Future proof yourself against AI.

Data Engineering Excellence.

Photo by Micaela Parente on Unsplash

Ignore ChatGPT and CoPilot at your own risk. Just like the old folk riding around horses bemoaning the iron horses, you are liable to throw the baby out with the bath water. It can be hard to tell sometimes if the hype is real if the headlines are true, and how much you should care.

I mean, I did write a fairly decent PySpark pipeline using CoPilot.

There are three types of folks when it comes to AI and programming I think.

• One’s who’ve never used AI to help with writing code, and the poopoo it.

• Folks who’ve used it to become more efficient and better at their job.

• The lazy want it to do everything for them and become terrible engineers because of it.

I think personally the truth is somewhere in the middle.

Can AI write better code than more than most new and junior software engineers? Yes. Have the best programmers always been the ones who can do more than just write code? Yes.

Is the future of software engineering going to change if AI can write decent code for us? Yes, I don’t know exactly what that will look like, and change takes time, but things are probably going to change.

What are we going to cover today?

• What will AI change about Data Engineering?

• What makes an excellent Data Engineer?

• Future-proof yourself against AI.

Thanks to Delta for sponsoring this newsletter! I personally use Delta Lake on a daily basis, and I believe this technology represents the future of Data Engineering. Check out their website below.

What will AI change about Data Engineering?