The best kept secret to ML success.

Databricks + Delta Lake.

image courtesy of DalL-E

Note: Yes, Delta Lake supports this Substack. However, the opinions presented here are from a decade of working as a Data Engineering on large ML projects. I’ve seen it all. I speak the truth.

So I’m just going to state the truth up front. There is a never-ending stream of Machine Learning content being poured down your throat in a never-ending stream of drivel about as good as late-night as-seen-on-TV adds. Seems all shiny and nice at 3 a.m., but the reality is somewhat different.

Today, I want to talk about the best-kept secret of Machine Learning, which is Databricks + Delta Lake. I will spend the rest of the time explaining that.