In this episode, I sit down with Matt Martin, Staff Engineer, data architect, ETL practitioner, and author of a new book on DuckDB coming soon, to talk about the past, present, and future of data engineering.
Matt has spent decades building and architecting data platforms across technologies such as SQL Server, Oracle, DB2, Hadoop, Redshift, and BigQuery, and now focuses on modern tools such as DuckDB and single-node analytics.
We discuss how the data industry has evolved, what actually makes data platforms succeed, and where tools like DuckDB, Polars, Databricks, and Snowflake fit into the future of analytics.
We also dive into the impact of AI on coding and data engineering, and whether distributed compute clusters will remain dominant — or if more workloads will move toward high-performance single-node systems.
Topics Covered
Matt’s early career and journey into data engineering
The evolution of data warehousing and ETL frameworks
Traditional enterprise data systems vs modern cloud platforms
DuckDB and the rise of single-node analytics
Polars vs DuckDB: where each tool shines
Databricks vs Snowflake
AI-assisted coding and its impact on engineers
The current data engineering job market
Lessons learned from decades of building data systems
Writing a book on DuckDB
