In this episode of the Data Engineering Central Podcast, I sit down with Ethan, a maintainer of delta-rs and an expert in modern lakehouse architecture working in the pharmaceutical industry.
We discuss Ethan’s journey into tech and data engineering, the evolution of open table formats like Delta Lake and Apache Iceberg, and what it actually takes to build scalable enterprise data platforms in highly regulated environments like big pharma.
We also dive into:
delta-rs and the future of Delta Lake outside Spark
Lakehouse architecture and open catalogs
Rust in the modern data ecosystem
Data platform governance and scalability
Enterprise analytics and infrastructure
The future of agentic analytics and AI-enabled data systems
Lessons learned building large-scale data platforms
If you’re interested in modern data engineering, open source infrastructure, lakehouses, or the future of analytics engineering, this is a great conversation.
