Why Aren’t You Filtering More?

Save Money and Time, and Money

A story as old as time, you and I’ve been around the block plenty before eh? Yet, after more than a decade of doing Data Engineering work … I still forget sometimes. Old habits die hard.

If I think back even further, to my old SQL Server Data Warehouse days, the same thing applied. I spent years tuning queries, and even today, piddling around Spark pipelines looking for a little efficiency here and there.

It comes down to one thing … filter early, filter often.