4 Comments

Surprised to not see a mention of Astronomer - they've bet big on Airflow and are contributors to the open source distro as well. They definitely support your argument here!

Expand full comment
author

Very true. I know they are key to pushing Airflow forward. But it's a double edged sword. Will it always be so? Will they start holding back features for only their paying customers? Will they always make the right decisions when money is involved. Such things have been known to also signal the decline and downfall of tools.

Expand full comment

I'm sure they are holding back some features, but as I understand it a lot of their clients are quite large and are paying for high-scale and support related features on top of Airflow. It's an Apache project though so I I'd think many features from Astronomer would eventually flow back into the open source at some point.

Making money in open source is hard! 😅

Expand full comment

Cloudera is also betting big on Apache AIrflow by integrating it well with in its Data Engineering stack, taking away some of the pain points mentioned above wrt scaling and verbosity. They are also contributing to the open source comm as well.

Expand full comment