9 Comments
founding

As always, your content is ๐Ÿ”ฅ ๐Ÿ”ฅ ๐Ÿ”ฅ . Thanks so much for sharing this ๐Ÿ™ ๐Ÿ™ ๐Ÿ™

Expand full comment

Dagster didnโ€™t get a look in? Any reason why?

Expand full comment

Thank you for breaking down your thoughts and experience. Very written as usual!

Expand full comment

Thanks, I think you have missed out on Dagster though, so maybe that would be worth a look as well! I really like it.

Expand full comment

Any view on Kestra? I am looking at it vs Mage and Prefect.

Expand full comment

Xcom can be a pain in Airflow (to explain to users primarily). Also, the feature is pretty new so buggy but dynamic task mappings leave something to be desired on Airflow as well. Wonder if Mage or Prefect do a better job here.

Expand full comment
Apr 25, 2023ยทedited Apr 25, 2023

This is great. I was looking at mage recently as well. One thing one could do in Airflow is to dynamically generate DAGs based on some config. This is commonly used in many home grown config driven pipelines. I do not see how to do that easily in mage.

Expand full comment

Thank you for steering my attention (curiosity) in the right (maze.ai) direction ๐Ÿ‘

Expand full comment