The Battle for the Orchestration Future.
As always, your content is 🔥 🔥 🔥 . Thanks so much for sharing this 🙏 🙏 🙏
Dagster didn’t get a look in? Any reason why?
Thank you for breaking down your thoughts and experience. Very written as usual!
Thanks, I think you have missed out on Dagster though, so maybe that would be worth a look as well! I really like it.
Any view on Kestra? I am looking at it vs Mage and Prefect.
Xcom can be a pain in Airflow (to explain to users primarily). Also, the feature is pretty new so buggy but dynamic task mappings leave something to be desired on Airflow as well. Wonder if Mage or Prefect do a better job here.
This is great. I was looking at mage recently as well. One thing one could do in Airflow is to dynamically generate DAGs based on some config. This is commonly used in many home grown config driven pipelines. I do not see how to do that easily in mage.
Thank you for steering my attention (curiosity) in the right (maze.ai) direction 👍