Orchestration
Schedule and monitor pipelines: Airflow, Prefect, Dagster, Luigi and Kedro.
Intermediate
This is an overview — pick a sub-topic below.
Sub-topics (3)
Apache Airflow
Define workflows as Python DAGs with scheduling, retries and a web UI.
Intermediate♥ 0🔖 0💬 0
Prefect and Dagster
Modern orchestrators: Prefect turns functions into tracked flows; Dagster models data assets and lineage.
Intermediate♥ 0🔖 0💬 0
Luigi and Kedro
Luigi (Spotify) builds dependency pipelines; Kedro structures data-science projects into reproducible pipelines.
Intermediate♥ 0🔖 0💬 0
Spotted a mistake or want to add detail? Suggest an improvement — Content Contributors' edits are reviewed by editors before they go live.
Discussion (0)
Sign in or join free to ask questions and discuss this topic.