Data Pipelines

Automated chains of steps that move and transform data on a schedule or trigger.

Intermediate

Good pipelines are idempotent (safe to re-run), monitored, and versioned. Orchestrators like Airflow, Prefect and Dagster run them.

Spotted a mistake or want to add detail? Suggest an improvement — Content Contributors' edits are reviewed by editors before they go live.

Discussion (0)