Skip to content
PHP2Shell: The PHP Bug
Akshay Thakare

Apache Airflow

DAG design, idempotent tasks, sensors and operating Airflow.

4 notes · all topicsCloud & orchestration

Start here · beginner1

  1. 1Airflow Core Concepts: DAGs, Tasks, Operators and RunsThe vocabulary you need to read any Airflow project: DAGs, tasks, operators, sensors, XComs, data intervals and the scheduler.

Next · intermediate3

  1. 2Sensors, Reschedule Mode and Deferrable OperatorsWaiting without wasting workers: poke vs reschedule mode, deferrable operators and the triggerer, and timeouts that fail loudly.
  2. 3Writing Idempotent Airflow DAGsLogical dates, partition overwrites and safe retries — the habits that make re-running a DAG boring.
  3. 4Airflow Backfills and Catchup Without SurprisesHow data intervals, catchup and backfills work in Airflow, and how to design DAGs so rerunning last month is safe and boring.