Sensors, Reschedule Mode and Deferrable Operators
Waiting without wasting workers: poke vs reschedule mode, deferrable operators and the triggerer, and timeouts that fail loudly.
Apache AirflowIntermediate2 min readUpdated 11 Oct 2026
The cost of waiting
A sensor in default poke mode holds a worker slot for its whole wait. Fifty sensors waiting an hour can block every other task.
| Mode | Holds a worker while waiting? | Use for |
|---|---|---|
poke | Yes | Waits of seconds |
reschedule | No; it frees the slot between checks | Waits of minutes to hours |
| Deferrable | No; an async triggerer does the waiting | Many long waits |
from airflow.providers.amazon.aws.sensors.s3 import S3KeySensor
wait = S3KeySensor(
task_id="wait_for_file",
bucket_key="s3://lake/raw/orders/{{ ds }}/_SUCCESS",
mode="reschedule",
poke_interval=300,
timeout=6 * 60 * 60, # fail after 6 hours instead of waiting forever
)Deferrable operators
A deferrable operator hands its wait to the triggerer process, which can watch thousands of conditions with asyncio. The task resumes on a worker only when the condition is met. Many provider operators accept deferrable=True.
Always set a timeout
Without timeout, a sensor waiting for a file that never arrives sits there silently. A timeout turns "data is late" into a failed task that alerts someone.
Finished this note?