retries
and retry_delay_seconds
parameters to your
task. If the task fails, Prefect will retry it up to the number of retries
times, waiting
retry_delay_seconds
seconds between each attempt. If the task fails on the
final retry, Prefect marks the task as crashed if the task raised an exception;
or failed if it returned a string.
Retries do not create new task runsA new task run is not created when a task is retried. A new state is added to the state
history of the original task run.
Example: making an API request
Consider the real-world problem of making an API request. This example uses thehttpx
library to make an HTTP
request.
Custom retry behavior
Theretry_delay_seconds
option accepts a list of delays for more custom retry behavior.
The following task waits for successively increasing intervals of 1, 10, and 100 seconds,
respectively, before the next attempt starts:
retry_condition_fn
option accepts a callable that returns a boolean.
If the callable returns True
, the task is retried. If the callable returns False
,
the task is not retried. The callable accepts three arguments: the task, the task run,
and the state of the task run. The following task retries on HTTP status codes other than 401 or 404:
exponential_backoff
utility. This utility automatically generates a list of retry delays that correspond to an exponential backoff retry
strategy. The following flow waits for 10, 20, then 40 seconds before each retry:
Advanced topic: adding “jitter”
With exponential backoff, you may want to add jitter to the delay times. Jitter is a random amount of time added to retry periods that helps prevent “thundering herd” scenarios. This is when many tasks all retry at the exact same time, potentially overwhelming systems. You can use theretry_jitter_factor
option to add variance to the base delay. For example, a retry
delay of 10 seconds with a retry_jitter_factor
of 0.5 allows a delay up to 15 seconds.
Large values of retry_jitter_factor
provide more protection against thundering herds, while keeping
the average retry delay time constant. For example, the following task adds jitter to its exponential
backoff so the retry delays will vary up to a maximum delay time of 20, 40, and 80 seconds respectively:
Configure retry behavior globally with settings
You can set retries and retry delays with the right global settings. These settings do not override theretries
or retry_delay_seconds
that are set in the flow or task decorator.