Exponential Backoff with Jitter: Retrying Failed Requests the Right Way

Retrying a failed request seems simple: wait a bit, try again. But retry the same fixed delay across thousands of clients and you get a thundering herd — every client wakes up and hammers the server at the exact same moment, often taking down whatever was just starting to recover.

Exponential backoff spaces retries further apart each time (200ms, 400ms, 800ms...), but on its own it still leaves every client synchronized to the same schedule. Jitter fixes that by randomizing the delay, so retries spread out instead of arriving in lockstep.

The Retry Timeline

Each hop roughly doubles the base delay, capped at some maximum, with a random component layered on top so no two clients retry at exactly the same instant.

Fixed vs Exponential vs Jittered

A fixed delay (always wait 1s) is simple but synchronizes every client that failed at the same time. Plain exponential backoff (200ms, 400ms, 800ms, 1600ms) spreads attempts further apart over time, but clients that failed together are still retrying together at each step. Adding jitter — picking a random delay instead of the fixed exponential value — breaks that synchronization entirely.

Full Jitter

The full jitter strategy (used internally by AWS's own SDKs) computes the exponential delay as a ceiling, then picks a uniformly random value between zero and that ceiling:

delay = random_between(0, min(cap, base * 2^attempt))

This is simpler than "equal jitter" or "decorrelated jitter" variants and performs well in practice: it spreads retries across the full window instead of just adding a small amount of noise around the exponential value.

Implementation

function calculateBackoff(int $attempt, int $baseDelayMs = 200, int $maxDelayMs = 30000): int
{
    $exponential = $baseDelayMs * (2 ** $attempt);
    $capped = min($exponential, $maxDelayMs);

    return random_int(0, $capped);
}

function retryWithBackoff(callable $operation, int $maxAttempts = 5, int $baseDelayMs = 200, int $maxDelayMs = 30000)
{
    $attempt = 0;

    while (true) {
        try {
            return $operation();
        } catch (\Throwable $e) {
            $attempt++;

            if ($attempt >= $maxAttempts) {
                throw $e;
            }

            $delayMs = calculateBackoff($attempt, $baseDelayMs, $maxDelayMs);
            usleep($delayMs * 1000);
        }
    }
}

import random
import time
from typing import Callable, TypeVar

T = TypeVar("T")


def calculate_backoff(attempt: int, base_delay_ms: int = 200, max_delay_ms: int = 30000) -> int:
    exponential = base_delay_ms * (2 ** attempt)
    capped = min(exponential, max_delay_ms)

    return random.randint(0, capped)


def retry_with_backoff(
    operation: Callable[[], T],
    max_attempts: int = 5,
    base_delay_ms: int = 200,
    max_delay_ms: int = 30000,
) -> T:
    attempt = 0

    while True:
        try:
            return operation()
        except Exception:
            attempt += 1

            if attempt >= max_attempts:
                raise

            delay_ms = calculate_backoff(attempt, base_delay_ms, max_delay_ms)
            time.sleep(delay_ms / 1000)

Choosing the Parameters

baseDelayMs should roughly match how fast the downstream service actually recovers from a blip — too low and you're retrying into a still-failing service, too high and users wait needlessly on transient errors. maxDelayMs caps the worst case so a client doesn't end up waiting minutes between attempts. maxAttempts should be low enough that a genuinely broken dependency fails fast instead of hanging the caller for a long chain of doubling delays.

Common Pitfalls

Retrying without any cap on the delay means a client can end up waiting minutes for an attempt that will just fail again. Retrying without jitter defeats the entire purpose — synchronized clients stay synchronized no matter how far apart the delays get. Retrying on every kind of error, including 4xx client errors that will never succeed, wastes the retry budget on failures that a delay can't fix. And retrying indefinitely with no maxAttempts turns a transient blip into a request that never gives up and never surfaces an error to the caller.

When to Reach for This

Any call to a dependency that can fail transiently and recover on its own: HTTP calls to another service, database connections during a failover, queue consumers hitting a rate-limited API. If failures are rarely transient — a malformed request, a missing resource — backoff and retries just delay an error that was never going to succeed.