Rate Limiting APIs: Token Bucket vs Sliding Window

Without rate limiting, one noisy client can degrade an API for everyone else. But the naive approach to rate limiting has a nasty boundary bug that most teams don't discover until production.

The Problem: Fixed Window Counter

The simplest approach counts requests in a fixed time box and resets the counter every period:

// Naive fixed window with Redis
async function isAllowed(userId) {
  const key = `rl:${userId}:${Math.floor(Date.now() / 60000)}` // per-minute bucket
  const count = await redis.incr(key)
  await redis.expire(key, 60)
  return count <= 100 // 100 req/min
}

This looks fine until a client sends 100 requests at 0:59 and another 100 at 1:00 — 200 requests in two seconds, twice the intended limit, simply because they straddle the window boundary.

Token Bucket

A bucket holds up to N tokens. Tokens refill at a fixed rate. Every request consumes one token; if the bucket is empty, the request is rejected. This allows short bursts (up to the bucket size) while still enforcing a strict average rate over time — it's what Stripe and AWS API Gateway use under the hood.

class TokenBucket {
  constructor(capacity, refillPerSecond) {
    this.capacity = capacity
    this.tokens = capacity
    this.refillPerSecond = refillPerSecond
    this.lastRefill = Date.now()
  }

  tryConsume() {
    this.refill()
    if (this.tokens < 1) return false
    this.tokens -= 1
    return true
  }

  refill() {
    const now = Date.now()
    const elapsedSeconds = (now - this.lastRefill) / 1000
    this.tokens = Math.min(this.capacity, this.tokens + elapsedSeconds * this.refillPerSecond)
    this.lastRefill = now
  }
}

Sliding Window Counter

A cheaper alternative to a full sliding log: weight the previous window's count by how much of it still "overlaps" the current moment.

// weighted average of previous + current fixed windows
const weight = 1 - (elapsedInCurrentWindow / windowSize)
const estimatedCount = previousWindowCount * weight + currentWindowCount

This smooths out the boundary burst problem without the memory cost of storing a timestamp per request.

How a Rate-Limited Request Flows

When to Use Which

• Fixed window: simplest to implement, fine for coarse per-day/per-hour limits where bursts don't matter.
• Token bucket: the default choice for public APIs — allows reasonable bursts while holding a strict average rate.
• Sliding window: most accurate under sustained load, worth the extra complexity for strict SLAs or billing-sensitive limits.

Common Pitfalls

• Limiting by IP alone breaks for users behind shared NAT or corporate proxies — key on API key or user ID as well as IP.
• Skipping rate-limit headers (X-RateLimit-Remaining, Retry-After) forces clients to guess when they can retry, causing thundering-herd retries.
• Per-instance in-memory counters in a multi-server deployment mean each instance enforces the limit independently — a client can get N times the intended limit by hitting N different instances. Always back the counter with a shared store like Redis.
• Not rate-limiting authentication endpoints separately — login/signup routes need tighter limits than general API traffic to slow down credential stuffing.

Start with a token bucket backed by Redis. It's simple to reason about, allows reasonable bursts, and the same primitive scales from a single endpoint to a full API gateway.