What is the thundering herd problem?

The thundering herd problem occurs when many processes or requests are released at the same instant and all contend for one resource, overwhelming it. Its most common form in application infrastructure is the cache stampede: a popular cache entry expires, every request that wanted it misses simultaneously, and all of them query the database at once.

The result is that a cache built to shield a database delivers a load spike to it instead, at the moment the database is least prepared for one. This page covers where the term comes from, what triggers a stampede, why the amplification is worse than it first appears, and the six mitigations that hold up in production — with Java implementations on Valkey and Redis.

Thundering herd, cache stampede, and dogpile effect

The three terms are used interchangeably in most engineering conversations, but they are not identical in origin or scope.

A note on the operating system sense: it is a real and separate problem, and it has its own fixes at the kernel level — EPOLLEXCLUSIVE to wake a single waiter per event, or SO_REUSEPORT to give each worker its own accept queue. Those are unrelated to anything you would do in a cache. The rest of this page concerns the caching sense.

What triggers a cache stampede

Expiry of a hot key is the textbook trigger, but it is not the most common one in practice. Anything that empties or invalidates many entries at once will do it:

• Hot key expiry. One entry serving a large share of traffic reaches its TTL and every in-flight request for it misses together.
• Synchronized TTLs. A batch of entries written at the same moment — during a deploy, a warm-up job, or a bulk import — carries the same TTL and therefore expires at the same moment. This is the trigger most teams overlook, and it produces a far larger herd than a single hot key.
• Cold start. A cache flush, a node restart, or a failover leaves the cache empty while traffic continues at full rate. Every request is a compulsory miss until the working set is rebuilt.
• Client reconnect storms. A network partition heals and every application instance reconnects and re-reads at once. Local caches that clear on disconnect make this worse.
• Retry storms. A slow backend causes clients to time out and retry. Without jitter, retries synchronize into waves that grow rather than decay.

Why the amplification is worse than it looks

The size of the herd is not the request rate. It is the request rate multiplied by the time the rebuild takes, because every request arriving during the rebuild window also misses:

herd size ≈ request rate × rebuild duration

10,000 req/s × 0.2 s rebuild = ~2,000 concurrent identical queries

Two things make this compound. First, the queries are identical, so 1,999 of those 2,000 are pure waste — the same rows, computed the same way, discarded. Second, the effect is self-reinforcing: 2,000 concurrent queries slow the database, which lengthens the rebuild, which widens the window in which further requests miss. A stampede that the database can nearly absorb and one that takes it down are separated by a small margin.

The practical consequence is that mitigations which shorten the rebuild are worth as much as mitigations which reduce the number of waiters.

How to prevent a thundering herd

1. Locking (single-flight)

Let one request rebuild the entry and make the others wait for its result. A distributed lock keyed on the cache key is the standard mechanism, and it is the strongest guarantee available: exactly one database query per key per rebuild.

The cost is latency for the waiters and a new failure mode. If the holder crashes mid-rebuild, the lock has to expire before anyone else can proceed, and every waiter is stalled until it does. If the lease is too short, it expires during a slow rebuild and a second herd is released behind the first. Lease management is the part that hand-rolled SETNX implementations usually get wrong.

2. Bounded concurrency

Serializing every rebuild behind a single lock can be more conservative than necessary. Allowing a small fixed number of concurrent rebuilds — three or five rather than one — keeps database load bounded while avoiding a queue of waiters behind one slow holder. A distributed semaphore gives you this.

3. TTL jitter

Add a small random offset to every TTL so that entries written together do not expire together:

ttl = base_ttl + random(0, jitter)

This is the cheapest mitigation on the list and it addresses the most common trigger. It does nothing for a single hot key — that entry still expires exactly once and releases its herd — so treat it as a prerequisite rather than a solution.

4. Probabilistic early recomputation

Rather than waiting for expiry, let each reader independently decide to refresh the entry early, with a probability that rises as the expiry approaches. Most readers do nothing; one refreshes before the entry ever expires, so the miss never happens.

The best-known formulation is XFetch, from Optimal Probabilistic Cache Stampede Prevention (Vattani, Chierichetti and Lowenstein, VLDB 2015). It stores the previous rebuild duration alongside the value and uses it to scale the window:

value, delta, expiry = cache.read(key)

if value == null or time() - delta * beta * log(rand()) >= expiry:
    start = time()
    value = recompute()
    delta = time() - start
    cache.write(key, value, delta, ttl)

return value

delta is how long the last rebuild took and beta tunes how eagerly the refresh happens — expensive entries are refreshed further ahead of expiry than cheap ones. The appeal is that no request ever waits and no coordination is required between nodes. The trade-off is that the guarantee is probabilistic: under sufficient concurrency two nodes can still recompute simultaneously.

5. Stale-while-revalidate

Store a logical expiry that falls earlier than the physical TTL. Once the logical expiry passes, one request refreshes the entry while every other request continues to receive the stale value. Nobody waits and the database sees one query.

This is often the best answer for data where slightly stale is clearly preferable to slow, which describes more workloads than teams tend to assume. It is the wrong answer where a stale read is a correctness problem rather than a cosmetic one.

6. Near cache and request coalescing

A client-side cache in each application process absorbs a large share of the herd before it reaches Valkey or Redis at all. With ten application instances, a remote miss produces at most ten requests rather than ten thousand — and combined with locking, one.

A near cache also shortens the rebuild path for everything it serves, which per the arithmetic above reduces the herd for the entries it does not.

Request coalescing is the complementary technique inside a single process. A near cache serves values that have already been computed; coalescing deals with the window during which one is still being computed, by having concurrent callers for the same key share a single in-flight computation rather than each starting their own. It is single-flight applied at the JVM level, and it is what caps each instance's contribution to the herd at exactly one request. The two work together: the near cache handles repeat reads, coalescing handles the first one.

Choosing a strategy

These are layers, not alternatives. A common production configuration is jittered TTLs as the baseline, a near cache to absorb what it can, and a lock around the remaining rebuild path.

Preventing cache stampede in Java with Redisson

Valkey and Redis are not usable from Java without a client. Redisson exposes them as Java objects and provides the locking, semaphore and near-cache primitives these strategies need, so the mitigations become configuration and a few lines of application code rather than protocol work.

Start with the remote layer. RMapCache supports an expiry per entry rather than one TTL for the whole map, which is what makes jitter possible, and RLock guards the rebuild so that only one instance queries the database:

RMapCache<String, Product> products = redisson.getMapCache("products");

public Product get(String id) throws InterruptedException {
    Product cached = products.get(id);
    if (cached != null) {
        return cached;
    }

    RLock lock = redisson.getLock("rebuild:products:" + id);
    // no leaseTime argument, so the watchdog renews the lock while this
    // instance is alive and a slow rebuild cannot release a second herd
    if (lock.tryLock(2, TimeUnit.SECONDS)) {
        try {
            // re-check: the winner may have populated the entry while we waited
            cached = products.get(id);
            if (cached != null) {
                return cached;
            }
            Product fresh = productRepository.findById(id);
            // 10 minutes plus up to 2 minutes of jitter, so entries written
            // together do not expire together
            long ttl = 600 + ThreadLocalRandom.current().nextLong(120);
            products.fastPut(id, fresh, ttl, TimeUnit.SECONDS);
            return fresh;
        } finally {
            lock.unlock();
        }
    }

    // lock not acquired inside the wait window: read once more, then degrade
    // rather than adding this request to the pile on the database
    return products.get(id);
}

Two details matter more than they look. The re-check after acquiring the lock is what makes this correct — without it, every waiter rebuilds in turn once the holder releases, and you have converted a simultaneous herd into a sequential one. And calling tryLock without a leaseTime argument hands lease renewal to Redisson's lock watchdog, which extends the lock while the holding instance is alive and releases it if that instance dies. Supplying an explicit leaseTime disables the watchdog, which is the correct choice only when you know the rebuild's worst-case duration.

On a recent Valkey or Redis version, RMapCacheNative is the better choice for straightforward TTL-based caching, because expiry is handled by the server natively rather than by a Redisson-side eviction task. That difference matters here: a lagging eviction task means entries outlive their nominal TTL and then disappear in a batch when the task catches up, which is a stampede trigger in itself. Keep the classic RMapCache when you need max-idle eviction or have to support older servers.

Absorbing the herd with a near cache

A RLocalCachedMap keeps hot entries inside the application process, so most reads never reach the cache server and the herd that arrives there is bounded by the number of instances rather than the number of requests:

LocalCachedMapOptions<String, Product> options =
        LocalCachedMapOptions.<String, Product>name("products")
            .cacheSize(10_000)
            .evictionPolicy(EvictionPolicy.LRU)
            // time to live for each entry in the local cache
            .timeToLive(Duration.ofMinutes(10))
            // drop the entry on other nodes when it changes
            .syncStrategy(SyncStrategy.INVALIDATE)
            // repair the local cache after a disconnect instead of clearing it,
            // which would turn every reconnect into a herd of its own
            .reconnectionStrategy(ReconnectionStrategy.LOAD);

RLocalCachedMap<String, Product> products = redisson.getLocalCachedMap(options);

One thing to be clear about: timeToLive on LocalCachedMapOptions governs the local copy only. The entries written to Valkey or Redis behind an RLocalCachedMap carry no expiry of their own, so a near cache alone does not give you the jittered remote TTL described above. Combining both in one object — a near cache with per-entry remote expiry — is what RLocalCachedMapCache provides in Redisson PRO. With the open-source client, either accept a map-level expiry or apply the lock-and-jitter pattern against RMapCache and treat the near cache as a separate layer in front of it.

ReconnectionStrategy.LOAD is worth setting deliberately. The default leaves a reconnecting instance to serve whatever it still holds; CLEAR drops the entire local cache, which empties the absorption layer at exactly the moment a partition heals and every instance starts reading again. LOAD replays the missed invalidations instead, which is the behaviour you want when reconnect storms are the trigger you are defending against. Note that the invalidation log is bounded — a disconnect longer than the retention window falls back to clearing the cache anyway, so LOAD protects against brief network blips rather than extended outages.

Bounded concurrency instead of a single lock

Where serializing every rebuild behind one lock is too strict, RPermitExpirableSemaphore allows a bounded number of concurrent rebuilds instead:

RPermitExpirableSemaphore semaphore =
        redisson.getPermitExpirableSemaphore("rebuild:products:" + id);
semaphore.trySetPermits(3);

String permitId = semaphore.tryAcquire(2, 30, TimeUnit.SECONDS);
if (permitId != null) {
    try {
        // rebuild — at most three instances are here at once
    } finally {
        semaphore.tryRelease(permitId);
    }
}

As a last line of defence, a rate limiter on the database read path caps what can reach the backing store if every other mitigation fails at once. See how to use Redis locks in Java for the locking API in full, and Java caching strategies for how these fit into cache-aside, read-through and write-behind.

Spring @Cacheable and the sync attribute

Spring's caching abstraction offers @Cacheable(sync = true), which is often presented as the fix:

@Cacheable(value = "products", key = "#id", sync = true)
public Product getProduct(String id) {
    return productRepository.findById(id);
}

It is worth understanding precisely what this does. The sync flag instructs Spring to route the call through the cache provider's get(key, valueLoader) method rather than performing a get-then-load itself. Whether that produces a single load across the whole cluster, or a single load per JVM, depends entirely on how the provider implements that method — Spring itself only guarantees that concurrent callers within one application context wait on the same computation.

That distinction is the one that bites in production. With a local cache provider and twelve pods, sync = true reduces twelve thousand concurrent loads to twelve — a large improvement, and still twelve identical queries arriving together. If you need a genuine cluster-wide single-flight guarantee, verify the behaviour of your provider or apply an explicit distributed lock as shown above. See the Spring Boot caching guide for the surrounding configuration.

Related failure modes

Cache stampede is one member of a family, and the same reasoning applies across it:

• Retry storms. Clients timing out and retrying in lockstep. The fix is exponential backoff with jitter — backoff alone preserves synchronization and simply spaces the waves further apart.
• Connection storms. Every instance reconnecting at once after a partition heals, exhausting connection slots before any useful work happens. Staggered reconnect delays and connection pooling contain it.
• Cache penetration. Repeated lookups for keys that do not exist, each one reaching the database and returning nothing. Distinct from a stampede — the entries were never cached at all — and addressed with negative caching or a Bloom filter. See cache miss.

The common thread is synchronization. Anything that causes independent clients to act at the same instant will produce a herd, and randomization is the general antidote.

Frequently asked questions

What is the thundering herd problem?

It is the pattern in which many waiting processes or requests are released simultaneously and contend for one resource, overwhelming it. It originates in operating system scheduling, where a single event wakes every process blocked on it. In application infrastructure the usual form is a cache stampede.

What is a cache stampede?

A cache stampede is what happens when a cached entry becomes unavailable and every concurrent request for it misses at the same moment, so all of them query the database simultaneously. It is the caching-specific instance of the thundering herd problem, and it is also called the dogpile effect.

How do I handle cache stampede and thundering herd?

Layer the mitigations. Add jitter to every TTL so entries do not expire in batches, put a near cache in front of the remote cache to absorb requests in-process, and guard the rebuild path with a distributed lock so that only one request queries the database per key. Where waiting is unacceptable, serve stale data while one request refreshes in the background.

What is Redis cache stampede?

The same problem in a Redis or Valkey deployment: a hot key expires, all concurrent readers miss, and the database absorbs the burst that the cache was there to prevent. Redis and Valkey have no built-in protection against it, so the mitigation lives in the client or the application.

Does a longer TTL prevent cache stampedes?

No. A longer TTL makes stampedes less frequent without making them any smaller, because the size of a herd is set by the request rate and the rebuild duration, not by how long the entry survived beforehand. It also increases staleness. Randomizing TTLs helps; extending them only moves the problem further apart in time.

What is the dogpile effect?

An informal synonym for cache stampede, more common in Python and Ruby codebases than in Java ones. It describes the same behaviour: many requests piling onto the same expensive recomputation at once.

Similar terms

• Cache Invalidation
• Cache Memory
• Cache Miss
• Distributed Caching
• Distributed lock
• Java Semaphore
• Race Condition
• Rate Limiter
• Redis Caching
• Redis Client-Side Caching
• Redis TTL
• Redlock Algorithm
• Spring Cache
• Write-through and write-behind caching