What Is Rate Limiting?

Rate limiting is the practice of capping how many requests a client may make to a system within a given period of time. Requests that arrive over the cap are rejected, delayed or queued rather than served. The component that enforces the cap is called a rate limiter.

What is a rate limiter?

A rate limiter sits in front of a resource and answers one question on every call: has this client already used its allowance for the current period? If the answer is no, the request proceeds and the limiter records the usage. If the answer is yes, the request is refused — conventionally with an HTTP 429 Too Many Requests status and a Retry-After header telling the caller when to come back.

Two parameters define any limiter: a limit (how many requests are permitted) and a window (the period the limit applies to). "100 requests per minute" is a complete specification. A third parameter, the key, decides who the limit applies to — an API key, a user ID, an IP address, a tenant, or the service as a whole.

The word permit is often used for a single unit of allowance. Some requests are worth more than others; a bulk export might reasonably consume ten permits where a status check consumes one. Limiters that support weighted acquisition let you price expensive operations accordingly.

Why is rate limiting used?

Rate limiting protects capacity that cannot be scaled instantly, and it protects it against traffic that is not always malicious:

• Traffic spikes. A launch, a link on an aggregator or a marketing email can push volume well past what the system was provisioned for. A limiter degrades gracefully by refusing the excess instead of letting everything slow to a crawl.
• Abuse and brute force. Credential stuffing, scraping and enumeration attacks all depend on issuing a high volume of requests. Capping attempts per account or per IP makes them impractical.
• Noisy neighbours. In a multi-tenant system one aggressive client can consume the capacity everyone else depends on. Per-tenant limits contain the blast radius.
• Buggy clients. A retry loop without backoff, a misconfigured cron job or a mobile app shipped with a polling interval of one second will generate the same load as an attack with none of the intent.
• Third-party quotas. When your service calls an upstream API that bills per call or cuts you off past a threshold, an outbound limiter keeps your own fleet inside someone else's budget.
• Cost control. In autoscaling environments, unbounded traffic converts directly into unbounded spend. A limit is a ceiling on the invoice as well as on the load.

Rate limiting is not always the right answer. Where every request carries data the user genuinely needs — a live feed, a trading system, a telemetry ingest path — refusing requests changes the correctness of the product rather than just its performance. In those cases the problem is capacity, and the fix is capacity.

Rate limiting vs throttling

The two terms are used interchangeably in casual conversation, but they describe different responses to the same condition. A rate limiter rejects excess requests. Throttling slows them — the request is queued, delayed or served at reduced quality rather than refused outright.

In practice most systems do both, and the same underlying limiter supports either behaviour. A blocking acquisition call throttles; a non-blocking one that returns a boolean rate limits.

Rate limiting algorithms

Every limiter has to decide how to count requests over time, and the algorithm it uses determines how much state it keeps, how precisely it holds the limit, and how it behaves at the edges of a window. Six cover essentially all production use:

Token bucket is the usual default for user-facing APIs, since it tolerates the bursty traffic real clients produce without letting the long-run average drift. Fixed window is the cheapest but admits up to twice the configured limit across a window boundary. For how each one works, worked examples, and the trade-offs between them, see rate limiting algorithms.

Rate limiters in Valkey and Redis

Inside a single JVM, rate limiting is a solved problem — a counter in a ConcurrentHashMap or Guava's RateLimiter is enough. Neither survives horizontal scaling, because both keep the count inside the process rather than in a store every node shares — see stateless vs stateful. Once an application runs behind a load balancer, every instance keeps its own counter and enforces its own copy of the limit, so three nodes configured for 100 requests per second will collectively admit 300 — and the number changes every time the deployment scales.

Enforcing one limit across a fleet needs shared state that every node can read and update atomically. Valkey and Redis suit the job: command execution is effectively serialized, so counters increment atomically without client-side coordination; the data lives in memory, so the check costs a sub-millisecond round trip; and keys carry a native TTL, so windows clean themselves up. One key becomes the single source of truth for a client's usage, whichever node happened to serve the request.

In Java, Redisson models this as a distributed object rather than a hand-written script, so one limit holds across every JVM sharing the key:

RRateLimiter limiter = redisson.getRateLimiter("api:global");
limiter.trySetRate(RateType.OVERALL, 100, 1, RateIntervalUnit.SECONDS);

if (limiter.tryAcquire(1)) {
    // handle the request
} else {
    // over the limit — reject with 429
}

Redisson also provides RGcra, a GCRA limiter built on the native Redis command, for cases needing strict burst control and exact retry timings. For working code — why hand-rolled INCR and EXPIRE limiters break, how to scope a limit per user or per fleet, and a reusable Spring Boot @RateLimit annotation — see distributed rate limiting in Java with Valkey or Redis and Spring Boot.

Rate limiter vs semaphore

A rate limiter and a semaphore both restrict access to a resource, and the two are easy to confuse. The difference is what they count.

A semaphore limits concurrency: how many callers may hold the resource at the same time. Its permits are borrowed and must be released when the caller is done. A rate limiter limits frequency: how many requests may be made per unit of time. Its permits are consumed rather than borrowed, and are never released — they replenish on a schedule.

"No more than 20 simultaneous connections to the database" is a semaphore. "No more than 20 queries per second" is a rate limiter. Systems that need both use both.

Rate limiting best practices

• Return 429 with Retry-After. A rejected caller that knows when to come back will back off correctly. One that does not will hammer you in a retry loop.
• Publish the limits. X-RateLimit-Limit, X-RateLimit-Remaining and X-RateLimit-Reset headers let well-behaved clients pace themselves before hitting the wall.
• Choose the key deliberately. Limiting by IP punishes everyone behind a corporate NAT and does little against a distributed attacker. Authenticated identity is a better key wherever it is available.
• Expire idle keys. Per-user, per-endpoint limiters multiply quickly. Whatever stores the counters needs a way to drop a limiter once it has sat unused for a full window, or the keyspace grows without bound.
• Make retried operations safe. A limiter guarantees the client will retry. If the endpoint is not idempotent, those retries create duplicates.
• Fail open or closed on purpose. Decide in advance what happens when the limiter's backing store is unreachable, and make it a deliberate choice rather than whatever the exception handler happens to do.

Frequently asked questions

What is rate limiting?

Rate limiting is the practice of capping how many requests a client may make within a set period of time, and refusing the excess. It protects a system from traffic spikes, abuse and buggy clients, keeps one tenant from consuming capacity that belongs to everyone, and holds usage of paid upstream APIs inside budget. Rejected requests are conventionally answered with HTTP 429 and a Retry-After header.

What is the difference between rate limiting and throttling?

Rate limiting rejects requests over the cap; throttling slows them instead, by queueing or delaying rather than refusing. The distinction is the response to excess load, not the mechanism that detects it, and the same limiter usually supports both — a non-blocking acquisition rate limits, a blocking one throttles.

Should rate limits be applied per user or per IP address?

Per authenticated user wherever identity is available. An IP is a poor proxy for a client: everyone behind a corporate NAT or a mobile carrier gateway shares one, so an IP limit punishes innocent users together with the abusive one, while a distributed attacker simply spreads across addresses to avoid it. IP limits still have a place on unauthenticated endpoints such as sign-up and login, where no better key exists.

What does HTTP 429 Too Many Requests mean?

429 Too Many Requests is the status a server returns when a client has exceeded its rate limit. It is normally sent with a Retry-After header giving either the number of seconds to wait or a date after which the client may try again. A 429 says the request was well-formed and the client is entitled to make it — only the timing was wrong — so the correct response is to back off and retry, not to treat it as a permanent failure.

What is the difference between a rate limiter and a semaphore?

A semaphore limits how many callers hold a resource at once, and its permits are released when the caller finishes. A rate limiter limits how many requests occur per unit of time, and its permits are consumed rather than returned, replenishing on a schedule instead. Concurrency versus frequency.

Similar terms

• Rate limiting algorithms
• Java Semaphore
• Distributed lock
• Idempotency
• Race condition
• Redis TTL
• Redis Lua scripting
• Microservices

Redisson provides distributed rate limiters, semaphores and locks through one Java API across Valkey and Redis, with the atomic logic already written and tested. Start a free trial.