What Is a Message Queue?

A message queue is an ordered buffer that holds messages between the service that produces them and the service that processes them. A producer appends a message and returns immediately; a consumer retrieves it later, does the work, and acknowledges it. Neither side needs the other to be running at the same moment, which is what makes the communication asynchronous.

The defining property is that each message is delivered to one consumer. Point several workers at a single queue and they compete for messages rather than each receiving a copy, so doubling the worker count roughly halves the time it takes to drain a backlog. That single characteristic separates a queue from publish/subscribe, where every subscriber receives its own copy of every message, and it determines almost everything else about how queues are used: task distribution, background jobs, and absorbing traffic spikes that would otherwise overwhelm a downstream service.

How a Message Queue Works

Message queuing predates almost everything else in modern distributed systems — IBM commercialised it as MQSeries in December 1993 — and the core mechanics have barely moved since: a four-step loop, of which the fourth step is the one that matters.

1. A producer creates a message — a payload plus metadata such as headers, a timestamp, or a priority — and sends it to a named queue.
2. The queue stores the message, in memory, on disk, or both, and returns an acknowledgment to the producer. From this point the producer is free.
3. A consumer retrieves the message. Most queues immediately make it invisible to other consumers for a visibility timeout, so two workers cannot pick up the same job.
4. The consumer processes the message and sends an acknowledgment. Only then does the queue delete it. If the consumer crashes first, the visibility timeout lapses and the message becomes available again.

Step four is what makes a queue a queue rather than a buffer. A plain buffer hands you data and forgets it existed; a message queue holds the message until someone confirms the work was done. Everything else — retries, delivery limits, dead letter queues — is built on top of that acknowledgment loop.

The gap between step three and step four is also where most production incidents live. A visibility timeout shorter than your worst-case processing time causes a healthy but slow consumer to have its message redelivered and processed a second time. Amazon SQS defaults to 30 seconds, which is far too short for anything involving a PDF, a payment provider, or a third-party API.

Message Queue Architecture

A queue has three moving parts — producers, the queue itself, and consumers — but the architectural decision that matters is where the queue actually lives.

A managed message queue service such as SQS or Azure Service Bus removes the operational work entirely, at the cost of per-request billing and semantics tied to one cloud. The data-store-backed model gets discussed least and is often the most practical: if Redis or Valkey is already in production for caching or sessions, putting a queue on it adds nothing new to run, patch, or monitor — though what guarantees you get then depends entirely on the client library, which is the subject of the last two sections here.

Consumers pull or are pushed to. Pull (SQS, Redis, Kafka) means the consumer asks for work when it has capacity, which gives natural backpressure — a saturated worker simply stops asking. Push (RabbitMQ's basic consume, JMS MessageListener) delivers as messages arrive and needs an explicit prefetch or window limit to avoid overwhelming a slow consumer. Neither is better; the failure modes differ, and a push consumer without a prefetch limit is a common way to turn one slow dependency into an out-of-memory error.

Queues are also frequently misdrawn as a single pipe between two services. In practice a mature system has many named queues, often one per unit of work, precisely so a backlog in report generation does not delay password-reset emails. Sharing one queue across unrelated work types is the fastest route to head-of-line blocking.

Delivery Guarantees: At-Most-Once, At-Least-Once, and Exactly-Once

Every queue makes a promise about how many times a message will be delivered, and the promise is determined by when the acknowledgment happens relative to the work.

At-most-once acknowledges on receipt. It is fast and it loses messages whenever a consumer dies mid-work. Native Redis pub/sub behaves this way, and for cache invalidation or presence updates that is a reasonable trade. For anything a customer paid for, it is not.

At-least-once acknowledges after the work completes, so a crash means redelivery rather than loss. This is the practical default across SQS standard queues, RabbitMQ with manual acks, Redis Streams with consumer groups, and Kafka. The cost is duplicates, and they are not rare edge cases: any consumer that finishes its work but dies before its acknowledgment reaches the broker will produce one.

Exactly-once is where marketing outruns distributed systems theory. Exactly-once delivery over an unreliable network is not achievable — an acknowledgment can always be lost in flight, and the sender cannot distinguish that from a consumer that never received the message. What systems actually provide is exactly-once processing: at-least-once delivery combined with either a deduplication window on the broker or an idempotent consumer. SQS FIFO queues, for example, deduplicate on a message deduplication ID within a five-minute window. When a product claims exactly-once, this combination is what it means.

The practical consequence: design consumers to be idempotent regardless of what your broker promises. Key the operation on a business identifier — an order ID, a payment reference — and make a second execution a no-op. That single discipline removes an entire class of incident, and it is cheaper than any broker feature.

Ordering, Durability, and Backpressure

Ordering is the guarantee people assume they have and usually do not. FIFO holds when one consumer reads one queue. Add a second consumer and ordering breaks immediately: message 1 goes to worker A, message 2 to worker B, and B finishes first. Every queue that offers both parallelism and ordering does so by partitioning — SQS FIFO uses a message group ID, Kafka uses the partition key — so that messages sharing a key go to the same consumer while different keys process concurrently. Global ordering across a whole queue and parallel consumption are mutually exclusive, and choosing ordering means accepting a throughput ceiling.

Durability is a spectrum, not a switch. An in-memory queue loses everything on restart. A persisted queue survives process failure but not necessarily disk failure. A replicated queue survives node failure, but only if the write was confirmed on the replica before the producer was told it succeeded — asynchronous replication means a narrow window where an acknowledged message exists on exactly one machine. If a message represents money, ask specifically whether the acknowledgment your producer receives is confirmed on more than one node.

There is a related gap on the producer side that queues cannot close for you. If a service writes to its database and the enqueue then fails — or the process dies in the gap between the two — the state change is durable but the message never existed, and retrying inside the request cannot help when the request itself is what died. The outbox pattern is the standard remedy: write the message into the same database transaction as the state change, then relay it to the queue separately.

Backpressure is what happens when producers outrun consumers. An unbounded queue absorbs the spike and converts it into latency — which is often exactly the point, since load leveling is a core reason to use a queue at all. But unbounded really means bounded by memory or disk, and discovering that limit in production is unpleasant. A bounded queue forces a decision: block the producer, reject the message, or drop the oldest. Deciding deliberately beats discovering it at 3am.

Monitor queue depth and the age of the oldest message. Depth alone is ambiguous — a queue holding 500 messages that arrived a second ago is healthy, while one holding 3 messages that arrived an hour ago is not.

When to Use a Message Queue — and When Not To

In system design terms the question is rarely whether a queue would work — it usually would — but whether the coupling it removes is coupling you actually suffer from. Queues earn their place when work can be separated from the request that triggered it.

• Offloading slow work. Accepting an upload and returning 202 while thumbnailing happens elsewhere keeps the request path fast and the timeout budget generous.
• Load leveling. A sale generates ten times normal traffic for twenty minutes. A queue absorbs it so the fulfilment service works steadily instead of falling over.
• Isolating failure. If the email provider is down, queued messages wait. Without a queue, the checkout fails because the receipt could not be sent.
• Decoupling deployments. Producers and consumers can be released, scaled, and restarted independently, which matters most in microservices architectures.
• Scheduled and delayed work. Reminders, retries with backoff, and follow-ups that should fire hours later.

They are the wrong tool when the caller needs an answer now — a queue in a synchronous read path adds latency and complexity to buy nothing — or when the traffic does not justify the operational cost. A single application with a background thread pool does not need a broker. Queues are also wrong when what you actually want is a durable, replayable history that several independent consumers read at their own pace: that is an event log, and building it out of a queue means fighting the fact that queues delete messages once they are acknowledged.

One more caution: a queue converts a fast failure into a slow one. A synchronous call fails visibly in milliseconds; a queued message fails somewhere else, minutes later, in a consumer nobody is watching. Queues need dead letter queues and alerting before they go to production, not after the first incident.

Message Queues vs. Pub/Sub, Brokers, and Event Streams

Versus pub/sub. A queue delivers each message to one consumer; pub/sub delivers a copy to every subscriber. The test is what a second consumer should do: if it should share the work, you want a queue; if it should also react, you want pub/sub. Most real systems use both.

Versus a message broker. A queue is a component; a message broker is the product you run to get one. The broker supplies everything around the queue — routing rules, both messaging patterns side by side, acknowledgment bookkeeping, failure destinations — which is why "should we use a queue?" and "should we deploy RabbitMQ?" are different questions with different answers.

Versus an event stream. A stream is an append-only log that retains messages after they are read, so consumers track their own position and can replay history. A queue deletes on acknowledgment. Kafka is the usual point of confusion, and the answer changed in 2026: share groups, added by KIP-932 and generally available in Kafka 4.2, brought per-record acknowledgment and broker-side delivery counting, so Kafka can now serve queue workloads directly. It still has no dead letter queue for share groups, no message priority, and no delayed delivery. Our comparison of Redis and Kafka covers the decision in more detail.

Message Queues in Java

Java has queues built in. java.util.Queue defines the contract, and BlockingQueue adds the operations that make a producer-consumer pipeline practical — put blocks when the queue is full, take blocks when it is empty, and both hand off between threads safely.

BlockingQueue<Order> queue = new ArrayBlockingQueue<>(1000);

// producer thread
queue.put(order);

// consumer thread
Order order = queue.take();
process(order);

This is a genuine message queue, and for coordinating threads inside one application it is the right answer. Its limits are structural rather than fixable: the queue lives in one JVM's heap, so a second instance of the service cannot see it, and everything still in the queue is lost on restart or crash. There is no acknowledgment either — take removes the element, and if the worker dies immediately afterwards the work is simply gone. See our overview of the Java queue interfaces for the full hierarchy, including priority queues.

Crossing the process boundary means a distributed queue, and the standards-based way to talk to one in Java is JMS (Jakarta Messaging since the javax.jms to jakarta.jms rename), which gives you JmsTemplate, @JmsListener, and a provider-independent API.

Message Queues on Redis and Valkey

Redis and Valkey are frequently used as queues because they are already in the stack and extremely fast. There are two native approaches, and their limitations are worth understanding precisely.

Lists are the naive queue. LPUSH appends, BRPOP blocks until something arrives, and you have a working producer-consumer pipeline in two commands.

LPUSH orders '{"id":"ORD-123"}'
BRPOP orders 0

The problem is that BRPOP removes the element. If the consumer crashes between popping and finishing, the message is gone — there is no acknowledgment, no visibility timeout, no redelivery. LMOVE improves on this by atomically moving the message to a processing list, but you are then responsible for detecting stalled consumers and moving abandoned entries back, which is real code with real edge cases. See Redis queue for the patterns in detail.

Streams are the serious option. XADD appends, XREADGROUP reads within a consumer group, and XACK confirms processing. Unacknowledged messages sit in the pending entries list with a delivery count, and XAUTOCLAIM reassigns them from a consumer that has stalled. That is a durable at-least-once queue with genuine acknowledgments.

What Streams do not provide is everything above the acknowledgment. The pending entries list records a delivery count but nothing acts on it — moving an exhausted message to a dead letter destination is application code you write, test, and maintain. The same applies to message priority, delayed delivery, deduplication, and delivery limits. Streams give you the primitive; the reliability features a broker ships in the box remain yours to build.

Reliable Message Queues With Redisson PRO

Redisson fills in what the raw primitives leave out. Its open-source edition turns the familiar Java message queue interfaces into distributed objects — RQueue, RBlockingQueue, RDeque, and RPriorityQueue behave like their java.util counterparts but are visible to every JVM connected to the same Redis, and their contents survive a restart.

For work where losing a message matters, Redisson PRO adds Reliable Queue (RReliableQueue), which brings broker-grade semantics to the data store you already operate. Messages are removed only on acknowledgment, delivery attempts are counted, and exhausted messages are routed to a dead letter queue rather than dropped.

RReliableQueue<Order> queue = redisson.getReliableQueue("orders");
queue.setConfigIfAbsent(QueueConfig.defaults()
    .deliveryLimit(5)
    .visibility(Duration.ofSeconds(60))
    .timeToLive(Duration.ofHours(24))
    .deadLetterQueueName("orders-dlq"));

// Producer
queue.add(QueueAddArgs.messages(MessageArgs.payload(new Order("ORD-123", 99.99))));

// Consumer
Message<Order> msg = queue.poll(QueuePollArgs.defaults()
    .visibility(Duration.ofSeconds(30))
    .acknowledgeMode(AcknowledgeMode.MANUAL));

if (msg != null) {
    if (processOrder(msg.getPayload())) {
        queue.acknowledge(QueueAckArgs.ids(msg.getId()));
    } else {
        queue.negativeAcknowledge(QueueNegativeAckArgs
            .failed(msg.getId())
            .delay(Duration.ofSeconds(10)));
    }
}

Beyond the acknowledgment loop, Reliable Queue covers the features that otherwise become application code: priority levels from 0 to 9, delayed delivery, deduplication by message ID or payload hash within a configurable window, size caps on both individual messages and the queue as a whole, and a choice between processing messages in parallel or strictly in order. Durability is tunable per operation too — a single message can be required to land on replicas before the call returns, which closes the asynchronous-replication window described earlier. Teams who want a standards-based API rather than a proprietary one can reach the same queue through Redisson's JMS 2.0, 3.0, and 3.1 implementations, using JmsTemplate and @JmsListener as normal.

The full walkthrough is in our guide to Reliable Queue for Valkey and Redis.

Message Queue: Frequently Asked Questions

What Is a Message Queue Used For?

A message queue is used to move work out of a request path so it can be processed asynchronously — sending emails, generating documents, processing payments, resizing images. It also levels load by absorbing traffic spikes, isolates failure so an unavailable downstream service does not break the caller, and lets producers and consumers be deployed and scaled independently.

What Is the Difference Between a Message Queue and a Message Broker?

Scope. A queue is one thing — a buffer with a delivery model attached. A broker is the software that runs many of them, and it adds the parts a bare queue has no opinion about: which destination a message should land in, how topics and queues coexist, what happens on repeated failure. You can have a queue without a broker, which is what using Redis lists or a database table amounts to. You cannot have a broker without queues underneath it. See message broker for the full comparison.

Is Kafka a Message Queue?

Kafka is a distributed log rather than a queue, but as of Kafka 4.2 it can act as one. Share groups, introduced by KIP-932, allow more consumers than partitions to consume a topic cooperatively with per-record acknowledgment and broker-side delivery counting. Share groups do not yet provide a dead letter queue, message priority, or delayed delivery, and they trade ordering for elastic scaling, so a purpose-built queue is still the better fit for job processing that needs those features. Our post on whether Kafka is a message queue covers what share groups did and did not change.

What Is the Difference Between a Message Queue and Pub/Sub?

A message queue delivers each message to exactly one consumer, so adding consumers distributes the work. Pub/sub delivers a copy of each message to every subscriber, so adding a subscriber adds a recipient rather than capacity. Queues suit task distribution; pub/sub suits broadcasting events that several services react to differently.

Does a Message Queue Guarantee Message Order?

Only with a single consumer. Queues are first-in, first-out by design, but as soon as several consumers read the same queue in parallel they finish at different times and ordering is lost. Systems that offer both ordering and parallelism partition the queue — SQS FIFO by message group ID, Kafka by partition key — so messages sharing a key are handled in order while different keys process concurrently.

Can Redis Be Used as a Message Queue?

Yes. Redis lists support a basic queue through LPUSH and BRPOP, but with no acknowledgment, so a message is lost if the consumer crashes mid-processing. Redis Streams add consumer groups and acknowledgments for durable at-least-once delivery, though delivery limits, dead letter queues, priority, and delayed delivery must still be implemented in application code. A client such as Redisson PRO provides those features as configuration. The same applies to Valkey, which shares the same primitives.

Similar terms

• Message Broker
• Redis Queue
• Dead Letter Queue
• Pub/sub
• Redis Streams
• JMS
• Java Queue
• Event-Driven Architecture
• Idempotency
• Microservices
• Outbox Pattern