Google Pub/Sub vs Kafka: Key Differences, Costs and How to Choose

Google Cloud Pub/Sub is a serverless messaging service: you create topics and subscriptions, Google supplies the capacity, and you pay for the bytes that pass through. Apache Kafka is an open-source, partitioned log that you run yourself or rent, and on Google Cloud you can now rent it from Google, as Managed Service for Apache Kafka. Pub/Sub tracks every message in every subscription and lets consumers acknowledge messages one at a time. Kafka keeps records in order within partitions, and each consumer group records how far it has read.

The usual shorthand, that Pub/Sub is a queue and Kafka is a log, is out of date in both directions. Pub/Sub topics and subscriptions can keep messages for up to 31 days and replay them, and Kafka's share groups, production-ready since Kafka 4.2, acknowledge records one by one. What still separates the two is who runs the capacity and how you pay for it, how much history you can keep, whether you can have per-message acknowledgement and per-key ordering at the same time, and whether your code is tied to Google Cloud.

Redisson is our product. We sell neither Pub/Sub nor Kafka; Redisson PRO's Reliable PubSub, which runs on Valkey or Redis, is covered near the end of this page, limits included. Google Cloud details were checked against Google's documentation and price lists on November 5, 2026, and Kafka details against Apache Kafka 4.4.0, released on October 22, 2026.

Google Pub/Sub vs Kafka at a Glance

On Google Cloud, the Choice Is Usually Pub/Sub or Managed Kafka

For a team on Google Cloud, choosing Kafka no longer means building a cluster. Google Cloud Managed Service for Apache Kafka became generally available on November 12, 2024, after a preview under the name Apache Kafka for BigQuery. Google runs the brokers in KRaft mode across three zones, applies critical security patches and moves older data to tiered storage, which it describes as "effectively unlimited". You still size the cluster, at least 3 vCPUs with 1 to 8 GiB of memory each, and you choose partition counts, replication and retention per topic. Managed Kafka Connect clusters have been generally available since October 1, 2025, without custom connector plugins; a schema registry for Avro and Protobuf is in preview; and clients authenticate with SASL/OAUTHBEARER and Google credentials, SASL/PLAIN with a service account key, or mutual TLS.

The service's documentation lists Apache Kafka 3.7.1 as the version its clusters run and says minor versions are not upgraded automatically. Kafka 4.x features such as share groups therefore need a cluster you run yourself, on Compute Engine or GKE, or a third-party Kafka service that offers them; check the version your managed cluster reports before counting on them.

Google's own advice calls the decision "a tradeoff between operational simplicity and portability" and sums it up in two sentences: "For new workloads or users new to streaming on Google Cloud, Pub/Sub is recommended due to its ease of use. If you want to move your existing Kafka workloads to the Cloud with minimal code changes, Managed Service for Apache Kafka is the ideal choice." Its comparison table has not kept up, though. It still says the managed service has no data-plane SLA (the SLA has covered the Produce API since October 2026), needs user-provided Kafka Connect clusters and has no managed schema registry.

Pub/Sub Lite, a third Google option with zonal or regional topics, reserved capacity and partitions, stopped accepting new customers after September 24, 2024. Google first set its turndown for March 18, 2026; its documentation now gives January 31, 2027, although the pricing page still shows the earlier date. Google's migration guides lead to Managed Service for Apache Kafka or to Pub/Sub.

Running Kafka yourself stays an option. Google's own price comparison puts a self-run cluster on Compute Engine at about $900 a month against $1,100 for the managed service at 10 MiB/s, before counting the people who operate it.

Acknowledgements, Retries and Dead Letters

In Pub/Sub, acknowledgement state belongs to each message in each subscription. A subscriber must acknowledge a message before its deadline, which is 10 to 600 seconds and 10 by default, and the client libraries extend the deadline automatically while your code works, for up to an hour by default. A nack, or a deadline that runs out, makes the message available again, and unless it carries an ordering key nothing else in the subscription waits for it. The subscription's retry policy decides how soon: immediately by default, or with exponential backoff between a minimum and a maximum of your choosing, at most 600 seconds. Google lists this per-message tracking "to minimize pipeline effects due to single bad messages" among its reasons to choose Pub/Sub.

A Kafka consumer group keeps no state per record. Each consumer owns some partitions, reads them in order and commits an offset, and everything before the offset counts as processed. There is no nack. When a record fails, the application retries it, writes it to a topic of its own for later, or holds the partition until the problem is fixed. Frameworks such as Spring for Apache Kafka package those choices, Kafka Streams has had a dead-letter queue since Kafka 4.2, and Kafka Connect has one for sink connectors. A consumer must also poll at least every five minutes by default (max.poll.interval.ms), or the group hands its partitions to another consumer.

Share groups, production-ready since Kafka 4.2 in February 2026, bring per-record acknowledgement to Kafka. Consumers in a share group take records from the same partitions; each record is locked for 30 seconds by default, and the consumer accepts, releases or rejects it, or renews the lock. A released record is delivered again until it has been delivered 5 times, the default limit, and is then archived. Is Kafka a Message Queue? walks through share groups and shows a share-group consumer.

Dead-Letter Topics

Pub/Sub and Kafka share groups can both move a message that keeps failing to a separate topic, with different defaults:

• Pub/Sub: a subscription gets a dead-letter topic and a maximum of 5 to 100 delivery attempts, 5 by default. The Pub/Sub service account needs permission to publish to the dead-letter topic and to subscribe to the source subscription. Google calls the attempt count approximate, because forwarding is best effort, and the Java client reports it in the googclient_deliveryattempt attribute.
• Kafka share groups: since Kafka 4.4 (KIP-1191), once the share.version feature is upgraded to 2, a share group can write rejected records, and records that reach the delivery limit, to a dead-letter topic named with errors.deadletterqueue.topic.name. The topic name must start with dlq. by default, the topic has to opt in with errors.deadletterqueue.group.enable, and the dead-letter record carries only headers that describe the original unless errors.deadletterqueue.copy.record.enable is set.
• Kafka consumer groups: no dead-letter topic; the application or framework writes failed records to one.

In every case someone has to read the dead-letter topic. A message lands there only after its retries are used up, so the useful next steps are an alert, a fix and a controlled replay, as our dead letter queue guide describes.

Ordering and Parallelism

Pub/Sub delivers in order only when asked, and only per ordering key. The subscription must be created with message ordering enabled, a setting that cannot be changed later, the publisher must enable ordering, and messages with the same key must be published in the same region. Each key carries at most 1 MBps. According to Google's ordering documentation, a redelivery of one message triggers the redelivery of every later message with the same key, "even acknowledged ones", so consumers must tolerate repeats. When a publish fails, the client library also fails later messages with that key until the application calls resumePublish.

Kafka orders records within a partition. A record's key chooses its partition, so records with the same key stay in order, and a consumer group can run at most one consumer per partition. Parallelism is therefore planned up front through the partition count, and adding partitions later sends existing keys to different partitions, which breaks per-key order across the change. Google's comparison table describes it as "Per-partition ordering up to throughput capacity of a partition", and its guidance recommends Managed Kafka when "You require message ordering at high throughput per key".

Pub/Sub gives you per-message acknowledgement and per-key ordering together, which neither Kafka consumer model does on its own. Retries and dead letters are run by the service, so a message that keeps failing mainly holds up the messages with its own key. The same documentation adds two caveats: unacknowledged messages for one key "can potentially delay delivery of messages for other ordering keys", especially during server restarts or traffic changes, and "order might not be preserved when messages are written to a dead-letter topic".

On Kafka you get both only by writing the code. Consumer groups keep partition order but acknowledge by offset, so per-message retries and parking are application code, as the Java sample below shows, and a record that keeps failing holds up its whole partition until that code moves it aside. Share groups acknowledge each record, but Kafka's documentation says they share records "at the expense of record ordering", and KIP-932 lists key-based ordering only as future work.

For Java services, Redisson PRO's Reliable PubSub, covered below, also pairs per-message acknowledgement with per-key order, through message groups, though it does not guarantee the order of redelivered messages within a group.

Exactly-Once Means Different Things

Both products advertise exactly-once, for different parts of the pipeline. Pub/Sub's exactly-once delivery, generally available since December 2022, is a subscription setting. With it on, a message is not redelivered after a successful acknowledgement or while it is still outstanding, and the subscriber learns whether each acknowledgement succeeded. In Java that comes through AckReplyConsumerWithResponse, which Google still marks as beta. It works only on pull subscriptions, StreamingPull included, and only while a subscription's subscribers all connect to Pub/Sub in the same region; push and export subscriptions don't support it. It also leaves publish-side duplicates alone: after publisher retries, a subscription "might receive multiple copies of the same message".

Kafka's exactly-once semantics come from idempotent producers, on by default since Kafka 3.0, and transactions. An application that reads from one topic and writes to another can commit its output and its consumer offsets in one transaction, so a crash neither loses nor duplicates the output. Google's comparison table states the boundary: "exactly-once side effects when reading from one topic and writing to another".

Neither covers the step most services care about, which is writing to a database or calling another API. That still takes an idempotent consumer, for example a processed-ID check inside the same database transaction, or the outbox pattern on the producing side.

Retention and Replay

A Pub/Sub subscription keeps unacknowledged messages for 7 days by default, configurable from 10 minutes to 31 days. Two settings keep acknowledged messages for replay: topic retention, up to 31 days, which every subscription on the topic can use, and retention of acknowledged messages on a single subscription. With either, you can seek a subscription back to a timestamp, and Pub/Sub delivers everything after it again. Snapshots record a subscription's acknowledgement state for a later seek and last at most 7 days. Retained messages cost $0.27 per GiB-month, and Google points out that topic retention is the cheapest way to keep them, because all subscriptions share it.

With a 31-day ceiling, Pub/Sub cannot be the system of record for an event-sourced service, and Google's own list of reasons to choose Managed Kafka includes "You want to use the event sourcing pattern with an event log as a source of truth".

Kafka keeps a topic's records for as long as its retention says, 7 days by default and indefinitely if you choose, and a compacted topic keeps the latest record for each key without a time limit. Tiered storage, production-ready since Kafka 3.9, moves older segments to object storage, though not for compacted topics. Managed Service for Apache Kafka tiers older data automatically, again except on compacted topics, and bills long-term storage for one copy of the data at about $0.10 per GiB-month. A new consumer group can start at the earliest retained record, and an existing group can reset its offsets to a point in time.

The Same Consumer in Java

The samples below process order events. The Pub/Sub version publishes with an ordering key and consumes with per-message acknowledgements, and leaves retries and dead letters to the subscription's configuration.

import com.google.api.core.ApiFutureCallback;
import com.google.api.core.ApiFutures;
import com.google.api.gax.batching.FlowControlSettings;
import com.google.cloud.pubsub.v1.AckReplyConsumer;
import com.google.cloud.pubsub.v1.MessageReceiver;
import com.google.cloud.pubsub.v1.Publisher;
import com.google.cloud.pubsub.v1.Subscriber;
import com.google.common.util.concurrent.MoreExecutors;
import com.google.protobuf.ByteString;
import com.google.pubsub.v1.ProjectSubscriptionName;
import com.google.pubsub.v1.PubsubMessage;
import com.google.pubsub.v1.TopicName;
import java.util.concurrent.TimeUnit;

// Publisher: messages with the same ordering key arrive in order if the
// subscription has message ordering enabled and every message with that
// key is published in one region, hence the regional endpoint.
Publisher publisher = Publisher.newBuilder(TopicName.of(project, "orders"))
        .setEndpoint("us-central1-pubsub.googleapis.com:443")
        .setEnableMessageOrdering(true)
        .build();
PubsubMessage event = PubsubMessage.newBuilder()
        .setData(ByteString.copyFromUtf8(json))
        .setOrderingKey(customerId)           // at most 1 MBps per key
        .putAttributes("type", "OrderPlaced") // subscriptions can filter on it
        .build();
ApiFutures.addCallback(publisher.publish(event), new ApiFutureCallback<>() {
    @Override
    public void onSuccess(String messageId) {
    }

    @Override
    public void onFailure(Throwable error) {
        // The key is now paused, and every message queued behind this one
        // fails too. Keep the failures to republish in order, then resume.
        keepForRetry(event, error);
        publisher.resumePublish(customerId);
    }
}, MoreExecutors.directExecutor());
// publish() queues the message for a batch; shutdown() sends what is still
// queued, so skipping it on exit loses messages.
publisher.shutdown();
publisher.awaitTermination(1, TimeUnit.MINUTES);

// Subscriber: each message is acknowledged on its own. nack() asks for a
// redelivery, delayed by the subscription's retry policy if it has a
// backoff; after the dead-letter policy's maximum delivery attempts,
// Pub/Sub forwards the message to the dead-letter topic.
MessageReceiver receiver = (PubsubMessage message, AckReplyConsumer reply) -> {
    try {
        process(message);
        reply.ack();
    } catch (RuntimeException e) {
        reply.nack();
    }
};
Subscriber subscriber = Subscriber.newBuilder(
                ProjectSubscriptionName.of(project, "orders-billing"), receiver)
        .setFlowControlSettings(FlowControlSettings.newBuilder()
                .setMaxOutstandingElementCount(500L) // unacked, per client
                .build())
        .build();
subscriber.startAsync().awaitRunning();

The Kafka version uses a consumer group, the model that Managed Service for Apache Kafka supports today. The record key provides the ordering, and retries and dead letters move into the application.

import static java.nio.charset.StandardCharsets.UTF_8;

import java.time.Duration;
import java.util.List;
import java.util.Properties;
import org.apache.kafka.clients.consumer.ConsumerRecord;
import org.apache.kafka.clients.consumer.ConsumerRecords;
import org.apache.kafka.clients.consumer.KafkaConsumer;
import org.apache.kafka.clients.producer.ProducerRecord;
import org.apache.kafka.common.serialization.StringDeserializer;

Properties props = new Properties();
props.put("bootstrap.servers", bootstrapServers);
props.put("group.id", "orders-billing");
props.put("enable.auto.commit", "false");
props.put("key.deserializer", StringDeserializer.class.getName());
props.put("value.deserializer", StringDeserializer.class.getName());
// Managed Service for Apache Kafka also needs its SASL settings, with
// OAUTHBEARER and Google credentials as the recommended option.

try (KafkaConsumer<String, String> consumer = new KafkaConsumer<>(props)) {
    consumer.subscribe(List.of("orders"));   // records keyed by customer ID
    while (running) {
        ConsumerRecords<String, String> batch =
                consumer.poll(Duration.ofSeconds(1));
        for (ConsumerRecord<String, String> record : batch) {
            try {
                processWithRetries(record);  // retries are application code
            } catch (RuntimeException e) {
                // Consumer groups have no nack and no dead-letter topic. Park
                // the record yourself: retrying forever blocks the partition,
                // and committing past it loses the record.
                ProducerRecord<String, String> parked = new ProducerRecord<>(
                        "orders.dlt", null, record.key(), record.value(),
                        record.headers());
                parked.headers().add("error", e.toString().getBytes(UTF_8));
                deadLetters.send(parked).get();
            }
        }
        // One position per partition: every record before it counts as done.
        // If the batch took longer than max.poll.interval.ms, the group has
        // reassigned the partitions, and commitSync() throws.
        consumer.commitSync();
    }
}

Three things change when code moves between the two. The unit of acknowledgement grows from a message to a position in a partition. Retry and dead-letter behavior that a Pub/Sub subscription is configured with becomes application code, or share-group settings on a self-run Kafka 4.4 cluster. And the publishing side changes shape: Pub/Sub's ordering key becomes the Kafka record key, and an attribute filter on the subscription becomes a check in the consumer.

Filtering, Push and Integrations

A Pub/Sub subscription filter matches message attributes only, with an expression of up to 256 bytes that cannot be changed after the subscription is created. Messages that don't match are acknowledged automatically and still billed for throughput. Filtering on the payload takes a single message transform, described below, which drops a message by returning null. Kafka has no filter in the broker: each consumer group reads every record of the topics it subscribes to and discards what it doesn't need, or a Kafka Streams or Flink job writes a filtered topic.

Pub/Sub can also deliver without a consumer process. Push subscriptions send each message to an HTTPS endpoint, such as a Cloud Run service, which acknowledges it by returning a success status. Export subscriptions write straight to BigQuery or Cloud Storage at $50 per TiB, and Bigtable subscriptions are in preview. Kafka's equivalent is a sink connector on Kafka Connect, which Google now runs as a managed service.

Single message transforms, generally available since June 2025, run a JavaScript function on each message, on a topic before Pub/Sub stores it or on a subscription before delivery, with up to five transforms per topic or subscription and 500 milliseconds per message. An AI Inference transform, which enriches each message with a model's response, followed in April 2026. Import topics bring data into Pub/Sub from Amazon Kinesis Data Streams, Cloud Storage, Azure Event Hubs, Amazon MSK and Confluent Cloud. Kafka's processing side is larger: Kafka Streams, ksqlDB and Flink run stateful jobs with windows and joins, work that Pub/Sub leaves to Dataflow. Our stream processing guide covers what those jobs involve.

What Each Costs

Pub/Sub charges for throughput: $40 per TiB after the first 10 GiB each month, counted once when a message is published and again for every subscription that receives it. Three details move the bill. Each publish request is billed at no less than 1 KB, so small messages should be batched. Messages a filter discards still count. And retained data costs $0.27 per GiB-month, whether it comes from topic retention, retained acknowledged messages, snapshots or a backlog older than a day. Pub/Sub charges nothing for traffic between zones, and charges for traffic that crosses a region boundary.

Managed Service for Apache Kafka charges for capacity. Compute is billed in data compute units at $0.09 per DCU-hour, where a vCPU counts as 0.6 DCU and a GiB of memory as 0.1, which comes to $0.054 per vCPU-hour and $0.009 per GiB-hour. Local storage is billed at 100 GB per vCPU, or at the actual size if you configure larger broker disks, data in long-term storage is billed for one copy, and replicating writes to the other two zones costs $0.01 per GiB. Google says this inter-zone transfer becomes the largest part of the bill once a cluster runs above 20 percent utilization. For sizing, it estimates 20 MBps per vCPU of write-equivalent traffic, a figure that counts replication.

Google publishes a monthly estimate for each service. The assumptions differ: the Pub/Sub example uses pull or push subscriptions and a 24-hour storage period, and the Kafka estimate uses three replicas, 4 GiB of memory per vCPU, 50 percent utilization, 24 hours of storage, and consumers that read as much as producers write, from a follower replica in their own zone. Use them to see which way the comparison leans, and price your own workload before deciding.

Monthly list prices from Google's Pub/Sub and Managed Service for Apache Kafka pricing pages (North America and us-central1), November 2026.

At steady volume, Kafka is cheaper, and every additional Pub/Sub subscription widens the gap. Each subscription costs another $40 per TiB, about 4 cents per GiB, while an extra Kafka consumer group adds read load and at most a cent per GiB for reads across zones. The order flips at the bottom of the range. Pub/Sub costs nothing for the first 10 GiB, while the smallest Managed Kafka cluster, 3 vCPUs, costs about $140 a month in compute with the minimum 1 GiB of memory per vCPU, or about $200 with the 4 GiB that Google recommends, plus about $50 for local storage, before any traffic. Low or bursty volume favors Pub/Sub, and high, steady volume read by several consumers favors Kafka. The other cost is people. Pub/Sub has no capacity to plan, and Google lists "You are willing to handle capacity management" among the reasons to choose Managed Kafka.

Moving Between Them

Pub/Sub has no Kafka protocol endpoint, so moving from Kafka to Pub/Sub means rewriting every producer and consumer against Pub/Sub's client libraries. Data can flow while that happens. Import topics read from Amazon MSK, Confluent Cloud and Azure Event Hubs, at $80 per TiB. A cluster you run yourself can use Google's Pub/Sub Group Kafka Connector instead: its CloudPubSubSinkConnector publishes Kafka records to Pub/Sub, its CloudPubSubSourceConnector does the reverse, and Google's Kafka-to-Pub/Sub migration guide builds a phased migration on it.

Moving existing Kafka applications onto Managed Service for Apache Kafka keeps the client code. What changes is authentication, to SASL/OAUTHBEARER with Google credentials, SASL/PLAIN with a service account key or mutual TLS, and the copying of data, for which Kafka's MirrorMaker 2 runs on Kafka Connect. Compare the Kafka version Google documents with the client features you rely on before you plan the cut-over.

Going the other way, from Pub/Sub to Kafka, the ordering key becomes the record key and the partition count becomes a design decision. Retry policies and dead-letter topics move into consumer code, or into share-group settings on a Kafka 4.4 cluster you run, and push subscriptions become sink connectors.

Pub/Sub Lite users have a deadline of January 31, 2027. Google's guides cover both targets: Managed Service for Apache Kafka, whose guide was updated in June 2026 to use Kafka Connect, and Pub/Sub. Lite's partitions and reserved capacity map more directly onto Kafka, while Pub/Sub removes capacity planning at the price of a new API.

Whichever direction you take, run old and new side by side for a while and make consumers idempotent, because a bridge that retries can deliver a message twice. Retire the old path once nothing reads from it.

A Third Option for Java Services: Reliable PubSub

Some teams comparing Pub/Sub and Kafka want Pub/Sub's per-message model without the 31-day limit or the tie to Google Cloud, and some want Kafka's replay without running brokers. If your services run on the JVM and already use Valkey or Redis, for example on Memorystore for Valkey, which Google made generally available in April 2025, there is a third option.

Redisson PRO's Reliable PubSub is a topic object stored in Valkey or Redis. It has Pub/Sub's shape: topics with subscriptions, consumers that acknowledge or reject each message, a visibility timeout, a delivery limit and a dead-letter topic per subscription. It also keeps history like a log, so a subscription can start at the earliest retained message or be moved with seek(). And it adds what neither Pub/Sub nor Kafka has: priorities from 0 to 9, delay per message, and deduplication by ID or by payload hash. The Reliable PubSub documentation lists every setting.

import java.time.Duration;
import java.time.Instant;
import org.redisson.api.MessageArgs;
import org.redisson.api.RReliablePubSubTopic;
import org.redisson.api.SyncFailureMode;
import org.redisson.api.pubsub.MessageAckArgs;
import org.redisson.api.pubsub.MessageListenerArgs;
import org.redisson.api.pubsub.MessageNegativeAckArgs;
import org.redisson.api.pubsub.Position;
import org.redisson.api.pubsub.PublishArgs;
import org.redisson.api.pubsub.Subscription;
import org.redisson.api.pubsub.SubscriptionConfig;
import org.redisson.api.pubsub.TopicConfig;

RReliablePubSubTopic<Order> topic = redisson.getReliablePubSubTopic("orders");
// Three days of history. Without a time-to-live the topic keeps every
// message, and the history grows until memory runs out.
topic.setConfig(TopicConfig.defaults()
        .timeToLive(Duration.ofDays(3)));

// The counterpart of a Pub/Sub subscription with a dead-letter policy
Subscription<Order> billing = topic.createSubscription(
        SubscriptionConfig.name("orders-billing")
                .deliveryLimit(5)                  // maximum delivery attempts
                .visibility(Duration.ofMinutes(1)) // the ack deadline
                .deadLetterTopicName("orders-billing-dlt")
                .retainAfterAck());                // replay acked messages too

billing.createPushConsumer().registerListener(
        MessageListenerArgs.listener((message, ack) -> {
            try {
                process(message.getPayload());
                ack.acknowledge(MessageAckArgs.ids(message.getId()));
            } catch (RuntimeException e) {
                // Redelivered after 30 seconds; at the limit, dead-lettered
                ack.negativeAcknowledge(MessageNegativeAckArgs
                        .failed(message.getId())
                        .delay(Duration.ofSeconds(30)));
            }
        }));

// A publisher retry that repeats the order ID within an hour is dropped.
// THROW_EXCEPTION makes publish() fail when the write is not confirmed in
// time; by default an unconfirmed write only logs a warning.
topic.publish(PublishArgs.messages(MessageArgs.payload(order)
                .deduplicationById(order.getId(), Duration.ofHours(1)))
        .syncFailureMode(SyncFailureMode.THROW_EXCEPTION));

// Replay the last hour for this subscription only
billing.seek(Position.timestamp(Instant.now().minus(Duration.ofHours(1))));

The visibility timeout plays the part of Pub/Sub's ack deadline, 30 seconds by default where Pub/Sub's is 10; the sample sets a minute. For per-customer order, set a group ID on each message with MessageArgs.payload(event).groupId(customerId) (Redisson PRO 4.0.0 and later). It is the closest thing to an ordering key, with one difference: Pub/Sub redelivers the later messages of a key along with a failed one, while Redisson's documentation says the order of redelivered messages in a group isn't guaranteed. Google Pub/Sub vs Redis for Messaging compares Reliable PubSub with Pub/Sub feature by feature, and Apache Kafka vs Reliable PubSub does the same for Kafka.

It does not fit every case:

• It is a JVM library. Services written in other languages need Pub/Sub's or Kafka's clients.
• History lives in Valkey or Redis memory, so how far back you can replay depends on how much memory you pay for. If you do not need replay, the SUBSCRIPTION_REQUIRED_DELETE_PROCESSED retention mode keeps a message only until every subscription that existed when it was published has processed it, so memory holds just the backlog. Give topics a time-to-live, size the instance for the deepest backlog you expect, and change Memorystore for Valkey's default eviction policy, volatile-lru, to noeviction, so that memory pressure can never evict messaging data.
• Memorystore for Valkey writes to disk only if you enable RDB snapshots or AOF, AOF syncs once a second by default, and replicas are asynchronous, so a failover can lose recent writes. For confirmed writes, keep the default syncMode, AUTO, which waits for the AOF on the primary and replicas when AOF is enabled and for replica acknowledgement otherwise, and set syncFailureMode(SyncFailureMode.THROW_EXCEPTION) as the sample does; by default a write that misses the one-second sync timeout only logs a warning. Our persistence guide covers the trade-offs.
• There is no push to HTTPS endpoints, no export to BigQuery or Cloud Storage, no transforms, and no Kafka Connect or Kafka Streams; integrations are your own code.
• It runs in one region. Memorystore for Valkey's cross-region secondaries are read-only and asynchronous, so they are for disaster recovery, and publishing stays in one region.

How to Choose

The two can also run together. With a connector or an import topic between them, Pub/Sub handles ingestion and push delivery at the edges while Kafka keeps the history in the middle. For options beyond these two, see Apache Kafka alternatives, Redis vs Kafka, and the managed services on other clouds: Amazon Kinesis and Azure Service Bus.

Frequently Asked Questions

Are Kafka and Pub/Sub the same?

No. Google Cloud Pub/Sub is a managed messaging service with its own API, billed per byte, that tracks and acknowledges each message per subscription and keeps messages for at most 31 days. Apache Kafka is an open-source partitioned log that you run or rent, with its own protocol, in which consumers track an offset per partition and history can be kept indefinitely. Both implement publish/subscribe: one topic, many independent readers.

Is Kafka a pub/sub or a queue?

Both. Consumer groups give publish/subscribe, because every group reads every record. Inside a group, consumers split the partitions between them, which works like a set of ordered queues. Since Kafka 4.2, share groups add queue semantics proper: several consumers take records from the same partitions and acknowledge each one, at the cost of ordering.

What is the GCP equivalent of Kafka?

Kafka itself, as Google Cloud Managed Service for Apache Kafka, generally available since November 2024. It runs open-source Kafka, so existing Kafka clients and tools work with it. Pub/Sub is Google's native alternative, with a different API. Pub/Sub Lite, the partitioned service that once sat between them, is closed to new customers and will be turned down on January 31, 2027.

What is Pub/Sub on GCP?

Pub/Sub is Google Cloud's serverless messaging service. Publishers send messages to topics, and each subscription attached to a topic receives a copy, delivered by pull, by push to an HTTPS endpoint, or by export to BigQuery or Cloud Storage. Google scales it without capacity settings and charges $40 per TiB of throughput after the first 10 GiB each month.

Is Google Pub/Sub cheaper than Kafka?

At low or bursty volume, yes: the first 10 GiB a month are free and there is no cluster to pay for, while the smallest Managed Service for Apache Kafka cluster costs about $190 to $250 a month before any traffic, depending on its memory. At steady volume Kafka usually costs less. Google's pricing pages put 10 MiB/s at about $2,000 a month on Pub/Sub with one subscription and about $1,100 on Managed Service for Apache Kafka, and every extra Pub/Sub subscription adds to the bill.

Does Google Pub/Sub guarantee message order?

Only per ordering key, and only when it is turned on. The subscription needs message ordering enabled when it is created, the publisher sets an ordering key on each message, and messages with the same key must be published in the same region. Each key is limited to 1 MBps, and a redelivery replays the later messages with that key as well.

Does Google Pub/Sub support exactly-once delivery?

Yes, on pull subscriptions within a single region, since December 2022. With it enabled, an acknowledged message is not redelivered and subscribers can confirm that each acknowledgement succeeded. Push and export subscriptions don't support it, and duplicates created by publisher retries can still arrive, so consumers with side effects still need to be idempotent.

Can Kafka clients connect to Google Pub/Sub?

No. Pub/Sub has its own API and no Kafka protocol endpoint. Kafka applications on Google Cloud either keep their clients and move to Managed Service for Apache Kafka, or are rewritten against Pub/Sub's client libraries. Google's Pub/Sub Group Kafka Connector and Pub/Sub import topics can move data between the two during a migration.

Similar articles

• Google Pub/Sub vs Redis for Messaging
• Apache Kafka vs Valkey & Redis Based Reliable PubSub
• Is Kafka a Message Queue?
• Apache Kafka Alternatives: 7 Options Compared
• Amazon Kinesis vs Valkey & Redis Based Reliable PubSub and Queue
• Azure Service Bus vs Valkey & Redis Based Reliable Queue and PubSub
• All About Reliable PubSub for Valkey and Redis