Amazon Kinesis vs Kafka: Key Differences, Costs and How to Choose

Published on
November 17, 2026

Amazon Kinesis Data Streams and Apache Kafka are both logs. A producer writes records with a key, the key picks a shard in Kinesis or a partition in Kafka, records stay in order within it, and each consuming application keeps its own position, so it can go back and read again. The difference starts with who owns the log. Kinesis is an AWS service, reached through its own API and billed by the shard or by the gigabyte, and it runs nowhere else. Kafka is open source with its own protocol, and the managed version on AWS is Amazon MSK, which comes as Standard brokers, Express brokers or MSK Serverless.

On AWS the practical choice is therefore Kinesis or MSK, and it turns on whether your code and tools already speak Kafka, how many applications read each stream, how far back they need to read, what a failed record should do, and how the bills compare at your volume. Both services have changed a lot since late 2025. Kinesis gained 10 MiB records, a cheaper On-demand Advantage mode with an account minimum, up to 50 enhanced fan-out consumers per stream and direct delivery to Amazon S3. MSK gained Kafka 4.2 on Express brokers and its own delivery to S3. One Kafka feature is still only a preview on MSK: share groups, the per-record acknowledgement that Apache Kafka declared production-ready in version 4.2.

Redisson, a Java client for Valkey and Redis, is our product, and we sell neither Kinesis nor Kafka. Reliable PubSub, a messaging feature of Redisson PRO, gets its own section near the end, along with what it can't do. We checked the AWS details against AWS documentation and price lists on November 17, 2026, the Kafka facts against the Apache Kafka 4.4.0 release of October 22, 2026, and KCL behavior against KCL 3.5.4, released October 30, 2026. The Java samples compile against those versions.

Amazon Kinesis vs Kafka at a Glance

Amazon Kinesis Data StreamsApache Kafka on Amazon MSK
What it isAn AWS service: a managed, sharded log that runs only on AWSAn open-source, partitioned, replicated log; on AWS, MSK runs it as Standard brokers, Express brokers or Serverless
APIThe Kinesis API, through the AWS SDKs, the Kinesis Client Library (KCL) and the Kinesis Producer LibraryThe Kafka protocol: any Kafka client, Kafka Connect, Kafka Streams, Flink
Unit of order and scaleShard, picked by hashing the partition key with MD5Partition, picked by hashing the record key
Write capacityEach shard takes 1 MB/s or 1,000 records/s; an on-demand stream begins at 4 MB/s and can grow to 10 GB/s in three Regions (200 MB/s in the rest unless AWS raises it)No fixed limit per partition on Standard brokers; Express brokers sustain 15.6 to 500 MB/s each and cap a partition at 15 MB/s; Serverless takes 200 MB/s per cluster
Largest record1 MiB, or up to 10 MiB on a stream configured for large recordsRoughly 1 MB unless raised (message.max.bytes); 8 MiB on Serverless
Retention24 hours unless extended; 365 days at mostA week unless changed, and as long as you like; tiered storage on Standard brokers, unlimited storage on Express
Compacted topicsNoYes
ReadingAll readers of a shard split 2 MB/s and five reads a second between them, unless they register for enhanced fan-out, which gives each its own 2 MB/s per shard (20 registrations per stream, 50 with On-demand Advantage)Each consumer group reads on its own, limited by broker capacity rather than a per-partition read quota
Latency, AWS figuresRoughly 200 ms for a single polling reader and 1,000 ms for five; roughly 70 ms through enhanced fan-outNo published figure; a consumer's fetch returns as soon as records arrive
Consumer progressA checkpoint per shard, kept by KCL in a DynamoDB lease table, or by LambdaA committed offset per partition, kept in Kafka
A record that failsIf your code throws, KCL moves past the batch; Lambda retries the batch until it succeeds or its records expireYour code retries it, stalling the partition, or sets it aside; Lambda behaves as it does for Kinesis
Per-record acknowledgementNoShare groups in Kafka 4.2 and later; on MSK only as a preview in Kafka 4.1
Exactly-onceNo; delivery is at least once, and you remove duplicates with an ID you embedIdempotent producer by default; transactions that commit a consumer's output and its offsets together
AWS integrationsLambda, Data Firehose, Managed Service for Apache Flink, Amazon Redshift, EventBridge Pipes; on-demand streams write to S3 and S3 Tables themselvesThe same services, plus MSK Connect for Kafka Connect connectors; Express brokers write to S3 and S3 Tables themselves
PricingShard-hours and PUT units (provisioned), stream-hours and per GB (On-demand Standard), or per GB with an account minimum (On-demand Advantage)Broker-hours and storage (Standard), plus $0.01 per GB written (Express), or cluster-hours, partition-hours and per GB (Serverless)
Runs outside AWSNoYes, on any cloud or in your own data center

On AWS, the Choice Is Kinesis or Amazon MSK

Each comes in several forms, with different prices and limits.

Kinesis Data Streams: Provisioned or On-Demand

A provisioned stream has exactly the shards you give it, each accepting 1 MB/s or 1,000 records a second of writes, and the US East price is $0.015 per shard-hour plus $0.014 per million 25 KB PUT units. Changing the shard count is your job. On-demand Standard needs no shard count: a new stream takes 4 MB/s of writes, accommodates up to double its peak of the previous 30 days, and can reach 10 GB/s in N. Virginia, Oregon and Ireland, or 200 MB/s by default in other Regions. AWS's sizing guide warns that traffic more than doubling its previous peak within 15 minutes can still be throttled. On-demand Standard costs $0.04 per stream-hour, $0.08 per GB written and $0.04 per GB read.

On-demand Advantage, launched in November 2025, is an account-level setting that covers every on-demand stream in a Region. It drops the stream-hour fee, charges $0.032 for each GB written and $0.016 for each GB read, includes enhanced fan-out in that read rate, and raises the enhanced fan-out limit from 20 consumers per stream to 50. In exchange the account is billed for no less than 25 MB/s of writes and 25 MB/s of reads, a floor that AWS's own pricing example puts at $3,037.50 a month. AWS calls it the most cost-effective way to stream with Kinesis once an account ingests at least 10 MB/s in total, fans out to more than two consumer applications or runs hundreds of streams in a Region; the cost section below tests that against provisioned shards.

Data is consumed through the Kinesis Client Library (a Java library, with other languages supported through a Java process called the MultiLangDaemon), AWS Lambda, Amazon Data Firehose and Managed Service for Apache Flink. KCL 1.x lost support on January 30, 2026, so Java consumers still on it have to be ported to KCL 3.x, and the port is a natural moment to ask whether the stream should stay on Kinesis at all.

Amazon MSK: Standard Brokers, Express Brokers or Serverless

Standard brokers are Kafka as most teams know it. You choose the broker size and count and provision EBS storage, up to 16 TiB per broker, with optional tiered storage that moves older segments to $0.06 per GB-month. They run Kafka 3.8, 3.9, 4.0 and 4.1, according to AWS's version list, which recommends 3.9, the last release with both ZooKeeper and KRaft, and promises extended support for it for at least two years. A kafka.m7g.large broker costs $0.204 an hour in US East.

Express brokers, generally available since November 2024, keep the Kafka API and replace the broker disks with storage AWS manages: nothing to size, no upper limit, $0.10 per GB-month. AWS says each broker delivers up to three times the throughput of a Standard broker, scales up to 20 times faster and recovers 90 percent faster, and its quota page gives sustained throughput for every size, from 15.6 MB/s in and 31.2 MB/s out for express.m7g.large ($0.408 an hour, plus $0.01 per GB written) to 500 MB/s in for the 16xlarge. Express clusters run in three Availability Zones only, come with preset configuration and enforced client quotas, and run Kafka 3.8, 3.9 and, since July 2026, 4.2, which MSK offers on Express brokers only. AWS also notes that Express brokers don't yet fully support the Kafka Streams API.

MSK Serverless removes sizing within fixed quotas: 200 MB/s in and 400 MB/s out per cluster, 5 MB/s in per partition, 8 MiB messages, 2,400 partitions and 500 consumer groups. Clients must authenticate with IAM. It costs $0.75 per cluster-hour, $0.0015 per partition-hour, $0.10 per GB in and $0.05 per GB out.

None of the three offers Kafka's queue mode for production yet. Share groups (KIP-932) let consumers acknowledge records one at a time; Apache Kafka declared them production-ready in 4.2 and added dead-letter topics for them in 4.4. On MSK they exist only as a preview in Kafka 4.1, and AWS's version page says KIP-932 is not yet supported on Express brokers, the only brokers where MSK runs 4.2. Using them in production today means running Kafka yourself or buying it from another provider.

AWS's own Kinesis Data Streams FAQ frames the decision by familiarity. It points teams new to streaming at Kinesis, and teams with existing Kafka applications, or a preference for open-source technology, at MSK.

Shards and Partitions

Both services hash a key to decide where a record goes, and both keep records in order only within that shard or partition. Kinesis hashes the partition key with MD5 to choose a shard, and Kafka's default partitioner hashes the record key to choose a partition. Order therefore holds per key, while different keys spread out for parallel processing.

What limits each unit differs. A Kinesis shard has fixed limits, 1 MB/s or 1,000 records a second in and 2 MB/s out, so writes for a busy partition key fail with ProvisionedThroughputExceededException once its shard is full, while an on-demand stream splits any shard that takes more than 500 KB/s, within 15 minutes. A Kafka partition on Standard brokers has no fixed limit; its leader broker's disk, network and CPU bound it. Express brokers cap a partition at 15 MB/s, and Serverless at 5 MB/s in.

Scaling affects ordering differently. Resharding a provisioned stream splits or merges shards, and KCL finishes reading a parent shard before it reads the shards that replaced it, so per-key order survives. Kafka lets you add partitions but never remove them, and adding them remaps many existing keys, so a key's older records sit in one partition and its newer ones in another. Plan partition counts up front, or create a new topic and move to it. Express brokers move partitions between brokers up to 20 times faster than Standard brokers, which makes adding brokers quicker but doesn't change which partition a key maps to.

Producers can reorder records as well. Kinesis's PutRecords call, which writes up to 500 records at once, doesn't guarantee their order, and when a caller resends the records it rejected, they land after records that succeeded. Strict per-key order from one producer needs serial PutRecord calls with SequenceNumberForOrdering. Kafka's idempotent producer, the default since Kafka 3.0, keeps order within a partition through retries with up to five requests in flight. On-demand Kinesis streams can also use service-managed partition keys, added in September 2026, which spread records across shards for you and are intended for workloads where order doesn't matter.

Record size is the last structural limit. Kinesis accepts records of up to 1 MiB, or 10 MiB on a stream configured for large records, which AWS expects to stay a small share of a stream's traffic. Kafka's limit is roughly 1 MB (message.max.bytes) until you raise it in the broker, topic and client settings, and MSK Serverless accepts up to 8 MiB. On both, large payloads usually go to S3, and the record carries a pointer to the object.

Reading: Fan-Out, Latency and Parallelism

All the consumers of a Kinesis shard divide its read capacity, 2 MB/s and five GetRecords calls a second, unless they register for enhanced fan-out. According to AWS, a record takes about 200 ms to reach a lone polling consumer and about 1,000 ms when five consumers poll the same shard. A registered consumer instead receives records pushed to it, with 2 MB/s per shard reserved for it alone and propagation of about 70 ms on average, no matter whether one or five are registered. A stream allows 20 registrations, or 50 under On-demand Advantage. What it costs depends on the mode: on a provisioned stream, $0.015 for each consumer and shard per hour plus $0.013 per GB; on On-demand Standard, $0.05 per GB read instead of $0.04; on On-demand Advantage, the usual $0.016 per GB read and nothing more. A consumer built on KCL 2.0 or later registers this way automatically.

A Kafka consumer group has no per-partition read budget. Each group reads as fast as its brokers' network and disks can serve it, so a new consuming application is just another group: no registration, no limit on consumers per topic, and no charge per reader on MSK Standard or Express. AWS publishes egress figures for each Express broker size, starting at 31.2 MB/s sustained for express.m7g.large, and Serverless allows 400 MB/s out per cluster, 10 MB/s per partition and 500 consumer groups. AWS publishes no latency figure for MSK. A Kafka consumer's fetch request waits on the broker and returns as soon as data is available (fetch.min.bytes is 1 by default), so latency depends mostly on producer batching and on the brokers.

One cost doesn't appear on MSK's price list. AWS charges standard data transfer rates for traffic in and out of MSK clusters, so a client in one Availability Zone that talks to a broker in another pays $0.01 per GB in each direction. Consumers avoid most of it by setting client.rack to their zone ID, as AWS's client best practices recommend, once the cluster's replica.selector.class is set to RackAwareReplicaSelector, so they fetch from a replica in their own zone. Producers can't avoid it, because every write goes to the partition's leader.

Parallelism inside one consuming application has a ceiling on both sides. KCL gives each shard to exactly one record processor, so workers beyond the shard count wait for a lease, and a classic Kafka consumer group gives each partition to one consumer, so partitions cap consumers in the same way. Lambda raises the ceiling for Kinesis with ParallelizationFactor, processing as many as 10 batches from one shard at once while keeping each partition key in order, and it reads each Kafka partition in sequence. Share groups remove the cap for Kafka, which is one more reason their absence on MSK matters.

Failed Records, Retries and Dead Letters

Because both are logs, a consumer's progress is a position, a checkpoint per Kinesis shard or a committed offset per Kafka partition, and everything before the position counts as done. Neither a KCL application nor a classic Kafka consumer group can mark one record as failed and return to it later while the records behind it move on, so what a failure does depends on the consumer.

On Kinesis

If a KCL processRecords method throws, the KCL troubleshooting guide says the library absorbs the exception and does not resend the batch, so the application ends up with skipped records. KCL 3.5.4 logs them and counts them in a RecordsFailedProcessing metric, and the next checkpoint moves past them. A KCL consumer that can't afford to lose records catches every failure itself and parks the record somewhere, usually an SQS queue, before it checkpoints. Our Kinesis vs Reliable PubSub comparison shows that code and the ways it can still go wrong.

On Kafka

A Kafka consumer group leaves the decision to your code, and both obvious choices hurt. Retrying the record in place stalls its partition, and once processing exceeds max.poll.interval.ms, five minutes by default, the group removes the consumer and hands its partitions to another. Committing past the record loses it. The usual answer is to publish the record to a dead-letter topic of your own and carry on, which frameworks package for you: Spring for Apache Kafka's DefaultErrorHandler with a DeadLetterPublishingRecoverer, Kafka Connect's errors.deadletterqueue.topic.name for sink connectors, and the exception handlers in Kafka Streams.

With Lambda on Either

Lambda handles the two alike. By default it retries the whole batch until the function succeeds or the records expire, and because it reads each shard or partition in order, everything behind the failing batch waits. BisectBatchOnFunctionError, MaximumRetryAttempts, MaximumRecordAgeInSeconds and ReportBatchItemFailures limit the damage for both, although for Kafka sources these error handling controls require the event source mapping's provisioned mode. Where discarded records go differs. A Kinesis mapping's SQS or SNS destination gets a description of the batch, its shard and sequence-number range, without the data, so the records must be read back from the stream before they expire; an S3 destination gets the data too. A Kafka mapping in provisioned mode can also write failures to a Kafka topic in the same cluster, and AWS says those arrive with the original records and the failure details.

Share Groups, Not Yet on MSK

Kafka has an answer to all of this that Kinesis lacks. In a share group, consumers accept, release or reject each record, the broker counts deliveries against a limit, five by default, and since Kafka 4.4 records that exhaust it can go to a dead-letter topic. That is a work queue inside Kafka, and a share-group consumer is shown step by step in Is Kafka a Message Queue? On MSK it is a 4.1 preview at best. Until that changes, per-record retries on AWS mean SQS, which the Kinesis FAQ itself recommends for message-level acknowledgement and visibility timeouts (see our SQS comparison), or a message bus that tracks each message, such as the Reliable PubSub option at the end of this page. The dead letter queue guide covers what to do with records once they land in one.

Retention, Replay and Compaction

A Kinesis stream retains data for 24 hours unless you extend it, to a maximum of 365 days, and keeping data past the first day costs extra. Up to seven days, a provisioned stream pays an extra $0.020 per shard-hour, an On-demand Standard stream $0.10 per GB-month and an On-demand Advantage stream $0.023 per GB-month. Beyond seven days every mode pays $0.023 per GB-month, and a provisioned stream also pays $0.021 per GB to read that older data back. A year is the ceiling, so anything kept longer has to go to S3.

Kafka retains a topic for a week unless you change it, and the setting has no maximum. On MSK, Standard brokers hold the data on EBS volumes you provision at $0.10 per GB-month, so with three replicas every gigabyte is paid for three times, and tiered storage moves older segments to $0.06 per GB-month. Express brokers and Serverless store any amount at $0.10 per GB-month. Per gigabyte kept, Kinesis's long-term rate is the cheapest of these, as long as a year is enough.

Kafka can also compact a topic, so that only the newest record per key survives, indefinitely. That is how Kafka holds current state, such as each customer's latest address, a Kafka Streams changelog or a change-data-capture table. Kinesis has no equivalent. MSK's tiered storage does not apply to compacted topics, and Serverless allows only 120 partitions for them.

Replaying is routine on Kafka and awkward with KCL. A Kafka consumer group can be moved back to any retained offset or point in time once its consumers are stopped, and a new group can start from the earliest record. KCL applies its initial position only when it creates a lease and ignores it for any shard that already has a checkpoint. To read a KCL stream again, you give the application a new name, and with it a new lease table, or you stop it and delete the lease table. Here is the Kafka side, run against an MSK cluster with IAM authentication:

# Move the billing group's offsets on "orders" back to 09:00 UTC (a time
# without an offset is read as UTC). Stop the group's consumers first;
# --dry-run in place of --execute prints the new offsets without moving them.
kafka-consumer-groups.sh --bootstrap-server "$BROKERS" \
    --command-config client-iam.properties \
    --group billing --topic orders \
    --reset-offsets --to-datetime 2026-11-16T09:00:00.000 --execute

And the KCL side, a second application that starts from the same point in time:

// KCL reads its initial position only when it creates a lease, so an
// application that has checkpoints ignores a new one. To reread from 09:00
// UTC, run the consumer under a new application name, with a new lease table.
Date from = Date.from(Instant.parse("2026-11-16T09:00:00Z"));
ConfigsBuilder configs = new ConfigsBuilder(
        new SingleStreamTracker("orders", InitialPositionInStreamExtended
                .newInitialPositionAtTimestamp(from)),
        "billing-replay-20261116",
        kinesis, dynamoDb, cloudWatch,
        UUID.randomUUID().toString(),
        BillingProcessor::new);
Scheduler scheduler = new Scheduler(
        configs.checkpointConfig(),
        configs.coordinatorConfig(),
        configs.leaseManagementConfig(),
        configs.lifecycleConfig(),
        configs.metricsConfig(),
        configs.processorConfig(),
        configs.retrievalConfig());
new Thread(scheduler).start();

The original application's lease table stays in DynamoDB until you delete it, and the replaying application keeps its new name, so dashboards and alarms keyed to the old name need updating too.

Delivery Guarantees and Exactly-Once

Kinesis delivers at least once, at both ends. A producer that retries a write after a timeout can store a record twice, and a KCL worker that restarts goes back to its last checkpoint and processes some records again. AWS's guidance on duplicates is to give every record a unique ID and drop repeats when processing, or to rely on an idempotent destination. PutRecords adds a trap of its own. The call can succeed while rejecting some of its records, most often because a shard's write limit was exceeded, so the caller must check each result and resend what failed:

// PutRecords can return HTTP 200 and still reject some of the records, most
// often with ProvisionedThroughputExceededException. Resend only those.
List<PutRecordsRequestEntry> pending = batch;      // up to 500 entries
for (int attempt = 1; !pending.isEmpty(); attempt++) {
    if (attempt > 5) {
        throw new IllegalStateException(pending.size() + " records not written");
    }
    if (attempt > 1) {
        Thread.sleep(100L << attempt);     // back off: 400 ms, 800 ms, 1.6 s...
    }
    PutRecordsResponse response = kinesis.putRecords(PutRecordsRequest.builder()
            .streamName("orders")
            .records(pending)
            .build());
    List<PutRecordsRequestEntry> rejected = new ArrayList<>();
    for (int i = 0; i < pending.size(); i++) {
        // results come back in request order; a rejected one has an error code
        if (response.records().get(i).errorCode() != null) {
            rejected.add(pending.get(i));
        }
    }
    pending = rejected;
}

Kafka removes producer duplicates by default. Since Kafka 3.0, producers use acks=all and idempotence, so the broker discards a retried batch it has already written during the producer's session. Transactions go further for applications that read from one topic and write to another: the output and the consumer's offsets commit together, which is what Kafka Streams' exactly-once mode is built on. On MSK with IAM authentication, a producer looks like this:

Properties props = new Properties();
props.put("bootstrap.servers", brokers);     // the cluster's IAM endpoint
props.put("security.protocol", "SASL_SSL");
props.put("sasl.mechanism", "AWS_MSK_IAM");
props.put("sasl.jaas.config",
        "software.amazon.msk.auth.iam.IAMLoginModule required;");
props.put("sasl.client.callback.handler.class",
        "software.amazon.msk.auth.iam.IAMClientCallbackHandler");
props.put("key.serializer", StringSerializer.class.getName());
props.put("value.serializer", StringSerializer.class.getName());
// acks=all and enable.idempotence=true are the defaults since Kafka 3.0: the
// producer retries for up to delivery.timeout.ms (two minutes), and the
// broker discards the duplicates those retries would otherwise write.
try (KafkaProducer<String, String> producer = new KafkaProducer<>(props)) {
    producer.send(new ProducerRecord<>("orders", customerId, json),
            (metadata, error) -> {
                if (error != null) {
                    // reached only when retries have run out, or for an
                    // error that retrying cannot fix
                    keepForRetry(customerId, json, error);
                }
            });
}

With IAM authentication, the producer's IAM policy needs kafka-cluster:WriteDataIdempotently as well as kafka-cluster:WriteData: AWS lists both for producing idempotently, which Kafka producers do by default.

Neither service's guarantee extends past its own boundary. A consumer that charges a card or updates a database still has to tolerate seeing a record twice, for example by recording processed IDs in the same database transaction as the update, which makes it an idempotent consumer. A service that writes to its database and publishes an event needs the outbox pattern to keep the two consistent.

Integrations on AWS

The integration gap inside AWS is narrower than it used to be. Lambda, Amazon Data Firehose, Managed Service for Apache Flink, Amazon Redshift streaming ingestion and EventBridge Pipes all read from Kinesis Data Streams and from MSK, and the AWS Glue Schema Registry works with both. Both can now also write straight to Amazon S3, and to Apache Iceberg tables in S3 Tables, with no Firehose stream or connector in between. MSK Express brokers have done it since July 2026, at $0.008 per GB to S3 and $0.01 per GB to S3 Tables (see AWS's data delivery guide). On-demand Kinesis streams followed in August 2026, at $0.0275 and $0.035 per GB on On-demand Standard, or $0.011 and $0.014 on On-demand Advantage. Provisioned Kinesis streams, Standard brokers and Serverless clusters don't have it.

Kafka brings its own ecosystem on top. Kafka Connect, which MSK Connect runs for $0.11 per MCU-hour, has a large catalog of source and sink connectors, Kafka Streams does stateful processing inside your services, and Debezium, the open-source change-data-capture project, is built on Kafka Connect. Kinesis has the Kinesis Producer Library, which packs small records together to save PUT units, and an ecosystem that is mostly AWS's own services. If stream processing is the main job, see what it involves in our stream processing guide.

One Kinesis service has gone. Kinesis Data Analytics for SQL stopped accepting new applications on October 15, 2025, and AWS began deleting the remaining ones on January 27, 2026, according to its discontinuation notice. Its successor, Managed Service for Apache Flink, was called Kinesis Data Analytics for Apache Flink until 2023, and it reads from Kinesis and Kafka alike.

Kinesis vs MSK Pricing: What Each Costs

The two bill on different axes. Kinesis charges for the stream: shard-hours and PUT units on a provisioned stream, stream-hours plus every gigabyte written and every gigabyte each consumer reads on On-demand Standard, and per gigabyte with an account minimum on On-demand Advantage. MSK charges mostly for the cluster: broker-hours and storage, plus $0.01 per GB written on Express brokers, with nothing per consumer. MSK Serverless is the exception, billing cluster-hours, partitions and every gigabyte in and out.

To see where each wins, we priced three steady workloads with AWS's US East list prices for November 2026 (Kinesis pricing, MSK pricing): records of 10 KiB, 24 hours of retention, a 30-day month, and two or six consumer applications that each read everything. Provisioned streams use AWS's shard formula, which leaves no headroom. MSK Express uses the smallest cluster, three express.m7g.large brokers, which stays within AWS's sustained throughput figures in all three cases.

Monthly cost1 MiB/s, 2 readers10 MiB/s, 2 readers10 MiB/s, 6 readers
Kinesis provisioned, readers sharing each shard$15 (1 shard)$145 (10 shards)$361 (30 shards), but each reader polls less than once a second
Kinesis provisioned, enhanced fan-out for every reader$102$1,019$2,768
Kinesis On-demand Standard$434$4,079$9,648, with enhanced fan-out
Kinesis On-demand Advantage$3,038 (account minimum)$3,038 (account minimum)$4,455
MSK Express, 3 × express.m7g.large$915$1,219$1,219
MSK Serverless, 10 partitions$1,065$5,698$10,760

These are list prices for the two services alone. They leave out KCL's DynamoDB lease tables, Lambda, the free tier and data transfer, and data transfer can matter. MSK clients pay inter-AZ transfer to brokers in other zones, about $338 a month for producers spread over three zones at 10 MiB/s, and Kinesis clients in private subnets pay for the NAT gateway or interface VPC endpoint they reach Kinesis through. Standard brokers are missing from the table because AWS publishes no throughput figure for them; three kafka.m7g.large brokers cost $441 a month before storage.

Three results stand out. A steady stream read by one or two applications costs far less on provisioned Kinesis than on any MSK option, provided someone manages the shard count, and doubling the shards for headroom would still leave it well below MSK. On-demand Standard costs nearly 30 times as much as provisioned at a steady load, which is the price of never planning capacity. And an MSK Express or Standard cluster's bill barely moves as consumers multiply, while Kinesis charges for every reader that needs low latency, so with six such readers MSK Express is the cheapest option in the table.

AWS's advice on On-demand Advantage holds against On-demand Standard. With two readers per stream, the two modes cost the same at about 7 MiB/s written, close to the 10 MB/s that AWS cites, and Advantage is cheaper above that, long before an account uses up its minimum. Against provisioned shards it lost in all three workloads, even with enhanced fan-out for every reader, so it pays off mainly for accounts that run their streams on demand anyway.

Moving Between Kinesis and Kafka

The APIs are unrelated, so a move in either direction rewrites every producer and consumer. The concepts carry over: a stream becomes a topic, a shard a partition, a partition key a record key, a sequence number an offset, and a KCL application or enhanced fan-out consumer a consumer group. Positions don't carry over: KCL checkpoints and Kafka's committed offsets have nothing in common, so each consuming application starts on the new system from a point in time, and Lambda event source mappings are recreated against the new source.

Data can flow between the two while you move. AWS Labs publishes a Kinesis Kafka connector that writes Kafka topics into Kinesis, and Confluent sells a Kinesis source connector, licensed after a 30-day trial, that copies Kinesis into Kafka; a short KCL program that produces each record to Kafka does the same job. Any bridge can repeat records after a restart, so give each copied record an ID built from its source position, the shard and sequence number or the partition and offset, drop duplicates downstream, and keep the partition key as the record key so that each key keeps its order. For existing Kafka clusters, the Kinesis FAQ also mentions an MSK migration program and guide.

Move consumers before producers. Bridge the old system into the new one, move each consuming application with its starting point set just before the last record it processed, run both in parallel until their output agrees, then switch the producers, and retire the bridge once it has copied the last records the old producers wrote. Consumers should be idempotent throughout, because the overlap delivers some records twice.

When the Stream Is Really an Event Bus: Reliable PubSub

Some Kinesis streams and Kafka topics on AWS are neither analytics pipelines nor event stores. They carry business events between a few Java services, and the team's real problem is the retry and dead-letter code around each consumer. If those services already use Valkey or Redis on Amazon ElastiCache or MemoryDB, there is an option that is not a log.

Reliable PubSub, part of Redisson PRO, keeps a topic in Valkey or Redis. A subscription per consuming service tracks that service's position and sets its own delivery limit, visibility timeout and dead-letter topic, and consumers acknowledge or reject messages one at a time, optionally asking for redelivery after a delay. Because the topic retains messages, a new subscription can begin with the oldest one, and an existing one can be moved back to a timestamp. It also offers three things Kinesis and Kafka lack: message priorities (0 to 9), delayed delivery, and duplicate suppression by message ID or payload hash. Message groups pin related messages to one consumer, which is the nearest equivalent of a partition key, although redelivered messages in a group can arrive out of order. Every setting is in the Reliable PubSub documentation.

KinesisKafka on MSKReliable PubSub
KCL application or enhanced fan-out consumerConsumer groupSubscription, from createSubscription()
Checkpoint per shardCommitted offset per partitionAcknowledgement per message
Skipped batch (KCL) or batch retries (Lambda)Your code's retriesfailed negative acknowledgement, redelivered after an optional delay
Nothing in the stream; a Lambda on-failure destinationA dead-letter topic your code or framework writesdeadLetterTopicName, after deliveryLimit deliveries (10 by default)
Partition keyRecord keyMessage group ID
New application name, or a deleted lease tableOffset reset with the group stoppedseek() to a timestamp, a message ID, the earliest or the latest message
No delay or priorityNo delay or prioritydelay() and priority() on each message
RReliablePubSubTopic<OrderEvent> topic =
        redisson.getReliablePubSubTopic("orders");
// two days of history; without a time-to-live the topic keeps every message
topic.setConfig(TopicConfig.defaults().timeToLive(Duration.ofDays(2)));

// create the subscription once, when the service is first deployed
Subscription<OrderEvent> billing = topic.createSubscription(
        SubscriptionConfig.name("billing")
                .deliveryLimit(5)
                .deadLetterTopicName("orders-billing-dlt"));
PullConsumer<OrderEvent> consumer =
        billing.createPullConsumer(ConsumerConfig.name("billing-1"));

List<Message<OrderEvent>> batch = consumer.pullMany(PullArgs.defaults()
        .count(10)
        .timeout(Duration.ofSeconds(20))      // wait up to 20 s for messages
        // each message's timeout starts when it is pulled, so cover the batch
        .visibility(Duration.ofMinutes(2)));
for (Message<OrderEvent> msg : batch) {
    try {
        charge(msg.getPayload());
        consumer.acknowledge(MessageAckArgs.ids(msg.getId()));
    } catch (RuntimeException e) {
        // only this message waits a minute; the rest of the batch goes on
        consumer.negativeAcknowledge(MessageNegativeAckArgs
                .failed(msg.getId())
                .delay(Duration.ofMinutes(1)));
    }
}

Durability comes from the service underneath. ElastiCache for Valkey has been able to keep a Multi-AZ transactional log of writes since June 2026, and MemoryDB always has; the durability section of our Kinesis comparison covers the requirements and the price. Setup guides cover connecting Redisson to ElastiCache and to MemoryDB.

Its limits:

  • Only JVM services can use it; anything else needs the Kinesis or Kafka clients.
  • History lives in memory, so replay windows are hours or days, not months. Give topics a time-to-live, and set maxmemory-policy to noeviction so a full node rejects writes instead of evicting messages.
  • Redisson documents no partitioning for Reliable PubSub, so assume each topic lives on a single Valkey or Redis primary, and split heavy traffic over several topics.
  • There is no Firehose, Flink, Kafka Connect or S3 delivery, so any integration is code you write.
  • It is part of Redisson PRO, which is commercial.

Amazon Kinesis vs Reliable PubSub and Apache Kafka vs Reliable PubSub compare it with each service feature by feature.

How to Choose

Your situationChoose
A new streaming workload on AWS, an analytics pipeline into S3, Redshift or Flink, or a team new to streamingKinesis Data Streams, on demand while traffic is unpredictable
Steady traffic read by one or two applications, where cost mattersKinesis, provisioned, with someone watching the shard count
Existing Kafka applications, Kafka Connect or Kafka Streams, or the same platform outside AWSMSK: Express brokers for throughput and storage you don't size, Standard brokers for full configuration control and tiered storage
Many applications reading the same stream, each needing low latencyMSK Express or Standard brokers, which charge nothing per reader
History beyond a year, compacted topics, or an event log as the system of recordKafka on MSK
Per-record acknowledgement, retries and dead letters, as in a work queueNot Kinesis or MSK as they are today: Amazon SQS, Kafka share groups on a Kafka 4.2 or later cluster you run yourself, or, for Java services, Redisson PRO's Reliable PubSub or Reliable Queue
Java services that pass business events to each other, already run ElastiCache or MemoryDB, and keep only as much history as memory holdsRedisson PRO's Reliable PubSub

The two can also run side by side, with Kinesis feeding AWS's analytics services and MSK holding the event log between services, joined by a connector. Wider shortlists are in Apache Kafka alternatives and Redis vs Kafka, and the Google Cloud version of this choice is Google Pub/Sub vs Kafka.

Frequently Asked Questions

Is AWS Kinesis the same as Kafka?

No. Kinesis Data Streams is AWS's own service with its own API, available only on AWS. The concepts line up: a stream is like a topic, a shard like a partition, a sequence number like an offset and a KCL checkpoint like a committed offset. But Kafka clients cannot connect to Kinesis, and code written for one does not run on the other. AWS's managed Kafka service is Amazon MSK.

What is the AWS equivalent of Kafka?

Amazon MSK, which runs Apache Kafka itself on Standard brokers, Express brokers or MSK Serverless, so Kafka applications connect to it unchanged apart from authentication. Kinesis Data Streams is AWS's native service for the same job, with a different API. AWS's own FAQ recommends MSK for teams with existing Kafka applications or a preference for open source, and Kinesis for teams new to streaming.

Is Kafka cheaper than Kinesis?

It depends on how many applications read the stream. At a steady 10 MiB/s read by two applications, AWS's US East list prices put a provisioned Kinesis stream at about $145 a month and the smallest MSK Express cluster at about $1,219. With six readers that each need low latency, Kinesis needs enhanced fan-out and costs about $2,768, while the MSK cluster still costs about $1,219. On-demand Standard costs nearly 30 times as much as provisioned Kinesis at a steady load, and a production-sized MSK cluster costs several hundred dollars a month before any traffic: three kafka.m7g.large Standard brokers come to about $441.

Is Kinesis better than Kafka?

Neither is better in general. Kinesis suits teams that want no brokers to manage, AWS integrations out of the box and no more than a year of history. Kafka on MSK suits teams that need Kafka's ecosystem, portability beyond AWS, compacted topics or unlimited retention, or many consumers per stream at low cost. Share groups, Kafka's per-record acknowledgement, are not yet production-ready on MSK.

Is MSK the same as Kafka?

MSK runs open-source Apache Kafka, so Kafka clients, Kafka Connect and Kafka Streams work with it. Standard brokers run Kafka 3.8 to 4.1 with the configuration you choose. Express brokers keep the Kafka API with storage AWS manages and preset defaults, run Kafka up to 4.2, and don't yet support share groups or fully support the Kafka Streams API. MSK Serverless is Kafka-compatible within fixed quotas and requires IAM authentication.

Can Kafka clients connect to Kinesis?

No. Kinesis has no Kafka protocol endpoint, so Kafka applications either move to Amazon MSK or are rewritten against the Kinesis API and the KCL. To move data between the two, AWS Labs' Kinesis Kafka connector writes from Kafka into Kinesis, and Confluent's licensed Kinesis source connector reads from Kinesis into Kafka.

Is Kinesis Data Analytics deprecated?

The SQL version is discontinued. Kinesis Data Analytics for SQL stopped accepting new applications on October 15, 2025, and AWS began deleting existing applications and ended support on January 27, 2026. AWS points users to Amazon Managed Service for Apache Flink, the renamed Kinesis Data Analytics for Apache Flink, which reads from both Kinesis and Kafka.

Is there anything better than Kafka?

For some jobs, yes. On AWS, Kinesis Data Streams gives you a log with no brokers to size when you don't need Kafka's ecosystem. For work queues with per-message retries, Amazon SQS or a broker such as RabbitMQ fits the job more directly than a log. For Java services exchanging events on Valkey or Redis, Redisson PRO's Reliable PubSub adds per-message acknowledgement, dead letters, delays and priorities. Our guide to Kafka alternatives compares seven options by the job they do.

Similar articles