Probabilistic structures
Probabilistic structures trade exact answers for dramatic memory savings. Rather than storing the elements themselves, they keep a small, fixed-size summary and answer questions approximately - is this element present? (with a tunable false-positive rate), how many distinct elements are there? (within a small error margin), which elements are most frequent?, or what value falls at a given percentile? - using far less memory than an exact structure would.
Redisson provides six: the Bloom filter and its Bloom filter (Native) variant and the Cuckoo filter for membership testing (the Cuckoo filter also supports deletion), HyperLogLog for estimating the number of distinct elements, TopK for tracking the most frequent elements, and TDigest for estimating quantiles and the distribution of a stream of values.
Bloom filter¶
Java implementation of Valkey or Redis based RBloomFilter object is a Bloom filter - a compact, probabilistic structure that tests whether an element has been added to a set. It never reports a false negative (an element that was added always tests as present), but allows a tunable rate of false positives (an element that was never added may occasionally test as present), in return for using far less memory than holding the elements themselves. The number of contained bits is limited to 2^32, raised to 2^63 with data partitioning, and the object is thread-safe.
It must be initialized with a capacity before use by calling tryInit(expectedInsertions, falseProbability): the bit-array size and the number of hash functions are derived from the expected number of insertions and the acceptable false-positive probability.
Code examples:
RBloomFilterAsync<SomeObject> bloomFilter = redisson.getBloomFilter("sample");
RFuture<Boolean> initFuture = bloomFilter.tryInitAsync(55000000L, 0.03);
RFuture<Boolean> addFuture = bloomFilter.addAsync(new SomeObject("field1Value", "field2Value"));
RFuture<Boolean> containsFuture = bloomFilter.containsAsync(new SomeObject("field1Value", "field8Value"));
RFuture<Long> countFuture = bloomFilter.countAsync();
RedissonReactiveClient redissonReactive = redisson.reactive();
RBloomFilterReactive<SomeObject> bloomFilter = redissonReactive.getBloomFilter("sample");
Mono<Boolean> initMono = bloomFilter.tryInit(55000000L, 0.03);
Mono<Boolean> addMono = bloomFilter.add(new SomeObject("field1Value", "field2Value"));
Mono<Boolean> containsMono = bloomFilter.contains(new SomeObject("field1Value", "field8Value"));
Mono<Long> countMono = bloomFilter.count();
RedissonRxClient redissonRx = redisson.rxJava();
RBloomFilterRx<SomeObject> bloomFilter = redissonRx.getBloomFilter("sample");
Single<Boolean> initSingle = bloomFilter.tryInit(55000000L, 0.03);
Single<Boolean> addSingle = bloomFilter.add(new SomeObject("field1Value", "field2Value"));
Single<Boolean> containsSingle = bloomFilter.contains(new SomeObject("field1Value", "field8Value"));
Single<Long> countSingle = bloomFilter.count();
Adding elements¶
add inserts a single element and returns true if it changed the filter, or false if the element was already present. The collection overload add(Collection) inserts many elements at once and returns how many were newly added.
RBloomFilter<SomeObject> bloomFilter = redisson.getBloomFilter("sample");
// true if newly added, false if already present
boolean added = bloomFilter.add(new SomeObject("field1Value", "field2Value"));
// add many at once, returns how many were newly added
long addedCount = bloomFilter.add(Arrays.asList(new SomeObject("a", "b"), new SomeObject("c", "d")));
RedissonReactiveClient redissonReactive = redisson.reactive();
RBloomFilterReactive<SomeObject> bloomFilter = redissonReactive.getBloomFilter("sample");
Mono<Boolean> addedMono = bloomFilter.add(new SomeObject("field1Value", "field2Value"));
Mono<Long> addedCountMono = bloomFilter.add(Arrays.asList(new SomeObject("a", "b"), new SomeObject("c", "d")));
RedissonRxClient redissonRx = redisson.rxJava();
RBloomFilterRx<SomeObject> bloomFilter = redissonRx.getBloomFilter("sample");
Single<Boolean> addedSingle = bloomFilter.add(new SomeObject("field1Value", "field2Value"));
Single<Long> addedCountSingle = bloomFilter.add(Arrays.asList(new SomeObject("a", "b"), new SomeObject("c", "d")));
Testing membership¶
contains returns true if an element is probably present; because of the false-positive rate a true result is not a guarantee, but a false result is definitive. contains(Collection) returns how many of the given elements are probably present, and exists(Collection) returns the subset of them that are.
RBloomFilter<SomeObject> bloomFilter = redisson.getBloomFilter("sample");
// true means probably present (subject to the false-positive rate); false is definitive
boolean maybePresent = bloomFilter.contains(new SomeObject("field1Value", "field2Value"));
// how many of these are probably present
long presentCount = bloomFilter.contains(Arrays.asList(new SomeObject("a", "b"), new SomeObject("c", "d")));
// which of these are probably present
Set<SomeObject> present = bloomFilter.exists(Arrays.asList(new SomeObject("a", "b"), new SomeObject("c", "d")));
RBloomFilterAsync<SomeObject> bloomFilter = redisson.getBloomFilter("sample");
RFuture<Boolean> maybeFuture = bloomFilter.containsAsync(new SomeObject("field1Value", "field2Value"));
RFuture<Long> presentCountFuture = bloomFilter.containsAsync(Arrays.asList(new SomeObject("a", "b"), new SomeObject("c", "d")));
RFuture<Set<SomeObject>> presentFuture = bloomFilter.existsAsync(Arrays.asList(new SomeObject("a", "b"), new SomeObject("c", "d")));
RedissonReactiveClient redissonReactive = redisson.reactive();
RBloomFilterReactive<SomeObject> bloomFilter = redissonReactive.getBloomFilter("sample");
Mono<Boolean> maybeMono = bloomFilter.contains(new SomeObject("field1Value", "field2Value"));
Mono<Long> presentCountMono = bloomFilter.contains(Arrays.asList(new SomeObject("a", "b"), new SomeObject("c", "d")));
Mono<Set<SomeObject>> presentMono = bloomFilter.exists(Arrays.asList(new SomeObject("a", "b"), new SomeObject("c", "d")));
RedissonRxClient redissonRx = redisson.rxJava();
RBloomFilterRx<SomeObject> bloomFilter = redissonRx.getBloomFilter("sample");
Single<Boolean> maybeSingle = bloomFilter.contains(new SomeObject("field1Value", "field2Value"));
Single<Long> presentCountSingle = bloomFilter.contains(Arrays.asList(new SomeObject("a", "b"), new SomeObject("c", "d")));
Single<Set<SomeObject>> presentSingle = bloomFilter.exists(Arrays.asList(new SomeObject("a", "b"), new SomeObject("c", "d")));
Filter information¶
count returns the estimated number of elements added so far. The configured sizing can be read back with getExpectedInsertions (the capacity passed to tryInit) and getFalseProbability (the target false-positive rate), and the derived structure with getSize (the number of bits) and getHashIterations (the number of hash functions).
RBloomFilter<SomeObject> bloomFilter = redisson.getBloomFilter("sample");
long inserted = bloomFilter.count(); // estimated number of added elements
long capacity = bloomFilter.getExpectedInsertions(); // capacity passed to tryInit
double rate = bloomFilter.getFalseProbability(); // target false-positive rate
long bits = bloomFilter.getSize(); // number of bits
int hashes = bloomFilter.getHashIterations(); // number of hash functions
RBloomFilterAsync<SomeObject> bloomFilter = redisson.getBloomFilter("sample");
RFuture<Long> insertedFuture = bloomFilter.countAsync();
RFuture<Long> capacityFuture = bloomFilter.getExpectedInsertionsAsync();
RFuture<Double> rateFuture = bloomFilter.getFalseProbabilityAsync();
RFuture<Long> bitsFuture = bloomFilter.getSizeAsync();
RFuture<Integer> hashesFuture = bloomFilter.getHashIterationsAsync();
RedissonReactiveClient redissonReactive = redisson.reactive();
RBloomFilterReactive<SomeObject> bloomFilter = redissonReactive.getBloomFilter("sample");
Mono<Long> insertedMono = bloomFilter.count();
Mono<Long> capacityMono = bloomFilter.getExpectedInsertions();
Mono<Double> rateMono = bloomFilter.getFalseProbability();
Mono<Long> bitsMono = bloomFilter.getSize();
Mono<Integer> hashesMono = bloomFilter.getHashIterations();
RedissonRxClient redissonRx = redisson.rxJava();
RBloomFilterRx<SomeObject> bloomFilter = redissonRx.getBloomFilter("sample");
Single<Long> insertedSingle = bloomFilter.count();
Single<Long> capacitySingle = bloomFilter.getExpectedInsertions();
Single<Double> rateSingle = bloomFilter.getFalseProbability();
Single<Long> bitsSingle = bloomFilter.getSize();
Single<Integer> hashesSingle = bloomFilter.getHashIterations();
Data partitioning¶
This feature available only in Redisson PRO edition.
Although 'RBloomFilter' object is cluster compatible its content isn't scaled across multiple master nodes. Bloom Filter data partitioning support available only in cluster mode and implemented by separate RClusteredBloomFilter object. This implementation uses more efficient distributed memory allocation algorithm. It allows to "shrink" memory space consumed by unused bits across all Valkey or Redis nodes. State of each instance is partitioned across all nodes in Valkey or Redis cluster. Number of contained bits is limited to 2^63. More details about partitioning here.
Below is the list of all available BloomFilter implementations:
| RedissonClient method name |
Data partitioning support |
Ultra-fast read/write | Bits amount limit |
|---|---|---|---|
| getBloomFilter() open-source version |
❌ | ❌ | 2^32 |
| getBloomFilter() Redisson PRO version |
❌ | ✔️ | 2^32 |
| getClusteredBloomFilter() available only in Redisson PRO |
✔️ | ✔️ | 2^63 |
RClusteredBloomFilter<SomeObject> bloomFilter = redisson.getClusteredBloomFilter("sample");
// initialize bloom filter with
// expectedInsertions = 255000000
// falseProbability = 0.03
bloomFilter.tryInit(255000000L, 0.03);
bloomFilter.add(new SomeObject("field1Value", "field2Value"));
bloomFilter.add(new SomeObject("field5Value", "field8Value"));
bloomFilter.contains(new SomeObject("field1Value", "field8Value"));
Use Cases¶
A Bloom filter is a compact probabilistic structure for set membership: contains may report a false positive but never a false negative, so a negative answer is definitive while a positive answer means the element is probably present. Elements are only ever added, never removed, which keeps it extremely space-efficient. The filter is sized up front with tryInit(expectedInsertions, falseProbability) for a target false-positive rate, and count() reports the approximate number of elements added. It suits workloads dominated by membership questions over very large sets, where a small, tunable false-positive rate is an acceptable trade for tiny, fixed memory.
Skipping Lookups for Absent Keys
The classic use is a guard in front of a slow backend. Before querying a database, remote cache, or object store, contains is checked first: because a negative answer is definitive, a false means the key is certainly absent and the expensive lookup can be skipped entirely. This blocks "cache penetration", where a flood of requests for non-existent keys would otherwise reach the database. Every stored key is also added to the filter.
RBloomFilter<String> stored = redisson.getBloomFilter("keys:products");
stored.tryInit(50000000L, 0.01);
// on write: record the key
stored.add(productId);
// on read: a false result means the key is definitely absent
if (!stored.contains(productId)) {
return null; // skip the database entirely
}
return database.load(productId); // possibly present - confirm in the store
Pre-Screening Against Large Reference Sets
Screening a value against a huge reference set - breached password hashes, known-malicious domains, reserved usernames - is a fast negative check. A false from contains proves the value is not in the set, so it clears immediately; only a positive needs to fall through to an authoritative lookup. The reference set, however large, compresses to a fixed-size filter, and a whole batch can be screened in one call: exists returns just the subset of candidates that might be present.
RBloomFilter<String> breached = redisson.getBloomFilter("breached-passwords");
breached.tryInit(600000000L, 0.001);
// load the reference set once (bulk add)
breached.add(knownBreachedHashes);
// single check: false proves the password was never breached
if (breached.contains(candidateHash)) {
// possible match - verify against the authoritative source
}
// batch screen: only the hashes that may be present come back
Set<String> suspicious = breached.exists(candidateHashes);
Suppressing Duplicate Work
For best-effort deduplication over a high-volume stream - notifications already sent, records already processed, URLs already crawled - add doubles as a test: it returns false when the element was already present, so a single call both records the element and reports whether it is new. A Bloom filter never removes elements and stores no ids, so memory stays small and fixed even as the seen-set grows into the millions. A rare false positive skips a genuinely new item, which is acceptable when occasional misses are tolerable, and count() tracks the approximate number of distinct items seen.
RBloomFilter<String> seen = redisson.getBloomFilter("sent-notifications");
seen.tryInit(20000000L, 0.01);
// add returns false if the id was already present
if (seen.add(notificationId)) {
send(notificationId); // first time seen - deliver it
}
// approximate number of distinct ids recorded
long distinct = seen.count();
Bloom filter (Native)¶
Java implementation of Valkey or Redis based RBloomFilterNative object is a Bloom filter based on native BF.* commands. This object is thread-safe.
Must be initialized with error rate and capacity by init(errorRate, capacity) method before usage.
Initialization¶
Initialize the filter with an error rate and expected capacity before use:
Use BloomFilterInitArgs builder for advanced initialization parameters such as expansion rate and non-scaling mode. Parameters expansionRate and nonScaling are mutually exclusive.
Non-scaling mode prevents creation of sub-filters when capacity is reached:
Adding elements¶
Add single elements, or a collection of elements in one call:
RBloomFilterNative<String> bloomFilter = redisson.getBloomFilterNative("sample");
RFuture<Boolean> addFuture = bloomFilter.addAsync("field1Value");
RFuture<Boolean> addFuture2 = bloomFilter.addAsync("field2Value");
RFuture<Set<String>> addedFuture = bloomFilter.addAsync(Arrays.asList("field3Value", "field4Value", "field5Value"));
RedissonReactiveClient redisson = redissonClient.reactive();
RBloomFilterNativeReactive<String> bloomFilter = redisson.getBloomFilterNative("sample");
Mono<Boolean> addMono = bloomFilter.add("field1Value");
Mono<Boolean> addMono2 = bloomFilter.add("field2Value");
Mono<Set<String>> addedMono = bloomFilter.add(Arrays.asList("field3Value", "field4Value", "field5Value"));
RedissonRxClient redisson = redissonClient.rxJava();
RBloomFilterNativeRx<String> bloomFilter = redisson.getBloomFilterNative("sample");
Single<Boolean> addRx = bloomFilter.add("field1Value");
Single<Boolean> addRx2 = bloomFilter.add("field2Value");
Single<Set<String>> addedRx = bloomFilter.add(Arrays.asList("field3Value", "field4Value", "field5Value"));
Inserting elements¶
The insert() method combines filter auto-creation (if it doesn't yet exist) with element insertion. It supports optional parameters including capacity, errorRate, expansionRate, nonScaling, and noCreate.
RedissonReactiveClient redisson = redissonClient.reactive();
RBloomFilterNativeReactive<String> bloomFilter = redisson.getBloomFilterNative("sample");
Mono<Set<String>> addedMono = bloomFilter.insert(
BloomFilterInsertArgs.<String>elements(Arrays.asList("field1Value", "field2Value", "field3Value"))
.capacity(1000000L)
.errorRate(0.01));
RedissonRxClient redisson = redissonClient.rxJava();
RBloomFilterNativeRx<String> bloomFilter = redisson.getBloomFilterNative("sample");
Single<Set<String>> addedRx = bloomFilter.insert(
BloomFilterInsertArgs.<String>elements(Arrays.asList("field1Value", "field2Value", "field3Value"))
.capacity(1000000L)
.errorRate(0.01));
Setting noCreate to true prevents the filter from being created automatically and causes the command to fail if the filter does not already exist:
RedissonReactiveClient redisson = redissonClient.reactive();
RBloomFilterNativeReactive<String> bloomFilter = redisson.getBloomFilterNative("sample");
Mono<Set<String>> addedMono = bloomFilter.insert(
BloomFilterInsertArgs.<String>elements(Arrays.asList("field1Value", "field2Value"))
.noCreate(true));
Testing membership¶
Test whether elements are present. The collection-based call returns the subset of supplied elements found in the filter:
RedissonReactiveClient redisson = redissonClient.reactive();
RBloomFilterNativeReactive<String> bloomFilter = redisson.getBloomFilterNative("sample");
Mono<Boolean> existsMono = bloomFilter.exists("field1Value");
Mono<Set<String>> existsAllMono = bloomFilter.exists(Arrays.asList("field1Value", "field8Value"));
Counting elements¶
count() returns the approximate number of elements added to the filter:
Filter information¶
The getInfo() method returns a BloomFilterInfo object containing filter details: capacity, size, sub-filter count, item count, and expansion rate. Use getInfo(BloomFilterInfoOption) to query a specific metric individually.
RBloomFilterNative<String> bloomFilter = redisson.getBloomFilterNative("sample");
BloomFilterInfo info = bloomFilter.getInfo();
long capacity = info.getCapacity();
long size = info.getSize();
long subFilterCount = info.getSubFilterCount();
long itemCount = info.getItemCount();
long expansionRate = info.getExpansionRate();
// query a specific metric
long currentCapacity = bloomFilter.getInfo(BloomFilterInfoOption.CAPACITY);
long currentItems = bloomFilter.getInfo(BloomFilterInfoOption.ITEMS);
RBloomFilterNative<String> bloomFilter = redisson.getBloomFilterNative("sample");
RFuture<BloomFilterInfo> infoFuture = bloomFilter.getInfoAsync();
// query a specific metric
RFuture<Long> capacityFuture = bloomFilter.getInfoAsync(BloomFilterInfoOption.CAPACITY);
RFuture<Long> itemsFuture = bloomFilter.getInfoAsync(BloomFilterInfoOption.ITEMS);
RedissonReactiveClient redisson = redissonClient.reactive();
RBloomFilterNativeReactive<String> bloomFilter = redisson.getBloomFilterNative("sample");
Mono<BloomFilterInfo> infoMono = bloomFilter.getInfo();
// query a specific metric
Mono<Long> capacityMono = bloomFilter.getInfo(BloomFilterInfoOption.CAPACITY);
Mono<Long> itemsMono = bloomFilter.getInfo(BloomFilterInfoOption.ITEMS);
RedissonRxClient redisson = redissonClient.rxJava();
RBloomFilterNativeRx<String> bloomFilter = redisson.getBloomFilterNative("sample");
Single<BloomFilterInfo> infoRx = bloomFilter.getInfo();
// query a specific metric
Single<Long> capacityRx = bloomFilter.getInfo(BloomFilterInfoOption.CAPACITY);
Single<Long> itemsRx = bloomFilter.getInfo(BloomFilterInfoOption.ITEMS);
Available BloomFilterInfoOption values: CAPACITY, SIZE, FILTERS, ITEMS, EXPANSION.
Data dump and restore¶
The scanDump() and loadChunk() methods allow serialization and deserialization of a Bloom filter for backup, replication, or migration between Valkey or Redis instances. Iteration starts from 0 and completes when the returned iterator is 0 with empty data. Requires Redis Bloom 1.0.0 and higher.
RBloomFilterNative<String> sourceFilter = redisson.getBloomFilterNative("source");
RBloomFilterNative<String> targetFilter = redisson.getBloomFilterNative("target");
// dump all chunks from source filter
long iterator = 0;
do {
BloomFilterScanDumpInfo dumpInfo = sourceFilter.scanDump(iterator);
iterator = dumpInfo.getIterator();
if (dumpInfo.getData() != null && dumpInfo.getData().length > 0) {
// load chunk into target filter
targetFilter.loadChunk(iterator, dumpInfo.getData());
}
} while (iterator != 0);
RBloomFilterNative<String> sourceFilter = redisson.getBloomFilterNative("source");
RBloomFilterNative<String> targetFilter = redisson.getBloomFilterNative("target");
// dump all chunks from source filter
long iterator = 0;
do {
RFuture<BloomFilterScanDumpInfo> dumpFuture = sourceFilter.scanDumpAsync(iterator);
BloomFilterScanDumpInfo dumpInfo = dumpFuture.get();
iterator = dumpInfo.getIterator();
if (dumpInfo.getData() != null && dumpInfo.getData().length > 0) {
// load chunk into target filter
RFuture<Void> loadFuture = targetFilter.loadChunkAsync(iterator, dumpInfo.getData());
}
} while (iterator != 0);
RedissonReactiveClient redisson = redissonClient.reactive();
RBloomFilterNativeReactive<String> sourceFilter = redisson.getBloomFilterNative("source");
RBloomFilterNativeReactive<String> targetFilter = redisson.getBloomFilterNative("target");
// dump all chunks from source filter
long iterator = 0;
do {
BloomFilterScanDumpInfo dumpInfo = sourceFilter.scanDump(iterator).block();
iterator = dumpInfo.getIterator();
if (dumpInfo.getData() != null && dumpInfo.getData().length > 0) {
// load chunk into target filter
targetFilter.loadChunk(iterator, dumpInfo.getData()).block();
}
} while (iterator != 0);
RedissonRxClient redisson = redissonClient.rxJava();
RBloomFilterNativeRx<String> sourceFilter = redisson.getBloomFilterNative("source");
RBloomFilterNativeRx<String> targetFilter = redisson.getBloomFilterNative("target");
// dump all chunks from source filter
long iterator = 0;
do {
BloomFilterScanDumpInfo dumpInfo = sourceFilter.scanDump(iterator).blockingGet();
iterator = dumpInfo.getIterator();
if (dumpInfo.getData() != null && dumpInfo.getData().length > 0) {
// load chunk into target filter
targetFilter.loadChunk(iterator, dumpInfo.getData()).blockingAwait();
}
} while (iterator != 0);
Cuckoo filter¶
Java implementation of Valkey or Redis based RCuckooFilter object is a cuckoo filter. A cuckoo filter is a probabilistic data structure for fast set membership testing, similar to a Bloom filter but with support for element deletion and counting. Covers CF.* commands of the Redis Bloom module. This object is thread-safe.
Initialization
The filter must be initialized before use. Simple initialization requires only a capacity value. Advanced initialization allows tuning of bucket size, max iterations, and expansion rate through CuckooFilterInitArgs.
Code examples:
RCuckooFilter<String> filter = redisson.getCuckooFilter("myCuckooFilter");
// simple initialization with capacity only
RFuture<Void> future = filter.initAsync(100000);
// advanced initialization with detailed parameters
RFuture<Void> advFuture = filter.initAsync(CuckooFilterInitArgs.capacity(100000)
.bucketSize(4)
.maxIterations(500)
.expansion(2));
RedissonReactiveClient redisson = redissonClient.reactive();
RCuckooFilterReactive<String> filter = redisson.getCuckooFilter("myCuckooFilter");
// simple initialization with capacity only
Mono<Void> mono = filter.init(100000);
// advanced initialization with detailed parameters
Mono<Void> advMono = filter.init(CuckooFilterInitArgs.capacity(100000)
.bucketSize(4)
.maxIterations(500)
.expansion(2));
RedissonRxClient redisson = redissonClient.rxJava();
RCuckooFilterRx<String> filter = redisson.getCuckooFilter("myCuckooFilter");
// simple initialization with capacity only
Completable rx = filter.init(100000);
// advanced initialization with detailed parameters
Completable advRx = filter.init(CuckooFilterInitArgs.capacity(100000)
.bucketSize(4)
.maxIterations(500)
.expansion(2));
Adding elements
Elements can be added individually or in bulk. The add() method allows adding the same element multiple times. The addIfAbsent() method adds an element only if it does not already exist in the filter. Bulk operations accept CuckooFilterAddArgs with optional capacity and noCreate parameters.
Code examples:
RCuckooFilter<String> filter = redisson.getCuckooFilter("myCuckooFilter");
// add a single element (allows duplicates)
boolean added = filter.add("element1");
// add element only if it does not already exist
boolean addedNew = filter.addIfAbsent("element2");
// bulk add with optional parameters
Set<String> addedItems = filter.add(
CuckooFilterAddArgs.<String>items(List.of("a", "b", "c"))
.capacity(50000)
.noCreate());
// bulk add only absent elements
Set<String> newItems = filter.addIfAbsent(
CuckooFilterAddArgs.<String>items(List.of("d", "e", "f"))
.capacity(50000));
RCuckooFilter<String> filter = redisson.getCuckooFilter("myCuckooFilter");
// add a single element (allows duplicates)
RFuture<Boolean> addFuture = filter.addAsync("element1");
// add element only if it does not already exist
RFuture<Boolean> addNxFuture = filter.addIfAbsentAsync("element2");
// bulk add with optional parameters
RFuture<Set<String>> bulkFuture = filter.addAsync(
CuckooFilterAddArgs.<String>items(List.of("a", "b", "c"))
.capacity(50000)
.noCreate());
// bulk add only absent elements
RFuture<Set<String>> bulkNxFuture = filter.addIfAbsentAsync(
CuckooFilterAddArgs.<String>items(List.of("d", "e", "f"))
.capacity(50000));
RedissonReactiveClient redisson = redissonClient.reactive();
RCuckooFilterReactive<String> filter = redisson.getCuckooFilter("myCuckooFilter");
// add a single element (allows duplicates)
Mono<Boolean> addMono = filter.add("element1");
// add element only if it does not already exist
Mono<Boolean> addNxMono = filter.addIfAbsent("element2");
// bulk add with optional parameters
Mono<Set<String>> bulkMono = filter.add(
CuckooFilterAddArgs.<String>items(List.of("a", "b", "c"))
.capacity(50000)
.noCreate());
// bulk add only absent elements
Mono<Set<String>> bulkNxMono = filter.addIfAbsent(
CuckooFilterAddArgs.<String>items(List.of("d", "e", "f"))
.capacity(50000));
RedissonRxClient redisson = redissonClient.rxJava();
RCuckooFilterRx<String> filter = redisson.getCuckooFilter("myCuckooFilter");
// add a single element (allows duplicates)
Single<Boolean> addRx = filter.add("element1");
// add element only if it does not already exist
Single<Boolean> addNxRx = filter.addIfAbsent("element2");
// bulk add with optional parameters
Single<Set<String>> bulkRx = filter.add(
CuckooFilterAddArgs.<String>items(List.of("a", "b", "c"))
.capacity(50000)
.noCreate());
// bulk add only absent elements
Single<Set<String>> bulkNxRx = filter.addIfAbsent(
CuckooFilterAddArgs.<String>items(List.of("d", "e", "f"))
.capacity(50000));
Checking element existence and counting occurrences
The exists() method checks if an element may exist in the filter. A return value of false guarantees the element is not present. A return value of true means the element may exist (false positives are possible). Multiple elements can be checked at once with exists(Collection). The count() method returns the approximate number of times an element has been added to the filter.
Code examples:
RCuckooFilter<String> filter = redisson.getCuckooFilter("myCuckooFilter");
// check single element
boolean mayExist = filter.exists("element1");
// check multiple elements at once
Set<String> existing = filter.exists(List.of("a", "b", "c", "d"));
// get approximate count of times an element was added
long approxCount = filter.count("element1");
RCuckooFilter<String> filter = redisson.getCuckooFilter("myCuckooFilter");
// check single element
RFuture<Boolean> existsFuture = filter.existsAsync("element1");
// check multiple elements at once
RFuture<Set<String>> mExistsFuture = filter.existsAsync(List.of("a", "b", "c", "d"));
// get approximate count of times an element was added
RFuture<Long> countFuture = filter.countAsync("element1");
RedissonReactiveClient redisson = redissonClient.reactive();
RCuckooFilterReactive<String> filter = redisson.getCuckooFilter("myCuckooFilter");
// check single element
Mono<Boolean> existsMono = filter.exists("element1");
// check multiple elements at once
Mono<Set<String>> mExistsMono = filter.exists(List.of("a", "b", "c", "d"));
// get approximate count of times an element was added
Mono<Long> countMono = filter.count("element1");
RedissonRxClient redisson = redissonClient.rxJava();
RCuckooFilterRx<String> filter = redisson.getCuckooFilter("myCuckooFilter");
// check single element
Single<Boolean> existsRx = filter.exists("element1");
// check multiple elements at once
Single<Set<String>> mExistsRx = filter.exists(List.of("a", "b", "c", "d"));
// get approximate count of times an element was added
Single<Long> countRx = filter.count("element1");
Removing elements
Unlike Bloom filters, cuckoo filters support element deletion. The remove() method deletes an element from the filter and returns true if it was found and removed.
Note: Deleting an element that was never added to the filter may cause false negatives for other elements.
Code examples:
Filter information
The getInfo() method returns a CuckooFilterInfo object containing filter statistics: memory size in bytes, number of buckets, number of sub-filters, number of inserted items, number of deleted items, bucket size, expansion rate, and maximum iterations.
Code examples:
RCuckooFilter<String> filter = redisson.getCuckooFilter("myCuckooFilter");
CuckooFilterInfo info = filter.getInfo();
info.getSize(); // memory size in bytes
info.getNumberOfBuckets(); // number of buckets
info.getNumberOfFilters(); // number of sub-filters
info.getNumberOfInsertedItems();// number of inserted items
info.getNumberOfDeletedItems(); // number of deleted items
info.getBucketSize(); // items per bucket
info.getExpansionRate(); // expansion rate
info.getMaxIterations(); // max swap attempts
Use Cases¶
Cuckoo filters provide fast, memory-efficient set membership testing where occasional false positives are acceptable but false negatives are not. Beyond what a Bloom filter offers, they support deletion and approximate counting, which makes them a fit for membership sets that change over time and for soft frequency limits.
Deduplication of Processed Items
Idempotent message handling, web-crawler URL frontiers, and "process each item once" pipelines need a compact record of what has already been seen. addIfAbsent both records and tests in a single call - it returns true only the first time an element is seen - so duplicates are skipped without a per-id lookup in a backing store. Because membership has no false negatives, a "seen" verdict is safe to act on, and unlike a Bloom filter an id can later be removed once it is safe to reprocess.
RCuckooFilter<String> seen = redisson.getCuckooFilter("processed-events");
seen.init(1_000_000);
// addIfAbsent returns true only the first time this id is seen
if (seen.addIfAbsent(eventId)) {
process(event); // first occurrence - handle it
}
// else: duplicate, already processed - skip
// allow the id to be processed again once the window has passed
seen.remove(eventId);
RCuckooFilter<String> seen = redisson.getCuckooFilter("processed-events");
// true only on first sight of the id
RFuture<Boolean> firstSeen = seen.addIfAbsentAsync(eventId);
firstSeen.whenComplete((isNew, exception) -> {
if (isNew) {
process(event);
}
});
// allow reprocessing later
RFuture<Boolean> removed = seen.removeAsync(eventId);
RedissonReactiveClient redisson = redissonClient.reactive();
RCuckooFilterReactive<String> seen = redisson.getCuckooFilter("processed-events");
// true only on first sight of the id
Mono<Boolean> firstSeen = seen.addIfAbsent(eventId);
// allow reprocessing later
Mono<Boolean> removed = seen.remove(eventId);
Mutable Allow/Deny Lists
Revoked-token denylists, blocked IPs, and ban lists are membership sets that change over time. The cuckoo filter's defining advantage over a Bloom filter is deletion: when a token is reinstated or a ban is lifted, its entry is removed with remove instead of forcing a full rebuild. exists serves as a cheap gate - a false result is authoritative (the value was never added, so no backend call is needed), while a true result is a possible match that can be confirmed against the source of truth.
RCuckooFilter<String> revoked = redisson.getCuckooFilter("revoked-tokens");
revoked.init(500_000);
// on revocation
revoked.add(tokenId);
// on validation - false is authoritative: the token was never revoked
if (revoked.exists(tokenId)) {
// possible match (or rare false positive) - confirm against the store
} else {
// definitely not revoked - accept without a backend lookup
}
// reinstating a token removes it - not possible with a Bloom filter
revoked.remove(tokenId);
RCuckooFilter<String> revoked = redisson.getCuckooFilter("revoked-tokens");
RFuture<Boolean> addFuture = revoked.addAsync(tokenId);
// false is authoritative - the token was never revoked
RFuture<Boolean> mayBeRevoked = revoked.existsAsync(tokenId);
// reinstate
RFuture<Boolean> removed = revoked.removeAsync(tokenId);
RedissonReactiveClient redisson = redissonClient.reactive();
RCuckooFilterReactive<String> revoked = redisson.getCuckooFilter("revoked-tokens");
Mono<Boolean> added = revoked.add(tokenId);
// false is authoritative - the token was never revoked
Mono<Boolean> mayBeRevoked = revoked.exists(tokenId);
// reinstate
Mono<Boolean> removed = revoked.remove(tokenId);
RedissonRxClient redisson = redissonClient.rxJava();
RCuckooFilterRx<String> revoked = redisson.getCuckooFilter("revoked-tokens");
Single<Boolean> added = revoked.add(tokenId);
// false is authoritative - the token was never revoked
Single<Boolean> mayBeRevoked = revoked.exists(tokenId);
// reinstate
Single<Boolean> removed = revoked.remove(tokenId);
Frequency Capping with Approximate Counting
Soft limits - ad impressions per user, retries per key, showing a tip at most a few times - need an approximate occurrence count rather than exact accounting. Because add permits duplicates and count returns the approximate number of times an element was added, each occurrence is a single add and each cap check is a single count, with no separate per-key counter to maintain. Counts may slightly over-report, which suits soft caps well.
RCuckooFilter<String> impressions = redisson.getCuckooFilter("ad-impressions");
impressions.init(2_000_000);
String key = userId + ":" + campaignId;
// stop before exceeding the cap (count is approximate)
if (impressions.count(key) < 3) {
showAd(campaignId);
impressions.add(key); // record this impression (duplicates allowed)
}
RCuckooFilter<String> impressions = redisson.getCuckooFilter("ad-impressions");
String key = userId + ":" + campaignId;
// approximate number of prior impressions
RFuture<Long> seen = impressions.countAsync(key);
seen.whenComplete((times, exception) -> {
if (times < 3) {
showAd(campaignId);
impressions.addAsync(key);
}
});
RedissonReactiveClient redisson = redissonClient.reactive();
RCuckooFilterReactive<String> impressions = redisson.getCuckooFilter("ad-impressions");
String key = userId + ":" + campaignId;
// approximate number of prior impressions
Mono<Long> seen = impressions.count(key);
// record an impression (duplicates allowed)
Mono<Boolean> recorded = impressions.add(key);
RedissonRxClient redisson = redissonClient.rxJava();
RCuckooFilterRx<String> impressions = redisson.getCuckooFilter("ad-impressions");
String key = userId + ":" + campaignId;
// approximate number of prior impressions
Single<Long> seen = impressions.count(key);
// record an impression (duplicates allowed)
Single<Boolean> recorded = impressions.add(key);
HyperLogLog¶
Java implementation of Valkey or Redis based RHyperLogLog object is a probabilistic data structure that estimates the number of distinct elements (the cardinality) of a very large set using a small, fixed amount of memory - on the order of a few kilobytes - no matter how many elements are added. The estimate carries a small standard error of about 0.81%, and the object is thread-safe.
Because it keeps only the estimate and not the elements themselves, it cannot list members or test whether a specific one is present - it answers "how many distinct items" rather than "is this item here". Elements are added one at a time or in bulk, the running estimate is read with count, and several logs can be combined or merged to count distinct items across them.
Code examples:
Adding elements¶
add records a single element and addAll records a whole collection at once. Both return true if the addition changed the structure's internal state - that is, if the element was probably new - and false if it almost certainly had no effect on the estimate.
Counting distinct elements¶
count returns the estimated number of distinct elements added to this log. The result is approximate - HyperLogLog trades exactness for a fixed, tiny memory footprint - so it suits large-scale counting where an exact figure is not required.
Combining multiple logs¶
Several logs can be counted together. countWith returns the estimated number of distinct elements across this log and the named ones without changing any of them - a combined total over, say, daily logs. mergeWith instead folds the named logs into this one, so its own count reflects their union from then on.
RHyperLogLog<String> today = redisson.getHyperLogLog("visitors:today");
// combined distinct count across several logs, without modifying them
long weekly = today.countWith("visitors:mon", "visitors:tue", "visitors:wed");
// merge other logs into this one; its count now reflects the union
today.mergeWith("visitors:mon", "visitors:tue");
RedissonReactiveClient redisson = redissonClient.reactive();
RHyperLogLogReactive<String> today = redisson.getHyperLogLog("visitors:today");
Mono<Long> weeklyMono = today.countWith("visitors:mon", "visitors:tue", "visitors:wed");
Mono<Void> mergeMono = today.mergeWith("visitors:mon", "visitors:tue");
Use Cases¶
HyperLogLog estimates the number of distinct elements in a set using a fixed, tiny amount of memory (around 12KB) no matter how many items are counted, with a typical error of about 1%. It fits problems where the count of unique items matters but the items themselves do not need to be stored, and where exact counting would otherwise require memory proportional to the cardinality.
Unique Visitor and Audience Counting
Counting distinct users, IP addresses, or devices over high-volume traffic is the canonical use: each event is added to the log, and count() returns the approximate number of unique values seen. A set holding every id would grow with the audience, while the log stays a fixed ~12KB whether it has seen a thousand ids or a billion.
RHyperLogLog<String> visitors = redisson.getHyperLogLog("visitors:2026-06-01");
// record each visit - duplicates collapse automatically
visitors.add(userId);
// ingest a batch in a single call
visitors.addAll(List.of("user:a", "user:b", "user:c"));
// approximate number of unique visitors (about 1% error)
long unique = visitors.count();
Rolling Unique Counts Across Time Windows
Reporting unique visitors per day, week, and month runs into a trap: distinct counts cannot simply be summed, because a user active on several days would be counted more than once. HyperLogLogs are mergeable, which solves this directly. Keeping one log per day, countWith estimates the de-duplicated union over a range without modifying the daily logs, while mergeWith folds several daily logs into a persistent weekly or monthly log.
RHyperLogLog<String> mon = redisson.getHyperLogLog("visitors:2026-06-01");
RHyperLogLog<String> tue = redisson.getHyperLogLog("visitors:2026-06-02");
RHyperLogLog<String> wed = redisson.getHyperLogLog("visitors:2026-06-03");
// unique visitors across the three days, counting returning users once
long weekToDate = mon.countWith(tue.getName(), wed.getName());
// roll the daily logs up into a persistent weekly log
RHyperLogLog<String> week = redisson.getHyperLogLog("visitors:2026-W23");
week.mergeWith(mon.getName(), tue.getName(), wed.getName());
long weeklyUnique = week.count();
RHyperLogLogAsync<String> mon = redisson.getHyperLogLog("visitors:2026-06-01");
// de-duplicated union across days without modifying the daily logs
RFuture<Long> weekToDate = mon.countWithAsync("visitors:2026-06-02", "visitors:2026-06-03");
// persistent roll-up into a weekly log
RHyperLogLogAsync<String> week = redisson.getHyperLogLog("visitors:2026-W23");
RFuture<Void> merged = week.mergeWithAsync("visitors:2026-06-01", "visitors:2026-06-02");
RedissonReactiveClient redisson = redissonClient.reactive();
RHyperLogLogReactive<String> mon = redisson.getHyperLogLog("visitors:2026-06-01");
// de-duplicated union across days without modifying the daily logs
Mono<Long> weekToDate = mon.countWith("visitors:2026-06-02", "visitors:2026-06-03");
// persistent roll-up into a weekly log
RHyperLogLogReactive<String> week = redisson.getHyperLogLog("visitors:2026-W23");
Mono<Void> merged = week.mergeWith("visitors:2026-06-01", "visitors:2026-06-02");
RedissonRxClient redisson = redissonClient.rxJava();
RHyperLogLogRx<String> mon = redisson.getHyperLogLog("visitors:2026-06-01");
// de-duplicated union across days without modifying the daily logs
Single<Long> weekToDate = mon.countWith("visitors:2026-06-02", "visitors:2026-06-03");
// persistent roll-up into a weekly log
RHyperLogLogRx<String> week = redisson.getHyperLogLog("visitors:2026-W23");
Completable merged = week.mergeWith("visitors:2026-06-01", "visitors:2026-06-02");
Estimating Audience Overlap
Since the union of two logs can be estimated with countWith, the overlap between two large sets follows from inclusion-exclusion: the number of common elements is approximately the sum of the two individual counts minus the count of their union. This estimates how many users are active on both web and mobile, or how many customers two campaigns reached in common, without storing either set of ids.
RHyperLogLog<String> web = redisson.getHyperLogLog("active:web");
RHyperLogLog<String> mobile = redisson.getHyperLogLog("active:mobile");
long onWeb = web.count(); // users seen on web
long onMobile = mobile.count(); // users seen on mobile
long onEither = web.countWith(mobile.getName());// users on either platform
// inclusion-exclusion: both = web + mobile - either
long onBoth = onWeb + onMobile - onEither;
TopK¶
Java implementation of Redis based RTopK object is a probabilistic data structure that keeps track of the k most frequent items in a stream using the HeavyKeeper algorithm, with a fixed amount of memory regardless of the number of distinct items seen. It is backed by the TOPK.* commands of the Redis Bloom module. This object is thread-safe.
A Top-K must be initialized once before items are added, after which occurrences are recorded and the current leaders can be queried at any time.
Initialization¶
init(int) reserves space for the given number of top items using default tuning, while init(TopKInitArgs) additionally controls the underlying sketch: width (counters per array, default 8), depth (number of counter arrays, default 7) and decay (probability of a counter being decreased on collision, default 0.9).
Adding items¶
Items are recorded with add(), or with an explicit weight using incrementBy(). Both return the item, if any, that was pushed out of the top-K list as a result of the call, or null when nothing was evicted. The list- and map-based overloads return their results positionally aligned to the input.
RTopK<String> topK = redisson.getTopK("searchTerms");
// record an occurrence; returns the evicted item or null
String evicted = topK.add("redis");
// record several at once (result aligned to input)
List<String> evictedItems = topK.add(List.of("redis", "valkey", "redis"));
// record with an explicit weight
topK.incrementBy("redis", 5);
// weight several items at once
topK.incrementBy(Map.of("redis", 5, "valkey", 2));
Querying membership and leaders¶
contains() reports whether an item is currently among the leaders, list() returns the current top-K items and listWithCount() returns them together with their approximate counts. The older count() method is deprecated since Redis Bloom 2.4.0 because its estimate can be inaccurate; prefer listWithCount().
RTopK<String> topK = redisson.getTopK("searchTerms");
// is an item currently among the leaders?
boolean present = topK.contains("redis");
List<Boolean> presence = topK.contains(List.of("redis", "valkey"));
// current leaders, optionally with their approximate counts
List<String> leaders = topK.list();
Map<String, Long> leadersWithCount = topK.listWithCount();
Top-K information¶
getInfo() returns the configured parameters of the structure - the number of tracked items together with the sketch width, depth and decay.
Use Cases¶
Top-K answers which items are most frequent? over an unbounded stream while using only a small, fixed amount of memory, which makes it a fit for trend detection and heavy-hitter analysis at high volume, where keeping an exact counter per distinct item would be too expensive.
Trending content
Leaderboards of trending search terms, hashtags, products or pages can be maintained directly from the event stream. Each view or query is fed in with add (or weighted with incrementBy), and the current leaders - with their approximate counts - are read back with listWithCount, without storing a counter per distinct item.
Heavy-hitter detection
In high-volume traffic - API calls, log lines, network flows - a Top-K surfaces the handful of clients, IPs or keys responsible for a disproportionate share of requests, so hot keys and potential abuse can be spotted in bounded memory. contains then gives a cheap test for whether a given client is currently among the top talkers.
TDigest¶
Java implementation of Redis based RTDigest object is a t-digest - a probabilistic data structure that estimates quantiles, ranks and the cumulative distribution of a stream of observations with sub-linear memory and high accuracy at the distribution's tails. It is backed by the TDIGEST.* commands of the Redis Bloom module. This object is thread-safe.
The sketch must be created with create() before use.
Initialization¶
Create the sketch before adding observations. An optional compression factor trades memory for accuracy - a higher value yields more accurate estimates, particularly at the tails, at the cost of more memory. reset() empties the sketch and re-initializes it.
Adding observations¶
Feed observations into the sketch one at a time or several at once.
Merging sketches¶
Several sketches can be merged into one destination, which is useful for combining per-node or per-shard sketches into a single global view. mergeWith(String...) merges the named source sketches on top of the destination's current contents, while mergeWith(TDigestMergeArgs) additionally controls the resulting compression and can override the destination, discarding its existing observations before the merge.
RTDigest global = redisson.getTDigest("latencies:global");
global.create();
// merge per-node sketches into the destination
global.mergeWith("latencies:node1", "latencies:node2");
// set the resulting compression and discard the destination's current contents
global.mergeWith(TDigestMergeArgs.keys("latencies:node1", "latencies:node2")
.compression(200)
.override());
Quantiles and cumulative distribution¶
quantile() estimates, for each input fraction, the value below which that fraction of observations fall - for example the median or the 99th percentile. cumulativeProbability() is the inverse: for each input value it estimates the fraction of observations less than or equal to it. Both accept several inputs and return one result per input.
Ranks¶
rank() returns, for each value, the number of observations less than it, and revRank() the number greater than it (both return -1 for a value outside the observed range). byRank() and byRevRank() are the inverse, returning the value at a given rank counting from the smallest or largest observation respectively.
RTDigest tdigest = redisson.getTDigest("latencies");
// how many observations fall below / above given values
List<Long> ranks = tdigest.rank(100.0, 250.0);
List<Long> reverseRanks = tdigest.revRank(100.0, 250.0);
// the value at given ranks, from the smallest / largest observation
List<Double> values = tdigest.byRank(0, 99);
List<Double> valuesFromTop = tdigest.byRevRank(0, 9);
RTDigest tdigest = redisson.getTDigest("latencies");
RFuture<List<Long>> ranksFuture = tdigest.rankAsync(100.0, 250.0);
RFuture<List<Long>> reverseRanksFuture = tdigest.revRankAsync(100.0, 250.0);
RFuture<List<Double>> valuesFuture = tdigest.byRankAsync(0, 99);
RFuture<List<Double>> valuesFromTopFuture = tdigest.byRevRankAsync(0, 9);
Summary statistics¶
getMin() and getMax() return the smallest and largest observations (or NaN when the sketch is empty), and trimmedMean() returns the mean of the observations between two cut quantiles, ignoring outliers at the tails.
Sketch information¶
getInfo() returns a TDigestInfo describing the sketch: its compression and capacity, the number of merged and unmerged nodes and their weights, the total number of observations, the number of compressions performed, and the estimated memory usage.
Use Cases¶
A t-digest answers what does the value distribution look like? over an unbounded stream of numbers - any quantile, rank or trimmed mean - while using only a small, fixed amount of memory and staying most accurate at the tails, where percentile monitoring needs it most. This makes it a fit for latency and SLA tracking and for distribution-based thresholds at high volume, where retaining every observation to compute exact percentiles would be too expensive.
Latency and SLA monitoring
Request or operation latencies are fed into the sketch with add as they complete, and tail percentiles - p50, p95, p99 - are read back at any time with quantile, giving a continuously updated picture of the latency distribution in a tiny, fixed footprint. cumulativeProbability answers the inverse SLA question directly - what fraction of requests came in under the target - and per-instance sketches gathered across a fleet can be folded into one cluster-wide view with mergeWith.
RTDigest latencies = redisson.getTDigest("latencies:checkout");
latencies.create();
// record each request's latency as it completes
latencies.add(requestMillis);
// p50 / p95 / p99 for the dashboard
List<Double> percentiles = latencies.quantile(0.5, 0.95, 0.99);
// SLA check: fraction of requests served under the 200ms target
double withinSla = latencies.cumulativeProbability(200.0).get(0);
Adaptive thresholds and outlier detection
Because the sketch tracks the whole distribution, it can supply thresholds that adapt to the data instead of being hard-coded. quantile yields a dynamic cutoff - flag anything above the current 99th percentile - while cumulativeProbability and rank place a new observation within the historical distribution to show how extreme it is. trimmedMean provides a stable central estimate that ignores the heaviest outliers at both tails.
RTDigest amounts = redisson.getTDigest("tx:amounts");
amounts.create();
// dynamic threshold: the current 99th percentile
double cutoff = amounts.quantile(0.99).get(0);
// how extreme is a new value relative to history?
double fractionBelow = amounts.cumulativeProbability(newAmount).get(0);
boolean isOutlier = newAmount > cutoff;
// robust baseline that ignores the top and bottom 5%
double baseline = amounts.trimmedMean(0.05, 0.95);