Package: fastbloom-rs
fastbloomA fastbloom filter|counting bloom filterimplemented by Rust for Rust and
Python!Language:简体中文setupPythonrequirementsinstallRustJavaExamplesBloomFilterPythonRustCountingBloomFilterPythonRustbenchmarkcomputer infoadd one str to bloom filteradd one million to bloom filtercheck one contains in bloom filtercheck one not contains in bloom filteradd one str to counting bloom filteradd one million to counting bloom filtersetupPythonrequirementsPython >= 3.7installInstall the latest fastbloom version with:pipinstallfastbloom-rsRustfastbloom-rs="{latest}"Javamaven<dependency><groupId>io.github.yankun1992</groupId><artifactId>fastbloom</artifactId><version>{latest-version}</version></dependency>ExamplesBloomFilterA Bloom filter is a space-efficient probabilistic data structure, conceived by Burton Howard
Bloom in 1970, that is used to test whether an element is a member of a set. False positive
matches are possible, but false negatives are not.Reference: Bloom, B. H. (1970). Space/time trade-offs in hash coding with allowable errors.
Communications of the ACM, 13(7), 422-426.Full text articlePythonbasic usagefromfastbloom_rsimportBloomFilterbloom=BloomFilter(100_000_000,0.01)bloom.add_str('hello')bloom.add_bytes(b'world')bloom.add_int(9527)assertbloom.contains('hello')assertbloom.contains(b'world')assertbloom.contains(9527)assertnotbloom.contains('hello world')build bloom filter from bytes or listfromfastbloom_rsimportBloomFilterbloom=BloomFilter(100_000_000,0.01)bloom.add_str('hello')assertbloom.contains('hello')bloom2=BloomFilter.from_bytes(bloom.get_bytes(),bloom.hashes())assertbloom2.contains('hello')bloom3=BloomFilter.from_int_array(bloom.get_int_array(),bloom.hashes())assertbloom3.contains('hello')there are some bulk api for python to reduce ffi cost between python and rustbloom=BloomFilter(100_000_000,0.01)inserts=[1,2,3,4,5,6,7,9,18,68,90,100]checks=[1,2,3,4,5,6,7,9,18,68,90,100,190,290,390]results=[True,True,True,True,True,True,True,True,True,True,True,True,False,False,False]bloom.add_int_batch(inserts)contains=bloom.contains_int_batch(checks)assertcontains==resultsbloom.add_str_batch(list(map(lambdax:str(x),inserts)))assertbloom.contains_str_batch(list(map(lambdax:str(x),checks)))==resultsbloom.add_bytes_batch(list(map(lambdax:bytes(x),inserts)))assertbloom.contains_bytes_batch(list(map(lambdax:bytes(x),checks)))==resultsmore examples atpy_tests.Rustusefastbloom_rs::{BloomFilter,FilterBuilder};letmutbloom=FilterBuilder::new(100_000_000,0.01).build_bloom_filter();bloom.add(b"helloworld");assert_eq!(bloom.contains(b"helloworld"),true);assert_eq!(bloom.contains(b"helloworld!"),false);more examples atdocs.rsCountingBloomFilterA Counting Bloom filter works in a similar manner as a regular Bloom filter; however, it is
able to keep track of insertions and deletions. In a counting Bloom filter, each entry in the
Bloom filter is a small counter associated with a basic Bloom filter bit.Reference: F. Bonomi, M. Mitzenmacher, R. Panigrahy, S. Singh, and G. Varghese, “An Improved
Construction for Counting Bloom Filters,” in 14th Annual European Symposium on
Algorithms, LNCS 4168, 2006Pythonfromfastbloom_rsimportCountingBloomFiltercbf=CountingBloomFilter(1000_000,0.01)cbf.add('hello')cbf.add('hello')assert'hello'incbfcbf.remove('hello')assert'hello'incbf# because 'hello' added twice.# If add same element larger than 15 times, then remove 15 times the filter will not contain the element.cbf.remove('hello')assert'hello'notincbfA CountingBloomFilter has a four bits counter to save hash index, so when insert an
element repeatedly, the counter will spill over quickly. So, you can setenable_repeat_inserttoFalseto check whether the element has added.
if it has added, it will not add again.enable_repeat_insertdefault set toTrue.fromfastbloom_rsimportCountingBloomFiltercbf=CountingBloomFilter(1000_000,0.01,False)cbf.add('hello')cbf.add('hello')# because enable_repeat_insert=False, this addition will not take effect.assert'hello'incbfcbf.remove('hello')assert'hello'notincbfmore examples atpy_tests.Rustusefastbloom_rs::{CountingBloomFilter,FilterBuilder};letmutbuilder=FilterBuilder::new(100_000,0.01);letmutcbf=builder.build_counting_bloom_filter();cbf.add(b"helloworld");assert_eq!(bloom.contains(b"helloworld"),true);benchmarkcomputer infoCPUMemoryOSAMD Ryzen 7 5800U with Radeon Graphics16GWindows 10add one str to bloom filterBenchmark insert one str to bloom filter:bloom_add_test          time:   [41.168 ns 41.199 ns 41.233 ns]
                        change: [-0.4891% -0.0259% +0.3417%] (p = 0.91 > 0.05)
                        No change in performance detected.
Found 13 outliers among 100 measurements (13.00%)
  1 (1.00%) high mild
  12 (12.00%) high severeadd one million to bloom filterBenchmark loop insert(1..1_000_000).map(|n| { n.to_string() })to bloom filter:bloom_add_all_test      time:   [236.24 ms 236.86 ms 237.55 ms]
                        change: [-3.4346% -2.9050% -2.3524%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 5 outliers among 100 measurements (5.00%)
  4 (4.00%) high mild
  1 (1.00%) high severecheck one contains in bloom filterbloom_contains_test     time:   [42.065 ns 42.102 ns 42.156 ns]
                        change: [-0.7830% -0.5901% -0.4029%] (p = 0.00 < 0.05)
                        Change within noise threshold.
Found 15 outliers among 100 measurements (15.00%)
  1 (1.00%) low mild
  5 (5.00%) high mild
  9 (9.00%) high severecheck one not contains in bloom filterbloom_not_contains_test time:   [22.695 ns 22.727 ns 22.773 ns]
                        change: [-3.1948% -2.9695% -2.7268%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 12 outliers among 100 measurements (12.00%)
  4 (4.00%) high mild
  8 (8.00%) high severeadd one str to counting bloom filtercounting_bloom_add_test time:   [60.822 ns 60.861 ns 60.912 ns]
                        change: [+0.2427% +0.3772% +0.5579%] (p = 0.00 < 0.05)
                        Change within noise threshold.
Found 10 outliers among 100 measurements (10.00%)
  1 (1.00%) low severe
  4 (4.00%) low mild
  1 (1.00%) high mild
  4 (4.00%) high severeadd one million to counting bloom filterBenchmark loop insert(1..1_000_000).map(|n| { n.to_string() })to counting bloom filter:counting_bloom_add_million_test
                        time:   [272.48 ms 272.58 ms 272.68 ms]
Found 2 outliers among 100 measurements (2.00%)
  1 (1.00%) low mild
  1 (1.00%) high mild