Package: blooms
Lightweight Bloom filter data structure derived from the built-in bytearray type.PurposeThis library provides a simple and lightweight data structure for representingBloom filtersthat is derived from the built-inbytearraytype. The data structure has methods for the insertion, membership, union, and subset operations. In addition, methods for estimating capacity and for converting to and from Base64 strings are available.Installation and UsageThis library is available as apackage on PyPI:python-mpipinstallbloomsThe library can be imported in the usual ways:importbloomsfrombloomsimportbloomsExamplesThis library makes it possible to concisely create, populate, and query simpleBloom filters. The example below constructs a Bloom filter that is 32 bits (i.e., four bytes) in size:>>>frombloomsimportblooms>>>b=blooms(4)A bytes-like object can be inserted into an instance using theinsertion operator@=. It is the responsibility of the user of the library to hash and truncate the bytes-like object being inserted. Only the bytes that remain after truncation contribute to the membership of the bytes-like object within the Bloom filter:>>>fromhashlibimportsha256>>>x='abc'# Value to insert.>>>h=sha256(x.encode()).digest()# Hash of value.>>>t=h[:2]# Truncated hash.>>>b@=t# Insert the value into the Bloom filter.>>>b.hex()'00000004'When testing whether a bytes-like object is a member using themembership operator@of an instance, the same hashing and truncation operations should be applied:>>>sha256('abc'.encode()).digest()[:2]@bTrue>>>sha256('xyz'.encode()).digest()[:2]@bFalseTheinsertion operator@=also accepts iterable containers:>>>x=sha256('x'.encode()).digest()[:2]>>>y=sha256('y'.encode()).digest()[:2]>>>z=sha256('z'.encode()).digest()[:2]>>>b@=[x,y,z]>>>b.hex()'02200006'The union of two Bloom filters (both having the same size) can be computed via thebuilt-in|operator:>>>c=blooms(4)>>>c@=sha256('xyz'.encode()).digest()[:2]>>>d=c|b>>>sha256('abc'.encode()).digest()[:2]@dTrue>>>sha256('xyz'.encode()).digest()[:2]@dTrueIt is also possible to check whether the members of one Bloom filterare a subsetof the members of another Bloom filter:>>>b.issubset(c)False>>>b.issubset(d)TrueThesaturationmethod calculates the saturation of a Bloom filter. Thesaturationis afloatvalue (between0.0and1.0) that represents an upper bound on the rate with which false positives will occur when testing bytes-like objects (of a specific length) for membership within the Bloom filter:>>>b=blooms(32)>>>fromsecretsimporttoken_bytes>>>for_inrange(8):...b@=token_bytes(4)>>>b.saturation(4)0.03125It is also possible to determine the approximate maximum capacity of a Bloom filter for a given saturation limit using thecapacitymethod. For example, the output below indicates that a saturation of0.05will likely be reached after more than28insertions of bytes-like objects of length8:>>>b=blooms(32)>>>b.capacity(8,0.05)28In addition, conversion methods to and from Base64 strings are included to support concise encoding and decoding:>>>b.to_base64()'AiAABg=='>>>sha256('abc'.encode()).digest()[:2]@blooms.from_base64('AiAABg==')TrueIf it is preferable to have a Bloom filter data structure that encapsulates a particular serialization, hashing, and truncation scheme, the recommended approach is to define a derived class. Thespecializemethod makes it possible to do so in a concise way:>>>encode=lambdax:sha256(x).digest()[:2]>>>blooms_custom=blooms.specialize(name='blooms_custom',encode=encode)>>>b=blooms_custom(4)>>>b@=bytes([1,2,3])>>>bytes([1,2,3])@bTrueThe user of the library is responsible for ensuring that Base64-encoded Bloom filters are converted back into an an instance of the appropriate derived class by using thefrom_base64method that belongs to that derived class:>>>isinstance(blooms_custom.from_base64(b.to_base64()),blooms_custom)TrueDevelopmentAll installation and development dependencies are fully specified inpyproject.toml. Theproject.optional-dependenciesobject is used tospecify optional requirementsfor various development tasks. This makes it possible to specify additional options (such asdocs,lint, and so on) when performing installation usingpip:python-mpipinstall.[docs,lint]DocumentationThe documentation can be generated automatically from the source files usingSphinx:python-mpipinstall.[docs]cddocssphinx-apidoc-f-E--templatedir=_templates-o_source..&&makehtmlTesting and ConventionsAll unit tests are executed and their coverage is measured when usingpytest(see thepyproject.tomlfile for configuration details):python-mpipinstall.[test]python-mpytestThe subset of the unit tests included in the module itself and can be executed usingdoctest:pythonsrc/blooms/blooms.py-vStyle conventions are enforced usingPylint:python-mpipinstall.[lint]python-mpylintsrc/bloomstest/test_blooms.pyContributionsIn order to contribute to the source code, open an issue or submit a pull request on theGitHub pagefor this library.VersioningThe version number format for this library and the changes to the library associated with version number increments conform withSemantic Versioning 2.0.0.PublishingThis library can be published as apackage on PyPIby a package maintainer. First, install the dependencies required for packaging and publishing:python-mpipinstall.[publish]Ensure that the correct version number appears inpyproject.toml, and that any links in this README document to the Read the Docs documentation of this package (or its dependencies) have appropriate version numbers. Also ensure that the Read the Docs project for this library has anautomation rulethat activates and sets as the default all tagged versions. Create and push a tag for this version (replacing?.?.?with the version number):gittag?.?.?gitpushorigin?.?.?Remove any old build/distribution files. Then, package the source into a distribution archive:rm-rfbuilddistsrc/*.egg-infopython-mbuild--sdist--wheel.Finally, upload the package distribution archive toPyPI:python-mtwineuploaddist/*