Package: xxhash
xxhash is a Python binding for thexxHashlibrary byYann Collet.Installation$pipinstallxxhashYou can also install using conda:$condainstall-cconda-forgepython-xxhashInstalling From Source$pipinstall--no-binaryxxhashxxhashPrerequisitesOn Debian/Ubuntu:$apt-getinstallpython-devgccOn CentOS/Fedora:$yuminstallpython-develgccredhat-rpm-configLinking to libxxhash.soBy default python-xxhash will use bundled xxHash,
we can change this by specifying ENV varXXHASH_LINK_SO:$XXHASH_LINK_SO=1pipinstall--no-binaryxxhashxxhashUsageModule version and its backend xxHash library version can be retrieved using
the module propertiesVERSIONANDXXHASH_VERSIONrespectively.>>>importxxhash>>>xxhash.VERSION'2.0.0'>>>xxhash.XXHASH_VERSION'0.8.0'This module is hashlib-compliant, which means you can use it in the same way ashashlib.md5.update() – update the current digest with an additional stringdigest() – return the current digest valuehexdigest() – return the current digest as a string of hexadecimal digitsintdigest() – return the current digest as an integercopy() – return a copy of the current xxhash objectreset() – reset statemd5 digest returns bytes, but the original xxh32 and xxh64 C APIs return integers.
While this module is made hashlib-compliant,intdigest()is also provided to
get the integer digest.Constructors for hash algorithms provided by this module arexxh32()andxxh64().For example, to obtain the digest of the byte stringb'Nobody inspects the spammish repetition':>>>importxxhash>>>x=xxhash.xxh32()>>>x.update(b'Nobody inspects')>>>x.update(b' the spammish repetition')>>>x.digest()b'\xe2);/'>>>x.digest_size4>>>x.block_size16More condensed:>>>xxhash.xxh32(b'Nobody inspects the spammish repetition').hexdigest()'e2293b2f'>>>xxhash.xxh32(b'Nobody inspects the spammish repetition').digest()==x.digest()TrueAn optional seed (default is 0) can be used to alter the result predictably:>>>importxxhash>>>xxhash.xxh64('xxhash').hexdigest()'32dd38952c4bc720'>>>xxhash.xxh64('xxhash',seed=20141025).hexdigest()'b559b98d844e0635'>>>x=xxhash.xxh64(seed=20141025)>>>x.update('xxhash')>>>x.hexdigest()'b559b98d844e0635'>>>x.intdigest()13067679811253438005Be careful that xxh32 takes an unsigned 32-bit integer as seed, while xxh64
takes an unsigned 64-bit integer. Although unsigned integer overflow is
defined behavior, it’s better not to make it happen:>>>xxhash.xxh32('I want an unsigned 32-bit seed!',seed=0).hexdigest()'f7a35af8'>>>xxhash.xxh32('I want an unsigned 32-bit seed!',seed=2**32).hexdigest()'f7a35af8'>>>xxhash.xxh32('I want an unsigned 32-bit seed!',seed=1).hexdigest()'d8d4b4ba'>>>xxhash.xxh32('I want an unsigned 32-bit seed!',seed=2**32+1).hexdigest()'d8d4b4ba'>>>>>>xxhash.xxh64('I want an unsigned 64-bit seed!',seed=0).hexdigest()'d4cb0a70a2b8c7c1'>>>xxhash.xxh64('I want an unsigned 64-bit seed!',seed=2**64).hexdigest()'d4cb0a70a2b8c7c1'>>>xxhash.xxh64('I want an unsigned 64-bit seed!',seed=1).hexdigest()'ce5087f12470d961'>>>xxhash.xxh64('I want an unsigned 64-bit seed!',seed=2**64+1).hexdigest()'ce5087f12470d961'digest()returns bytes of thebig-endianrepresentation of the integer
digest:>>>importxxhash>>>h=xxhash.xxh64()>>>h.digest()b'\xefF\xdb7Q\xd8\xe9\x99'>>>h.intdigest().to_bytes(8,'big')b'\xefF\xdb7Q\xd8\xe9\x99'>>>h.hexdigest()'ef46db3751d8e999'>>>format(h.intdigest(),'016x')'ef46db3751d8e999'>>>h.intdigest()17241709254077376921>>>int(h.hexdigest(),16)17241709254077376921Besides xxh32/xxh64 mentioned above, oneshot functions are also provided,
so we can avoid allocating XXH32/64 state on heap:xxh32_digest(bytes, seed=0)xxh32_intdigest(bytes, seed=0)xxh32_hexdigest(bytes, seed=0)xxh64_digest(bytes, seed=0)xxh64_intdigest(bytes, seed=0)xxh64_hexdigest(bytes, seed=0)>>>importxxhash>>>xxhash.xxh64('a').digest()==xxhash.xxh64_digest('a')True>>>xxhash.xxh64('a').intdigest()==xxhash.xxh64_intdigest('a')True>>>xxhash.xxh64('a').hexdigest()==xxhash.xxh64_hexdigest('a')True>>>xxhash.xxh64_hexdigest('xxhash',seed=20141025)'b559b98d844e0635'>>>xxhash.xxh64_intdigest('xxhash',seed=20141025)13067679811253438005L>>>xxhash.xxh64_digest('xxhash',seed=20141025)'\xb5Y\xb9\x8d\x84N\x065'In[1]:importxxhashIn[2]:%timeitxxhash.xxh64_hexdigest('xxhash')268ns±24.1nsperloop(mean±std.dev.of7runs,1000000loopseach)In[3]:%timeitxxhash.xxh64('xxhash').hexdigest()416ns±17.3nsperloop(mean±std.dev.of7runs,1000000loopseach)XXH3 hashes are available since v2.0.0 (xxHash v0.8.0), they are:Streaming classes:xxh3_64xxh3_128Oneshot functions:xxh3_64_digest(bytes, seed=0)xxh3_64_intdigest(bytes, seed=0)xxh3_64_hexdigest(bytes, seed=0)xxh3_128_digest(bytes, seed=0)xxh3_128_intdigest(bytes, seed=0)xxh3_128_hexdigest(bytes, seed=0)And aliases:xxh128 = xxh3_128xxh128_digest = xxh3_128_digestxxh128_intdigest = xxh3_128_intdigestxxh128_hexdigest = xxh3_128_hexdigestCaveatsSEED OVERFLOWxxh32 takes an unsigned 32-bit integer as seed, and xxh64 takes
an unsigned 64-bit integer as seed. Make sure that the seed is greater than
or equal to0.ENDIANNESSAs of python-xxhash 0.3.0,digest()returns bytes of thebig-endianrepresentation of the integer digest. It used
to be little-endian.DONT USE XXHASH IN HMACThough you can use xxhash as anHMAChash function, but it’s
highly recommended not to.xxhash isNOTa cryptographic hash function, it is a
non-cryptographic hash algorithm aimed at speed and quality.
Do not put xxhash in any position where cryptographic hash
functions are required.Copyright and LicenseCopyright (c) 2014-2020 Yue Du -https://github.com/ifduyueLicensed underBSD 2-Clause LicenseCHANGELOGv3.4.1 2023-10-05Remove setuptools_scmv3.4.0 2023-10-05Build wheels for Python 3.12v3.3.0 2023-07-29Upgrade xxHash to v0.8.2Drop support for Python 3.6v3.2.0 2022-12-28This is the last version to support Python 3.6Build Python 3.11 wheels.Remove setup.py test_suites, call unittest directlyv3.1.0 2022-10-19Type annotations.Enabled muslinux wheels building.v3.0.0 2022-02-25New setalgorithms_availablelists all implemented algorithms inxxhashpackage.Upgrade xxHash to v0.8.1.Drop support for EOL Python versions, require python >= 3.6 from now on.Migrate to github actions and build arm64 wheels for macOS.Always release GIL.v2.0.2 2021-04-15Fix Travis CI OSX dpl python2.7 get-pip.py errorv2.0.1 2021-04-15Only to trigger Python 3.9 wheels building.v2.0.0 2020-08-03Require xxHash version >= v0.8.0Upgrade xxHash to v0.8.0XXH3 hashes:xxh3_64,xxh3_128, and their oneshot functionsv1.4.4 2020-06-20Upgrade xxHash to v0.7.3Stop using PEP393 deprecated APIsUse XXH(32|64)_canonicalFromHash to replace u2bytes and ull2bytesv1.4.3 2019-11-12Upgrade xxHash to v0.7.2Python 3.8 wheelsv1.4.2 2019-10-13Fixed: setup.py fails when reading README.rst and the default encoding is not UTF-8v1.4.1 2019-08-27Fixed: xxh3.h in missing from source tarballv1.4.0 2019-08-25Upgrade xxHash to v0.7.1v1.3.0 2018-10-21Wheels are now built automaticallySplit CFFI variant into a separate packageifduyue/python-xxhash-cffiv1.2.0 2018-07-13Add oneshot functions xxh{32,64}_{,int,hex}digestv1.1.0 2018-07-05Allow input larger than 2GBRelease the GIL on sufficiently large inputDrop support for Python 3.2v1.0.1 2017-03-02Free state actively, instead of delegating it to ffi.gcv1.0.0 2017-02-10Fixed copy() segfaultAdded CFFI variantv0.6.3 2017-02-10Fixed copy() segfaultv0.6.2 2017-02-10Upgrade xxHash to v0.6.2v0.6.1 2016-06-26Upgrade xxHash to v0.6.1v0.5.0 2016-03-02Upgrade xxHash to v0.5.0v0.4.3 2015-08-21Upgrade xxHash to r42v0.4.1 2015-08-16Upgrade xxHash to r41v0.4.0 2015-08-05Added method resetUpgrade xxHash to r40v0.3.2 2015-01-27Fixed some typos in docstringsv0.3.1 2015-01-24Upgrade xxHash to r39v0.3.0 2014-11-11Change digest() from little-endian representation to big-endian representation of the integer digest.
This change breaks compatibility (digest() results are different).v0.2.0 2014-10-25Make this package hashlib-compliantv0.1.3 2014-10-23Update xxHash to r37v0.1.2 2014-10-19Improve: Check XXHnn_init() return value.Update xxHash to r36v0.1.1 2014-08-07Improve: Can now be built with Visual C++ Compiler.v0.1.0 2014-08-05New: XXH32 and XXH64 type, which support partially update.Fix: build under Python 3.4v0.0.2 2014-08-03NEW: Support Python 3v0.0.1 2014-07-30NEW: xxh32 and xxh64