Package: bitarray-hardbyte
bitarray: efficient arrays of booleansThis library provides an object type which efficiently represents an array
of booleans.  Bitarrays are sequence types and behave very much like usual
lists.  Eight bits are represented by one byte in a contiguous block of
memory.  The user can select between two representations: little-endian
and big-endian.  All functionality is implemented in C.
Methods for accessing the machine representation are provided, including the
ability to import and export buffers.  This allows creating bitarrays that
are mapped to other objects, including memory-mapped files.Key featuresThe bit endianness can be specified for each bitarray object, see below.Sequence methods: slicing (including slice assignment and deletion),
operations+,*,+=,*=, theinoperator,len()Fast methods for encoding and decoding variable bit length prefix codes.Bitwise operations:~,&,|,^,<<,>>(as well as
their in-place versions&=,|=,^=,<<=,>>=).Packing and unpacking to other binary data formats, e.g.numpy.ndarray.Bitarray objects support the buffer protocol (both importing and
exporting buffers).frozenbitarrayobjects which are hashablePickling and unpickling of bitarray objects.Sequential searchExtensive test suite with over 400 unittests.Utility modulebitarray.util:conversion to hexadecimal stringserializationpretty printingconversion to integerscreating Huffman codesvarious count functionsother helpful functionsInstallationIf you have a working C compiler, you can simply:$pipinstallbitarrayIf you rather want to use precompiled binaries, you can:conda install bitarray(both the default Anaconda repository as well
as conda-forge support bitarray)download Windows wheels fromChris GohlkeOnce you have installed the package, you may want to test it:$python-c'import bitarray; bitarray.test()'bitarray is installed in: /Users/ilan/bitarray/bitarray
bitarray version: 2.3.7
sys.version: 2.7.15 (default, Mar  5 2020, 14:58:04) [GCC Clang 9.0.1]
sys.prefix: /Users/ilan/Mini3/envs/py27
pointer size: 64 bit
sizeof(size_t): 8
sizeof(bitarrayobject): 80
PY_UINT64_T defined: 1
USE_WORD_SHIFT: 1
DEBUG: 0
.........................................................................
.........................................................................
................................................................
----------------------------------------------------------------------
Ran 409 tests in 0.484s

OKYou can always import the function test,
andtest().wasSuccessful()will returnTruewhen the test went well.Using the moduleAs mentioned above, bitarray objects behave very much like lists, so
there is not too much to learn.  The biggest difference from list
objects (except that bitarray are obviously homogeneous) is the ability
to access the machine representation of the object.
When doing so, the bit endianness is of importance; this issue is
explained in detail in the section below.  Here, we demonstrate the
basic usage of bitarray objects:>>>frombitarrayimportbitarray>>>a=bitarray()# create empty bitarray>>>a.append(1)>>>a.extend([1,0])>>>abitarray('110')>>>x=bitarray(2**20)# bitarray of length 1048576 (uninitialized)>>>len(x)1048576>>>bitarray('1001 011')# initialize from string (whitespace is ignored)bitarray('1001011')>>>lst=[1,0,False,True,True]>>>a=bitarray(lst)# initialize from iterable>>>abitarray('10011')>>>a.count(1)3>>>a.remove(0)# removes first occurrence of 0>>>abitarray('1011')Like lists, bitarray objects support slice assignment and deletion:>>>a=bitarray(50)>>>a.setall(0)# set all elements in a to 0>>>a[11:37:3]=9*bitarray('1')>>>abitarray('00000000000100100100100100100100100100000000000000')>>>dela[12::3]>>>abitarray('0000000000010101010101010101000000000')>>>a[-6:]=bitarray('10011')>>>abitarray('000000000001010101010101010100010011')>>>a+=bitarray('000111')>>>a[9:]bitarray('001010101010101010100010011000111')In addition, slices can be assigned to booleans, which is easier (and
faster) than assigning to a bitarray in which all values are the same:>>>a=20*bitarray('0')>>>a[1:15:3]=True>>>abitarray('01001001001001000000')This is easier and faster than:>>>a=20*bitarray('0')>>>a[1:15:3]=5*bitarray('1')>>>abitarray('01001001001001000000')Note that in the latter we have to create a temporary bitarray whose length
must be known or calculated.  Another example of assigning slices to Booleans,
is setting ranges:>>>a=bitarray(30)>>>a[:]=0# set all elements to 0 - equivalent to a.setall(0)>>>a[10:25]=1# set elements in range(10, 25) to 1>>>abitarray('000000000011111111111111100000')Bitwise operatorsBitarray objects support the bitwise operators~,&,|,^,<<,>>(as well as their in-place versions&=,|=,^=,<<=,>>=).  The behavior is very much what one would expect:>>>a=bitarray('101110001')>>>~a# invertbitarray('010001110')>>>b=bitarray('111001011')>>>a^bbitarray('010111010')>>>a&=b>>>abitarray('101000001')>>>a<<=2# in-place left shift by 2>>>abitarray('100000100')>>>b>>1bitarray('011100101')The C language does not specify the behavior of negative shifts and
of left shifts larger or equal than the width of the promoted left operand.
The exact behavior is compiler/machine specific.
This Python bitarray library specifies the behavior as follows:the length of the bitarray is never changed by any shift operationblanks are filled by 0negative shifts raiseValueErrorshifts larger or equal to the length of the bitarray result in
bitarrays with all values 0Bit endiannessUnless explicitly converting to machine representation, using
the.tobytes(),.frombytes(),.tofile()and.fromfile()methods, as well as usingmemoryview, the bit endianness will have no
effect on any computation, and one can skip this section.Since bitarrays allows addressing individual bits, where the machine
represents 8 bits in one byte, there are two obvious choices for this
mapping: little-endian and big-endian.When dealing with the machine representation of bitarray objects, it is
recommended to always explicitly specify the endianness.By default, bitarrays use big-endian representation:>>>a=bitarray()>>>a.endian()'big'>>>a.frombytes(b'A')>>>abitarray('01000001')>>>a[6]=1>>>a.tobytes()b'C'Big-endian means that the most-significant bit comes first.
Here,a[0]is the lowest address (index) and most significant bit,
anda[7]is the highest address and least significant bit.When creating a new bitarray object, the endianness can always be
specified explicitly:>>>a=bitarray(endian='little')>>>a.frombytes(b'A')>>>abitarray('10000010')>>>a.endian()'little'Here, the low-bit comes first because little-endian means that increasing
numeric significance corresponds to an increasing address.
Soa[0]is the lowest address and least significant bit,
anda[7]is the highest address and most significant bit.The bit endianness is a property of the bitarray object.
The endianness cannot be changed once a bitarray object is created.
When comparing bitarray objects, the endianness (and hence the machine
representation) is irrelevant; what matters is the mapping from indices
to bits:>>>bitarray('11001',endian='big')==bitarray('11001',endian='little')TrueBitwise operations (|,^,&=,|=,^=,~) are
implemented efficiently using the corresponding byte operations in C, i.e. the
operators act on the machine representation of the bitarray objects.
Therefore, it is not possible to perform bitwise operators on bitarrays
with different endianness.When converting to and from machine representation, using
the.tobytes(),.frombytes(),.tofile()and.fromfile()methods, the endianness matters:>>>a=bitarray(endian='little')>>>a.frombytes(b'\x01')>>>abitarray('10000000')>>>b=bitarray(endian='big')>>>b.frombytes(b'\x80')>>>bbitarray('10000000')>>>a==bTrue>>>a.tobytes()==b.tobytes()FalseAs mentioned above, the endianness can not be changed once an object is
created.  However, you can create a new bitarray with different endianness:>>>a=bitarray('111000',endian='little')>>>b=bitarray(a,endian='big')>>>bbitarray('111000')>>>a==bTrueBuffer protocolBitarray objects support the buffer protocol.  They can both export their
own buffer, as well as import another object’s buffer.  To learn more about
this topic, please readbuffer protocol.  There is also an example that shows how
to memory-map a file to a bitarray:mmapped-file.pyVariable bit length prefix codesThe.encode()method takes a dictionary mapping symbols to bitarrays
and an iterable, and extends the bitarray object with the encoded symbols
found while iterating.  For example:>>>d={'H':bitarray('111'),'e':bitarray('0'),...'l':bitarray('110'),'o':bitarray('10')}...>>>a=bitarray()>>>a.encode(d,'Hello')>>>abitarray('111011011010')Note that the string'Hello'is an iterable, but the symbols are not
limited to characters, in fact any immutable Python object can be a symbol.
Taking the same dictionary, we can apply the.decode()method which will
return a list of the symbols:>>>a.decode(d)['H','e','l','l','o']>>>''.join(a.decode(d))'Hello'Since symbols are not limited to being characters, it is necessary to return
them as elements of a list, rather than simply returning the joined string.
The above dictionarydcan be efficiently constructed using the functionbitarray.util.huffman_code().  I also wroteHuffman coding in Python
using bitarrayfor more
background information.When the codes are large, and you have many decode calls, most time will
be spent creating the (same) internal decode tree objects.  In this case,
it will be much faster to create adecodetreeobject, which can be
passed to bitarray’s.decode()and.iterdecode()methods, instead
of passing the prefix code dictionary to those methods itself:>>>frombitarrayimportbitarray,decodetree>>>t=decodetree({'a':bitarray('0'),'b':bitarray('1')})>>>a=bitarray('0110')>>>a.decode(t)['a','b','b','a']>>>''.join(a.iterdecode(t))'abba'Thedecodetreeobject is immutable and unhashable, and it’s sole purpose
is to be passed to bitarray’s.decode()and.iterdecode()methods.FrozenbitarraysAfrozenbitarrayobject is very similar to the bitarray object.
The difference is that this afrozenbitarrayis immutable, and hashable,
and can therefore be used as a dictionary key:>>>frombitarrayimportfrozenbitarray>>>key=frozenbitarray('1100011')>>>{key:'some value'}{frozenbitarray('1100011'):'some value'}>>>key[3]=1Traceback(mostrecentcalllast):...TypeError:frozenbitarrayisimmutableReferencebitarray version: 2.3.7 –change logIn the following,itemandvalueare usually a single bit -
an integer 0 or 1.The bitarray object:bitarray(initializer=0, /,endian='big',buffer=None)-> bitarrayReturn a new bitarray object whose items are bits initialized from
the optional initial object, and endianness.
The initializer may be of the following types:int: Create a bitarray of given integer length.  The initial values are
uninitialized.str: Create bitarray from a string of0and1.iterable: Create bitarray from iterable or sequence or integers 0 or 1.Optional keyword arguments:endian: Specifies the bit endianness of the created bitarray object.
Allowed values arebigandlittle(the default isbig).
The bit endianness effects the buffer representation of the bitarray.buffer: Any object which exposes a buffer.  When provided,initializercannot be present (or has to beNone).  The imported buffer may be
readonly or writable, depending on the object type.New in version 2.3: optionalbufferargument.A bitarray object supports the following methods:all()-> boolReturn True when all bits in the array are True.
Note thata.all()is faster thanall(a).any()-> boolReturn True when any bit in the array is True.
Note thata.any()is faster thanany(a).append(item, /)Appenditemto the end of the bitarray.buffer_info()-> tupleReturn a tuple containing:memory address of bufferbuffer size (in bytes)bit endianness as a stringnumber of unused padding bitsallocated memory for the buffer (in bytes)memory is read-onlybuffer is importednumber of buffer exportsbytereverse(start=0,stop=<endof buffer>, /)Reverse the bit order for the bytes in range(start, stop) in-place.
The start and stop indices are given in terms of bytes (not bits).
By default, all bytes in the buffer are reversed.
Note: This method only changes the buffer; it does not change the
endianness of the bitarray object.New in version 2.2.5: optional start and stop arguments.clear()Remove all items from the bitarray.New in version 1.4.copy()-> bitarrayReturn a copy of the bitarray.count(value=1, start=0,stop=<endof array>, step=1, /)-> intCount the number of occurrences ofvaluein the bitarray.New in version 1.1.0: optional start and stop arguments.New in version 2.3.7: optional step argument.decode(code, /)-> listGiven a prefix code (a dict mapping symbols to bitarrays, ordecodetreeobject), decode the content of the bitarray and return it as a list of
symbols.encode(code, iterable, /)Given a prefix code (a dict mapping symbols to bitarrays),
iterate over the iterable object with symbols, and extend the bitarray
with the corresponding bitarray for each symbol.endian()-> strReturn the bit endianness of the bitarray as a string (littleorbig).extend(iterable, /)Append all items fromiterableto the end of the bitarray.
If the iterable is a string, each0and1are appended as
bits (ignoring whitespace and underscore).fill()-> intAdd zeros to the end of the bitarray, such that the length of the bitarray
will be a multiple of 8, and return the number of bits added (0..7).find(sub_bitarray, start=0,stop=<endof array>, /)-> intReturn the lowest index where sub_bitarray is found, such that sub_bitarray
is contained within[start:stop].
Return -1 when sub_bitarray is not found.New in version 2.1.frombytes(bytes, /)Extend bitarray with raw bytes.  That is, each append byte will add eight
bits to the bitarray.fromfile(f,n=-1,/)Extend bitarray with up to n bytes read from the file object f.
When n is omitted or negative, reads all data until EOF.
When n is provided and positive but exceeds the data available,EOFErroris raised (but the available data is still read and appended.index(sub_bitarray, start=0,stop=<endof array>, /)-> intReturn the lowest index where sub_bitarray is found, such that sub_bitarray
is contained within[start:stop].
RaisesValueErrorwhen the sub_bitarray is not present.insert(index, value, /)Insertvalueinto the bitarray beforeindex.invert(index=<allbits>, /)Invert all bits in the array (in-place).
When the optionalindexis given, only invert the single bit at index.New in version 1.5.3: optional index argument.iterdecode(code, /)-> iteratorGiven a prefix code (a dict mapping symbols to bitarrays, ordecodetreeobject), decode the content of the bitarray and return an iterator over
the symbols.itersearch(sub_bitarray, /)-> iteratorSearches for the given sub_bitarray in self, and return an iterator over
the start positions where bitarray matches self.pack(bytes, /)Extend the bitarray from bytes, where each byte corresponds to a single
bit.  The byteb'\x00'maps to bit 0 and all other characters map to
bit 1.
This method, as well as the unpack method, are meant for efficient
transfer of data between bitarray objects to other python objects
(for example NumPy’s ndarray object) which have a different memory view.pop(index=-1,/)-> itemReturn the i-th (default last) element and delete it from the bitarray.
RaisesIndexErrorif bitarray is empty or index is out of range.remove(value, /)Remove the first occurrence ofvaluein the bitarray.
RaisesValueErrorif item is not present.reverse()Reverse all bits in the array (in-place).search(sub_bitarray,limit=<none>,/)-> listSearches for the given sub_bitarray in self, and return the list of start
positions.
The optional argument limits the number of search results to the integer
specified.  By default, all search results are returned.setall(value, /)Set all elements in the bitarray tovalue.
Note thata.setall(value)is equivalent toa[:]= value.sort(reverse=False)Sort the bits in the array (in-place).to01()-> strReturn a string containing ‘0’s and ‘1’s, representing the bits in the
bitarray.tobytes()-> bytesReturn the byte representation of the bitarray.tofile(f, /)Write the byte representation of the bitarray to the file object f.tolist()-> listReturn a list with the items (0 or 1) in the bitarray.
Note that the list object being created will require 32 or 64 times more
memory (depending on the machine architecture) than the bitarray object,
which may cause a memory error if the bitarray is very large.unpack(zero=b'\x00',one=b'\x01')-> bytesReturn bytes containing one character for each bit in the bitarray,
using the specified mapping.Other objects:frozenbitarray(initializer=0, /,endian='big',buffer=None)-> frozenbitarrayReturn a frozenbitarray object, which is initialized the same way a bitarray
object is initialized.  A frozenbitarray is immutable and hashable.
Its contents cannot be altered after it is created; however, it can be used
as a dictionary key.New in version 1.1.decodetree(code, /)-> decodetreeGiven a prefix code (a dict mapping symbols to bitarrays),
create a binary tree object to be passed to.decode()or.iterdecode().New in version 1.6.Functions defined in thebitarraymodule:bits2bytes(n, /)-> intReturn the number of bytes necessary to store n bits.get_default_endian()-> stringReturn the default endianness for new bitarray objects being created.
Unless_set_default_endian()is called, the return value isbig.New in version 1.3.test(verbosity=1, repeat=1)-> TextTestResultRun self-test, and return unittest.runner.TextTestResult object.Functions defined inbitarray.utilmodule:This sub-module was add in version 1.2.zeros(length, /, endian=None)-> bitarrayCreate a bitarray of length, with all values 0, and optional
endianness, which may be ‘big’, ‘little’.urandom(length, /, endian=None)-> bitarrayReturn a bitarray oflengthrandom bits (usesos.urandom).New in version 1.7.pprint(bitarray, /, stream=None, group=8, indent=4, width=80)Prints the formatted representation of object onstream(which defaults
tosys.stdout).  By default, elements are grouped in bytes (8 elements),
and 8 bytes (64 elements) per line.
Non-bitarray objects are printed by the standard library
functionpprint.pprint().New in version 1.8.make_endian(bitarray, /, endian)-> bitarrayWhen the endianness of the given bitarray is different fromendian,
return a new bitarray, with endiannessendianand the same elements
as the original bitarray.
Otherwise (endianness is alreadyendian) the original bitarray is returned
unchanged.New in version 1.3.rindex(bitarray, value=1, start=0,stop=<endof array>, /)-> intReturn the rightmost (highest) index ofvaluein bitarray.
RaisesValueErrorif the value is not present.New in version 2.3.0: optional start and stop arguments.strip(bitarray, /,mode='right')-> bitarrayReturn a new bitarray with zeros stripped from left, right or both ends.
Allowed values for mode are the strings:left,right,bothcount_n(a, n, value=1, /)-> intReturn lowest indexifor whicha[:i].count(value)== n.
RaisesValueError, when n exceeds total count (a.count(value)).New in version 2.3.6: optional value argument.parity(a, /)-> intReturn the parity of bitarraya.
This is equivalent toa.count() % 2(but more efficient).New in version 1.9.count_and(a, b, /)-> intReturn(a &b).count()in a memory efficient manner,
as no intermediate bitarray object gets created.count_or(a, b, /)-> intReturn(a |b).count()in a memory efficient manner,
as no intermediate bitarray object gets created.count_xor(a, b, /)-> intReturn(a ^b).count()in a memory efficient manner,
as no intermediate bitarray object gets created.subset(a, b, /)-> boolReturnTrueif bitarrayais a subset of bitarrayb.subset(a, b)is equivalent to(a &b).count()== a.count()but is more
efficient since we can stop as soon as one mismatch is found, and no
intermediate bitarray object gets created.ba2hex(bitarray, /)-> hexstrReturn a string containing the hexadecimal representation of
the bitarray (which has to be multiple of 4 in length).hex2ba(hexstr, /, endian=None)-> bitarrayBitarray of hexadecimal representation.  hexstr may contain any number
(including odd numbers) of hex digits (upper or lower case).ba2base(n, bitarray, /)-> strReturn a string containing the basenASCII representation of
the bitarray.  Allowed values fornare 2, 4, 8, 16, 32 and 64.
The bitarray has to be multiple of length 1, 2, 3, 4, 5 or 6 respectively.
Forn=16(hexadecimal),ba2hex()will be much faster, asba2base()does not take advantage of byte level operations.
Forn=32the RFC 4648 Base32 alphabet is used, and forn=64the
standard base 64 alphabet is used.See also:Bitarray representationsNew in version 1.9.base2ba(n, asciistr, /, endian=None)-> bitarrayBitarray of the basenASCII representation.
Allowed values fornare 2, 4, 8, 16, 32 and 64.
Forn=16(hexadecimal),hex2ba()will be much faster, asbase2ba()does not take advantage of byte level operations.
Forn=32the RFC 4648 Base32 alphabet is used, and forn=64the
standard base 64 alphabet is used.See also:Bitarray representationsNew in version 1.9.ba2int(bitarray, /, signed=False)-> intConvert the given bitarray to an integer.
The bit-endianness of the bitarray is respected.signedindicates whether two’s complement is used to represent the integer.int2ba(int, /, length=None, endian=None, signed=False)-> bitarrayConvert the given integer to a bitarray (with given endianness,
and no leading (big-endian) / trailing (little-endian) zeros), unless
thelengthof the bitarray is provided.  AnOverflowErroris raised
if the integer is not representable with the given number of bits.signeddetermines whether two’s complement is used to represent the integer,
and requireslengthto be provided.serialize(bitarray, /)-> bytesReturn a serialized representation of the bitarray, which may be passed todeserialize().  It efficiently represents the bitarray object (including
its endianness) and is guaranteed not to change in future releases.See also:Bitarray representationsNew in version 1.8.deserialize(bytes, /)-> bitarrayReturn a bitarray given the bytes representation returned byserialize().See also:Bitarray representationsNew in version 1.8.vl_encode(bitarray, /)-> bytesReturn variable length binary representation of bitarray.
This representation is useful for efficiently storing small bitarray
in a binary stream.  Usevl_decode()for decoding.See also:Variable length bitarray formatNew in version 2.2.vl_decode(stream, /, endian=None)-> bitarrayDecode binary stream (an integer iterator, or bytes object), and return
the decoded bitarray.  This function consumes only one bitarray and leaves
the remaining stream untouched.StopIterationis raised when no
terminating byte is found.
Usevl_encode()for encoding.See also:Variable length bitarray formatNew in version 2.2.huffman_code(dict, /, endian=None)-> dictGiven a frequency map, a dictionary mapping symbols to their frequency,
calculate the Huffman code, i.e. a dict mapping those symbols to
bitarrays (with given endianness).  Note that the symbols are not limited
to being strings.  Symbols may may be any hashable object (such asNone).