Package: huffmanfile
libhuffman - The Huffman coding libraryThe Huffman library is a simple, pure C library for encoding and decoding data using a
frequency-sorted binary tree.
The implementation of this library is pretty straightforward, additional information
regarding Huffman coding could be gained from theWikipedia.InstallationThe build mechanism of the library is based on theCMaketool, so
you could easily install it on your distribution in the following way:$sudocmakeinstallBy default the command above install the library into/usr/local/liband all
required headers into/usr/local/include. The installation process is organized
using CMake. Just create a new directorybuildand generate required makefiles:$mkdir-pbuild
$cmake..After that run theinstalltarget:$makeinstallUsageEncodingTo encode the data, use either a file streamhuf_fdopenorhuf_memopento use
an in-memory stream. Consider the following example, where the input is a memory
stream and output of the encoder is also memory buffer of 1MiB size.void*bufin,*bufout=NULL;huf_read_writer_t*input,*output=NULL;// Allocate the necessary memory.huf_memopen(&input,&bufin,HUF_1MIB_BUFFER);huf_memopen(&output,&bufout,HUF_1MIB_BUFFER);// Write the data for encoding to the input.size_tinput_len=10;input->write(input->stream,"0123456789",input_len);Create a configuration used to encode the input string using Huffman algorithm:huf_config_tconfig={.reader=input,.length=input_len,.writer=output,.blocksize=HUF_64KIB_BUFFER,};huf_error_terr=huf_encode(&config);printf("%s\n",huf_error_string(err));reader- input ready for the encoding.writer- output for the encoded data.length- length of the data in bytes to encode.blocksize- the length of each chunk in bytes (instead of reading the file twicelibhuffmanreads and encodes data by blocks).reader_buffer_size- this is opaque reader buffer size in bytes, if the buffer size
is set to zero, all reads will be unbuffered.writer_buffer_size- this is opaque writer buffer size ib bytes, if the buffer size
is set to zero, all writes will be unbuffered.After the encoding, the output memory buffer could be automatically scaled to fit all
necessary encoded bytes. To retrieve a new length of the buffer, use the following:size_tout_size=0;huf_memlen(output,&out_size);// The data is accessible through the `bufout` variable or using `read` function:uint8_tresult[10]={0};size_tresult_len=10;// result_len is inout parameter, and will contain the length of encoding// after the reading from the stream.output->read(output->stream,result,&result_len);DecodingDecoding is similar to the encoding, except that reader attribute of the configuration
should contain the data used to decode:input->write(input->stream,decoding,decoding_len);huf_config_tconfig={.reader=input,.length=input_len,.writer=output,.blockize=HUF_64KIB_BUFFER,};// After the decoding the original data will be writter to the `output`.huf_decode(&config);Resource DeallocationOnce the processing of the encoding is completed, consider freeing the allocated memory:// This does not free underlying buffer, call free for the buffer.huf_memclose(&mem_out);free(buf);For more examples, please, refer to thetestsdirectory.Python BindingsPython bindings forlibhuffmanlibrary are distributed as PyPI package, to install
that package, execute the following command:pipinstallhuffmanfileYou can use thelibhuffmanfor performant compression and decompression of Huffman
encoding. The interface of the Python library is similar to the interface of thebz2andlzmapackages from Python's standard
library.Examples of usageReading in a compressed file:importhuffmanfilewithhuffmanfile.open("file.hm")asf:file_content=f.read()Creating a compressed file:importhuffmanfiledata=b"Insert Data Here"withhuffmanfile.open("file.hm","w")asf:f.write(data)Compressing data in memory:importhuffmanfiledata_in=b"Insert Data Here"data_out=huffmanfile.compress(data_in)Incremental compression:importhuffmanfilehfc=huffmanfile.HuffmanCompressor()out1=hfc.compress(b"Some data\n")out2=hfc.compress(b"Another piece of data\n")out3=hfc.compress(b"Even more data\n")out4=hfc.flush()# Concatenate all the partial results:result=b"".join([out1,out2,out3,out4])Note, random data tends to compress poorly, while ordered, repetitive data usually
yields a high compression ratio.LicenseThe Huffman library is distributed under MIT license, therefore you are free to do with
code whatever you want. See theLICENSEfile for full license text.