Package: python-xz
python-xzPure Python implementation of the XZ file format with random access supportLeveraging the lzma module for fast (de)compression📖 Documentation|📃 ChangelogA XZ file can be composed of several streams and blocks. This allows for fast random
access when reading, but this is not supported by Python's builtinlzmamodule (which
would read all previous blocks for nothing).lzmalzmaffipython-xzmodule typebuiltincffi (C extension)pure Python📄readrandom access❌ no1✔️ yes2✔️ yes2several blocks✔️ yes✔️✔️ yes3✔️✔️ yes3several streams✔️ yes✔️ yes✔️✔️ yes4stream padding❌ no5✔️ yes✔️ yes📝writewmode✔️ yes✔️ yes✔️ yesxmode✔️ yes❌ no✔️ yesamode✔️ new stream✔️ new stream⏳ plannedr+/w+/… modes❌ no❌ no✔️ yesseveral blocks❌ no❌ no✔️ yesseveral streams❌ no6❌ no6✔️ yesstream padding❌ no❌ no⏳ plannedNotesReading from a position will read the file from the very beginningReading from a position will read the file from the beginning of the blockBlock positions available with theblock_boundariesattributeStream positions available with thestream_boundariesattributeRelatedissuePossible by manually closing and re-opening in append modeInstallInstallpython-xzwith pip:$python-mpipinstallpython-xzAn unofficial package for conda isalso available, seeissue #5for more information.UsageThe API is similar tolzma: you can use eitherxz.openorxz.XZFile.Read mode>>>withxz.open('example.xz')asfin:...fin.read(18)...fin.stream_boundaries# 2 streams...fin.block_boundaries# 4 blocks in first stream, 2 blocks in second stream...fin.seek(1000)...fin.read(31)...b'Hello, world!\xf0\x9f\x91\x8b'[0,2000][0,500,1000,1500,2000,3000]1000b'\xe2\x9c\xa8Random access is fast!\xf0\x9f\x9a\x80'Opening in text mode works as well, but notice that seek arguments as well as boundaries
are still in bytes (just like withlzma.open).>>>withxz.open('example.xz','rt')asfin:...fin.read(15)...fin.stream_boundaries...fin.block_boundaries...fin.seek(1000)...fin.read(26)...'Hello, world! 👋'[0,2000][0,500,1000,1500,2000,3000]1000'✨ Random access is fast! 🚀'Write modeWriting is only supported from the end of file. It is however possible to truncate the
file first. Note that truncating is only supported on block boundaries.>>>withxz.open('test.xz','w')asfout:...fout.write(b'Hello, world!\n')...fout.write(b'This sentence is still in the previous block\n')...fout.change_block()...fout.write(b'But this one is in its own!\n')...144528Advanced usage:Modes liker+/w+/x+allow to open for both read and write at the same time;
however in the current implementation, a block with writing in progress is
automatically closed when reading data from it.Thecheck,presetandfiltersarguments toxz.openandxz.XZFileallow to
configure the default values for new streams and blocks.Change block with thechange_blockmethod (thepresetandfiltersattributes can
be changed beforehand to apply to the new block).Change stream with thechange_streammethod (thecheckattribute can be changed
beforehand to apply to the new stream).FAQHow does random-access works?XZ files are made of a number of streams, and each stream is composed of a number of
block. This can be seen withxz --list:$xz--listfile.xz
StrmsBlocksCompressedUncompressedRatioCheckFilename11316.8MiB297.9MiB0.056CRC64file.xzTo read data from the middle of the 10th block, we will decompress the 10th block from
its start it until we reach the middle (and drop that decompressed data), then returned
the decompressed data from that point.Choosing the good block size is a tradeoff between seeking time during random access and
compression ratio.How can I create XZ files optimized for random-access?You can open the file for writing and use thechange_blockmethod to create several
blocks.Other tools allow to create XZ files with several blocks as well:XZ Utilsneeds to be called with flags:$xz-T0file# threading mode$xz--block-size16Mfile# same size for all blocks$xz--block-list16M,32M,8M,42Mfile# specific size for each blockPIXZcreates files with several blocks by default:$pixzfilePython version supportAs a general rule, all Python versions that are bothreleased and still officially
supportedare supported bypython-xzand tested against (both
CPython and PyPy implementations).If you have other use cases or find issues with some Python versions, feel free toopen a ticket!