Package: zipfile-deflate64
zipfile-deflate64Extract Deflate64 ZIP archives with Python'szipfileAPI.Installationpipinstallzipfile-deflate64Python 3.6, 3.7, 3.8, 3.9, and 3.10 are supported,
withmanylinux2014, macOS and Windows wheels published to PyPI.UsageAnywhere in a Python codebase:importzipfile_deflate64# This has the side effect of patching the zipfile module to support Deflate64Alternatively,zipfile_deflate64re-exports thezipfileAPI, as a convenience:importzipfile_deflate64aszipfilezipfile.ZipFile(...)...Design RationaleThe ProblemRecent versions of Microsoft Windows Exploreruse Deflate64 compression when creating ZIP files larger than 2GB.
With the ubiquity of Windows and the ease of using "Sent to compressed folder", a majority of newly-created large
ZIP files use Deflate64 compression.However,support for Deflate64 in the open-source ecosystem is very poor!
Most ZIP libraries have declined to implement Deflate64,
citingits proprietary nature.In the .NET ecosystem, theZipArchiveAPI supports decompression only.
In Java, theApache Commons Compress APIs support both compression and decompression.The 7-Zip project probably provides the best general-purpose support for compressing and decompressing
Deflate64, but there are several obstacles to general usability:7-Zip itselfis a Windows-only GUI application7-Zip is still issuing new releases, but has declined to implement certain new compression formats,
so themcmilk/7-Zip-zstdfork is notable.p7zip, the POSIX-compatible CLI version(which does include Deflate64),has not had a release since 2016 and is likely unmaintained.p7zip does not build an API for external software to invoke for decompression.p7zip seems to now be living on as thejinfeihan57/p7zipfork,
which is packaged by Arch Linux, amongst others.This seems to be active, and now can be built with CMake, but there's no support for building an external API.Many re-implementations of 7-Zip, such aspy7zrfor Python, do not support
Deflate64.In the Python ecosystem in particular, there have been several unfulfilled requests ([1][2][3]) for Deflate64 decompression support.A SolutionThe best hope seems to be theinfback9extension
to zlib. This was developed in 2003 by Mark Adler, an original author of zlib, and is kept in the source repository
of zlib, but it is not officially supported and contains no build tooling and is not distributed with zlib packages.
Additionally, infback9 provides only low-level support for working with Deflate64 bitstreams, with no support for
the ZIP archive format (which is out of scope for zlib).infback9's C-language API is relatively simple, but requires a non-trivial struct and function pointers for
initialization and some explicit memory management operations (resizing allocated buffers and proving a
Python-friendlymalloc) to operate efficiently, so wrapping it with onlyctypesseems to be inadequate.To manage ZIP archive extraction operations, the Python standard libraryzipfilemodule provides the essential features and is already
ubiquitous in availability and usage. However, zipfile is difficult to extend, as it hardcodes many conditionals for
compression formats and does not provide capabilities for easily augmenting or replacing parts of it. Monkey-patching
can overcome some of these problems, and the promise of a drop-in, API-compatible patch to a standard library module
outweighed the engineering benefits of basing a solution off a more naturally extensible third-party ZIP manipulation
package.