Package: python-zim
pyzim - a python package for working with ZIM filesNote:pyzimis published on PyPI aspython-zimdue to a naming conflict with an existing package.pyzimis a semi-pure python package for working with ZIM files. A ZIM file is basically a very highly compressed archive of a website. Examples for ZIM files include offline versions of wikipedia, stackoverflow, project gutenberg and many more.pyzimaims to provide a very flexible and open method of interacting with ZIM files. For example, this project aims to give developers the choice whether they want access entries in a ZIM file as fast as possible or with as little RAM usage as possible.pyzimitself is written in pure python and does not depend onlibzim. However, modern ZIM files use zstandard compression andpyzimdepends on a C library for working with such files.Featurespyzimis nearly fully implemented. It supports nearly all reader featurs and youshouldbe able to read all modern ZIM files, but some features (like search) are still missing. A writer also exists and is even capable of editing existing ZIM files.Basic features:Most read and write operations on ZIM files are implemented.Read and write ZIM filesall compression types are supported (at least at the time this document is being written)Access header informations and metadataaccess clusters and entries directlyiterate over entries and clustersedit existig ZIM files (add new entries, remove entries, edit them and clusters)a space allocation algorithm tries to recycle unused space in a ZIM file when it is being edited.work with ZIM files at a specified offset (untested)Missing features:The following features are still missing, but planned:search functionssimple illustration methods (you can already read metadata illustrations, but you will have to convert them to PIL images manually)various additional CLI toolssupport for ZIM files without namespacesAdditional features:In addition to regular ZIM functionality, the following features are also implemented:configurable caching of entries and clusters for better performancevarious alternative implementations of clusters for better performance at the cost of RAMa policy system to manage resource allocation behavior (e.g. use a policy to reduce RAM usage as much as possible at the cost of access speed)ZIM editing.General project features:extensive API documentation (but not yet hosted online)extensive software tests (branch-coverage of 98% at the time of writing)examples are providedInstallationpyzimis published on PyPI aspython-zimdue to a naming conflict with an existing package.Via pip from PyPITo install viapip, runpip install python-zim. Alternatively, runpip install python-zim[all]to install all additional dependencies (like compression and testing libraries).Here is a full ist of supported extra dependencies (usage:pip install python-zim[<extra>]):all:all extra dependencies.compression:compression related dependencies.testing:testing related dependencies. Please note thattoxwill install further dependencies during testing.From sourceDownload the source code using git:git clone https://github.com/IMayBeABitShy/pyzim.gitcdinto directory:cd pyzimInstall usingpip:pip install .[compression,testing]. See above for the meaning of the extras specified. You may have to usepython3 -m pipinstead and/or specify--user.ExamplePlease take a look at theexamples/directory for fully commented examples.# read a specific file from the ZIMimportargparseimportpyzimwithpyzim.Zim.open(zimpath)aszim:entry=zim.get_content_entry_by_url(entrypath)entry=entry.resolve()print("URL: ",entry.url)print("Full URL: ",entry.full_url)print("Redirect: ",entry.is_redirect)print("Title: ",entry.title)print("Mimetype: ",entry.mimetype)print("Content location:{}@{}".format(entry.blob_number,entry.cluster_number))print("\n\n=====CONTENT=====\n\n")print(entry.read())Documentationpyzimis extensively documented usingpydoctor. There is currently no online version of the documentation, but you can build it locally by runningtox -e docsin the project directory, which will output HTML documentation tohtml/apidocs/. This requirestoxto be installed.If you are a contributor looking to write you own documentation, you can find a pydoctor syntax guidehere.TestingAt the time of writing this document,pyzimachieves a (statement-based) test coverage of 98%. You can run the tests locally by executingtoxin the project directory. Specify thetestingextra during installation ofpyzimto automatically install all test dependencies.pyzimlogs a lot of low-level operations at numeric values below theDEBUGlevel. For example, each entry being read is logged, but normally aren't shown. See the documentation ofpyzim.constantsfor these log levels. Editingtox.iniand changing the log level may be helpful when debugging.FAQ**Why do I get anUnsupportedCompressionTypeexception with a ZIM file?pyzimdepends on other libraries to handle the decompression of data from the ZIM file. Luckily, the vast majority of these libraries come included with most python distributions. Unfortunately, these libraries may not be included when you build python yourself. Additionally, the most common compression in modern ZIM files iszstandard, for whichpyzimdepends onpyzstd. Please ensure that this library is installed.You can automatically install all optional compression dependencies by installing thecompressionextra forpyzim.**Why do I get aBindRequiredexception / what does "bound/unbound" mean?pyzimdifferentiates betweenboundandunboundentries/clusters/... . Anunboundobject is an object that is not attached to any ZIM object. By default, most objects should be automatically bound by the various methods for accessing them, but if you are accessing any class directly you may encounter unbound ones.You can bind any such objects by calling their.bind(zim_object)method.The idea behind this behavior is that we should be able to use the same code for readers and writers.See alsoThe following section lists various other resources related to ZIM files, which may be of interest to you. This includes enduser applications, alternative libraries, documentation and more. These lists are by no means exclusive.ZIM programming libraries and documentationlibzim: the reference implementation of the ZIM file formatpython-libzim: python bindings forlibzim, if you want to use another ZIM moduleZIM file format specification: for details about the ZIM file formatZIM filesThe kiwix Library: A library of ZIM files provided by the kiwix project. It also allows you to browse ZIM files directly.ZIM viewers (For endusers)The kiwix website: Kiwix provides a wide range of ZIM viewers. Desktop and mobikle Apps exist.kiwix-js, also available as aPWA: A ZIM browser implemented in javascript for webbrowsers, available as extensions and as a PWA.kiwix-tools:kiwix-toolscontainskiwix-serve, a dedicated HTTP-Server for ZIM files.