Package: pyempaq
PyEmpaqA simple but powerful Python packer to run any project with any virtualenv dependencies anywhwere.With PyEmpaq you can convert any Python project (see limitations below) into a single.pyzfile with all the project's content packed inside.Packing is super simple, see this demo:That single file is everything that needs to be distributed. When the final user executes it, the original project will be expanded, its dependencies installed in a virtualenv, and then executed. Note that no special permissions or privileges are required, as everything happens in the user environment.See that in action:Both the packaging and the execution are fully multiplatorm. This means that you can pack a project in Linux, Windows, Mac or whatever, and it will run ok in Linux, Windows, Mac or whatever. The only requirement is Python to be already installed.You can try yourself some packed with PyEmpaq examples, very easy, just download any of these files and run it with Python:in a terminal: a very small pure terminal example (this, of course, needs to be run in a terminal)a game: a simple game using thePython Arcadelibrary (actually, it's theexample #6 from their tutorial)desktop app: a full-fledged desktop application using PyQt5 (thisEncuentro app)You can installpyempaqdirectly from PyPI; seeinstructions below.PyEmpaq is security friendly, there is nothing obscure or secretly shipped when you distribute your project with it: anybody can just open thepyzfile (it's just a ZIP) and inspect it.It's also safe regarding licenses when distributing your packed software. Unlike packing mechanisms that rely on putting inside a big blob, with PyEmpaq you don't have to worry about the licenses of your software dependencies (and the dependencies' dependencies) in regard to distributing them.Limitations:There are some limitations:Only Python >= 3.7 is supportedOnly Linux, Windows and Mac is supportedOnly pip-installable dependencies are supported (from PyPI or whatever).Only dependencies that are pure Python or provide wheels are supported.All this means that the most majority of the projects could be packed and run by PyEmpaq. If you have any ideas on how to overcome any of these limitations, let's talk!How Does this Work?There are two phases: packing and execution.Thepacking phaseis executed by the project developer, only once, before distribution. It's a simple step where the developer runs PyEmpaq indicating all needed info, and PyEmpaq will produce a single<projectname>.pyzfile. That's all, and that only file is what is needed for distribution.In this packing phase, PyEmpaq builds the indicated packed file, putting inside:the payload project, with all the indicated modules and binary filesanunpackerscript from PyEmpaq, which will be run during the execution phasefew more stuff: some needed infrastructure details for the.pyzto run correctlyAfter packing, the developer will distribute the packed file, final users will download/receive/get it, and execute it.In theexecution phaseall that needs to be done by the final user is to run it using Python, which can be done from the command line (e.g.python3 supergame.pyz) or by doing double click from the file explorer in those systems that relate the.pyzextension to Python (e.g. Windows).In this execution phase, theunpackerscript put by PyEmpaq inside the packed file will be run, running the following steps:check if the needed setup exists from a previous run; if yes, it will just run the payload project with almost no extra work; otherwise...create a directory in the user data dir (or the indicated one, see below), and expand the.pyzfile therecreate a virtualenv in that new directory, and install all the payload's project dependenciesrun the payload's project inside that virtualenvThe verification that the unpacker does to see if has a reusable setup from the past is based on the.pyztimestamp; if it changed (a new file was distributed), a new setup will be created and used.The environment variablePYEMPAQ_UNPACK_BASE_PATHcan be used to specify in which base directory PyEmpaq will unpack the different projects. If indicated, the path must exist and be a directory.PyEmpaq transparently returns the payload's exit code except when it must exit before the execution of the payload. In that case, it uses special codes equal or above 64, meaning:64:unpack-restrictionsare not met on the final user's system.65: the indicatedPYEMPAQ_UNPACK_BASE_PATHdoes not exist66: the indicatedPYEMPAQ_UNPACK_BASE_PATHis not a directory67: the indicatedPYEMPAQ_ACTIONis not validSpecial ActionsWhen running the packed project, some special actions can be indicated through the environment variablePYEMPAQ_ACTION. Note that if it is specified, the special action will take place and the packed program itself willnotbe run.Current special actions:info: Show information about installations of the given projectuninstall: Remove all installations of the given projectCommand Line OptionsThere is one mandatory parameter:source: it needs to point the configuration file or to the directory where the configuration will be located (it will be searched by its default namepyempaq.yaml).Examples:pyempaq .pyempaq /data/project/pyempaq ~/repo/proj/config.yamlYou can control verbosity by adding these command line parameters to control logging levels:-v, --verbose- Show detailed information, typically of interest only when diagnosing problems.-q, --quiet- Only events of WARNING level and above will be tracked.All that said, there is an special option-V, --version(new in v0.3) that if used will just print the version and exit.NoteIn theexecution phase, if you have an environment variablePYEMPAQ_DEBUG=1it will show the Pyempaq log lines during the execution.The Configuration FileAll the information that PyEmpaq needs to pack a project comes from a configuration file, which the developer would normally have in the project itself.The following is the structure of thepyempaq.yamlconfiguration file:name[mandatory]: the name of the project.basedir[optional]: the project's base directory; if not included it defaults to where the configuration file is located.exec[mandatory]: the section where is defined the information to execute the project once unpacked; it holds different subkeys (some subkeys are marked with †: ONE of those keys must be present, but only ONE):script[†]: the filepath of the python script to run; when unpacking PyEmpaq will dopython3 SCRIPT.module[†]: the name of the module to invoke; when unpacking PyEmpaq will dopython3 -m MODULE.entrypoint[†]: freeform, as a list of strings; when unpacking PyEmpaq will only insert the proper python3 at the beginning:python3 STR1 STR2 ....default-args[optional]: the default arguments to be passed to the script/module/entrypoint (if not overridden when the distributed.pyzis executed).requirements[optional]: a list of filepaths pointing to the requirement files with pip-installable dependencies.dependencies[optional]: a list of strings to directly specify packages to be installed bypipwithout needing to have a requirement file.include[optional, new in v0.3]: a list of strings, each one specifying a pattern to decide what files and directories to include in the pack (see below); if not included it defaults to["./**"]which means all the files and directories in the project.exclude[optional, new in v0.3]: a list of strings, each one specifying a pattern to decide what files and directories to exclude from the pack (see below).unpack-restrictions[optional, new in v0.3]: a section to specify different restrictions to be verified/enforced during unpack.minimum-python-version[optional, new in v0.3]: a string specifying the minimum version possible to run correctly.All specified filepaths must exist inside the project and must be relative (to the project's base directory), with the exception ofbasediritself which can be absolute or relative (to the configuration file location).Bothincludeandexcludeoptions use pattern matching according to the rules used by the Unix shell, using*,?, and character ranges expressed with[]. Also**will match any files and zero or more directories. For more information or subtleties check theglob.globdocumentation.Note that the final user may ignore/bypass any unpack restrictions using thePYEMPAQ_IGNORE_RESTRICTIONSenvironment variable with a value of comma-separated names of which restrictions to ignore.The following are examples of different configuration files (which were the ones used to build the packed examples mentioned before):the terminal scriptthe simple gamethe full desktop appContext of the Project Being RunWhen the packed script/module/whatever is run in the unpacking phase, it will be executed inside the virtualenv with the needed requirements/dependencies (freshly created and installed or reused from a previous run).Also, the following environment variable will be set:PYEMPAQ_PYZ_PATH: the absolute path to the.pyzrun by the userHow to Install PyEmpaqDirectly from PyPI:pip install --user --upgrade --ignore-installed pyempaqIt's handy to install it usingpipx, if you have it:pipx install pyempaqIf you havefadesyou don't even need to install pyempaq, just run it:fades -d pyempaq -x pyempaqIn the future there will be more ways to install it. Please open an issue if you desire an installation method (extra points if you specify how), thanks!Try Packing an Example ProjectPyEmpaq sources come with a small example project if you want to play a little packing it. Just a couple of dir/files underexamples/srcproject:asrcandmedia, with stuff to be imported and accessed.apyempaq.yamlwith the configuration for PyEmpaq.aep.pyfile which is the project's entrypoint; all it does is to inform it started, import the internal module, access the media files, and use the declared dependency, reporting every step.asecrets.txtfile that must not be included in the packed file.This explores most of the needs of any project. You can try this example, and surely after you will be ready to actually pack any other project you want.So, let's pack the example source project. As you're working with the PyEmpaq project itself (as you're packing its example), you don't really need to have it installed yet. In that case, if you havefadesinstalled is super easy:fades -r requirements.txt -m pyempaq examples/srcproject/Otherwise, you would need to create and use a virtual environment:python3 -m venv env
source env/bin/activate
pip install -r requirements
python3 -m pyempaq examples/srcproject/After running that command, you will see apyempaq-example.pyzfile. That is thewhole project encoded in a single file.At this point you may move thatpyempaq-example.pyzto another directory, or to another machine, even that other machine having another operating system.Then, try it:python3 pyempaq-example.pyzYou should see the project's reportings that we mentioned above (note: these lines will be surrounded by debug ones that will be hidden by default in the future):Hello world
Code access ok .../pyempaq/projectname-20210722013526/orig/src/foo.py
Media access ok
Module requests imported .../pyempaq/projectname-20210722013526/venv/lib/python3.8/site-packages/requests/__init__.pyThis shows that what you've run actually started, accessed the internal modules and other files, and imported correctly a custom-installed dependency.How PyEmpaq Relates to Other Similar Tools?There are other tools that are similar in their work. The more known areShivandPex.Shivwould create a fully self-contained Python zip app with all their dependencies included, which means that it ships not only your project but also the dependencies! this makes the resulting .pyz NOT multiplatform (unless your project and ALL your dependencies are pure Python...).On the other hand, PyEmpaq will create the virtualenv and install the needed dependencies the first time the .pyz is executed in the destination machine. This allows for example to pack a fully graphical desktop program that works with Qt, like the example shown above).A side detail of this is that we normally include dependencies in our programs, and those dependencies have sub-dependencies, etc. How sure are you that the whole set of dependencies and sub dependencies have a license that allows you to distribute them? Shiv will distribute them, PyEmpaq will not!Pexlooks like it also includes the dependencies in the resulting file, but goes an extra mile pointing to the Python that was used to build the thing. So for example I built a .pex with Py3.10 and moved the file to a machine that has only 3.9, and it didn’t work, and both were Ubuntus! No multiplatform at all, no even mention to pack in Linux (once!) and run in Windows (or everywhere), as PyEmpaq allows to do.These both tools are "ahead of time" dense (larger size) archives that include application code and dependencies for a single target platform. Hermetic and deterministic installation. Something that you would also get if yousnapyour project.On the other hand PyEmpaq is a sparse (smaller size) archive. It only includes the application code and at first-run will create and install the rest into a virtualenv. A bit like using pipx to install an application, except without needing to teach the end-user about pipx. It’s self installing. Non-hermetic and non-deterministic installation, but may not matter in practice.How to Contribute to the Project?Please checkthe how to contributeinstructions.