Package: dotlock
Fast and safe dependency management for Python applications.Supports Linux and OS X.MotivationThe de-facto standard way to keep track of Python dependencies is with arequirements.txtfile,
listing the required packages and specifying what versions of them can be used.
There are two strategies for specifying versions in arequirements.txtfile:
adding only the top-level dependencies and constraints you know to be necessary,
or adding every recursive dependency and pinning them to specific versions you know work.
The first strategy makes installing dependencies non-repeatable.
The second makes upgrading difficult, and is hard to manage with standard python tools.Dotlock enables you to do both: keep track of top-level requirements and known constraints
inpackage.json, and generate repeatable requirement sets inpackage.lock.jsonby running a single command:dotlock lock.Dotlock is partly inspired bypipenv, which also provides
dependency-locking functionality. However, dotlock has a different philosophy: instead of acting as a wrapper
around pip, dotlock handles package resolution natively. This gives dotlock more flexibility, better performance,
and a smaller surface area for bugs, but at the cost of the wide platform and package support the pip developers have
put so much work into. This should make dotlock better for deploying to common environments such as linux servers
with common packages, but there will always be edge cases of platforms or packages it does not support.Dotlock’s main goals are:Accuracy:dotlock locklocks to the level of specific distributions, not just versions.
That means you know exactly what code will be installed bydotlock install, andpackage.lock.jsonwill contain a single hash for each package. You can even vendor your dependencies withdotlock bundle.Speed:dotlock lockuses caching and asyncio to re-lock after changes topackage.lockin 1s or less.
Similarly,dotlock installanddotlock bundledownload dependencies in parallel.Extras Support: Unlike pipenv which only supports “default” dependencies and “dev” dependencies,dotlocksupports arbitrary extra dependency groups, e.g.dotlock install--extrastests.InstallationDotlock can be installed withpip, i.e.pip install dotlock.
It can be installed in an application’s virtual environment, at the user level, or globally.Development SetupOn your development machine, rundotlock initto create a virtualenv and a skeletonpackage.lockfile.
Add your sources and dependencies topackage.lock:{"sources":[// PyPI-like package index hosting the dependencies.// If multiple indexes are included, each is tried in order during dependency resolution."https://pypi.org/pypi"],"default":{// Requirements in the form "package-name": "specifier".// Version specifiers may be "*", or a version number preceded by any of <, <=, >, >=, or ==.// Multiple specifiers can be separated by commas, e.g. ">=2.1,<3.0"."setuptools":">=39.0","virtualenv":"*",// Git, Mercurial and Subversion dependencies are also supported."requests":"git+git://github.com/requests/requests@v2.19.1",// If you need extras or markers, supply a dictionary instead of a string."idna_ssl":{"specifier":"*","marker":"python_version < 3.7",// See PEP 496"extras":["tests"],},// Local file paths can be used too, but this loses integrity guarantees."mypackage":"~/projects/mypackage"},"extras":{// You can specify groups of additional dependencies that will be installed by// dotlock install --extras [names]"dev":{"ipython":"*"},"tests":{"pytest":"*"}}}Then you can lock and install your dependencies:dotlocklock# Creates package.lock.json.dotlockinstall# Installs exactly the distributions in package.lock.json.
# Either source venv/bin/activate to enter the virtualenv, or use dotlock run.dotlockrun[program][args]# Runs [program] in the virtualenv.For more information, rundotlock-hordotlock [command]-h.Developing in Different EnvironmentsIf your development environment differs significantly from your target deployed environment,
e.g. you use a different operating system or a different version of Python, you will have to
do some extra work and lose some of the benefits ofdotlock.In order to resolve dependencies and select distributions correctly,dotlockneeds to know
certain features of the deployed environment. Rundotlockdump-envon the deployed environment
to create anenv.jsonfile. This file should live alongside yourpackage.jsonfile, and
will be used bydotlock lock.Sincepackage.lock.jsoncontains only the distributions appropriate for your deployed environment,
runningdotlock installon an incompatible environment will error. Instead, you can rundotlock install--skip-lock, which will bypasspackage.lock.json, looking just atpackage.json.DeploymentThere are two ways to install your locked dependencies during deployment:Installdotlockand rundotlock installin the application root directory.Usedotlock bundleto createbundle.tar.gzandinstall.shprior to deployment,
include these files in the deployment, and run./install.shduring deployment.Usingdotlock bundleis preferred because it does not require installingdotlockin
the deployed environment and does not depend on external services during deploy.Once the dependencies are installed, run your application with one of:source venv/bin/activate; [program] [args]Assumingdotlockis installed:dotlock run [program] [args]Roadmap and LimitationsPlanned features:Interpolate environment variables insourcesAllow specifying indices for individual packagesFeatures under consideration:Support virtualenvs other than./venvSupport installing into system python (useful in certain contexts, e.g. containers)Support versions of Python before 3.6Support locking for other platforms. This is not possible to do with perfect reliability,
since the dependencies discovered by runningsetup.pymay differ depending on what
platform the script is run on.