Package: conda-lock
conda-lockConda lock is a lightweight library that can be used to generate fully reproducible lock files forcondaenvironments.It does this by performing a conda solve for each platform you desire a lockfile for.This also has the added benefit of acting as an external pre-solve for conda as the lockfiles it generates
results in the conda solvernotbeing invoked when installing the packages from the generated lockfile.Why?Condaenvironment.ymlfiles are very useful for defining desired environments but there are times when we want to
be able to EXACTLY reproduce an environment by just installing and downloading the packages needed.This is particularly handy in the context of a gitops style setup where you use conda to provision environments in
various places.InstallationUseoneof the following commands:pipxinstallconda-lock
condaxinstallconda-lock
pipinstallconda-lock
condainstall--channel=conda-forge--name=baseconda-lock
mambainstall--channel=conda-forge--name=baseconda-lockThe first two options are recommended since they install conda-lock into an isolated environment. (Otherwise there is a risk of dependency conflicts.)ContributingIf you would like to contribute to conda-lock, please refer to theContributing Guidefor instructions on how to set up your development environment.Basic usage# generate a multi-platform lockfileconda-lock-fenvironment.yml-posx-64-plinux-64# optionally, update the previous solution, using the latest version of# pydantic that is compatible with the source specificationconda-lock--updatepydantic# create an environment from the lockfileconda-lockinstall[-p{prefix}|-n{name}]# alternatively, render a single-platform lockfile and use conda command directlyconda-lockrender-plinux-64
condacreate-nmy-locked-env--fileconda-linux-64.lockPre 1.0 compatible usage (explicit per platform locks)If you were making use of conda-lock before the 1.0 release that added unified lockfiles
you can still get that behaviour by making use of theexplicitoutput kind.conda-lock--kindexplicit-fenvironment.ymlAdvanced usageFile namingBy default,conda-lockstore its output inconda-lock.ymlin the current
working directory. This file will also be used by default for render, install,
and update operations. You can supply a different filename with e.g.conda-lock--lockfilesuperspecial.conda-lock.ymlRenderedexplicitandenvlockfiles will be named as"conda-{platform}.lock"and"conda-{platform}.lock.ymlrespectively by default.If you want to override that call conda-lock as follows.conda-lock-kexplicit--filename-template"specific-{platform}.conda.lock"Compound specificationConda-lock will build a spec list from several files if requested.conda-lock-fbase.yml-fspecific.yml-plinux-64-kexplicit--filename-template"specific-{platform}.lock"In this case all dependencies are combined, and the ordered union of allchannelsis used as the final
specification.This works for all supported file types.channel overridesYou can override the channels that are used by conda-lock in case you need to override the ones specified in
anenvironment.ymlconda-lock-cconda-forge-plinux-64platform specificationYou may specify the platforms you wish to target by default directly in anenvironment.ymlusing the (nonstandard)platformskey:# environment.ymlchannels:-conda-forgedependencies:-python=3.9-pandasplatforms:-linux-64-osx-64-win-64-osx-arm64# For Apple Silicon, e.g. M1/M2-linux-aarch64# aka arm64, use for Docker on Apple Silicon-linux-ppc64leIf you specify target platforms on the command line with-p, these will
override the values in the environment specification. If neitherplatformsnor-pare provided,conda-lockwill fall back to a default set of platforms.default categoryYou can may wish to split your dependencies into separate files for better
organization, e.g. aenvironment.ymlfor production dependencies and adev-environment.ymlfor development dependencies. You can assign all the
dependencies parsed from a single file to a category using the (nonstandard)categorykey.# dev-environment.ymlchannels:-conda-forgedependencies:-pytest-mypy=0.910category:devThe default category ismain.pip supportconda-lockcan also lock thedependencies.pipsection ofenvironment.yml, using a vendored copy ofPoetry'sdependency solver.private pip repositoriesRight nowconda-lockonly supportslegacypypi repos with basic auth. Most self-hosted repositories like Nexus, Artifactory etc. use this. You can configure private pip repositories in a similar way to channels, for example:channels:-conda-forgepip-repositories:-http://$PIP_USER:$PIP_PASSWORD@private-pypi.org/api/pypi/simpledependencies:-python=3.11-requests=2.26-pip:-fake-private-package==1.0.0Seethe related docs for private channelsto understand the rules regarding environment variable substitution.Alternatively, you can use thepoetryconfiguration file format to configure private PyPi repositories. The configuration file should be namedconfig.tomland have the following format:[repositories.example]url="https://username:password@example.repo/simple"The location of this file can be determined withpython -c 'from conda_lock._vendor.poetry.locations import CONFIG_DIR; print(CONFIG_DIR)'Private repositories will be used in addition topypi.org. For projects usingpyproject.toml, it is possible todisablepypi.orgentirely.--dev-dependencies/--no-dev-dependenciesBy default conda-lock will include dev dependencies in the specification of the lock (if the files that the lock
is being built from support them).  This can be disabled easilyconda-lock--no-dev-dependencies-f./recipe/meta.yaml--check-input-hashUnder some situation you may want to run conda lock in some kind of automated way (eg as a precommit) and want to not
need to regenerate the lockfiles if the underlying input specification for that particular lock as not changed.conda-lock--check-input-hash-plinux-64When the input_hash of the input files, channels match those present in a given lockfile, that lockfile will not be regenerated.--strip-auth, --auth and --auth-fileBy defaultconda-lockwill leave basic auth credentials for private conda channels in the lock file. If you wish to strip authentication from the file, provide the--strip-authargument.conda-lock--strip-auth-fenvironment.ymlIn order toconda-lock installa lock file with its basic auth credentials stripped, you will need to create an authentication file in.jsonformat like this:{"domain":"username:password"}If you have multiple channels that require different authentication within the same domain, you can additionally specify the channel like this:{"domain.org/channel1":"username1:password1","domain.org/channel2":"username2:password2"}You can provide the authentication either as string through--author as a filepath through--auth-file.conda-lockinstall--auth-fileauth.jsonconda-linux-64.lock--virtual-package-specConda makes use ofvirtual packagesthat are available at
runtime to gate dependency on system features.  Due to these not generally existing on your local execution platform conda-lock will inject
them into the solution environment with a reasonable guess at what a default system configuration should be.If you want to override which virtual packages are injected you can create a file like# virtual-packages.ymlsubdirs:linux-64:packages:__glibc:"2.17"__cuda:"11.4"win-64:packages:__cuda:"11.4"conda-lock will automatically use avirtual-packages.ymlit finds in the the current working directory.  Alternatively one can be specified
explicitly via the flag.condalock--virtual-package-specvirtual-packages-cuda.yml-plinux-64Input hash stabilityVirtual packages take part in the input hash so if you build an environment with a different set of virtual packages the input hash will change.
Additionally the default set of virtual packages may be augmented in future versions of conda-lock.  If you desire very stable input hashes
we recommend creating avirtual-packages.ymlfile to lock down the virtual packages considered.⚠️ in conjunction with micromambaMicromamba does not presently support some of the overrides to remove all discovered virtual packages, consequently the set of virtual packages
available at solve time may be larger than those specified in your specification.Supported file sourcesConda lock supports more than justenvironment.ymlspecifications!Additionally conda-lock supportsmeta.yaml(conda-build)
andpyproject.toml(flit,pdmandpoetrybased).  These do come with some gotchas but
are generally good enough for the 90% use-case.meta.yamlConda-lock will attempt to make an educated guess at the desired environment spec in a meta.yaml.  This is
not guaranteed to work for complex recipes with many selectors and outputs.  For multi-output recipes, conda-lock
will fuse all the dependencies together.  If that doesn't work for your case fall back to specifying the specification
as anenvironment.ymlSince a meta.yaml doesn't contain channel information we make use of the following extra key to specify channels# meta.yamlextra:channels:-conda-forge-defaultspyproject.tomlSincepyproject.tomlfiles are commonly used by python packages it can be desirable to create a lock
file directly from those dependencies to single-source a package's dependencies.  This makes use of some
conda-forge infrastructure (pypi-mapping) to do a lookup of the PyPI package name to a corresponding
conda package name (e.g.docker->docker-py).  In cases where there exists no lookup for the package it assumes that
the PyPI name, and the conda name are the same.Channels# pyproject.toml[tool.conda-lock]channels=['conda-forge','defaults']PlatformsLike inenvironment.yml, you can specify default platforms to target:# pyproject.toml[tool.conda-lock]platforms=['osx-arm64','linux-64']ExtrasIf your pyproject.toml file contains optional dependencies/extras these can be referred to by using the--extrasflag# pyproject.toml[tool.poetry.dependencies]mandatory="^1.0"psycopg2={version="^2.7",optional=true}mysqlclient={version="^1.3",optional=true}[tool.poetry.extras]mysql=["mysqlclient"]pgsql=["psycopg2"]These can be referened as followsconda-lock--extramysql--extrapgsql-fpyproject.tomlWhen generating lockfiles that make use of extras it is recommended to make use of--filename-templatecoveredhere.Filtering extrasBy default conda-lock will attempt to solve forALLextras/categories it discovers in sources.  This allows you to render explicit locks with subset of extras, without needing a new solve.However this does make the assumption that your extras can all be installed in conjunction with each other.  If you want extras filtering
to happen at the solve stage use the flag--filter-extrasconda-lock--extraincompatiblea--filter-extras-fpyproject.tomlExtra conda dependenciesSince in apyproject.tomlall the definitions are python dependencies if you need
to specify some non-python dependencies as well this can be accomplished by adding
the following sections to thepyproject.toml# pyproject.toml[tool.conda-lock.dependencies]sqlite=">=3.34"pip dependenciesIf a dependency refers directly to a URL rather than a package name and version,conda-lockwill assume it is pip-installable, e.g.:# pyproject.toml[tool.poetry.dependencies]python="3.9"pymage={url="https://github.com/MickaelRigault/pymage/archive/v1.0.tar.gz#sha256=11e99c4ea06b76ca7fb5b42d1d35d64139a4fa6f7f163a2f0f9cc3ea0b3c55eb"}Similarly, if a dependency is explicitly marked withsource = "pypi", it will
be treated as apipdependency, e.g.:[tool.poetry.dependencies]python="3.9"ampel-ztf={version="^0.8.0-alpha.2",source="pypi"}A dependency will also be treated as apipdependency if explicitly marked withsource = "pypi"in the[tool.conda-lock.dependencies]section, e.g.:[tool.conda-lock.dependencies]ampel-ztf={source="pypi"}Defaulting non-conda dependency sources to PyPIAlternatively, the above behavior is defaulted for all dependencies defined outside of[tool.conda-lock.dependencies], i.e.:Default topipdependencies for[tool.poetry.dependencies],[project.dependencies], etc.Default tocondadependencies for[tool.conda-lock.dependencies]by explicitly providingdefault-non-conda-source = "pip"in the[tool.conda-lock]section, e.g.:[tool.conda-lock]default-non-conda-source="pip"In all cases, the dependencies ofpip-installable packages will also be
installed withpip, unless they were already requested by acondadependency.Lock only conda-lock dependenciesTo lock only dependencies specified under[tool.conda-lock](i.e. skipping all dependencies specified elsewhere), explicitly provideskip-non-conda-lock = truein the[tool.conda-lock]section, e.g.:[tool.conda-lock]skip-non-conda-lock=trueDisabling pypi.orgWhen using private pip repos, it is possible to disablepypi.orgentirely. This can be useful when usingconda-lockbehind a network proxy that does not allow access topypi.org.[tool.conda-lock]allow-pypi-requests=falseDockerfile exampleIn order to use conda-lock in a docker-style context you want to add the lockfile to the
docker container.  In order to refresh the lock file just runconda-lockagain.Given a file tree likeDockerfile
  environment.yaml
* conda-linux-64.lockYou want a dockerfile that is structured something similar to this# Dockerfile# Build containerFROMcontinuumio/miniconda:latestascondaADDconda-linux-64.lock/locks/conda-linux-64.lockRUNcondacreate-p/opt/env--copy--file/locks/conda-linux-64.lock# Primary containerFROMgcr.io/distroless/base-debian10COPY--from=conda/opt/env/opt/env