Package: unidep
🚀 UniDep - Unified Conda and Pip Dependency Management 🚀UniDep streamlines Python project dependency management by unifying Conda and Pip packages in a single system.Learn when to use UniDepin ourFAQ.Handling dependencies in Python projects can be challenging, especially when juggling Python and non-Python packages.
This often leads to confusion and inefficiency, as developers juggle between multiple dependency files.📝 Unified Dependency File: Use eitherrequirements.yamlorpyproject.tomlto manage both Conda and Pip dependencies in one place.⚙️ Build System Integration: Integrates with Setuptools and Hatchling for automatic dependency handling duringpip install ./your-package.💻 One-Command Installation:unidep installhandles Conda, Pip, and local dependencies effortlessly.🏢 Monorepo-Friendly: Render (multiple)requirements.yamlorpyproject.tomlfiles into one Condaenvironment.yamlfile and maintain fully consistent globalandper sub packageconda-lockfiles.🌍 Platform-Specific Support: Specify dependencies for different operating systems or architectures.🔧pip-compileIntegration: Generate fully pinnedrequirements.txtfiles fromrequirements.yamlorpyproject.tomlfiles usingpip-compile.🔒 Integration withconda-lock: Generate fully pinnedconda-lock.ymlfiles from (multiple)requirements.yamlorpyproject.tomlfile(s), leveragingconda-lock.🤓 Nerd stats: written in Python, >99% test coverage, fully-typed, all Ruff's rules enabled, easily extensible, and minimal dependenciesunidepis designed to make dependency management in Python projects as simple and efficient as possible.
Try it now and streamline your development process![!TIP]
Check out theexamplerequirements.yamlandpyproject.tomlbelow.:books: Table of Contents:package: Installation:memo:requirements.yamlandpyproject.tomlstructureExampleExamplerequirements.yamlExamplepyproject.tomlKey PointsSupported Version PinningsConflict ResolutionHow It WorksPlatform SelectorsSupported SelectorsUsageImplementation:jigsaw: Build System IntegrationExample packagesSetuptools IntegrationHatchling Integration:desktop_computer: As a CLIunidep mergeunidep installunidep install-allunidep conda-lockunidep pip-compileunidep pipunidep conda❓ FAQQ: When to use UniDep?Q: Just show me a full example!Q: Uses of UniDep in the wild?Q: How is this different from conda/mamba/pip?Q: I found a project using unidep, now what?Q: How to handle local dependencies that do not use UniDep?Q: Can't Conda already do this?Q: What is the difference betweenconda-lockandunidep conda-lock?Q: What is the difference betweenhatch-conda/pdm-condaandunidep?:hammer_and_wrench: Troubleshootingpip installfails withFileNotFoundError:warning: Limitations:package: InstallationTo installunidep, run the following command:pipinstall"unidep[all]"orcondainstall-cconda-forgeunidep:memo:requirements.yamlandpyproject.tomlstructureunidepallows either using arequirements.yamlfile with a specific format (similar butnotthe same as a Condaenvironment.yamlfile) orpyproject.tomlfile with a[tool.unidep]section.Both files contain the following keys:name(Optional): For documentation, not used in the output.channels: List of conda channels for packages, such asconda-forge.dependencies: Mix of Conda and Pip packages.local_dependencies(Optional): List of paths to otherrequirements.yamlorpyproject.tomlfiles to include.platforms(Optional): List of platforms that are supported (used inconda-lock).optional_dependencies(Optional): Dictionary with lists of optional dependencies.Whether you use arequirements.yamlorpyproject.tomlfile, the same information can be specified in either.
Choose the format that works best for your project.ExampleExamplerequirements.yamlExample of arequirements.yamlfile:name:example_environmentchannels:-conda-forgedependencies:-numpy# same name on conda and pip-conda:python-graphviz# When names differ between Conda and Pippip:graphviz-pip:slurm-usage >=1.1.0,<2# pip-only-conda:mumps# conda-only# Use platform selectors-conda:cuda-toolkit =11.8# [linux64]local_dependencies:-../other-project-using-unidep# include other projects that use unidep-../common-requirements.yaml# include other requirements.yaml files-../project-not-managed-by-unidep# 🚨 Skips its dependencies!platforms:# (Optional) specify platforms that are supported (used in conda-lock)-linux-64-osx-arm64optional_dependencies:# (Optional) specify optional dependenciestest:-pytestfull:-../other-local-dep[test]# include its optional dependencies[!IMPORTANT]unidepcan process this duringpip installand create a Conda installableenvironment.yamlorconda-lock.ymlfile, and more![!NOTE]
For a more in-depth example containing multiple installable projects, see theexampledirectory.Examplepyproject.tomlAlternatively, one can fully configure the dependencies in thepyproject.tomlfile in the[tool.unidep]section:[tool.unidep]channels=["conda-forge"]dependencies=["numpy",# same name on conda and pip{conda="python-graphviz",pip="graphviz"},# When names differ between Conda and Pip{pip="slurm-usage >=1.1.0,<2"},# pip-only{conda="mumps"},# conda-only{conda="cuda-toolkit =11.8:linux64"}# Use platform selectors by appending `:linux64`]local_dependencies=["../other-project-using-unidep",# include other projects that use unidep"../common-requirements.yaml"# include other requirements.yaml files"../project-not-managed-by-unidep"# 🚨 Skips its dependencies!]platforms=[# (Optional) specify platforms that are supported (used in conda-lock)"linux-64","osx-arm64"]optional_dependencies={#(Optional)specifyoptionaldependenciestest=["pytest"],full=["../other-local-dep[test]"]# include its optional dependencies}This data structure isidenticalto therequirements.yamlformat, with the exception of thenamefield and theplatform selectors.
In therequirements.yamlfile, one can use e.g.,# [linux64], which in thepyproject.tomlfile is:linux64at the end of the package name.SeeBuild System Integrationfor more information on how to set upunidepwith different build systems (Setuptools or Hatchling).[!IMPORTANT]
In these docs, we often mention therequirements.yamlformat for simplicity, but the same information can be specified inpyproject.tomlas well.
Everything that is possible inrequirements.yamlis also possible inpyproject.toml!Key PointsStandard names (e.g.,- numpy) are assumed to be the same for Conda and Pip.Use a dictionary withconda: <package>andpip: <package>to specify different names across platforms.Usepip:to specify packages that are only available through Pip.Useconda:to specify packages that are only available through Conda.Use# [selector](YAML only) orpackage:selectorto specify platform-specific dependencies.Useplatforms:to specify the platforms that are supported.Uselocal_dependencies:to include otherrequirements.yamlorpyproject.tomlfiles and merge them into one. Also allows projects that are not managed byunidepto be included, but be aware that this skips their dependencies!Useoptional_dependencies:to specify optional dependencies. Can be installed likeunidep install ".[test]"orpip install ".[test]".We use the YAML notation here, but the same information can be specified inpyproject.tomlas well.Supported Version PinningsUniDep supports a range of version pinning operators (the same as Conda):Standard Version Constraints: Specify exact versions or ranges with standard operators like=,>,<,>=,<=.Example:=1.0.0,>1.0.0, <2.0.0.Version Exclusions: Exclude specific versions using!=.Example:!=1.5.0.Redundant Pinning Resolution: Automatically resolves redundant version specifications.Example:>1.0.0, >0.5.0simplifies to>1.0.0.Contradictory Version Detection: Errors are raised for contradictory pinnings to maintain dependency integrity. See theConflict Resolutionsection for more information.Example: Specifying>2.0.0, <1.5.0triggers aVersionConflictError.Invalid Pinning Detection: Detects and raises errors for unrecognized or improperly formatted version specifications.Conda Build Pinning: UniDep also supports Conda's build pinning, allowing you to specify builds in your pinning patterns.Example: Conda supports pinning builds likeqsimcirq * cuda*orvtk * *egl*.Limitation: While UniDep allows such build pinning, it requires that there be a single pin per package. UniDep cannot resolve conflicts where multiple build pinnings are specified for the same package.Example: UniDep can handleqsimcirq * cuda*, but it cannot resolve a scenario with bothqsimcirq * cuda*andqsimcirq * cpu*.Other Special Cases: In addition to Conda build pins, UniDep supports all special pinning formats, such as VCS (Version Control System) URLs or local file paths. This includes formats likepackage @ git+https://git/repo/hereorpackage @ file:///path/to/package. However, UniDep has a limitation: it can handle only one special pin per package. These special pins can be combined with an unpinned version specification, but not with multiple special pin formats for the same package.Example: UniDep can manage dependencies specified aspackage @ git+https://git/repo/hereandpackagein the samerequirements.yaml. However, it cannot resolve scenarios where bothpackage @ git+https://git/repo/hereandpackage @ file:///path/to/packageare specified for the same package.[!WARNING]Pinning Validation and Combination: UniDep actively validates and/or combines pinnings only whenmultiple different pinningsare specified for the same package.
This means if yourrequirements.yamlfiles include multiple pinnings for a single package, UniDep will attempt to resolve them into a single, coherent specification.
However, if the pinnings are contradictory or incompatible, UniDep will raise an error to alert you of the conflict.Conflict Resolutionunidepfeatures a conflict resolution mechanism to manage version conflicts and platform-specific dependencies inrequirements.yamlorpyproject.tomlfiles.How It WorksVersion Pinning Priority:unidepgives priority to version-pinned packages when the same package is specified multiple times. For instance, if bothfooandfoo <1are listed,foo <1is selected due to its specific version pin.Platform-Specific Version Pinning:unidepresolves platform-specific dependency conflicts by preferring the version with the narrowest platform scope. For instance, givenfoo <3 # [linux64]andfoo >1, it installsfoo >1,<3exclusively on Linux-64 andfoo >1on all other platforms.Intractable Conflicts: When conflicts are irreconcilable (e.g.,foo >1vs.foo <1),unidepraises an exception.Platform SelectorsThis tool supports a range of platform selectors that allow for specific handling of dependencies based on the user's operating system and architecture. This feature is particularly useful for managing conditional dependencies in diverse environments.Supported SelectorsThe following selectors are supported:linux: For all Linux-based systems.linux64: Specifically for 64-bit Linux systems.aarch64: For Linux systems on ARM64 architectures.ppc64le: For Linux on PowerPC 64-bit Little Endian architectures.osx: For all macOS systems.osx64: Specifically for 64-bit macOS systems.arm64: For macOS systems on ARM64 architectures (Apple Silicon).macos: An alternative toosxfor macOS systems.unix: A general selector for all UNIX-like systems (includes Linux and macOS).win: For all Windows systems.win64: Specifically for 64-bit Windows systems.UsageSelectors are used inrequirements.yamlfiles to conditionally include dependencies based on the platform:dependencies:-some-package >=1# [unix]-another-package# [win]-special-package# [osx64]-pip:cirq# [macos win]conda:cirq# [linux]Or when usingpyproject.tomlinstead ofrequirements.yaml:[tool.unidep]dependencies=["some-package >=1:unix","another-package:win","special-package:osx64",{pip="cirq:macos win",conda="cirq:linux"},]In this example:some-packageis included only in UNIX-like environments (Linux and macOS).another-packageis specific to Windows.special-packageis included only for 64-bit macOS systems.cirqis managed bypipon macOS and Windows, and bycondaon Linux. This demonstrates how you can specify different package managers for the same package based on the platform.Note that thepackage-name:unixsyntax can also be used in therequirements.yamlfile, but thepackage-name # [unix]syntax is not supported inpyproject.toml.Implementationunidepparses these selectors and filters dependencies according to the platform where it's being installed.
It is also used for creating environment and lock files that are portable across different platforms, ensuring that each environment has the appropriate dependencies installed.:jigsaw: Build System Integration[!TIP]
Seeexample/for working examples of usingunidepwith different build systems.unidepseamlessly integrates with popular Python build systems to simplify dependency management in your projects.Example packagesExplore these installableexamplepackages to understand howunidepintegrates with different build tools and configurations:ProjectBuild Toolpyproject.tomlrequirements.yamlsetup.pysetup_py_projectsetuptools✅✅✅setuptools_projectsetuptools✅✅❌pyproject_toml_projectsetuptools✅❌❌hatch_projecthatch✅✅❌hatch2_projecthatch✅❌❌Setuptools IntegrationFor projects usingsetuptools, configureunidepinpyproject.tomland either specify dependencies in arequirements.yamlfile or include them inpyproject.tomltoo.Usingpyproject.tomlonly: The[project.dependencies]field inpyproject.tomlgets automatically populated fromrequirements.yamlor from the[tool.unidep]section inpyproject.toml.Usingsetup.py: Theinstall_requiresfield insetup.pyautomatically reflects dependencies specified inrequirements.yamlorpyproject.toml.Examplepyproject.tomlConfiguration:[build-system]build-backend="setuptools.build_meta"requires=["setuptools","unidep"][project]dynamic=["dependencies"]Hatchling IntegrationFor projects managed withHatch,unidepcan be configured inpyproject.tomlto automatically process the dependencies fromrequirements.yamlor from the[tool.unidep]section inpyproject.toml.Example Configuration for Hatch:[build-system]requires=["hatchling","unidep"]build-backend="hatchling.build"[project]dynamic=["dependencies"]# Additional project configurations[tool.hatch]# Additional Hatch configurations[tool.hatch.metadata.hooks.unidep]:desktop_computer: As a CLISeeexamplefor more information or check the output ofunidep -hfor the available sub commands:usage:unidep[-h]{merge,install,install-all,conda-lock,pip-compile,pip,conda,version}...

UnifiedCondaandPiprequirementsmanagement.

positionalarguments:{merge,install,install-all,conda-lock,pip-compile,pip,conda,version}SubcommandsmergeCombinemultiple(orasingle)`requirements.yaml`or`pyproject.toml`filesintoasingleCondainstallable`environment.yaml`file.installAutomaticallyinstallalldependenciesfromoneormore`requirements.yaml`or`pyproject.toml`files.ThiscommandfirstinstallsdependencieswithConda,thenwithPip.Finally,itinstallslocalpackages(thosecontainingthe`requirements.yaml`or`pyproject.toml`files)using`pipinstall[-e]./project`.install-allInstalldependenciesfromall`requirements.yaml`or`pyproject.toml`filesfoundinthecurrentdirectoryorspecifieddirectory.ThiscommandfirstinstallsdependenciesusingConda,thenPip,andfinallythelocalpackages.conda-lockGenerateaglobal`conda-lock.yml`fileforacollectionof`requirements.yaml`or`pyproject.toml`files.Additionally,createindividual`conda-lock.yml`filesforeach`requirements.yaml`or`pyproject.toml`fileconsistentwiththegloballockfile.pip-compileGenerateafullypinned`requirements.txt`filefromoneormore`requirements.yaml`or`pyproject.toml`filesusing`pip-compile`from`pip-tools`.Thiscommandconsolidatesallpipdependenciesdefinedinthe`requirements.yaml`or`pyproject.toml`filesandcompilesthemintoasingle`requirements.txt`file,takingintoaccountthespecificversionsanddependenciesofeachpackage.pipGetthepiprequirementsforthecurrentplatformonly.condaGetthecondarequirementsforthecurrentplatformonly.versionPrintversioninformationofunidep.

options:-h,--helpshowthishelpmessageandexitunidep mergeUseunidep mergeto scan directories forrequirements.yamlfile(s) and combine them into anenvironment.yamlfile.
Seeunidep merge -hfor more information:usage:unidepmerge[-h][-oOUTPUT][-nNAME][--stdout][--selector{sel,comment}][-dDIRECTORY][--depthDEPTH][-v][--platform{linux-64,linux-aarch64,linux-ppc64le,osx-64,osx-arm64,win-64}][--skip-dependencySKIP_DEPENDENCY][--ignore-pinIGNORE_PIN][--overwrite-pinOVERWRITE_PIN]Combinemultiple(orasingle)`requirements.yaml`or`pyproject.toml`files
intoasingleCondainstallable`environment.yaml`file.Exampleusage:`unidepmerge--directory.--depth1--outputenvironment.yaml`tosearchfor`requirements.yaml`or`pyproject.toml`filesinthecurrentdirectoryandits
subdirectoriesandcreate`environment.yaml`.Thesearethedefaults,soyou
canalsojustrun`unidepmerge`.

options:-h,--helpshowthishelpmessageandexit-oOUTPUT,--outputOUTPUTOutputfileforthecondaenvironment,bydefault`environment.yaml`-nNAME,--nameNAMENameofthecondaenvironment,bydefault`myenv`--stdoutOutputtostdoutinsteadofafile--selector{sel,comment}Theselectortousefortheenvironmentmarkers,if`sel`then`-numpy# [linux]` becomes `sel(linux):numpy`,if`comment`thenitremains`-numpy#[linux]`,bydefault`sel`-dDIRECTORY,--directoryDIRECTORYBasedirectorytoscanfor`requirements.yaml`or`pyproject.toml`file(s),bydefault`.`--depthDEPTHMaximumdepthtoscanfor`requirements.yaml`or`pyproject.toml`files,bydefault1-v,--verbosePrintverboseoutput--platform{linux-64,linux-aarch64,linux-ppc64le,osx-64,osx-arm64,win-64},-p{linux-64,linux-aarch64,linux-ppc64le,osx-64,osx-arm64,win-64}Theplatform(s)togettherequirementsfor.Multipleplatformscanbespecified.Bydefault,thecurrentplatform(`linux-64`)isused.--skip-dependencySKIP_DEPENDENCYSkipinstallingaspecificdependencythatisinoneofthe`requirements.yaml`or`pyproject.toml`files.Thisoptioncanbeusedmultipletimes,eachtimespecifyingadifferentpackagetoskip.Forexample,use`--skip-dependencypandas`toskipinstallingpandas.--ignore-pinIGNORE_PINIgnoretheversionpinforaspecificpackage,e.g.,`--ignore-pinnumpy`.Thisoptioncanberepeatedtoignoremultiplepackages.--overwrite-pinOVERWRITE_PINOverwritetheversionpinforaspecificpackage,e.g.,`--overwrite-pin'numpy==1.19.2'`.Thisoptioncanberepeatedtooverwritethepinsofmultiplepackages.unidep installUseunidep installon one or morerequirements.yamlfiles and install the dependencies on the current platform using conda, then install the remaining dependencies with pip, and finally install the current package withpip install [-e] ..
Seeunidep install -hfor more information:usage:unidepinstall[-h][-v][-e][--skip-local][--skip-pip][--skip-conda][--skip-dependencySKIP_DEPENDENCY][--no-dependencies][--conda-executable{conda,mamba,micromamba}][--conda-env-nameCONDA_ENV_NAME|--conda-env-prefixCONDA_ENV_PREFIX][--dry-run][--ignore-pinIGNORE_PIN][--overwrite-pinOVERWRITE_PIN]files[files...]Automaticallyinstallalldependenciesfromoneormore`requirements.yaml`or`pyproject.toml`files.ThiscommandfirstinstallsdependencieswithConda,thenwithPip.Finally,itinstallslocalpackages(thosecontainingthe`requirements.yaml`or`pyproject.toml`files)using`pipinstall[-e]./project`.Exampleusage:`unidepinstall.`forasingleproject.For
multipleprojects:`unidepinstall./project1./project2`.Thecommandaccepts
bothfilepathsanddirectoriescontaininga`requirements.yaml`or`pyproject.toml`file.Use`--editable`or`-e`toinstallthelocalpackagesineditablemode.See`unidepinstall-all`toinstallall`requirements.yaml`or`pyproject.toml`filesinandbelowthecurrentfolder.

positionalarguments:filesThe`requirements.yaml`or`pyproject.toml`file(s)toparseorfolder(s)thatcontainthosefile(s),bydefault`.`options:-h,--helpshowthishelpmessageandexit-v,--verbosePrintverboseoutput-e,--editableInstalltheprojectineditablemode--skip-localSkipinstallinglocaldependencies--skip-pipSkipinstallingpipdependenciesfrom`requirements.yaml`or`pyproject.toml`--skip-condaSkipinstallingcondadependenciesfrom`requirements.yaml`or`pyproject.toml`--skip-dependencySKIP_DEPENDENCYSkipinstallingaspecificdependencythatisinoneofthe`requirements.yaml`or`pyproject.toml`files.Thisoptioncanbeusedmultipletimes,eachtimespecifyingadifferentpackagetoskip.Forexample,use`--skip-dependencypandas`toskipinstallingpandas.--no-dependenciesSkipinstallingdependenciesfrom`requirements.yaml`or`pyproject.toml`file(s)andonlyinstalllocalpackage(s).Usefulafterinstallinga`conda-lock.yml`filebecausethenalldependencieshavealreadybeeninstalled.--conda-executable{conda,mamba,micromamba}Thecondaexecutabletouse--conda-env-nameCONDA_ENV_NAMENameofthecondaenvironment,ifnotprovided,thecurrentlyactiveenvironmentnameisused,unless`--conda-env-prefix`isprovided--conda-env-prefixCONDA_ENV_PREFIXPathtothecondaenvironment,ifnotprovided,thecurrentlyactiveenvironmentpathisused,unless`--conda-env-name`isprovided--dry-run,--dryOnlyprintthecommandsthatwouldberun--ignore-pinIGNORE_PINIgnoretheversionpinforaspecificpackage,e.g.,`--ignore-pinnumpy`.Thisoptioncanberepeatedtoignoremultiplepackages.--overwrite-pinOVERWRITE_PINOverwritetheversionpinforaspecificpackage,e.g.,`--overwrite-pin'numpy==1.19.2'`.Thisoptioncanberepeatedtooverwritethepinsofmultiplepackages.unidep install-allUseunidep install-allon a folder with packages that containrequirements.yamlfiles and install the dependencies on the current platform using conda, then install the remaining dependencies with pip, and finally install the current package withpip install [-e] ./package1 ./package2.
Seeunidep install-all -hfor more information:usage:unidepinstall[-h][-v][-e][--skip-local][--skip-pip][--skip-conda][--skip-dependencySKIP_DEPENDENCY][--no-dependencies][--conda-executable{conda,mamba,micromamba}][--conda-env-nameCONDA_ENV_NAME|--conda-env-prefixCONDA_ENV_PREFIX][--dry-run][--ignore-pinIGNORE_PIN][--overwrite-pinOVERWRITE_PIN]files[files...]Automaticallyinstallalldependenciesfromoneormore`requirements.yaml`or`pyproject.toml`files.ThiscommandfirstinstallsdependencieswithConda,thenwithPip.Finally,itinstallslocalpackages(thosecontainingthe`requirements.yaml`or`pyproject.toml`files)using`pipinstall[-e]./project`.Exampleusage:`unidepinstall.`forasingleproject.For
multipleprojects:`unidepinstall./project1./project2`.Thecommandaccepts
bothfilepathsanddirectoriescontaininga`requirements.yaml`or`pyproject.toml`file.Use`--editable`or`-e`toinstallthelocalpackagesineditablemode.See`unidepinstall-all`toinstallall`requirements.yaml`or`pyproject.toml`filesinandbelowthecurrentfolder.

positionalarguments:filesThe`requirements.yaml`or`pyproject.toml`file(s)toparseorfolder(s)thatcontainthosefile(s),bydefault`.`options:-h,--helpshowthishelpmessageandexit-v,--verbosePrintverboseoutput-e,--editableInstalltheprojectineditablemode--skip-localSkipinstallinglocaldependencies--skip-pipSkipinstallingpipdependenciesfrom`requirements.yaml`or`pyproject.toml`--skip-condaSkipinstallingcondadependenciesfrom`requirements.yaml`or`pyproject.toml`--skip-dependencySKIP_DEPENDENCYSkipinstallingaspecificdependencythatisinoneofthe`requirements.yaml`or`pyproject.toml`files.Thisoptioncanbeusedmultipletimes,eachtimespecifyingadifferentpackagetoskip.Forexample,use`--skip-dependencypandas`toskipinstallingpandas.--no-dependenciesSkipinstallingdependenciesfrom`requirements.yaml`or`pyproject.toml`file(s)andonlyinstalllocalpackage(s).Usefulafterinstallinga`conda-lock.yml`filebecausethenalldependencieshavealreadybeeninstalled.--conda-executable{conda,mamba,micromamba}Thecondaexecutabletouse--conda-env-nameCONDA_ENV_NAMENameofthecondaenvironment,ifnotprovided,thecurrentlyactiveenvironmentnameisused,unless`--conda-env-prefix`isprovided--conda-env-prefixCONDA_ENV_PREFIXPathtothecondaenvironment,ifnotprovided,thecurrentlyactiveenvironmentpathisused,unless`--conda-env-name`isprovided--dry-run,--dryOnlyprintthecommandsthatwouldberun--ignore-pinIGNORE_PINIgnoretheversionpinforaspecificpackage,e.g.,`--ignore-pinnumpy`.Thisoptioncanberepeatedtoignoremultiplepackages.--overwrite-pinOVERWRITE_PINOverwritetheversionpinforaspecificpackage,e.g.,`--overwrite-pin'numpy==1.19.2'`.Thisoptioncanberepeatedtooverwritethepinsofmultiplepackages.unidep conda-lockUseunidep conda-lockon one or multiplerequirements.yamlfiles and output the conda-lock file.
Optionally, when using a monorepo with multiple subpackages (with their ownrequirements.yamlfiles), generate a lock file for each subpackage.
Seeunidep conda-lock -hfor more information:usage:unidepconda-lock[-h][--only-global][--lockfileLOCKFILE][--check-input-hash][-dDIRECTORY][--depthDEPTH][-fFILE][-v][--platform{linux-64,linux-aarch64,linux-ppc64le,osx-64,osx-arm64,win-64}][--skip-dependencySKIP_DEPENDENCY][--ignore-pinIGNORE_PIN][--overwrite-pinOVERWRITE_PIN]...

Generateaglobal`conda-lock.yml`fileforacollectionof`requirements.yaml`or`pyproject.toml`files.Additionally,createindividual`conda-lock.yml`filesforeach`requirements.yaml`or`pyproject.toml`file
consistentwiththegloballockfile.Exampleusage:`unidepconda-lock
--directory./projects`togenerateconda-lockfilesforall`requirements.yaml`or`pyproject.toml`filesinthe`./projects`directory.
Use`--only-global`togenerateonlythegloballockfile.The`--check-input-hash`optioncanbeusedtoavoidregeneratinglockfilesiftheinputhasn'tchanged.positional arguments:extra_flags           Extra flags to pass to `conda-lock lock`. These flagsare passed directly and should be provided in theformat expected by `conda-lock lock`. For example,`unidep conda-lock -- --micromamba`. Note that the`--` is required to separate the flags for `unidepconda-lock` from the flags for `conda-lock lock`.options:-h, --help            show this help message and exit--only-global         Only generate the global lock file--lockfile LOCKFILE   Specify a path for the global lockfile (default:`conda-lock.yml` in current directory). Path should berelative, e.g., `--lockfile ./locks/example.conda-lock.yml`.--check-input-hash    Check existing input hashes in lockfiles beforeregenerating lock files. This flag is directly passedto `conda-lock`.-d DIRECTORY, --directory DIRECTORYBase directory to scan for `requirements.yaml` or`pyproject.toml` file(s), by default `.`--depth DEPTH         Maximum depth to scan for `requirements.yaml` or`pyproject.toml` files, by default 1-f FILE, --file FILE  A single `requirements.yaml` or `pyproject.toml` fileto use, or folder that contains that file. This is analternative to using `--directory` which searches forall `requirements.yaml` or `pyproject.toml` files inthe directory and its subdirectories.-v, --verbose         Print verbose output--platform {linux-64,linux-aarch64,linux-ppc64le,osx-64,osx-arm64,win-64}, -p {linux-64,linux-aarch64,linux-ppc64le,osx-64,osx-arm64,win-64}The platform(s) to get the requirements for. Multipleplatforms can be specified. By default, the currentplatform (`linux-64`) is used.--skip-dependency SKIP_DEPENDENCYSkip installing a specific dependency that is in oneof the `requirements.yaml` or `pyproject.toml` files.This option can be used multiple times, each timespecifying a different package to skip. For example,use `--skip-dependency pandas` to skip installingpandas.--ignore-pin IGNORE_PINIgnore the version pin for a specific package, e.g.,`--ignore-pin numpy`. This option can be repeated toignore multiple packages.--overwrite-pin OVERWRITE_PINOverwrite the version pin for a specific package,e.g., `--overwrite-pin 'numpy==1.19.2'`.Thisoptioncanberepeatedtooverwritethepinsofmultiplepackages.unidep pip-compileUseunidep pip-compileon one or multiplerequirements.yamlfiles and output a fully lockedrequirements.txtfile usingpip-compilefrompip-tools.
Seeunidep pip-compile -hfor more information:usage:unideppip-compile[-h][-oOUTPUT_FILE][-dDIRECTORY][--depthDEPTH][-v][--platform{linux-64,linux-aarch64,linux-ppc64le,osx-64,osx-arm64,win-64}][--skip-dependencySKIP_DEPENDENCY][--ignore-pinIGNORE_PIN][--overwrite-pinOVERWRITE_PIN]...

Generateafullypinned`requirements.txt`filefromoneormore`requirements.yaml`or`pyproject.toml`filesusing`pip-compile`from`pip-
tools`.Thiscommandconsolidatesallpipdependenciesdefinedinthe`requirements.yaml`or`pyproject.toml`filesandcompilesthemintoasingle`requirements.txt`file,takingintoaccountthespecificversionsand
dependenciesofeachpackage.Exampleusage:`unideppip-compile--directory
./projects`togeneratea`requirements.txt`fileforall`requirements.yaml`or`pyproject.toml`filesinthe`./projects`directory.Use`--output-file
requirements.txt`tospecifyadifferentoutputfile.

positionalarguments:extra_flagsExtraflagstopassto`pip-compile`.Theseflagsarepasseddirectlyandshouldbeprovidedintheformatexpectedby`pip-compile`.Forexample,`unideppip-compile----generate-hashes--allow-unsafe`.Notethatthe`--`isrequiredtoseparatetheflagsfor`unideppip-compile`fromtheflagsfor`pip-compile`.

options:-h,--helpshowthishelpmessageandexit-oOUTPUT_FILE,--output-fileOUTPUT_FILEOutputfileforthepiprequirements,bydefault`requirements.txt`-dDIRECTORY,--directoryDIRECTORYBasedirectorytoscanfor`requirements.yaml`or`pyproject.toml`file(s),bydefault`.`--depthDEPTHMaximumdepthtoscanfor`requirements.yaml`or`pyproject.toml`files,bydefault1-v,--verbosePrintverboseoutput--platform{linux-64,linux-aarch64,linux-ppc64le,osx-64,osx-arm64,win-64},-p{linux-64,linux-aarch64,linux-ppc64le,osx-64,osx-arm64,win-64}Theplatform(s)togettherequirementsfor.Multipleplatformscanbespecified.Bydefault,thecurrentplatform(`linux-64`)isused.--skip-dependencySKIP_DEPENDENCYSkipinstallingaspecificdependencythatisinoneofthe`requirements.yaml`or`pyproject.toml`files.Thisoptioncanbeusedmultipletimes,eachtimespecifyingadifferentpackagetoskip.Forexample,use`--skip-dependencypandas`toskipinstallingpandas.--ignore-pinIGNORE_PINIgnoretheversionpinforaspecificpackage,e.g.,`--ignore-pinnumpy`.Thisoptioncanberepeatedtoignoremultiplepackages.--overwrite-pinOVERWRITE_PINOverwritetheversionpinforaspecificpackage,e.g.,`--overwrite-pin'numpy==1.19.2'`.Thisoptioncanberepeatedtooverwritethepinsofmultiplepackages.unidep pipUseunidep pipon arequirements.yamlfile and output the pip installable dependencies on the current platform (default).
Seeunidep pip -hfor more information:usage:unideppip[-h][-fFILE][-v][--platform{linux-64,linux-aarch64,linux-ppc64le,osx-64,osx-arm64,win-64}][--skip-dependencySKIP_DEPENDENCY][--ignore-pinIGNORE_PIN][--overwrite-pinOVERWRITE_PIN][--separatorSEPARATOR]Getthepiprequirementsforthecurrentplatformonly.Exampleusage:`unidep
pip--filefolder1--filefolder2/requirements.yaml--separator' '--platform
linux-64`toextractallthepipdependenciesspecifictothelinux-64
platform.Notethatthe`--file`argumentcanbeusedmultipletimesto
specifymultiple`requirements.yaml`or`pyproject.toml`filesandthat--file
canalsobeafolderthatcontainsa`requirements.yaml`or`pyproject.toml`file.

options:-h,--helpshowthishelpmessageandexit-fFILE,--fileFILEThe`requirements.yaml`or`pyproject.toml`filetoparse,orfolderthatcontainsthatfile,bydefault`.`-v,--verbosePrintverboseoutput--platform{linux-64,linux-aarch64,linux-ppc64le,osx-64,osx-arm64,win-64},-p{linux-64,linux-aarch64,linux-ppc64le,osx-64,osx-arm64,win-64}Theplatform(s)togettherequirementsfor.Multipleplatformscanbespecified.Bydefault,thecurrentplatform(`linux-64`)isused.--skip-dependencySKIP_DEPENDENCYSkipinstallingaspecificdependencythatisinoneofthe`requirements.yaml`or`pyproject.toml`files.Thisoptioncanbeusedmultipletimes,eachtimespecifyingadifferentpackagetoskip.Forexample,use`--skip-dependencypandas`toskipinstallingpandas.--ignore-pinIGNORE_PINIgnoretheversionpinforaspecificpackage,e.g.,`--ignore-pinnumpy`.Thisoptioncanberepeatedtoignoremultiplepackages.--overwrite-pinOVERWRITE_PINOverwritetheversionpinforaspecificpackage,e.g.,`--overwrite-pin'numpy==1.19.2'`.Thisoptioncanberepeatedtooverwritethepinsofmultiplepackages.--separatorSEPARATORTheseparatorbetweenthedependencies,bydefault``unidep condaUseunidep condaon arequirements.yamlfile and output the conda installable dependencies on the current platform (default).
Seeunidep conda -hfor more information:usage:unidepconda[-h][-fFILE][-v][--platform{linux-64,linux-aarch64,linux-ppc64le,osx-64,osx-arm64,win-64}][--skip-dependencySKIP_DEPENDENCY][--ignore-pinIGNORE_PIN][--overwrite-pinOVERWRITE_PIN][--separatorSEPARATOR]Getthecondarequirementsforthecurrentplatformonly.Exampleusage:`unidepconda--filefolder1--filefolder2/requirements.yaml--separator' '--platformlinux-64`toextractallthecondadependenciesspecifictothe
linux-64platform.Notethatthe`--file`argumentcanbeusedmultipletimestospecifymultiple`requirements.yaml`or`pyproject.toml`filesandthat
--filecanalsobeafolderthatcontainsa`requirements.yaml`or`pyproject.toml`file.

options:-h,--helpshowthishelpmessageandexit-fFILE,--fileFILEThe`requirements.yaml`or`pyproject.toml`filetoparse,orfolderthatcontainsthatfile,bydefault`.`-v,--verbosePrintverboseoutput--platform{linux-64,linux-aarch64,linux-ppc64le,osx-64,osx-arm64,win-64},-p{linux-64,linux-aarch64,linux-ppc64le,osx-64,osx-arm64,win-64}Theplatform(s)togettherequirementsfor.Multipleplatformscanbespecified.Bydefault,thecurrentplatform(`linux-64`)isused.--skip-dependencySKIP_DEPENDENCYSkipinstallingaspecificdependencythatisinoneofthe`requirements.yaml`or`pyproject.toml`files.Thisoptioncanbeusedmultipletimes,eachtimespecifyingadifferentpackagetoskip.Forexample,use`--skip-dependencypandas`toskipinstallingpandas.--ignore-pinIGNORE_PINIgnoretheversionpinforaspecificpackage,e.g.,`--ignore-pinnumpy`.Thisoptioncanberepeatedtoignoremultiplepackages.--overwrite-pinOVERWRITE_PINOverwritetheversionpinforaspecificpackage,e.g.,`--overwrite-pin'numpy==1.19.2'`.Thisoptioncanberepeatedtooverwritethepinsofmultiplepackages.--separatorSEPARATORTheseparatorbetweenthedependencies,bydefault``❓ FAQHere is a list of questions we have either been asked by users or potential pitfalls we hope to help users avoid:Q: When to use UniDep?A:UniDep is particularly useful for setting up full development environments that require both Pythonandnon-Python dependencies (e.g., CUDA, compilers, etc.) with a single command.In fields like research, data science, robotics, AI, and ML projects, it is common to work from a locally cloned Git repository.Setting up a full development environment can be a pain, especially if you need to install non Python dependencies like compilers, low-level numerical libraries, or CUDA (luckily Conda has all of them).
Typically, instructions are different for each OS and their corresponding package managers (apt,brew,yum,winget, etc.).With UniDep, you can specify all your Pip and Conda dependencies in a single file.
To get set up on a new machine, you just need to install Conda (we recommendmicromamba) and runpip install unidep; unidep install-all -ein your project directory, to install all dependencies and local packages in editable mode in the current Conda environment.For fully reproducible environments, you can rununidep conda-lockto generate aconda-lock.ymlfile.
Then, runconda env create -f conda-lock.yml -n myenvto create a new Conda environment with all the third-party dependencies.
Finally, rununidep install-all -e --no-dependenciesto install all your local packages in editable mode.For those who prefer not to use Conda, you can simply runpip install -e .on a project using UniDep.
You'll need to install the non-Python dependencies yourself, but you'll have a list of them in therequirements.yamlfile.In summary, use UniDep if you:Prefer installing packages with conda but still want your package to be pip installable.Are tired of synchronizing your Pip requirements (requirements.txt) and Conda requirements (environment.yaml).Want a low-effort, comprehensive development environment setup.Q: Just show me a full example!A:Check out theexamplefolder.Q: Uses of UniDep in the wild?A:UniDep really shines when used in a monorepo with multiple dependent projects, however, since these are typically private, we cannot share them.However, an example of a single package that is public ishome-assistant-streamdeck-yaml.
This is a Python package that allows to interact withHome Assistantfrom an Elgato Stream Deck connected via USB to e.g., a Raspberry Pi.
It requires a couple of system dependencies (e.g.,libusbandhidapi), which are typically installed withaptorbrew.
TheREADME.mdshows different installation instructions on Linux, MacOS, and Windows for non-Conda installs, however, with UniDep, we can just useunidep install .on all platforms.
It is fully configured viapyproject.toml.
The 2Dockerfiles show 2 different ways of using UniDep:Dockerfile.locked: Installingconda-lock.yml(generated withunidep conda-lock) and thenpip install .the local package.Dockerfile.latest: Usingunidep install .to install all dependencies, first with conda, then pip, then the local package.Q: How is this different from conda/mamba/pip?A:UniDep uses pip and conda under the hood to install dependencies, but it is not a replacement for them. UniDep will print the commands it runs, so you can see exactly what it is doing.Q: I found a project using unidep, now what?A:You can install it likeany other Python packageusingpip install.
However, to take full advantage of UniDep's functionality, clone the repository and rununidep install-all -ein the project directory.
This installs all dependencies in editable mode in the current Conda environment.Q: How to handle local dependencies that do not use UniDep?A:You can use thelocal_dependenciesfield in therequirements.yamlorpyproject.tomlfile to specify local dependencies.
However,ifa local dependency isnotmanaged by UniDep, it will skip installing its dependencies!To include all its dependencies, either convert the package to use UniDep (🏆), or maintain a separaterequirements.yamlfile, e.g., for a package calledfoocreate,foo-requirements.yaml:dependencies:# List the dependencies of foo here-numpy-scipy-matplotlib-barlocal_dependencies:-./path/to/foo# This is the path to the packageThen, in therequirements.yamlorpyproject.tomlfile of the package that usesfoo, listfoo-requirements.yamlas a local dependency:local_dependencies:-./path/to/foo-requirements.yamlQ: Can't Conda already do this?A:Not quite. Conda can indeed install both Conda and Pip dependencies via anenvironment.yamlfile, however, it does not work the other way around.
Pip cannot install thepipdependencies from anenvironment.yamlfile.
This means, that if you want your package to be installable withpip install -e .andsupport Conda, you need to maintain two separate files:environment.yamlandrequirements.txt(or specify these dependencies inpyproject.tomlorsetup.py).Q: What is the difference betweenconda-lockandunidep conda-lock?A:conda-lockis a standalone tool that creates aconda-lock.ymlfile from aenvironment.yamlfile.
On the other hand,unidep conda-lockis a command within the UniDep tool that also generates aconda-lock.ymlfile (leveragingconda-lock), but it does so from one or morerequirements.yamlorpyproject.tomlfiles.
When managing multiple dependent projects (e.g., in a monorepo), a unique feature ofunidep conda-lockis its ability to createconsistentindividualconda-lock.ymlfiles for eachrequirements.yamlorpyproject.tomlfile, ensuring consistency with a globalconda-lock.ymlfile.
This feature is not available in the standaloneconda-locktool.Q: What is the difference betweenhatch-conda/pdm-condaandunidep?A:hatch-condais a plugin forhatchthat integrates Conda environments intohatch.
A key difference is thathatch-condakeeps Conda and Pip dependencies separate, choosing to install packages with either CondaorPip.
This results in Conda being a hard requirement, for example, ifnumbais specified for Conda, it cannot be installed with Pip despite its availability on PyPI.In contrast,UniDepdoes not require Conda.
Without Conda, it can still install any dependency that is available on PyPI (e.g.,numbais both Conda and Pip installable).
However, without Conda, UniDep will not install dependencies exclusive to Conda.
These Conda-specific dependencies can often be installed through alternative package managers likeapt,brew,yum, or by building them from source.Another key difference is thathatch-condais managingHatch environmentswhereasunidepcan install Pip dependencies in the current Python environment (venv, Conda, Hatch, etc.), however, to optimally use UniDep, we recommend using Conda environments to additionally install non-Python dependencies.Similar tohatch-conda,unidepalso integrates with Hatchling, but it works in a slightly different way.A:pdm-condais a plugin forpdmdesigned to facilitate the use of Conda environments in conjunction withpdm.
Likehatch-conda,pdm-condaopts to install packages either with Conda or Pip.
It is closely integrated withpdm, primarily enabling the inclusion of Conda packages inpdm's lock file (pdm.lock).
However,pdm-condalacks extensive cross-platform support.
For instance, when adding a package like Numba usingpdm-conda, it gets locked to the current platform (e.g., osx-arm64) without the flexibility to specify compatibility for other platforms such as linux64.
In contrast, UniDep allows for cross-platform compatibility, enabling the user to specify dependencies for multiple platforms.
UniDep currently does not supportpdm, but it does support Hatchling and Setuptools.UniDep stands out from bothpdm-condaandhatch-condawith its additional functionalities, particularly beneficial for monorepos and projects spanning multiple operating systems. For instance:Conda Lock Files: Createconda-lock.ymlfiles for all packages with consistent sub-lock files per package.CLI tools: Provides tools likeunidep install-all -ewhich will install multiple local projects (e.g., in monorepo) and all its dependencies first with Conda, then remaining ones with Pip, and finally the local dependencies in editable mode with Pip.Conda Environment Files: Can create standard Condaenvironment.yamlfiles by combining the dependencies from manyrequirements.yamlorpyproject.tomlfiles.Platform-Specific Dependencies: Allows specifying dependencies for certain platforms (e.g., linux64, osx-arm64), enhancing cross-platform compatibility.:hammer_and_wrench: Troubleshootingpip installfails withFileNotFoundErrorWhen using a project that useslocal_dependencies: [../not/current/dir]in therequirements.yamlfile:local_dependencies:# File in a different directory than the pyproject.toml file-../common-requirements.yamlYou might get an error like this when using apipversion older than22.0:$pipinstall/path/to/your/project/using/unidep...File"/usr/lib/python3.8/pathlib.py",line1222,inopenreturnio.open(self,mode,buffering,encoding,errors,newline,File"/usr/lib/python3.8/pathlib.py",line1078,in_openerreturnself._accessor.open(self,flags,mode)FileNotFoundError:[Errno2]Nosuchfileordirectory:'/tmp/common-requirements.yaml'The solution is to upgradepipto version22.0or newer:pipinstall--upgradepip:warning: LimitationsConda-Focused: Best suited for Conda environments. However, note that havingcondais not a requirement to install packages that use UniDep.Setuptools and Hatchling only: Currently only works with setuptools and Hatchling, not flit, poetry, or other build systems. Open an issue if you'd like to see support for other build systems.Nologic operators in platform selectorsandno Python selectors.Tryunideptoday for a streamlined approach to managing your Conda environment dependencies across multiple projects! 🎉👏