Package: pip-compile-cross-platform
pip-compile-cross-platformPin your Python dependencies as you would withpip-compile, except in a cross-platform way.Usagepip install --user pip-compile-cross-platformpip-compile-cross-platform requirements.inWhat is "pinning"?"Pinning" your Python dependencies makes them predictable and deterministic by resolving all transitive dependencies
(dependencies of dependencies) and hard-coding them up-front.For example, you can define arequirements.infile with the top-level requirements thatyouhave:requests==2.27.1Then, you runpip-compile-cross-platform, which produces arequirements.txtfile with the "pinned" packages:certifi==2022.5.18.1 ; python_full_version >= "3.6.0" \
    --hash=sha256:f1d53542ee8cbedbe2118b5686372fb33c297fcd6379b050cca0ef13a597382a \
    --hash=sha256:9c5705e395cd70084351dd8ad5c41e65655e08ce46f2ec9cf6c2c08390f71eb7
charset-normalizer==2.0.12 ; python_full_version >= "3.6.0" \
    --hash=sha256:2857e29ff0d34db842cd7ca3230549d1a697f96ee6d3fb071cfa6c7393832597 \
    --hash=sha256:6881edbebdb17b39b4eaaa821b438bf6eddffb4468cf344f09f89def34a8b1df
idna==3.3 ; python_full_version >= "3.6.0" \
    --hash=sha256:84d9dd047ffa80596e0f246e2eab0b391788b0503584e8945f2368256d2735ff \
    --hash=sha256:9d643ff0a55b762d5cdb124b8eaa99c66322e2157b69160bc32796e824360e6d
requests==2.27.1 \
    --hash=sha256:f22fa1e554c9ddfd16e6e41ac79759e17be9e492b3587efa038054674760e72d \
    --hash=sha256:68d7c56fd5a8999887728ef304a6d12edc7be74f1cfa47714fc8b414525c9a61
urllib3==1.26.9 ; python_version >= "2.7" and python_full_version < "3.0.0" or python_full_version >= "3.6.0" and python_version < "4" \
    --hash=sha256:44ece4d53fb1706f667c9bd1c648f5469a2ec925fcf3a776667042d645472c14 \
    --hash=sha256:aabaf16477806a5e1dd19aa41f8c2b7950dd3c746362d7e3223dbe6de6ac448eNow, when developers install dependencies into their environment, they use therequirements.txtfile:pip install -r requirements.txt.Since they're using this "lockfile" rather than the simplerrequirements.in, they're guaranteed to have the exact
same packages installed as everyone else working on your project. This sidesteps a major cause of "works on my machine, but for
some reason not on yours" issues that aren't fun to diagnose 😎How does this compare topip-compile?pip-compileis an incredible tool built bythe Jazzband crewthat pins dependencies exactly as described above.
It can comfortably be described as a primary solution to the "pinning" problem in the Python ecosystem over the last
half-decade. In summary:pip-compileis great :)However, there's one main limitation:cross-environment usage is
unsupported.
This means that conditional dependencies (such as Windows-only dependencies likecolorama,
or backport libraries likeiso8601which are only needed for Python < 3.7) aren't
properly captured bypip-compile.pip-compilerecommends creating a separaterequirements.txtfor each environment, but this
can lead to a lot ofrequirements-$python-version-$os.txtfiles:lockfilesrequirements-3.7-windows.txtrequirements-3.7-linux.txtrequirements-3.7-macos.txtrequirements-3.8-windows.txt...This has a few downsides:Developers have to consider whichrequirements-...txtfile to install, and if the wrong one is chosen, the error
message (pip complaining about missing hashes) doesn't make the root cause clear.Generating lockfiles requires many operating system environments, which can be a hassle to set up. There'sa GitHub
Actions workflow templatethat can be used to lower this cost, but
it's still a time-consuming (and perhaps eventually expensive) solution.There's a minor, but possible other concern: there's a race condition where a new version of a transitive
dependency is published in between lockfiles being created, causing differentrequirements...txtfiles to
havedifferent versions of the same package.pip-compile-cross-platformsidesteps these downsides by generating a singlerequirements.txtlockfile that
is valid for all environments.Usage compared topip-compilepip-compile-cross-platformis planned to be compatible with the same command-line usage aspip-compile. However,
many flags and options are missing so far, sohelp to improve this compatibility would be much appreciated.How it works under-the-hoodEnvironment-specific dependencies are defined usingenvironment markers.pip-compileevaluates environment markers up-front according to the current environment, which is why packages not
needed for the lockfile-generating machine are dropped.pip-compile-cross-platform"carries forward" the environment markers into the created lockfile, which is why it is
able to work in all environments.Howpip-compile-cross-platformis able to do this is by leveraging the fantasticpoetrytool.poetryis a "dependency management" tool for Python projects, whose main downside is that it requires that all
developers/environments working with a project havepoetryinstalled. This isn't alargecost, but it still carries
migration risk, which is (my theory about) why many projects still usepip-compiletoday instead.So, part ofpoetry's core logic requires carefully resolving dependencies and keeping track of environment markers.pip-compile-cross-platformwraps around this logic, feeding in your project's top-level requirements and extracting
out apip-compatiblerequirements.txtfile.The best way to visualizepip-compile-cross-platformis that it's a thin wrapper aroundpoetrythat mimicks the
interface ofpip-compile: this allows projects that still usepip-compileto get the benefits ofpoetrywithout
having to make a significant time investment.