Repository: jazzband/pip-tools

Issue:
piptools depcache is shared between implementations causing incorrect dependencies
<!-- Describe the issue briefly here. -->

#### Environment Versions

```console
$ uname -a
Linux asottile-MacBookPro 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
$ lsb_release -a
No LSB modules are available.
Distributor ID:	Ubuntu
Description:	Ubuntu 18.04.3 LTS
Release:	18.04
Codename:	bionic
$ /tmp/x/venv36/bin/pip freeze --all
Click==7.0
pip==20.0.2
pip-tools==4.4.0
setuptools==45.1.0
six==1.14.0
wheel==0.34.1
$ /tmp/x/venv36/bin/python --version --version
Python 3.6.9 (default, Nov  7 2019, 10:44:02) 
[GCC 8.3.0]
$ /tmp/x/venvpp36/bin/python --version --version
Python 3.6.9 (1608da62bfc7, Dec 23 2019, 10:50:04)
[PyPy 7.3.0 with GCC 7.3.1 20180303 (Red Hat 7.3.1-5)]
```

#### Steps to replicate

```bash
set -euxo pipefail

cache="$PWD/cache"
export XDG_CACHE_HOME="$cache"
export CUSTOM_COMPILE_COMMAND='<<redacted>>'

rm -rf "$cache" venv36 venvpp36 reqs*.txt

virtualenv venv36 -ppython3.6 >& /dev/null
venv36/bin/pip install -qq pip-tools
virtualenv venvpp36 -ppypy3 >& /dev/null
venvpp36/bin/pip install -qq pip-tools

echo 'cffi' > requirements.in

venv36/bin/pip-compile --output-file reqs36_1.txt
venvpp36/bin/pip-compile --output-file reqspp36_1.txt

rm -rf "$cache"
venvpp36/bin/pip-compile --output-file reqspp36_2.txt
venv36/bin/pip-compile --output-file reqs36_2.txt

diff -u reqs36_1.txt reqs36_2.txt
diff -u reqspp36_1.txt reqspp36_2.txt
```

#### Expected result

I expect no diff to be produced and the script to exit 0

#### Actual result

```console
$ bash t.sh 
+ cache=/tmp/x/cache
+ export XDG_CACHE_HOME=/tmp/x/cache
+ XDG_CACHE_HOME=/tmp/x/cache
+ export 'CUSTOM_COMPILE_COMMAND=<<redacted>>'
+ CUSTOM_COMPILE_COMMAND='<<redacted>>'
+ rm -rf /tmp/x/cache venv36 venvpp36 reqs36_1.txt reqs36_2.txt reqspp36_1.txt reqspp36_2.txt
+ virtualenv venv36 -ppython3.6
+ venv36/bin/pip install -qq pip-tools
+ virtualenv venvpp36 -ppypy3
+ venvpp36/bin/pip install -qq pip-tools
+ echo cffi
+ venv36/bin/pip-compile --output-file reqs36_1.txt
#
# This file is autogenerated by pip-compile
# To update, run:
#
#    <<redacted>>
#
cffi==1.13.2
pycparser==2.19           # via cffi
+ venvpp36/bin/pip-compile --output-file reqspp36_1.txt
#
# This file is autogenerated by pip-compile
# To update, run:
#
#    <<redacted>>
#
cffi==1.13.2
pycparser==2.19           # via cffi
+ rm -rf /tmp/x/cache
+ venvpp36/bin/pip-compile --output-file reqspp36_2.txt
#
# This file is autogenerated by pip-compile
# To update, run:
#
#    <<redacted>>
#
cffi==1.13.2
+ venv36/bin/pip-compile --output-file reqs36_2.txt
#
# This file is autogenerated by pip-compile
# To update, run:
#
#    <<redacted>>
#
cffi==1.13.2
+ diff -u reqs36_1.txt reqs36_2.txt
--- reqs36_1.txt	2020-01-30 08:46:36.913122945 -0800
+++ reqs36_2.txt	2020-01-30 08:46:41.696946374 -0800
@@ -5,4 +5,3 @@
 #    <<redacted>>
 #
 cffi==1.13.2
-pycparser==2.19           # via cffi
```


as you can see here, the pypy 3.6 cache is poisoning the result of the cpython 3.6 pip-compile


Discussion:

Resolving patch:
diff --git a/piptools/cache.py b/piptools/cache.py
--- a/piptools/cache.py
+++ b/piptools/cache.py
@@ -3,6 +3,7 @@
 
 import json
 import os
+import platform
 import sys
 
 from pip._vendor.packaging.requirements import Requirement
@@ -10,6 +11,17 @@
 from .exceptions import PipToolsError
 from .utils import as_tuple, key_from_req, lookup_table
 
+_PEP425_PY_TAGS = {"cpython": "cp", "pypy": "pp", "ironpython": "ip", "jython": "jy"}
+
+
+def _implementation_name():
+    """similar to PEP 425, however the minor version is separated from the
+    major to differentation "3.10" and "31.0".
+    """
+    implementation_name = platform.python_implementation().lower()
+    implementation = _PEP425_PY_TAGS.get(implementation_name, "??")
+    return "{}{}.{}".format(implementation, *sys.version_info)
+
 
 class CorruptCacheError(PipToolsError):
     def __init__(self, path):
@@ -45,14 +57,14 @@ class DependencyCache(object):
 
         ~/.cache/pip-tools/depcache-pyX.Y.json
 
+    Where py indicates the Python implementation.
     Where X.Y indicates the Python version.
     """
 
     def __init__(self, cache_dir):
         if not os.path.isdir(cache_dir):
             os.makedirs(cache_dir)
-        py_version = ".".join(str(digit) for digit in sys.version_info[:2])
-        cache_filename = "depcache-py{}.json".format(py_version)
+        cache_filename = "depcache-{}.json".format(_implementation_name())
 
         self._cache_file = os.path.join(cache_dir, cache_filename)
         self._cache = None


Test patch: