Repository: pypa/pip

Issue:
Wheel cache ignores subdirectory URL fragment
**Environment**

* pip version: master
* Python version: any
* OS: any

**Description**

Pip wheel cache ignores `subdirectory` URL fragment. This can result in the wrong wheel being downloaded or installed.

**Expected behavior**

URLs with different `subdirectory` fragments must not result in the same cache entry.

**How to Reproduce**

1. cd to an empty directory
2. `pip --cache-dir=$PWD/cache wheel --no-deps "git+https://github.com/OCA/mis-builder@c647390ee34d4b051ac1ae8094ebbf7fd222b08b#egg=odoo12-addon-mis_builder&subdirectory=setup/mis_builder"`
3. verify that `odoo12_addon_mis_builder-12.0.3.5.0.99.dev1-py3-none-any.whl` is created
4. same command with a different `subdirectory` and `egg` fragments: `pip --cache-dir=$PWD/cache wheel --no-deps "git+https://github.com/OCA/mis-builder@c647390ee34d4b051ac1ae8094ebbf7fd222b08b#egg=odoo12-addon-mis_builder_budget&subdirectory=setup/mis_builder_budget"`
5. a new wheel should be created, but is not, due to the cache not taking into account the `subdirectory` fragment (nor the package name since it assumes no two URLs can point to the same package)



Discussion:
The fix can be done in one of those two places, or both:
1. adding the `subdirectory` fragment in `key_parts` [here](https://github.com/pypa/pip/blob/deecb36de914da7e1fe7559b37f4281a671a2c80/src/pip/_internal/cache.py#L58-L61).
2. verifying that the canonicalized package name matches `wheel.name` in addition to supported tags [here](https://github.com/pypa/pip/blob/deecb36de914da7e1fe7559b37f4281a671a2c80/src/pip/_internal/cache.py#L176-L178).

I think 2. is more robust and sufficient.
The comment in `get_path_for_link` states

> Because there are M wheels for any one sdist, we provide a directory
        to cache them in, and then consult that directory when looking up
        cache hits.

which to me indicates the intent was one project (e.g. directory with setup.py) to one cache dir. In keeping with the same model we would want to add the subdirectory fragment to the cache key.

Adding a check for the wheel name would be good too as a sanity check. I think we can assert that the project name is not `None` at that point since otherwise we wouldn't have any candidates.

Resolving patch:
diff --git a/src/pip/_internal/cache.py b/src/pip/_internal/cache.py
--- a/src/pip/_internal/cache.py
+++ b/src/pip/_internal/cache.py
@@ -58,6 +58,10 @@ def _get_cache_path_parts(self, link):
         key_parts = [link.url_without_fragment]
         if link.hash_name is not None and link.hash is not None:
             key_parts.append("=".join([link.hash_name, link.hash]))
+        if link.subdirectory_fragment:
+            key_parts.append(
+                "=".join(["subdirectory", link.subdirectory_fragment])
+            )
         key_url = "#".join(key_parts)
 
         # Encode our key url with sha224, we'll use this because it has similar
@@ -73,19 +77,18 @@ def _get_cache_path_parts(self, link):
 
         return parts
 
-    def _get_candidates(self, link, package_name):
+    def _get_candidates(self, link, canonical_package_name):
         # type: (Link, Optional[str]) -> List[Any]
         can_not_cache = (
             not self.cache_dir or
-            not package_name or
+            not canonical_package_name or
             not link
         )
         if can_not_cache:
             return []
 
-        canonical_name = canonicalize_name(package_name)
         formats = self.format_control.get_allowed_formats(
-            canonical_name
+            canonical_package_name
         )
         if not self.allowed_formats.intersection(formats):
             return []
@@ -168,11 +171,23 @@ def get(
         # type: (...) -> Link
         candidates = []
 
-        for wheel_name in self._get_candidates(link, package_name):
+        if not package_name:
+            return link
+
+        canonical_package_name = canonicalize_name(package_name)
+        for wheel_name in self._get_candidates(link, canonical_package_name):
             try:
                 wheel = Wheel(wheel_name)
             except InvalidWheelFilename:
                 continue
+            if canonicalize_name(wheel.name) != canonical_package_name:
+                logger.debug(
+                    "Ignoring cached wheel {} for {} as it "
+                    "does not match the expected distribution name {}.".format(
+                        wheel_name, link, package_name
+                    )
+                )
+                continue
             if not wheel.supported(supported_tags):
                 # Built for a different python/arch/etc
                 continue


Test patch:
diff --git a/tests/unit/test_cache.py b/tests/unit/test_cache.py
--- a/tests/unit/test_cache.py
+++ b/tests/unit/test_cache.py
@@ -1,13 +1,44 @@
+import os
+
 from pip._internal.cache import WheelCache
+from pip._internal.models.format_control import FormatControl
+from pip._internal.models.link import Link
 from pip._internal.utils.compat import expanduser
+from pip._internal.utils.misc import ensure_dir
+
+
+def test_expands_path():
+    wc = WheelCache("~/.foo/", None)
+    assert wc.cache_dir == expanduser("~/.foo/")
+
+
+def test_falsey_path_none():
+    wc = WheelCache(False, None)
+    assert wc.cache_dir is None
 
 
-class TestWheelCache:
+def test_subdirectory_fragment():
+    """
+    Test the subdirectory URL fragment is part of the cache key.
+    """
+    wc = WheelCache("~/.foo/", None)
+    link1 = Link("git+https://g.c/o/r#subdirectory=d1")
+    link2 = Link("git+https://g.c/o/r#subdirectory=d2")
+    assert wc.get_path_for_link(link1) != wc.get_path_for_link(link2)
 
-    def test_expands_path(self):
-        wc = WheelCache("~/.foo/", None)
-        assert wc.cache_dir == expanduser("~/.foo/")
 
-    def test_falsey_path_none(self):
-        wc = WheelCache(False, None)
-        assert wc.cache_dir is None
+def test_wheel_name_filter(tmpdir):
+    """
+    Test the wheel cache filters on wheel name when several wheels
+    for different package are stored under the same cache directory.
+    """
+    wc = WheelCache(tmpdir, FormatControl())
+    link = Link("https://g.c/package.tar.gz")
+    cache_path = wc.get_path_for_link(link)
+    ensure_dir(cache_path)
+    with open(os.path.join(cache_path, "package-1.0-py3-none-any.whl"), "w"):
+        pass
+    # package matches wheel name
+    assert wc.get(link, "package", [("py3", "none", "any")]) is not link
+    # package2 does not match wheel name
+    assert wc.get(link, "package2", [("py3", "none", "any")]) is link