Repository: pypa/pip

Issue:
Remove support for pip<20.0 cache entries
Cf https://github.com/pypa/pip/pull/7319/files#diff-e455c1048a1016c5fa2c413f52146488R142


Discussion:
A quick note. Pip 21.0 is due for release in about 2 weeks (see #9282). This item is on the 21.0 milestone, so it either needs to be implemented and merged to master before then, or it will miss the release (and I'll move it to the 21.1 milestone).
@pfmoore if there's any deprecations scheduled for this release, we'd either need to update the timeline in the codebose for them, or remove the deprecated code.

That's a consequence of how the `deprecated` function works - otherwise we'd actually create a hard break, since it raises an exception in the deprecated call instead of printing a warning.
I’m not quite sure what you mean here. Are you saying this is essential to avoid a broken release? If so, then as RM I need to go through the code checking for “deprecated in 21.0” calls and doing “something” for each of them. That’s fine and thanks for the heads up - is it covered in the “how to do a release” docs? Particularly what the “something” is... If it isn’t I should probably add it.

Also, should this be on the release issue rather than here? I’m not sure how it’s specific to this issue.
Hmm, I just checked. There are two `deprecated(... gone_in"21.0")` in the current code base:

* Extras after specifiers (`name>=1.0[extras]`)
* `git+git@` VCS specifiers

But thinking about it, after 21.0, `deprecated` will raise - is that not what we want to happen? The previously-deprecated approach is now an error? I'd be happy to accept PRs to give a *better* failure than the `raise PipDeprecationError` that `deprecated` raises, but that's mostly a cosmetic matter (and likely better handled by someone who understands the relevant code, not by the RM).

Did I miss your point here?
Actually I reviewed the link from this issue, and it looks like what you (and this issue) are saying is that because the original code was merged, old cache entries will get ignored once 20.0 gets released, without any warning.

Yep, that's what this issue is about - and I guess at this point, it's too late, as we won't have a release between now and 21.0 in which we warn. Looks like we missed the chance, so we either just close this issue, or we revert the change (which is likely hard, after all this time).

With my "release what's on master" philosophy, I'm going to say close this issue. But I'll leave it for now in case someone else wants to take a different approach. After 21.0 is released, if this is still open I'll just close it rather than moving it to the 21.0 milestone.
@pfmoore this particular issue is "just" about cleaning up legacy code. In #7319 (released in 20.0) the wheel cache layout was changed, but we made sure that pre-existing entries in the old layout were still used. What we implied in the release notes is that in 21.0 such entries would become ignored. There is no particular urgency, but since it should be easy to do and we announced it I'll see if I can do it.

Resolving patch:
diff --git a/src/pip/_internal/cache.py b/src/pip/_internal/cache.py
--- a/src/pip/_internal/cache.py
+++ b/src/pip/_internal/cache.py
@@ -55,34 +55,6 @@ def __init__(self, cache_dir, format_control, allowed_formats):
         _valid_formats = {"source", "binary"}
         assert self.allowed_formats.union(_valid_formats) == _valid_formats
 
-    def _get_cache_path_parts_legacy(self, link):
-        # type: (Link) -> List[str]
-        """Get parts of part that must be os.path.joined with cache_dir
-
-        Legacy cache key (pip < 20) for compatibility with older caches.
-        """
-
-        # We want to generate an url to use as our cache key, we don't want to
-        # just re-use the URL because it might have other items in the fragment
-        # and we don't care about those.
-        key_parts = [link.url_without_fragment]
-        if link.hash_name is not None and link.hash is not None:
-            key_parts.append("=".join([link.hash_name, link.hash]))
-        key_url = "#".join(key_parts)
-
-        # Encode our key url with sha224, we'll use this because it has similar
-        # security properties to sha256, but with a shorter total output (and
-        # thus less secure). However the differences don't make a lot of
-        # difference for our use case here.
-        hashed = hashlib.sha224(key_url.encode()).hexdigest()
-
-        # We want to nest the directories some to prevent having a ton of top
-        # level directories where we might run out of sub directories on some
-        # FS.
-        parts = [hashed[:2], hashed[2:4], hashed[4:6], hashed[6:]]
-
-        return parts
-
     def _get_cache_path_parts(self, link):
         # type: (Link) -> List[str]
         """Get parts of part that must be os.path.joined with cache_dir
@@ -139,17 +111,8 @@ def _get_candidates(self, link, canonical_package_name):
         if os.path.isdir(path):
             for candidate in os.listdir(path):
                 candidates.append((candidate, path))
-        # TODO remove legacy path lookup in pip>=21
-        legacy_path = self.get_path_for_link_legacy(link)
-        if os.path.isdir(legacy_path):
-            for candidate in os.listdir(legacy_path):
-                candidates.append((candidate, legacy_path))
         return candidates
 
-    def get_path_for_link_legacy(self, link):
-        # type: (Link) -> str
-        raise NotImplementedError()
-
     def get_path_for_link(self, link):
         # type: (Link) -> str
         """Return a directory to store cached items in for link.
@@ -177,12 +140,6 @@ def __init__(self, cache_dir, format_control):
         # type: (str, FormatControl) -> None
         super().__init__(cache_dir, format_control, {"binary"})
 
-    def get_path_for_link_legacy(self, link):
-        # type: (Link) -> str
-        parts = self._get_cache_path_parts_legacy(link)
-        assert self.cache_dir
-        return os.path.join(self.cache_dir, "wheels", *parts)
-
     def get_path_for_link(self, link):
         # type: (Link) -> str
         """Return a directory to store cached wheels for link
@@ -286,10 +243,6 @@ def __init__(self, cache_dir, format_control):
         self._wheel_cache = SimpleWheelCache(cache_dir, format_control)
         self._ephem_cache = EphemWheelCache(format_control)
 
-    def get_path_for_link_legacy(self, link):
-        # type: (Link) -> str
-        return self._wheel_cache.get_path_for_link_legacy(link)
-
     def get_path_for_link(self, link):
         # type: (Link) -> str
         return self._wheel_cache.get_path_for_link(link)


Test patch:
diff --git a/tests/unit/test_cache.py b/tests/unit/test_cache.py
--- a/tests/unit/test_cache.py
+++ b/tests/unit/test_cache.py
@@ -51,47 +51,6 @@ def test_cache_hash():
     assert h == "f83b32dfa27a426dec08c21bf006065dd003d0aac78e7fc493d9014d"
 
 
-def test_get_path_for_link_legacy(tmpdir):
-    """
-    Test that an existing cache entry that was created with the legacy hashing
-    mechanism is returned by WheelCache._get_candidates().
-    """
-    wc = WheelCache(tmpdir, FormatControl())
-    link = Link("https://g.c/o/r")
-    path = wc.get_path_for_link(link)
-    legacy_path = wc.get_path_for_link_legacy(link)
-    assert path != legacy_path
-    ensure_dir(path)
-    with open(os.path.join(path, "test-1.0.0-pyz-none-any.whl"), "w"):
-        pass
-    ensure_dir(legacy_path)
-    with open(os.path.join(legacy_path, "test-1.0.0-pyx-none-any.whl"), "w"):
-        pass
-    expected_candidates = {
-        "test-1.0.0-pyx-none-any.whl", "test-1.0.0-pyz-none-any.whl"
-    }
-    candidates = {c[0] for c in wc._get_candidates(link, "test")}
-    assert candidates == expected_candidates
-
-
-def test_get_with_legacy_entry_only(tmpdir):
-    """
-    Test that an existing cache entry that was created with the legacy hashing
-    mechanism is actually returned in WheelCache.get().
-    """
-    wc = WheelCache(tmpdir, FormatControl())
-    link = Link("https://g.c/o/r")
-    legacy_path = wc.get_path_for_link_legacy(link)
-    ensure_dir(legacy_path)
-    with open(os.path.join(legacy_path, "test-1.0.0-py3-none-any.whl"), "w"):
-        pass
-    cached_link = wc.get(link, "test", [Tag("py3", "none", "any")])
-    assert (
-        os.path.normcase(os.path.dirname(cached_link.file_path)) ==
-        os.path.normcase(legacy_path)
-    )
-
-
 def test_get_cache_entry(tmpdir):
     wc = WheelCache(tmpdir, FormatControl())
     persi_link = Link("https://g.c/o/r/persi")