Repository: pypa/setuptools

Issue:
[FR] Cache supported tags in Wheel.is_compatible
### What's the problem this feature will solve?

Calling `is_compatible` on an instance of the `Wheel` class takes mere milliseconds. But when calling this on thousands of wheels, this quickly adds up.

I think this method is called when setuptools or pip reads an index page, for example https://pypi.org/simple/setuptools/, and for each link checks if the wheel is compatible with the current interpreter and platform. (Not completely sure if this is how it works.)

My own use case is with Buildout. If this downloads a distribution, it saves it in a directory. Buildout uses this directory as an extra find-link. So the next time you call buildout, these distributions are available. This can help for the case where you have no internet, or someone has removed a distribution from PyPI (which happens a lot less these days, I am glad to say.) With thousands of wheels in there, and Buildout/setuptools calling `is_compatible` on each wheel, this takes too much time.

I created an [issue in Buildout](https://github.com/buildout/buildout/issues/626) to track this, so some more details are there. There it seems it is worse with the combination of the very latest setuptools (67.0.0) and pip (23.0.0), and extra worse on Python 3.8 compared to 3.11. But this is a bit unclear.

### Describe the solution you'd like

This is fixable in the `_is_compatible` method in [`setuptools/wheel.py`](https://github.com/pypa/setuptools/blob/v67.0.0/setuptools/wheel.py#L85-L89) by calculating the supported tags once, outside of the class.
When I checked on my system, this gives a set of 1700 supported tags. With 1000 wheels, we would calculate 1.7 million tags. A tad much. ;-)

The assumption is that calling `sys_tags` from the vendored packaging returns the same result each time.

I am preparing a PR.


### Alternative Solutions

I suppose Buildout could add another to its existing [patches](https://github.com/buildout/buildout/blob/master/src/zc/buildout/patches.py), which already includes `setuptools.package_index.PackageIndex` which is involved here.
But I think pip would benefit from a faster method as well.

### Additional context

Here is a test file to get some timings on your own system. Prerequisite: a directory with some wheel, the more the better.

```
from setuptools.wheel import Wheel
from time import time

import os

DIR = "/Users/maurits/cached-downloads/dist"
print(f"Looking for compatible wheels in {DIR}...")
wheels = 0
compatible = 0
start = time()
for filename in os.listdir(DIR):
    if not filename.endswith(".whl"):
        continue
    wheel = Wheel(os.path.join(DIR, filename))
    wheels += 1
    if wheel.is_compatible():
        compatible += 1
stop = time()
print(f"""
Processed {wheels} wheels.
There were {compatible} compatible wheels.
Time taken: {stop - start} seconds.
""")
```

Save this as `test.py`.
With a clone of the pip repo and using the main branch, this is the result:

```
$ .tox/python/bin/python test.py 
Looking for compatible wheels in /Users/maurits/cached-downloads/dist...

Processed 2284 wheels.
There were 1776 compatible wheels.
Time taken: 7.127894639968872 seconds.
```

With my branch I get the same numbers, except the time:

```
Time taken: 0.04627823829650879 seconds.
```

That is about 150 times faster.

### Code of Conduct

- [X] I agree to follow the PSF Code of Conduct


Discussion:

Resolving patch:
diff --git a/setuptools/wheel.py b/setuptools/wheel.py
--- a/setuptools/wheel.py
+++ b/setuptools/wheel.py
@@ -2,6 +2,7 @@
 
 import email
 import itertools
+import functools
 import os
 import posixpath
 import re
@@ -28,6 +29,14 @@
     "__import__('pkg_resources').declare_namespace(__name__)\n"
 
 
+@functools.lru_cache(maxsize=None)
+def _get_supported_tags():
+    # We calculate the supported tags only once, otherwise calling
+    # this method on thousands of wheels takes seconds instead of
+    # milliseconds.
+    return {(t.interpreter, t.abi, t.platform) for t in sys_tags()}
+
+
 def unpack(src_dir, dst_dir):
     '''Move everything under `src_dir` to `dst_dir`, and delete the former.'''
     for dirpath, dirnames, filenames in os.walk(src_dir):
@@ -82,10 +91,8 @@ def tags(self):
         )
 
     def is_compatible(self):
-        '''Is the wheel is compatible with the current platform?'''
-        supported_tags = set(
-            (t.interpreter, t.abi, t.platform) for t in sys_tags())
-        return next((True for t in self.tags() if t in supported_tags), False)
+        '''Is the wheel compatible with the current platform?'''
+        return next((True for t in self.tags() if t in _get_supported_tags()), False)
 
     def egg_name(self):
         return _egg_basename(


Test patch:
diff --git a/setuptools/tests/test_wheel.py b/setuptools/tests/test_wheel.py
--- a/setuptools/tests/test_wheel.py
+++ b/setuptools/tests/test_wheel.py
@@ -609,9 +609,11 @@ def test_wheel_no_dist_dir():
 
 def test_wheel_is_compatible(monkeypatch):
     def sys_tags():
-        for t in parse_tag('cp36-cp36m-manylinux1_x86_64'):
-            yield t
-    monkeypatch.setattr('setuptools.wheel.sys_tags', sys_tags)
+        return {
+            (t.interpreter, t.abi, t.platform)
+            for t in parse_tag('cp36-cp36m-manylinux1_x86_64')
+        }
+    monkeypatch.setattr('setuptools.wheel._get_supported_tags', sys_tags)
     assert Wheel(
         'onnxruntime-0.1.2-cp36-cp36m-manylinux1_x86_64.whl').is_compatible()