Repository: jazzband/pip-tools

Issue:
Generating hashes from files for incompatible wheels isn't working
<!-- Describe the issue briefly here. -->

#### Environment Versions

1. OS Type: macOS
1. Python version: 3.9.6
1. pip version: 21.3.1
1. pip-tools version: 6.4.0

#### Steps to replicate

(These replication steps require that you run this on a machine **not** compatible with the manylinux wheels provided in the [demo repository](https://gitlab.com/RazerM/pip-tools-repro/-/packages/3680771) used in the following commands)

1. Create a requirements file using the project I prepared

   ```bash
   echo regex > requirements.in
   pip-compile --index-url https://gitlab.com/api/v4/projects/31206336/packages/pypi/simple --no-header --generate-hashes -o normal.txt
   ```
2. Patch the installed pip to remove [this line](https://github.com/pypa/pip/blob/d81c65ace124d3cf903909bc770272566b9b832c/src/pip/_internal/index/package_finder.py#L779).
3. Generate a new requirements file:

   ```bash
   pip-compile --index-url https://gitlab.com/api/v4/projects/31206336/packages/pypi/simple --no-header --generate-hashes -o patched.txt
   ```
4. Compare the files
   ```bash
   git diff --no-index normal.txt patched.txt
   ```

#### Expected result

Both files should include all hashes

#### Actual result

```
diff --git a/normal.txt b/patched.txt
index da086ab..23bc894 100644
--- a/normal.txt
+++ b/patched.txt
@@ -1,5 +1,10 @@
 --index-url https://gitlab.com/api/v4/projects/31206336/packages/pypi/simple

 regex==2021.11.10 \
+    --hash=sha256:05b7d6d7e64efe309972adab77fc2af8907bb93217ec60aa9fe12a0dad35874f \
+    --hash=sha256:30ab804ea73972049b7a2a5c62d97687d69b5a60a67adca07eb73a0ddbc9e29f \
+    --hash=sha256:85bfa6a5413be0ee6c5c4a663668a2cad2cbecdee367630d097d7823041bdeec \
+    --hash=sha256:962b9a917dd7ceacbe5cd424556914cb0d636001e393b43dc886ba31d2a1e449 \
+    --hash=sha256:ca5f18a75e1256ce07494e245cdb146f5a9267d3c702ebf9b65c7f8bd843431e \
     --hash=sha256:f341ee2df0999bfdf7a95e448075effe0db212a59387de1a70690e4acb03d4c6
     # via -r requirements.in
```

Only hashes compatible with my system are generated.

The easy solution I see would be to call `self.finder.find_all_candidates.cache_clear()` in `PyPIRepository.allow_all_wheels`, but as maintainers of the library you'll probably have more insight. Maybe even creating a new finder instance rather than rely on this implementation detail?



Discussion:

Resolving patch:
diff --git a/piptools/repositories/pypi.py b/piptools/repositories/pypi.py
--- a/piptools/repositories/pypi.py
+++ b/piptools/repositories/pypi.py
@@ -436,6 +436,10 @@ def _wheel_support_index_min(self: Wheel, tags: List[Tag]) -> int:
         Wheel.support_index_min = _wheel_support_index_min
         self._available_candidates_cache = {}
 
+        # If we don't clear this cache then it can contain results from an
+        # earlier call when allow_all_wheels wasn't active. See GH-1532
+        self.finder.find_all_candidates.cache_clear()
+
         try:
             yield
         finally:


Test patch:
diff --git a/tests/test_data/minimal_wheels/small-fake-multi-arch-0.1.tar.gz b/tests/test_data/minimal_wheels/small-fake-multi-arch-0.1.tar.gz
new file mode 100644
diff --git a/tests/test_repository_pypi.py b/tests/test_repository_pypi.py
--- a/tests/test_repository_pypi.py
+++ b/tests/test_repository_pypi.py
@@ -15,9 +15,16 @@ def test_generate_hashes_all_platforms(capsys, pip_conf, from_line, pypi_reposit
         "sha256:8d4d131cd05338e09f461ad784297efea3652e542c5fabe04a62358429a6175e",
         "sha256:ad05e1371eb99f257ca00f791b755deb22e752393eb8e75bc01d651715b02ea9",
         "sha256:24afa5b317b302f356fd3fc3b1cfb0aad114d509cf635ea9566052424191b944",
+        "sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
     }
 
     ireq = from_line("small-fake-multi-arch==0.1")
+
+    # pip caches the candidates for the current system, which means
+    # allow_all_wheels won't have the desired effect unless the cache is
+    # cleared. See GH-1532
+    assert pypi_repository.get_hashes(ireq) < expected
+
     with pypi_repository.allow_all_wheels():
         assert pypi_repository.get_hashes(ireq) == expected
     captured = capsys.readouterr()