Repository: pypa/pip

Issue:
"No matching distribution" due to wheel tags not matching in pip 20.0
**Environment**

* pip version: 20.0.1
* Python version: 3.6.9
* OS: Ubuntu 18.04

<!-- Feel free to add more information about your environment here -->

**Description**
Pip can seemingly no longer install any version of `mxnet` newer than 0.9.5.

**Expected behavior**
It should be able to. :-) This worked before pip 20.

**How to Reproduce**
Try to install `mxnet==1.3.1` in a virtual environment.

**Output**

```
$ virtualenv -ppython3 /tmp/venv
Already using interpreter /usr/bin/python3
Using base prefix '/usr'
New python executable in /tmp/venv/bin/python3
Also creating executable in /tmp/venv/bin/python
Installing setuptools, pkg_resources, pip, wheel...done.
$ /tmp/venv/bin/pip install mxnet==1.3.1
ERROR: Could not find a version that satisfies the requirement mxnet==1.3.1 (from versions: 0.9.5)
ERROR: No matching distribution found for mxnet==1.3.1
```

Running `pip install` with `--verbose` produces a huge log, in which this seems relevant:

```
  Skipping link: none of the wheel's tags match: py2-none-manylinux1_x86_64, py3-none-manylinux1_x86_64: https://files.pythonhosted.org/packages/f0/2e/b26eb7273aed1945f59993b3b306442eb41684f931b5380821c39cf50a31/mxnet-1.3.1-py2.py3-none-manylinux1_x86_64.whl#sha256=939575fddd45e8ba39177dd3d53ccce64dea312bc08f493392b1ecace9e1b117 (from https://pypi.org/simple/mxnet/)
```


Discussion:
We are also having this error using version 20.0.1 with our in-house wheel
```
(venv) C:\depot\bitbucket\mytests\tests_pti>pip -vvv install C:\Users\otrejoso\Downloads\pti-2.0.510-py3-none-win_amd64.whl
Non-user install because user site-packages disabled
Created temporary directory: C:\Users\otrejoso\AppData\Local\Temp\pip-ephem-wheel-cache-wquw3si6
Created temporary directory: C:\Users\otrejoso\AppData\Local\Temp\pip-req-tracker-ik56de2r
Initialized build tracking at C:\Users\otrejoso\AppData\Local\Temp\pip-req-tracker-ik56de2r
Created build tracker: C:\Users\otrejoso\AppData\Local\Temp\pip-req-tracker-ik56de2r
Entered build tracker: C:\Users\otrejoso\AppData\Local\Temp\pip-req-tracker-ik56de2r
Created temporary directory: C:\Users\otrejoso\AppData\Local\Temp\pip-install-vb0u5yy4
Cleaning up...
Removed build tracker: 'C:\\Users\\otrejoso\\AppData\\Local\\Temp\\pip-req-tracker-ik56de2r'
ERROR: pti-2.0.510-py3-none-win_amd64.whl is not a supported wheel on this platform.
Exception information:
....
pip._internal.exceptions.InstallationError: pti-2.0.510-py3-none-win_amd64.whl is not a supported wheel on this platform.
```
Using `pip install pip==19.3.1`  works fine.
Same here with an in-house wheel.

Doesn't work:
` pip install -U pip==20.0.1; pip install <wheel>
`
ERROR: <wheel> is not a supported wheel on this platform.

Works:
` pip install -U pip==19.3.1; pip install <wheel>
`

Seems that the platform tags are the problem here: tags 'any' are working, but this specify wheel has 'linux_x86_64'.

Note that I have:
```
uname -a
Linux <propretiery> 3.10.0-957.5.1.el7.x86_64 #1 SMP Fri Feb 1 14:54:57 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

 python -c "import wheel.pep425tags as w; print(w.get_supported())"
[('cp27', 'cp27mu', 'linux_x86_64'), ('cp27', 'none', 'linux_x86_64'), ('cp27', 'none', 'any'), ('cp2', 'none', 'any'), ('cp26', 'none', 'any'), ('cp25', 'none', 'any'), ('cp24', 'none', 'any'), ('cp23', 'none', 'any'), ('cp22', 'none', 'any'), ('cp21', 'none', 'any'), ('cp20', 'none', 'any'), ('py2', 'none', 'linux_x86_64'), ('py27', 'none', 'any'), ('py2', 'none', 'any'), ('py26', 'none', 'any'), ('py25', 'none', 'any'), ('py24', 'none', 'any'), ('py23', 'none', 'any'), ('py22', 'none', 'any'), ('py21', 'none', 'any'), ('py20', 'none', 'any')]

```
Same here.

19.3.1 works but 20.0.1 gives:
pip._internal.exceptions.InstallationError: pyenchant-2.0.0-py2.py3.cp27.cp32.cp33.cp34.cp35.cp36.pp27.pp33.pp35-none-win32.whl is not a supported wheel on this platform.

Tags for my pc: [('cp37', 'cp37m', 'win32'), ('cp37', 'none', 'win32'), ('cp37', 'none', 'any'), ('cp3', 'none', 'any'), ('cp36', 'none', 'any'), ('cp35', 'none', 'any'), ('cp34', 'none', 'any'), ('cp33', 'none', 'any'), ('cp32', 'none', 'any'), ('cp31', 'none', 'any'), ('cp30', 'none', 'any'), ('py3', 'none', 'win32'), ('py37', 'none', 'any'), ('py3', 'none', 'any'), ('py36', 'none', 'any'), ('py35', 'none', 'any'), ('py34', 'none', 'any'), ('py33', 'none', 'any'), ('py32', 'none', 'any'), ('py31', 'none', 'any'), ('py30', 'none', 'any')]

Tags for file can be seen in filename.

Could you print the differences between `pip debug -v` in pip 20.0.1 & pip 19.3.1 ?
```diff
--- /tmp/old.txt	2020-01-21 17:22:10.221211433 +0300
+++ /tmp/new.txt	2020-01-21 17:22:30.725552363 +0300
@@ -1,4 +1,4 @@
-pip version: pip 19.3.1 from /tmp/venv/lib/python3.6/site-packages/pip (python 3.6)
+pip version: pip 20.0.1 from /tmp/venv/lib/python3.6/site-packages/pip (python 3.6)
 sys.version: 3.6.9 (default, Nov  7 2019, 10:44:02) 
 [GCC 8.3.0]
 sys.executable: /tmp/venv/bin/python3
@@ -8,7 +8,11 @@
 sys.platform: linux
 sys.implementation:
   name: cpython
-Compatible tags: 42
+'cert' config value: global
+REQUESTS_CA_BUNDLE: None
+CURL_CA_BUNDLE: None
+pip._vendor.certifi.where(): /tmp/venv/lib/python3.6/site-packages/pip/_vendor/certifi/cacert.pem
+Compatible tags: 41
   cp36-cp36m-manylinux2014_x86_64
   cp36-cp36m-manylinux2010_x86_64
   cp36-cp36m-manylinux1_x86_64
@@ -37,12 +41,11 @@
   cp32-abi3-manylinux2010_x86_64
   cp32-abi3-manylinux1_x86_64
   cp32-abi3-linux_x86_64
-  py3-none-manylinux2014_x86_64
-  py3-none-manylinux2010_x86_64
-  py3-none-manylinux1_x86_64
-  py3-none-linux_x86_64
+  py36-none-manylinux2014_x86_64
+  py36-none-manylinux2010_x86_64
+  py36-none-manylinux1_x86_64
+  py36-none-linux_x86_64
   cp36-none-any
-  cp3-none-any
   py36-none-any
   py3-none-any
   py35-none-any
```
```diff
-pip version: pip 19.3.1 from c:\sdks\python37-32\lib\site-packages\pip (python 3.7)
+pip version: pip 20.0.1 from c:\sdks\python37-32\lib\site-packages\pip (python 3.7)
 sys.version: 3.7.6 (tags/v3.7.6:43364a7ae0, Dec 18 2019, 23:46:00) [MSC v.1916 32 bit (Intel)]
 sys.executable: c:\sdks\python37-32\python.exe
 sys.getdefaultencoding: utf-8
@@ -8,14 +8,21 @@ locale.getpreferredencoding: cp1252
 sys.platform: win32
 sys.implementation:
   name: cpython
-Config variable 'Py_DEBUG' is unset, Python ABI tag may be incorrect
-Config variable 'WITH_PYMALLOC' is unset, Python ABI tag may be incorrect
-Compatible tags: 14
+'cert' config value: global
+REQUESTS_CA_BUNDLE: None
+CURL_CA_BUNDLE: None
+pip._vendor.certifi.where(): c:\sdks\python37-32\lib\site-packages\pip\_vendor\certifi\cacert.pem
+Compatible tags: 19
   cp37-cp37m-win32
+  cp37-abi3-win32
   cp37-none-win32
-  py3-none-win32
+  cp36-abi3-win32
+  cp35-abi3-win32
+  cp34-abi3-win32
+  cp33-abi3-win32
+  cp32-abi3-win32
+  py37-none-win32
   cp37-none-any
-  cp3-none-any
   py37-none-any
   py3-none-any
   py36-none-any
Similar on Windows - the tag section of the output:
```diff
--- ".\\pip19.txt"      2020-01-21 14:30:16 +0000
+++ ".\\pip20.txt"      2020-01-21 14:26:54 +0000
@@ -1,9 +1,15 @@
-Compatible tags: 15
+Compatible tags: 21
   cp38-cp38-win_amd64
+  cp38-abi3-win_amd64
   cp38-none-win_amd64
-  py3-none-win_amd64
+  cp37-abi3-win_amd64
+  cp36-abi3-win_amd64
+  cp35-abi3-win_amd64
+  cp34-abi3-win_amd64
+  cp33-abi3-win_amd64
+  cp32-abi3-win_amd64
+  py38-none-win_amd64
   cp38-none-any
-  cp3-none-any
   py38-none-any
   py3-none-any
   py37-none-any
```

Looks like `packaging.tags` has different values than the version pip used internally in pip 19. The main difference is the lack of `{py3,cp3}-none-win_amd64`. Which isn't a standard tag generated by `bdist_wheel` AFAIK, so at least the impact will be limited to people who set custom tags.

The specs don't say much about what custom tags like this are valid, so this is arguably in the area of "undefined behaviour". While that doesn't help the people affected by this, it does suggest that being more specific in the standards would be good.

BTW, I'm not entirely sure what `mxnet-1.5.1.post0-py2.py3-none-manylinux1_x86_64.whl` even *means* - the MacOS releases of mxnet have a specific ABI set, why does the manylinux build not? Numpy's manylinux builds have an ABI, so it doesn't appear to be a general issue with the manylinux toolchain. The tags for pyenchant also look a bit bizarre...
> the MacOS releases of mxnet have a specific ABI set, why does the manylinux build not?

I briefly checked the Linux package, and it appears that none of the native libraries there refer to Python symbols. It looks like MXNet uses `ctypes` for interop with native code, so not having an ABI makes sense.
Have the same issue installing  icc-rt  (from intel-numpy) (2020.0.133) using pip==20.0.1
> I briefly checked the Linux package, and it appears that none of the native libraries there refer to Python symbols. It looks like MXNet uses ctypes for interop with native code, so not having an ABI makes sense.

OK. Why does it need a "manylinux" tag in that case, if it's using ctypes for everything? Actually, don't spend any time on that question, I'm not a Linux expert so I probably wouldn't follow the answer anyway.

At a minimum, this sounds like it should be raised as an issue against the `packaging` library. Regardless of what pip does, if these are valid tags they should be supported in `packaging.tags`, and a general discussion of what tags should be supported is probably better had there than here.
> OK. Why does it need a "manylinux" tag in that case, if it's using ctypes for everything? Actually, don't spend any time on that question, I'm not a Linux expert so I probably wouldn't follow the answer anyway.

I will answer anyway: the wheel has native Linux libraries in it, so the `manylinux1` tag makes sense.
In https://github.com/pypa/pip/issues/7620#issuecomment-576743862 @tomasaschan reported what I think is this same issue for `xgboost`, which ships as `xgboost-0.90-py2.py3-none-manylinux1_x86_64.whl`. It also appears to contain native libs, perhaps for the JVM.
@IRDonch Thanks. I *did* actually follow that explanation 🙂 Makes sense.
@jamadden Agreed, that looks like the same issue.
@jamadden What can I do locally to help you determine whether this is the same?
@tomasaschan Can you paste here the output of `pip debug -v`?
```diff
 λ diff pip19.log pip20.log 
1c1
- pip version: pip 19.3.1 from /usr/local/lib/python3.6/dist-packages/pip (python 3.6)
---
+ pip version: pip 20.0.1 from /usr/local/lib/python3.6/dist-packages/pip (python 3.6)
11c11,15
- Compatible tags: 42
---
+ 'cert' config value: global
+ REQUESTS_CA_BUNDLE: None
+ CURL_CA_BUNDLE: None
+ pip._vendor.certifi.where(): /usr/local/lib/python3.6/dist-packages/pip/_vendor/certifi/cacert.pem
+ Compatible tags: 41
40,43c44,47
-   py3-none-manylinux2014_x86_64
-   py3-none-manylinux2010_x86_64
-   py3-none-manylinux1_x86_64
-   py3-none-linux_x86_64
---
+   py36-none-manylinux2014_x86_64
+   py36-none-manylinux2010_x86_64
+   py36-none-manylinux1_x86_64
+   py36-none-linux_x86_64
45d48
-   cp3-none-any
```
<details>

```
 λ cat pip19.log 
pip version: pip 19.3.1 from /usr/local/lib/python3.6/dist-packages/pip (python 3.6)
sys.version: 3.6.9 (default, Nov  7 2019, 10:44:02) 
[GCC 8.3.0]
sys.executable: /usr/bin/python
sys.getdefaultencoding: utf-8
sys.getfilesystemencoding: utf-8
locale.getpreferredencoding: UTF-8
sys.platform: linux
sys.implementation:
  name: cpython
Compatible tags: 42
  cp36-cp36m-manylinux2014_x86_64
  cp36-cp36m-manylinux2010_x86_64
  cp36-cp36m-manylinux1_x86_64
  cp36-cp36m-linux_x86_64
  cp36-abi3-manylinux2014_x86_64
  cp36-abi3-manylinux2010_x86_64
  cp36-abi3-manylinux1_x86_64
  cp36-abi3-linux_x86_64
  cp36-none-manylinux2014_x86_64
  cp36-none-manylinux2010_x86_64
  cp36-none-manylinux1_x86_64
  cp36-none-linux_x86_64
  cp35-abi3-manylinux2014_x86_64
  cp35-abi3-manylinux2010_x86_64
  cp35-abi3-manylinux1_x86_64
  cp35-abi3-linux_x86_64
  cp34-abi3-manylinux2014_x86_64
  cp34-abi3-manylinux2010_x86_64
  cp34-abi3-manylinux1_x86_64
  cp34-abi3-linux_x86_64
  cp33-abi3-manylinux2014_x86_64
  cp33-abi3-manylinux2010_x86_64
  cp33-abi3-manylinux1_x86_64
  cp33-abi3-linux_x86_64
  cp32-abi3-manylinux2014_x86_64
  cp32-abi3-manylinux2010_x86_64
  cp32-abi3-manylinux1_x86_64
  cp32-abi3-linux_x86_64
  py3-none-manylinux2014_x86_64
  py3-none-manylinux2010_x86_64
  py3-none-manylinux1_x86_64
  py3-none-linux_x86_64
  cp36-none-any
  cp3-none-any
  py36-none-any
  py3-none-any
  py35-none-any
  py34-none-any
  py33-none-any
  py32-none-any
  py31-none-any
  py30-none-any
```

```
 λ cat pip20.log 
pip version: pip 20.0.1 from /usr/local/lib/python3.6/dist-packages/pip (python 3.6)
sys.version: 3.6.9 (default, Nov  7 2019, 10:44:02) 
[GCC 8.3.0]
sys.executable: /usr/bin/python
sys.getdefaultencoding: utf-8
sys.getfilesystemencoding: utf-8
locale.getpreferredencoding: UTF-8
sys.platform: linux
sys.implementation:
  name: cpython
'cert' config value: global
REQUESTS_CA_BUNDLE: None
CURL_CA_BUNDLE: None
pip._vendor.certifi.where(): /usr/local/lib/python3.6/dist-packages/pip/_vendor/certifi/cacert.pem
Compatible tags: 41
  cp36-cp36m-manylinux2014_x86_64
  cp36-cp36m-manylinux2010_x86_64
  cp36-cp36m-manylinux1_x86_64
  cp36-cp36m-linux_x86_64
  cp36-abi3-manylinux2014_x86_64
  cp36-abi3-manylinux2010_x86_64
  cp36-abi3-manylinux1_x86_64
  cp36-abi3-linux_x86_64
  cp36-none-manylinux2014_x86_64
  cp36-none-manylinux2010_x86_64
  cp36-none-manylinux1_x86_64
  cp36-none-linux_x86_64
  cp35-abi3-manylinux2014_x86_64
  cp35-abi3-manylinux2010_x86_64
  cp35-abi3-manylinux1_x86_64
  cp35-abi3-linux_x86_64
  cp34-abi3-manylinux2014_x86_64
  cp34-abi3-manylinux2010_x86_64
  cp34-abi3-manylinux1_x86_64
  cp34-abi3-linux_x86_64
  cp33-abi3-manylinux2014_x86_64
  cp33-abi3-manylinux2010_x86_64
  cp33-abi3-manylinux1_x86_64
  cp33-abi3-linux_x86_64
  cp32-abi3-manylinux2014_x86_64
  cp32-abi3-manylinux2010_x86_64
  cp32-abi3-manylinux1_x86_64
  cp32-abi3-linux_x86_64
  py36-none-manylinux2014_x86_64
  py36-none-manylinux2010_x86_64
  py36-none-manylinux1_x86_64
  py36-none-linux_x86_64
  cp36-none-any
  py36-none-any
  py3-none-any
  py35-none-any
  py34-none-any
  py33-none-any
  py32-none-any
  py31-none-any
  py30-none-any
```
</details>
```diff
pip/_vendor/packaging/tags.py
332c332
-         platforms = _platform_tags
---
+         platforms = _platform_tags()
334c334
-         for platform_ in platforms():
---
+         for platform_ in platforms:
```

seems to solve the problem
Here's a Dockerfile that reproduces our error message:

```
FROM ubuntu:bionic-20190912.1

RUN set -ex \
  && apt-get update \
  && apt-get install -y --no-install-recommends \
  ca-certificates \
  python3 python3-dev python3-pip

RUN pip3 install --upgrade pip==20.0.1 setuptools

RUN echo "xgboost==0.81" >> requirements.txt

RUN pip3 install -r requirements.txt
```
@jeroendecroos Good catch - that looks like it may be a straight bug in `packaging.tags` (re-using an iterator rather than re-creating it each time). Could you open an issue against https://github.com/pypa/packaging for this - and if you could make your fix into a PR that would be even better!
Not sure if this helps but I'm facing the same issue trying to install `dotnetcore2`
Facing the same issue with freetype-py on macOS: https://github.com/rougier/freetype-py/issues/119 ("fixed" by pinning to 19.3.1)
Expect a bugfix release for this tomorrow, assuming I recover from my current headache by then. :)
Same issue with our in-house wheels (pip 20.0.1), a workaround is to use pip<20 for now. Hopefully your upcoming today's fix will resolve it. Thanks!

Resolving patch:
diff --git a/src/pip/_vendor/packaging/__about__.py b/src/pip/_vendor/packaging/__about__.py
--- a/src/pip/_vendor/packaging/__about__.py
+++ b/src/pip/_vendor/packaging/__about__.py
@@ -18,7 +18,7 @@
 __summary__ = "Core utilities for Python packages"
 __uri__ = "https://github.com/pypa/packaging"
 
-__version__ = "20.0"
+__version__ = "20.1"
 
 __author__ = "Donald Stufft and individual contributors"
 __email__ = "donald@stufft.io"
diff --git a/src/pip/_vendor/packaging/tags.py b/src/pip/_vendor/packaging/tags.py
--- a/src/pip/_vendor/packaging/tags.py
+++ b/src/pip/_vendor/packaging/tags.py
@@ -315,7 +315,7 @@ def _py_interpreter_range(py_version):
 def compatible_tags(
     python_version=None,  # type: Optional[PythonVersion]
     interpreter=None,  # type: Optional[str]
-    platforms=None,  # type: Optional[Iterator[str]]
+    platforms=None,  # type: Optional[Iterable[str]]
 ):
     # type: (...) -> Iterator[Tag]
     """
@@ -328,8 +328,7 @@ def compatible_tags(
     """
     if not python_version:
         python_version = sys.version_info[:2]
-    if not platforms:
-        platforms = _platform_tags()
+    platforms = list(platforms or _platform_tags())
     for version in _py_interpreter_range(python_version):
         for platform_ in platforms:
             yield Tag(version, "none", platform_)


Test patch: