Repository: pytorch/pytorch

Issue:
PyTorch 1.13 cannot be installed using `poetry` on Mac M1
### 🐛 Describe the bug

This is an installation issue.

Poetry parses the endpoint at `https://pypi.org/pypi/torch/1.13/json` to get dependency metadata.  The endpoint returns:

```   
"requires_dist": [
      "typing-extensions",
      "nvidia-cuda-runtime-cu11 (==11.7.99)",
      "nvidia-cudnn-cu11 (==8.5.0.96)",
      "nvidia-cublas-cu11 (==11.10.3.66)",
      "nvidia-cuda-nvrtc-cu11 (==11.7.99)",
      "opt-einsum (>=3.3) ; extra == 'opt-einsum'"
    ],
```

However, the packages `nvidia-cuda-runtime-cu11` , `nvidia-cudnn-cu11`, `nvidia-cuda-nvrtc-cu11`, and `nvidia-cuda-nvrtc-cu11` do not exist for M1, and `poetry update` fails with 

```
  Unable to find installation candidates for nvidia-cublas-cu11 (11.10.3.66)
  Unable to find installation candidates for nvidia-cuda-nvrtc-cu11 (11.7.99)
  Unable to find installation candidates for nvidia-cuda-runtime-cu11 (11.7.99)
  Unable to find installation candidates for nvidia-cudnn-cu11 (8.5.0.96)
```

This is specific to v1.13.  This problem did not exist in v1.12.1

### Versions

This probably isn't relevant until after it's installed, but I get:

```
Collecting environment information...
PyTorch version: 1.13.0
Is debug build: False
CUDA used to build PyTorch: None
ROCM used to build PyTorch: N/A

OS: macOS 12.6 (arm64)
GCC version: Could not collect
Clang version: 14.0.0 (clang-1400.0.29.102)
CMake version: Could not collect
Libc version: N/A

Python version: 3.10.8 (main, Oct 13 2022, 09:48:40) [Clang 14.0.0 (clang-1400.0.29.102)] (64-bit runtime)
Python platform: macOS-12.6-arm64-arm-64bit
Is CUDA available: False
CUDA runtime version: No CUDA
CUDA_MODULE_LOADING set to: N/A
GPU models and configuration: No CUDA
Nvidia driver version: No CUDA
cuDNN version: No CUDA
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

Versions of relevant libraries:
[pip3] mypy==0.982
[pip3] mypy-extensions==0.4.3
[pip3] numpy==1.23.4
[pip3] torch==1.13.0
[conda] Could not collect
```

cc @ezyang @seemethere @malfet


Discussion:
cc @malfet @seemethere 

This issue maybe related to pypi repo ? I just downloaded the M1 version of wheel from pypi:
```
https://files.pythonhosted.org/packages/79/b3/eaea3fc35d0466b9dae1e3f9db08467939347b3aaa53c0fd81953032db33/torch-1.13.0-cp310-none-macosx_11_0_arm64.whl
```

I see following Metadata requirements:
```
Requires-Python: >=3.7.0
Description-Content-Type: text/markdown
License-File: LICENSE
License-File: NOTICE
Requires-Dist: typing-extensions
Provides-Extra: opt-einsum
Requires-Dist: opt-einsum (>=3.3) ; extra == 'opt-einsum'
```

No mentions of the
```
"nvidia-cuda-runtime-cu11 (==11.7.99)",
"nvidia-cudnn-cu11 (==8.5.0.96)",
"nvidia-cublas-cu11 (==11.10.3.66)",
"nvidia-cuda-nvrtc-cu11 (==11.7.99)",
```

These are Linux specific requirements that  should not be used in Mac M1
Dependencies to linux package should have `; platform_system=="Linux"` suffix to make it more friendly to poetry
FWIW, this also affects Macs with Intel chips because (at a minimum), nvidia-cuda-nvrtc-cu11 does not supply any Mac wheels on PyPI. Hence, Poetry goes out and finds nothing and throws a RuntimeError about installing torch 1.13.0.

Specifying the `; platform_system=="Linux"` should address this for Intel Macs, though.
The issue here is that Poetry does not support split metadata across wheels; instead, markers should be used with unified metadata. This is not just a Poetry decision; PyPI's metadata API currently gathers metadata from the first distfile to be uploaded. Until such time that [PEP 658](https://peps.python.org/pep-0658/) is implemented in PyPI, having inconsistent metadata between distfiles is likely to confuse non-Pip tools.
`torch>1.12` also can't be installed on Windows with Intel Chip. `pip` and `conda` installatoin works fine however. Also, `torch<1.13` installation via `poetry` works. This might not be a `torch` issue and seems to be related how `poetry` chooses the wheel files but I just wanted to mention that it might be a workaround for someone looking for this to limit `torch<1.13` if `poetry` is mandatory.
I was really looking to get the improved M1 support in 1.13.  For now, my work-around is to hard-code a path to the wheel in `pyproject.toml`:

```
[tool.poetry.dependencies]
torch = {url = "https://files.pythonhosted.org/packages/79/b3/eaea3fc35d0466b9dae1e3f9db08467939347b3aaa53c0fd81953032db33/torch-1.13.0-cp310-none-macosx_11_0_arm64.whl"}
```
It seems that the PyPi API doesn't support different requirements on different platforms.

I submitted https://github.com/pytorch/pytorch/pull/88826 to address this for future releases. 🤞

It doesn't solve the [larger issue](https://github.com/pytorch/pytorch/issues/88049#issuecomment-1302555269) @neersighted mentioned of split metadata across wheels, but should at least have the platform marker. I guess the opposite problem of Poetry inspecting metadata for one of the non-Linux wheels and thus _omitting_ these Linux deps on Linux systems might be possible (I'm not sure if the order the wheels are picked is deterministic?).
It's based on the order we get them back from PyPI (and PyPI uses upload order iirc; this is modeled on the fact that the PyPI JSON API uses the first artifact to be uploaded).

Resolving patch:
diff --git a/.github/scripts/generate_binary_build_matrix.py b/.github/scripts/generate_binary_build_matrix.py
--- a/.github/scripts/generate_binary_build_matrix.py
+++ b/.github/scripts/generate_binary_build_matrix.py
@@ -219,9 +219,9 @@ def generate_wheels_matrix(os: str,
                         "container_image": WHEEL_CONTAINER_IMAGES[arch_version],
                         "package_type": package_type,
                         "pytorch_extra_install_requirements":
-                        "nvidia-cuda-runtime-cu11;"
-                        "nvidia-cudnn-cu11==8.5.0.96;"
-                        "nvidia-cublas-cu11==11.10.3.66",
+                        "nvidia-cuda-runtime-cu11; platform_system == 'Linux' | "
+                        "nvidia-cudnn-cu11==8.5.0.96; platform_system == 'Linux' | "
+                        "nvidia-cublas-cu11==11.10.3.66; platform_system == 'Linux'",
                         "build_name":
                         f"{package_type}-py{python_version}-{gpu_arch_type}{gpu_arch_version}-with-pypi-cudnn"
                         .replace(
diff --git a/setup.py b/setup.py
--- a/setup.py
+++ b/setup.py
@@ -848,7 +848,7 @@ def configure_extension_build():
     pytorch_extra_install_requirements = os.getenv("PYTORCH_EXTRA_INSTALL_REQUIREMENTS", "")
     if pytorch_extra_install_requirements:
         report(f"pytorch_extra_install_requirements: {pytorch_extra_install_requirements}")
-        extra_install_requires += pytorch_extra_install_requirements.split(";")
+        extra_install_requires += pytorch_extra_install_requirements.split("|")
 
 
     # Cross-compile for M1


Test patch: