Torch+cuda installation on CPU-only machine

dsethz: <p>Hey,</p>
<p><strong>Question</strong>: Is it feasible to install a CUDA-compatible version of <code>torch</code> (and <code>torchvision</code>) on a machine without a GPU (and no CUDA installed) (e.g. <code>pip install pip install torch==1.10.1+cu111</code>)?</p>
<p><strong>Context</strong>: I want to declare <code>torch</code> as a dependency in my packaging meta-data. The project is a plug-in for a GUI-based software → intended for users without command line experience. Further, <code>torch</code> must be functional irrespective of <code>OS</code> and on GPU and CPU machines. However, automatically recognising if the machine has GPUs available at installation time does not seem to be possible (?, currently using <a href="https://python-poetry.org" rel="noopener nofollow ugc"><code>poetry</code></a>).</p>
<p>Similar issues have been discussed here:</p><aside class="onebox stackexchange" data-onebox-src="https://stackoverflow.com/questions/74388129/poetry-how-to-deal-with-hardware-dependant-libraries">
  <header class="source">

      <a href="https://stackoverflow.com/questions/74388129/poetry-how-to-deal-with-hardware-dependant-libraries" target="_blank" rel="noopener nofollow ugc">stackoverflow.com</a>
  </header>

  <article class="onebox-body">
      <a href="https://stackoverflow.com/users/6084245/robin" target="_blank" rel="noopener nofollow ugc">
    <img alt="Robin" src="https://www.gravatar.com/avatar/a9d8725b1bb91943421a0615a94839be?s=256&amp;d=identicon&amp;r=PG&amp;f=1" class="thumbnail onebox-avatar" width="256" height="256">
  </a>

<h4>
  <a href="https://stackoverflow.com/questions/74388129/poetry-how-to-deal-with-hardware-dependant-libraries" target="_blank" rel="noopener nofollow ugc">Poetry: how to deal with hardware dependant libraries</a>
</h4>

<div class="tags">
  <strong>pytorch, python-poetry</strong>
</div>

<div class="date">
  asked by
  
  <a href="https://stackoverflow.com/users/6084245/robin" target="_blank" rel="noopener nofollow ugc">
    Robin
  </a>
  on <a href="https://stackoverflow.com/questions/74388129/poetry-how-to-deal-with-hardware-dependant-libraries" target="_blank" rel="noopener nofollow ugc">11:18AM - 10 Nov 22 UTC</a>
</div>

  </article>

  <div class="onebox-metadata">
    
    
  </div>

  <div style="clear: both"></div>
</aside>
<aside class="onebox githubissue" data-onebox-src="https://github.com/python-poetry/poetry/issues/6409">
  <header class="source">

      <a href="https://github.com/python-poetry/poetry/issues/6409" target="_blank" rel="noopener nofollow ugc">github.com/python-poetry/poetry</a>
  </header>

  <article class="onebox-body">
    <div class="github-row">
  <div class="github-icon-container" title="Issue">
	  <svg width="60" height="60" class="github-icon" viewbox="0 0 14 16" aria-hidden="true"><path fill-rule="evenodd" d="M7 2.3c3.14 0 5.7 2.56 5.7 5.7s-2.56 5.7-5.7 5.7A5.71 5.71 0 0 1 1.3 8c0-3.14 2.56-5.7 5.7-5.7zM7 1C3.14 1 0 4.14 0 8s3.14 7 7 7 7-3.14 7-7-3.14-7-7-7zm1 3H6v5h2V4zm0 6H6v2h2v-2z"></path></svg>
  </div>

  <div class="github-info-container">
    <h4>
      <a href="https://github.com/python-poetry/poetry/issues/6409" target="_blank" rel="noopener nofollow ugc">Instructions for installing PyTorch</a>
    </h4>

    <div class="github-info">
      <div class="date">
        opened <span class="discourse-local-date" data-format="ll" data-date="2022-09-05" data-time="08:59:06" data-timezone="UTC">08:59AM - 05 Sep 22 UTC</span>
      </div>


      <div class="user">
        <a href="https://github.com/davidgilbertson" target="_blank" rel="noopener nofollow ugc">
          <img alt="davidgilbertson" src="https://avatars.githubusercontent.com/u/4443482?v=4" class="onebox-avatar-inline" width="20" height="20">
          davidgilbertson
        </a>
      </div>
    </div>

    <div class="labels">
        <span style="display:inline-block;margin-top:2px;background-color: #B8B8B8;padding: 2px;border-radius: 4px;color: #fff;margin-left: 3px;">
          area/docs
        </span>
        <span style="display:inline-block;margin-top:2px;background-color: #B8B8B8;padding: 2px;border-radius: 4px;color: #fff;margin-left: 3px;">
          status/triage
        </span>
    </div>
  </div>
</div>

  <div class="github-row">
    <p class="github-body-container">&lt;!--
  Hi there! Thank you for discovering and submitting an issue with our doc<span class="show-more-container"><a href="" rel="noopener" class="show-more">…</a></span><span class="excerpt hidden">umentation.

  Before you submit this; let's make sure of a few things.
  Please make sure the following boxes are ticked if they are correct.
  If not, please try and fulfill these first.
--&gt;


- [x] I have searched the [issues](https://github.com/python-poetry/poetry/issues) of this repo and believe that this is not a duplicate.

## Issue

As mentioned in issue https://github.com/python-poetry/poetry/issues/4231 there is some confusion around installing PyTorch with CUDA but it is now somewhat resolved. It still requires a few steps, and all options have pretty serious flaws. Below are two options that worked for me, on Poetry version `1.2.0`.

# Option 1 - wheel URLs for a specific platform

* You will need to pick the specific wheels you want. These are listed here: https://download.pytorch.org/whl/torch_stable.html. E.g. if you want CUDA 11.6, Python 3.10 and Windows, search that page for `cu116-cp310-cp310-win_amd64.whl` to see the matches for `torch`, `torchaudio` and `torchvision`
* In your `pyproject.toml` file add the URLs like:

```toml
[tool.poetry.dependencies]
python = "^3.10"
numpy = "^1.23.2"
torch = { url = "https://download.pytorch.org/whl/cu116/torch-1.12.1%2Bcu116-cp310-cp310-win_amd64.whl"}
torchaudio = { url = "https://download.pytorch.org/whl/cu116/torchaudio-0.12.1%2Bcu116-cp310-cp310-win_amd64.whl"}
torchvision = { url = "https://download.pytorch.org/whl/cu116/torchvision-0.13.1%2Bcu116-cp310-cp310-win_amd64.whl"}
```
* Run `poetry update`. It will download a _lot_ of data (many GB) and take quite some time. And this doesn't seem to cache reliably (at least, I've waited 30 minutes+ at 56 Mbps three separate times while troubleshooting this, for the exact same wheels)

Note that each subsequent `poetry update` will do _another_ huge download and you'll see this message:
```
  • Updating torch (1.12.1+cu116 -&gt; 1.12.1+cu116 https://download.pytorch.org/whl/cu116/torch-1.12.1%2Bcu116-cp310-cp310-win_amd64.whl)
  • Updating torchaudio (0.12.1+cu116 -&gt; 0.12.1+cu116 https://download.pytorch.org/whl/cu116/torchaudio-0.12.1%2Bcu116-cp310-cp310-win_amd64.whl)
  • Updating torchvision (0.13.1+cu116 -&gt; 0.13.1+cu116 https://download.pytorch.org/whl/cu116/torchvision-0.13.1%2Bcu116-cp310-cp310-win_amd64.whl)
```

# Option 2 - alternate source

```toml
[tool.poetry.dependencies]
python = "^3.10"
numpy = "^1.23.2"
torch = { version = "1.12.1", source="torch"}
torchaudio = { version = "0.12.1", source="torch"}
torchvision = { version = "0.13.1", source="torch"}

[[tool.poetry.source]]
name = "torch"
url = "https://download.pytorch.org/whl/cu116"
secondary = true
```

This seems to have worked (although I already had the packages installed) but it reports errors like `Source (torch): Authorization error accessing https://download.pytorch.org/whl/cu116/pillow/`, but I think they get installed anyway (maybe a better message would be "Can't access pillow at 'https://download.pytorch.org/whl/cu116', falling back to pypi")

Also, if you later go on to do, say `poetry add pandas` (a completely unrelated library) you'll get a wall of messages like:
```
Source (torch): Authorization error accessing https://download.pytorch.org/whl/cu116/pandas/
Source (torch): Authorization error accessing https://download.pytorch.org/whl/cu116/pandas/
Source (torch): Authorization error accessing https://download.pytorch.org/whl/cu116/pytz/
Source (torch): Authorization error accessing https://download.pytorch.org/whl/cu116/python-dateutil/
Source (torch): Authorization error accessing https://download.pytorch.org/whl/cu116/numpy/
Source (torch): Authorization error accessing https://download.pytorch.org/whl/cu116/pillow/
Source (torch): Authorization error accessing https://download.pytorch.org/whl/cu116/requests/
Source (torch): Authorization error accessing https://download.pytorch.org/whl/cu116/typing-extensions/
Source (torch): Authorization error accessing https://download.pytorch.org/whl/cu116/certifi/
Source (torch): Authorization error accessing https://download.pytorch.org/whl/cu116/urllib3/
Source (torch): Authorization error accessing https://download.pytorch.org/whl/cu116/idna/
Source (torch): Authorization error accessing https://download.pytorch.org/whl/cu116/charset-normalizer/
Source (torch): Authorization error accessing https://download.pytorch.org/whl/cu116/python-dateutil/
Source (torch): Authorization error accessing https://download.pytorch.org/whl/cu116/six/
Source (torch): Authorization error accessing https://download.pytorch.org/whl/cu116/pytz/
Source (torch): Authorization error accessing https://download.pytorch.org/whl/cu116/six/
```

This happens with or without `secondary = true` in the source config.

Maintainers: please feel free to edit the text of this if I've got something wrong.</span></p>
  </div>

  </article>

  <div class="onebox-metadata">
    
    
  </div>

  <div style="clear: both"></div>
</aside>

<p>Best,<br>
dsethz</p>

ptrblck: <p><code>pip install torch torchvision</code> will install the default wheels hosted on PyPI, which ship with the CUDA 11.7 runtime, so it seems this would already solve your issue.</p>

dsethz: <p>ok, so do I understand you correctly, that installing a <code>torch</code> version that ships with e.g. CUDA 11.7 does not introduce unintended behaviour on a machine that does not have GPUs available?</p>

ptrblck: <p>Yes, I don’t think installing the default pip wheels with the CUDA 11.7 runtime has any side effects besides the size increase (since CUDA libs will be downloaded and stored).</p>