PyTorch runs on CPU instead of 4090, Windows 11

Anton_Uramer: <p>I’m using the nightly PyTorch (for CUDA 11.8) installed with conda, conda was installed with the standard visual installer.</p>
<pre><code class="lang-auto">python -m torch.utils.collect_env

Collecting environment information...
PyTorch version: 2.0.0.dev20230130
Is debug build: False
CUDA used to build PyTorch: Could not collect
ROCM used to build PyTorch: N/A

OS: Microsoft Windows 11 Pro
GCC version: Could not collect
Clang version: Could not collect
CMake version: version 3.25.2
Libc version: N/A

Python version: 3.9.13 (main, Aug 25 2022, 23:51:50) [MSC v.1916 64 bit (AMD64)] (64-bit runtime)
Python platform: Windows-10-10.0.22621-SP0
Is CUDA available: False
CUDA runtime version: Could not collect
CUDA_MODULE_LOADING set to: N/A
GPU models and configuration: GPU 0: NVIDIA GeForce RTX 4090
Nvidia driver version: 528.24
cuDNN version: Could not collect
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

Versions of relevant libraries:
[pip3] mypy-extensions==0.4.3
[pip3] numpy==1.21.5
[pip3] numpydoc==1.4.0
[pip3] rotary-embedding-torch==0.2.1
[pip3] torch==2.0.0.dev20230130
[pip3] torchaudio==2.0.0.dev20230130
[pip3] torchvision==0.15.0.dev20230130
[conda] blas                      1.0                         mkl
[conda] mkl                       2021.4.0           haa95532_640
[conda] mkl-service               2.4.0            py39h2bbff1b_0
[conda] mkl_fft                   1.3.1            py39h277e83a_0
[conda] mkl_random                1.2.2            py39hf11a4ad_0
[conda] numpy                     1.20.0                   pypi_0    pypi
[conda] numpy-base                1.21.5           py39hca35cd5_3
[conda] numpydoc                  1.4.0            py39haa95532_0
[conda] pytorch                   2.0.0.dev20230130     py3.9_cpu_0    pytorch-nightly
[conda] pytorch-cuda              11.8                 h8dd9ede_2    pytorch-nightly
[conda] pytorch-mutex             1.0                         cpu    pytorch-nightly
[conda] rotary-embedding-torch    0.2.1                    pypi_0    pypi
[conda] torchaudio                2.0.0.dev20230130        py39_cpu    pytorch-nightly
[conda] torchvision               0.15.0.dev20230130        py39_cpu    pytorch-nightly
</code></pre>

Anton_Uramer: <p>Using stable PyTorch with CUDA 11.7 has the same issue</p>
<pre><code class="lang-auto">Collecting environment information...
PyTorch version: 1.13.1
Is debug build: False
CUDA used to build PyTorch: Could not collect
ROCM used to build PyTorch: N/A

OS: Microsoft Windows 11 Pro
GCC version: Could not collect
Clang version: Could not collect
CMake version: version 3.25.2
Libc version: N/A

Python version: 3.9.13 (main, Aug 25 2022, 23:51:50) [MSC v.1916 64 bit (AMD64)] (64-bit runtime)
Python platform: Windows-10-10.0.22621-SP0
Is CUDA available: False
CUDA runtime version: 11.7.99
CUDA_MODULE_LOADING set to: N/A
GPU models and configuration: GPU 0: NVIDIA GeForce RTX 4090
Nvidia driver version: 528.24
cuDNN version: Could not collect
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

Versions of relevant libraries:
[pip3] mypy-extensions==0.4.3
[pip3] numpy==1.21.5
[pip3] numpydoc==1.4.0
[pip3] rotary-embedding-torch==0.2.1
[pip3] torch==1.13.1
[pip3] torchaudio==0.13.1
[pip3] torchvision==0.14.1
[conda] blas                      1.0                         mkl
[conda] mkl                       2021.4.0           haa95532_640
[conda] mkl-service               2.4.0            py39h2bbff1b_0
[conda] mkl_fft                   1.3.1            py39h277e83a_0
[conda] mkl_random                1.2.2            py39hf11a4ad_0
[conda] numpy                     1.20.0                   pypi_0    pypi
[conda] numpy-base                1.21.5           py39hca35cd5_3
[conda] numpydoc                  1.4.0            py39haa95532_0
[conda] pytorch                   1.13.1              py3.9_cpu_0    pytorch
[conda] pytorch-cuda              11.7                 h67b0de4_0    pytorch
[conda] pytorch-mutex             1.0                         cpu    pytorch-nightly
[conda] rotary-embedding-torch    0.2.1                    pypi_0    pypi
[conda] torchaudio                0.13.1                 py39_cpu    pytorch
[conda] torchvision               0.14.1                 py39_cpu    pytorch
</code></pre>

ptrblck: <p>I don’t know why (as I’m not using Windows), but the CPU-only binaries were installed as indicate by the <code>cpu</code> tag:</p>
<pre><code class="lang-auto">[conda] pytorch                   2.0.0.dev20230130     py3.9_cpu_0    pytorch-nightly
</code></pre>
<p>Maybe try to install the pip wheels and see if this would work.</p>

Anton_Uramer: <p>I used</p>
<pre><code class="lang-auto">conda install pytorch torchvision torchaudio pytorch-cuda=11.8 -c pytorch-nightly -c nvidia
</code></pre>
<p>and</p>
<pre><code class="lang-auto">conda install pytorch torchvision torchaudio pytorch-cuda=11.7 -c pytorch -c nvidia
</code></pre>
<p>to install these versions. Is there a convenient way to prohibit installing the CPU version with Conda?</p>

Anton_Uramer: <p>Trying to install the wheels with this command now</p>
<pre><code class="lang-auto">pip3 install --pre torch torchvision torchaudio --index-url https://download.pytorch.org/whl/nightly/cu118
</code></pre>
<p>Will report back how it goes</p>

vince62s: <p>there is no wheel for 11.8<br>
You still can install 11.7 wheel on cuda 11.8 but then if you intend to use puytroch 2.0 with sm_89 arch you’ll need to re-install triton on your own for now.</p><aside class="onebox githubissue" data-onebox-src="https://github.com/pytorch/pytorch/issues/90170">
  <header class="source">

      <a href="https://github.com/pytorch/pytorch/issues/90170" target="_blank" rel="noopener nofollow ugc">github.com/pytorch/pytorch</a>
  </header>

  <article class="onebox-body">
    <div class="github-row">
  <div class="github-icon-container" title="Issue">
	  <svg width="60" height="60" class="github-icon" viewbox="0 0 14 16" aria-hidden="true"><path fill-rule="evenodd" d="M7 2.3c3.14 0 5.7 2.56 5.7 5.7s-2.56 5.7-5.7 5.7A5.71 5.71 0 0 1 1.3 8c0-3.14 2.56-5.7 5.7-5.7zM7 1C3.14 1 0 4.14 0 8s3.14 7 7 7 7-3.14 7-7-3.14-7-7-7zm1 3H6v5h2V4zm0 6H6v2h2v-2z"></path></svg>
  </div>

  <div class="github-info-container">
    <h4>
      <a href="https://github.com/pytorch/pytorch/issues/90170" target="_blank" rel="noopener nofollow ugc">Trying pytorch 2.0 snippet from the website - sm89 error</a>
    </h4>

    <div class="github-info">
      <div class="date">
        opened <span class="discourse-local-date" data-format="ll" data-date="2022-12-05" data-time="11:41:23" data-timezone="UTC">11:41AM - 05 Dec 22 UTC</span>
      </div>

        <div class="date">
          closed <span class="discourse-local-date" data-format="ll" data-date="2023-01-11" data-time="06:56:52" data-timezone="UTC">06:56AM - 11 Jan 23 UTC</span>
        </div>

      <div class="user">
        <a href="https://github.com/vince62s" target="_blank" rel="noopener nofollow ugc">
          <img alt="vince62s" src="https://avatars.githubusercontent.com/u/15141326?v=4" class="onebox-avatar-inline" width="20" height="20">
          vince62s
        </a>
      </div>
    </div>

    <div class="labels">
        <span style="display:inline-block;margin-top:2px;background-color: #B8B8B8;padding: 2px;border-radius: 4px;color: #fff;margin-left: 3px;">
          high priority
        </span>
        <span style="display:inline-block;margin-top:2px;background-color: #B8B8B8;padding: 2px;border-radius: 4px;color: #fff;margin-left: 3px;">
          triaged
        </span>
        <span style="display:inline-block;margin-top:2px;background-color: #B8B8B8;padding: 2px;border-radius: 4px;color: #fff;margin-left: 3px;">
          module: third_party
        </span>
        <span style="display:inline-block;margin-top:2px;background-color: #B8B8B8;padding: 2px;border-radius: 4px;color: #fff;margin-left: 3px;">
          module: inductor
        </span>
    </div>
  </div>
</div>

  <div class="github-row">
    <p class="github-body-container">### 🐛 Describe the bug

trying the snippet from the website
```
import torch
<span class="show-more-container"><a href="" rel="noopener" class="show-more">…</a></span><span class="excerpt hidden">import torchvision.models as models

model = models.resnet18().cuda()
optimizer = torch.optim.SGD(model.parameters(), lr=0.01)
compiled_model = torch.compile(model)

x = torch.randn(16, 3, 224, 224).cuda()
optimizer.zero_grad()
out = compiled_model(x)
out.sum().backward()
optimizer.step()
```

I got Value 'sm_89' is not defined for option 'gpu-name'
I am indeed using a rtx4090 and cuda 11.6
BUT works fine without torch.compile


### Versions

Collecting environment information...
PyTorch version: 1.14.0.dev20221205+cu116
Is debug build: False
CUDA used to build PyTorch: 11.6
ROCM used to build PyTorch: N/A

OS: Ubuntu 20.04.5 LTS (x86_64)
GCC version: (Ubuntu 9.4.0-1ubuntu1~20.04.1) 9.4.0
Clang version: Could not collect
CMake version: version 3.25.0
Libc version: glibc-2.31

Python version: 3.8.10 (default, Jun 22 2022, 20:18:18)  [GCC 9.4.0] (64-bit runtime)
Python platform: Linux-5.15.0-56-generic-x86_64-with-glibc2.29
Is CUDA available: True
CUDA runtime version: Could not collect
CUDA_MODULE_LOADING set to: LAZY
GPU models and configuration: GPU 0: NVIDIA GeForce RTX 4090
Nvidia driver version: 525.60.11
cuDNN version: Probably one of the following:
/usr/lib/x86_64-linux-gnu/libcudnn.so.8.6.0
/usr/lib/x86_64-linux-gnu/libcudnn_adv_infer.so.8.6.0
/usr/lib/x86_64-linux-gnu/libcudnn_adv_train.so.8.6.0
/usr/lib/x86_64-linux-gnu/libcudnn_cnn_infer.so.8.6.0
/usr/lib/x86_64-linux-gnu/libcudnn_cnn_train.so.8.6.0
/usr/lib/x86_64-linux-gnu/libcudnn_ops_infer.so.8.6.0
/usr/lib/x86_64-linux-gnu/libcudnn_ops_train.so.8.6.0
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

Versions of relevant libraries:
[pip3] numpy==1.22.3
[pip3] pytorch-lightning==1.7.7
[pip3] torch==1.14.0.dev20221205+cu116
[pip3] torch-tb-profiler==0.4.0
[pip3] torchaudio==0.14.0.dev20221204+cu116
[pip3] torchtext==0.14.0
[pip3] torchtriton==2.0.0+0d7e753227
[pip3] torchvision==0.15.0.dev20221204+cpu
[conda] blas                      1.0                         mkl  
[conda] cudatoolkit               11.3.1               h2bc3f7f_2  
[conda] faiss-gpu                 1.7.2           py3.8_h28a55e0_0_cuda11.3    pytorch
[conda] libfaiss                  1.7.2           hfc2d529_0_cuda11.3    pytorch
[conda] mkl                       2021.4.0           h06a4308_640  
[conda] mkl-service               2.4.0            py38h7f8727e_0  
[conda] mkl_fft                   1.3.1            py38hd3c417c_0  
[conda] mkl_random                1.2.2            py38h51133e4_0  
[conda] numpy                     1.23.4           py38h14f4228_0  
[conda] numpy-base                1.23.4           py38h31eccc5_0  
[conda] pytorch                   1.11.0          py3.8_cuda11.3_cudnn8.2.0_0    pytorch
[conda] pytorch-mutex             1.0                        cuda    pytorch
[conda] torch                     1.14.0.dev20221204+cu117          pypi_0    pypi
[conda] torchaudio                0.14.0.dev20221204+cu117          pypi_0    pypi
[conda] torchtriton               2.0.0+0d7e753227          pypi_0    pypi
[conda] torchvision               0.15.0.dev20221204+cpu          pypi_0    pypi


cc @ezyang @gchanan @zou3519 @mlazos @soumith @voznesenskym @yanboliang @penguinwu @anijain2305 @EikanWang @jgong5 @Guobing-Chen @chunyuan-w @XiaobingSuper @zhuhaozhe @blzheng @Xia-Weiwen @wenzhe-nrv @jiayisunx @peterbell10 @desertfire</span></p>
  </div>

  </article>

  <div class="onebox-metadata">
    
    
  </div>

  <div style="clear: both"></div>
</aside>


Anton_Uramer: <p>That works, I’ve confirmed it by running tortoise-tts about 100 times faster than my 7950X did.<br>
Thanks for the help. Maybe Conda shouldn’t be recommended by default on the website?</p>
<pre><code class="lang-auto">Collecting environment information...
PyTorch version: 2.0.0.dev20230129+cu118
Is debug build: False
CUDA used to build PyTorch: 11.8
ROCM used to build PyTorch: N/A

OS: Microsoft Windows 11 Pro
GCC version: Could not collect
Clang version: Could not collect
CMake version: version 3.25.2
Libc version: N/A

Python version: 3.9.13 (main, Aug 25 2022, 23:51:50) [MSC v.1916 64 bit (AMD64)] (64-bit runtime)
Python platform: Windows-10-10.0.22621-SP0
Is CUDA available: True
CUDA runtime version: 11.7.99
CUDA_MODULE_LOADING set to: LAZY
GPU models and configuration: GPU 0: NVIDIA GeForce RTX 4090
Nvidia driver version: 528.24
cuDNN version: Could not collect
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

Versions of relevant libraries:
[pip3] mypy-extensions==0.4.3
[pip3] numpy==1.21.5
[pip3] numpydoc==1.4.0
[pip3] rotary-embedding-torch==0.2.1
[pip3] torch==2.0.0.dev20230129+cu118
[pip3] torchaudio==2.0.0.dev20230130+cu118
[pip3] torchvision==0.15.0.dev20230130+cu118
[conda] blas                      1.0                         mkl
[conda] mkl                       2021.4.0           haa95532_640
[conda] mkl-service               2.4.0            py39h2bbff1b_0
[conda] mkl_fft                   1.3.1            py39h277e83a_0
[conda] mkl_random                1.2.2            py39hf11a4ad_0
[conda] numpy                     1.20.0                   pypi_0    pypi
[conda] numpy-base                1.21.5           py39hca35cd5_3
[conda] numpydoc                  1.4.0            py39haa95532_0
[conda] pytorch-cuda              11.7                 h67b0de4_0    pytorch
[conda] pytorch-mutex             1.0                         cpu    pytorch-nightly
[conda] rotary-embedding-torch    0.2.1                    pypi_0    pypi
[conda] torch                     2.0.0.dev20230129+cu118          pypi_0    pypi
[conda] torchaudio                2.0.0.dev20230130+cu118          pypi_0    pypi
[conda] torchvision               0.15.0.dev20230130+cu118          pypi_0    pypi
</code></pre>

Anton_Uramer: <p>I see that it still installed CUDA 11.7 though, as vince mentioned</p>