Cuda not available for RTX 6000 mobile

orionflame: <p>Hi,</p>
<p>I have mobile RTX 6000 on my laptop:</p><aside class="onebox allowlistedgeneric" data-onebox-src="https://www.asus.com/us/laptops/for-creators/proart-studiobook/proart-studiobook-one-w590/">
  <header class="source">
      <img src="https://www.asus.com/new_asus_ico_256x256.png" class="site-icon" width="256" height="256">

      <a href="https://www.asus.com/us/laptops/for-creators/proart-studiobook/proart-studiobook-one-w590/" target="_blank" rel="noopener nofollow ugc">ASUS USA</a>
  </header>

  <article class="onebox-body">
    <img width="500" height="500" src="https://dlcdnwebimgs.asus.com/gain/2496af20-0c43-41ed-94d7-1eda080bd857/" class="thumbnail onebox-avatar">

<h3><a href="https://www.asus.com/us/laptops/for-creators/proart-studiobook/proart-studiobook-one-w590/" target="_blank" rel="noopener nofollow ugc">ProArt Studiobook One W590｜Laptops For Creators｜ASUS USA</a></h3>

  <p>ASUS ProArt series is a mighty powerhouse that brings ideas to life. It features the NVIDIA® Quadro GPU, and boasts a slim-bezel NanoEdge display for immersive visuals. Its combination of cutting-edge graphics and portability makes it perfect for...</p>


  </article>

  <div class="onebox-metadata">
    
    
  </div>

  <div style="clear: both"></div>
</aside>

<p>When I run this code, it returns False:<br>
import torch<br>
print(torch.cuda.is_available())</p>
<p>And this returns “Torch not compiled with CUDA enabled”<br>
print(torch.cuda.get_device_name(0))</p>
<p>I am using torch from RVC, but I don’t remember if I installed it myself for another project like OpenAI whisper variants.</p>
<p>Mon May  6 11:37:19 2024<br>
±----------------------------------------------------------------------------------------+<br>
| NVIDIA-SMI 551.23                 Driver Version: 551.23         CUDA Version: 12.4     |<br>
|-----------------------------------------±-----------------------±---------------------+<br>
| GPU  Name                     TCC/WDDM  | Bus-Id          Disp.A | Volatile Uncorr. ECC |<br>
| Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |<br>
|                                         |                        |               MIG M. |<br>
|=========================================+========================+======================|<br>
|   0  Quadro RTX 6000              WDDM  |   00000000:01:00.0  On |                  N/A |<br>
| N/A   66C    P2             58W /  132W |    5957MiB /  24576MiB |      3%      Default |<br>
|                                         |                        |                  N/A |<br>
±----------------------------------------±-----------------------±---------------------+</p>
<p>What should I do to fix this?</p>

AlphaBetaGamma96: <p>Hi <a class="mention" href="/u/orionflame">@orionflame</a>,</p>
<p>When you installed PyTorch did you install PyTorch with CUDA? It seems you may have installed the cpu only version of PyTorch, as <code>torch.cuda.is_available()</code> returns <code>False</code>.</p>
<aside class="quote no-group" data-username="orionflame" data-post="1" data-topic="202160">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/orionflame/48/69785_2.png" class="avatar"> orionflame:</div>
<blockquote>
<p>And this returns “Torch not compiled with CUDA enabled”</p>
</blockquote>
</aside>
<p>Make sure to select an appropriate CUDA version for the PyTorch install via the getting-started page, here: <a href="https://pytorch.org/get-started/locally/" class="inline-onebox" rel="noopener nofollow ugc">Start Locally | PyTorch</a></p>
<p>These other threads on the forum might help you debug any issues:</p>
<ol>
<li><a href="https://discuss.pytorch.org/t/torch-cuda-is-available-returns-false-even-cuda-is-installed/159016" class="inline-onebox">Torch.cuda.is_available() returns False even CUDA is installed</a></li>
<li><a href="https://discuss.pytorch.org/t/torch-cuda-is-available-is-false-cuda-version-11-4/130030" class="inline-onebox">Torch.cuda.is_available() is false CUDA Version: 11.4</a></li>
</ol>

orionflame: <p>Thanks I just installed cuda 12.1 and nvcc reports that I have it. But when I run the same code now I get this:</p>
<p>The specified module could not be found. Error loading local-packages\Python312\site-packages\torch\lib\shm.dll" or one of its dependencies.</p>
<p>This is after running this install:<br>
pip3 install torch torchvision torchaudio --index-url <a href="https://download.pytorch.org/whl/cu121" rel="noopener nofollow ugc">https://download.pytorch.org/whl/cu121</a></p>
<p>I will try to see how to fix it.</p>

AlphaBetaGamma96: <p>If you already have torch installed, you might have to update the existing install (as well as installing the CUDA add-on) rather than just using the install command directly from the getting-started page, but I’m not 100% sure.</p>
<p>Perhaps it might be best to create a new environment (which has CUDA support from the get go?) and try the install command from the getting-started page?</p>
<p>If not, <a class="mention" href="/u/ptrblck">@ptrblck</a> might be able to help! (apologies for the ping!)</p>

orionflame: <p>thx but pip install updates the relevant components based on the command no? I thought it updated it to support cuda 12.1 based on the command I posted.</p>
<p>what’s the cuda addon? i installed it from nvidia first.</p>
<p>also what do you mean by new env? like conda? i just wanted to use RVC</p>

AlphaBetaGamma96: <p>I’m just trying to figure out if I might be missing something obvious (which I most definitely am).</p>
<p>The issue seems to be pytorch fails loading a particular <code>.dll</code> file, so perhaps a fresh install in a clean environment might be the best initial solution (before <a class="mention" href="/u/ptrblck">@ptrblck</a> hopefully responds). Or perhaps you could force reinstall the <code>pip3</code> command you stated above?</p>

AlphaBetaGamma96: <p>I just read on this thread: <a href="https://discuss.pytorch.org/t/error-loading-lib-site-packages-torch-lib-shm-dll-or-one-of-its-dependencie/201695/2" class="inline-onebox">Error loading "\lib\site-packages\torch\lib\shm.dll" or one of its dependencie - #2 by Alexander_DE</a> (which you’re also on). The error might be due to using pytorch2.3 (which was released quite recently).</p>
<p>Perhaps do a clean install of PyTorch 2.2 and see if the error is resolved?</p>

orionflame: <p>Is there a way to specify the version? When I click previous versions on the official website, it shows conda commands. The same GUI for forming the pip command is not available so not sure if I can specify the version for all or I have to figure out individual versions for each module.</p>

AlphaBetaGamma96: <p>You should be able to find install commands for previous install version here: <a href="https://pytorch.org/get-started/previous-versions/" class="inline-onebox" rel="noopener nofollow ugc">Previous PyTorch Versions | PyTorch</a></p>
<p>For CUDA 12.1 install, the command is:<br>
<code>pip install torch==2.2.2 torchvision==0.17.2 torchaudio==2.2.2 --index-url https://download.pytorch.org/whl/cu121</code></p>

orionflame: <p>Thanks that seems to work, I get this printed now:</p>
<p>CUDA Available:  True<br>
CUDA Device Name:  Quadro RTX 6000</p>

orionflame: <p>Now I need to figure out why RVC still says " Unfortunately there is no compatible GPU available to support your training." I imagine they dont support pytorch 2.2.</p>

ptrblck: <p>Thanks for the ping <a class="mention" href="/u/alphabetagamma96">@AlphaBetaGamma96</a>.<br>
The issue is tracked <a href="https://github.com/pytorch/pytorch/issues/125109">here</a> and might come from an MKL update on Windows:</p>
<blockquote>
<p>Aaaand I found the solution. The MKL package, when installed in user context, installs the libraries that shm.dll is linked to into %APPDATA%\Python\Library\Bin - which per default isn’t in the PATH env var. Adding that path to the PATH env var allows for the DLL loader to actually find the MKL dlls</p>
</blockquote>
<p>The topic also suggests workarounds <a class="mention" href="/u/orionflame">@orionflame</a>.</p>

orionflame: <p>If i am not wrong this issue is fixed right?</p>
<p>I am trying to use this:</p><aside class="onebox githubblob" data-onebox-src="https://github.com/RVC-Project/Retrieval-based-Voice-Conversion-WebUI/blob/main/docs/en/README.en.md">
  <header class="source">

      <a href="https://github.com/RVC-Project/Retrieval-based-Voice-Conversion-WebUI/blob/main/docs/en/README.en.md" target="_blank" rel="noopener nofollow ugc">github.com</a>
  </header>

  <article class="onebox-body">
    <h4><a href="https://github.com/RVC-Project/Retrieval-based-Voice-Conversion-WebUI/blob/main/docs/en/README.en.md" target="_blank" rel="noopener nofollow ugc">RVC-Project/Retrieval-based-Voice-Conversion-WebUI/blob/main/docs/en/README.en.md</a></h4>


      <pre><code class="lang-md">&lt;div align="center"&gt;

&lt;h1&gt;Retrieval-based-Voice-Conversion-WebUI&lt;/h1&gt;
An easy-to-use voice conversion framework based on VITS.&lt;br&gt;&lt;br&gt;

[![madewithlove](https://img.shields.io/badge/made_with-%E2%9D%A4-red?style=for-the-badge&amp;labelColor=orange
)](https://github.com/RVC-Project/Retrieval-based-Voice-Conversion-WebUI)

&lt;img src="https://counter.seku.su/cmoe?name=rvc&amp;theme=r34" /&gt;&lt;br&gt;
  
[![RVC v1](https://img.shields.io/badge/RVCv1-F9AB00?style=for-the-badge&amp;logo=googlecolab&amp;color=525252)](https://colab.research.google.com/github/RVC-Project/Retrieval-based-Voice-Conversion-WebUI/blob/main/tools/ipynb/v1.ipynb)
[![RVC v2](https://img.shields.io/badge/RVCv2-F9AB00?style=for-the-badge&amp;logo=googlecolab&amp;color=525252)](https://colab.research.google.com/github/RVC-Project/Retrieval-based-Voice-Conversion-WebUI/blob/main/tools/ipynb/v2.ipynb)
[![Licence](https://img.shields.io/github/license/RVC-Project/Retrieval-based-Voice-Conversion-WebUI?style=for-the-badge)](https://github.com/RVC-Project/Retrieval-based-Voice-Conversion-WebUI/blob/main/LICENSE)
[![Huggingface](https://img.shields.io/badge/🤗%20-Spaces-yellow.svg?style=for-the-badge)](https://huggingface.co/lj1995/VoiceConversionWebUI/tree/main/)

[![Discord](https://img.shields.io/badge/RVC%20Developers-Discord-7289DA?style=for-the-badge&amp;logo=discord&amp;logoColor=white)](https://discord.gg/HcsmBBGyVk)

[**Changelog**](https://github.com/RVC-Project/Retrieval-based-Voice-Conversion-WebUI/blob/main/docs/Changelog_EN.md) | [**FAQ (Frequently Asked Questions)**](https://github.com/RVC-Project/Retrieval-based-Voice-Conversion-WebUI/wiki/FAQ-(Frequently-Asked-Questions)) 

[**English**](../en/README.en.md) | [**中文简体**](../../README.md) | [**日本語**](../jp/README.ja.md) | [**한국어**](../kr/README.ko.md) ([**韓國語**](../kr/README.ko.han.md)) | [**Français**](../fr/README.fr.md) | [**Türkçe**](../tr/README.tr.md) | [**Português**](../pt/README.pt.md)
</code></pre>



  This file has been truncated. <a href="https://github.com/RVC-Project/Retrieval-based-Voice-Conversion-WebUI/blob/main/docs/en/README.en.md" target="_blank" rel="noopener nofollow ugc">show original</a>

  </article>

  <div class="onebox-metadata">
    
    
  </div>

  <div style="clear: both"></div>
</aside>

<p>But it will says it can’t find any supported GPU.</p>
<p>On the page that say this:</p>
<ol>
<li>If you are using Nvidia Ampere architecture (RTX30xx) in Windows, according to the experience of <span class="hashtag-raw">#21</span>, you need to specify the cuda version corresponding to pytorch.</li>
</ol>
<p>pip install torch torchvision torchaudio --index-url <a href="https://download.pytorch.org/whl/cu117" rel="noopener nofollow ugc">https://download.pytorch.org/whl/cu117</a></p>
<p>If I am not wrong my GPU is Turing so I shouldn’t be using the latest pytorch?</p>
<p>I asked there but they didn’t reply anyway for a week.</p>
<p>Any ideas? <img src="https://discuss.pytorch.org/images/emoji/apple/pray.png?v=12" title=":pray:" class="emoji" alt=":pray:" loading="lazy" width="20" height="20"></p>

AlphaBetaGamma96: <p>If you’re trying to install CUDA 11.7, perhaps try PyTorch 2.0.1 via the install page?</p><aside class="onebox allowlistedgeneric" data-onebox-src="https://pytorch.org/get-started/previous-versions/#v201">
  <header class="source">
      <img src="https://pytorch.org/favicon.ico?" class="site-icon" width="32" height="32">

      <a href="https://pytorch.org/get-started/previous-versions/#v201" target="_blank" rel="noopener nofollow ugc">PyTorch</a>
  </header>

  <article class="onebox-body">
    <div class="aspect-image" style="--aspect-ratio:690/362;"><img src="https://pytorch.org/assets/images/social-share.jpg" class="thumbnail" width="690" height="362"></div>

<h3><a href="https://pytorch.org/get-started/previous-versions/#v201" target="_blank" rel="noopener nofollow ugc">Previous PyTorch Versions</a></h3>

  <p>Installing previous versions of PyTorch</p>


  </article>

  <div class="onebox-metadata">
    
    
  </div>

  <div style="clear: both"></div>
</aside>


orionflame: <p>thx i will try this now.</p>
<p>can multiple cuda versions exist together? i assume i dont have to uninstall the other one.</p>

ptrblck: <aside class="quote no-group" data-username="orionflame" data-post="13" data-topic="202160">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/orionflame/48/69785_2.png" class="avatar"> orionflame:</div>
<blockquote>
<p>my GPU is Turing so I shouldn’t be using the latest pytorch?</p>
</blockquote>
</aside>
<p>Turing GPUs are still supported in the latest CUDA releases and all of our current binaries with CUDA 11.8, 12.1, and 12.4 support these.</p>
<aside class="quote no-group" data-username="orionflame" data-post="15" data-topic="202160">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/orionflame/48/69785_2.png" class="avatar"> orionflame:</div>
<blockquote>
<p>can multiple cuda versions exist together?</p>
</blockquote>
</aside>
<p>PyTorch binaries ship with their own CUDA dependencies and your locally installed CUDA toolkit won’t be used unless you build PyTorch from source or a custom CUDA extension.<br>
You cannot install multiple PyTorch binaries into one environment.</p>

orionflame: <p>Oh I see, I thought i had to install cuda separately for pytorch to work. Then I will just install PyTorch 2.0.1 as Alpha recommended.</p>

orionflame: <p>I am trying to install it for 11.7 but I got an error saying no matching distribution found for torch==2.0.0</p>
<pre><code class="lang-auto"># CUDA 11.7
pip install torch==2.0.0 torchvision==0.15.1 torchaudio==2.0.1
# CUDA 11.8
pip install torch==2.0.0 torchvision==0.15.1 torchaudio==2.0.1 --index-url https://download.pytorch.org/whl/cu118
</code></pre>
<p>from the help above: <a href="https://pytorch.org/get-started/previous-versions/#v201" class="inline-onebox" rel="noopener nofollow ugc">Previous PyTorch Versions | PyTorch</a></p>
<p>also shouldnt 11.7 command have this?</p>
<p>–index-url <a href="https://download.pytorch.org/whl/cu117" rel="noopener nofollow ugc">https://download.pytorch.org/whl/cu117</a></p>
<p>It’s a bit confusing.</p>

AlphaBetaGamma96: <p>Shouldn’t it be <code>torch==2.0.1</code>?<br>
The command on the website is:<br>
<code>conda install pytorch==2.0.1 torchvision==0.15.2 torchaudio==2.0.2 pytorch-cuda=11.7 -c pytorch -c nvidia</code></p>

orionflame: <p>Yes I wanted to use that but I dont have conda that’s why I was looking for comparable one to get cuda 11.7.</p>
<p>I dont know how to convert that condo command into regular pip. Because in pip it’s using torch name and pytorch name so not sure why they are different like that.</p>