Cuda.is_available always returns false

mich1: <p>I have been messing around with some machine learning code and always stumble upon gpu related issues. I am almost certain that my gpu can be used because I have used applications such as stable diffusion webui and it works fine. However, the moment I try to set it up myself it just never works. I have been looking for solutions and I am guessing that my cuda version is too low? (6.1 I think) Any suggestions?</p>
<p>Python version: 3.10.10 (tags/v3.10.10:aad5f6a, Feb  7 2023, 17:20:36) [MSC v.1929 64 bit (AMD64)] (64-bit runtime)<br>
Python platform: Windows-10-10.0.19044-SP0<br>
Is CUDA available: False<br>
CUDA runtime version: 12.1.66<br>
CUDA_MODULE_LOADING set to: N/A<br>
GPU models and configuration: GPU 0: NVIDIA GeForce GTX 1060 with Max-Q Design<br>
Nvidia driver version: 531.14<br>
cuDNN version: Could not collect<br>
HIP runtime version: N/A<br>
MIOpen runtime version: N/A<br>
Is XNNPACK available: True</p>

ptrblck: <aside class="quote no-group" data-username="mich1" data-post="1" data-topic="175221">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/mich1/48/58514_2.png" class="avatar"> mich1:</div>
<blockquote>
<p>I am almost certain that my gpu can be used because I have used applications such as stable diffusion webui and it works fine. However, the moment I try to set it up myself it just never works.</p>
</blockquote>
</aside>
<p>Could you describe in more detail how these two environments might differ and how you’ve installed each of these?</p>

mich1: <p>Stable diffusion provides installation script for users and this comes in the form of a web ui. However, there is also the option for people who might prefer a more programmatic way, for these people there is the source code.</p>
<p>I am not 100% sure but for the webui version I would assume that it is installed with a container so all the dependencies are resolved within the container. If you want to use the source code, you would have to setup all the stuffs yourself, all the pip packages and whatnot.</p>

ege_b: <p>I believe there is no support for CUDA 12.1 with any PyTorch version. If you refer to the installation guide from: <a href="https://pytorch.org/" rel="noopener nofollow ugc">https://pytorch.org/</a> you shouldn’t have any problems. What <a class="mention" href="/u/ptrblck">@ptrblck</a> meant is probably the commands that you used to install pytorch with, e.g.:<br>
<code> pip install torch...</code> and the way you installed the cuda runtime, e.g. manually or through pytorch etc.</p>

ptrblck: <aside class="quote no-group" data-username="ege_b" data-post="4" data-topic="175221">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ege_b/48/32255_2.png" class="avatar"> ege_b:</div>
<blockquote>
<p>I believe there is no support for CUDA 12.1 with any PyTorch version.</p>
</blockquote>
</aside>
<p>We do not publish binaries with CUDA 12.1, but source builds will work as we are already using them.</p>
<aside class="quote no-group" data-username="ege_b" data-post="4" data-topic="175221">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ege_b/48/32255_2.png" class="avatar"> ege_b:</div>
<blockquote>
<p>What <a class="mention" href="/u/ptrblck">@ptrblck</a> meant is probably the commands that you used to install pytorch with</p>
</blockquote>
</aside>
<p>Yes, since <a class="mention" href="/u/mich1">@mich1</a> explained that “stable diffusion webui and it works fine” while the custom setup fails.</p>

mich1: <p>That is a hard question. I cannot remember but I believe followed the setups on github and installed pytorch with the command they provided.</p>
<p>I have downgrade my cuda version to 11.6, however, after running <code>python -m torch.utils.collect_env</code> I still get false on cuda’s availability. On top of that I am now receiving <code>RuntimeError: don't know how to restore data location of torch.storage.UntypedStorage (tagged with gpu)</code> from running whisper (openai). Any cool tips?</p>

mich1: <p>cool, fixed.</p>
<p>what i did:</p>
<ul>
<li>verify cuda version<br>
– cuda version is not supported because its too new<br>
– downgrade cuda version</li>
<li>verify pytorch version to match cuda version<br>
– uninstall uncessary cuda versions from pip to ensure pytorch and cuda version is the same<br>
– e.g pytorch cuda117 should be used with cuda117 (not sure why i thought its backward compatible and wouldn’t be an issue if you have different versions but i made sure it is same)</li>
</ul>
<p><code>RuntimeError: don't know how to restore data location of torch.storage.UntypedStorage (tagged with gpu)</code> Ignore this, I simply passed the wrong option in the command</p>