Pytorch says GPU busy/unavailable even though it returns 'True' with " cuda.is_available() "

lukecage: <p>Hi all!</p>
<p>I have been trying to make Pytorch work with NVIDIA A100 - 80GB GPU .<br>
Following is the <code>nvidia-smi</code> output :</p>
<pre><code class="lang-auto">+-----------------------------------------------------------------------------+
| NVIDIA-SMI 450.203.03   Driver Version: 450.203.03   CUDA Version: 11.0     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  A100 80GB PCIe      On   | 00000000:01:01.0 Off |                   On |
| N/A   30C    P0    42W / 300W |      0MiB / 81252MiB |     N/A      Default |
|                               |                      |              Enabled |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| MIG devices:                                                                |
+------------------+----------------------+-----------+-----------------------+
| GPU  GI  CI  MIG |         Memory-Usage |        Vol|         Shared        |
|      ID  ID  Dev |           BAR1-Usage | SM     Unc| CE  ENC  DEC  OFA  JPG|
|                  |                      |        ECC|                       |
|==================+======================+===========+=======================|
|  No MIG devices found                                                       |
+-----------------------------------------------------------------------------+

+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
|  No running processes found                                                 |
+-----------------------------------------------------------------------------+

</code></pre>
<p>I have installed Pytorch using the command available on official website :</p>
<pre><code class="lang-auto">pip3 install torch torchvision torchaudio --extra-index-url https://download.pytorch.org/whl/cu113

</code></pre>
<p>which gives me " torch==1.12.1+cu113 " version of Pytorch. Upon checking if Pytorch detects GPU or not using <code>torch.cuda.is_available()</code> I get <code>True</code> as return.<br>
While a basic Tensor <code> torch.Tensor([1,2,3]).to("cuda")</code> gives me the following output</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/a/0/a08e8c41a264c79ae4803c07da8d698a26a4d0d3.png" data-download-href="https://discuss.pytorch.org/uploads/default/a08e8c41a264c79ae4803c07da8d698a26a4d0d3" title="image"><img src="https://discuss.pytorch.org/uploads/default/original/3X/a/0/a08e8c41a264c79ae4803c07da8d698a26a4d0d3.png" alt="image" data-base62-sha1="mUlNLxckYX65OCsoQMxGIsVW1r5" width="690" height="73" data-dominant-color="0B0B0B"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">945×100 4.71 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>I have multiple combinations of CUDA Drivers - Pytorch versions but nothing works.</p>
<p>Can anyone please help?</p>

ptrblck: <p>Try to disable MIG and rerun your script.</p>

lukecage: <aside class="quote no-group" data-username="ptrblck" data-post="2" data-topic="163555">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/48/1823_2.png" class="avatar"> ptrblck:</div>
<blockquote>
<p>ble MIG and rerun your script</p>
</blockquote>
</aside>
<p>Yes, I figured that out eventually. Thanks for the response though!</p>