Really strange OOM error on new pytorch install

qmeeus: <p>I am experiencing a really strange issue with out of memory errors on pytorch:</p>
<pre><code class="lang-bash">$ python -c "import torch; torch.randn(12,12).cuda()"
Traceback (most recent call last):
  File "&lt;string&gt;", line 1, in &lt;module&gt;
RuntimeError: CUDA error: out of memory
</code></pre>
<p>Background information:<br>
I just created a new environment with the following (from the documentation):</p>
<pre><code class="lang-bash">conda create -n pytorch-test
conda install -n pytorch-test -c pytorch pytorch cudatoolkit=11.0
</code></pre>
<p>And my GPU:</p>
<pre><code class="lang-bash">$ echo $CUDA_VISIBLE_DEVICES                                                                                           
2
$ nvidia-smi -i 2                                                                                                      
Thu Nov 19 12:52:56 2020
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 455.38       Driver Version: 455.38       CUDA Version: 11.1     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   2  Tesla P100-PCIE...  Off  | 00000000:0E:00.0 Off |                    2 |
| N/A   47C    P0    29W / 250W |      2MiB / 16280MiB |      0%   E. Process |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
|  No running processes found                                                 |
+-----------------------------------------------------------------------------+
</code></pre>
<p>Any help is much appreciated, I kinda desperate here <img src="https://discuss.pytorch.org/images/emoji/apple/sweat_smile.png?v=9" title=":sweat_smile:" class="emoji" alt=":sweat_smile:"></p>
<p>[EDIT]: This seems to be more of a hardware issue. I am experiencing the same problem with a new installation of tensorflow, following the exact same steps… Still, if you have any pointer to solve this, I take it</p>

ptrblck: <p>Could you switch the GPU from <code>EXCLUSIVE Process</code> to the default via <code>nvidia-smi -i 0 -c 0</code> and check if it’s changing the behavior?</p>

qmeeus: <p>Thank you for your answer. Unfortunately, I do not have sufficient permissions to do that.<br>
Nonetheless,here is some more information that makes me think that your approach is probably the right one:</p>
<pre><code class="lang-bash">$ nvidia-smi -q | grep "Compute Mode"
    Compute Mode                          : Exclusive_Process
</code></pre>
<p>Any way I can go around this problem without changing the compute mode?</p>