How totally remove data and free gpu

orena1: <p>Hello,<br>
I have a agile algorithm that use a changing amount of GPUs, increasing and decreasing the number of GPUs that are needs.It constantly checked for free GPUs, but it seems that I can not really free a GPU after I already sent an object to the GPU.<br>
For example:</p>
<pre><code class="lang-auto">import torch
tm = torch.Tensor([1,2]).to("cuda")
!nvidia-smi

|===============================+======================+======================|
|   0  GeForce RTX 208...  On   | 00000000:3D:00.0 Off |                  N/A |
|  0%   37C    P2    52W / 250W |    730MiB / 10989MiB |      0%      Default
</code></pre>
<p>So I use 730MiB…<br>
Now no matter what I try I can not make the 730MiB go to zero:</p>
<pre><code class="lang-auto">del tm                                                                                                                                                                                                 
torch.cuda.empty_cache()                                                                                                                                                                               
import sys;sys.modules[__name__].__dict__.clear()                                                                                                                                                      
%reset                                                                                                                                                                                                 
Once deleted, variables cannot be recovered. Proceed (y/[n])? y
!nvidia-smi
|   0  GeForce RTX 208...  On   | 00000000:3D:00.0 Off |                  N/A |
|  0%   35C    P8     1W / 250W |    728MiB / 10989MiB |      0%      Default |
</code></pre>
<p>Any help?<br>
Thanks</p>

albanD: <p>Hi,</p>
<p>I’m afraid this is the cuda driver memory use.<br>
As you can see in your first example: putting (almost) nothing on the GPU does the same.<br>
There is no way to clear that driver’s memory I’m afraid unless you stop your process <img src="https://discuss.pytorch.org/images/emoji/apple/confused.png?v=9" title=":confused:" class="emoji" alt=":confused:"></p>

orena1: <p>Thanks <a class="mention" href="/u/alband">@albanD</a> I was sure it is possible. Damm… It completely change the way the algorithm works… Thanks.<br>
I’ve opened a ticket  - <a href="https://github.com/pytorch/pytorch/issues/37664" rel="nofollow noopener">https://github.com/pytorch/pytorch/issues/37664</a></p>

albanD: <p>We’ll see what the cuda experts say. But this is most likely almost impossible to do <img src="https://discuss.pytorch.org/images/emoji/apple/confused.png?v=9" title=":confused:" class="emoji" alt=":confused:"></p>

orena1: <p>I’ve checked with TensorFlow and there is a workaround -<br>
<aside class="onebox stackexchange">
  <header class="source">
      <a href="https://stackoverflow.com/questions/39758094/clearing-tensorflow-gpu-memory-after-model-execution/51731238#51731238" target="_blank" rel="nofollow noopener">stackoverflow.com</a>
  </header>
  <article class="onebox-body">
      <a href="https://stackoverflow.com/users/4331167/liviaerxin" target="_blank" rel="nofollow noopener">
    <img alt="liviaerxin" src="https://www.gravatar.com/avatar/57f6185edb1171dbcff2e684db20a7e5?s=128&amp;d=identicon&amp;r=PG&amp;f=1" class="thumbnail onebox-avatar" width="60" height="60">
  </a>
<h4>
  <a href="https://stackoverflow.com/questions/39758094/clearing-tensorflow-gpu-memory-after-model-execution/51731238#51731238" target="_blank" rel="nofollow noopener">Clearing Tensorflow GPU memory after model execution</a>
</h4>

<div class="tags">
  <strong>python, tensorflow, gpu</strong>
</div>

<div class="date">
  
  answered by
  <a href="https://stackoverflow.com/users/4331167/liviaerxin" target="_blank" rel="nofollow noopener">
    liviaerxin
  </a>
  on <a href="https://stackoverflow.com/questions/39758094/clearing-tensorflow-gpu-memory-after-model-execution/51731238#51731238" target="_blank" rel="nofollow noopener">04:12PM - 07 Aug 18 UTC</a>
</div>

  </article>
  <div class="onebox-metadata">
    
    
  </div>
  <div style="clear: both"></div>
</aside>
<br>
But it is an annoying one… not sure how it would work with pytorch</p>

albanD: <p>This will work on pytorch as well: run your code in a different process. When this process finishes, it will free up all the GPU memory.</p>