CUDA is out of memory

username: zhishen_nie
cooked: <p>Recently, I use pytorch to generate some adversarial samples, and the algothim is FGSM. Just like<br>
<a href="https://pytorch.org/tutorials/beginner/fgsm_tutorial.html" rel="nofollow noopener">https://pytorch.org/tutorials/beginner/fgsm_tutorial.html</a> . It was very smooth at the beginning of mu program. But soon pytorch told me that cuda is out of memory. I guess there will be a part of the GPU memory has not been released. And I know torch.cuda.empty_cache(）but it didn’t work. So how could I resolve this problems? Thanks for anyone who could help me! <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=9" title=":slight_smile:" class="emoji" alt=":slight_smile:"></p>

username: zhishen_nie
cooked: <p>def fgsm_attack(image, epsilon, data_grad):</p>
<pre><code>sign_data_grad = data_grad.sign()

perturbed_image = image + epsilon*sign_data_grad

perturbed_image = t.clamp(perturbed_image, 0, 255)

return perturbed_image
</code></pre>
<p>def gen_adv(model, device, data, target, epsilon):</p>
<pre><code>data, target = data.to(device), target.to(device)
data.requires_grad = True

output = model(data)
init_pred = output.max(1, keepdim=True)[1]
if init_pred.item() != target.item():
    return

loss = F.nll_loss(output, target)
model.zero_grad()
loss.backward()
data_grad = data.grad.data
data = unnormalized_show(data)

perturbed_data = fgsm_attack(data, epsilon, data_grad)
perturbed_data = perturbed_data.cpu().detach().numpy()
perturbed_data = perturbed_data.reshape(3, 224, 224)
perturbed_data = np.transpose(perturbed_data, (1, 2, 0))

return perturbed_data</code></pre>

username: albanD
cooked: <p>Hi,</p>
<p>Have you tried reducing the batch size?<br>
Also make sure that you do not save any Tensor in a list or something similar during training.</p>

username: zhishen_nie
cooked: <aside class="quote no-group" data-username="albanD" data-post="3" data-topic="70208">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/alband/48/215_2.png" class="avatar"> albanD:</div>
<blockquote>
<p>make sure that you do not save any Tensor in a list or something similar during tr</p>
</blockquote>
</aside>
<p>Well, the model has been trained,  and when I use the model to generate adversarial samples , the problem appear<br>
the error is :<br>
Traceback (most recent call last):<br>
File “gen_adv.py”, line 28, in <br>
perturbed_data = Attack.gen_adv(model, device, data, target, eps)<br>
File “/content/drive/My Drive/Colab Notebooks/animal10/Attack.py”, line 42, in gen_adv<br>
loss.backward()<br>
File “/usr/local/lib/python3.6/dist-packages/torch/tensor.py”, line 195, in backward<br>
torch.autograd.backward(self, gradient, retain_graph, create_graph)<br>
File “/usr/local/lib/python3.6/dist-packages/torch/autograd/<strong>init</strong>.py”, line 99, in backward<br>
allow_unreachable=True)  # allow_unreachable flag<br>
RuntimeError: CUDA out of memory. Tried to allocate 392.00 MiB (GPU 0; 7.43 GiB total capacity; 5.78 GiB already allocated; 392.94 MiB free; 6.53 GiB reserved in total by PyTorch)</p>
<p>It seems that "  loss.backward() " is the key, and I don’t save any Tensor in a list or something similar…<br>
The model can withstand the input of dozens of pictures, but when the number of pictures exceeds thousands, the program fails…<br>
Thanks for answer my question:)</p>

username: JakeAndFinn
cooked: <p>Sometimes I found the restarting you computer works.<br>
It may have something in the memory or holding on to something that you dont know.<br>
If it still keeps doing it, make a data loader and do mini-batches.<br>
That may help</p>

username: zhishen_nie
cooked: <p>well, I find the key of this problem. The following function, each time it runs, the GPU memory consumed will increase by 1mb. But I really don’t know how to solve it. QAQ</p>
<p>def gen_adv(model, device, data, target, epsilon):</p>
<pre><code>data, target = data.to(device), target.to(device)


data.requires_grad = True

# 1067mb  1068mb
output = model(data)
# 1202mb  1203mb

init_pred = output.max(1, keepdim=True)[1]
if init_pred.item() != target.item():
    return
 # 1202mb  1203mb
loss = F.nll_loss(output, target)
 # 1202mb  1203mb
model.zero_grad()
 # 1202mb  1203mb
loss.backward()
 # 1068mb  1069mb
data_grad = data.grad.data
 # 1068mb  1069mb
data = unnormalized_show(data)
 # 1068 mb 1069mb


perturbed_data = fgsm_attack(data, epsilon, data_grad)
perturbed_data = perturbed_data.cpu().detach().numpy()


perturbed_data = perturbed_data.reshape(3, 224, 224)
perturbed_data = np.transpose(perturbed_data, (1, 2, 0))

return perturbed_data</code></pre>

username: ptrblck
cooked: <p>Duplicate post from <a href="https://discuss.pytorch.org/t/1mb-gpu-memory-cant-be-relased/70327">here</a>.<br>
Let’s please keep the discussion in your created topic.</p>