GPU memory doesn't released

masawdah: <p>I’m trying to send my CNN model to the GPU device, but each time I run <code> model = model.to(device)</code> I got an error “RuntimeError: CUDA error: out of memory”.</p>
<p>I tried to use</p>
<pre><code class="lang-auto">import torch
torch.cuda.empty_cache()
</code></pre>
<p>but that did not work, I’ve restarted the Kernal but that didn’t solve the problem. I checked the free/used memory, it looks full, I’ve tried to clean the memory using <code>torch.cuda.empty_cache()</code> that did not work, the below image shows the free/used memory.</p>
<p>I don’t have any idea why this error pops-up even I don’t send or train any model on the GPU.</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/4/e/4e371a37be8d572bb4413cfb7972cf8adfdb0957.jpeg" data-download-href="https://discuss.pytorch.org/uploads/default/4e371a37be8d572bb4413cfb7972cf8adfdb0957" title="nivida_gpu"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/4/e/4e371a37be8d572bb4413cfb7972cf8adfdb0957_2_689x172.jpeg" alt="nivida_gpu" data-base62-sha1="b9VixCTJg5AhyPHHchFheMXx6U7" width="689" height="172" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/4/e/4e371a37be8d572bb4413cfb7972cf8adfdb0957_2_689x172.jpeg, https://discuss.pytorch.org/uploads/default/optimized/3X/4/e/4e371a37be8d572bb4413cfb7972cf8adfdb0957_2_1033x258.jpeg 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/4/e/4e371a37be8d572bb4413cfb7972cf8adfdb0957.jpeg 2x" data-dominant-color="EFEFF0"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">nivida_gpu</span><span class="informations">1293×323 41.1 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

gphilip: <ul>
<li>Is someone else using the same machine to run GPU-intensive tasks?</li>
<li>Are you running another GPU-intensive task (e.g.: a game, or 3D-rendering)?</li>
</ul>

arya47: <p>Try these following steps to figure out where the problem is:</p>
<ul>
<li>Use the model with a simple most basic training loop. If the problem is solved check your training loop for any accumulation or (maybe make sure you’re not calling model.to(device) for each epoch)</li>
<li>If the error persists try using a different model. To find out if the error is in your model. (Has happened to me before and was  due to some linear layers)</li>
</ul>
<p>Simple training loop: (no autograd or scalers)</p>
<pre><code class="lang-auto">
def train_loop(dataloader, model, loss_fn, optimizer):
    size = len(dataloader.dataset)
    for batch, (X, y) in enumerate(dataloader):
        pred = model(X)
        loss = loss_fn(pred, y)
        optimizer.zero_grad()
        loss.backward()
        optimizer.step()
    
        if batch%100 ==0:
            loss, current = loss.item(), batch*len(X)
            print(f"loss: {loss:&gt;7f}  [{current:&gt;5d}/{size:&gt;5d}]")

epochs = 10
for t in range(epochs):
    print(f"Epoch {t+1}\n-------------------------------")
    train_loop(train_dataloader, model, loss_fn, optimizer)
print("Done!")


</code></pre>

masawdah: <p>No, it’s an online platform and I don’t run anything else</p>

masawdah: <p>Thanks for reply, the problem with sending the model to GPU, I didn’t reach training step</p>

arya47: <p>Can you share the model?</p>

masawdah: <p>Thanks, It was a problem within the platform and they’re working to fix it</p>