Cuda Out of memory when model training started

bolt25: <p>When I tried to run my training loop</p>
<pre><code class="lang-auto">iters=0
lossesTrain = []
lossesTest = []


print('Training Started!')

torch.backends.cudnn.benchmark = True # Optimizes cudnn
for epochs in range(num_epochs):
    for i,(c, n) in enumerate(trainLoader):
        
        #converting images and labels to variables so as to calculate gradient 
        if torch.cuda.is_available():
            clean = Variable(c.cuda())
            noisy = Variable(n.cuda())
        else:
            clean = Variable(c)
            noisy = Variable(n)
            
            
        #clearing gradient buffers
        optimizer.zero_grad()

        #finding outputs
        output = model(noisy)

        #calculating losses
        loss=criterion(output,clean)

        #backpropagating the loss
        loss.backward()

        #updating the parameters
        optimizer.step()

        #updating counter
        iters+=1
        
        print(i)
        if iters % 25 == 0:

            for c,n in testLoader:
                cleanT = Variable(c)
                noisyT = Variable(n)
                if torch.cuda.is_available():
                    cleanT = Variable(c.cuda())
                    noisyT = Variable(n.cuda())
                else:
                    cleanT = Variable(c)
                    noisyT = Variable(n)
                
                outputT = model(noisyT)
                
                lossesTest.append(criterion(outputT, cleanT).detach())
                lossesTrain.append(criterion(output, clean).detach())
                
                print('Epoch: {}, Iter: {}, lossesTrain: {}, lossesTest: {}'.format(epochs, iters, sum(lossesTrain[-1]), sum(lossesTest[-1])))
                
#             print(plt.imshow(copy.reshape((512,512,-1)).detach().numpy()))
print('Training Done!! Save the model')
</code></pre>
<p>I got this as a RunTimeError:-<br>
<code>RuntimeError: CUDA out of memory. Tried to allocate 2.00 GiB (GPU 0; 4.00 GiB total capacity; 1.13 GiB already allocated; 212.59 MiB free; 2.75 GiB reserved in total by PyTorch)</code></p>
<p>Any ideas on what could be done to avoid this and why is this happening?</p>

chinmay5: <p>Try to reduce the batch size and see if it is able to train. The error seems to indicate you do not have sufficient memory but it would be better to confirm that. Try and run your code with a smaller batch size</p>

bolt25: <p>I halved my batchSize, but I’m still getting the same error message.</p>

chinmay5: <p>Maybe your model is itself too big. I would run the model with batch size of 1 and see how much memory the model itself needs. That gives us an idea about how much data we can upload without OOM error</p>

bolt25: <p>My model has around 195k trainable parameters. So i dont think that model size would be an issue.</p>

chinmay5: <p>Does it work with a smaller batch size like 2, 4, 8 etc?</p>

bolt25: <p>I’ll have to try with these values. I’ll text back here with results</p>

bolt25: <p>Even with batchSize of 1… same error</p>

chinmay5: <aside class="onebox whitelistedgeneric">
  <header class="source">
      <img src="https://github.githubassets.com/favicons/favicon.svg" class="site-icon" width="16" height="16">
      <a href="https://github.com/jacobkimmel/pytorch_modelsize" target="_blank" rel="nofollow noopener">GitHub</a>
  </header>
  <article class="onebox-body">
    <img src="https://avatars2.githubusercontent.com/u/2465221?s=400&amp;v=4" class="thumbnail onebox-avatar" width="60" height="60">

<h3><a href="https://github.com/jacobkimmel/pytorch_modelsize" target="_blank" rel="nofollow noopener">jacobkimmel/pytorch_modelsize</a></h3>

<p>Estimates the size of a PyTorch model in memory. Contribute to jacobkimmel/pytorch_modelsize development by creating an account on GitHub.</p>


  </article>
  <div class="onebox-metadata">
    
    
  </div>
  <div style="clear: both"></div>
</aside>

<p>Let us use this to get the model size because if batch size is 1 and still you run into the issue, it is bad. Also, most likely you should be able to run training for a few iterations and then get OOM becuase you are also putting val set onto GPU along with the train. But first thing would be to test the model size</p>