Torch.not enough memory

vahid_sadeghi: <p>Hello all<br>
I tried to run a convolutional neural network on GPU. for the first time it run and I got the results but for the second time  I got the below error while I didn’t change my code at all and it is weird.</p>
<p>$Torch: not enough memory: you tried to allocate 0GB. Buy new RAM!</p>
<p>I couldn’t figure out why this might be happening.<br>
how can I solve this problem?<br>
Any help will be appreciated<br>
here is my code:</p>
<h1>-<em>- coding: utf-8 -</em>-</h1>
<p>“”"<br>
Created on Thu Nov  7 13:25:00 2019</p>
<p><span class="mention">@author</span>: Vahid<br>
“”"<br>
from torch.autograd import Variable<br>
import torch<br>
import torch.cuda as cuda<br>
import torchvision<br>
from torchvision import transforms<br>
import torch.nn as nn<br>
import matplotlib.pyplot as plt</p>
<h1>Parameters</h1>
<p>batch_size =64<br>
n_class =4<br>
lr = 0.001<br>
num_epochs =25<br>
transform=transforms.Compose([transforms.Resize((128,128)),<br>
transforms.ToTensor(),<br>
transforms.Normalize((.5,.5,.5),(.5,.5,.5))])</p>
<h1>Load Custom Dataset</h1>
<p>train_dataset = torchvision.datasets.ImageFolder(‘H:\vahid-spectrogram-class1\train’,<br>
transform=transform)<br>
valid_dataset = torchvision.datasets.ImageFolder(‘H:\vahid-spectrogram-class1\validation’,<br>
transform=transform)<br>
test_dataset = torchvision.datasets.ImageFolder(‘H:\vahid-spectrogram-class1\test’,<br>
transform=transform)<br>
train_loader = torch.utils.data.DataLoader(dataset=train_dataset,<br>
batch_size=batch_size,<br>
shuffle=True)<br>
valid_loader = torch.utils.data.DataLoader(dataset=valid_dataset,<br>
batch_size=batch_size,<br>
shuffle=True)<br>
test_loader = torch.utils.data.DataLoader(dataset=test_dataset,<br>
batch_size=batch_size,<br>
shuffle=True)</p>
<h1>Convolutional neural network</h1>
<p>class convnet(nn.Module):<br>
def <strong>init</strong>(self):<br>
super(convnet,self).<strong>init</strong>()<br>
self.layer1 = nn.Sequential(nn.Conv2d(3,16, 5,1,2),<br>
nn.BatchNorm2d(16),<br>
nn.ReLU(),<br>
nn.MaxPool2d(2, 2))<br>
self.layer2 = nn.Sequential(nn.Conv2d(16, 8, 5,1,2),<br>
nn.BatchNorm2d(8),<br>
nn.ReLU(),<br>
nn.MaxPool2d(2, 2))<br>
self.layer3 = nn.Sequential(nn.Conv2d(8,4, 5,1,2),<br>
nn.BatchNorm2d(4),<br>
nn.ReLU(),<br>
nn.MaxPool2d(2, 2))<br>
self.fc1 = nn.Linear(16<em>16</em>4, 1000)<br>
self.drop_out = nn.Dropout(p=.75)<br>
self.fc2 = nn.Linear(1000,n_class)</p>
<pre><code>def forward(self, x):
    out = self.layer1(x)
    out = self.layer2(out)
    out = self.layer3(out)
    out = out.reshape(out.size(0), -1)
    out = self.fc1(out)
    out = self.drop_out(out)
    out = self.fc2(out)
    return out
</code></pre>
<h1>Model CNN</h1>
<p>convmodel = convnet()<br>
if cuda.is_available():<br>
convmodel=convmodel.cuda()</p>
<h1>loss</h1>
<p>loss_fn = nn.CrossEntropyLoss()</p>
<h1>Optimizer</h1>
<p>optimizer= torch.optim.Adam(convmodel.parameters(), lr=lr)</p>
<h1>LR</h1>
<p>lr_sch = torch.optim.lr_scheduler.StepLR(optimizer, 1, gamma=0.5)<br>
num_steps = len(train_loader)<br>
valid_num_steps = len(valid_loader)<br>
train_loss = []<br>
valid_loss = []<br>
train_accuracy = []<br>
valid_accuracy = []<br>
for epoch in range(num_epochs):<br>
iter_loss = 0.0<br>
correct = 0<br>
iterations = 0<br>
convmodel.train()<br>
<span class="hashtag">#print</span>(lr_sch.get_lr())<br>
for j, (imgs, lbls) in enumerate(train_loader):</p>
<pre><code>            # Convert torch tensor to Variable
    imgs= Variable(imgs)
    lbls= Variable(lbls)
    
    # If we have GPU, shift the data to GPU
    if cuda.is_available():
        imgs= imgs.cuda()
        lbls =lbls.cuda()
    optimizer.zero_grad()
    out = convmodel(imgs)
    loss_tr = loss_fn(out, lbls)
    iter_loss += loss_tr.item()
    loss_tr.backward()
    optimizer.step()
    _, predicted = torch.max(out.data, 1)
    correct += (predicted == lbls.data).sum()
    #if (j+1) % 1 == 0:
    # print('Train, Epoch [{}/{}] Step [{}/{}] Loss_Train: {:.2f} Accuracy_Train:{:.2f}'.format(epoch+1, num_epochs, j+1, num_steps, loss_tr.item(),100*(correct.item()/(batch_size*(j+1)))))
    iterations += 1
    # Record the training loss
train_loss.append(iter_loss/iterations)
# Record the training accuracy
train_accuracy.append(100*correct/39888)
loss= 0.0
correct = 0
iterations = 0
convmodel.eval() 
for j, (items, classes) in enumerate(valid_loader):
    
    items=Variable(items)
    classes=Variable(classes)
            # If we have GPU, shift the data to GPU
    if cuda.is_available():
        items = items.cuda()
        classes = classes.cuda()
    out= convmodel(items)      # Do the forward pass
    loss += loss_fn(out,classes.data)# Calculate the loss
    _, predicted = torch.max(out.data, 1)
    correct += (predicted == classes.data).sum()
    #if (j+1) % 1 == 0:
      #print('validation, Epoch [{}/{}] Step [{}/{}] Loss_Validation: {:.2f} Accuracy_Validation:{:.2f} '.format(epoch+1, num_epochs, j+1, valid_num_steps, loss_va.item(),100*(correct.item()/(batch_size*(j+1)))))

    iterations += 1

# Record the validation loss
valid_loss.append(loss/iterations)
# Record the validation accuracy
valid_accuracy.append(correct/8568)
print ('Epoch %d/%d, Tr Loss: %.4f, Tr Acc: %.4f, Val Loss: %.4f, Val Acc: %.4f'
       %(epoch+1, num_epochs, train_loss[-1], train_accuracy[-1], 
         valid_loss[-1], valid_accuracy[-1]))
</code></pre>
<p>convmodel.eval()<br>
corrects = 0<br>
num_steps = len(test_loader)<br>
for j, (imgs, lbls) in enumerate(test_loader):<br>
out = convmodel(imgs)<br>
predicted = torch.argmax(out, 1)<br>
corrects += torch.sum(predicted == lbls)<br>
print('Step [{}/{}] Acc {:.4f}: '.format(j+1, num_steps, 100.*corrects/((j+1)*batch_size)))</p>
<p>f = plt.figure(figsize=(10, 8))<br>
plt.plot(train_loss, label=‘training loss’)<br>
plt.plot(valid_loss, label=‘validation loss’)<br>
plt.legend()<br>
plt.show()</p>
<h1>In[13]:</h1>
<h1>Accuracy</h1>
<p>f = plt.figure(figsize=(10, 8))<br>
plt.plot(train_accuracy, label=‘training accuracy’)<br>
plt.plot(valid_accuracy, label=‘validation accuracy’)<br>
plt.legend()<br>
plt.show()</p>

albanD: <p>This error means that you are running out of RAM. Do you have other people using the same machine at the same time? Can you check if other processes are using RAM on your machine?</p>

vahid_sadeghi: <p>Thanks for your answer<br>
For the first time i run my code and i got good results but for the second time i got the Cuda error:not enough memory<br>
Only my code is run on the system<br>
Would you hepl me to solve the problem?<br>
I get stuck in for a couple of<br>
Days</p>

albanD: <p>A cuda error? The error you mentionned above  “Torch: not enough memory: you tried to allocate 0GB. Buy new RAM!” is about CPU memory only.<br>
Or do you see another error message?</p>