Help moving code to run on a GPU - Expected object of device type cuda but got device type cpu for argument #1 'self' in call to _thnn_nll_loss_forward

AnswersPls: <pre><code class="lang-python">import torch
from torchvision import datasets, models, transforms
device = torch.device("cuda:0" if torch.cuda.is_available() else "cpu")
device

</code></pre>
<pre><code>device(type='cuda', index=0)
</code></pre>
<pre><code class="lang-python">mean = [0.485, 0.456, 0.406]
std = [0.229, 0.224, 0.225]
</code></pre>
<pre><code class="lang-python">train_transform = transforms.Compose([
                                transforms.Resize(256),
                                transforms.RandomResizedCrop(224),
                                transforms.RandomHorizontalFlip(),
                                transforms.ToTensor(),
                                transforms.Normalize(mean, std)])

test_transform = transforms.Compose([
                                transforms.Resize(256),
                                transforms.CenterCrop(224),
                                transforms.ToTensor(),
                                transforms.Normalize(mean, std)])
</code></pre>
<pre><code class="lang-python">#we perform these on the training data, test data is not randomly flipped or cropped
</code></pre>
<pre><code class="lang-python">pwd
</code></pre>
<pre><code>'C:\\Users\\cam12\\datasets'
</code></pre>
<pre><code class="lang-python">data_dir = 'datasetsToUse/HPV/'
</code></pre>
<pre><code class="lang-python">#holds training and test data
image_datasets ={}
</code></pre>
<pre><code class="lang-python">#image folder allows us to load images and apply a series of transformations on the, read everything from train and apply 
#transforms etc 
image_datasets['train']= datasets.ImageFolder(data_dir + '/HPVTraining', train_transform)
</code></pre>
<pre><code class="lang-python">image_datasets['test']= datasets.ImageFolder(data_dir + '/HPVVal', test_transform)
</code></pre>
<pre><code class="lang-python">#both stored as dictionaries
print("Training data size - %d" %  len(image_datasets['train']))
print("Test data size - %d" %  len(image_datasets['test']))
</code></pre>
<pre><code>Training data size - 24520
Test data size - 5996
</code></pre>
<pre><code class="lang-python">class_names = image_datasets['train'].classes
class_names
</code></pre>
<pre><code>['HPV+', 'HPV-']
</code></pre>
<pre><code class="lang-python">image_datasets['train'].class_to_idx
#This shows that HPV+ is a 0 and HPV positive is a 1
</code></pre>
<pre><code>{'HPV+': 0, 'HPV-': 1}
</code></pre>
<pre><code class="lang-python">dataloaders ={}
</code></pre>
<pre><code class="lang-python">dataloaders['train'] = torch.utils.data.DataLoader(image_datasets['train'],
                                                   batch_size=8,
                                                   shuffle=True,
                                                   num_workers=4)
</code></pre>
<pre><code class="lang-python">dataloaders['test'] = torch.utils.data.DataLoader(image_datasets['test'],
                                                  batch_size=8,
                                                  shuffle=True,
                                                  num_workers=4)
</code></pre>
<pre><code class="lang-python">dataloaders
</code></pre>
<pre><code>{'train': &lt;torch.utils.data.dataloader.DataLoader at 0x27fe824e3a0&gt;,
 'test': &lt;torch.utils.data.dataloader.DataLoader at 0x27fe824ee50&gt;}
</code></pre>
<pre><code class="lang-python">inputs, labels = next(iter(dataloaders['train']))
</code></pre>
<pre><code class="lang-python">inputs.shape
</code></pre>
<pre><code>torch.Size([8, 3, 224, 224])
</code></pre>
<pre><code class="lang-python">labels
#we have 2 folders in our training folder, 0 and 1 
</code></pre>
<pre><code>tensor([0, 0, 0, 0, 0, 0, 0, 0])
</code></pre>
<pre><code class="lang-python">import torchvision
#make grid to view using matplotlib, stack side by side
inp = torchvision.utils.make_grid(inputs)
</code></pre>
<pre><code class="lang-python">inp.shape
#include padding
</code></pre>
<pre><code>torch.Size([3, 228, 1810])
</code></pre>
<pre><code class="lang-python">inp.max()
#maxiumum value for RGB is 2.64
</code></pre>
<pre><code>tensor(2.5354)
</code></pre>
<pre><code class="lang-python">import numpy as np

np.clip(inp.cpu(), 0, 1).max()
</code></pre>
<pre><code>tensor(1.)
</code></pre>
<pre><code class="lang-python">inp.numpy().transpose((1, 2, 0)).shape
</code></pre>
<pre><code>(228, 1810, 3)
</code></pre>
<pre><code class="lang-python">import matplotlib.pyplot as plt

plt.ion()
</code></pre>
<pre><code class="lang-python">model = models.resnet18(pretrained=True)
model.to(device)
</code></pre>
<pre><code>ResNet(
  (conv1): Conv2d(3, 64, kernel_size=(7, 7), stride=(2, 2), padding=(3, 3), bias=False)
  (bn1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (relu): ReLU(inplace=True)
  (maxpool): MaxPool2d(kernel_size=3, stride=2, padding=1, dilation=1, ceil_mode=False)
  (layer1): Sequential(
    (0): BasicBlock(
      (conv1): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      (bn1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (relu): ReLU(inplace=True)
      (conv2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      (bn2): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    )
    (1): BasicBlock(
      (conv1): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      (bn1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (relu): ReLU(inplace=True)
      (conv2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      (bn2): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    )
  )
  (layer2): Sequential(
    (0): BasicBlock(
      (conv1): Conv2d(64, 128, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
      (bn1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (relu): ReLU(inplace=True)
      (conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      (bn2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (downsample): Sequential(
        (0): Conv2d(64, 128, kernel_size=(1, 1), stride=(2, 2), bias=False)
        (1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
    )
    (1): BasicBlock(
      (conv1): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      (bn1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (relu): ReLU(inplace=True)
      (conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      (bn2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    )
  )
  (layer3): Sequential(
    (0): BasicBlock(
      (conv1): Conv2d(128, 256, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
      (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (relu): ReLU(inplace=True)
      (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (downsample): Sequential(
        (0): Conv2d(128, 256, kernel_size=(1, 1), stride=(2, 2), bias=False)
        (1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
    )
    (1): BasicBlock(
      (conv1): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (relu): ReLU(inplace=True)
      (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    )
  )
  (layer4): Sequential(
    (0): BasicBlock(
      (conv1): Conv2d(256, 512, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
      (bn1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (relu): ReLU(inplace=True)
      (conv2): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      (bn2): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (downsample): Sequential(
        (0): Conv2d(256, 512, kernel_size=(1, 1), stride=(2, 2), bias=False)
        (1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
    )
    (1): BasicBlock(
      (conv1): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      (bn1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (relu): ReLU(inplace=True)
      (conv2): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      (bn2): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    )
  )
  (avgpool): AdaptiveAvgPool2d(output_size=(1, 1))
  (fc): Linear(in_features=512, out_features=1000, bias=True)
)
</code></pre>
<pre><code class="lang-python">#add last linear layer that works with our data, need number of input features
num_ftrs = model.fc.in_features
num_ftrs
</code></pre>
<pre><code>512
</code></pre>
<pre><code class="lang-python">import torch.nn as nn
</code></pre>
<pre><code class="lang-python">#takes in 512 features as extracted above, and it ouputs to 2, + or -
#This layer replaces the layer in the pretrained model
model.fc = nn.Linear(num_ftrs, 2)
</code></pre>
<pre><code class="lang-python">criterion = nn.CrossEntropyLoss()
criterion.to(device)
</code></pre>
<pre><code>CrossEntropyLoss()
</code></pre>
<pre><code class="lang-python">import torch.optim as optim

optimizer = optim.SGD(model.parameters(), 
                      lr=0.001, 
                      momentum=0.9)
</code></pre>
<pre><code class="lang-python">from torch.optim import lr_scheduler

exp_lr_scheduler = lr_scheduler.StepLR(optimizer, 
                                       step_size=7, 
                                       gamma=0.1)
</code></pre>
<pre><code class="lang-python">def calculate_accuracy(phase, running_loss, running_corrects):

    epoch_loss = running_loss / len(image_datasets[phase])
    epoch_acc = running_corrects.double() / len(image_datasets[phase])

    print('{} Loss: {:.4f} Acc: {:.4f}'.format( phase, epoch_loss, epoch_acc))
    
    return (epoch_loss, epoch_acc)
</code></pre>
<pre><code class="lang-python">def phase_train(model, criterion, optimizer, scheduler):
    
    
    model.train()
    running_loss = 0.0
    running_corrects = 0
    
    #setup for training, loop through inputs, zero gradients, enable for training
    #run predictions and extract predicted values, calculate loss
    #run backward and update optimiser
    for inputs, labels in dataloaders['train']:
        inputs, labels = inputs.to(device), labels.to(device)
        optimizer.zero_grad()
        
        with torch.set_grad_enabled(True):
            outputs = model(inputs)
            _, preds = torch.max(outputs, 1)
            loss = criterion(outputs, labels)
            
            loss.backward()
            optimizer.step()
            scheduler.step()
            
        running_loss += loss.item() * inputs.size(0)
        running_corrects += torch.sum(preds == labels.data)
    
    calculate_accuracy('train', running_loss, running_corrects) 
</code></pre>
<pre><code class="lang-python">#We are going to train the model and evaluate the accuracy,
#we will only save params that give best accuracy 
import copy
</code></pre>
<pre><code class="lang-python">best_acc = 0.0
</code></pre>
<pre><code class="lang-python">def phase_test(model, criterion, optimizer):
    
    model.eval()
    running_loss = 0.0
    running_corrects = 0
    global best_acc
    #keeps track of model weights that predicts best accuracy
    
    for inputs, labels in dataloaders['test']:
        inputs, labels = inputs.to(device), labels.to(device)
        optimizer.zero_grad()
        #run with no grads and zero grads before 
        with torch.no_grad():
            outputs = model(inputs)
            _, preds = torch.max(outputs, 1)
            loss = criterion(outputs, labels)
        #get predicted output and caclulate loss
        running_loss += loss.item() * inputs.size(0)
        running_corrects += torch.sum(preds == labels.data)
        
    epoch_loss, epoch_acc = calculate_accuracy('test', running_loss, running_corrects)
    #If we get a more accurate result - store the model params in best model wts 
    if epoch_acc &gt; best_acc:
        best_acc = epoch_acc
        best_model_wts = copy.deepcopy(model.state_dict())
        
    return best_model_wts
</code></pre>
<pre><code class="lang-python">def build_model(model, criterion, optimizer, scheduler, num_epochs=10):
    #will be updated if we getrr a better accuracy
    best_model_wts = copy.deepcopy(model.state_dict())

    for epoch in range(num_epochs):
        print('Epoch {}/{}'.format(epoch, num_epochs - 1))
        print('-' * 10)
        
        phase_train(model, criterion, optimizer, scheduler)
        best_model_wts = phase_test(model, criterion, optimizer)
        #once we train on an epoch - we update our model wts
        print()
    
    print('Best test Acc: {:4f}'.format(best_acc))

    model.load_state_dict(best_model_wts)
    return model
</code></pre>
<pre><code class="lang-python">#runs for 1 epoch just for demonstration purposes
print ("Got here")
model = build_model(model, 
                    criterion, 
                    optimizer, 
                    exp_lr_scheduler, 
                    num_epochs=1)
</code></pre>
<pre><code>Got here
Epoch 0/0
----------



---------------------------------------------------------------------------

RuntimeError                              Traceback (most recent call last)

&lt;ipython-input-143-05c4ef04f42d&gt; in &lt;module&gt;
      1 #runs for 1 epoch just for demonstration purposes
      2 print ("Got here")
----&gt; 3 model = build_model(model, 
      4                     criterion,
      5                     optimizer,


&lt;ipython-input-142-fa210670951b&gt; in build_model(model, criterion, optimizer, scheduler, num_epochs)
      7         print('-' * 10)
      8 
----&gt; 9         phase_train(model, criterion, optimizer, scheduler)
     10         best_model_wts = phase_test(model, criterion, optimizer)
     11         #once we train on an epoch - we update our model wts


&lt;ipython-input-138-9d206a49c1bd&gt; in phase_train(model, criterion, optimizer, scheduler)
     16             outputs = model(inputs)
     17             _, preds = torch.max(outputs, 1)
---&gt; 18             loss = criterion(outputs, labels)
     19 
     20             loss.backward()


~\anaconda3\lib\site-packages\torch\nn\modules\module.py in _call_impl(self, *input, **kwargs)
    720             result = self._slow_forward(*input, **kwargs)
    721         else:
--&gt; 722             result = self.forward(*input, **kwargs)
    723         for hook in itertools.chain(
    724                 _global_forward_hooks.values(),


~\anaconda3\lib\site-packages\torch\nn\modules\loss.py in forward(self, input, target)
    945 
    946     def forward(self, input: Tensor, target: Tensor) -&gt; Tensor:
--&gt; 947         return F.cross_entropy(input, target, weight=self.weight,
    948                                ignore_index=self.ignore_index, reduction=self.reduction)
    949 


~\anaconda3\lib\site-packages\torch\nn\functional.py in cross_entropy(input, target, weight, size_average, ignore_index, reduce, reduction)
   2420     if size_average is not None or reduce is not None:
   2421         reduction = _Reduction.legacy_get_string(size_average, reduce)
-&gt; 2422     return nll_loss(log_softmax(input, 1), target, weight, None, ignore_index, None, reduction)
   2423 
   2424 


~\anaconda3\lib\site-packages\torch\nn\functional.py in nll_loss(input, target, weight, size_average, ignore_index, reduce, reduction)
   2216                          .format(input.size(0), target.size(0)))
   2217     if dim == 2:
-&gt; 2218         ret = torch._C._nn.nll_loss(input, target, weight, _Reduction.get_enum(reduction), ignore_index)
   2219     elif dim == 4:
   2220         ret = torch._C._nn.nll_loss2d(input, target, weight, _Reduction.get_enum(reduction), ignore_index)


RuntimeError: Expected object of device type cuda but got device type cpu for argument #1 'self' in call to _thnn_nll_loss_forward
</code></pre>
<pre><code class="lang-python">with torch.no_grad():
    
    inputs, labels = iter(dataloaders['test']).next()
    inp = torchvision.utils.make_grid(inputs)
    
    outputs = model(inputs)
    _, preds = torch.max(outputs, 1)
    
    for j in range(len(inputs)):
        inp = inputs.data[j]
        img_show(inp, 'predicted:' + class_names[preds[j]])
</code></pre>
<pre><code class="lang-python">
</code></pre>
<h3>Freezing a model’s layers</h3>
<p>Here, we freeze all the layers in the network except the final layer.<br><br>
We need to set requires_grad == False to freeze the parameters so that the gradients are not computed in backward().</p>
<pre><code class="lang-python">frozen_model = models.resnet18(pretrained=True)
frozen_model.to(device)
</code></pre>
<pre><code class="lang-python">for param in frozen_model.parameters():
    param.requires_grad = False
</code></pre>
<h4>We add the final layer which is not frozen</h4>
<pre><code class="lang-python">frozen_model.fc = nn.Linear(num_ftrs, 5)
</code></pre>
<h4>Add an optimizer - for only the final layer</h4>
<p>There is no gradient calculation for the other layers, so we cannot use an optimizer for those</p>
<pre><code class="lang-python">optimizer = optim.SGD(frozen_model.fc.parameters(), 
                      lr=0.001, 
                      momentum=0.9)
</code></pre>
<h4>Create a scheduler for this model</h4>
<p>Just like with the earlier model, we attach a scheduler to this optimizer in order to slow down the learning rate</p>
<pre><code class="lang-python">exp_lr_scheduler = lr_scheduler.StepLR(optimizer, 
                                       step_size=7, 
                                       gamma=0.1)
</code></pre>
<h4>The loss calculation is the same CrossEntropyLoss as earlier</h4>
<p>This is just included here as a reminder as we are not changing the definition</p>
<pre><code class="lang-python">criterion = nn.CrossEntropyLoss()
</code></pre>
<h4>Reset the best_acc score</h4>
<p>This is so that the new frozen model can start from scratch</p>
<pre><code class="lang-python">best_acc = 0.0
</code></pre>
<h4>Build the new model with the frozen layers</h4>
<p>This is similar to what we did previously. Except this time, the gradient calculation is turned off for the parameters when a call is made to backward()</p>
<pre><code class="lang-python">frozen_model = build_model(frozen_model, 
                           criterion, 
                           optimizer, 
                           exp_lr_scheduler, 
                           num_epochs=8)
</code></pre>
<h4>We test a batch of flowers on this new model</h4>
<pre><code class="lang-python">with torch.no_grad():
    
    inputs, labels = iter(dataloaders['test']).next()
    inp = torchvision.utils.make_grid(inputs)
    
    outputs = frozen_model(inputs)
    _, preds = torch.max(outputs, 1)
    
    for j in range(len(inputs)):
        inp = inputs.data[j]
        img_show(inp, 'predicted:' + class_names[preds[j]])
</code></pre>
<pre><code class="lang-python">
</code></pre>
<pre><code class="lang-python">
</code></pre>

ptrblck: <p>Your code is quite unreadable at the moment, which makes debugging hard.<br>
You can post code snippets by wrapping them into three backticks ```.</p>
<p>Based on the error message, the model output or more likely the target tensor passed to <code>nn.CrossEntropyLoss</code> or <code>nn.NLLLoss</code> is on the CPU while it should be on the GPU.</p>

AnswersPls: <p>Sorry, I posted the thread out of frustration and didn’t really think about cleaning it up.</p>
<p>Does that look better now? All of the tensors that I can think of that are necessary have had the to(device) performed on them as is suggested in this article and to me anyway, thats pretty much my code looks.</p>
<p><a href="https://towardsdatascience.com/pytorch-switching-to-the-gpu-a7c0b21e8a99" class="onebox" target="_blank" rel="noopener nofollow ugc">https://towardsdatascience.com/pytorch-switching-to-the-gpu-a7c0b21e8a99</a></p>
<p>Any help is much appreciated!</p>

AnswersPls: <p>I fixed the issue and it’s running just fine now.</p>
<p>model.fc = nn.Linear(num_ftrs, 2).to(device)</p>
<p>I created this layer after i had already put the model onto the GPU in the code above it, and this layer defaults to the CPU so that just needed overidden</p>