RuntimeError: only batches of spatial targets supported (3D tensors) but got targets of dimension: 1 I keep getting this error

Chedi_Morchdi: <blockquote>
<p>Hello, I’m trying to train this model:</p>
</blockquote>
<p>class Perceptron(nn.Module):</p>
<blockquote>
<pre><code>def __init__(self):
    super(Perceptron, self).__init__()
    self.fc = nn.Linear(28,10)
    self.relu = torch.nn.ReLU() 
def forward(self, x):
    output = self.fc(x)
    output = self.relu(x) 
    return output
</code></pre>
</blockquote>
<p>On the MNIST dataset uploaded as follows:<br>
trainset = torchvision.datasets.MNIST(root=‘./data’, train=True, download=True,</p>
<blockquote>
<pre><code>                         transform=torchvision.transforms.Compose([
                           torchvision.transforms.ToTensor(),
                           torchvision.transforms.Normalize(
                             (0.1307,), (0.3081,))
                         ]))
</code></pre>
</blockquote>
<p>trainloader = torch.utils.data.DataLoader(trainset, batch_size=batch_size_train,<br>
shuffle=True, num_workers=2, pin_memory=pin_mem)</p>
<p>But I get this error:<br>
RuntimeError: only batches of spatial targets supported (3D tensors) but got targets of dimension: 1</p>
<p>I’m using nn.CrossEntropyLoss()</p>

ptrblck: <p>Your model output shape is most likely wrong since your model architecture seems to apply the linear and relu layers on a 4-dimensional input and would thus output <code>[batch_size, 1, 28, 10]</code> which I doubt is your intended use case.<br>
Flatten the inputs before via <code>x = x.view(x.size(0), -1)</code> and change the <code>in_features</code> of the linear layer to <code>28*28</code>.</p>

Chedi_Morchdi: <aside class="quote no-group" data-username="ptrblck" data-post="2" data-topic="166387">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/48/1823_2.png" class="avatar"> ptrblck:</div>
<blockquote>
<p>x = x.view(x.size(0), -1)</p>
</blockquote>
</aside>
<p>Thank you for your answer, I did that, now i’m getting this error:<br>
RuntimeError: element 0 of tensors does not require grad and does not have a grad_fn</p>

ptrblck: <p>Could you post a minimal and executable code snippet showing this error, please? The <code>view</code> operation won’t detach the computation graph so something else must be wrong in your code.</p>

Chedi_Morchdi: <aside class="onebox allowlistedgeneric" data-onebox-src="https://colab.research.google.com/drive/1BjCzuo6bBjJqDs56_GzydM5QCuzn5jrk#scrollTo=ULgwcyAwlRKO&amp;line=115&amp;uniqifier=1">
  <header class="source">
      <img src="https://colab.research.google.com/img/favicon.ico?vrz=colab-20221117-060045-RC00_489179157" class="site-icon" width="128" height="128">

      <a href="https://colab.research.google.com/drive/1BjCzuo6bBjJqDs56_GzydM5QCuzn5jrk#scrollTo=ULgwcyAwlRKO&amp;line=115&amp;uniqifier=1" target="_blank" rel="noopener nofollow ugc">colab.research.google.com</a>
  </header>

  <article class="onebox-body">
    <img src="https://colab.research.google.com/img/colab_favicon_256px.png" class="thumbnail onebox-avatar" width="260" height="260">

<h3><a href="https://colab.research.google.com/drive/1BjCzuo6bBjJqDs56_GzydM5QCuzn5jrk#scrollTo=ULgwcyAwlRKO&amp;line=115&amp;uniqifier=1" target="_blank" rel="noopener nofollow ugc">Google Colaboratory</a></h3>



  </article>

  <div class="onebox-metadata">
    
    
  </div>

  <div style="clear: both"></div>
</aside>


Chedi_Morchdi: <p>In the snippet i’m trying to gather the gradients. I tried x.requires_grad(), it solved the first error, but then i found that the param.grad were none</p>

ptrblck: <p>I don’t have access to your notebook as it’s restricted so please post the code snippet here.</p>

Chedi_Morchdi: <blockquote>
<p>import torch<br>
import random<br>
import torchvision<br>
import torchvision.transforms as transforms<br>
import matplotlib.pyplot as plt<br>
import numpy as np<br>
import torch.nn as nn<br>
import torch.nn.functional as F<br>
from torch.optim.optimizer import Optimizer<br>
from scipy import stats<br>
from torch.optim.optimizer import Optimizer, required<br>
import seaborn as sns<br>
import matplotlib.pylab as plt<br>
import scipy<br>
import torchvision.models as models<br>
import copy<br>
import sys<br>
import matplotlib as mpl</p>
<p>seed=142857<br>
torch.manual_seed(seed)<br>
torch.cuda.manual_seed(seed)<br>
torch.cuda.manual_seed_all(seed)  # if you are using multi-GPU.<br>
np.random.seed(seed)  # Numpy module.<br>
random.seed(seed)  # Python random module.<br>
torch.manual_seed(seed)<br>
torch.backends.cudnn.benchmark = False<br>
torch.backends.cudnn.deterministic = True</p>
<p>batch_size_train= 20 <span class="hashtag-raw">#batch_size</span><br>
tr_split_len= 20 <span class="hashtag-raw">#number_of_total</span> training data samples</p>
</blockquote>
<blockquote>
<p>train_on_gpu=True<br>
lr=0.01<br>
num_epochs=30<br>
cuda = torch.device(‘cuda’)</p>
<p>device = torch.device(‘cuda’ if torch.cuda.is_available() else ‘cpu’)</p>
<p>te_split_len=10<br>
batch_size_test=1	<br>
pin_mem=True</p>
</blockquote>
<blockquote>
<p>trainset = torchvision.datasets.MNIST(root=‘./data’, train=True, download=True,</p>
</blockquote>
<blockquote>
<pre><code>                           transform=torchvision.transforms.Compose([
                             torchvision.transforms.ToTensor(),
                           torchvision.transforms.Normalize(
                             (0.1307,), (0.3081,))
                         ]))
</code></pre>
<p>part_tr = torch.utils.data.random_split(trainset, [tr_split_len, len(trainset)-tr_split_len])[0]</p>
<p>trainloader = torch.utils.data.DataLoader(part_tr, batch_size=batch_size_train,<br>
shuffle=True, num_workers=2, pin_memory=pin_mem)</p>
</blockquote>
<blockquote>
<p>class Perceptron(nn.Module):</p>
</blockquote>
<blockquote>
<pre><code>def __init__(self):
    super(Perceptron, self).__init__()
    self.fc = nn.Linear(28*28,10)
    self.relu = torch.nn.ReLU() 
def forward(self, x):
    x = x.view(x.size(0), -1)
    output = self.fc(x)

    output = self.relu(x) 
    return output
</code></pre>
<p>criterion1 = nn.CrossEntropyLoss()</p>
<p>net1=Perceptron()</p>
</blockquote>
<blockquote></blockquote>
<blockquote>
<p>net1 = nn.DataParallel(net1)<br>
net1.to(device)</p>
<p>optimizer1 = torch.optim.SGD(net1.parameters(), lr=lr, momentum=0)</p>
<p>for epoch in range(num_epochs):  # loop over the dataset multiple times</p>
<pre><code>epoch_loss = 0.0
for i, data in enumerate(trainloader, 0):
    # get the inputs; data is a list of [inputs, labels]
    inputs, labels = data
    if train_on_gpu:
      inputs, labels = inputs.to(device), labels.to(device)
    
    current_grads=[]
    # zero the parameter gradients
    optimizer1.zero_grad()

    # forward + backward + optimize
    outputs = net1(inputs)

    
    loss = criterion(outputs, labels)

    loss.backward()
        
    for param in net1.parameters():
      current_grads.append(param.grad.view(-1))
    current_grads=torch.cat(current_grads)
    grads.append(current_grads)

    

    
    optimizer1.step()
    epoch_loss += loss.item()


print('epoch_loss',epoch,':',epoch_loss)
</code></pre>
</blockquote>

ptrblck: <p>You are passing <code>x</code> again to <code>self.relu</code> instead of <code>output</code>, which makes the output non-differentiable, since no trainable parameters were used.<br>
After fixing this as well as more issues in your code, which made it non-executable, the code works fine for me.</p>

Chedi_Morchdi: <p>it worked, thank you</p>