RuntimeError: invalid argument 3: only batches of spatial targets supported (3D tensors) but got targets of dimension: 1 at /pytorch/aten/src/THNN/generic/SpatialClassNLLCriterion.c:61

username: video_analytics
cooked: <p>Hi</p>
<p>I am getting this error RuntimeError: invalid argument 3: only batches of spatial targets supported (3D tensors) but got targets of dimension: 1 at /pytorch/aten/src/THNN/generic/SpatialClassNLLCriterion.c:61</p>
<p>whenever I used my saved model for transfer learning</p>
<p>Can anyone please help me out with this problem ?</p>
<p>Thanks in advance</p>

username: ptrblck
cooked: <p>This error is usually raised, when you try to pass a spatial model output to a non-spatial target.<br>
Have a look at this small code to reproduce the issue:</p>
<pre><code class="lang-python"># Reproduced error
N, nb_classes, H, W = 1, 10, 24, 24
data = torch.randn(N, nb_classes, H, W, requires_grad=True)
target = torch.randint(0, nb_classes, (N,))

criterion = nn.CrossEntropyLoss()
loss = criterion(data, target) 

# Fixed by spatial target
target = torch.randint(0, nb_classes, (N, H, W))
loss = criterion(data, target)
</code></pre>
<p>The error is fixed once you provide a spatial target as <code>[N, H, W]</code>.</p>

username: video_analytics
cooked: <h1>Implementation of CNN/ConvNet Model using PyTorch (depicted in the picture above)</h1>
<p>class CNN(torch.nn.Module):</p>
<pre><code>def __init__(self):
    super(CNN, self).__init__()
    # L1 ImgIn shape=(?, 28, 28, 1)
    #    Conv     -&gt; (?, 28, 28, 32)
    #    Pool     -&gt; (?, 14, 14, 32)
    self.layer1 = torch.nn.Sequential(
        torch.nn.Conv2d(3, 32, kernel_size=3, stride=1, padding=1),
        torch.nn.ReLU(),
        torch.nn.MaxPool2d(kernel_size=2, stride=2),
        torch.nn.Dropout(p=1 - keep_prob))
    # L2 ImgIn shape=(?, 14, 14, 32)
    #    Conv      -&gt;(?, 14, 14, 64)
    #    Pool      -&gt;(?, 7, 7, 64)
    self.layer2 = torch.nn.Sequential(
        torch.nn.Conv2d(32, 64, kernel_size=3, stride=1, padding=1),
        torch.nn.ReLU(),
        torch.nn.MaxPool2d(kernel_size=2, stride=2),
        torch.nn.Dropout(p=1 - keep_prob))
    # L3 ImgIn shape=(?, 7, 7, 64)
    #    Conv      -&gt;(?, 7, 7, 128)
    #    Pool      -&gt;(?, 4, 4, 128)
    self.layer3 = torch.nn.Sequential(
        torch.nn.Conv2d(64, 128, kernel_size=3, stride=1, padding=1),
        torch.nn.ReLU(),
        torch.nn.MaxPool2d(kernel_size=2, stride=2, padding=1),
        torch.nn.Dropout(p=1 - keep_prob))

    # L4 FC 4x4x128 inputs -&gt; 625 outputs
    self.fc1 = torch.nn.Linear(4 * 4 * 8192, 240, bias=True)
    torch.nn.init.xavier_uniform(self.fc1.weight)
    self.layer4 = torch.nn.Sequential(
        self.fc1,
        torch.nn.ReLU(),
        torch.nn.Dropout(p=1 - keep_prob))
    # L5 Final FC 625 inputs -&gt; 10 outputs
    self.fc2 = torch.nn.Linear(240, 2, bias=True)
    torch.nn.init.xavier_uniform_(self.fc2.weight) # initialize parameters

def forward(self, x):
    out = self.layer1(x)
    out = self.layer2(out)
    out = self.layer3(out)
    out = out.view(out.size(0), -1)   # Flatten them for FC
    out = self.fc1(out)
    out = self.fc2(out)
    return out
</code></pre>
<h1>instantiate CNN model</h1>
<p>model = CNN()<br>
model</p>

username: video_analytics
cooked: <p>I have used the above CNN network for training two classes and then I want to reuse the trained network for training other two classes.</p>
<p>The first training part run successfully and I saved that model file but when I loading the trianed network and reusing it gives me the same run time error.</p>

username: ptrblck
cooked: <p>Could you print the shapes of your model output as well as the target before passing them to your criterion, please?<br>
I still think the error is due to a shape mismatch in your target tensor.</p>

username: video_analytics
cooked: <hr>
<pre><code>    Layer (type)               Output Shape         Param #
</code></pre>
<h1>================================================================<br>
Conv2d-1        [128, 32, 250, 250]             896<br>
ReLU-2        [128, 32, 250, 250]               0<br>
MaxPool2d-3        [128, 32, 125, 125]               0<br>
Dropout-4        [128, 32, 125, 125]               0<br>
Conv2d-5        [128, 64, 125, 125]          18,496<br>
ReLU-6        [128, 64, 125, 125]               0<br>
MaxPool2d-7          [128, 64, 62, 62]               0<br>
Dropout-8          [128, 64, 62, 62]               0<br>
Conv2d-9         [128, 128, 62, 62]          73,856<br>
ReLU-10         [128, 128, 62, 62]               0<br>
MaxPool2d-11         [128, 128, 32, 32]               0<br>
Dropout-12         [128, 128, 32, 32]               0<br>
Linear-13                 [128, 240]      31,457,520<br>
Linear-14                 [128, 240]      31,457,520<br>
Linear-15                   [128, 2]             482</h1>
<h2>Total params: 63,008,770<br>
Trainable params: 63,008,770<br>
Non-trainable params: 0</h2>
<p>Input size (MB): 91.55<br>
Forward/backward pass size (MB): 8533.91<br>
Params size (MB): 240.36<br>
Estimated Total Size (MB): 8865.82</p>

username: video_analytics
cooked: <p>this is overall model shape so I am intially passing images of [3, 250, 250] with 2 classes and once tge model is trained I want to retrain it using smaller amount of dataset but it is giving me error…</p>

username: DoubleEleven1111
cooked: <p>Hi, I’m a new learner and I have a same question, the error is: only batches of spatial targets supported (3D tensors) but got targets of dimension: 1. My target size is [128], my predict data size is [128, 256, 6, 1000]. I noticed your solution which is provide a spatial target as  <code>[N, H, W]</code> . But I don’t know how to do it . I’ve google it for many times but I still didn’t find a way to do it. Can you give me some examples about how to provide a spatial target as  <code>[N, H, W]</code> . Thanks a lot.</p>

username: ptrblck
cooked: <p>You shouldn’t simply reshape your target to a spatial shape, if you have scalar values.<br>
Instead of changing the target, you should change your model architecture to return logits in the right shape.<br>
Based on the provided target shape (multi-class classification), your model should return logits in the shape <code>[batch_size, nb_classes]</code>.</p>

username: DoubleEleven1111
cooked: <p>Thank you! It works😉</p>