Training working but getting error when testing

username: PyAP
cooked: <p>When I’m running my CNN the training looks to be working as the loss is decreasing through each Epoch but when it reaches the testing I’m getting this error to do with tensor size.</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/8/d/8dcbd7d22c12f5d30d41dc7d9a5944128d0b9b04.png" data-download-href="https://discuss.pytorch.org/uploads/default/8dcbd7d22c12f5d30d41dc7d9a5944128d0b9b04" title="image"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/8/d/8dcbd7d22c12f5d30d41dc7d9a5944128d0b9b04_2_690x236.png" alt="image" data-base62-sha1="keo3pDdGiK0jXgXjrdIemxoeuBC" width="690" height="236" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/8/d/8dcbd7d22c12f5d30d41dc7d9a5944128d0b9b04_2_690x236.png, https://discuss.pytorch.org/uploads/default/original/3X/8/d/8dcbd7d22c12f5d30d41dc7d9a5944128d0b9b04.png 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/8/d/8dcbd7d22c12f5d30d41dc7d9a5944128d0b9b04.png 2x" data-dominant-color="312F30"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">871×298 31 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>Here is my code for the CNN and training and testing. Batch size is 256, and the output I am trying to classify out of 11 different signals. Not sure why it working during training but not testing</p>
<p>‘’'<span class="hashtag-raw">#Input</span> [N , 1 , 2, 128]<br>
class ConvNet(nn.Module):<br>
def <strong>init</strong>(self,dr=0.5,in_channels=1,num_classes=11):<br>
super(ConvNet,self).<strong>init</strong>()<br>
self.pad = nn.ConstantPad2d((2,2,0,0),0)<br>
self.conv1 = nn.Conv2d(1,256,(1,3)) <span class="hashtag-raw">#First</span> convolutional Layer (Input_channels, Output_channels, Kernel)<br>
self.drop = nn.Dropout(dr) <span class="hashtag-raw">#Zeros</span> some of the elements of the input tensor, probability of an element to be zeroed (p=0.5)<br>
self.zeropad2 = nn.ConstantPad2d((2,2,0,0),0)<br>
self.bnorm = nn.BatchNorm2d(256)<br>
self.conv2 = nn.Conv2d(256,80,(2,3)) <span class="hashtag-raw">#Second</span> convolutional Layer<br>
self.bnorm2 = nn.BatchNorm2d(80)<br>
self.Dense = nn.Linear(10560,256) # at the point will get flatten, the number of neurons required will be 10560<br>
self.bnorm3 = nn.BatchNorm1d(256)<br>
self.Dense2 = nn.Linear(256,num_classes)</p>
<pre><code>def forward(self, x):

    x = self.pad(x)
    x = F.relu(self.conv1(x))
    x = self.drop(x)
    x = self.pad(x)
    x = self.bnorm(x)
    x = F.relu(self.conv2(x))
    x = self.bnorm2(x)
    x = torch.flatten(x,1)
    x = F.relu(self.Dense(x))
    x = self.bnorm3(x)
    x = self.Dense2(x)

    return x
</code></pre>
<p>conv = ConvNet().to(device)</p>
<p><span class="hashtag-raw">#Loss</span> and optimizer<br>
criterion = nn.CrossEntropyLoss()<br>
optimizer = torch.optim.Adam(conv.parameters(), lr=learning_rate)</p>
<p><span class="hashtag-raw">#Train</span> the Model</p>
<p>total_step = len(train_dl)<br>
conv.train()<br>
for epoch in range(num_epochs):<br>
for i, (data, labels) in enumerate(train_dl):<br>
# Get data to cuda if possible<br>
data = data.reshape([data.size(dim=0), 1]+input_shape,[data.size(dim=0), 1]+input_shape)<br>
data = data.to(device)<br>
labels = labels.to(device)</p>
<pre><code>    #forward
    output = conv(data)
    loss = criterion(output, torch.max(labels, 1)[1])

    #backward
    optimizer.zero_grad()
    loss.backward()
    optimizer.step()

    if (i + 1) % 100 == 0:
        print('Epoch [{}/{}], Step [{}/{}], Loss: {:.4f}'
             .format(epoch + 1, num_epochs, i + 1, total_step, loss.item()))
</code></pre>
<p>print(‘Finished Training’)</p>
<h1><a name="test-the-model-1" class="anchor" href="#test-the-model-1"></a>Test the model</h1>
<p>with torch.no_grad():<br>
conv.eval()<br>
correct = 0<br>
total = 0<br>
for data, labels in test_dl:<br>
data = data.reshape([data.size(dim=0), 1]+input_shape,[data.size(dim=0), 1]+input_shape)<br>
data = data.to(device)<br>
labels = labels.to(device)</p>
<pre><code>    #forward
    output = conv(data)

    _, predicted = torch.max(F.softmax(output,dim=1), 1)
    total += labels.size(0)
    correct += (predicted == labels).sum().item()

print('Test Accuracy of the model on the test signals: {} %'.format(100 * correct / total))
</code></pre>

username: blueeagle
cooked: <p>As the error message shows, the error is caused by this line of code</p>
<pre><code class="lang-auto">correct += (predicted == labels).sum().item()
</code></pre>
<p>i.e. it is not (directly) caused by your model. As you are not calculating the number of correctly classified samples during training, the error only appears during testing.<br>
In this line you are comparing the tensor <code>predicted</code> with <code>labels</code> and it seems that they have different shapes.<br>
As far as I see, your predictions are of dimensions <code>[256, 11]</code>, how about your labels?<br>
Can you just print the dimensions of your labels (and predictions) and post them here?</p>
<p>Suppose your labels are not one-hot encoded, i.e. of shape <code>[256]</code>, then</p>
<pre><code class="lang-auto">correct += torch.sum(labels==torch.argmax(predictions, dim=1)).item()
</code></pre>
<p>should do the trick for you</p>

username: PyAP
cooked: <p>Thanks for replying to my post <a class="mention" href="/u/blueeagle">@blueeagle</a><br>
Yes I’ve just done what you have said and the shapes are different.<br>
dimensions of labels [256,11]<br>
dimensions of predictions [256]</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/9/9/99bf8a33231ca5202255ded7dce6c14a4b405e77.png" data-download-href="https://discuss.pytorch.org/uploads/default/99bf8a33231ca5202255ded7dce6c14a4b405e77" title="image"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/9/9/99bf8a33231ca5202255ded7dce6c14a4b405e77_2_690x206.png" alt="image" data-base62-sha1="lW7qmsTg6hojo8WtqTzhoosKQwn" width="690" height="206" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/9/9/99bf8a33231ca5202255ded7dce6c14a4b405e77_2_690x206.png, https://discuss.pytorch.org/uploads/default/original/3X/9/9/99bf8a33231ca5202255ded7dce6c14a4b405e77.png 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/9/9/99bf8a33231ca5202255ded7dce6c14a4b405e77.png 2x" data-dominant-color="312F30"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">874×262 26 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

username: blueeagle
cooked: <p>Ah now I see. Based on the architecture of your network, especially the last layer being <code>Linear(256,_num_classes)</code> I assumed the predictions to be of shape <code>[Batchsize, num_classes]</code> (in your case <code>[256, 11]</code>).<br>
However, you are doing <code>_, predicted = torch.max(F.softmax(output,dim=1), 1)</code>, which leads to your predictions being class indices of shape <code>[256]</code>.<br>
The problem consists in your predictions being class indices and your labels being one-hot encoded.<br>
Therefore the following should work for you:</p>
<pre><code class="lang-auto">correct += torch.sum(torch.argmax(labels, dim=1)==predictions).item()
</code></pre>

username: PyAP
cooked: <p>Ahh thank you so much <a class="mention" href="/u/blueeagle">@blueeagle</a> , that’s solved it. Cheers for explaining it so clearly.</p>
<p><img src="https://discuss.pytorch.org/uploads/default/original/3X/1/1/119471c9202b2969ad2e6759e952b1f7106062cc.png" alt="image" data-base62-sha1="2vw9MEFuCoi5BTixvPcGuGDcJ7m" width="579" height="168"></p>