Mlp classification using pytorch

tejal_mehta: <p>Hello</p>
<p>I am using pytorch for build MLP neural network for multiclass classification.</p>
<p>epochs = 100 # to have 100 training epochs<br>
for epoch in range(epochs):<br>
batch_loss = 0</p>
<pre><code>for data in train_loader:
    input, output = data
    
    #it is required to zero out the gradients so that we update parameter update correctly. Otherwise the gradient may not point towards the indetended direction than towards the minimum 
    optimizer.zero_grad() 

    predicted = network.forward(input) # a batch of 10 will be applied. neural network will apply the sum function and the sigmoid activation function and it will do prediction on that basis
    print(predicted.shape)
    print("output",output.shape)
   
    #_,pred_label = torch.max(predicted, dim = 1)
    #correct = (pred_label == output).float()

    #loss = criterion(y_pred.squeeze(), y_train)
    loss = loss(predicted.squeeze(), output) #predicted.shape (10, 5) , output.shape - 10
    loss.backward() #to apply the back propogation algorithm
    optimizer.step() #to update the weights

    batch_loss += loss.item() #error which is calculated for each of the batch

#to print the loss for each epoch 
print('Epoch: ' + str(epoch + 1) + ' has a average loss of : ' + str(batch_loss / len(train_loader)))
</code></pre>
<p>on running the above code, I am getting following error -</p>
<p>RuntimeError                              Traceback (most recent call last)<br>
 in <br>
17<br>
18         <span class="hashtag">#loss</span> = criterion(y_pred.squeeze(), y_train)<br>
—&gt; 19         loss = loss(predicted.squeeze(), output) <span class="hashtag">#to</span> calculate the mean squared error defined in above cell.<br>
20         loss.backward() <span class="hashtag">#to</span> apply the back propogation algorithm<br>
21         optimizer.step() <span class="hashtag">#to</span> update the weights</p>
<p>C:\ProgramData\Anaconda3\lib\site-packages\torch\nn\modules\module.py in _call_impl(self, *input, **kwargs)<br>
887             result = self._slow_forward(*input, **kwargs)<br>
888         else:<br>
 → 889             result = self.forward(*input, **kwargs)<br>
890         for hook in itertools.chain(<br>
891                 _global_forward_hooks.values(),</p>
<p>C:\ProgramData\Anaconda3\lib\site-packages\torch\nn\modules\loss.py in forward(self, input, target)<br>
526<br>
527     def forward(self, input: Tensor, target: Tensor) → Tensor:<br>
 → 528         return F.mse_loss(input, target, reduction=self.reduction)<br>
529<br>
530</p>
<p>C:\ProgramData\Anaconda3\lib\site-packages\torch\nn\functional.py in mse_loss(input, target, size_average, reduce, reduction)<br>
2923         reduction = _Reduction.legacy_get_string(size_average, reduce)<br>
2924<br>
 → 2925     expanded_input, expanded_target = torch.broadcast_tensors(input, target)<br>
2926     return torch._C._nn.mse_loss(expanded_input, expanded_target, _Reduction.get_enum(reduction))<br>
2927</p>
<p>C:\ProgramData\Anaconda3\lib\site-packages\torch\functional.py in broadcast_tensors(*tensors)<br>
72     if has_torch_function(tensors):<br>
73         return handle_torch_function(broadcast_tensors, tensors, *tensors)<br>
—&gt; 74     return _VF.broadcast_tensors(tensors)  # type: ignore<br>
75<br>
76</p>
<p>RuntimeError: The size of tensor a (5) must match the size of tensor b (10) at non-singleton dimension 1</p>
<p>How can I rectify it?</p>
<p>Thanks</p>

ptrblck: <p>I guess the <code>squeeze</code> operation is not needed and will create an error, e.g. if the last batch contains only one sample:</p>
<pre><code class="lang-python">loss = loss(predicted.squeeze(), output)
</code></pre>
<p>Based on the comment <code>predicted</code> should have a shape of <code>[10, 5]</code>. If that’s the shape before applying the <code>squeeze</code> operation, just remove it and pass the tensors directly to the criterion:</p>
<pre><code class="lang-python">loss = criterion(predicted, output)
</code></pre>
<p>Also, rename the <code>loss</code> function to <code>criterion</code> or any other name, as you are overriding it with the <code>loss</code> tensor.</p>
<p>Unrelated to this issue, but call the model directly to execute the forward pass as:</p>
<pre><code class="lang-python">predicted = network(input)
</code></pre>
<p>instead of the <code>.forward</code> method.</p>
<p>PS: you can post code snippets by wrapping them into three backticks ```, which makes debugging easier. <img src="https://discuss.pytorch.org/images/emoji/apple/wink.png?v=9" title=":wink:" class="emoji" alt=":wink:"></p>

tejal_mehta: <p>Hey<br>
I tried all your suggestions. But still, I am getting the same error <img src="https://discuss.pytorch.org/images/emoji/apple/frowning.png?v=9" title=":frowning:" class="emoji" alt=":frowning:"></p>

ptrblck: <p>Could you post the model definition as well as the input shapes, so that we could reproduce this issue, please?</p>