ValueError while running multi-class predictions

username: sharky
cooked: <p>I am doing multi-class predicition with softmax and CrossEntropyLoss. My output is [0.4559, 0.2230, 0.3211] and label is [1]. I am very new to pytorch and have no idea how to correct this error I am getting: ValueError: If <code>preds</code> and <code>target</code> are of shape (N, …) and <code>preds</code> are floats, <code>target</code> should be binary. Any suggestions would be really appreciated.</p>

username: safakkbilici
cooked: <p>The task that you are working on seems a little bit ambiguous for me. So I would be more helpful if you give more information, like, what is your batch size? what is your exact dimensions of target and prediction tensor shapes?</p>
<p>Assuming tensors as you say: <code>preds: [0.45, 0.22, 0.32], target: 1</code>, the corresponding loss calculation should be</p>
<pre><code class="lang-python">import torch

criterion = torch.nn.CrossEntropyLoss()
criterion(torch.tensor([0.45, 0.22, 0.32]).unsqueeze(1).unsqueeze(0), torch.tensor([1]).unsqueeze(0).long())
#tensor(1.2131)
</code></pre>
<p>where .unsqueeze(0) for batch size of 1 and .unsqueeze(1) is for sequence length.</p>
<pre><code class="lang-python">print(torch.tensor([0.45, 0.22, 0.32]).unsqueeze(1).unsqueeze(0).shape)
# torch.Size([1, 3, 1])

print(torch.tensor([1]).unsqueeze(0).long().shape)
# torch.Size([1, 1])
</code></pre>
<p>I hope this would be helpful.</p>

username: sharky
cooked: <p>Hi Safak,</p>
<p>Many thanks for your response. My batch size is 12, dim=1. An example tensor shape for my output is torch.Size([10, 3]) and for my labels it is torch.Size([10, 1]). I applied labels.flatten() - it did not show the error and calculated the loss. See below the code:<br>
<img src="https://discuss.pytorch.org/uploads/default/original/3X/0/e/0e15b2098181d0e72c362ce96d7b503450f59034.png" alt="Screenshot 2021-07-15 at 15-56-04 transformer-Copy2 - Jupyter Notebook" data-base62-sha1="20BalwuqBrPduZM2Z7qD8aVEPIM" width="669" height="333"></p>
<p>Predictions look ok to me but need to convert it into the label (get the max index). Do you know the best way of doing that? Thanks</p>

username: safakkbilici
cooked: <p>If your batch size is 12, sequence length is 10 and number of your classes is 3; your output before passing it loss fn should have dimension <code>[batch_size, n_classes, seq_len]</code> and your target should have dimension <code>[batch_size, seq_len]</code>. An example: <code>[12, 3, 10]</code> and <code>[12, 10]</code>. It is okey to pass this to <code>nn.CrossEntropyLoss</code> without any flattening.</p>
<p>If you have batch size at dimension 1; y_pred: <code>[10,12,3]</code> and target: <code>[10,12,1]</code>, you can pass it like <code>criterion(y_pred.permute(1,2,0), target.permute(1,2,0).squeeze(1))</code></p>
<pre><code class="lang-python">print(y_pred.permute(1,2,0).shape)
# torch.Size([12, 3, 10])

print(target.permute(1,2,0).squeeze(1))
# torch.Size([12, 10])
</code></pre>
<p>After organizing your shapes, you can get your highest probability classes with</p>
<pre><code class="lang-python">print(y_pred.permute(1,2,0).shape)
# torch.Size([12, 3, 10])

argmax_preds = torch.argmax(y_pred.permute(1,2,0), dim=1)
</code></pre>
<p>Then calculate your accuracy:</p>
<pre><code class="lang-auto">accuracy = (argmax_preds == target).cpu().numpy().mean() * 100
</code></pre>