Multi-Class Ground Truth Annotation and Classification Head for Three Elements with Three Classes

username: Chuong_Vo
cooked: <p>Hello,</p>
<p>I’m currently working on a problem that involves annotating the classification ground truth for three elements, where each element can belong to one of three classes. An example of this type of annotation is: <code>[0, 1, 2]</code>, <code>[2, 0, 0]</code>, or <code>[0, 2, 2]</code>. This form of annotation is reminiscent of semantic segmentation, as it classifies each element within the array.</p>
<p>I’m seeking guidance on two main aspects of my project:</p>
<p><strong>Classification Head:</strong> To tackle this problem, I’m unsure about the best way to design the classification head for my neural network. Specifically, I’m unsure whether I should use fully-connected layers (FC layers) or conv2d layers, given the nature of my task. Should I opt for one approach over the other, or perhaps a combination of both?</p>
<p><strong>Loss Function:</strong> I’d also like advice on selecting an appropriate loss function for this task. Given the multi-class nature of my problem, I’d like to know which loss function is best suited for training my model.</p>
<p><strong>Additional Information:</strong></p>
<ul>
<li>I’m using PyTorch for this project, and my dataset includes annotations in the format of <code>[0, 1, 2]</code> for each element.</li>
<li>The model architecture I’m using incorporates both CNN and fully-connected layers, but I’m open to recommendations on how to structure the classification head.</li>
</ul>
<p><strong>Expected Outcome:</strong> I’m looking for guidance on the best approach to designing the classification head and selecting the appropriate loss function for my problem. Any insights, recommendations, or best practices from the community would be greatly appreciated.</p>
<p>Thank you for your help and advice.</p>

username: ptrblck
cooked: <p>I don’t fully understand the use case since you describe it first as:</p>
<aside class="quote no-group" data-username="Chuong_Vo" data-post="1" data-topic="189989">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/c/b2d939/48.png" class="avatar"> Chuong_Vo:</div>
<blockquote>
<p>that involves annotating the classification ground truth for three elements, where each element can belong to one of three classes. An example of this type of annotation is: <code>[0, 1, 2]</code>, <code>[2, 0, 0]</code>, or <code>[0, 2, 2]</code>.</p>
</blockquote>
</aside>
<p>which sounds as if these <code>list</code>s contain  class indices for 3 elements while later you explain:</p>
<aside class="quote no-group" data-username="Chuong_Vo" data-post="1" data-topic="189989">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/c/b2d939/48.png" class="avatar"> Chuong_Vo:</div>
<blockquote>
<p>and my dataset includes annotations in the format of <code>[0, 1, 2]</code> for each element.</p>
</blockquote>
</aside>
<p>I assume each batch contains 3 elements each belonging to one of 3 possible classes?<br>
If so I also assume your target has the shape <code>[batch_size, 3]</code> containing values in <code>[0, 2]</code>?<br>
In this case you could use <code>nn.CrossEntropyLoss</code> and output logits in the shape <code>[batch_size, nb_classes=3, 3]</code>.</p>

username: Chuong_Vo
cooked: <p>Thank you very much for the answer. Your understanding is exactly what I have in mind.<br>
So I’m making the data shapes like this</p>
<pre><code class="lang-auto">'output' shape: (batch_size, 3, 3)
'target' shape: (batch_size, 3)
</code></pre>
<p>the ‘output’ is one hot encoder according to each element that looks like</p>
<pre><code class="lang-auto">[[0, 1, 0],
 [1, 0, 0],
 [0, 0, 1]]
</code></pre>
<p>And the groundtruth looks like</p>
<pre><code class="lang-auto">[ 1, 0, 2]
</code></pre>
<p>Using <code>nn.CrossEntropyLoss</code> the code runs without error.<br>
However, during training, the accuracy for this is quite low ~0.51 both train and val.<br>
I use the code below to get the accuracy</p>
<pre><code class="lang-auto">predicted_labels = torch.argmax(output, dim=2)
correct_predictions = (predicted_labels == gt)
position_accuracy = correct_predictions.float().mean(dim=0)
overall_accuracy = position_accuracy.mean()
</code></pre>
<p>Is there any problem with the inference code?</p>

username: ptrblck
cooked: <p>The output looks wrong since it should contain logits while yours is One-shot encoded for some reason.<br>
Also, the calculation of <code>predicted_labels</code> is wrong as the <code>argmax</code> should be applied on <code>dim1</code>, which is the class dimension.</p>

username: Chuong_Vo
cooked: <p>Could you provide me with an example of how the data should be and how to work with them?</p>

username: ptrblck
cooked: <p>I’ve mentioned the expected output shapes including their meaning in my first post. Only the last dimension with a size of 3 is “undefined” as it seems to be specific to your use case.</p>