How should I compute the accuracy for a multilable dataset?

isalirezag: <p>I am wondering how should I get the accuracy for a multi-lable classification task?<br>
Can someone please provide a toy example? <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=9" title=":slight_smile:" class="emoji" alt=":slight_smile:"></p>

acobobby: <p>For multiclass classification you should have an output tensor of size <code>(batch, num_classes)</code> while the target label tensor should be (a LongTensor) of size <code>(batch)</code>, where <code>batch</code> is the number of samples.</p>
<p>To compute accuracy you should first compute a softmax in order to have probabilities of each class for each sample, i.e.:</p>
<pre><code class="lang-auto">probs = torch.softmax(out, dim=1)
</code></pre>
<p>Then you should select the most probable class for each sample, i.e.:</p>
<pre><code class="lang-auto">winners = probs.argmax(dim=1)
</code></pre>
<p>Now you can compare target with winners:</p>
<pre><code class="lang-auto">corrects = (winners == target)
</code></pre>
<p>And take the average over all samples:</p>
<pre><code class="lang-auto">accuracy = corrects.sum().float() / float( target.size(0) )
</code></pre>
<p>If you want to do this all in one you can do it like:</p>
<pre><code class="lang-auto">accuracy = (torch.softmax(out, dim=1).argmax(dim=1) == target).sum().float() / float( target.size(0) )
</code></pre>

isalirezag: <p>Lets for the simplicity we say we have one batch in each time. and you have 20 classes.</p>
<p>so the output is gonna be in shape of <code>torch.Size([1, 20])</code>, am I correct?</p>
<aside class="quote no-group" data-username="acobobby" data-post="2" data-topic="43511">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/a/f0a364/48.png" class="avatar"> acobobby:</div>
<blockquote>
<p>To compute accuracy you should first compute a softmax in order to have probabilities of each class for each sample, i.e.:</p>
<pre><code class="lang-auto">probs = torch.softmax(out, dim=1)
</code></pre>
<p>Then you should select the most probable class for each sample, i.e.:</p>
<pre><code class="lang-auto">winners = probs.argmax(dim=1)
</code></pre>
</blockquote>
</aside>
<p>But in multi lable classification you might have multi class in one time, when you do <code>winners = probs.argmax(dim=1)</code> you are considering just one class that I dont think is correct.</p>

acobobby: <p>Sorry, I answered a completely different question. My answer is for multiclass classification, while you asked for multilabel classification. My mistake!</p>
<p>A brute-force way to approach your original problem, multilabel classification, is to transform it into multiclass classification by creating all possible combination of the labels and consider each combination a single class.<br>
In this way you can use the method I explained above, but you have to consider the powerset of the set of all labels, which can be huge.</p>

vs74: <p>Although your answer if for Multiclass class classification different from what was asked in the question, it really helped a lot as I was searching how to compute the accuracy for multiclass classification.</p>

Rohan_Sharma: <p>Why do the softmax before the argmax instead of doing argmax directly on the logits? The answer shouldn’t change…</p>

Bibarys_Mussagaliyev: <p>yes, you are right. The results should be the same.</p>