Multi-target classification problem:**Error** multi-target not supported for CrossEntropyLoss

username: Animesh_Kumar_Paul
cooked: <p>I have a dataset which has multiple targets attributes.</p>
<p>Sample targets <strong>for 12 data points</strong> (<strong>4 attributes as target outputs: not one-hot encoded</strong>):<br>
<img src="https://discuss.pytorch.org/uploads/default/original/2X/9/9822ad91ef5648a11c47601b2b50b6f45005c148.png" alt="image" data-base62-sha1="lHQS729WppQ3KJuApAjdl3vMnpe" width="434" height="206"><br>
<img src="https://discuss.pytorch.org/uploads/default/original/2X/b/b749ddb900d0175f1a84e9cb6a59fa0a98244283.png" alt="image" data-base62-sha1="q9rDkYQK99OhJI8gjpbrKzzeVOz" width="428" height="183"></p>
<pre><code> class Classifier(nn.Module):
    def __init__(self,input_nodes):
        super(Classifier, self).__init__()
        self.fc3 = nn.Linear(75000, 500)
        self.fc4 = nn.Linear(500, 200)
        self.fc5 = nn.Linear(200, 100)
        self.fc_out = nn.Linear(100, 4)
        self.dropout = nn.Dropout(0.5)
   
    def forward(self, x):
        x = F.relu(self.fc3(x))
        x = self.dropout(x)
        x = F.relu(self.fc4(x))
        x = self.dropout(x)
        x = F.relu(self.fc5(x))
        x = self.dropout(x)
        x = self.fc_out(x)
        return x

criterion = nn.CrossEntropyLoss()

 
for epoch in range(n_epochs):
	running_loss = 0
	i = 0
	model.train()
	for data, label in trainloader:
		 y_hat  = model(data) # Shape of y_hat = 6*4 (as 4 targets)
		 loss = criterion(y_hat, label) # Shape of label = 6*4
		 #--&gt; Give error: RuntimeError: multi-target not supported at ..\aten\src\THNN/generic/ClassNLLCriterion.c:20
		
		 optimizer.zero_grad()
		 loss.backward()
		 optimizer.step()
		 train_loss.append(loss.item())
		 train_losses.append(loss.item())
		 running_loss += loss.item()
</code></pre>
<p><strong>Error: <em>RuntimeError:</em> multi-target not supported at …\aten\src\THNN/generic/ClassNLLCriterion.c:20</strong></p>
<p>How can I train and evaluate the multi-target classification problem?</p>
<p>Please help.</p>

username: KFrank
cooked: <p>Hello Animesh!</p>
<aside class="quote no-group quote-modified" data-username="Animesh_Kumar_Paul" data-post="1" data-topic="50152" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/animesh_kumar_paul/48/11990_2.png" class="avatar"> Animesh_Kumar_Paul:</div>
<blockquote>
<p>I have a dataset which has multiple targets attributes.</p>
<p>Sample targets fo<strong>r 6 data points</strong> (<strong>4 attributes as target outputs: not one-hot encoded</strong>):<br>
<img src="https://discuss.pytorch.org/uploads/default/original/2X/9/9822ad91ef5648a11c47601b2b50b6f45005c148.png" alt="image" data-base62-sha1="lHQS729WppQ3KJuApAjdl3vMnpe" width="434" height="206"></p>
</blockquote>
</aside>
<p>Just to confirm, are your really working with a “multi-target”<br>
(multi-class) classification problem?  I ask because the sample<br>
targets you show in your .png image have either 0 (in just row 2)<br>
or 1 (in the rest of the rows) of the labels set.</p>
<p>That is, even though you didn’t show such a row, could you have<br>
a row (say, row 17) for which all four fields are set to 1?</p>
<p>If not, you have a multi-class (but not multi-label) classification<br>
problem, and you should recast it as such, and (most likely)<br>
use <code>nn.CrossEntropyLoss</code> as your loss function.</p>
<aside class="quote no-group">
<blockquote>
<p><strong>Error: <em>RuntimeError:</em> multi-target not supported at …\aten\src\THNN/generic/ClassNLLCriterion.c:20</strong></p>
<p>How can I train and evaluate the multi-target classification problem?</p>
</blockquote>
</aside>
<p>The general consensus is that <a href="https://pytorch.org/docs/stable/nn.html#multilabelsoftmarginloss" rel="noopener nofollow ugc">MultiLabelSoftMarginLoss</a><br>
is the loss function to start with for a multi-label classification<br>
problem.</p>
<p>Best.</p>
<p>K. Frank</p>

username: Animesh_Kumar_Paul
cooked: <p>Hi <a class="mention" href="/u/kfrank">@KFrank</a>,</p>
<p>Thanks for your help.</p>
<blockquote>
<p>That is, even though you didn’t show such a row, could you have<br>
a row (say, row 17) for which all four fields are set to 1?</p>
</blockquote>
<p>Yes,  I have some rows with all 1s and some rows with more than ones there.</p>
<p>The targets can be like this:<br>
<img src="https://discuss.pytorch.org/uploads/default/original/2X/b/b749ddb900d0175f1a84e9cb6a59fa0a98244283.png" alt="image" data-base62-sha1="q9rDkYQK99OhJI8gjpbrKzzeVOz" width="428" height="183"></p>

username: Prerna_Dhareshwar
cooked: <p>Hi Animesh,</p>
<p>I think you’re looking for BCEloss which can be found <a href="https://pytorch.org/docs/stable/nn.html#bceloss" rel="nofollow noopener">here.</a></p>

username: KFrank
cooked: <p>Hi Prerna and Animesh!</p>
<aside class="quote no-group" data-username="Prerna_Dhareshwar" data-post="4" data-topic="50152" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/prerna_dhareshwar/48/14256_2.png" class="avatar"> Prerna_Dhareshwar:</div>
<blockquote>
<p>I think you’re looking for BCEloss which can be found <a href="https://pytorch.org/docs/stable/nn.html#bceloss" rel="noopener nofollow ugc">here.</a></p>
</blockquote>
</aside>
<p>You don’t want – in the typical case – <code>BCELoss</code> for classification<br>
problems.  This is because <code>BCELoss</code> requires the predictions fed<br>
into it to be numbers in (0, 1) (probability-like numbers).</p>
<p>For  example, Animesh’s output layer is a <code>Linear</code></p>
<pre data-code-wrap="python"><code class="lang-python">        self.fc_out = nn.Linear(100, 4)
</code></pre>
<p>that will, in general, output numbers ranging from -infinity to<br>
+infinity.  You also don’t want to pass these outputs through<br>
a <code>Sigmoid</code> (or <code>Softmax</code>) layer to map them to (0, 1) because<br>
of the risk of overflow.</p>
<p>You want, instead, a loss function that takes <em>logit</em>-like<br>
predictions (that run from -infinity to +infinity), such as<br>
<code>BCEWithLogitsLoss</code>.</p>
<p><code>MultiLabelSoftMarginLoss</code> and <code>BCEWithLogitsLoss</code><br>
are essentially the same function, as Peter explains here:</p>
<aside class="quote quote-modified" data-post="2" data-topic="14944">
  <div class="title">
    <div class="quote-controls"></div>
    <img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/48/1823_2.png" class="avatar">
    <a href="https://discuss.pytorch.org/t/what-is-the-difference-between-bcewithlogitsloss-and-multilabelsoftmarginloss/14944/2">What is the difference between BCEWithLogitsLoss and MultiLabelSoftMarginLoss</a> 
  </div>
  <blockquote>
    You are right. Both loss functions seem to return the same loss values: 
x = Variable(torch.randn(10, 3))
y = Variable(torch.FloatTensor(10, 3).random_(2))

# double the loss for class 1
class_weight = torch.FloatTensor([1.0, 2.0, 1.0])
# double the loss for last sample
element_weight = torch.FloatTensor([1.0]*9 + [2.0]).view(-1, 1)
element_weight = element_weight.repeat(1, 3)

bce_criterion = nn.BCEWithLogitsLoss(weight=None, reduce=False)
multi_criterion = nn.MultiLabelSoftMarginLoss(weight=No…
  </blockquote>
</aside>

<p>Good luck.</p>
<p>K. Frank</p>