How to use CrossEntroyLoss to multiple Bi-classification task

Feywell: <p>I use a simple Classification CNN to realize  multiple Bi-classification task. My output shape of model like<br>
[10,2]. How can I use loss?</p>

phan_phan: <p>Here, is 10 the number of bi-classifications ?<br>
If it is :<br>
Since it is bi-classification, you could instead get an output shape of [10, 1] and use <a href="https://pytorch.org/docs/stable/nn.html#bcewithlogitsloss" rel="nofollow noopener"><code>nn.BCEWithLogitsLoss</code></a>. It combines Sigmoid + BinaryCrossEntropy (so do not use a sigmoid activation just before that).<br>
For the multiple classification aspect I suppose your output shape is [batch_size, 10, 1], and your label shape is also [batch_size, 10, 1] filled with ones and zeros. The loss function by default will average over the 10 different tasks, which is probably what you want.</p>

Feywell: <p>Thank you! Is it better than mse?</p>

phan_phan: <p>Yes I’ve always heard that for classification it was better than MSE, but I have nothing to back it up.</p>

ptrblck: <p>If I’m not mistaken, the loss for a classification task using <code>nn.MSELoss</code> will be scaled down the closer the output gets to the target, which might slow down the training towards the end.<br>
<a class="mention" href="/u/rasbt">@rasbt</a> derives the formula <a href="https://sebastianraschka.com/faq/docs/mse-derivative.html" rel="nofollow noopener">here</a> and you can find some additional information in <a href="https://jamesmccaffrey.wordpress.com/2013/11/05/why-you-should-use-cross-entropy-error-instead-of-classification-error-or-mean-squared-error-for-neural-network-classifier-training/" rel="nofollow noopener">this blog post</a> (at the end).</p>

rasbt: <p>Both are fine. Empirically, the cross tends to result in better performance of the resulting models though. Also, the the derivative/gradient calculation plays nicely with sigmoid or softmax in the last layer, i.e., it simplifies as follows (where “a” is the sigmoid activation, y is the true label, and z is the weighted input w^T x):</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/2/c/2c54933ef2c096343a1db2aa24dd549208a5d687.png" data-download-href="https://discuss.pytorch.org/uploads/default/2c54933ef2c096343a1db2aa24dd549208a5d687" title="31%20PM"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/2/c/2c54933ef2c096343a1db2aa24dd549208a5d687_2_690x227.png" alt="31%20PM" data-base62-sha1="6kad1X7D1cVGCsZwC3ORUc1pqwn" width="690" height="227" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/2/c/2c54933ef2c096343a1db2aa24dd549208a5d687_2_690x227.png, https://discuss.pytorch.org/uploads/default/optimized/3X/2/c/2c54933ef2c096343a1db2aa24dd549208a5d687_2_1035x340.png 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/2/c/2c54933ef2c096343a1db2aa24dd549208a5d687.png 2x" data-dominant-color="F8F9F9"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">31%20PM</span><span class="informations">1312×432 57.7 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

phan_phan: <p>Thanks for the info !</p>