BCELoss() computation for images

username: krish13
cooked: <p>This topic seems to be widely discussed but I got a question. The document in the Pytorch indicate following formula:</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/7/4/74341c1d82936bc23fdddc422c0700ef300f2ac4.png" data-download-href="https://discuss.pytorch.org/uploads/default/74341c1d82936bc23fdddc422c0700ef300f2ac4" title="image"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/7/4/74341c1d82936bc23fdddc422c0700ef300f2ac4_2_516x150.png" alt="image" data-base62-sha1="gzZ1WxspmT3OLeshZwtSFXoyLnC" width="516" height="150" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/7/4/74341c1d82936bc23fdddc422c0700ef300f2ac4_2_516x150.png, https://discuss.pytorch.org/uploads/default/optimized/3X/7/4/74341c1d82936bc23fdddc422c0700ef300f2ac4_2_774x225.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/7/4/74341c1d82936bc23fdddc422c0700ef300f2ac4_2_1032x300.png 2x" data-dominant-color="F4F4F4"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">1374×402 47.4 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div><br>
My question is related to N.</p>
<p>Let’s say my target size is [1* 1 * 10 * 10] - B,C,H,W<br>
The output from my network is also [1* 1 * 10 * 10]  - B,C,H,W</p>
<p>In this case N = 1. Thus, according to the formula listed above, we should have following:<br>
l(x,y) = L = { l1 } = -[ y1 log( x1 ) + (1 - y1) log(1 - x1)]<br>
Question:<br>
This problem have a target image, and output image instead of a particular scalar for target and output values. In such case how is l1 computed? Will it be computed pixel by pixel (10*10) and mean is taken to get l1?</p>
<p>The definition says mean can be used for batch N. So if I have  {l1, l2,…,lN}, it makes sense to compute L using mean formula, but nothing is said above how is l1 computed in the case of images?</p>

username: Unity05
cooked: <p>Hi krish,</p>
<p>quite interesting, I haven’t found anything specific about it either.<br>
I assume, you want to do binary image segmentation. Then, it should be nothing different to having a bunch of single binary classification cases and therefore it should be handled as ‘batch size’. At least that would make sense to me. Please correct me, if I’m wrong.</p>
<p>Regards,<br>
Unity05</p>

username: KFrank
cooked: <p>Hi Krish!</p>
<aside class="quote no-group" data-username="krish13" data-post="1" data-topic="120879" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/krish13/48/38128_2.png" class="avatar"> krish13:</div>
<blockquote>
<p>In such case how is l1 computed? Will it be computed pixel by pixel (10*10) and mean is taken to get l1?</p>
</blockquote>
</aside>
<p>As an aside, you should use <code>BCEWithLogitsLoss</code> rather than <code>BCELoss</code><br>
for reasons of numerical stability.</p>
<p>To answer your question, the documentation is a little imperfect on this<br>
point.  <code>BCELoss</code> simply computes the loss for each pair of matching<br>
elements of the <code>output</code> and <code>target</code> tensors, and then computes the<br>
mean (if that is what you have chosen for the <code>reduction</code>).  It doesn’t<br>
care how the dimensions are divvied up into <code>B, C, H, W</code>.</p>
<p>This example shows what is going on:</p>
<pre><code class="lang-plaintext">&gt;&gt;&gt; import torch
&gt;&gt;&gt; torch.__version__
'1.7.1'
&gt;&gt;&gt; _ = torch.manual_seed (2021)
&gt;&gt;&gt; output = torch.rand (1, 1, 10, 10)
&gt;&gt;&gt; target = torch.rand (1, 1, 10, 10)
&gt;&gt;&gt; torch.nn.BCELoss() (output, target)
tensor(1.0348)
&gt;&gt;&gt; torch.nn.BCELoss() (output.flatten(), target.flatten())
tensor(1.0348)
&gt;&gt;&gt; torch.nn.BCELoss (reduction = 'none') (output, target).shape
torch.Size([1, 1, 10, 10])
&gt;&gt;&gt; torch.nn.BCELoss (reduction = 'none') (output.flatten(), target.flatten()).shape
torch.Size([100])
</code></pre>
<p>Best.</p>
<p>K. Frank</p>

username: krish13
cooked: <p>Thank you Unity05, I think you are correct. The way I understand this now is you compute the loss value for all channel or batches whatever it be (pixel by pixel) and take a mean of it if reduction is “mean”. Example if I have [1 * 1* 10* 10] for output and target. It computes loss pixel by pixel per channels. And at the end take the mean if the reduction is mean.</p>

username: krish13
cooked: <p>Thank KFrank,</p>
<p>So, given target of [1 * 1 * 10 * 10] and output from the network as [ 1 * 1 * 10 * 10]. I understood the loss would be given by (assuming the reduction is mean):</p>
<p><img src="https://discuss.pytorch.org/uploads/default/original/3X/d/b/db2851541b2cccb19517c164d001d32eeffbad41.png" alt="image" data-base62-sha1="vgKWFv0r7aIWtTox01IIs6gC5pf" width="688" height="92"></p>
<p>Where M and N are the height (value of 10) and width (value of 10) in our case. Am I correct in this regard?</p>

username: KFrank
cooked: <p>Hi Krish!</p>
<aside class="quote no-group" data-username="krish13" data-post="5" data-topic="120879" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/krish13/48/38128_2.png" class="avatar"> krish13:</div>
<blockquote>
<p>Am I correct in this regard?</p>
</blockquote>
</aside>
<p>No.  To calculate the mean you divide your double sum by <code>M * N</code> (in this<br>
case <code>100</code>) (not by <code>M + N</code>, which would be <code>20</code>).</p>
<p>Best.</p>
<p>K. Frank</p>

username: krish13
cooked: <p>Oh yeah I agree. That was a silly mistake on my part. Thanks make total sense.</p>