Defining loss function inside train loop

username: hwaseem04
cooked: <p>I am trying to implement balanced binary focal loss(Paper: <a href="https://arxiv.org/abs/1708.02002v2" class="inline-onebox" rel="noopener nofollow ugc">[1708.02002v2] Focal Loss for Dense Object Detection</a>) , where I need to give different weight (ratio of negative to negative class) based on each input tensor.</p>
<p>So if I have batch size as 6, I would create a weight tensor of size 6 that is passed as below,<br>
<code>loss_fn = nn.BCEWithLogitsLoss(pos_weight = weight_tensor, reduction=None)</code></p>
<p>Wont it affect backpropagation if I define such <code>loss_fn</code> in each iteration in order to give weight_tensor dynamically based on inputs of each batch?</p>
<p>What I mean is, If my forward function is like below, wont it be a problem for backpropagation (i.e affecting computational graph of pytorch)?</p>
<pre><code class="lang-auto">def forward(self, img, gt_img, count_pixels):
        # ------ Some code is removed for better context to this question------

        # --------------BCE loss-----------------
        # AlphaC component in focal loss
        ratio = []
        for counts in count_pixels:
            ratio.append(counts[0]/counts[1])
        ratio = np.array(ratio)
        ratio = torch.from_numpy(ratio).to(torch.float32)

        bce_criterion = nn.BCEWithLogitsLoss(pos_weight=ratio, reduction=None)

        # --------------Focal Loss---------------
        # Yet to implement

        loss = bce_criterion(final_pred, gt_patches)

        return loss, patches, pred_pixel_values

</code></pre>
<p><a class="mention" href="/u/ptrblck">@ptrblck</a> please advise</p>

username: KFrank
cooked: <p>Hi Muhammad!</p>
<aside class="quote no-group quote-modified" data-username="hwaseem04" data-post="1" data-topic="181926" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/hwaseem04/48/55728_2.png" class="avatar"> hwaseem04:</div>
<blockquote>
<pre><code class="lang-auto">        for counts in count_pixels:
            ratio.append(counts[0]/counts[1])
        ratio = np.array(ratio)
        ratio = torch.from_numpy(ratio).to(torch.float32)

        bce_criterion = nn.BCEWithLogitsLoss(pos_weight=ratio, reduction=None)
...
        loss = bce_criterion(final_pred, gt_patches)
</code></pre>
</blockquote>
</aside>
<p>You will not be able to backpropagate through the computation of <code>ratio</code> (for<br>
three reasons) – however, I expect that you don’t want to.  Assuming that<br>
<code>final_pred</code> is the output of some properly differentiable model, you will be<br>
able to backpropagate through <code>bce_criterion</code> and <code>final_pred</code>.</p>
<p>As an aside, for stylistic reasons, I would probably use the functional version<br>
of <code>BCEWithLogitsLoss</code>, <a href="https://pytorch.org/docs/2.0/generated/torch.nn.functional.binary_cross_entropy_with_logits.html" rel="noopener nofollow ugc">binary_cross_entropy_with_logits(),</a> rather than<br>
repeatedly instantiating <code>BCEWithLogitsLoss</code>, but, either way, you’ll be doing<br>
the same computation.</p>
<p>Best.</p>
<p>K. Frank</p>