Runtime Error while using Focal Loss

username: bing
cooked: <p>Hi everyone,<br>
I am doing a multi-class classification.<br>
I am trying to use focal loss instead of cross-entropy but it is throwing me an error -</p>
<pre><code>class WeightedFocalLoss(nn.Module):
    def __init__(self, alpha=.25, gamma=2):
        super(WeightedFocalLoss, self).__init__()
        self.alpha = alpha
        self.gamma = gamma

    def forward(self, inputs, targets):
       CE_loss = nn.CrossEntropyLoss(inputs, targets, reduce='none')
       pt = torch.exp(-CE_loss)
       F_loss = self.alpha * (1-pt)**self.gamma * CE_loss

   return F_loss
</code></pre>
<p>In the training script, I used in the following way -</p>
<blockquote>
<p>criterion = focal_loss.WeightedFocalLoss()<br>
<span class="hashtag-raw">#In</span> training loop -<br>
output = model(input_Data)<br>
loss = criterion(output, target)</p>
</blockquote>
<p>I am getting the below error -</p>
<blockquote>
<p>RuntimeError: bool value of Tensor with more than one value is ambiguous</p>
</blockquote>

username: planet_pluto
cooked: <p>There’s some issue with your predictions and targets.</p>
<p>From the docs:<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/3/f/3f07f4caa5a5e60ea243abd2801f909773875a4e.png" data-download-href="https://discuss.pytorch.org/uploads/default/3f07f4caa5a5e60ea243abd2801f909773875a4e" title="image"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/3/f/3f07f4caa5a5e60ea243abd2801f909773875a4e_2_690x170.png" alt="image" data-base62-sha1="8ZB7RffdW9HHyoHO3ptJHJX368e" width="690" height="170" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/3/f/3f07f4caa5a5e60ea243abd2801f909773875a4e_2_690x170.png, https://discuss.pytorch.org/uploads/default/original/3X/3/f/3f07f4caa5a5e60ea243abd2801f909773875a4e.png 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/3/f/3f07f4caa5a5e60ea243abd2801f909773875a4e.png 2x" data-dominant-color="F4F4F5"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">922×228 24.9 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>Let’s assume we have 4 classes. The predictions should be of shape <code>torch.Size([1, 4])</code> and targets of shape <code>torch.Size([1])</code></p>
<p>I made a few modifications to your code and this seems to work for me.</p>
<pre><code class="lang-auto">class WeightedFocalLoss(nn.Module):
    def __init__(self, alpha=.25, gamma=2):
        super(WeightedFocalLoss, self).__init__()
        self.alpha = alpha
        self.gamma = gamma
        self.criterion = nn.CrossEntropyLoss(reduce='none')

    def forward(self, inputs, targets):
        
        CE_loss = self.criterion(inputs, targets)
        pt = torch.exp(-CE_loss)
        F_loss = self.alpha * (1-pt)**self.gamma * CE_loss

        return F_loss

criterion = WeightedFocalLoss()

outputs = torch.Tensor([[0, 0, 0, 1]])
targets = torch.Tensor([3]).long()

loss = criterion(outputs, targets)

print(loss)
#tensor(0.0512)
</code></pre>