Penalize FN in binary cross enthropy

Anita: <p>Hi all,</p>
<p>I am trying to implement a weighted binary cross entropy loss function with dice loss, basically:</p>
<p>total_loss = Weighted_bce_loss + dice_loss</p>
<p>I am using the code below: (SR - segmentation result, GT - ground truth)</p>
<p>def bce_dice_loss(GT, SR):</p>
<pre><code>SR.flatten()
GT.flatten()
criterion = torch.nn.BCELoss()

loss = criterion(GT, SR) + dice_coef_loss(GT, SR)
return loss
</code></pre>
<p>def dice_coef_loss(GT, SR):<br>
SR.flatten()<br>
GT.flatten()<br>
loss = 1 - dice_coeff(GT, SR)<br>
return loss</p>
<p>def dice_coeff(GT, SR):</p>
<pre><code>smooth = 1.
GT.flatten()
SR.flatten()
production = GT * SR
intersection = production.sum()

return (2. * intersection + smooth) / (GT.sum() + SR.sum() + smooth)
</code></pre>
<p>I am doing an medical image segmentation and I deal with a highly imbalanced dataset. In my error loss function (in the BCE part) I want to penalize False Negatives in order to get a higher Recall value.</p>
<p>Does anyone has an idea how to proceed with this?</p>
<p>Thank you in advance <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=9" title=":slight_smile:" class="emoji" alt=":slight_smile:"></p>

ptrblck: <p>You could use <a href="https://pytorch.org/docs/stable/nn.html#torch.nn.BCEWithLogitsLoss" rel="nofollow noopener">nn.BCEWithLogitsLoss</a> instead of <code>nn.BCELoss</code> (remove the sigmoid therefore) and set the <code>pos_weight &gt; 1</code> to increase the recall.</p>

Anita: <p>This is my train loop:</p>
<p>def per_epoch_train(self,epoch):</p>
<pre><code>avg_meter = defaultdict(float)
pbar = tqdm(enumerate(self.train_loader), total=len(self.train_loader), desc="Epoch {}".format(epoch), ncols=0)
self.attUnet.train(True)
meter = {}

acc_detection = 0.
recall_detection = 0.
precision_detection = 0.
DC_detection = 0.
    
length = 0
avg_loss = 0

for i, data in pbar:
  images = data[0]
  GT = data[1]
  
  images, GT = Variable(images.cuda()), Variable(GT.cuda())
  
  SR = self.attUnet(images)
  SR_probs = torch.nn.functional.sigmoid(SR)

  length += images.size(0)
  
  loss = bce_dice_loss(SR_probs, GT)
  avg_loss += loss.item()
  
  self.optimizer.zero_grad()
  loss.backward()
  self.optimizer.step()
</code></pre>
<p>Does this mean I have to get rid of this line to remove Sigmoid function:</p>
<pre><code>  SR_probs = torch.nn.functional.sigmoid(SR)
</code></pre>
<p>And also by set the <code>pos_weight &gt; 1</code>, is this the right implementation:<br>
pos_weight = torch.FloatTensor([1, 1.2])<br>
criterion = torch.nn.BCEWithLogitsLoss(pos_weight)<br>
loss = criterion(GT, SR)</p>
<p>Thank you a lot for your help!!</p>

Anita: <p>HI!</p>
<p>I am getting an error in the pos_weight declaration. How i am supposed to use it?</p>

ptrblck: <p>If you are working with a binary classification task, you should pass a single value:</p>
<pre><code class="lang-python">criterion = nn.BCEWithLogitsLoss(pos_weight=torch.tensor(5.))
x = torch.randn(10, 1)
y = torch.randint(0, 2, (10, 1)).float()

loss = criterion(x, y)
</code></pre>

Anita: <p>I am doing pixel wise classification for medical image segmentation. How should I use pos_weight in this case?</p>