Repository: kornia/kornia
Pull request: adding weights to positive examples
#### Changes
<!-- Please include a summary of the change and which issue is fixed. -->
<!-- Please also include relevant motivation and context. -->
<!-- List any dependencies that are required for this change. -->

Fixes #1744


#### Type of change
<!-- Please delete options that are not relevant. -->
- [ ] 📚  Documentation Update
- [ ] 🧪 Tests Cases
- [ ] 🐞 Bug fix (non-breaking change which fixes an issue)
- [x] 🔬 New feature (non-breaking change which adds functionality)
- [ ] 🚨 Breaking change (fix or feature that would cause existing functionality to not work as expected)
- [ ] 📝 This change requires a documentation update


#### Checklist

- [x] My code follows the style guidelines of this project
- [x] I have performed a self-review of my own code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] Did you update CHANGELOG in case of a major change?


Problem statement:
Improve binary focal loss to trade off recall and precision by adding weights to positive examples like torch.nn.BCEWithLogitsLoss
## 🚀 Feature
Improve binary focal loss to trade off recall and precision by adding weights to positive examples. Just like `pos_weight` parameter in torch.nn.BCEWithLogitsLoss. `pos_weight` must be a vector with length equal to the number of classes.

Hints from issue discussion:
@xiaoyuan0203 Open to send a PR ?
@xiaoyuan0203 @edgarriba Looks interesting! Can I take it?
@MrShevan sure ! go ahead
> @xiaoyuan0203 Open to send a PR ?

Sorry for reply to you until now. Could you tell me how to do it? 
Oh sorry. I see you are interested in creating your own PR. Let me know if it's free


It's free of cource. It's my first time to take part in open-source,so I just want to know how to do. If you are interested in make a pull request, you can do it. Do you mean you want to improve the focal loss?
@xiaoyuan0203 i have other suggestions for losses to add e.g for semantic segmentation. or the ones from this model we are going to include: https://github.com/xavysp/LDC
@xiaoyuan0203 Well, I can help you get started :)
We can try to make this PR together. You can fork this and create your own branch. [Here](https://devblog.pytorchlightning.ai/quick-contribution-guide-86d977171b3a) it is well written in paragraph `Prepare your Workspace`. And then I'll join you)
you can also check our less graphical contributor guides: https://github.com/kornia/kornia/blob/master/CONTRIBUTING.rst#developing-kornia
@MrShevan Here? https://github.com/xiaoyuan0203/kornia/tree/feat/focal_loss

Commit message: Update kornia/losses/focal.py

Co-authored-by: Edgar Riba <edgar.riba@gmail.com>

Patch to review:
diff --git a/CHANGELOG.md b/CHANGELOG.md
index aec6a26936..5397043a02 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -15,6 +15,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 - Add Lovasz-Hinge/Softmax losses (#1682)
 - Add `adjust_sigmoid` and `adjust_log` initial implementation (#1685)
 - Added distribution mapper (#1667)
+- `pos_weight` param to focal loss (#1744)
 
 ### :lady_beetle: Bug fixes
 - Fixes filter2d's output shape shrink when padding='same' (#1661)
diff --git a/kornia/losses/focal.py b/kornia/losses/focal.py
index a4dd344d63..d5c04da230 100644
--- a/kornia/losses/focal.py
+++ b/kornia/losses/focal.py
@@ -157,6 +157,7 @@ def binary_focal_loss_with_logits(
     gamma: float = 2.0,
     reduction: str = 'none',
     eps: Optional[float] = None,
+    pos_weight: Optional[torch.Tensor] = None,
 ) -> torch.Tensor:
     r"""Function that computes Binary Focal loss.
 
@@ -178,6 +179,9 @@ def binary_focal_loss_with_logits(
           the number of elements in the output, ``'sum'``: the output will be
           summed.
         eps: Deprecated: scalar for numerically stability when dividing. This is no longer used.
+        pos_weight: a weight of positive examples.
+          It’s possible to trade off recall and precision by adding weights to positive examples.
+          Must be a vector with length equal to the number of classes.
 
     Returns:
         the computed loss.
@@ -207,9 +211,17 @@ def binary_focal_loss_with_logits(
     if input.size(0) != target.size(0):
         raise ValueError(f'Expected input batch_size ({input.size(0)}) to match target batch_size ({target.size(0)}).')
 
+    if pos_weight is None:
+        pos_weight = torch.ones(input.size(-1), device=input.device, dtype=input.dtype)
+    elif not isinstance(pos_weight, torch.Tensor):
+        raise TypeError(f"Input type is not a torch.Tensor. Got {type(input)}")
+    elif input.size(-1) != pos_weight.size(0):
+        raise ValueError(f"Expected pos_weight size ({pos_weight.size(0)}) to match number of "
+                         f"classes ({input.size(1)})")
+
     probs_pos = torch.sigmoid(input)
     probs_neg = torch.sigmoid(-input)
-    loss_tmp = -alpha * torch.pow(probs_neg, gamma) * target * F.logsigmoid(input) - (
+    loss_tmp = -alpha * pos_weight * torch.pow(probs_neg, gamma) * target * F.logsigmoid(input) - (
         1 - alpha
     ) * torch.pow(probs_pos, gamma) * (1.0 - target) * F.logsigmoid(-input)
 
@@ -237,13 +249,16 @@ class BinaryFocalLossWithLogits(nn.Module):
        - :math:`p_t` is the model's estimated probability for each class.
 
     Args:
-        alpha): Weighting factor for the rare class :math:`\alpha \in [0, 1]`.
+        alpha: Weighting factor for the rare class :math:`\alpha \in [0, 1]`.
         gamma: Focusing parameter :math:`\gamma >= 0`.
         reduction: Specifies the reduction to apply to the
           output: ``'none'`` | ``'mean'`` | ``'sum'``. ``'none'``: no reduction
           will be applied, ``'mean'``: the sum of the output will be divided by
           the number of elements in the output, ``'sum'``: the output will be
           summed.
+        pos_weight: a weight of positive examples.
+          It’s possible to trade off recall and precision by adding weights to positive examples.
+          Must be a vector with length equal to the number of classes.
 
     Shape:
         - Input: :math:`(N, *)`.
@@ -258,11 +273,20 @@ class BinaryFocalLossWithLogits(nn.Module):
         >>> output.backward()
     """
 
-    def __init__(self, alpha: float, gamma: float = 2.0, reduction: str = 'none') -> None:
+    def __init__(
+        self,
+        alpha: float,
+        gamma: float = 2.0,
+        reduction: str = 'none',
+        pos_weight: Optional[torch.Tensor] = None
+    ) -> None:
         super().__init__()
         self.alpha: float = alpha
         self.gamma: float = gamma
         self.reduction: str = reduction
+        self.pos_weight: Optional[torch.Tensor] = pos_weight
 
     def forward(self, input: torch.Tensor, target: torch.Tensor) -> torch.Tensor:
-        return binary_focal_loss_with_logits(input, target, self.alpha, self.gamma, self.reduction)
+        return binary_focal_loss_with_logits(
+            input, target, self.alpha, self.gamma, self.reduction, pos_weight=self.pos_weight
+        )
diff --git a/test/losses/test_losses.py b/test/losses/test_losses.py
index ed88793649..158a008542 100644
--- a/test/losses/test_losses.py
+++ b/test/losses/test_losses.py
@@ -39,6 +39,17 @@ def test_smoke_mean_flat(self, device, dtype):
         assert (kornia.losses.binary_focal_loss_with_logits(
             logits, labels, alpha=0.5, gamma=2.0, reduction="mean").shape == ())
 
+    @pytest.mark.parametrize("pos_weight", [None, 1, 5])
+    def test_smoke_pos_weight(self, device, dtype, pos_weight):
+        logits = torch.rand(2, 3, 2, dtype=dtype, device=device)
+        labels = torch.rand(2, 3, 2, dtype=dtype, device=device)
+
+        assert (kornia.losses.binary_focal_loss_with_logits(
+            logits, labels, alpha=0.5, gamma=2.0, reduction="mean",
+            pos_weight=None if pos_weight is None
+            else torch.full([logits.size(-1)], pos_weight, dtype=dtype, device=device)
+        ).shape == ())
+
     def test_jit(self, device, dtype):
         logits = torch.rand(2, 3, 2, dtype=dtype, device=device)
         labels = torch.rand(2, 3, 2, dtype=dtype, device=device)


Code review comments:
diff_hunk: @@ -157,6 +157,7 @@ def binary_focal_loss_with_logits(
     gamma: float = 2.0,
     reduction: str = 'none',
     eps: Optional[float] = None,
+    pos_weight: Optional[torch.Tensor] = None,
path: kornia/losses/focal.py
text: @user1:
```suggestion
    pos_weight: Optional[Tensor] = None,
```
now you can do `from kornia.core import Tensor` -- maybe update others too. Context: we are isolating torch import as much as possible to clean code and understand pytorch dependency