Repository: kornia/kornia (Python)

Issue:
Binary focal loss is numerically unstable
### Describe the bug

Binary Focal Loss (kornia.losses.focal.binary_focal_loss_with_logits) is numerically unstable.
It looks like a regression bug - issue reproduces with kornia==0.6.9 but does not with kornia==0.6.4.

It probably happens because of `y = (1 - target)* log(sigmoid(-logit))` that happens for prob neg [here](https://github.com/kornia/kornia/blob/v0.6.9/kornia/losses/focal.py#L217)

When `logit->inf`:
* sigmoid(-logit) -> 0
* log(sigmoid(-logit)) -> log(0) -> (-inf)
* (1-target) == 0
* y = 0 * (-inf) = nan

### Reproduction steps

```bash
1. `import torch; from kornia.losses.focal import binary_focal_loss_with_logits`
2. `binary_focal_loss_with_logits(torch.tensor(100.).view(1, 1, 1).float(), torch.tensor(1.).view(1, 1, 1).float()).item()`

Returns nan for kornia==0.6.9, though it should be 0 (positive class=1 with huge positive logit=100)
```


### Expected behavior

Expect to get 0 value in the example instead of nan.

### Environment

```shell
wget https://raw.githubusercontent.com/pytorch/pytorch/master/torch/utils/collect_env.py
# For security purposes, please check the contents of collect_env.py before running it.
python collect_env.py
```
- PyTorch Version (e.g., 1.0): 1.12.1
- OS (e.g., Linux): Linux
- How you installed PyTorch (`conda`, `pip`, source): conda
- Build command you used (if compiling from source):
- Python version: 3.9.0
- CUDA/cuDNN version:
- GPU models and configuration: CPU
- Any other relevant information:
```


### Additional context

_ I have checked that both on Windows and Linux for kornia==0.6.4 issue does not happen, which probably means that it depends only on kornia but not on third-party libs or env_


Discussion:
Probably we need to limit the lower bound for:

https://github.com/kornia/kornia/blob/d6882a24235839ebd5b82bf0993115be5371c990/kornia/losses/focal.py#L212-L213

@zimka Can you try fix it then throw a PR?
PR that fixes the issue https://github.com/kornia/kornia/pull/2125
@shijianjian FYI



Patch:
diff --git a/kornia/losses/focal.py b/kornia/losses/focal.py
index e37b39c5f0..cc6209588a 100644
--- a/kornia/losses/focal.py
+++ b/kornia/losses/focal.py
@@ -212,9 +212,12 @@ def binary_focal_loss_with_logits(
     probs_pos = input.sigmoid()
     probs_neg = (-input).sigmoid()
 
+    log_probs_pos = nn.functional.logsigmoid(input)
+    log_probs_neg = nn.functional.logsigmoid(-input)
+
     loss_tmp = (
-        -alpha * pos_weight * probs_neg.pow(gamma) * target * probs_pos.log()
-        - (1 - alpha) * probs_pos.pow(gamma) * (1.0 - target) * probs_neg.log()
+        -alpha * pos_weight * probs_neg.pow(gamma) * target * log_probs_pos
+        - (1 - alpha) * probs_pos.pow(gamma) * (1.0 - target) * log_probs_neg
     )
 
     if reduction == 'none':


Test patch:
diff --git a/test/losses/test_losses.py b/test/losses/test_losses.py
index dee30646fa..42346c2e15 100644
--- a/test/losses/test_losses.py
+++ b/test/losses/test_losses.py
@@ -99,6 +99,15 @@ def test_same_output(self, device, dtype):
             logits, labels, **kwargs
         ) == kornia.losses.BinaryFocalLossWithLogits(**kwargs)(logits, labels)
 
+    def test_numeric_stability(self, device, dtype):
+        logits = torch.tensor([[100.0, -100]], dtype=dtype, device=device)
+        labels = torch.tensor([[1.0, 0.0]], dtype=dtype, device=device)
+        expected = torch.tensor([[0.0, 0.0]], dtype=dtype, device=device)
+
+        kwargs = {"alpha": 0.25, "gamma": 2.0, "reduction": 'none'}
+        actual = kornia.losses.binary_focal_loss_with_logits(logits, labels, **kwargs)
+        assert_close(actual, expected, atol=1e-3, rtol=1e-3)
+
 
 class TestFocalLoss:
     def test_smoke_none(self, device, dtype):