Repository: Lightning-AI/torchmetrics

Issue:
Behaviors of AUROC and Average Precision are inconsistent when all labels are equal
## 🐛 Bug

When all labels are equal (either all zeros or all ones), the current implementation of `AUROC` and `AveragePrecision` have pretty different behaviors.
When labels are all ones, AUROC gives 0, while AP gives 1.
When labels are all zeros, AUROC gives 0, while Average Precision gives NaN.

I think it is better to add a flag such that both metrics would return NaN when all labels are equal to better inform users.

### To Reproduce

```
>>> from torchmetrics import AUROC, AveragePrecision
>>> import torch
>>> auroc = AUROC(task = "binary")
>>> ap = AveragePrecision(task = "binary")
>>> preds = torch.randn(10)
>>> labels = torch.ones(10, dtype = torch.long)
>>> auroc(preds, labels)
/opt/homebrew/anaconda3/lib/python3.9/site-packages/torchmetrics/utilities/prints.py:43: UserWarning: No negative samples in targets, false positive value should be meaningless. Returning zero tensor in false positive score
  warnings.warn(*args, **kwargs)  # noqa: B028
tensor(0.)
>>> ap(preds, labels)
tensor(1.)
>>> labels = torch.zeros(10, dtype = torch.long)
>>> auroc(preds, labels)
/opt/homebrew/anaconda3/lib/python3.9/site-packages/torchmetrics/utilities/prints.py:43: UserWarning: No positive samples in targets, true positive value should be meaningless. Returning zero tensor in true positive score
  warnings.warn(*args, **kwargs)  # noqa: B028
tensor(0.)
>>> ap(preds, labels)
tensor(nan)
```

### Expected behavior

when labels are all equal, it should return a NaN all the time.
at least, there can be a flag like `equal_label_mode`.
```python
>>> ap = AveragePrecision(task = "binary", equal_label_mode = "nan")
>>> ap = AveragePrecision(task = "binary", equal_label_mode = "nan")
```
that give the expected behavior.

### Environment

- TorchMetrics version: installed via pip. '1.3.0.post0'
- Python & PyTorch Version (e.g., 1.0): Python 3.9.12,  torch 2.1.0
- Any other relevant information such as OS (e.g., Linux): Linus



Discussion:

Resolving patch:
diff --git a/src/torchmetrics/functional/classification/precision_recall_curve.py b/src/torchmetrics/functional/classification/precision_recall_curve.py
--- a/src/torchmetrics/functional/classification/precision_recall_curve.py
+++ b/src/torchmetrics/functional/classification/precision_recall_curve.py
@@ -23,6 +23,7 @@
 from torchmetrics.utilities.compute import _safe_divide, interp
 from torchmetrics.utilities.data import _bincount, _cumsum
 from torchmetrics.utilities.enums import ClassificationTask
+from torchmetrics.utilities.prints import rank_zero_warn
 
 
 def _binary_clf_curve(
@@ -274,6 +275,12 @@ def _binary_precision_recall_curve_compute(
     fps, tps, thresholds = _binary_clf_curve(state[0], state[1], pos_label=pos_label)
     precision = tps / (tps + fps)
     recall = tps / tps[-1]
+    if (state[1] == 0).all():  # all labels are negative, recall is undefined
+        rank_zero_warn(
+            "No positive samples found in target, recall is undefined. Setting recall to one for all thresholds.",
+            UserWarning,
+        )
+        recall = torch.ones_like(recall)
 
     # need to call reversed explicitly, since including that to slice would
     # introduce negative strides that are not yet supported in pytorch


Test patch:
diff --git a/tests/unittests/classification/test_average_precision.py b/tests/unittests/classification/test_average_precision.py
--- a/tests/unittests/classification/test_average_precision.py
+++ b/tests/unittests/classification/test_average_precision.py
@@ -143,6 +143,14 @@ def test_binary_average_precision_threshold_arg(self, inputs, threshold_fn):
             assert torch.allclose(ap1, ap2)
 
 
+def test_warning_on_no_positives():
+    """Test that a warning is raised when there are no positive samples in the target."""
+    preds = torch.rand(100)
+    target = torch.zeros(100).long()
+    with pytest.warns(UserWarning, match="No positive samples found in target, recall is undefined. Setting recall.*"):
+        binary_average_precision(preds, target)
+
+
 def _reference_sklearn_avg_precision_multiclass(preds, target, average="macro", ignore_index=None):
     preds = np.moveaxis(preds.numpy(), 1, -1).reshape((-1, preds.shape[1]))
     target = target.numpy().flatten()