Repository: Lightning-AI/torchmetrics

Issue:
binary_calibration_error() throw RuntimeError when preds tensor contains some 0
## 🐛 Bug
Hello,

The binary_calibration_error() function from torchmetrics.functional.classification throw "RuntimeError: index -1 is out of bounds for dimension 0 with size 2" when the preds tensor contains some confidences with a value of 0.

### To Reproduce

 `import torch`
`from torchmetrics.functional.classification import binary_calibration_error`
`preds = torch.tensor([1., 1., 1., 1., 0.])`
`target = torch.tensor([0, 0, 1, 1, 1])`
`binary_calibration_error(preds, target, n_bins=2, norm='l1')`

throw the following :

`Traceback (most recent call last):`
  `File "<stdin>", line 1, in <module>`
  `File "/...../.local/lib/python3.8/site-packages/torchmetrics/functional/classification/calibration_error.py", line 203, in binary_calibration_error
    return _ce_compute(confidences, accuracies, n_bins, norm)`
  `File "/...../.local/lib/python3.8/site-packages/torchmetrics/functional/classification/calibration_error.py", line 90, in _ce_compute
    acc_bin, conf_bin, prop_bin = _binning_bucketize(confidences, accuracies, bin_boundaries)`
  `File "/...../.local/lib/python3.8/site-packages/torchmetrics/functional/classification/calibration_error.py", line 48, in _binning_bucketize
    count_bin.scatter_add_(dim=0, index=indices, src=torch.ones_like(confidences))`
`RuntimeError: index -1 is out of bounds for dimension 0 with size 2`
`

### Expected behavior

The function should be able to compute a calibration error even if some confidences are 0, or maybe I am missing something ?

### Environment

- TorchMetrics version: 0.11.4 (installed via pip)
- Python & PyTorch Version: python 3.8.8 and pytorch 1.13.0

## EDIT:

The bug also appears for the BinaryCalibrationError class in torchmetrics.classification



Discussion:
Hi! thanks for your contribution!, great first issue!

Resolving patch:
diff --git a/src/torchmetrics/functional/classification/calibration_error.py b/src/torchmetrics/functional/classification/calibration_error.py
--- a/src/torchmetrics/functional/classification/calibration_error.py
+++ b/src/torchmetrics/functional/classification/calibration_error.py
@@ -40,11 +40,11 @@ def _binning_bucketize(
         tuple with binned accuracy, binned confidence and binned probabilities
     """
     accuracies = accuracies.to(dtype=confidences.dtype)
-    acc_bin = torch.zeros(len(bin_boundaries) - 1, device=confidences.device, dtype=confidences.dtype)
-    conf_bin = torch.zeros(len(bin_boundaries) - 1, device=confidences.device, dtype=confidences.dtype)
-    count_bin = torch.zeros(len(bin_boundaries) - 1, device=confidences.device, dtype=confidences.dtype)
+    acc_bin = torch.zeros(len(bin_boundaries), device=confidences.device, dtype=confidences.dtype)
+    conf_bin = torch.zeros(len(bin_boundaries), device=confidences.device, dtype=confidences.dtype)
+    count_bin = torch.zeros(len(bin_boundaries), device=confidences.device, dtype=confidences.dtype)
 
-    indices = torch.bucketize(confidences, bin_boundaries) - 1
+    indices = torch.bucketize(confidences, bin_boundaries, right=True) - 1
 
     count_bin.scatter_add_(dim=0, index=indices, src=torch.ones_like(confidences))
 


Test patch:
diff --git a/tests/unittests/classification/test_calibration_error.py b/tests/unittests/classification/test_calibration_error.py
--- a/tests/unittests/classification/test_calibration_error.py
+++ b/tests/unittests/classification/test_calibration_error.py
@@ -133,6 +133,13 @@ def test_binary_calibration_error_dtype_gpu(self, input, dtype):
         )
 
 
+def test_binary_with_zero_pred():
+    """Test that metric works with edge case where confidence is zero for a bin."""
+    preds = torch.tensor([1.0, 1.0, 1.0, 1.0, 0.0])
+    target = torch.tensor([0, 0, 1, 1, 1])
+    assert binary_calibration_error(preds, target, n_bins=2, norm="l1") == torch.tensor(0.6)
+
+
 def _netcal_multiclass_calibration_error(preds, target, n_bins, norm, ignore_index):
     preds = preds.numpy()
     target = target.numpy().flatten()