internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: Lightning-AI/torchmetrics

Issue:
off-by-one in MultiClassJaccard ignore_index handling
## 🐛 Bug

I believe the bugfix https://github.com/Lightning-AI/torchmetrics/pull/1386 introduced aother bug when computing `MulticlassJaccardIndex`.

This bug appears when `num_classes == ignore_index`.

### To Reproduce
<details>
  <summary>Code sample</summary>

```python
import torch
from torchmetrics.classification import MulticlassJaccardIndex
target = torch.tensor([2, 1, 0, 0])
preds = torch.tensor([2, 1, 0, 1])
metric = MulticlassJaccardIndex(num_classes=3, ignore_index=3)
metric(preds, target)
print(metric.confmat)
print(metric.compute())
```
results in: 

`IndexError: index 3 is out of bounds for dimension 0 with size 3`

</details>

### Expected behavior
setting `ignore_index == num_classes` should work

### Environment

```
pip show torchmetrics
Name: torchmetrics
Version: 0.11.4
```


Fix: https://github.com/Lightning-AI/torchmetrics/pull/1860/files



Discussion:

Resolving patch:
diff --git a/src/torchmetrics/functional/classification/jaccard.py b/src/torchmetrics/functional/classification/jaccard.py
--- a/src/torchmetrics/functional/classification/jaccard.py
+++ b/src/torchmetrics/functional/classification/jaccard.py
@@ -65,7 +65,7 @@ def _jaccard_index_reduce(
     if average == "binary":
         return confmat[1, 1] / (confmat[0, 1] + confmat[1, 0] + confmat[1, 1])
 
-    ignore_index_cond = ignore_index is not None and 0 <= ignore_index <= confmat.shape[0]
+    ignore_index_cond = ignore_index is not None and 0 <= ignore_index < confmat.shape[0]
     multilabel = confmat.ndim == 3
     if multilabel:
         num = confmat[:, 1, 1]


Test patch:
diff --git a/tests/unittests/classification/test_jaccard.py b/tests/unittests/classification/test_jaccard.py
--- a/tests/unittests/classification/test_jaccard.py
+++ b/tests/unittests/classification/test_jaccard.py
@@ -131,7 +131,7 @@ def _sklearn_jaccard_index_multiclass(preds, target, ignore_index=None, average=
     preds = preds.flatten()
     target = target.flatten()
     target, preds = remove_ignore_index(target, preds, ignore_index)
-    if ignore_index is not None and 0 <= ignore_index <= NUM_CLASSES:
+    if ignore_index is not None and 0 <= ignore_index < NUM_CLASSES:
         labels = [i for i in range(NUM_CLASSES) if i != ignore_index]
         res = sk_jaccard_index(y_true=target, y_pred=preds, average=average, labels=labels)
         return np.insert(res, ignore_index, 0.0) if average is None else res
Repository: Lightning-AI/torchmetrics

Issue:
CLIPScore demo code is incorrect

Page: [CLIPScore](https://torchmetrics.readthedocs.io/en/stable/multimodal/clip_score.html)

Demo import CLIPScore as:
```python
from torchmetrics.multimodal import CLIPScore
```

But the correct import is: 
```python
from torchmetrics.multimodal.clip_score import CLIPScore
```


Discussion:
Hi! thanks for your contribution!, great first issue!

Resolving patch:
diff --git a/src/torchmetrics/multimodal/clip_score.py b/src/torchmetrics/multimodal/clip_score.py
--- a/src/torchmetrics/multimodal/clip_score.py
+++ b/src/torchmetrics/multimodal/clip_score.py
@@ -76,7 +76,7 @@ class CLIPScore(Metric):
     Example:
         >>> import torch
         >>> _ = torch.manual_seed(42)
-        >>> from torchmetrics.multimodal import CLIPScore
+        >>> from torchmetrics.multimodal.clip_score import CLIPScore
         >>> metric = CLIPScore(model_name_or_path="openai/clip-vit-base-patch16")
         >>> score = metric(torch.randint(255, (3, 224, 224)), "a photo of a cat")
         >>> print(score.detach())
@@ -150,7 +150,7 @@ def plot(self, val: Union[Tensor, Sequence[Tensor], None] = None, ax: Optional[_
 
             >>> # Example plotting a single value
             >>> import torch
-            >>> from torchmetrics.multimodal import CLIPScore
+            >>> from torchmetrics.multimodal.clip_score import CLIPScore
             >>> metric = CLIPScore(model_name_or_path="openai/clip-vit-base-patch16")
             >>> metric.update(torch.randint(255, (3, 224, 224)), "a photo of a cat")
             >>> fig_, ax_ = metric.plot()
@@ -160,7 +160,7 @@ def plot(self, val: Union[Tensor, Sequence[Tensor], None] = None, ax: Optional[_
 
             >>> # Example plotting multiple values
             >>> import torch
-            >>> from torchmetrics.multimodal import CLIPScore
+            >>> from torchmetrics.multimodal.clip_score import CLIPScore
             >>> metric = CLIPScore(model_name_or_path="openai/clip-vit-base-patch16")
             >>> values = [ ]
             >>> for _ in range(10):


Test patch:
Repository: Lightning-AI/torchmetrics

Issue:
Unable to cross-reference a metric in the sphinx doc.
## 🐛 Bug

When trying to cross-reference a metric in another sphinx doc using for instance `` :class:`torchmetrics.AUROC` ``, the link is not created. Looking at the result of `python -m sphinx.ext.intersphinx https://torchmetrics.readthedocs.io/en/latest/objects.inv | grep AUROC`, you can see that there are no entry for `torchmetrics.AUROC`.

### To Reproduce

1. Add torchmetrics entry into conf.py:
    ```python
    intersphinx_mapping = {
        'torchmetrics': ('https://torchmetrics.readthedocs.io/en/latest/', None),
    }
    ```
2. Create link `` :class:`torchmetrics.AUROC` ``.

### Expected behavior

There should be a link created leading to the documentation of the module.

### Additional context

In the [documentation source code](https://github.com/PyTorchLightning/metrics/blob/master/docs/source/references/modules.rst), there are `:noindex:` on each class. I suggest changing all of them to [:noindexentry:](https://github.com/sphinx-doc/sphinx/pull/7906) as I believe it was the original intention.



Discussion:
> I suggest changing all of them to [:noindexentry:](https://github.com/sphinx-doc/sphinx/pull/7906) as I believe it was the original intention.

didn't know it, mind sending a PR? :)
> > I suggest changing all of them to [:noindexentry:](https://github.com/sphinx-doc/sphinx/pull/7906) as I believe it was the original intention.
> 
> didn't know it, mind sending a PR? :)

Hi, unfortunately, it seems that this works only with "modules" and not autodoc. I just made an issue in Sphinx: https://github.com/sphinx-doc/sphinx/issues/10222 .

Resolving patch:
diff --git a/src/torchmetrics/functional/classification/exact_match.py b/src/torchmetrics/functional/classification/exact_match.py
--- a/src/torchmetrics/functional/classification/exact_match.py
+++ b/src/torchmetrics/functional/classification/exact_match.py
@@ -231,6 +231,7 @@ def exact_match(
     ``task`` argument to either ``'multiclass'`` or ``'multilabel'``. See the documentation of
     :func:`multiclass_exact_match` and :func:`multilabel_exact_match` for the specific details of
     each argument influence and examples.
+
     Legacy Example:
         >>> from torch import tensor
         >>> target = tensor([[[0, 1], [2, 1], [0, 2]], [[1, 1], [2, 0], [1, 2]]])


Test patch:
Repository: Lightning-AI/torchmetrics

Issue:
Backwards incompatible change to MSE for pixelwise regression
## 🐛 Bug

#1937 introduces an unintended consequence: pixelwise regression is no longer supported.

### To Reproduce

Run the following script:
```python
import torch
import torchmetrics

B = 4
H = W = 3

x = torch.rand(B, H, W)
y = torch.rand(B, H, W)

torchmetrics.functional.mean_squared_error(x, y)
```
This results in the following error msg:
```
Traceback (most recent call last):
  File "test.py", line 10, in <module>
    torchmetrics.functional.mean_squared_error(x, y, num_outputs=H * W)
  File "lib/python3.10/site-packages/torchmetrics/functional/regression/mse.py", line 84, in mean_squared_error
    sum_squared_error, n_obs = _mean_squared_error_update(preds, target, num_outputs=num_outputs)
  File "lib/python3.10/site-packages/torchmetrics/functional/regression/mse.py", line 35, in _mean_squared_error_update
    _check_data_shape_to_num_outputs(preds, target, num_outputs, allow_1d_reshape=True)
  File "lib/python3.10/site-packages/torchmetrics/functional/regression/utils.py", line 31, in _check_data_shape_to_num_outputs
    raise ValueError(
ValueError: Expected both predictions and target to be either 1- or 2-dimensional tensors, but got 3 and 3.
```

### Expected behavior

I would expect the MSE metrics to support pixelwise regression (predicting a single regression value for each pixel in an image). The above script works fine with torchmetrics 1.0.3.

### Environment

- TorchMetrics version: 1.1.0, spack
- Python & PyTorch Version: 3.10.10, 2.1.0
- Any other relevant information such as OS: macOS

### Additional context

@SkafteNicki @Borda @justusschock 


Discussion:

Resolving patch:
diff --git a/src/torchmetrics/functional/regression/mse.py b/src/torchmetrics/functional/regression/mse.py
--- a/src/torchmetrics/functional/regression/mse.py
+++ b/src/torchmetrics/functional/regression/mse.py
@@ -16,7 +16,6 @@
 import torch
 from torch import Tensor
 
-from torchmetrics.functional.regression.utils import _check_data_shape_to_num_outputs
 from torchmetrics.utilities.checks import _check_same_shape
 
 
@@ -32,7 +31,6 @@ def _mean_squared_error_update(preds: Tensor, target: Tensor, num_outputs: int)
 
     """
     _check_same_shape(preds, target)
-    _check_data_shape_to_num_outputs(preds, target, num_outputs, allow_1d_reshape=True)
     if num_outputs == 1:
         preds = preds.view(-1)
         target = target.view(-1)


Test patch:
Repository: Lightning-AI/torchmetrics

Issue:
Pearson Correlation Coefficient fails when updating one batch at a time
## 🐛 Bug

The `PearsonCorrCoef` fails when using a single batch per update.

### To Reproduce
```python
import torch
from torchmetrics import PearsonCorrCoef

metric = PearsonCorrCoef()

# Works
metric(torch.tensor([3.0, -0.5, 2.0, 7.0]), torch.tensor([2.5, 0.0, 2.0, 8.0]))
print(metric.compute())  # tensor(0.9849)

metric.reset()

# Doesn't work.
metric(torch.tensor([3.0]), torch.tensor([2.5]))
metric(torch.tensor([-0.5]), torch.tensor([0.0]))
metric(torch.tensor([2.0]), torch.tensor([2.0]))
metric(torch.tensor([7.0]), torch.tensor([8.0]))
print(metric.compute())  # tensor(nan)
```

### Expected behavior

Both ways of updating the metric should work.

### Environment
Python 3.10
torchmetrics==1.03
torch==2.01



Discussion:

Resolving patch:
diff --git a/src/torchmetrics/functional/regression/pearson.py b/src/torchmetrics/functional/regression/pearson.py
--- a/src/torchmetrics/functional/regression/pearson.py
+++ b/src/torchmetrics/functional/regression/pearson.py
@@ -52,9 +52,9 @@ def _pearson_corrcoef_update(
     # Data checking
     _check_same_shape(preds, target)
     _check_data_shape_to_num_outputs(preds, target, num_outputs)
-    cond = n_prior.mean() > 0
-
     n_obs = preds.shape[0]
+    cond = n_prior.mean() > 0 or n_obs == 1
+
     if cond:
         mx_new = (n_prior * mean_x + preds.sum(0)) / (n_prior + n_obs)
         my_new = (n_prior * mean_y + target.sum(0)) / (n_prior + n_obs)
@@ -67,7 +67,6 @@ def _pearson_corrcoef_update(
     if cond:
         var_x += ((preds - mx_new) * (preds - mean_x)).sum(0)
         var_y += ((target - my_new) * (target - mean_y)).sum(0)
-
     else:
         var_x += preds.var(0) * (n_obs - 1)
         var_y += target.var(0) * (n_obs - 1)


Test patch:
diff --git a/tests/unittests/regression/test_pearson.py b/tests/unittests/regression/test_pearson.py
--- a/tests/unittests/regression/test_pearson.py
+++ b/tests/unittests/regression/test_pearson.py
@@ -149,3 +149,20 @@ def test_pearsons_warning_on_small_input(dtype, scale):
     target = scale * torch.randn(100, dtype=dtype)
     with pytest.warns(UserWarning, match="The variance of predictions or target is close to zero.*"):
         pearson_corrcoef(preds, target)
+
+
+def test_single_sample_update():
+    """See issue: https://github.com/Lightning-AI/torchmetrics/issues/2014."""
+    metric = PearsonCorrCoef()
+
+    # Works
+    metric(torch.tensor([3.0, -0.5, 2.0, 7.0]), torch.tensor([2.5, 0.0, 2.0, 8.0]))
+    res1 = metric.compute()
+    metric.reset()
+
+    metric(torch.tensor([3.0]), torch.tensor([2.5]))
+    metric(torch.tensor([-0.5]), torch.tensor([0.0]))
+    metric(torch.tensor([2.0]), torch.tensor([2.0]))
+    metric(torch.tensor([7.0]), torch.tensor([8.0]))
+    res2 = metric.compute()
+    assert torch.allclose(res1, res2)
Repository: Lightning-AI/torchmetrics

Issue:
Running Wrapper documentation code example does not render
Pointing out a documentation issue in the Running wrapper:

The code example does not render properly: 
![image](https://github.com/Lightning-AI/torchmetrics/assets/28459495/ecb30567-4b65-4bda-b3ff-0e6a66f4ff2d)

https://github.com/Lightning-AI/torchmetrics/blob/99d6d9d6ac4eb1b3398241df558604e70521e6b0/src/torchmetrics/wrappers/running.py#L46-L83

I assume this is because the first line after each `Example:` does not start with `>>>`?


Discussion:

Resolving patch:
diff --git a/src/torchmetrics/wrappers/running.py b/src/torchmetrics/wrappers/running.py
--- a/src/torchmetrics/wrappers/running.py
+++ b/src/torchmetrics/wrappers/running.py
@@ -43,8 +43,7 @@ class Running(WrapperMetric):
         base_metric: The metric to wrap.
         window: The size of the running window.
 
-    Example:
-        # Single metric
+    Example (single metric):
         >>> from torch import tensor
         >>> from torchmetrics.wrappers import Running
         >>> from torchmetrics.aggregation import SumMetric
@@ -61,8 +60,7 @@ class Running(WrapperMetric):
         current_val=tensor(4.), running_val=tensor(9.), total_val=tensor(10)
         current_val=tensor(5.), running_val=tensor(12.), total_val=tensor(15)
 
-    Example:
-        # Metric collection
+    Example (metric collection):
         >>> from torch import tensor
         >>> from torchmetrics.wrappers import Running
         >>> from torchmetrics import MetricCollection


Test patch:
Repository: Lightning-AI/torchmetrics

Issue:
Documentation incorrectly specifies the name of "num_labels" attribute as "num_classes" for MultilabelMatthewsCorrCoef



Discussion:

Resolving patch:
diff --git a/src/torchmetrics/classification/matthews_corrcoef.py b/src/torchmetrics/classification/matthews_corrcoef.py
--- a/src/torchmetrics/classification/matthews_corrcoef.py
+++ b/src/torchmetrics/classification/matthews_corrcoef.py
@@ -276,7 +276,7 @@ class MultilabelMatthewsCorrCoef(MultilabelConfusionMatrix):
     - ``mlmcc`` (:class:`~torch.Tensor`): A tensor containing the Multi-label Matthews Correlation Coefficient.
 
     Args:
-        num_classes: Integer specifying the number of labels
+        num_labels: Integer specifying the number of labels
         threshold: Threshold for transforming probability to binary (0,1) predictions
         ignore_index:
             Specifies a target value that is ignored and does not contribute to the metric calculation


Test patch:
Repository: Lightning-AI/torchmetrics

Issue:
UniversalImageQualityIndex & SpectralDistortionIndex are still possible to give NaN results 
## 🐛 Bug

I noticed the issue [#1520](https://github.com/Lightning-AI/torchmetrics/issues/1520).
The fix may still cause "divided by zero" problem because eps is not directly added to the divisor.

### To Reproduce

```python
    upper = 2 * sigma_pred_target
    lower = sigma_pred_sq + sigma_target_sq + torch.finfo(sigma_pred_sq.dtype).eps

    uqi_idx = ((2 * mu_pred_target) * upper) / ((mu_pred_sq + mu_target_sq) * lower)
```

Here `(mu_pred_sq + mu_target_sq) * lower` still may contain zero values. For example, if there are several pictures with large black area, both `mu_pred_sq` and `mu_target_sq` can be zeros. To reproduce the problem, these two images can be as examples.

![Example](https://cdn.wallpaperhub.app/cloudcache/1/f/1/4/0/2/1f1402171f9c39cb111c0c3369132c13d0ea2224.jpg)
![Example](https://cdn.wallpaperhub.app/cloudcache/e/f/4/e/e/0/ef4ee0d1f81b62716529f091115bc5470e664c0f.jpg)
### Expected behavior

```python
    upper = 2 * sigma_pred_target
    lower = sigma_pred_sq + sigma_target_sq

    uqi_idx = ((2 * mu_pred_target) * upper) / ((mu_pred_sq + mu_target_sq) * lower + torch.finfo(lower.dtype).eps)
```

This fix is very easy to understand. Just thanks to your excellent jobs!

### Environment
|Name|Version|Build|Channel|
|:-:|:-:|:-:|:-:|
|_anaconda_depends|2023.07|py311_0|https://repo.anaconda.com/pkgs/main|
|conda|23.9.0|py311haa95532_0|https://repo.anaconda.com/pkgs/main|
|python|3.10.13|he1021f5_0|defaults|
|pytorch|2.1.0|py3.10_cuda12.1_cudnn8_0|pytorch|
|pytorch-cuda|12.1|hde6ce7c_5|pytorch|
|pytorch-mutex|1.0|cuda|pytorch|
|torchaudio|2.1.0|pypi_0|pypi|
|torchmetrics|1.2.0|pyhd8ed1ab_0|conda-forge|
|torchvision| 0.16.0|pypi_0|pypi|




Discussion:
Hi! thanks for your contribution!, great first issue!

Resolving patch:
diff --git a/src/torchmetrics/functional/image/uqi.py b/src/torchmetrics/functional/image/uqi.py
--- a/src/torchmetrics/functional/image/uqi.py
+++ b/src/torchmetrics/functional/image/uqi.py
@@ -108,9 +108,9 @@ def _uqi_compute(
     sigma_pred_target = output_list[4] - mu_pred_target
 
     upper = 2 * sigma_pred_target
-    lower = sigma_pred_sq + sigma_target_sq + torch.finfo(sigma_pred_sq.dtype).eps
-
-    uqi_idx = ((2 * mu_pred_target) * upper) / ((mu_pred_sq + mu_target_sq) * lower)
+    lower = sigma_pred_sq + sigma_target_sq
+    eps = torch.finfo(sigma_pred_sq.dtype).eps
+    uqi_idx = ((2 * mu_pred_target) * upper) / ((mu_pred_sq + mu_target_sq) * lower + eps)
     uqi_idx = uqi_idx[..., pad_h:-pad_h, pad_w:-pad_w]
 
     return reduce(uqi_idx, reduction)


Test patch:
Repository: Lightning-AI/torchmetrics

Issue:
Bug in "dim_zero_cat" reduction "Metric._reduce_states(...)"
## 🐛 Bug

The function Metric._reduce_states(...) is supposed to concat states when told to use "cat", but instead tries to add.

### To Reproduce

Try to update metric states initialized with `dist_reduce_fx='cat'`

First define the metric states in the Metric class init function
```py
# base class for metrics requiring an optimal threshold (like EER, BPCER@APCER1%)
class OptimalThresholdMetrics(Metric):
    # .............

    def __init__(self, iter_max=100, tolerance=0.01) -> None:
        super().__init__()

        # ................

        self.add_state("preds", default=torch.tensor([]), dist_reduce_fx='cat')
        self.add_state("targets", default=torch.tensor([]), dist_reduce_fx='cat')
```

And then, when trying to update the metric with
```py
metric_output = self.train_metrics[0](preds, targets)
```

I get some sort of dimensionality error (I don't remember the exact message) because `Metric._reduce_states(...)` tries to add the states instead of concatenating.

<!-- If you have a code sample, error messages, stack traces, please provide it here as well -->


<!-- Ideally attach a minimal code sample to reproduce the decried issue.
Minimal means having the shortest code but still preserving the bug. -->

</details>

### Expected behavior

When passing `dist_reduce_fx='cat'` metric states are supposed to be concatenated when updated, not added. 

### Environment

I encountered the bug in torchmetrics v1.2.0 installed via pip, but it can also be found in the current master branch

### Additional context

<!-- Add any other context about the problem here. -->

### How to fix
In `src/torchmetrics/metric.py` method `Meric._reduce_states(...)`, it currently says

```py
elif reduce_fn == dim_zero_cat:
    reduced = global_state + local_state
```
instead it should say something like
```py
elif reduce_fn == dim_zero_cat:
    reduced = torch.cat([global_state, local_state])
```


Discussion:
Hi! thanks for your contribution!, great first issue!

Resolving patch:
diff --git a/src/torchmetrics/metric.py b/src/torchmetrics/metric.py
--- a/src/torchmetrics/metric.py
+++ b/src/torchmetrics/metric.py
@@ -409,7 +409,10 @@ def _reduce_states(self, incoming_state: Dict[str, Any]) -> None:
             elif reduce_fn == dim_zero_min:
                 reduced = torch.min(global_state, local_state)
             elif reduce_fn == dim_zero_cat:
-                reduced = global_state + local_state
+                if isinstance(global_state, Tensor):
+                    reduced = torch.cat([global_state, local_state])
+                else:
+                    reduced = global_state + local_state
             elif reduce_fn is None and isinstance(global_state, Tensor):
                 reduced = torch.stack([global_state, local_state])
             elif reduce_fn is None and isinstance(global_state, list):


Test patch:
Repository: Lightning-AI/torchmetrics

Issue:
`dummy_image` in FID doesn't fit into custom feature extractors
## 🐛 Bug

When a custom feature extractor is passed into FID, a hard-coded `dummy_image` is used to find the number of features:
https://github.com/Lightning-AI/torchmetrics/blob/3e5a6ea49046d90574ae25151cebfe3b1d96496d/src/torchmetrics/image/fid.py#L298-L301

which will fail if the custom extractor does not accept input with dtype=`torch.uint8`, shape=`(1, 3, 299, 299)`, and device does not match the custom feature extractor.

### To Reproduce

```
import torchmetrics.image
import torchvision
torchmetrics.image.FrechetInceptionDistance(torchvision.models.resnet50().to('cuda'))
```


Discussion:
Hi! thanks for your contribution!, great first issue!
@asy51 mind sharing env detail fro reproducing the issue?
```
Python 3.10.4
torch 2.0.0+cu177
torchvision 0.15.1+cu117
torchmetrics 1.2.1
torch_fidelity 0.3.0
```

Resolving patch:
diff --git a/src/torchmetrics/image/fid.py b/src/torchmetrics/image/fid.py
--- a/src/torchmetrics/image/fid.py
+++ b/src/torchmetrics/image/fid.py
@@ -230,6 +230,12 @@ class FrechetInceptionDistance(Metric):
             your dataset does not change.
         kwargs: Additional keyword arguments, see :ref:`Metric kwargs` for more info.
 
+    .. note::
+        If a custom feature extractor is provided through the `feature` argument it is expected to either have a
+        attribute called ``num_features`` that indicates the number of features returned by the forward pass or
+        alternatively we will pass through tensor of shape ``(1, 3, 299, 299)`` and dtype ``torch.uint8``` to the
+        forward pass and expect a tensor of shape ``(1, num_features)`` as output.
+
     Raises:
         ValueError:
             If torch version is lower than 1.9
@@ -297,8 +303,11 @@ def __init__(
 
         elif isinstance(feature, Module):
             self.inception = feature
-            dummy_image = torch.randint(0, 255, (1, 3, 299, 299), dtype=torch.uint8)
-            num_features = self.inception(dummy_image).shape[-1]
+            if hasattr(self.inception, "num_features"):
+                num_features = self.inception.num_features
+            else:
+                dummy_image = torch.randint(0, 255, (1, 3, 299, 299), dtype=torch.uint8)
+                num_features = self.inception(dummy_image).shape[-1]
         else:
             raise TypeError("Got unknown input to argument `feature`")
 


Test patch: