internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: Lightning-AI/torchmetrics

Issue:
Update and unify a number of metrics in `torchmetrics` docs
## 📚 Documentation

Before the next feature release, it'd be nice to update the number of implemented metrics and unify this number over all occurrences within the docs/pages.

**Additional context:** It looks like we've already had almost 80 metrics, so it'd be pity to underestimate these before another feature release O:]

```bash
$ grep -w docs/source/references/functional.rst -e "func" | wc -l
      78
```



Discussion:

Resolving patch:
diff --git a/torchmetrics/__about__.py b/torchmetrics/__about__.py
--- a/torchmetrics/__about__.py
+++ b/torchmetrics/__about__.py
@@ -11,7 +11,7 @@
 [PyTorch Lightning](https://pytorch-lightning.readthedocs.io/en/stable/). It was originally a part of
 Pytorch Lightning, but got split off so users could take advantage of the large collection of metrics
 implemented without having to install Pytorch Lightning (even though we would love for you to try it out).
-We currently have around 60+ metrics implemented and we continuously are adding more metrics, both within
+We currently have around 80+ metrics implemented and we continuously are adding more metrics, both within
 already covered domains (classification, regression ect.) but also new domains (object detection ect.).
 We make sure that all our metrics are rigorously tested such that you can trust them.
 """


Test patch:
Repository: Lightning-AI/torchmetrics

Issue:
PSNR - Higher is better.
## 🐛 Bug

`PSNR.higher_is_better` should be `True`

### Additional context

This is a simple change, created [PR#959](https://github.com/PyTorchLightning/metrics/pull/959) with the change.



Discussion:

Resolving patch:
diff --git a/torchmetrics/image/psnr.py b/torchmetrics/image/psnr.py
--- a/torchmetrics/image/psnr.py
+++ b/torchmetrics/image/psnr.py
@@ -70,7 +70,7 @@ class PeakSignalNoiseRatio(Metric):
     """
     min_target: Tensor
     max_target: Tensor
-    higher_is_better = False
+    higher_is_better = True
 
     def __init__(
         self,


Test patch:
Repository: Lightning-AI/torchmetrics

Issue:
Compatibility with future PT 1.12
## 🐛 Bug

```
>       distance = x @ y.T
E       RuntimeError: "addmm_impl_cpu_" not implemented for 'Half'
```

### To Reproduce

https://github.com/PyTorchLightning/metrics/runs/6275393755?check_suite_focus=true

#### Code sample

```py
distance = x @ y.T
```

### Expected behavior

<!-- A clear and concise description of what you expected to happen. -->

### Environment

- OS (e.g., Linux): linux
- Python & PyTorch Version (e.g., 1.0): py3.8 & pt1.12

### Additional context

context see https://github.com/pytorch/pytorch/pull/75647
slack: https://pytorch.slack.com/archives/C3PDTEV8E/p1651742487294399

tl;dr: We don't have correct and fast linalg algorithms for half in CPU as none of the backend libraries that we use support them so it's better not to support these via half-baked implementations, and simply make the users cast their inputs if they want to use these methods

> but for GPU it shall be supported, correct? >> Yes

note as well that this just applies to Half on CPU and for linalg functios (think matmul and conv). Pointwise functions on Half on CPU will still be available, and Half on CUDA will still have full support

> it was implemented up till 1.11 but there was no real speed-up, correct?

Not only it was slower, but it was not numerically stable, so it was pretty much a bug (hence the removal without deprecation)


Discussion:

Resolving patch:
diff --git a/torchmetrics/functional/pairwise/cosine.py b/torchmetrics/functional/pairwise/cosine.py
--- a/torchmetrics/functional/pairwise/cosine.py
+++ b/torchmetrics/functional/pairwise/cosine.py
@@ -20,6 +20,16 @@
 from torchmetrics.functional.pairwise.helpers import _check_input, _reduce_distance_matrix
 
 
+def _safe_matmul(x: Tensor, y: Tensor) -> Tensor:
+    """Safe calculation of matrix multiplication.
+
+    If input is float16, will cast to float32 for computation and back again.
+    """
+    if x.dtype == torch.float16 or y.dtype == torch.float16:
+        return (x.float() @ y.T.float()).half()
+    return x @ y.T
+
+
 def _pairwise_cosine_similarity_update(
     x: Tensor, y: Optional[Tensor] = None, zero_diagonal: Optional[bool] = None
 ) -> Tensor:
@@ -37,7 +47,7 @@ def _pairwise_cosine_similarity_update(
     norm = torch.norm(y, p=2, dim=1)
     y /= norm.unsqueeze(1)
 
-    distance = x @ y.T
+    distance = _safe_matmul(x, y)
     if zero_diagonal:
         distance.fill_diagonal_(0)
     return distance


Test patch:
Repository: Lightning-AI/torchmetrics

Issue:
Pytorch<1.8 not working despite requirements.txt
## 🐛 Bug

In `requirements.txt` you depend on torch>=1.3.1 but while trying to use torchmetrics with torch1.4 I encountered an error in https://github.com/PyTorchLightning/metrics/blob/master/torchmetrics/utilities/data.py#L242 because `are_deterministic_algorithms_enabled` function is only available for torch>=1.8

### To Reproduce

Install torchmetrics and torch<1.8 and run the provided script.



#### Code sample

```python
import torch
from torchmetrics.functional import confusion_matrix
x = torch.tensor([[1, 2, 3], [1, 2, 3]]).cuda()
y = torch.tensor([[1, 2, 3], [1, 2, 3]]).cuda()
confusion_matrix(x, y, num_classes=4)
```

### Expected behavior

Should run.

### Environment

- TorchMetrics version (and how you installed TM, e.g. `conda`, `pip`, build from source): pip install torchmetrics~=0.7.2 
- Python & PyTorch Version (e.g., 1.0): python3.7 and pytorch1.4
- Any other relevant information such as OS (e.g., Linux): Ubuntu 20.04




Discussion:

Resolving patch:
diff --git a/torchmetrics/utilities/data.py b/torchmetrics/utilities/data.py
--- a/torchmetrics/utilities/data.py
+++ b/torchmetrics/utilities/data.py
@@ -16,6 +16,20 @@
 import torch
 from torch import Tensor, tensor
 
+from torchmetrics.utilities.imports import _TORCH_GREATER_EQUAL_1_6, _TORCH_GREATER_EQUAL_1_7, _TORCH_GREATER_EQUAL_1_8
+
+if _TORCH_GREATER_EQUAL_1_8:
+    deterministic = torch.are_deterministic_algorithms_enabled
+elif _TORCH_GREATER_EQUAL_1_7:
+    deterministic = torch.is_deterministic
+elif _TORCH_GREATER_EQUAL_1_6:
+    deterministic = torch._is_deterministic
+else:
+
+    def deterministic() -> bool:
+        return True
+
+
 METRIC_EPS = 1e-6
 
 
@@ -239,7 +253,7 @@ def _bincount(x: Tensor, minlength: Optional[int] = None) -> Tensor:
     Returns:
         Number of occurrences for each unique element in x
     """
-    if x.is_cuda and torch.are_deterministic_algorithms_enabled():
+    if x.is_cuda and deterministic():
         if minlength is None:
             minlength = len(torch.unique(x))
         output = torch.zeros(minlength, device=x.device, dtype=torch.long)


Test patch:
Repository: Lightning-AI/torchmetrics

Issue:
Minor discrepancy in AveragePrecision documentation for `average="none"`
## 📚 Documentation

The [documentation](https://torchmetrics.readthedocs.io/en/latest/classification/average_precision.html) for the `torchmetrics.AveragePrecision` class and `torchmetrics.functional.average_precision()` function state that setting `average="none"` is permitted. However, the source code only seems to allow `average=None` (see [here](https://github.com/Lightning-AI/metrics/blob/master/src/torchmetrics/classification/avg_precision.py#L98) and [here](https://github.com/Lightning-AI/metrics/blob/master/src/torchmetrics/functional/classification/average_precision.py#L175)).

I'd be happy to submit a PR for this but I would like some feedback on how to best resolve this, since I am not familiar with the design of this library. The two immediate directions I can think of are editing the documentation to only allow `average=None` or editing the source code to support `average="none"`.

Thanks!


Discussion:
Hi! thanks for your contribution!, great first issue!
Hi @alstonlo, thanks for raising this issue.
To be consistent with the remaining codebase it should allow `"none"` also. It essentially boils down to changing these lines of code:
https://github.com/Lightning-AI/metrics/blob/e46258c5555491474e246936d4972351656177cd/src/torchmetrics/functional/classification/average_precision.py#L173-L175
to 
```python
if average is None or average == "none":
    return res
allowed_average = ("micro", "macro", "weighted", "none", None)
```
Feel free to send a PR :]

Resolving patch:
diff --git a/src/torchmetrics/functional/classification/average_precision.py b/src/torchmetrics/functional/classification/average_precision.py
--- a/src/torchmetrics/functional/classification/average_precision.py
+++ b/src/torchmetrics/functional/classification/average_precision.py
@@ -167,9 +167,9 @@ def _average_precision_compute_with_precision_recall(
             return res[~torch.isnan(res)].mean()
         weights = torch.ones_like(res) if weights is None else weights
         return (res * weights)[~torch.isnan(res)].sum()
-    if average is None:
+    if average is None or average == "none":
         return res
-    allowed_average = ("micro", "macro", "weighted", None)
+    allowed_average = ("micro", "macro", "weighted", "none", None)
     raise ValueError(f"Expected argument `average` to be one of {allowed_average}" f" but got {average}")
 
 


Test patch:
Repository: Lightning-AI/torchmetrics

Issue:
notation typo in Cosine Similarity docs 
## 📚 Documentation

There is a typo in the notation for the [pairwise_cosine_similarity](https://torchmetrics.readthedocs.io/en/stable/pairwise/cosine_similarity.html)


![cosine_sim](https://user-images.githubusercontent.com/57797966/180202448-a2970270-9a16-4405-b0d7-909e011a6315.jpg)




Discussion:

Resolving patch:
diff --git a/src/torchmetrics/functional/pairwise/cosine.py b/src/torchmetrics/functional/pairwise/cosine.py
--- a/src/torchmetrics/functional/pairwise/cosine.py
+++ b/src/torchmetrics/functional/pairwise/cosine.py
@@ -54,7 +54,7 @@ def pairwise_cosine_similarity(
 
     .. math::
         s_{cos}(x,y) = \frac{<x,y>}{||x|| \cdot ||y||}
-                     = \frac{\sum_{d=1}^D x_d \cdot y_d }{\sqrt{\sum_{d=1}^D x_i^2} \cdot \sqrt{\sum_{d=1}^D x_i^2}}
+                     = \frac{\sum_{d=1}^D x_d \cdot y_d }{\sqrt{\sum_{d=1}^D x_i^2} \cdot \sqrt{\sum_{d=1}^D y_i^2}}
 
     If both :math:`x` and :math:`y` are passed in, the calculation will be performed pairwise
     between the rows of :math:`x` and :math:`y`.


Test patch:
Repository: Lightning-AI/torchmetrics

Issue:
Refine the warning message about `full_state`
The warning message hints a non-existed function because the function `check_forward_no_full_state` has been renamed to `check_forward_full_state_property` in this [PR](https://github.com/Lightning-AI/metrics/pull/1048/). (Please also see [checks.py](https://github.com/Lightning-AI/metrics/blob/master/src/torchmetrics/utilities/checks.py) for details of the function.) Therefore, the message should be refined.



Discussion:

Resolving patch:
diff --git a/src/torchmetrics/metric.py b/src/torchmetrics/metric.py
--- a/src/torchmetrics/metric.py
+++ b/src/torchmetrics/metric.py
@@ -143,7 +143,7 @@ def __init__(
                 default needs access to the full metric state. If this is not the case, significant speedups can be
                 achieved and we recommend setting this to `False`.
                 We provide an checking function
-                `from torchmetrics.utilities import check_forward_no_full_state`
+                `from torchmetrics.utilities import check_forward_full_state_property`
                 that can be used to check if the `full_state_update=True` (old and potential slower behaviour,
                 default for now) or if `full_state_update=False` can be used safely.
                 """,


Test patch:
Repository: Lightning-AI/torchmetrics

Issue:
`list index out of range` in `_sync_dist`
## 🐛 Bug

`list index out of range` out of range error when using a `list` state with a `default=[]`.

### To Reproduce

`self.add_state("p_true_positives", default=[], dist_reduce_fx="cat")`

when no value is added during update, it raises a list index out of range when hitting:

`if isinstance(output_dict[attr][0], Tensor):` in metric.py (l. 374).

#### Code sample

See above

### Expected behavior

Just add a len() > 0 check before

### Environment

- TorchMetrics version (and how you installed TM, e.g. `conda`, `pip`, build from source): 0.9.3, installed with pip
- Python & PyTorch Version (e.g., 1.0): torch 1.12.1 / python 3.10.4
- Any other relevant information such as OS (e.g., Linux): Linux

### Additional context



Discussion:
Hi! thanks for your contribution!, great first issue!

Resolving patch:
diff --git a/src/torchmetrics/metric.py b/src/torchmetrics/metric.py
--- a/src/torchmetrics/metric.py
+++ b/src/torchmetrics/metric.py
@@ -371,6 +371,10 @@ def _sync_dist(self, dist_sync_fn: Callable = gather_all_tensors, process_group:
         for attr, reduction_fn in self._reductions.items():
             # pre-processing ops (stack or flatten for inputs)
 
+            if isinstance(output_dict[attr], list) and len(output_dict[attr]) == 0:
+                setattr(self, attr, [])
+                continue
+
             if isinstance(output_dict[attr][0], Tensor):
                 output_dict[attr] = torch.stack(output_dict[attr])
             elif isinstance(output_dict[attr][0], list):


Test patch:
diff --git a/tests/unittests/bases/test_ddp.py b/tests/unittests/bases/test_ddp.py
--- a/tests/unittests/bases/test_ddp.py
+++ b/tests/unittests/bases/test_ddp.py
@@ -273,3 +273,16 @@ def _test_sync_on_compute_list_state(rank, worldsize, sync_on_compute):
 def test_sync_on_compute(sync_on_compute, test_func):
     """Test that syncronization of states can be enabled and disabled for compute."""
     torch.multiprocessing.spawn(test_func, args=(2, sync_on_compute), nprocs=2)
+
+
+def _test_sync_with_empty_lists(rank, worldsize):
+    setup_ddp(rank, worldsize)
+    dummy = DummyListMetric()
+    val = dummy.compute()
+    assert val == []
+
+
+@pytest.mark.skipif(sys.platform == "win32", reason="DDP not available on windows")
+def test_sync_with_empty_lists():
+    """Test that syncronization of states can be enabled and disabled for compute."""
+    torch.multiprocessing.spawn(_test_sync_with_empty_lists, args=(2,), nprocs=2)
Repository: Lightning-AI/torchmetrics

Issue:
source links are broken in docs
## 📚 Documentation


There's something wrong with metrics source redirect:
1. On `stable`, If you click on source, it doesn't show up anything on master which is fine I guess since it's the stable one but I think it should lead to the release branch instead ([click source here](https://torchmetrics.readthedocs.io/en/stable/classification/accuracy.html))
2. On the `latest`, it leads to some weird URL ([click source here](https://torchmetrics.readthedocs.io/en/latest/classification/accuracy.html))

cc @Borda 


Discussion:
Hi! thanks for your contribution!, great first issue!
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.


Resolving patch:
diff --git a/docs/source/conf.py b/docs/source/conf.py
--- a/docs/source/conf.py
+++ b/docs/source/conf.py
@@ -322,7 +322,7 @@ def find_source():
         if any(s in fname for s in ("readthedocs", "rtfd", "checkouts")):
             # /home/docs/checkouts/readthedocs.org/user_builds/pytorch_lightning/checkouts/
             #  devel/pytorch_lightning/utilities/cls_experiment.py#L26-L176
-            path_top = os.path.abspath(os.path.join("..", "..", ".."))
+            path_top = os.path.abspath(os.path.join("..", "..", "..", ".."))
             fname = os.path.relpath(fname, start=path_top)
         else:
             # Local build, imitate master


Test patch:
Repository: Lightning-AI/torchmetrics

Issue:
NLTK punkt is downloaded by Rouge even when unneeded
## 🐛 Bug

torchmetrics.ROUGEScore uses nltk's punkt library to compute the RougeLSum metric, however punkt is downloaded even if RougeLSum is not selected. In certain environments, e.g. multiprocessing environments, this results in an exception being thrown. Errors are also thrown and handled in the case of an environment without internet access, e.g. 
 ```
[nltk_data] Error loading punkt: <urlopen error [Errno -3] Temporary
[nltk_data]     failure in name resolution>
```

### To Reproduce

Run ROUGEScore on any input dataset, without selecting the RougeLSum metric. 

#### Code sample

```
from torchmetrics.text.rouge import ROUGEScore
rouge = ROUGEScore(rouge_keys=('rouge1', 'rouge2', 'rougeL'))
preds = ["this is a sentence", "this is another sentence"]
targets = ["this is a sentence", "this is a different sentence"]
rouge.update(preds, targets) # Errors and failures occur on this line
print(rouge.compute())
```

### Expected behavior

nltk punkt is not downloaded, as it is not ever used to produce output

### Environment

- TorchMetrics 0.9.1
- pytorch_lightning 1.6.4
- pytorch 1.11
- Python 3.8

### Additional context

I believe adding an if statement here that checks `if "Lsum" in rouge_keys_values:` before computing `pred_lsum` would address the issue
https://github.com/Lightning-AI/metrics/blob/627ed6ad153e41dfd6aae8e48022b6cdb1e9b075/src/torchmetrics/functional/text/rouge.py#L318

similar to what is done on line 326 with `target_lsum`:
https://github.com/Lightning-AI/metrics/blob/627ed6ad153e41dfd6aae8e48022b6cdb1e9b075/src/torchmetrics/functional/text/rouge.py#L326


Discussion:
Hi! thanks for your contribution!, great first issue!

Resolving patch:
diff --git a/src/torchmetrics/functional/text/rouge.py b/src/torchmetrics/functional/text/rouge.py
--- a/src/torchmetrics/functional/text/rouge.py
+++ b/src/torchmetrics/functional/text/rouge.py
@@ -315,10 +315,11 @@ def _rouge_score_update(
         result_avg: Dict[Union[int, str], List[Dict[str, Tensor]]] = {rouge_key: [] for rouge_key in rouge_keys_values}
         list_results = []
         pred = _normalize_and_tokenize_text(pred_raw, stemmer, normalizer, tokenizer)
-        pred_lsum = [
-            _normalize_and_tokenize_text(pred_sentence, stemmer, normalizer, tokenizer)
-            for pred_sentence in _split_sentence(pred_raw)
-        ]
+        if "Lsum" in rouge_keys_values:
+            pred_lsum = [
+                _normalize_and_tokenize_text(pred_sentence, stemmer, normalizer, tokenizer)
+                for pred_sentence in _split_sentence(pred_raw)
+            ]
 
         for target_raw_inner in target_raw:
             tgt = _normalize_and_tokenize_text(target_raw_inner, stemmer, normalizer, tokenizer)


Test patch:
Repository: Lightning-AI/torchmetrics

Issue:
inplace operation in pairwise_cosine_similarity
## 🐛 Bug
Hello !
The x, y values are modified inplace in the `pairwise_cosine_similarity` function. 
This is not documented and may cause bugs that are difficult to find. 
Thank you.

### To Reproduce

```python
import torch
from torchmetrics.functional import pairwise_cosine_similarity
x = torch.tensor([[2, 3], [3, 5], [5, 8]], dtype=torch.float32)
y = torch.tensor([[1, 0], [2, 1]], dtype=torch.float32)
print("Result:", pairwise_cosine_similarity(x, y))
print("X:", x)
print("Y:", y)
"""Out[0]
Result: tensor([[0.5547, 0.8682],
        [0.5145, 0.8437],
        [0.5300, 0.8533]])
X: tensor([[0.5547, 0.8321],
        [0.5145, 0.8575],
        [0.5300, 0.8480]])
Y: tensor([[1.0000, 0.0000],
        [0.8944, 0.4472]])
"""
```

### Environment
torchmetrics==0.10.0




Discussion:
Hi! thanks for your contribution!, great first issue!

Resolving patch:
diff --git a/src/torchmetrics/functional/pairwise/cosine.py b/src/torchmetrics/functional/pairwise/cosine.py
--- a/src/torchmetrics/functional/pairwise/cosine.py
+++ b/src/torchmetrics/functional/pairwise/cosine.py
@@ -34,9 +34,9 @@ def _pairwise_cosine_similarity_update(
     x, y, zero_diagonal = _check_input(x, y, zero_diagonal)
 
     norm = torch.norm(x, p=2, dim=1)
-    x /= norm.unsqueeze(1)
+    x = x / norm.unsqueeze(1)
     norm = torch.norm(y, p=2, dim=1)
-    y /= norm.unsqueeze(1)
+    y = y / norm.unsqueeze(1)
 
     distance = _safe_matmul(x, y)
     if zero_diagonal:


Test patch:
Repository: Lightning-AI/torchmetrics

Issue:
MatthewsCorrCoef has massive slow down on both CPU and especially GPU
## 🐛 Bug

It's extremely slow to compute the Matthew Correlation Coefficient since torchmetrics == 0.10.0.

### To Reproduce

During my testing, I am seeing massive slow down when computing the Matthew Correlation Coefficient, especially on the GPU (using pytorch-lightning to build, train and test a deep learning model's performance on this metric).

I have compiled a code sample (next section) and below shows the result over different versions of torchmetrics. 

ON CPU:
torchmetrics 0.7.3: 0.93961 s
torchmetrics 0.8.0: 0.93549 s
torchmetrics 0.8.2: 0.94494 s
torchmetrics 0.9.0: 0.92856 s
torchmetrics 0.9.2: 0.93682 s
torchmetrics 0.10.0: 1.10903 s

ON GPU:
torchmetrics 0.7.3: 0.11444 s
torchmetrics 0.8.0: 0.11682 s
torchmetrics 0.8.2: 0.11425 s
torchmetrics 0.9.0: 0.11433 s
torchmetrics 0.9.2: 0.11410 s
torchmetrics 0.10.0: 359.30208 s

So yeah testing over thousands of batches now takes almost a week for me to complete :rofl: Please take a look at this soon.

#### Code sample

```
from tqdm.auto import tqdm
import time
import torch
from torchmetrics import MatthewsCorrCoef

torch.manual_seed(1)

b, h, w = 10, 1080, 1920
device = "cpu"

def generate(b, h, w):
    prob = torch.rand(b, h, w).to(device)
    truth = torch.randint(0, 2, (b, h, w)).to(device)
    return prob, truth

batches = []
for _ in range(10):
    batches.append(generate(b, h, w))

mcc = MatthewsCorrCoef(num_classes=2).to(device)

t1 = time.time()
for detections, targets in tqdm(batches):
    mcc.update(detections, targets)
print(f"{time.time() - t1:.5f}")
```

### Expected behavior

It's supposed to be faster in the range of ~0.1 second for GPU and ~0.9 second for CPU in this naive benchmark.

### Environment

- GPU: NVIDIA RTX 3090
- TorchMetrics version (`pip`): 0.10.0
- Python & PyTorch Version (e.g., 1.0): python=3.9, pytorch=1.12.1
- Any other relevant information such as OS (e.g., Linux): ubuntu18.04, (same behavior with 20.04)


Discussion:
Hi! thanks for your contribution!, great first issue!
wow, for GPU it is huge :octopus: :face_with_spiral_eyes: 

Resolving patch:
diff --git a/src/torchmetrics/utilities/data.py b/src/torchmetrics/utilities/data.py
--- a/src/torchmetrics/utilities/data.py
+++ b/src/torchmetrics/utilities/data.py
@@ -225,8 +225,7 @@ def _bincount(x: Tensor, minlength: Optional[int] = None) -> Tensor:
         for i in range(minlength):
             output[i] = (x == i).sum()
         return output
-    z = torch.zeros(minlength, device=x.device, dtype=x.dtype)
-    return z.index_add_(0, x, torch.ones_like(x))
+    return torch.bincount(x, minlength=minlength)
 
 
 def _flexible_bincount(x: Tensor) -> Tensor:


Test patch: