Repository: Lightning-AI/torchmetrics

Issue:
Keyword Argument Filtering for Metric Compositions/Collections does not behave as expected
## 🐛 Bug
Metric collections / compositions (using metric arithmetics) do not filter keyword arguments based on update signature, if a metric does not expect any of the given keyword arguments.

### To Reproduce
Steps to reproduce the behavior:
```python
import torch
from torchmetrics import MetricCollection, Metric, Accuracy


class DummyMetric(Metric):
    def __init__(self):
        super(DummyMetric, self).__init__()

    def update(self, *args, kwarg):
        print("Entered DummyMetric")

    def compute(self):
        return


class MyAccuracy(Metric):
    def __init__(self):
        super(MyAccuracy, self).__init__()

    def update(self, preds, target, kwarg2):
        print("Entered MyAccuracy")

    def compute(self):
        return


if __name__ == "__main__":
    mc = MetricCollection([Accuracy(), DummyMetric()])      # Accuracy() + MyOwnMetric()
    mc2 = MetricCollection([MyAccuracy(), DummyMetric()])
    try:
        mc(torch.tensor([0, 1]), torch.tensor([0, 1]), kwarg="kwarg")
    except TypeError as e:
        print(e)
    mc2(torch.tensor([0, 1]), torch.tensor([0, 1]), kwarg="kwarg", kwarg2="kwarg2")
```
The output of this function is as follows:
```console
update() got an unexpected keyword argument 'kwarg'
Entered MyAccuracy
Entered MyAccuracy
Entered DummyMetric
Entered DummyMetric
```
### Expected Behavior
If a metric does not expect any of the given keyword arguments, it should be called with positional arguments only.


Discussion:
Hi! thanks for your contribution!, great first issue!
@justusschock mind have a look :rabbit: 
@Borda the issue is this line: https://github.com/PyTorchLightning/metrics/blob/master/torchmetrics/metric.py#L566 if no kwarg was found, the filter mecahanism returns all kwargs to match possible **kwargs. We could check if some **kwargs exist and only in that case return all kwargs

Resolving patch:
diff --git a/torchmetrics/metric.py b/torchmetrics/metric.py
--- a/torchmetrics/metric.py
+++ b/torchmetrics/metric.py
@@ -562,8 +562,14 @@ def _filter_kwargs(self, **kwargs: Any) -> Dict[str, Any]:
             k: v for k, v in kwargs.items() if (k in _sign_params.keys() and _sign_params[k].kind not in _params)
         }
 
-        # if no kwargs filtered, return al kwargs as default
-        if not filtered_kwargs:
+        exists_var_keyword = any([v.kind == inspect.Parameter.VAR_KEYWORD for v in _sign_params.values()])
+        # if no kwargs filtered, return all kwargs as default
+        if not filtered_kwargs and not exists_var_keyword:
+            # no kwargs in update signature -> don't return any kwargs
+            filtered_kwargs = {}
+        elif exists_var_keyword:
+            # kwargs found in update signature -> return all kwargs to be sure to not omit any.
+            # filtering logic is likely implemented within the update call.
             filtered_kwargs = kwargs
         return filtered_kwargs
 


Test patch:
diff --git a/tests/bases/test_collections.py b/tests/bases/test_collections.py
--- a/tests/bases/test_collections.py
+++ b/tests/bases/test_collections.py
@@ -18,6 +18,8 @@
 
 from tests.helpers import seed_all
 from tests.helpers.testers import DummyMetricDiff, DummyMetricSum
+from torchmetrics import Metric
+from torchmetrics.classification import Accuracy
 from torchmetrics.metric_collections import MetricCollection
 
 seed_all(42)
@@ -249,3 +251,30 @@ def test_collection_check_arg():
 
     with pytest.raises(ValueError, match="Expected input `postfix` to be a string, but got"):
         MetricCollection._check_arg(1, "postfix")
+
+
+def test_collection_filtering():
+    class DummyMetric(Metric):
+        def __init__(self):
+            super().__init__()
+
+        def update(self, *args, kwarg):
+            print("Entered DummyMetric")
+
+        def compute(self):
+            return
+
+    class MyAccuracy(Metric):
+        def __init__(self):
+            super().__init__()
+
+        def update(self, preds, target, kwarg2):
+            print("Entered MyAccuracy")
+
+        def compute(self):
+            return
+
+    mc = MetricCollection([Accuracy(), DummyMetric()])
+    mc2 = MetricCollection([MyAccuracy(), DummyMetric()])
+    mc(torch.tensor([0, 1]), torch.tensor([0, 1]), kwarg="kwarg")
+    mc2(torch.tensor([0, 1]), torch.tensor([0, 1]), kwarg="kwarg", kwarg2="kwarg2")