Repository: pytorch/ignite

Issue:
Precision raises an NonComputableError for all zeros model predictions
## 🐛 Bug description

Following https://github.com/pytorch/ignite/issues/1991#issuecomment-844860265

```python
from ignite.metrics import Precision
p = Precision()
p.update((torch.zeros(4), torch.randint(0, 2, (4,))))
p.compute()
> NotComputableError: Precision must have at least one example before it can be computed.
```
which is wrong as we do call `update`.

This is related to the check
```
/opt/conda/lib/python3.8/site-packages/ignite/metrics/precision.py in compute(self)
     51         is_scalar = not isinstance(self._positives, torch.Tensor) or self._positives.ndim == 0
     52         if is_scalar and self._positives == 0:
---> 53             raise NotComputableError(
     54                 f"{self.__class__.__name__} must have at least one example before it can be computed."
     55             )
```
where `self._positives` is 0 for all zeros predictions but is thought as uninitialized.

I think this should be related to a change in pytorch that previously had `torch.tensor([0, 0, 0]).sum()` as 1d tensor and now is 0d tensor.


## Environment

- PyTorch Version (e.g., 1.4): 1.8
- Ignite Version (e.g., 0.3.0): 0.4.4
- OS (e.g., Linux):
- How you installed Ignite (`conda`, `pip`, source):
- Python version:
- Any other relevant information:

cc @liebkne 


Discussion:
yes, it's true

`torch.sum` with default `keepdim=False` will squeeze the dim, 1 to 0 dim in our case.

Thus in a corner case withg zeros:

```python
prec.update((torch.zeros(4), ...))
```
will produce `self._positives` equals `tensor(0)` with size of `torch.Size([])`, `ndim=0`

and later when checking `self._positives == 0` will produce True and raise Error, since

```python
>>> torch.tensor(0) == 0
tensor(True)
```

After reading discussion #335 and some experiments I'd like to propose another solution. It is based on the code proposed in that discussion, but the approach is slightly different. It uses the  `__new__` magics, so it wraps the methods on instance creation, so no need to call `super()` and user cannot break it by overriding `__init__`, `reset`, `update` etc.

```python
class Metric(metaclass=ABCMeta):
...
    def __new__(cls, *args, **kwargs):
        """Prevents metric from being computed before updated.
        """
        _reset = cls.reset
        _update = cls.update
        _compute = cls.compute

        def wrapped_reset(self):
            _reset(self)
            self._updated = False

        cls.reset = wraps(cls.reset)(wrapped_reset)

        def wrapped_update(self, output):
            _update(self, output)
            self._updated = True

        cls.update = wraps(cls.update)(wrapped_update)

        def wrapped_compute(self):
            if not self._updated:
                raise NotComputableError(
                    f"{self.__class__.__name__} not updated before compute."
                )
            return _compute(self)

        cls.compute = wraps(cls.compute)(wrapped_compute)

        return super(Metric, cls).__new__(cls, *args, **kwargs)
...
    def __init__(...):
        ...
```

What do you think?
This solution is smart but IMO it should be a temporary fix. We have a `Metric` interface that could be adapted to handle this flag. However, that would mean 2 method levels : logic (defined by user) and API (the wrappers, used to compute). To be discussed in next version ?
@sdesrozis I'm not sure that we would like just patch the problem as it can be recurrent for other metrics once pytorch has changed some other behaviour on which we rely on (but should not) to assert `reset -> (update)+ -> compute` logic.
As proposed change reside in `Metric` there is no change for extended classes. 

> To be discussed in next version ?

Can you detail what you would like to discuss ?
@sdesrozis Thanks for the review, I have another idea, let's encapsulate `reset-update-comp` in a separate MetaClass, it looks similar to @zasdfgbnm proposed in #336 But here we put the code into metaclass `__call__`, to customize instance creation when "calling a class", like this:

```python
class MetaMetric(ABCMeta):

    def __call__(cls, *args, **kwargs):
        """Prevents metric from being computed before updated.
        """

        _reset = cls.reset
        _update = cls.update
        _compute = cls.compute

        def wrapped_reset(self):
            _reset(self)
            self._updated = False

        cls.reset = wraps(cls.reset)(wrapped_reset)

        def wrapped_update(self, output):
            _update(self, output)
            self._updated = True

        cls.update = wraps(cls.update)(wrapped_update)

        def wrapped_compute(self):
            if not self._updated:
                raise NotComputableError(
                    f"{self.__class__.__name__} not updated before compute."
                )
            return _compute(self)

        cls.compute = wraps(cls.compute)(wrapped_compute)

        return super(MetaMetric, cls).__call__(*args, **kwargs)


# No changes for the Metric class
class Metric(metaclass=ABCMeta):
    ...
```

Then, If we want to activate it in Custom Metric we can do:

```python
class Precision(_BasePrecisionRecall, metaclass=MetaMetric):
    ...
```

Thus precision instance will be created using `MetMetric`'s `__call__`  method with wrappers, if omitted - works as usual.
> class Precision(_BasePrecisionRecall, metaclass=MetaMetric):

@trsvchn I think we should not expose that for the user. It is also a breaking change to avoid for now.
Oh, ok, I see. in such case we can:

```python
class Metric(metaclass=MetaMetric):
    ...
```

Or  use implementation w/ `__new__`
@trsvchn your first solution based on `__new__` is perfect regarding to BC concerns.

> Can you detail what you would like to discuss ?

If we hadn't the BC constraint, we certainly would have changed how to define, and use a metric. The user would have to define the computing methods (not API methods) `reset-update-compute` as today, but the usage would have been different using new API methods, because of the internal logic added (counter for instance). This is the template method design pattern. Having a design based on the same methods for internal computing and API facade is the blocking point here.

Btw, I understand that it could be considered low interest regarding to the impact for existing codes.





Resolving patch:
diff --git a/ignite/metrics/precision.py b/ignite/metrics/precision.py
--- a/ignite/metrics/precision.py
+++ b/ignite/metrics/precision.py
@@ -22,6 +22,7 @@ def __init__(
 
         self._average = average
         self.eps = 1e-20
+        self._updated = False
         super(_BasePrecisionRecall, self).__init__(
             output_transform=output_transform, is_multilabel=is_multilabel, device=device
         )
@@ -30,6 +31,7 @@ def __init__(
     def reset(self) -> None:
         self._true_positives = 0  # type: Union[int, torch.Tensor]
         self._positives = 0  # type: Union[int, torch.Tensor]
+        self._updated = False
 
         if self._is_multilabel:
             init_value = 0.0 if self._average else []
@@ -39,8 +41,7 @@ def reset(self) -> None:
         super(_BasePrecisionRecall, self).reset()
 
     def compute(self) -> Union[torch.Tensor, float]:
-        is_scalar = not isinstance(self._positives, torch.Tensor) or self._positives.ndim == 0
-        if is_scalar and self._positives == 0:
+        if not self._updated:
             raise NotComputableError(
                 f"{self.__class__.__name__} must have at least one example before it can be computed."
             )
@@ -173,3 +174,5 @@ def update(self, output: Sequence[torch.Tensor]) -> None:
         else:
             self._true_positives += true_positives
             self._positives += all_positives
+
+        self._updated = True
diff --git a/ignite/metrics/recall.py b/ignite/metrics/recall.py
--- a/ignite/metrics/recall.py
+++ b/ignite/metrics/recall.py
@@ -121,3 +121,5 @@ def update(self, output: Sequence[torch.Tensor]) -> None:
         else:
             self._true_positives += true_positives
             self._positives += actual_positives
+
+        self._updated = True


Test patch:
diff --git a/tests/ignite/metrics/test_precision.py b/tests/ignite/metrics/test_precision.py
--- a/tests/ignite/metrics/test_precision.py
+++ b/tests/ignite/metrics/test_precision.py
@@ -15,45 +15,58 @@
 
 def test_no_update():
     precision = Precision()
+    assert precision._updated is False
     with pytest.raises(NotComputableError, match=r"Precision must have at least one example before it can be computed"):
         precision.compute()
+    assert precision._updated is False
 
     precision = Precision(is_multilabel=True, average=True)
+    assert precision._updated is False
     with pytest.raises(NotComputableError, match=r"Precision must have at least one example before it can be computed"):
         precision.compute()
+    assert precision._updated is False
 
 
 def test_binary_wrong_inputs():
     pr = Precision()
 
+    assert pr._updated is False
     with pytest.raises(ValueError, match=r"For binary cases, y must be comprised of 0's and 1's"):
         # y has not only 0 or 1 values
         pr.update((torch.randint(0, 2, size=(10,)).long(), torch.arange(0, 10).long()))
+    assert pr._updated is False
 
     with pytest.raises(ValueError, match=r"For binary cases, y_pred must be comprised of 0's and 1's"):
         # y_pred values are not thresholded to 0, 1 values
         pr.update((torch.rand(10,), torch.randint(0, 2, size=(10,)).long(),))
+    assert pr._updated is False
 
     with pytest.raises(ValueError, match=r"y must have shape of"):
         # incompatible shapes
         pr.update((torch.randint(0, 2, size=(10,)).long(), torch.randint(0, 2, size=(10, 5)).long()))
+    assert pr._updated is False
 
     with pytest.raises(ValueError, match=r"y must have shape of"):
         # incompatible shapes
         pr.update((torch.randint(0, 2, size=(10, 5, 6)).long(), torch.randint(0, 2, size=(10,)).long()))
+    assert pr._updated is False
 
     with pytest.raises(ValueError, match=r"y must have shape of"):
         # incompatible shapes
         pr.update((torch.randint(0, 2, size=(10,)).long(), torch.randint(0, 2, size=(10, 5, 6)).long()))
+    assert pr._updated is False
 
 
 @pytest.mark.parametrize("average", [False, True])
 def test_binary_input(average):
 
     pr = Precision(average=average)
+    assert pr._updated is False
 
     def _test(y_pred, y, batch_size):
         pr.reset()
+        assert pr._updated is False
+
         if batch_size > 1:
             n_iters = y.shape[0] // batch_size + 1
             for i in range(n_iters):
@@ -66,6 +79,7 @@ def _test(y_pred, y, batch_size):
         np_y_pred = y_pred.numpy().ravel()
 
         assert pr._type == "binary"
+        assert pr._updated is True
         assert isinstance(pr.compute(), float if average else torch.Tensor)
         pr_compute = pr.compute() if average else pr.compute().numpy()
         assert precision_score(np_y, np_y_pred, average="binary") == pytest.approx(pr_compute)
@@ -91,6 +105,9 @@ def get_test_cases():
             # updated batches
             (torch.randint(0, 2, size=(50, 12, 10)), torch.randint(0, 2, size=(50, 12, 10)), 16),
             (torch.randint(0, 2, size=(50, 1, 12, 10)), torch.randint(0, 2, size=(50, 1, 12, 10)), 16),
+            # Corner case with all zeros predictions
+            (torch.zeros(size=(10,)), torch.randint(0, 2, size=(10,)), 1),
+            (torch.zeros(size=(10, 1)), torch.randint(0, 2, size=(10, 1)), 1),
         ]
 
         return test_cases
@@ -104,51 +121,64 @@ def get_test_cases():
 
 def test_multiclass_wrong_inputs():
     pr = Precision()
+    assert pr._updated is False
 
     with pytest.raises(ValueError):
         # incompatible shapes
         pr.update((torch.rand(10, 5, 4), torch.randint(0, 2, size=(10,)).long()))
+    assert pr._updated is False
 
     with pytest.raises(ValueError):
         # incompatible shapes
         pr.update((torch.rand(10, 5, 6), torch.randint(0, 5, size=(10, 5)).long()))
+    assert pr._updated is False
 
     with pytest.raises(ValueError):
         # incompatible shapes
         pr.update((torch.rand(10), torch.randint(0, 5, size=(10, 5, 6)).long()))
+    assert pr._updated is False
 
     pr = Precision(average=True)
+    assert pr._updated is False
 
     with pytest.raises(ValueError):
         # incompatible shapes between two updates
         pr.update((torch.rand(10, 5), torch.randint(0, 5, size=(10,)).long()))
         pr.update((torch.rand(10, 6), torch.randint(0, 5, size=(10,)).long()))
+    assert pr._updated is True
 
     with pytest.raises(ValueError):
         # incompatible shapes between two updates
         pr.update((torch.rand(10, 5, 12, 14), torch.randint(0, 5, size=(10, 12, 14)).long()))
         pr.update((torch.rand(10, 6, 12, 14), torch.randint(0, 5, size=(10, 12, 14)).long()))
+    assert pr._updated is True
 
     pr = Precision(average=False)
+    assert pr._updated is False
 
     with pytest.raises(ValueError):
         # incompatible shapes between two updates
         pr.update((torch.rand(10, 5), torch.randint(0, 5, size=(10,)).long()))
         pr.update((torch.rand(10, 6), torch.randint(0, 5, size=(10,)).long()))
+    assert pr._updated is True
 
     with pytest.raises(ValueError):
         # incompatible shapes between two updates
         pr.update((torch.rand(10, 5, 12, 14), torch.randint(0, 5, size=(10, 12, 14)).long()))
         pr.update((torch.rand(10, 6, 12, 14), torch.randint(0, 5, size=(10, 12, 14)).long()))
+    assert pr._updated is True
 
 
 @pytest.mark.parametrize("average", [False, True])
 def test_multiclass_input(average):
 
     pr = Precision(average=average)
+    assert pr._updated is False
 
     def _test(y_pred, y, batch_size):
         pr.reset()
+        assert pr._updated is False
+
         if batch_size > 1:
             n_iters = y.shape[0] // batch_size + 1
             for i in range(n_iters):
@@ -162,6 +192,7 @@ def _test(y_pred, y, batch_size):
         np_y = y.numpy().ravel()
 
         assert pr._type == "multiclass"
+        assert pr._updated is True
         assert isinstance(pr.compute(), float if average else torch.Tensor)
         pr_compute = pr.compute() if average else pr.compute().numpy()
         sk_average_parameter = "macro" if average else None
@@ -191,6 +222,9 @@ def get_test_cases():
             # updated batches
             (torch.rand(50, 5, 18, 16), torch.randint(0, 5, size=(50, 18, 16)), 16),
             (torch.rand(50, 7, 20, 12), torch.randint(0, 7, size=(50, 20, 12)), 16),
+            # Corner case with all zeros predictions
+            (torch.zeros(size=(10, 6)), torch.randint(0, 6, size=(10,)), 1),
+            (torch.zeros(size=(10, 4)), torch.randint(0, 4, size=(10,)), 1),
         ]
 
         return test_cases
@@ -204,23 +238,28 @@ def get_test_cases():
 
 def test_multilabel_wrong_inputs():
     pr = Precision(average=True, is_multilabel=True)
+    assert pr._updated is False
 
     with pytest.raises(ValueError):
         # incompatible shapes
         pr.update((torch.randint(0, 2, size=(10,)), torch.randint(0, 2, size=(10,)).long()))
+    assert pr._updated is False
 
     with pytest.raises(ValueError):
         # incompatible y_pred
         pr.update((torch.rand(10, 5), torch.randint(0, 2, size=(10, 5)).long()))
+    assert pr._updated is False
 
     with pytest.raises(ValueError):
         # incompatible y
         pr.update((torch.randint(0, 5, size=(10, 5, 6)), torch.rand(10)))
+    assert pr._updated is False
 
     with pytest.raises(ValueError):
         # incompatible shapes between two updates
         pr.update((torch.randint(0, 2, size=(20, 5)), torch.randint(0, 2, size=(20, 5)).long()))
         pr.update((torch.randint(0, 2, size=(20, 6)), torch.randint(0, 2, size=(20, 6)).long()))
+    assert pr._updated is True
 
 
 def to_numpy_multilabel(y):
@@ -235,9 +274,12 @@ def to_numpy_multilabel(y):
 def test_multilabel_input(average):
 
     pr = Precision(average=average, is_multilabel=True)
+    assert pr._updated is False
 
     def _test(y_pred, y, batch_size):
         pr.reset()
+        assert pr._updated is False
+
         if batch_size > 1:
             n_iters = y.shape[0] // batch_size + 1
             for i in range(n_iters):
@@ -250,6 +292,7 @@ def _test(y_pred, y, batch_size):
         np_y = to_numpy_multilabel(y)
 
         assert pr._type == "multilabel"
+        assert pr._updated is True
         pr_compute = pr.compute() if average else pr.compute().mean().item()
         with warnings.catch_warnings():
             warnings.simplefilter("ignore", category=UndefinedMetricWarning)
@@ -257,9 +300,15 @@ def _test(y_pred, y, batch_size):
 
         pr1 = Precision(is_multilabel=True, average=True)
         pr2 = Precision(is_multilabel=True, average=False)
+        assert pr1._updated is False
+        assert pr2._updated is False
         pr1.update((y_pred, y))
         pr2.update((y_pred, y))
+        assert pr1._updated is True
+        assert pr2._updated is True
         assert pr1.compute() == pytest.approx(pr2.compute().mean().item())
+        assert pr1._updated is True
+        assert pr2._updated is True
 
     def get_test_cases():
 
@@ -282,6 +331,9 @@ def get_test_cases():
             # updated batches
             (torch.randint(0, 2, size=(50, 5, 18, 16)), torch.randint(0, 2, size=(50, 5, 18, 16)), 16),
             (torch.randint(0, 2, size=(50, 4, 20, 23)), torch.randint(0, 2, size=(50, 4, 20, 23)), 16),
+            # Corner case with all zeros predictions
+            (torch.zeros(size=(10, 5)), torch.randint(0, 2, size=(10, 5)), 1),
+            (torch.zeros(size=(10, 4)), torch.randint(0, 2, size=(10, 4)), 1),
         ]
 
         return test_cases
@@ -298,10 +350,12 @@ def test_incorrect_type():
 
     def _test(average):
         pr = Precision(average=average)
+        assert pr._updated is False
 
         y_pred = torch.softmax(torch.rand(4, 4), dim=1)
         y = torch.ones(4).long()
         pr.update((y_pred, y))
+        assert pr._updated is True
 
         y_pred = torch.randint(0, 2, size=(4,))
         y = torch.ones(4).long()
@@ -309,15 +363,21 @@ def _test(average):
         with pytest.raises(RuntimeError):
             pr.update((y_pred, y))
 
+        assert pr._updated is True
+
     _test(average=True)
     _test(average=False)
 
     pr1 = Precision(is_multilabel=True, average=True)
     pr2 = Precision(is_multilabel=True, average=False)
+    assert pr1._updated is False
+    assert pr2._updated is False
     y_pred = torch.randint(0, 2, size=(10, 4, 20, 23))
     y = torch.randint(0, 2, size=(10, 4, 20, 23)).long()
     pr1.update((y_pred, y))
     pr2.update((y_pred, y))
+    assert pr1._updated is True
+    assert pr2._updated is True
     assert pr1.compute() == pytest.approx(pr2.compute().mean().item())
 
 
@@ -325,12 +385,16 @@ def test_incorrect_y_classes():
     def _test(average):
         pr = Precision(average=average)
 
+        assert pr._updated is False
+
         y_pred = torch.randint(0, 2, size=(10, 4)).float()
         y = torch.randint(4, 5, size=(10,)).long()
 
         with pytest.raises(ValueError):
             pr.update((y_pred, y))
 
+        assert pr._updated is False
+
     _test(average=True)
     _test(average=False)
 
@@ -360,11 +424,13 @@ def update(engine, i):
 
         pr = Precision(average=average, device=metric_device)
         pr.attach(engine, "pr")
+        assert pr._updated is False
 
         data = list(range(n_iters))
         engine.run(data=data, max_epochs=n_epochs)
 
         assert "pr" in engine.state.metrics
+        assert pr._updated is True
         res = engine.state.metrics["pr"]
         if isinstance(res, torch.Tensor):
             assert res.device == metric_device
@@ -413,11 +479,13 @@ def update(engine, i):
 
         pr = Precision(average=average, is_multilabel=True, device=metric_device)
         pr.attach(engine, "pr")
+        assert pr._updated is False
 
         data = list(range(n_iters))
         engine.run(data=data, max_epochs=n_epochs)
 
         assert "pr" in engine.state.metrics
+        assert pr._updated is True
         res = engine.state.metrics["pr"]
         res2 = pr.compute()
         if isinstance(res, torch.Tensor):
@@ -447,10 +515,14 @@ def update(engine, i):
 
     pr1 = Precision(is_multilabel=True, average=True)
     pr2 = Precision(is_multilabel=True, average=False)
+    assert pr1._updated is False
+    assert pr2._updated is False
     y_pred = torch.randint(0, 2, size=(10, 4, 20, 23))
     y = torch.randint(0, 2, size=(10, 4, 20, 23)).long()
     pr1.update((y_pred, y))
     pr2.update((y_pred, y))
+    assert pr1._updated is True
+    assert pr2._updated is True
     assert pr1.compute() == pytest.approx(pr2.compute().mean().item())
 
 
@@ -460,6 +532,7 @@ def _test_distrib_accumulator_device(device):
     def _test(average, metric_device):
         pr = Precision(average=average, device=metric_device)
         assert pr._device == metric_device
+        assert pr._updated is False
         # Since the shape of the accumulated amount isn't known before the first update
         # call, the internal variables aren't tensors on the right device yet.
 
@@ -467,6 +540,7 @@ def _test(average, metric_device):
         y = torch.randint(0, 2, size=(10,)).long()
         pr.update((y_pred, y))
 
+        assert pr._updated is True
         assert (
             pr._true_positives.device == metric_device
         ), f"{type(pr._true_positives.device)}:{pr._true_positives.device} vs {type(metric_device)}:{metric_device}"
@@ -488,6 +562,7 @@ def _test_distrib_multilabel_accumulator_device(device):
     def _test(average, metric_device):
         pr = Precision(is_multilabel=True, average=average, device=metric_device)
 
+        assert pr._updated is False
         assert pr._device == metric_device
         assert (
             pr._true_positives.device == metric_device
@@ -500,6 +575,7 @@ def _test(average, metric_device):
         y = torch.randint(0, 2, size=(10, 4, 20, 23)).long()
         pr.update((y_pred, y))
 
+        assert pr._updated is True
         assert (
             pr._true_positives.device == metric_device
         ), f"{type(pr._true_positives.device)}:{pr._true_positives.device} vs {type(metric_device)}:{metric_device}"
diff --git a/tests/ignite/metrics/test_recall.py b/tests/ignite/metrics/test_recall.py
--- a/tests/ignite/metrics/test_recall.py
+++ b/tests/ignite/metrics/test_recall.py
@@ -15,45 +15,58 @@
 
 def test_no_update():
     recall = Recall()
+    assert recall._updated is False
     with pytest.raises(NotComputableError, match=r"Recall must have at least one example before it can be computed"):
         recall.compute()
+    assert recall._updated is False
 
     recall = Recall(is_multilabel=True, average=True)
+    assert recall._updated is False
     with pytest.raises(NotComputableError, match=r"Recall must have at least one example before it can be computed"):
         recall.compute()
+    assert recall._updated is False
 
 
 def test_binary_wrong_inputs():
     re = Recall()
 
+    assert re._updated is False
     with pytest.raises(ValueError, match=r"For binary cases, y must be comprised of 0's and 1's"):
         # y has not only 0 or 1 values
         re.update((torch.randint(0, 2, size=(10,)), torch.arange(0, 10).long()))
+    assert re._updated is False
 
     with pytest.raises(ValueError, match=r"For binary cases, y_pred must be comprised of 0's and 1's"):
         # y_pred values are not thresholded to 0, 1 values
         re.update((torch.rand(10, 1), torch.randint(0, 2, size=(10,)).long()))
+    assert re._updated is False
 
     with pytest.raises(ValueError, match=r"y must have shape of"):
         # incompatible shapes
         re.update((torch.randint(0, 2, size=(10,)), torch.randint(0, 2, size=(10, 5)).long()))
+    assert re._updated is False
 
     with pytest.raises(ValueError, match=r"y must have shape of"):
         # incompatible shapes
         re.update((torch.randint(0, 2, size=(10, 5, 6)), torch.randint(0, 2, size=(10,)).long()))
+    assert re._updated is False
 
     with pytest.raises(ValueError, match=r"y must have shape of"):
         # incompatible shapes
         re.update((torch.randint(0, 2, size=(10,)), torch.randint(0, 2, size=(10, 5, 6)).long()))
+    assert re._updated is False
 
 
 @pytest.mark.parametrize("average", [False, True])
 def test_binary_input(average):
 
     re = Recall(average=average)
+    assert re._updated is False
 
     def _test(y_pred, y, batch_size):
         re.reset()
+        assert re._updated is False
+
         if batch_size > 1:
             n_iters = y.shape[0] // batch_size + 1
             for i in range(n_iters):
@@ -66,6 +79,7 @@ def _test(y_pred, y, batch_size):
         np_y_pred = y_pred.numpy().ravel()
 
         assert re._type == "binary"
+        assert re._updated is True
         assert isinstance(re.compute(), float if average else torch.Tensor)
         re_compute = re.compute() if average else re.compute().numpy()
         assert recall_score(np_y, np_y_pred, average="binary") == pytest.approx(re_compute)
@@ -91,6 +105,9 @@ def get_test_cases():
             # updated batches
             (torch.randint(0, 2, size=(50, 12, 10)), torch.randint(0, 2, size=(50, 12, 10)), 16),
             (torch.randint(0, 2, size=(50, 1, 12, 10)), torch.randint(0, 2, size=(50, 1, 12, 10)), 16),
+            # Corner case with all zeros predictions
+            (torch.zeros(size=(10,)), torch.randint(0, 2, size=(10,)), 1),
+            (torch.zeros(size=(10, 1)), torch.randint(0, 2, size=(10, 1)), 1),
         ]
 
         return test_cases
@@ -104,51 +121,64 @@ def get_test_cases():
 
 def test_multiclass_wrong_inputs():
     re = Recall()
+    assert re._updated is False
 
     with pytest.raises(ValueError):
         # incompatible shapes
         re.update((torch.rand(10, 5, 4), torch.randint(0, 2, size=(10,)).long()))
+    assert re._updated is False
 
     with pytest.raises(ValueError):
         # incompatible shapes
         re.update((torch.rand(10, 5, 6), torch.randint(0, 5, size=(10, 5)).long()))
+    assert re._updated is False
 
     with pytest.raises(ValueError):
         # incompatible shapes
         re.update((torch.rand(10), torch.randint(0, 5, size=(10, 5, 6)).long()))
+    assert re._updated is False
 
     re = Recall(average=True)
+    assert re._updated is False
 
     with pytest.raises(ValueError):
         # incompatible shapes between two updates
         re.update((torch.rand(10, 5), torch.randint(0, 5, size=(10,)).long()))
         re.update((torch.rand(10, 6), torch.randint(0, 5, size=(10,)).long()))
+    assert re._updated is True
 
     with pytest.raises(ValueError):
         # incompatible shapes between two updates
         re.update((torch.rand(10, 5, 12, 14), torch.randint(0, 5, size=(10, 12, 14)).long()))
         re.update((torch.rand(10, 6, 12, 14), torch.randint(0, 5, size=(10, 12, 14)).long()))
+    assert re._updated is True
 
     re = Recall(average=False)
+    assert re._updated is False
 
     with pytest.raises(ValueError):
         # incompatible shapes between two updates
         re.update((torch.rand(10, 5), torch.randint(0, 5, size=(10,)).long()))
         re.update((torch.rand(10, 6), torch.randint(0, 5, size=(10,)).long()))
+    assert re._updated is True
 
     with pytest.raises(ValueError):
         # incompatible shapes between two updates
         re.update((torch.rand(10, 5, 12, 14), torch.randint(0, 5, size=(10, 12, 14)).long()))
         re.update((torch.rand(10, 6, 12, 14), torch.randint(0, 5, size=(10, 12, 14)).long()))
+    assert re._updated is True
 
 
 @pytest.mark.parametrize("average", [False, True])
 def test_multiclass_input(average):
 
     re = Recall(average=average)
+    assert re._updated is False
 
     def _test(y_pred, y, batch_size):
         re.reset()
+        assert re._updated is False
+
         if batch_size > 1:
             n_iters = y.shape[0] // batch_size + 1
             for i in range(n_iters):
@@ -162,6 +192,7 @@ def _test(y_pred, y, batch_size):
         np_y = y.numpy().ravel()
 
         assert re._type == "multiclass"
+        assert re._updated is True
         assert isinstance(re.compute(), float if average else torch.Tensor)
         re_compute = re.compute() if average else re.compute().numpy()
         sk_average_parameter = "macro" if average else None
@@ -191,6 +222,9 @@ def get_test_cases():
             # updated batches
             (torch.rand(50, 5, 18, 16), torch.randint(0, 5, size=(50, 18, 16)), 16),
             (torch.rand(50, 7, 20, 12), torch.randint(0, 7, size=(50, 20, 12)), 16),
+            # Corner case with all zeros predictions
+            (torch.zeros(size=(10, 6)), torch.randint(0, 6, size=(10,)), 1),
+            (torch.zeros(size=(10, 4)), torch.randint(0, 4, size=(10,)), 1),
         ]
 
         return test_cases
@@ -204,23 +238,28 @@ def get_test_cases():
 
 def test_multilabel_wrong_inputs():
     re = Recall(average=True, is_multilabel=True)
+    assert re._updated is False
 
     with pytest.raises(ValueError):
         # incompatible shapes
         re.update((torch.randint(0, 2, size=(10,)), torch.randint(0, 2, size=(10,)).long()))
+    assert re._updated is False
 
     with pytest.raises(ValueError):
         # incompatible y_pred
         re.update((torch.rand(10, 5), torch.randint(0, 2, size=(10, 5)).long()))
+    assert re._updated is False
 
     with pytest.raises(ValueError):
         # incompatible y
         re.update((torch.randint(0, 5, size=(10, 5, 6)), torch.rand(10)))
+    assert re._updated is False
 
     with pytest.raises(ValueError):
         # incompatible shapes between two updates
         re.update((torch.randint(0, 2, size=(20, 5)), torch.randint(0, 2, size=(20, 5)).long()))
         re.update((torch.randint(0, 2, size=(20, 6)), torch.randint(0, 2, size=(20, 6)).long()))
+    assert re._updated is True
 
 
 def to_numpy_multilabel(y):
@@ -235,9 +274,12 @@ def to_numpy_multilabel(y):
 def test_multilabel_input(average):
 
     re = Recall(average=average, is_multilabel=True)
+    assert re._updated is False
 
     def _test(y_pred, y, batch_size):
         re.reset()
+        assert re._updated is False
+
         if batch_size > 1:
             n_iters = y.shape[0] // batch_size + 1
             for i in range(n_iters):
@@ -250,6 +292,7 @@ def _test(y_pred, y, batch_size):
         np_y = to_numpy_multilabel(y)
 
         assert re._type == "multilabel"
+        assert re._updated is True
         re_compute = re.compute() if average else re.compute().mean().item()
         with warnings.catch_warnings():
             warnings.simplefilter("ignore", category=UndefinedMetricWarning)
@@ -257,9 +300,15 @@ def _test(y_pred, y, batch_size):
 
         re1 = Recall(is_multilabel=True, average=True)
         re2 = Recall(is_multilabel=True, average=False)
+        assert re1._updated is False
+        assert re2._updated is False
         re1.update((y_pred, y))
         re2.update((y_pred, y))
+        assert re1._updated is True
+        assert re2._updated is True
         assert re1.compute() == pytest.approx(re2.compute().mean().item())
+        assert re1._updated is True
+        assert re2._updated is True
 
     def get_test_cases():
 
@@ -282,6 +331,9 @@ def get_test_cases():
             # updated batches
             (torch.randint(0, 2, size=(50, 5, 18, 16)), torch.randint(0, 2, size=(50, 5, 18, 16)), 16),
             (torch.randint(0, 2, size=(50, 4, 20, 23)), torch.randint(0, 2, size=(50, 4, 20, 23)), 16),
+            # Corner case with all zeros predictions
+            (torch.zeros(size=(10, 5)), torch.randint(0, 2, size=(10, 5)), 1),
+            (torch.zeros(size=(10, 4)), torch.randint(0, 2, size=(10, 4)), 1),
         ]
 
         return test_cases
@@ -298,10 +350,12 @@ def test_incorrect_type():
 
     def _test(average):
         re = Recall(average=average)
+        assert re._updated is False
 
         y_pred = torch.softmax(torch.rand(4, 4), dim=1)
         y = torch.ones(4).long()
         re.update((y_pred, y))
+        assert re._updated is True
 
         y_pred = torch.zeros(4,)
         y = torch.ones(4).long()
@@ -309,15 +363,21 @@ def _test(average):
         with pytest.raises(RuntimeError):
             re.update((y_pred, y))
 
+        assert re._updated is True
+
     _test(average=True)
     _test(average=False)
 
     re1 = Recall(is_multilabel=True, average=True)
     re2 = Recall(is_multilabel=True, average=False)
+    assert re1._updated is False
+    assert re2._updated is False
     y_pred = torch.randint(0, 2, size=(10, 4, 20, 23))
     y = torch.randint(0, 2, size=(10, 4, 20, 23)).long()
     re1.update((y_pred, y))
     re2.update((y_pred, y))
+    assert re1._updated is True
+    assert re2._updated is True
     assert re1.compute() == pytest.approx(re2.compute().mean().item())
 
 
@@ -325,12 +385,16 @@ def test_incorrect_y_classes():
     def _test(average):
         re = Recall(average=average)
 
+        assert re._updated is False
+
         y_pred = torch.randint(0, 2, size=(10, 4)).float()
         y = torch.randint(4, 5, size=(10,)).long()
 
         with pytest.raises(ValueError):
             re.update((y_pred, y))
 
+        assert re._updated is False
+
     _test(average=True)
     _test(average=False)
 
@@ -361,11 +425,13 @@ def update(engine, i):
 
         re = Recall(average=average, device=metric_device)
         re.attach(engine, "re")
+        assert re._updated is False
 
         data = list(range(n_iters))
         engine.run(data=data, max_epochs=n_epochs)
 
         assert "re" in engine.state.metrics
+        assert re._updated is True
         res = engine.state.metrics["re"]
         if isinstance(res, torch.Tensor):
             assert res.device == metric_device
@@ -414,11 +480,13 @@ def update(engine, i):
 
         re = Recall(average=average, is_multilabel=True, device=metric_device)
         re.attach(engine, "re")
+        assert re._updated is False
 
         data = list(range(n_iters))
         engine.run(data=data, max_epochs=n_epochs)
 
         assert "re" in engine.state.metrics
+        assert re._updated is True
         res = engine.state.metrics["re"]
         res2 = re.compute()
         if isinstance(res, torch.Tensor):
@@ -448,10 +516,14 @@ def update(engine, i):
 
     re1 = Recall(is_multilabel=True, average=True)
     re2 = Recall(is_multilabel=True, average=False)
+    assert re1._updated is False
+    assert re2._updated is False
     y_pred = torch.randint(0, 2, size=(10, 4, 20, 23))
     y = torch.randint(0, 2, size=(10, 4, 20, 23)).long()
     re1.update((y_pred, y))
     re2.update((y_pred, y))
+    assert re1._updated is True
+    assert re2._updated is True
     assert re1.compute() == pytest.approx(re2.compute().mean().item())
 
 
@@ -461,6 +533,7 @@ def _test_distrib_accumulator_device(device):
     def _test(average, metric_device):
         re = Recall(average=average, device=metric_device)
         assert re._device == metric_device
+        assert re._updated is False
         # Since the shape of the accumulated amount isn't known before the first update
         # call, the internal variables aren't tensors on the right device yet.
 
@@ -468,6 +541,7 @@ def _test(average, metric_device):
         y = torch.randint(0, 2, size=(10,)).long()
         re.update((y_reed, y))
 
+        assert re._updated is True
         assert (
             re._true_positives.device == metric_device
         ), f"{type(re._true_positives.device)}:{re._true_positives.device} vs {type(metric_device)}:{metric_device}"
@@ -489,6 +563,7 @@ def _test_distrib_multilabel_accumulator_device(device):
     def _test(average, metric_device):
         re = Recall(is_multilabel=True, average=average, device=metric_device)
 
+        assert re._updated is False
         assert re._device == metric_device
         assert (
             re._true_positives.device == metric_device
@@ -501,6 +576,7 @@ def _test(average, metric_device):
         y = torch.randint(0, 2, size=(10, 4, 20, 23)).long()
         re.update((y_reed, y))
 
+        assert re._updated is True
         assert (
             re._true_positives.device == metric_device
         ), f"{type(re._true_positives.device)}:{re._true_positives.device} vs {type(metric_device)}:{metric_device}"