Repository: Project-MONAI/MONAI

Issue:
New type hint in `Inferer` implies additional restriction
The new type hint added to `Inferer` is not implying the same design as before and is implicitly adding additional restriction on `Inferer` base class which may not be desirable. It suggests that the output of the network and the output of `Inferer` call should have the same type, which might not expected as the `Inferer` should be allowed to produce results (via post-processing for instance) different than the network's output.
https://github.com/Project-MONAI/MONAI/blob/385ac0bb6612f88f92e0862a6aee033fb8df5de7/monai/inferers/inferer.py#L51

ModuleNotFoundError: No module named 'numpy.typing'
**Describe the bug**
```
[2023-02-06T23:22:07.279Z] + bash runtests.sh --clean
[2023-02-06T23:22:08.211Z] PYTHONPATH: /home/jenkins/agent/workspace/Monai-pip/monai:
[2023-02-06T23:22:08.211Z] Python: /opt/conda/bin/python
[2023-02-06T23:22:11.481Z] Traceback (most recent call last):
[2023-02-06T23:22:11.482Z]   File "/home/jenkins/agent/workspace/Monai-pip/monai/monai/utils/module.py", line 209, in load_submodules
[2023-02-06T23:22:11.482Z]     mod = import_module(name)
[2023-02-06T23:22:11.482Z]   File "/opt/conda/lib/python3.8/importlib/__init__.py", line 127, in import_module
[2023-02-06T23:22:11.482Z]     return _bootstrap._gcd_import(name[level:], package, level)
[2023-02-06T23:22:11.482Z]   File "/home/jenkins/agent/workspace/Monai-pip/monai/monai/apps/pathology/__init__.py", line 16, in <module>
[2023-02-06T23:22:11.482Z]     from .losses import HoVerNetLoss
[2023-02-06T23:22:11.482Z]   File "/home/jenkins/agent/workspace/Monai-pip/monai/monai/apps/pathology/losses/__init__.py", line 14, in <module>
[2023-02-06T23:22:11.482Z]     from .hovernet_loss import HoVerNetLoss
[2023-02-06T23:22:11.482Z]   File "/home/jenkins/agent/workspace/Monai-pip/monai/monai/apps/pathology/losses/hovernet_loss.py", line 19, in <module>
[2023-02-06T23:22:11.482Z]     from monai.losses import DiceLoss
[2023-02-06T23:22:11.482Z]   File "/home/jenkins/agent/workspace/Monai-pip/monai/monai/losses/__init__.py", line 37, in <module>
[2023-02-06T23:22:11.482Z]     from .ssim_loss import SSIMLoss
[2023-02-06T23:22:11.482Z]   File "/home/jenkins/agent/workspace/Monai-pip/monai/monai/losses/ssim_loss.py", line 17, in <module>
[2023-02-06T23:22:11.482Z]     from monai.metrics.regression import SSIMMetric
[2023-02-06T23:22:11.482Z]   File "/home/jenkins/agent/workspace/Monai-pip/monai/monai/metrics/__init__.py", line 27, in <module>
[2023-02-06T23:22:11.482Z]     from .rocauc import ROCAUCMetric, compute_roc_auc
[2023-02-06T23:22:11.482Z]   File "/home/jenkins/agent/workspace/Monai-pip/monai/monai/metrics/rocauc.py", line 18, in <module>
[2023-02-06T23:22:11.482Z]     import numpy.typing as npt
[2023-02-06T23:22:11.482Z] ModuleNotFoundError: No module named 'numpy.typing'
```

tests with numpy 1.19.2, base image 21.02
New type hint in `Inferer` implies additional restriction
The new type hint added to `Inferer` is not implying the same design as before and is implicitly adding additional restriction on `Inferer` base class which may not be desirable. It suggests that the output of the network and the output of `Inferer` call should have the same type, which might not expected as the `Inferer` should be allowed to produce results (via post-processing for instance) different than the network's output.
https://github.com/Project-MONAI/MONAI/blob/385ac0bb6612f88f92e0862a6aee033fb8df5de7/monai/inferers/inferer.py#L51



Discussion:
@Shadow-Devil @wyli @Nic-Ma @ericspod 
I am very sorry that my type annotations were not correct. I just looked through all currently implemended subclasses and found this relationship. To what should it be changed? Replace both `T`s with `Any`s?
No problem, @Shadow-Devil! You are right that the current implementation of the concrete classes follow that pattern but I don't see the need to restrict it. Returning `Any` makes more sense to me. The network outputs on the other hand are limited to torch.Tensor or sequence or dict of torch.Tensors.

@Nic-Ma on a side note, maybe we should revisit the `Inferer` call signature to include other inputs too. Currently, the input can only be torch Tensor while in the case of whole slide images, a string for filename or an image object might be preferable but not allowed here. What do you think?
Hi @drbeh ,

I think we have the `*args: Any, **kwargs: Any` in the signature?

Thanks.

I'm fine with either `Any` or the current dev branch approach, just wanted to mention that when we use the inferer as an input parameter, the consistent `Callable[..., T] -> T` seems to be more robust if the user wants to tune the inference method 
https://github.com/Project-MONAI/MONAI/blob/385ac0bb6612f88f92e0862a6aee033fb8df5de7/monai/engines/trainer.py#L147

and in the case of postprocessing types that are different than the network definitions can also be achieved with a thin wrapper, for example https://github.com/Project-MONAI/MONAI/blob/fdd07f36ecb91cfcd491533f4792e1a67a9f89fc/monai/apps/detection/utils/predict_utils.py#L86-L88
> 

Hi @Nic-Ma, you are right but I am referring to the `inputs` value here:
https://github.com/Project-MONAI/MONAI/blob/385ac0bb6612f88f92e0862a6aee033fb8df5de7/monai/inferers/inferer.py#L51
The inputs is not necessarily what we are feeding to the network, e.g., for WSI, the inputs can be `(filename, location)`, which is uses to extract a torch.Tensr and being fed to the network.
> I'm fine with either `Any` or the current dev branch approach, just wanted to mention that when we use the inferer as an input parameter, the consistent `Callable[..., T] -> T` seems to be more robust if the user wants to tune the inference method
> 
> https://github.com/Project-MONAI/MONAI/blob/385ac0bb6612f88f92e0862a6aee033fb8df5de7/monai/engines/trainer.py#L147
> 
> and in the case of postprocessing types that are different than the network definitions can also be achieved with a thin wrapper, for example
> 
> https://github.com/Project-MONAI/MONAI/blob/fdd07f36ecb91cfcd491533f4792e1a67a9f89fc/monai/apps/detection/utils/predict_utils.py#L86-L88

@wyli I agree that is a viable option that people can create wrapper around any class in MONAI.
My point is that it restricts the `Inferer` so, in this *new* design, a developer is not allowed to create a concrete implementation of `Inferer` which returns something that have a different type than the network output. It is absolutely fine with me if we think that we should do this but the point is that design changes should not go silently.
Given the entire module `inferer.py` implicitly follows `Callable[..., T] -> T` over the past three years, to me it's quite reasonable to amend the type annotations to be explicit. if you have a new feature request, we can also discuss it. 
I agree and as I said, it is fine with me to add a restriction to `Inferer` but we should be explicit about design changes and not *accidentally* change it to something that might be fine or not. The good practice is to consider any unintended design change as a bug and fix it by either reverting it or doing an explicit review.

If people agree with this design change, we can close this issue (agree: 👍 disagree: 👎).

OLD:
https://github.com/Project-MONAI/MONAI/blob/a77de4adff49037ffb24ea2cae217fac40302181/monai/inferers/inferer.py#L50

NEW:
https://github.com/Project-MONAI/MONAI/blob/385ac0bb6612f88f92e0862a6aee033fb8df5de7/monai/inferers/inferer.py#L51

@wyli @Nic-Ma @ericspod 
Sorry to draw this out but I've had a chance to look at it and I'd say `T` should be `Any` in the signature. I can see later inferrers being defined which use networks returning tuples of values and producing a result through sliding window that combines those so the result is a single tensor. 

@Shadow-Devil @wyli @Nic-Ma @ericspod 
I am very sorry that my type annotations were not correct. I just looked through all currently implemended subclasses and found this relationship. To what should it be changed? Replace both `T`s with `Any`s?
No problem, @Shadow-Devil! You are right that the current implementation of the concrete classes follow that pattern but I don't see the need to restrict it. Returning `Any` makes more sense to me. The network outputs on the other hand are limited to torch.Tensor or sequence or dict of torch.Tensors.

@Nic-Ma on a side note, maybe we should revisit the `Inferer` call signature to include other inputs too. Currently, the input can only be torch Tensor while in the case of whole slide images, a string for filename or an image object might be preferable but not allowed here. What do you think?
Hi @drbeh ,

I think we have the `*args: Any, **kwargs: Any` in the signature?

Thanks.

I'm fine with either `Any` or the current dev branch approach, just wanted to mention that when we use the inferer as an input parameter, the consistent `Callable[..., T] -> T` seems to be more robust if the user wants to tune the inference method 
https://github.com/Project-MONAI/MONAI/blob/385ac0bb6612f88f92e0862a6aee033fb8df5de7/monai/engines/trainer.py#L147

and in the case of postprocessing types that are different than the network definitions can also be achieved with a thin wrapper, for example https://github.com/Project-MONAI/MONAI/blob/fdd07f36ecb91cfcd491533f4792e1a67a9f89fc/monai/apps/detection/utils/predict_utils.py#L86-L88
> 

Hi @Nic-Ma, you are right but I am referring to the `inputs` value here:
https://github.com/Project-MONAI/MONAI/blob/385ac0bb6612f88f92e0862a6aee033fb8df5de7/monai/inferers/inferer.py#L51
The inputs is not necessarily what we are feeding to the network, e.g., for WSI, the inputs can be `(filename, location)`, which is uses to extract a torch.Tensr and being fed to the network.
> I'm fine with either `Any` or the current dev branch approach, just wanted to mention that when we use the inferer as an input parameter, the consistent `Callable[..., T] -> T` seems to be more robust if the user wants to tune the inference method
> 
> https://github.com/Project-MONAI/MONAI/blob/385ac0bb6612f88f92e0862a6aee033fb8df5de7/monai/engines/trainer.py#L147
> 
> and in the case of postprocessing types that are different than the network definitions can also be achieved with a thin wrapper, for example
> 
> https://github.com/Project-MONAI/MONAI/blob/fdd07f36ecb91cfcd491533f4792e1a67a9f89fc/monai/apps/detection/utils/predict_utils.py#L86-L88

@wyli I agree that is a viable option that people can create wrapper around any class in MONAI.
My point is that it restricts the `Inferer` so, in this *new* design, a developer is not allowed to create a concrete implementation of `Inferer` which returns something that have a different type than the network output. It is absolutely fine with me if we think that we should do this but the point is that design changes should not go silently.
Given the entire module `inferer.py` implicitly follows `Callable[..., T] -> T` over the past three years, to me it's quite reasonable to amend the type annotations to be explicit. if you have a new feature request, we can also discuss it. 
I agree and as I said, it is fine with me to add a restriction to `Inferer` but we should be explicit about design changes and not *accidentally* change it to something that might be fine or not. The good practice is to consider any unintended design change as a bug and fix it by either reverting it or doing an explicit review.

If people agree with this design change, we can close this issue (agree: 👍 disagree: 👎).

OLD:
https://github.com/Project-MONAI/MONAI/blob/a77de4adff49037ffb24ea2cae217fac40302181/monai/inferers/inferer.py#L50

NEW:
https://github.com/Project-MONAI/MONAI/blob/385ac0bb6612f88f92e0862a6aee033fb8df5de7/monai/inferers/inferer.py#L51

@wyli @Nic-Ma @ericspod 
Sorry to draw this out but I've had a chance to look at it and I'd say `T` should be `Any` in the signature. I can see later inferrers being defined which use networks returning tuples of values and producing a result through sliding window that combines those so the result is a single tensor. 

Resolving patch:
diff --git a/monai/inferers/inferer.py b/monai/inferers/inferer.py
--- a/monai/inferers/inferer.py
+++ b/monai/inferers/inferer.py
@@ -13,7 +13,8 @@
 
 import warnings
 from abc import ABC, abstractmethod
-from typing import Any, Callable, Dict, Generic, Mapping, Sequence, TypeVar, Union
+from collections.abc import Callable, Mapping, Sequence
+from typing import Any
 
 import torch
 import torch.nn as nn
@@ -24,10 +25,8 @@
 
 __all__ = ["Inferer", "SimpleInferer", "SlidingWindowInferer", "SaliencyInferer", "SliceInferer"]
 
-T = TypeVar("T")
 
-
-class Inferer(ABC, Generic[T]):
+class Inferer(ABC):
     """
     A base class for model inference.
     Extend this class to support operations during inference, e.g. a sliding window method.
@@ -48,7 +47,7 @@ class Inferer(ABC, Generic[T]):
     """
 
     @abstractmethod
-    def __call__(self, inputs: torch.Tensor, network: Callable[..., T], *args: Any, **kwargs: Any) -> T:
+    def __call__(self, inputs: torch.Tensor, network: Callable, *args: Any, **kwargs: Any) -> Any:
         """
         Run inference on `inputs` with the `network` model.
 
@@ -65,7 +64,7 @@ def __call__(self, inputs: torch.Tensor, network: Callable[..., T], *args: Any,
         raise NotImplementedError(f"Subclass {self.__class__.__name__} must implement this method.")
 
 
-class SimpleInferer(Inferer[torch.Tensor]):
+class SimpleInferer(Inferer):
     """
     SimpleInferer is the normal inference method that run model forward() directly.
     Usage example can be found in the :py:class:`monai.inferers.Inferer` base class.
@@ -91,7 +90,7 @@ def __call__(
         return network(inputs, *args, **kwargs)
 
 
-class SlidingWindowInferer(Inferer[Union[torch.Tensor, Sequence[torch.Tensor], Dict[Any, torch.Tensor]]]):
+class SlidingWindowInferer(Inferer):
     """
     Sliding window method for model inference,
     with `sw_batch_size` windows for every model.forward().


Test patch: