internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: Project-MONAI/MONAI

Issue:
Basis UNet
On what paper is your UNet based?
The original 2D seems to differ quite a lot from the 3D U-Net by Özgün Çiçek which I'd like to use. 

Thanks.


Discussion:

Resolving patch:
diff --git a/monai/networks/nets/unet.py b/monai/networks/nets/unet.py
--- a/monai/networks/nets/unet.py
+++ b/monai/networks/nets/unet.py
@@ -38,6 +38,11 @@ def __init__(
         dropout=0,
     ) -> None:
         """
+        Enhanced version of UNet which has residual units implemented with the ResidualUnit class.
+        The residual part uses a convolution to change the input dimensions to match the output dimensions
+        if this is necessary but will use nn.Identity if not.
+        Refer to: https://link.springer.com/chapter/10.1007/978-3-030-12029-0_40.
+
         Args:
             dimensions: number of spatial dimensions.
             in_channels: number of input channels.


Test patch:
Repository: Project-MONAI/MONAI

Issue:
CI errors with python 3.9
**Describe the bug**
some of the CI pipelines by default uses the latest python which is now Python 3.9
this breaks the tests because pytorch py3.9 wheel is not fully available yet.

**To Reproduce**
please see the log https://github.com/Project-MONAI/MONAI/runs/1253216926?check_suite_focus=true#step:8:14


Discussion:

Resolving patch:
diff --git a/monai/transforms/utility/array.py b/monai/transforms/utility/array.py
--- a/monai/transforms/utility/array.py
+++ b/monai/transforms/utility/array.py
@@ -424,19 +424,16 @@ class LabelToMask(Transform):
 
     """
 
-    def __init__(
+    def __init__(  # pytype: disable=annotation-type-mismatch
         self,
         select_labels: Union[Sequence[int], int],
         merge_channels: bool = False,
-    ) -> None:  # pytype: disable=annotation-type-mismatch # pytype bug with bool
+    ) -> None:
         self.select_labels = ensure_tuple(select_labels)
         self.merge_channels = merge_channels
 
     def __call__(
-        self,
-        img: np.ndarray,
-        select_labels: Optional[Union[Sequence[int], int]] = None,
-        merge_channels: Optional[bool] = None,
+        self, img: np.ndarray, select_labels: Optional[Union[Sequence[int], int]] = None, merge_channels: bool = False
     ) -> np.ndarray:
         """
         Args:
@@ -450,15 +447,13 @@ def __call__(
             select_labels = self.select_labels
         else:
             select_labels = ensure_tuple(select_labels)
-        if merge_channels is None:
-            merge_channels = self.merge_channels
 
         if img.shape[0] > 1:
             data = img[[*select_labels]]
         else:
             data = np.where(np.in1d(img, select_labels), True, False).reshape(img.shape)
 
-        return np.any(data, axis=0, keepdims=True) if merge_channels else data
+        return np.any(data, axis=0, keepdims=True) if (merge_channels or self.merge_channels) else data
 
 
 class FgBgToIndices(Transform):
diff --git a/monai/transforms/utility/dictionary.py b/monai/transforms/utility/dictionary.py
--- a/monai/transforms/utility/dictionary.py
+++ b/monai/transforms/utility/dictionary.py
@@ -506,14 +506,14 @@ class LabelToMaskd(MapTransform):
 
     """
 
-    def __init__(
+    def __init__(  # pytype: disable=annotation-type-mismatch
         self,
         keys: KeysCollection,
         select_labels: Union[Sequence[int], int],
         merge_channels: bool = False,
-    ) -> None:  # pytype: disable=annotation-type-mismatch # pytype bug with bool
+    ) -> None:
         super().__init__(keys)
-        self.converter = LabelToMask(select_labels, merge_channels)
+        self.converter = LabelToMask(select_labels=select_labels, merge_channels=merge_channels)
 
     def __call__(self, data: Mapping[Hashable, np.ndarray]) -> Dict[Hashable, np.ndarray]:
         d = dict(data)


Test patch:
Repository: Project-MONAI/MONAI

Issue:
AHNet can't support input channels > 1
**Describe the bug**
I found that the AHNet implementation hard-code input channels to 1 for the first conv layer, need to support multi-channels.



Discussion:

Resolving patch:
diff --git a/monai/networks/nets/ahnet.py b/monai/networks/nets/ahnet.py
--- a/monai/networks/nets/ahnet.py
+++ b/monai/networks/nets/ahnet.py
@@ -354,6 +354,7 @@ class AHNet(nn.Module):
     Args:
         layers: number of residual blocks for 4 layers of the network (layer1...layer4). Defaults to ``(3, 4, 6, 3)``.
         spatial_dims: spatial dimension of the input data. Defaults to 3.
+        in_channels: number of input channels for the network. Default to 1.
         out_channels: number of output channels for the network. Defaults to 1.
         upsample_mode: [``"transpose"``, ``"bilinear"``, ``"trilinear"``]
             The mode of upsampling manipulations.
@@ -368,6 +369,7 @@ def __init__(
         self,
         layers: tuple = (3, 4, 6, 3),
         spatial_dims: int = 3,
+        in_channels: int = 1,
         out_channels: int = 1,
         upsample_mode: str = "transpose",
     ):
@@ -393,7 +395,7 @@ def __init__(
         assert spatial_dims == 2 or spatial_dims == 3, "spatial_dims can only be 2 or 3."
 
         self.conv1 = conv_type(
-            1,
+            in_channels,
             64,
             kernel_size=(7, 7, 3)[-spatial_dims:],
             stride=(2, 2, 1)[-spatial_dims:],
@@ -526,7 +528,8 @@ def copy_from(self, net):
         p2d, p3d = next(net.conv1.parameters()), next(self.conv1.parameters())
 
         # From 64x3x7x7 -> 64x3x7x7x1 -> 64x1x7x7x3
-        p3d.data = p2d.data.unsqueeze(dim=4).permute(0, 4, 2, 3, 1).clone()
+        weights = p2d.data.unsqueeze(dim=4).permute(0, 4, 2, 3, 1).clone()
+        p3d.data = weights.repeat([1, p3d.shape[1], 1, 1, 1])
 
         # Copy the initial module BN0
         copy_bn_param(net.bn0, self.bn0)


Test patch:
diff --git a/tests/test_ahnet.py b/tests/test_ahnet.py
--- a/tests/test_ahnet.py
+++ b/tests/test_ahnet.py
@@ -92,6 +92,12 @@
     (2, 2, 128, 128, 64),
     {"out_channels": 1, "upsample_mode": "bilinear"},
 ]
+TEST_CASE_AHNET_3D_WITH_PRETRAIN_3 = [
+    {"spatial_dims": 3, "upsample_mode": "transpose", "in_channels": 2, "out_channels": 3},
+    torch.randn(2, 2, 128, 128, 64),
+    (2, 3, 128, 128, 64),
+    {"out_channels": 1, "upsample_mode": "bilinear"},
+]
 
 
 class TestFCN(unittest.TestCase):
@@ -135,7 +141,13 @@ def test_ahnet_shape(self, input_param, input_data, expected_shape):
 
 
 class TestAHNETWithPretrain(unittest.TestCase):
-    @parameterized.expand([TEST_CASE_AHNET_3D_WITH_PRETRAIN_1, TEST_CASE_AHNET_3D_WITH_PRETRAIN_2])
+    @parameterized.expand(
+        [
+            TEST_CASE_AHNET_3D_WITH_PRETRAIN_1,
+            TEST_CASE_AHNET_3D_WITH_PRETRAIN_2,
+            TEST_CASE_AHNET_3D_WITH_PRETRAIN_3,
+        ]
+    )
     @skip_if_quick
     def test_ahnet_shape(self, input_param, input_data, expected_shape, fcn_input_param):
         net = AHNet(**input_param)
Repository: Project-MONAI/MONAI

Issue:
Can`t load JIT model from .trcd file
I saved 3D DensNet from monai:
```python
traced = torch.jit.trace(model, torch.rand(1, 3, 160, 160, 160).cuda())
torch.jit.save(traced, f'monai3_160_baseline.trcd')
```


Now I am trying to load it from trcd file with the following function:

```torch.jit.load(x, map_location="cuda:0")```

And I get the following error:
```python
---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
<ipython-input-28-60944cabc718> in <module>()
----> 1 model = load_jit_model("./model_training/monai3_160_baseline.trcd").cuda()

<ipython-input-22-2020de3a15ce> in load_jit_model(x, cuda_id)
     14 
     15 def load_jit_model(x, cuda_id=0):
---> 16     return torch.jit.load(x, map_location=f'cuda:{cuda_id}' if torch.cuda.is_available() else 'cpu')

/home/yeliseev/anaconda3/envs/rsna/lib/python3.7/site-packages/torch/jit/__init__.py in load(f, map_location, _extra_files)
    273     cu = torch._C.CompilationUnit()
    274     if isinstance(f, str) or isinstance(f, pathlib.Path):
--> 275         cpp_module = torch._C.import_ir_module(cu, f, map_location, _extra_files)
    276     else:
    277         cpp_module = torch._C.import_ir_module_from_buffer(cu, f.read(), map_location, _extra_files)

RuntimeError: expected ident but found 'class' here:
Serialized   File "code/__torch__/torch/nn/modules/container/___torch_mangle_423.py", line 8
  norm : __torch__.torch.nn.modules.pooling.AdaptiveAvgPool3d
  flatten : __torch__.torch.nn.modules.flatten.Flatten
  class : __torch__.torch.nn.modules.linear.Linear
  ~~~~~ <--- HERE
  def forward(self: __torch__.torch.nn.modules.container.___torch_mangle_423.Sequential,
    argument_1: Tensor) -> Tensor:
```


Discussion:

Resolving patch:
diff --git a/monai/networks/nets/densenet.py b/monai/networks/nets/densenet.py
--- a/monai/networks/nets/densenet.py
+++ b/monai/networks/nets/densenet.py
@@ -174,7 +174,7 @@ def __init__(
                     ("relu", nn.ReLU(inplace=True)),
                     ("norm", avg_pool_type(1)),
                     ("flatten", nn.Flatten(1)),
-                    ("class", nn.Linear(in_channels, out_channels)),
+                    ("out", nn.Linear(in_channels, out_channels)),
                 ]
             )
         )


Test patch:
Repository: Project-MONAI/MONAI

Issue:
Add pretrained weights parameter for AHNet
**Is your feature request related to a problem? Please describe.**
Currently, if users want to use pretrained weights for AHNet, they need to create a `FCN` object and a `AHNet`, then manually call `copy_from()` to initialize weights, which is not easy to use as `DenseNet` and other networks.

**Describe the solution you'd like**
Need to add a `pretrained` parameter for `AHNet` directly.



Discussion:

Resolving patch:
diff --git a/monai/networks/nets/ahnet.py b/monai/networks/nets/ahnet.py
--- a/monai/networks/nets/ahnet.py
+++ b/monai/networks/nets/ahnet.py
@@ -16,6 +16,7 @@
 import torch.nn as nn
 import torch.nn.functional as F
 
+from monai.networks.blocks import FCN
 from monai.networks.layers.factories import Act, Conv, Norm, Pool
 
 
@@ -363,6 +364,8 @@ class AHNet(nn.Module):
             - ``"transpose"``, uses transposed convolution layers.
             - ``"bilinear"``, uses bilinear interpolate.
             - ``"trilinear"``, uses trilinear interpolate.
+        pretrained: whether to load pretrained weights from ResNet50 to initialize convolution layers, default to False.
+        progress: If True, displays a progress bar of the download of pretrained weights to stderr.
     """
 
     def __init__(
@@ -372,6 +375,8 @@ def __init__(
         in_channels: int = 1,
         out_channels: int = 1,
         upsample_mode: str = "transpose",
+        pretrained: bool = False,
+        progress: bool = True,
     ):
         self.inplanes = 64
         super(AHNet, self).__init__()
@@ -458,6 +463,10 @@ def __init__(
                 m.weight.data.fill_(1)
                 m.bias.data.zero_()
 
+        if pretrained:
+            net2d = FCN(pretrained=True, progress=progress)
+            self.copy_from(net2d)
+
     def _make_layer(
         self,
         block: Type[Bottleneck3x3x1],


Test patch:
diff --git a/tests/test_ahnet.py b/tests/test_ahnet.py
--- a/tests/test_ahnet.py
+++ b/tests/test_ahnet.py
@@ -158,6 +158,20 @@ def test_ahnet_shape(self, input_param, input_data, expected_shape, fcn_input_pa
             result = net.forward(input_data)
             self.assertEqual(result.shape, expected_shape)
 
+    def test_initialize_pretrained(self):
+        net = AHNet(
+            spatial_dims=3,
+            upsample_mode="transpose",
+            in_channels=2,
+            out_channels=3,
+            pretrained=True,
+            progress=True,
+        )
+        input_data = torch.randn(2, 2, 128, 128, 64)
+        with torch.no_grad():
+            result = net.forward(input_data)
+            self.assertEqual(result.shape, (2, 3, 128, 128, 64))
+
 
 if __name__ == "__main__":
     unittest.main()
Repository: Project-MONAI/MONAI

Issue:
CopyItemsd transform can't support tensor from model
**Describe the bug**
Thanks for @madil90 's bug report, `CopyItemsd` transform can work on normal `torch.Tensor` data but can't work on a tensor in graph:
```
ERROR - Exception: Applying transform <monai.transforms.compose.Compose object at 0x7f0700e9deb8>.
Traceback (most recent call last):
  File "/home/madil/dlmed/code/MONAI/monai/transforms/utils.py", line 309, in apply_transform
    return transform(data)
  File "/home/madil/dlmed/code/MONAI/monai/transforms/utility/dictionary.py", line 406, in __call__
    d[new_key] = copy.deepcopy(d[key])
  File "/opt/conda/lib/python3.6/copy.py", line 161, in deepcopy
    y = copier(memo)
  File "/opt/conda/lib/python3.6/site-packages/torch/tensor.py", line 38, in __deepcopy__
    raise RuntimeError("Only Tensors created explicitly by the user "
RuntimeError: Only Tensors created explicitly by the user (graph leaves) support the deepcopy protocol at the moment
```
Need to fix it for post transforms.



Discussion:

Resolving patch:
diff --git a/monai/transforms/utility/dictionary.py b/monai/transforms/utility/dictionary.py
--- a/monai/transforms/utility/dictionary.py
+++ b/monai/transforms/utility/dictionary.py
@@ -403,7 +403,10 @@ def __call__(self, data):
         for key, new_key in zip(self.keys * self.times, self.names):
             if new_key in d:
                 raise KeyError(f"Key {new_key} already exists in data.")
-            d[new_key] = copy.deepcopy(d[key])
+            if type(d[key]) == torch.Tensor:
+                d[new_key] = d[key].detach().clone()
+            else:
+                d[new_key] = copy.deepcopy(d[key])
         return d
 
 


Test patch:
diff --git a/tests/test_copy_itemsd.py b/tests/test_copy_itemsd.py
--- a/tests/test_copy_itemsd.py
+++ b/tests/test_copy_itemsd.py
@@ -58,6 +58,19 @@ def test_array_values(self):
         np.testing.assert_allclose(result["img"], [[0, 1], [1, 2]])
         np.testing.assert_allclose(result["img_1"], [[1, 1], [1, 2]])
 
+    def test_graph_tensor_values(self):
+        device = torch.device("cuda:0") if torch.cuda.is_available() else torch.device("cpu:0")
+        net = torch.nn.PReLU().to(device)
+        net.eval()
+        with torch.no_grad():
+            pred = net(torch.tensor([[0.0, 1.0], [1.0, 2.0]], device=device))
+        input_data = {"pred": pred, "seg": torch.tensor([[0.0, 1.0], [1.0, 2.0]], device=device)}
+        result = CopyItemsd(keys="pred", times=1, names="pred_1")(input_data)
+        self.assertTrue("pred_1" in result)
+        result["pred_1"] += 1.0
+        torch.testing.assert_allclose(result["pred"], torch.tensor([[0.0, 1.0], [1.0, 2.0]], device=device))
+        torch.testing.assert_allclose(result["pred_1"], torch.tensor([[1.0, 2.0], [2.0, 3.0]], device=device))
+
 
 if __name__ == "__main__":
     unittest.main()
Repository: Project-MONAI/MONAI

Issue:
Can't save classification result if "info" is empty in meta_data
**Describe the bug**
A clear and concise description of what the bug is.
Thanks @madil90 for raising this issue.
Most of the PNG images don't have `info` field in the metadata, and we saved an empty {} in metadata.
But PyTorch `collate_fn` can't collate a batch of {} into `"info": [{}, {}, {}, ...]`, just a `info: {}`:
```
{'filename_or_obj': ['/workspace/data/CXR/PLCO/PLCO_256_original/AB02720121407113025_v2.png'], 'spatial_shape': [tensor([256]), tensor([256])], 'format': ['PNG'], 'mode': ['I'], 'width': tensor([256]), 'height': tensor([256]), 'info': {}}
```
So we can't load meta_data correctly in `ClassificationSaver` handler with index `i`:
```
  File "/home/madil/dlmed/code/MONAI/monai/handlers/classification_saver.py", line 82, in __call__
    self.saver.save_batch(engine_output, meta_data)
  File "/home/madil/dlmed/code/MONAI/monai/data/csv_saver.py", line 91, in save_batch
    self.save(data, {k: meta_data[k][i] for k in meta_data} if meta_data else None)
  File "/home/madil/dlmed/code/MONAI/monai/data/csv_saver.py", line 91, in <dictcomp>
    self.save(data, {k: meta_data[k][i] for k in meta_data} if meta_data else None)
KeyError: 0
```



Discussion:

Resolving patch:
diff --git a/monai/data/image_reader.py b/monai/data/image_reader.py
--- a/monai/data/image_reader.py
+++ b/monai/data/image_reader.py
@@ -579,7 +579,6 @@ def _get_meta_dict(self, img) -> Dict:
         meta["mode"] = img.mode
         meta["width"] = img.width
         meta["height"] = img.height
-        meta["info"] = img.info
         return meta
 
     def _get_spatial_shape(self, img) -> Sequence:
diff --git a/monai/transforms/io/array.py b/monai/transforms/io/array.py
--- a/monai/transforms/io/array.py
+++ b/monai/transforms/io/array.py
@@ -233,7 +233,6 @@ def __call__(self, filename: Union[Sequence[Union[Path, str]], Path, str]):
             meta["mode"] = img.mode
             meta["width"] = img.width
             meta["height"] = img.height
-            meta["info"] = img.info
             if not compatible_meta:
                 compatible_meta = meta
             else:


Test patch:
Repository: Project-MONAI/MONAI

Issue:
Missing up_modules in AHNet when TorchScript
**Describe the bug**
When exporting model in TorchScript, it output error:
```
RuntimeError: 
Module 'PSP' has no attribute 'up_modules' :
  File "/workspace/data/medical/MONAI/monai/networks/nets/ahnet.py", line 290
        if self.upsample_mode == "transpose":
            for (project_module, pool_module, up_module) in zip(
                self.project_modules, self.pool_modules, self.up_modules
                                                         ~~~~~~~~~~~~~~~ <--- HERE
            ):
                output = up_module(project_module(pool_module(x)))
```
I will fix it ASAP.


Discussion:

Resolving patch:
diff --git a/monai/networks/nets/ahnet.py b/monai/networks/nets/ahnet.py
--- a/monai/networks/nets/ahnet.py
+++ b/monai/networks/nets/ahnet.py
@@ -243,7 +243,7 @@ def forward(self, x):
 class PSP(nn.Module):
     def __init__(self, spatial_dims: int, psp_block_num: int, in_ch: int, upsample_mode: str = "transpose"):
         super(PSP, self).__init__()
-
+        self.up_modules = nn.ModuleList()
         conv_type = Conv[Conv.CONV, spatial_dims]
         pool_type: Type[Union[nn.MaxPool2d, nn.MaxPool3d]] = Pool[Pool.MAX, spatial_dims]
 
@@ -269,7 +269,6 @@ def __init__(self, spatial_dims: int, psp_block_num: int, in_ch: int, upsample_m
 
         if self.upsample_mode == "transpose":
             conv_trans_type = Conv[Conv.CONVTRANS, spatial_dims]
-            self.up_modules = nn.ModuleList()
             for i in range(psp_block_num):
                 size = (2 ** (i + 3), 2 ** (i + 3), 1)[-spatial_dims:]
                 pad_size = (2 ** (i + 3), 2 ** (i + 3), 0)[-spatial_dims:]


Test patch:
Repository: Project-MONAI/MONAI

Issue:
c++/cuda extention source files not included in build if in nested directory
The build process will currently fail to discover c++/cuda source which is more than a single nested directory deep in the csrc folder. 

The issue is that while we use glob with ** to find the source files, we must add the recursive=True flag to the glob call. 

This has been fixed just need to make the PR.


Discussion:

Resolving patch:
diff --git a/setup.py b/setup.py
--- a/setup.py
+++ b/setup.py
@@ -86,15 +86,14 @@ def get_extensions():
     ext_dir = os.path.join(this_dir, "monai", "csrc")
     include_dirs = [ext_dir]
 
-    main_src = glob.glob(os.path.join(ext_dir, "*.cpp"))
-    source_cpu = glob.glob(os.path.join(ext_dir, "**", "*.cpp"))
-    source_cuda = glob.glob(os.path.join(ext_dir, "**", "*.cu"))
+    source_cpu = glob.glob(os.path.join(ext_dir, "**", "*.cpp"), recursive=True)
+    source_cuda = glob.glob(os.path.join(ext_dir, "**", "*.cu"), recursive=True)
 
     extension = None
     define_macros = [(f"{torch_parallel_backend()}", 1), ("MONAI_TORCH_VERSION", TORCH_VERSION)]
     extra_compile_args = {}
     extra_link_args = []
-    sources = main_src + source_cpu
+    sources = source_cpu
     if BUILD_CPP:
         extension = CppExtension
         extra_compile_args.setdefault("cxx", [])


Test patch:
Repository: Project-MONAI/MONAI

Issue:
ScanningWindowInferer accumulates gradient data and inference fails with out of memory
**Bug Description**
The documentation does not explain that the MONAI inferers accumulate backpropagation information with each additional batch. The name "inferer" inherently implies no gradient information should be stored.

This took a few hours of detailed debugging to find the root cause, and creates a difficult user experience.

**To Reproduce**
Write a simple inference program using `ScanningWindowInferer` where the inference is performed on GPU. On CPU gradient info accumulation drastically slows down inference speed.

**Expected behavior**
The documentation should state that the inferers should be wrapped with `torch.no_grad()`, or the current inferer classes should be renamed with `ScanningWindowForwardPropagator` and wrapped with `ScanningWindowInferer` which internally uses `torch.no_grad()`.




Discussion:
Hi @aihsani ,

Thanks for your feedback, I will enhance the document to make it more clear.


Resolving patch:
diff --git a/monai/inferers/inferer.py b/monai/inferers/inferer.py
--- a/monai/inferers/inferer.py
+++ b/monai/inferers/inferer.py
@@ -24,6 +24,19 @@ class Inferer(ABC):
     """
     A base class for model inference.
     Extend this class to support operations during inference, e.g. a sliding window method.
+
+    Example code::
+
+        device = torch.device("cuda:0")
+        data = ToTensor()(LoadImage()(filename=img_path)).to(device)
+        model = UNet(...).to(device)
+        inferer = SlidingWindowInferer(...)
+
+        model.eval()
+        with torch.no_grad():
+            pred = inferer(inputs=data, network=model)
+        ...
+
     """
 
     @abstractmethod
@@ -53,6 +66,7 @@ def __call__(
 class SimpleInferer(Inferer):
     """
     SimpleInferer is the normal inference method that run model forward() directly.
+    Usage example can be found in the :py:class:`monai.inferers.Inferer` base class.
 
     """
 
@@ -83,6 +97,7 @@ class SlidingWindowInferer(Inferer):
     """
     Sliding window method for model inference,
     with `sw_batch_size` windows for every model.forward().
+    Usage example can be found in the :py:class:`monai.inferers.Inferer` base class.
 
     Args:
         roi_size: the window size to execute SlidingWindow evaluation.


Test patch: