RuntimeError: stack expects each tensor to be equal size, but got [1, 1024, 1024] at entry 0 and [4, 1024, 1024] at entry 1 Collate error on the key 'fixed_image' of dictionary data. MONAI hint: if your transforms intentionally create images of different

username: Hassan_Ghavidel
cooked: <p>I am doing a project about image registration. received this error:</p>
<p>Epoch 0/99:  33%|████████████████████████████████████████████████████████████████████████▎                                                                                                                                                | 5/15 [00:04&lt;00:05,  1.74batch/s]&gt; collate dict key “fixed_image” out of 2 keys</p>
<blockquote>
<blockquote>
<p>collate/stack a list of tensors<br>
E: stack expects each tensor to be equal size, but got [1, 1024, 1024] at entry 0 and [4, 1024, 1024] at entry 1, shape [torch.Size([1, 1024, 1024]), torch.Size([4, 1024, 1024])] in collate([metatensor([[[1.8903e-02, 1.6555e-02, 1.4817e-02,  …, 1.8006e-01,<br>
1.8057e-01, 1.8318e-01],<br>
[1.7500e-02, 1.5912e-02, 1.5085e-02,  …, 1.7924e-01,<br>
1.7876e-01, 1.8112e-01],<br>
[1.7203e-02, 1.4965e-02, 1.5309e-02,  …, 1.7940e-01,<br>
1.7656e-01, 1.7810e-01],<br>
…,<br>
[5.5115e-04, 1.3683e-04, 0.0000e+00,  …, 1.6299e-01,<br>
1.6520e-01, 1.6168e-01],<br>
[1.5516e-03, 7.7386e-04, 2.7677e-05,  …, 1.6049e-01,<br>
1.6329e-01, 1.6351e-01],<br>
[2.6995e-03, 1.3090e-03, 2.9158e-04,  …, 1.6152e-01,<br>
1.6472e-01, 1.6307e-01]]]), metatensor([[[0.1033, 0.1383, 0.1528,  …, 0.1248, 0.1120, 0.0833],<br>
[0.1207, 0.1617, 0.1792,  …, 0.1439, 0.1293, 0.0963],<br>
[0.1208, 0.1617, 0.1793,  …, 0.1421, 0.1276, 0.0951],<br>
…,<br>
[0.0073, 0.0099, 0.0111,  …, 0.1347, 0.1210, 0.0900],<br>
[0.0074, 0.0100, 0.0112,  …, 0.1343, 0.1206, 0.0898],<br>
[0.0064, 0.0085, 0.0095,  …, 0.1145, 0.1028, 0.0765]],</p>
</blockquote>
</blockquote>
<pre><code>    [[0.1033, 0.1383, 0.1528,  ..., 0.1248, 0.1120, 0.0833],
     [0.1207, 0.1617, 0.1792,  ..., 0.1439, 0.1293, 0.0963],
     [0.1208, 0.1617, 0.1793,  ..., 0.1421, 0.1276, 0.0951],
     ...,
     [0.0073, 0.0099, 0.0111,  ..., 0.1347, 0.1210, 0.0900],
     [0.0074, 0.0100, 0.0112,  ..., 0.1343, 0.1206, 0.0898],
     [0.0064, 0.0085, 0.0095,  ..., 0.1145, 0.1028, 0.0765]],

    [[0.1033, 0.1383, 0.1528,  ..., 0.1248, 0.1120, 0.0833],
     [0.1207, 0.1617, 0.1792,  ..., 0.1439, 0.1293, 0.0963],
     [0.1208, 0.1617, 0.1793,  ..., 0.1421, 0.1276, 0.0951],
     ...,
     [0.0073, 0.0099, 0.0111,  ..., 0.1347, 0.1210, 0.0900],
     [0.0074, 0.0100, 0.0112,  ..., 0.1343, 0.1206, 0.0898],
     [0.0064, 0.0085, 0.0095,  ..., 0.1145, 0.1028, 0.0765]],

    [[0.1412, 0.1900, 0.2119,  ..., 0.2119, 0.1900, 0.1412],
     [0.1645, 0.2213, 0.2469,  ..., 0.2469, 0.2213, 0.1645],
     [0.1646, 0.2215, 0.2471,  ..., 0.2471, 0.2215, 0.1646],
     ...,
     [0.1646, 0.2215, 0.2471,  ..., 0.2471, 0.2215, 0.1646],
     [0.1645, 0.2213, 0.2469,  ..., 0.2469, 0.2213, 0.1645],
     [0.1412, 0.1900, 0.2119,  ..., 0.2119, 0.1900, 0.1412]]])])
</code></pre>
<blockquote>
<p>collate dict key “moving_image” out of 2 keys</p>
<blockquote>
<p>collate/stack a list of tensors<br>
Epoch 0/99:  80%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████▊                                           | 12/15 [00:06&lt;00:01,  1.77batch/s]<br>
Traceback (most recent call last):<br>
File “/project/med/Hassan_Ghavidel/transformer_target_localization/code/main_train_unsup_TransMorph.py”, line 269, in <br>
train_val_test.train_val_model_unsupervised(model, train_loader, optimizer, config.loss_name, config.loss_weights,<br>
File “/project/med/Hassan_Ghavidel/transformer_target_localization/code/auxiliary/train_val_test.py”, line 49, in train_val_model_unsupervised<br>
for train_batch_data in tepoch:<br>
File “/project/med/Hassan_Ghavidel/TransMorph_Transformer_for_Medical_Image_Registration/myenv/lib/python3.8/site-packages/tqdm/std.py”, line 1181, in <strong>iter</strong><br>
for obj in iterable:<br>
File “/project/med/Hassan_Ghavidel/TransMorph_Transformer_for_Medical_Image_Registration/myenv/lib/python3.8/site-packages/torch/utils/data/dataloader.py”, line 628, in <strong>next</strong><br>
data = self._next_data()<br>
File “/project/med/Hassan_Ghavidel/TransMorph_Transformer_for_Medical_Image_Registration/myenv/lib/python3.8/site-packages/torch/utils/data/dataloader.py”, line 1333, in _next_data<br>
return self._process_data(data)<br>
File “/project/med/Hassan_Ghavidel/TransMorph_Transformer_for_Medical_Image_Registration/myenv/lib/python3.8/site-packages/torch/utils/data/dataloader.py”, line 1359, in _process_data<br>
data.reraise()<br>
File “/project/med/Hassan_Ghavidel/TransMorph_Transformer_for_Medical_Image_Registration/myenv/lib/python3.8/site-packages/torch/_utils.py”, line 543, in reraise<br>
raise exception<br>
RuntimeError: Caught RuntimeError in DataLoader worker process 0.<br>
Original Traceback (most recent call last):<br>
File “/project/med/Hassan_Ghavidel/TransMorph_Transformer_for_Medical_Image_Registration/myenv/lib/python3.8/site-packages/monai/data/utils.py”, line 514, in list_data_collate<br>
ret[key] = collate_fn(data_for_batch)<br>
File “/project/med/Hassan_Ghavidel/TransMorph_Transformer_for_Medical_Image_Registration/myenv/lib/python3.8/site-packages/torch/utils/data/_utils/collate.py”, line 265, in default_collate<br>
return collate(batch, collate_fn_map=default_collate_fn_map)<br>
File “/project/med/Hassan_Ghavidel/TransMorph_Transformer_for_Medical_Image_Registration/myenv/lib/python3.8/site-packages/torch/utils/data/_utils/collate.py”, line 120, in collate<br>
return collate_fn_map[elem_type](batch, collate_fn_map=collate_fn_map)<br>
File “/project/med/Hassan_Ghavidel/TransMorph_Transformer_for_Medical_Image_Registration/myenv/lib/python3.8/site-packages/monai/data/utils.py”, line 458, in collate_meta_tensor_fn<br>
collated = collate_fn(batch)  # type: ignore<br>
File “/project/med/Hassan_Ghavidel/TransMorph_Transformer_for_Medical_Image_Registration/myenv/lib/python3.8/site-packages/torch/utils/data/_utils/collate.py”, line 163, in collate_tensor_fn<br>
return torch.stack(batch, 0, out=out)<br>
File “/project/med/Hassan_Ghavidel/TransMorph_Transformer_for_Medical_Image_Registration/myenv/lib/python3.8/site-packages/monai/data/meta_tensor.py”, line 282, in <strong>torch_function</strong><br>
ret = super().<strong>torch_function</strong>(func, types, args, kwargs)<br>
File “/project/med/Hassan_Ghavidel/TransMorph_Transformer_for_Medical_Image_Registration/myenv/lib/python3.8/site-packages/torch/_tensor.py”, line 1279, in <strong>torch_function</strong><br>
ret = func(*args, **kwargs)<br>
RuntimeError: stack expects each tensor to be equal size, but got [1, 1024, 1024] at entry 0 and [4, 1024, 1024] at entry 1</p>
</blockquote>
</blockquote>
<p>The above exception was the direct cause of the following exception:</p>
<p>Traceback (most recent call last):<br>
File “/project/med/Hassan_Ghavidel/TransMorph_Transformer_for_Medical_Image_Registration/myenv/lib/python3.8/site-packages/torch/utils/data/_utils/worker.py”, line 302, in _worker_loop<br>
data = fetcher.fetch(index)<br>
File “/project/med/Hassan_Ghavidel/TransMorph_Transformer_for_Medical_Image_Registration/myenv/lib/python3.8/site-packages/torch/utils/data/_utils/fetch.py”, line 61, in fetch<br>
return self.collate_fn(data)<br>
File “/project/med/Hassan_Ghavidel/TransMorph_Transformer_for_Medical_Image_Registration/myenv/lib/python3.8/site-packages/monai/data/utils.py”, line 529, in list_data_collate<br>
raise RuntimeError(re_str) from re<br>
RuntimeError: stack expects each tensor to be equal size, but got [1, 1024, 1024] at entry 0 and [4, 1024, 1024] at entry 1<br>
Collate error on the key ‘fixed_image’ of dictionary data.</p>
<p>MONAI hint: if your transforms intentionally create images of different shapes, creating your <code>DataLoader</code> with <code>collate_fn=pad_list_data_collate</code> might solve this problem (check its documentation).</p>
<p>the line that the code is complaining is (for train_batch_data in tepoch:)<br>
in train_val_test.py script located in auxiliary folder:</p>
<pre><code>    # loop over all batches of data
    with tqdm.tqdm(train_loader, unit="batch", initial=0) as tepoch:
        for train_batch_data in tepoch:
            tepoch.set_description(f"Epoch {epoch}/{epoch_start + epoch_nr - 1}")
            if wandb_usage:
                wandb.log({"lr": optimizer.param_groups[0]['lr']})                  
            # clear stored gradients
            optimizer.zero_grad()
            
            # get batch of data
            train_inputs, train_targets = train_batch_data["moving_image"].to(device), train_batch_data["fixed_image"].to(device)
</code></pre>
<p>and dataloader in main_train_unsup_TransMorph.py script are as follows:</p>
<p>train_loader = DataLoader(train_ds, batch_size=config.batch_size, shuffle=False, num_workers=4)<br>
if config.unsupervised_validation:<br>
val_loader = DataLoader(val_ds, batch_size=config.batch_size, shuffle=False, num_workers=4)<br>
else:<br>
val_loader = None<br>
if config.supervised_validation:<br>
val_loader_supervised = DataLoader(val_ds_supervised, batch_size=200, shuffle=False, num_workers=2)<br>
else:<br>
val_loader_supervised = None</p>
<p>check_data = monai.utils.first(train_loader)</p>
<h1><a name="printfshape-of-check-data-check_datafixed_imageshape-hw-1" class="anchor" href="#printfshape-of-check-data-check_datafixed_imageshape-hw-1"></a>print(f"Shape of check data: {check_data[“fixed_image”].shape}")  # (h,w)</h1>
<p>fixed_image = check_data[“fixed_image”][0][0]  # eg (2,1,224,224)<br>
moving_image = check_data[“moving_image”][0][0]</p>
<p>i dont have any clue how can i fix this issue. please help me.<br>
this is the github link of the code:</p><aside class="onebox allowlistedgeneric" data-onebox-src="https://github.com/hasan-ghavidel/transformer_target_localization">
  <header class="source">
      <img src="https://github.githubassets.com/favicons/favicon.svg" class="site-icon" width="32" height="32">

      <a href="https://github.com/hasan-ghavidel/transformer_target_localization" target="_blank" rel="noopener nofollow ugc">GitHub</a>
  </header>

  <article class="onebox-body">
    <div class="aspect-image" style="--aspect-ratio:690/344;"><img src="https://opengraph.githubassets.com/08d6f984bf9f9abea00f3790fc9b7daa0652a2ace01283d1d8268a056f5c3e21/hasan-ghavidel/transformer_target_localization" class="thumbnail" width="690" height="345"></div>

<h3><a href="https://github.com/hasan-ghavidel/transformer_target_localization" target="_blank" rel="noopener nofollow ugc">GitHub - hasan-ghavidel/transformer_target_localization</a></h3>

  <p>Contribute to hasan-ghavidel/transformer_target_localization development by creating an account on GitHub.</p>


  </article>

  <div class="onebox-metadata">
    
    
  </div>

  <div style="clear: both"></div>
</aside>
<p>
i am trying to use this datasets of chests:</p><aside class="onebox allowlistedgeneric" data-onebox-src="https://www.kaggle.com/datasets/nih-chest-xrays/data">
  <header class="source">
      <img src="https://www.kaggle.com/static/images/favicon.ico" class="site-icon" width="32" height="32">

      <a href="https://www.kaggle.com/datasets/nih-chest-xrays/data" target="_blank" rel="noopener nofollow ugc">kaggle.com</a>
  </header>

  <article class="onebox-body">
    <img width="500" height="500" src="https://storage.googleapis.com/kaggle-datasets-images/5839/8684/9b6ad772f7f3ba299b1fcb712abf3a1a/dataset-card.jpg?t=2017-12-06-00-17-18" class="thumbnail onebox-avatar">

<h3><a href="https://www.kaggle.com/datasets/nih-chest-xrays/data" target="_blank" rel="noopener nofollow ugc">NIH Chest X-rays</a></h3>

  <p>Over 112,000 Chest X-ray images from more than 30,000 unique patients</p>


  </article>

  <div class="onebox-metadata">
    
    
  </div>

  <div style="clear: both"></div>
</aside>


username: ptrblck
cooked: <aside class="quote no-group" data-username="Hassan_Ghavidel" data-post="1" data-topic="205706">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/hassan_ghavidel/48/71175_2.png" class="avatar"> Hassan_Ghavidel:</div>
<blockquote>
<p>RuntimeError: stack expects each tensor to be equal size, but got [1, 1024, 1024] at entry 0 and [4, 1024, 1024] at entry 1<br>
Collate error on the key ‘fixed_image’ of dictionary data.</p>
</blockquote>
</aside>
<p>The issue points to samples using a different shape, in your case an image with a single channel (grayscale) and another one with 4 channels (I guess RGBA).<br>
You might want to transform all images to have the same number of channels, e.g. by repeating the single grayscale channel.</p>

username: Hassan_Ghavidel
cooked: <p>thanks ptrblck</p>
<p>is there any function for that?</p>

username: ptrblck
cooked: <p>You can simply call <code>repeat</code> on the tensor inside the <code>__getitem__</code> method of your <code>Dataset</code>:</p>
<pre data-code-wrap="python"><code class="lang-python">x = torch.randn(1, 224, 224)

if x.size(0) == 1:
    x = x.repeat(4, 1, 1)
</code></pre>
<p>This code assumes that the transformed grayscale image has a single channel (the channel dimension could also be missing, so you might want to make sure it was created) and that you want to keep all 4 channels (and do not want to drop the alpha channel).</p>

username: Hassan_Ghavidel
cooked: <p>Dear ptblck</p>
<p>i solved the issue… thanks a lot for you help.<br>
add EnsureSingleChannel function to ensure all images are greyscale:</p>
<p>class EnsureSingleChannel:<br>
def <strong>init</strong>(self, keys):<br>
self.keys = keys</p>
<pre><code>def __call__(self, data):
    for key in self.keys:
        img = data[key]
        # Convert to tensor if not already
        if not isinstance(img, torch.Tensor):
            img = torch.tensor(img)
        # Ensure it has exactly 1 channel
        if img.shape[0] &gt; 1:
            img = img[0:1]  # Take only the first channel
        elif img.shape[0] &lt; 1:
            raise ValueError("Image has less than 1 channel, cannot be processed")
        data[key] = img
    return data
</code></pre>
<p>and use it like this:</p>
<p>train_transforms = Compose(<br>
[<br>
LoadImaged(keys=[“fixed_image”, “moving_image”]),<br>
# EnsureChannelFirstd(keys=(“fixed_image”, “moving_image”), channel_dim=-1),  # images have shape (270,270,1)<br>
EnsureChannelFirstd(keys=(“fixed_image”, “moving_image”)),  # images have shape ??<br>
EnsureSingleChannel(keys=[“fixed_image”, “moving_image”]),<br>
ScaleIntensityRanged(<br>
keys=[“fixed_image”, “moving_image”],<br>
a_min=0,<br>
a_max=1000,<br>
b_min=0.0,<br>
b_max=1.0,<br>
clip=False,<br>
),<br>
RandAffined(<br>
keys=[“fixed_image”, “moving_image”],<br>
mode=(“bilinear”, “bilinear”),<br>
prob=config.prob_randaffine,<br>
spatial_size=(1024, 1024),<br>
rotate_range=(0, np.pi / 15),<br>
shear_range=(0.2, 0.2),<br>
translate_range=(20,20),<br>
scale_range=(0.2, 0.2),<br>
),</p>
<p>it is running now! God bless you.</p>

username: ptrblck
cooked: <p>Cool, good to hear you’ve solved the issue.<br>
Note however, that your approach is removing the color channels from some images by keeping only the first channel (which I assume would be the RED channel). You could experiment with a proper grayscale transformation (<a href="https://pytorch.org/vision/0.9/transforms.html#torchvision.transforms.Grayscale"><code>torchvision.transforms.Grayscale</code></a>) or could also compare against my proposal of keeping the color images and to repeat the grayscale channel instead for all grayscale images.</p>

username: Hassan_Ghavidel
cooked: <p>thanks,<br>
i will try that, as those colors are useful in my results i guess…</p>