RuntimeError: each element in list of batch should be of equal size | Image Caption Generator | Kaggle

tavishjain: <p>I am building an Image Caption Generator using the Flickr30k dataset on Kaggle, and am getting this message.</p>
<p>Here are the Dataset and DataLoader classes. Can someone help me how to fix this ?</p>
<pre><code class="lang-auto">class ImageCaptionDataset(Dataset):
    def __init__(self, data, word_dict, transform=None):
        self.data = data
        self.word_dict = word_dict
        if transform != None:
            self.transform = transform
        else:
            self.transform = transforms.Compose([transforms.ToTensor()])
    def __len__(self):
        return len(self.data)
    def __getitem__(self, idx):
        img_address = os.path.join('../input/flickr-image-dataset/flickr30k_images/flickr30k_images/', self.data['image_name'][idx])
        img = Image.open(img_address)
        img = self.transform(img)
        img = img.numpy().astype('float32')
        answer = str(self.data[' comment'][idx]).translate(str.maketrans('', '', string.punctuation)).split()
        label = [self.word_dict[word.lower()] for word in answer]
        return img, label
</code></pre>
<pre><code class="lang-auto">transform = transforms.Compose([
            transforms.Resize((256, 256)),
            transforms.CenterCrop(224),
            transforms.RandomHorizontalFlip(),
            transforms.ToTensor()
        ])

tempdb = ImageCaptionDataset(full_data, word_dict, transform)
print('Dataset Size: ',len(tempdb))

imageCaptionLoader = DataLoader(tempdb, batch_size = 16, shuffle = True)
print('Loader Size: ', len(imageCaptionLoader))

next(iter(imageCaptionLoader))
</code></pre>
<p>Here is the stack trace:</p>
<pre><code class="lang-auto">---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
&lt;ipython-input-7-afc3a3f8cb3d&gt; in &lt;module&gt;
     12 print('Loader Size: ', len(imageCaptionLoader))
     13 
---&gt; 14 next(iter(imageCaptionLoader))

/opt/conda/lib/python3.7/site-packages/torch/utils/data/dataloader.py in __next__(self)
    361 
    362     def __next__(self):
--&gt; 363         data = self._next_data()
    364         self._num_yielded += 1
    365         if self._dataset_kind == _DatasetKind.Iterable and \

/opt/conda/lib/python3.7/site-packages/torch/utils/data/dataloader.py in _next_data(self)
    401     def _next_data(self):
    402         index = self._next_index()  # may raise StopIteration
--&gt; 403         data = self._dataset_fetcher.fetch(index)  # may raise StopIteration
    404         if self._pin_memory:
    405             data = _utils.pin_memory.pin_memory(data)

/opt/conda/lib/python3.7/site-packages/torch/utils/data/_utils/fetch.py in fetch(self, possibly_batched_index)
     45         else:
     46             data = self.dataset[possibly_batched_index]
---&gt; 47         return self.collate_fn(data)

/opt/conda/lib/python3.7/site-packages/torch/utils/data/_utils/collate.py in default_collate(batch)
     82             raise RuntimeError('each element in list of batch should be of equal size')
     83         transposed = zip(*batch)
---&gt; 84         return [default_collate(samples) for samples in transposed]
     85 
     86     raise TypeError(default_collate_err_msg_format.format(elem_type))

/opt/conda/lib/python3.7/site-packages/torch/utils/data/_utils/collate.py in &lt;listcomp&gt;(.0)
     82             raise RuntimeError('each element in list of batch should be of equal size')
     83         transposed = zip(*batch)
---&gt; 84         return [default_collate(samples) for samples in transposed]
     85 
     86     raise TypeError(default_collate_err_msg_format.format(elem_type))

/opt/conda/lib/python3.7/site-packages/torch/utils/data/_utils/collate.py in default_collate(batch)
     80         elem_size = len(next(it))
     81         if not all(len(elem) == elem_size for elem in it):
---&gt; 82             raise RuntimeError('each element in list of batch should be of equal size')
     83         transposed = zip(*batch)
     84         return [default_collate(samples) for samples in transposed]

RuntimeError: each element in list of batch should be of equal size
</code></pre>

ptrblck: <p>Could you check the shape of the <code>img</code> and <code>label</code> in the dataset and make sure they have the same size? Based on your posted code and the error message, I guess that <code>label</code> might have different lengths, so you might need to pad this target tensor to the largest length in the batch.</p>

tavishjain: <p>Ummm yes, label doesn’t have a fixed size everytime. It is an encoded version of the caption for the image. So should I make all of them the same size first ?</p>

ptrblck: <p>It depends how you would like to handle the targets during training.<br>
You could use a custom <code>collate_fn</code> as a fix for this particular error in the <code>DataLoader</code> and return a list, which contains targets of different sizes.<br>
However, I guess you would need to use a valid batch to calculate the loss during training, so you could e.g. use <a href="https://pytorch.org/docs/stable/generated/torch.nn.utils.rnn.pad_sequence.html#torch.nn.utils.rnn.pad_sequence"><code>pad_sequence</code></a> or another valid approach.</p>

jcpinto54: <p>Hello, I have the same error and every feature set and label of my dataset has the same size. Should I apply the same solution? The one with collate_fn?</p>

ptrblck: <p>No, you should not have to use a custom collate function if all samples have the same shape and I would recommend to try to narrow down where the issue is coming from.<br>
E.g. print the shape of each sample in the <code>Dataset.__getitem__</code> and check if they have indeed the same shape or if some of them are different.</p>

jcpinto54: <p>Thank you for your answer. I have found the error to be mine. The labels had indeed different sizes and they should only be an integer in my implementation. Problem solved <img src="https://discuss.pytorch.org/images/emoji/apple/+1.png?v=12" title=":+1:" class="emoji" alt=":+1:" loading="lazy" width="20" height="20"></p>

poult: <p>I don’t know why you always give the valuable answer. I am jealous.</p>