DataLoader gives:stack expects each tensor to be equal size,due to different image has different objects number

username: AlexLuya
cooked: <p>Hello,I am preparing dataset for a detection task:</p>
<pre><code class="lang-auto">def __getitem__(self, idx):
    ......
   return from_numpy(image), from_numpy(bboxes)
</code></pre>
<p>As you can see,the second element of the return is bboxes,and different image will have different number of objects in it,so the shape of the bboxes will vary from each other,and this will cause Dataloder throws out an exception like:</p>
<p><code>RuntimeError: stack expects each tensor to be equal size, but got [1, 1, 5] at entry 0 and [1, 5, 5] at entry 1</code></p>
<p>when it tries to stack bboxes(label) in a batch,question is</p>
<p><code>what is general paradigm to handle this issue?</code></p>

username: mariosasko
cooked: <p>This discussion may be helpful.</p><aside class="quote" data-post="6" data-topic="87211">
  <div class="title">
    <div class="quote-controls"></div>
    <img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/chetan_patil/48/25122_2.png" class="avatar">
    <a href="https://discuss.pytorch.org/t/runtimeerror-stack-expects-each-tensor-to-be-equal-size-but-got-3-224-224-at-entry-0-and-3-224-336-at-entry-3/87211/6">RuntimeError: stack expects each tensor to be equal size, but got [3, 224, 224] at entry 0 and [3, 224, 336] at entry 3</a> <a class="badge-category__wrapper " href="/c/vision/5"><span data-category-id="5" data-drop-close="true" class="badge-category " title="Topics related to either pytorch/vision or vision research related topics"><span class="badge-category__name">vision</span></span></a>
  </div>
  <blockquote>
    So you mean to say, even if the transforms.Resize() is included in transforms.Compose(), there is still an error of size mismatch ?
  </blockquote>
</aside>
<p>
Usually, in such situations some sort of padding has to be introduced for each batch so all elements match in size. This can be achieved by defining your own <code>collate_fn</code> that is then passed to <code>DataLoader</code> as an argument.</p>

username: AlexLuya
cooked: <p>Thanks,I may haven’t express may question clearly,the size(width/height) of every transformed image is same,and the problem caused from that the numbers of object in each image in the same batch are different,not the height or width of images</p>

username: mariosasko
cooked: <p>The simplest solution to that is to define the <code>collate_fn</code> like this:</p>
<pre><code class="lang-auto">def collate_fn(batch):
    return tuple(zip(*batch))
</code></pre>
<p>And this is the for loop:</p>
<pre><code class="lang-python">for image_batch, bbox_batch in dataloader: 
    ...
</code></pre>
<p>Then, inside the loop you can stack the images before passing them to the model, etc.</p>
<p>If you need some additional functionality, you can even define your own object for handling batches (in the docs there is an example how to do this).</p>

username: Tomash
cooked: <p>I don’t get it. I have the same problem like OP had. I am loading data like this:</p>
<blockquote>
<p>my_loader = DataLoader(my_dataset, batch_size=128, shuffle=True)</p>
</blockquote>
<p>How should I use that collate_fn function (what is the “batch” argument? )? What should I do in the for loop?</p>

username: mariosasko
cooked: <p>By default, <code>Dataloader</code> tries to stack the tensors to form a batch (calls <code>torch.stack</code> on the current batch), but it fails if the tensors are not of equal size. With the <code>collate_fn</code> it is possible to override this behavior and define your own “stacking procedure”. In the example above, the batch arg contains a list of instances (an image-bbox pair; the batch arg is of type <code>List[Tuple[Image, Bbox]]</code>) and with  <code>tuple(zip(*batch))</code> we  form a batch, where batch[0] corresponds to the images, and batch[1]  to the bboxes in the batch.</p>

username: Tomash
cooked: <p>Ok, so I have a function:</p>
<blockquote>
<pre><code>def collate_fn(data):
    img, bbox = data
    zipped = zip(img, bbox)
    return zipped
</code></pre>
</blockquote>
<p>Where <code>data</code> is object of class based on <code>Dataset</code> like:</p>
<blockquote>
<p>…<br>
def <strong>getitem</strong>(self, idx):<br>
img = self.imgs[idx]<br>
bbox = self.bboxs[idx]<br>
return (<br>
img,<br>
bbox,<br>
)</p>
</blockquote>
<p>And how I am supposed to use it?<br>
When I do like this:</p>
<blockquote>
<p>my_loader = DataLoader(data, batch_size=8, shuffle=True,  collate_fn=collate_fn(data))</p>
</blockquote>
<p>there is an error that: <code>zip object is not callable</code></p>
<p>What am I doing wrong?</p>

username: mariosasko
cooked: <p>First consume the iterator returned by <code>zip</code> in the collate function:</p>
<pre><code class="lang-python">def collate_fn(data):
    img, bbox = data
    zipped = zip(img, bbox)
    return list(zipped)
</code></pre>
<p>The error is self-explantory. <code>collate_fn</code> has to be callable or in layman’s terms a function.</p>
<pre><code class="lang-python">my_loader = DataLoader(data, batch_size=8, shuffle=True, collate_fn=collate_fn)
</code></pre>

username: Mahdi_Farhadi
cooked: <p>I had same problem , but my targets were dict (bounding box in detection) and solved with a similar function :</p>
<pre><code class="lang-auto">def collate_fn(data):
    zipped = zip(data)
    return list(zipped)
</code></pre>

username: iliasslasri
cooked: <p>but what’s the default behaviour, does the DataLoader just skip the element ?</p>