Type error in ConcatDataset when using a BatchSampler

Brizar: <p>I’m using <code>ConcatDataset</code> to combine multiple (custom) <code>torch.Dataset</code> instances. Then I want to sample batches from the concatenation. Because I use a custom data structure I want to use the <code>__getitems__</code>-call, and not the single getitem for every sample and collate to combine. The following code shows my attempt, moving along the solutions of multiple other posts here in this forum.</p>
<pre data-code-wrap="python"><code class="lang-python">import torch
from torch.utils.data import BatchSampler, ConcatDataset, DataLoader, Dataset, SequentialSampler

DS_SIZE = 2
BATCH_SIZE = 5


class MyDataset(Dataset):
    def __init__(self, start: int) -&gt; None:
        assert start &gt;= 0
        self.data = torch.arange(start=start, end=start + DS_SIZE)

    def __len__(self) -&gt; int:
        return len(self.data)

    def __getitem__(self, idx: int) -&gt; torch.Tensor:
        print(f"single - idx: {idx}")
        return self.data[idx]

    def __getitems__(self, indices: list[int]) -&gt; torch.Tensor:
        print(f"batch - indices: {indices}")
        return self.data[indices]


if __name__ == "__main__":
    dataset = ConcatDataset([MyDataset(start=DS_SIZE * i) for i in range(10)])

    dl1 = DataLoader(dataset=dataset, batch_size=BATCH_SIZE)

    for batch in dl1:
        print(f"1: {batch}")

    dl2 = DataLoader(
        dataset=dataset,
        sampler=BatchSampler(SequentialSampler(dataset), batch_size=BATCH_SIZE, drop_last=False),
        batch_size=None,
    )

    for batch in dl2:
        print(f"2: {batch}")
</code></pre>
<p>As expected does the first DataLoader use the <code>__getitem__</code> and a (custom) collate route. But the second DataLoader fails due to the ConcatDataset trying to compare the batched list indices against a number without checking the type of the index first. See <a href="https://github.com/pytorch/pytorch/blob/2a99e6f299714f707e84e19cb6210d4ab4ae5149/torch/utils/data/dataset.py#L337" rel="noopener nofollow ugc">the code here</a>.  The step before that, in the fetch-call, it seems like a <code>possibly_batched_index</code> is still expected, but not so in ConcatDataset. Is this a Bug or am I doing something wrong?</p>
<p>Python: 3.10<br>
PyTorch: 2.2.0+cu121</p>
<p>The error message:</p>
<pre data-code-wrap="bash"><code class="lang-bash">  File ... 
    for batch in dl2:
  File ".../venv/lib/python3.10/site-packages/torch/utils/data/dataloader.py", line 631, in __next__
    data = self._next_data()
  File ".../venv/lib/python3.10/site-packages/torch/utils/data/dataloader.py", line 675, in _next_data
    data = self._dataset_fetcher.fetch(index)  # may raise StopIteration
  File ".../venv/lib/python3.10/site-packages/torch/utils/data/_utils/fetch.py", line 53, in fetch
    data = self.dataset[possibly_batched_index]
  File ".../venv/lib/python3.10/site-packages/torch/utils/data/dataset.py", line 326, in __getitem__
    if idx &lt; 0:
TypeError: '&lt;' not supported between instances of 'list' and 'int'
</code></pre>

akorneychuk: <p>The same for me! Unable to use ConcatDataset with BatchSamles. The same error occurs</p>