TypeError when concatenating datasets and shuffling

username: PavelCz
cooked: <p>Hello all,</p>
<p>I have noticed some to me unexpected behaviour when using <code>ConcatDataset</code>. I managed to reproduce it with the minimal example provided below.<br>
When <code>shuffle=True</code> is removed the script runs fine without any problems. Furthermore, the prints show that the sizes of all the elements in the concatenated dataset are the same.</p>
<pre><code class="lang-auto">import torch
from torch.utils.data import TensorDataset, DataLoader, ConcatDataset
import torchvision
from torchvision import transforms

nb_samples = 100
features = torch.randn(nb_samples, 1, 28, 28)
labels = torch.empty(nb_samples, dtype=torch.long).random_(10)

dataset = TensorDataset(features, labels)
DATASET_DIR = '../../data/datasets'

transform = transforms.Compose([transforms.ToTensor()])

mnist = torchvision.datasets.MNIST(root=DATASET_DIR, train=True, download=True, transform=transform)

set = ConcatDataset([dataset, mnist])

loader = DataLoader(
    set,
    batch_size=10, shuffle=True
)
for batch_idx, (x, y) in enumerate(loader):
    print(x.shape, y.shape)

</code></pre>
<p>Now if I include <code>shuffle=True</code> I get the a TypeError with the following stacktrace:</p>
<pre><code class="lang-auto">Traceback (most recent call last):
  File "test.py", line 23, in &lt;module&gt;
    for batch_idx, (x, y) in enumerate(loader):
  File "/Users/MyUser/opt/anaconda3/envs/conda-env-name/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 345, in __next__
    data = self._next_data()
  File "/Users/MyUser/opt/anaconda3/envs/conda-env-name/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 385, in _next_data
    data = self._dataset_fetcher.fetch(index)  # may raise StopIteration
  File "/Users/MyUser/opt/anaconda3/envs/conda-env-name/lib/python3.8/site-packages/torch/utils/data/_utils/fetch.py", line 47, in fetch
    return self.collate_fn(data)
  File "/Users/MyUser/opt/anaconda3/envs/conda-env-name/lib/python3.8/site-packages/torch/utils/data/_utils/collate.py", line 79, in default_collate
    return [default_collate(samples) for samples in transposed]
  File "/Users/MyUser/opt/anaconda3/envs/conda-env-name/lib/python3.8/site-packages/torch/utils/data/_utils/collate.py", line 79, in &lt;listcomp&gt;
    return [default_collate(samples) for samples in transposed]
  File "/Users/MyUser/opt/anaconda3/envs/conda-env-name/lib/python3.8/site-packages/torch/utils/data/_utils/collate.py", line 55, in default_collate
    return torch.stack(batch, 0, out=out)
TypeError: expected Tensor as element 1 in argument 0, but got int
</code></pre>
<p>Is this expected behaviour, i.e. is it somehow not possible to use shuffling with the ConcatDataset?<br>
With some debugging I did, it looks like some batches are transformed into tuples of values, where some are scalar tensors and some values are just int, the reason not being quite clear to me.</p>
<p>According to <code>python -c "import torch; print(torch.__version__)"</code> I am on pytorch version 1.5.0</p>
<p>Help with this would be greatly appreciated.</p>

username: ptrblck
cooked: <p>It seems that <a href="https://github.com/pytorch/vision/blob/42aa9b263eaea3553369ff39268cfe295c7a57b0/torchvision/datasets/mnist.py#L90">this line of code</a> is causing the error, as the target will be transformed to an <code>int</code>.</p>
<p>As a workaround you could pass a <code>target_transform</code> and transform the target back to a tensor via:</p>
<pre><code class="lang-python">mnist = torchvision.datasets.MNIST(
    root=DATASET_DIR, train=True, download=True, transform=transform,
    target_transform=transforms.Lambda(lambda y: torch.tensor(y)))
</code></pre>

username: PavelCz
cooked: <p>Thanks for having a look ptrblck!<br>
And thank you for the workaround, that indeed solved my problem.<br>
I didn’t consider putting a transform on the target.</p>