Error loading cityscape dataset

xavi0007: <p>I am trying to load the cityscape dataset leftimg8bit and gtFine, downloaded from the website.<br>
I am also using the DataLoader for city scape provided by PyTorch. However, I am getting the error " TypeError: default_collate: batch must contain tensors, numpy arrays, numbers, dicts or lists; found<br>
&lt;class ‘PIL.PngImagePlugin.PngImageFile’&gt;  " when enumerating the loaded data.</p>
<p>I have also already made sure to use transforms.ToTensor()<br>
Any idea what I did wrong here?</p>
<p>An excerpt of my code:</p>
<pre><code class="lang-auto">data_transforms_train = transforms.Compose([
        transforms.Resize(256),
        transforms.RandomResizedCrop(224),
        transforms.RandomHorizontalFlip(),
        transforms.ToTensor(),
        transforms.Normalize((0.485, 0.456, 0.406), (0.229, 0.224, 0.225)),
    ])

    data_transforms_val = transforms.Compose([
        transforms.Resize(256),
        transforms.CenterCrop(224),
        transforms.ToTensor(),
        transforms.Normalize((0.485, 0.456, 0.406), (0.229, 0.224, 0.225)),
    ])

    train_dataset = torchvision.datasets.Cityscapes(os.path.join(root_dir, 'data/cityscapes'),
        split='train', mode='fine', target_type='semantic', transform=data_transforms_train)
    val_dataset = torchvision.datasets.Cityscapes(os.path.join(root_dir, 'data/cityscapes'),
        split='val', mode='fine', target_type='semantic', transform=data_transforms_val)

    train_loader = data.DataLoader(
        train_dataset,
        batch_size=batch_size,
        num_workers=num_workers,
        pin_memory=False,
        shuffle=True
    )
    val_loader = data.DataLoader(
        val_dataset,
        batch_size=batch_size,
        num_workers=num_workers,
        pin_memory=True,
        shuffle=False
    )
</code></pre>
<blockquote>
<p>Blockquote<br>
Traceback (most recent call last):<br>
File “/Users/xavier/Programming/FishNet/main_semantic_segmentation.py”, line 271, in <br>
main()<br>
File “/Users/xavier/Programming/FishNet/main_semantic_segmentation.py”, line 259, in main<br>
train(train_loader, model, criterion, optimizer, epoch)<br>
File “/Users/xavier/Programming/FishNet/main_semantic_segmentation.py”, line 99, in train<br>
for i, data in enumerate(train_loader):<br>
File “/Users/xavier/miniforge3/lib/python3.9/site-packages/torch/utils/data/dataloader.py”, line 521, in <strong>next</strong><br>
data = self._next_data()<br>
File “/Users/xavier/miniforge3/lib/python3.9/site-packages/torch/utils/data/dataloader.py”, line 1203, in _next_data<br>
return self._process_data(data)<br>
File “/Users/xavier/miniforge3/lib/python3.9/site-packages/torch/utils/data/dataloader.py”, line 1229, in _process_data<br>
data.reraise()<br>
File “/Users/xavier/miniforge3/lib/python3.9/site-packages/torch/_utils.py”, line 425, in reraise<br>
raise self.exc_type(msg)<br>
TypeError: Caught TypeError in DataLoader worker process 0.<br>
Original Traceback (most recent call last):<br>
File “/Users/xavier/miniforge3/lib/python3.9/site-packages/torch/utils/data/_utils/worker.py”, line 287, in _worker_loop<br>
data = fetcher.fetch(index)<br>
File “/Users/xavier/miniforge3/lib/python3.9/site-packages/torch/utils/data/_utils/fetch.py”, line 47, in fetch<br>
return self.collate_fn(data)<br>
File “/Users/xavier/miniforge3/lib/python3.9/site-packages/torch/utils/data/_utils/collate.py”, line 84, in default_collate<br>
return [default_collate(samples) for samples in transposed]<br>
File “/Users/xavier/miniforge3/lib/python3.9/site-packages/torch/utils/data/_utils/collate.py”, line 84, in <br>
return [default_collate(samples) for samples in transposed]<br>
File “/Users/xavier/miniforge3/lib/python3.9/site-packages/torch/utils/data/_utils/collate.py”, line 86, in default_collate<br>
raise TypeError(default_collate_err_msg_format.format(elem_type))<br>
TypeError: default_collate: batch must contain tensors, numpy arrays, numbers, dicts or lists; found &lt;class ‘PIL.PngImagePlugin.PngImageFile’&gt;</p>
</blockquote>
<p>Thanks!</p>

techkang: <p>It is wired. I think you can try <code>print(train_dataset[0])</code> to varify the output of your dataset.</p>

xavi0007: <p>it prints out</p>
<pre><code class="lang-auto">(tensor([[[-1.0733, -1.0733, -1.0904,  ..., -0.9363, -0.3883, -0.3883],
         [-1.0733, -1.0904, -1.0904,  ..., -0.8678, -0.3541, -0.3541],
         [-1.0904, -1.0904, -1.0904,  ..., -0.8164, -0.2684, -0.2856],
         ...,
         [-0.6623, -1.0048, -0.9192,  ..., -0.5424, -0.6965, -0.6965],
         [-0.6109, -0.9363, -1.0562,  ..., -0.5938, -0.7650, -0.8849],
         [-0.5424, -0.8164, -1.1247,  ..., -0.7993, -0.8164, -0.8507]],

        [[-0.6352, -0.6527, -0.6527,  ..., -0.6877, -0.1450, -0.1099],
         [-0.6527, -0.6702, -0.6527,  ..., -0.6176, -0.0924, -0.0749],
         [-0.6527, -0.6527, -0.6702,  ..., -0.5651, -0.0224, -0.0049],
         ...,
         [-0.3025, -0.6702, -0.6702,  ..., -0.2850, -0.4426, -0.4426],
         [-0.2850, -0.6001, -0.8102,  ..., -0.3375, -0.4951, -0.6001],
         [-0.2325, -0.4776, -0.8627,  ..., -0.5476, -0.5826, -0.6001]],

        [[-0.4450, -0.4275, -0.4450,  ..., -0.6541, -0.1312, -0.0441],
         [-0.4450, -0.4450, -0.4624,  ..., -0.6018, -0.0441,  0.0605],
         [-0.4450, -0.4624, -0.4624,  ..., -0.5495,  0.0256,  0.1651],
         ...,
         [-0.3927, -0.6890, -0.6715,  ..., -0.3927, -0.4973, -0.4973],
         [-0.3578, -0.6193, -0.7761,  ..., -0.4450, -0.5670, -0.6018],
         [-0.3404, -0.5495, -0.8284,  ..., -0.5670, -0.5844, -0.6018]]]), &lt;PIL.PngImagePlugin.PngImageFile image mode=L size=2048x1024 at 0x12958A790&gt;)
</code></pre>

eqy: <p>That error does look strange, could you post how you are iterating over the dataloader? I think just a little additional code would be enough to potentially reproduce the issue.</p>

xavi0007: <p>My entire training code would be like so</p>
<pre><code class="lang-auto">def train(train_loader, model, criterion, optimizer, epoch):
    train_loss = AverageMeter()
    model.train()

    for i, (inputs, labels) in enumerate(train_loader):
     
        
        N = inputs.size(0)
        inputs = torch.FloatTensor(inputs).to(device)
        labels = torch.LongTensor(labels).to(device)

        optimizer.zero_grad()
        outputs = model(inputs)

        loss = criterion(outputs, labels.to(device)) / N
        loss.backward()
        optimizer.step()

        train_loss.update(loss.data[0], N)


    
</code></pre>

techkang: <aside class="quote no-group" data-username="xavi0007" data-post="3" data-topic="135488">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/xavi0007/48/43457_2.png" class="avatar"> xavi0007:</div>
<blockquote>
<p>it prints out</p>
</blockquote>
</aside>
<p>It is clearly that this default dataset return the input tensor and semenstic labels, but the labels is a PIL Image.</p>
<p>You can write your own dataset to process the PIL Image. For example:</p>
<pre><code class="lang-auto">class MyCitySpace(torchvision.datasets.Cityspaces):
    def __getitem__(self, item):
        inputs, target = super().__getitem__(item)
        return inputs, np.array(target)
</code></pre>

xavi0007: <p>Thanks! that works, additionally I also changed it data type to long using target.astype(np.long) due to requirements of PyTorch.</p>