Getting jfif while using data loader

username: shrutishrestha
cooked: <p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/e/c/ec5bc1fc5ce2b6e18ded22df98aa560db63d4549.png" data-download-href="https://discuss.pytorch.org/uploads/default/ec5bc1fc5ce2b6e18ded22df98aa560db63d4549" title="Screen Shot 2020-10-21 at 4.22.28 PM"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/e/c/ec5bc1fc5ce2b6e18ded22df98aa560db63d4549_2_690x222.png" alt="Screen Shot 2020-10-21 at 4.22.28 PM" data-base62-sha1="xIVgVQDCmJ6tMJ9qrAyqAXPKOO5" width="690" height="222" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/e/c/ec5bc1fc5ce2b6e18ded22df98aa560db63d4549_2_690x222.png, https://discuss.pytorch.org/uploads/default/optimized/3X/e/c/ec5bc1fc5ce2b6e18ded22df98aa560db63d4549_2_1035x333.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/e/c/ec5bc1fc5ce2b6e18ded22df98aa560db63d4549_2_1380x444.png 2x" data-dominant-color="ECECEC"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Screen Shot 2020-10-21 at 4.22.28 PM</span><span class="informations">2310×744 126 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>I am getting this error while using a data loader for the validation dataset. What can be the cause of this?</p>

username: AnimeshSinha1309
cooked: <p>Can you please add the code that’s causing the error.</p>
<p>It seems like the error is caused by the default collate function, which is responsible for taking the lists that you input and batching them into torch tensors. I would presume that ‘jfif’ is a key in your DataSet class which is missing in some of the objects, which the DataLoader is picking up on.</p>
<p>So the code that takes the dataset class and constructs a DataLoader out of it will be very helpful.</p>

username: shrutishrestha
cooked: <p>this is the place where dataset is transformed</p>
<p>def get_dataset(opts):<br>
“”" Dataset And Augmentation<br>
“”"<br>
train_path = os.path.join(opts.data_root, ‘new_data/trainset’)<br>
val_path = os.path.join(opts.data_root, ‘new_data/valset’)</p>
<pre><code># preprocess dataset
train_images = sorted(glob.glob(os.path.join(train_path, "images", "*.jpg")))
train_segs = sorted(glob.glob(os.path.join(train_path, "masks", "*.jpg")))
val_images = sorted(glob.glob(os.path.join(val_path, "images", "*.jpg")))
val_segs = sorted(glob.glob(os.path.join(val_path, "masks", "*.jpg")))
train_files = [{"image": img, "label": seg} for img, seg in zip(train_images, train_segs)]
val_files = [{"image": img, "label": seg} for img, seg in zip(val_images, val_segs)]
print("train_files", len(train_files))
print("val_files", len(val_files))

# for normalizing
mean, std = get_imagenet_mean_std()

train_transform = monai.transforms.Compose(
    [
        LoadPNGd(keys=["image", "label"]),
        AsChannelFirstd(keys=["image", "label"], channel_dim=-1),
        Lambdad(keys=['label'], func=lambda x: x[0:1] * 0.2125 + x[1:2] * 0.7154 + x[2:3] * 0.0721),
        Resized(["image", "label"], spatial_size=(256, 256)),
        ScaleIntensityd(keys=["image", "label"], minv=0.0, maxv=1.0),
        RandAdjustContrastd(keys=["image"], prob=0.3),
        RandGaussianNoised(keys=["image"], prob=0.5),
        RandRotate90d(keys=["image", "label"], prob=0.5),
        RandFlipd(keys=["image", "label"], prob=0.5),
        RandRotated(keys=["image", "label"], range_x=180, range_y=180, prob=0.5),
        RandZoomd(keys=["image", "label"], prob=0.2, min_zoom=1, max_zoom=2),
        RandAffined(keys=["image", "label"], prob=0.5),
        Rand2DElasticd(keys=["image", "label"], magnitude_range=(0, 1), spacing=(0.3, 0.3), prob=0.5),
        NormalizeIntensityd(keys=["image"], subtrahend=mean, divisor=std),
        ToTensord(keys=["image", "label"]),
        Lambdad(keys=['label'], func=lambda x: (x &gt; 0).float())
    ]
)

val_transform = monai.transforms.Compose(
    [
        LoadPNGd(keys=["image", "label"]),
        AsChannelFirstd(keys=["image", "label"], channel_dim=-1),
        Lambdad(keys=['label'], func=lambda x: x[0:1] * 0.2125 + x[1:2] * 0.7154 + x[2:3] * 0.0721),
        Resized(["image", "label"], spatial_size=(256, 256)),
        ScaleIntensityd(keys=["image", "label"], minv=0.0, maxv=1.0),
        NormalizeIntensityd(keys=["image"], subtrahend=mean, divisor=std),
        ToTensord(keys=["image", "label"])
    ]
)

# transforms
training_dataset = monai.data.PersistentDataset(train_files, transform=train_transform)
val_dataset = monai.data.Dataset(val_files, transform=val_transform)
return training_dataset, val_dataset
</code></pre>
<p>This is where data loader takes the dataset<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/e/c/ec601c9c3cadc2c6eff599466ca09f6e60ad898e.png" data-download-href="https://discuss.pytorch.org/uploads/default/ec601c9c3cadc2c6eff599466ca09f6e60ad898e" title="Screen Shot 2020-10-21 at 5.08.07 PM"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/e/c/ec601c9c3cadc2c6eff599466ca09f6e60ad898e_2_690x125.png" alt="Screen Shot 2020-10-21 at 5.08.07 PM" data-base62-sha1="xJ4Bif2tDTB3nERAkpj0KUksJNs" width="690" height="125" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/e/c/ec601c9c3cadc2c6eff599466ca09f6e60ad898e_2_690x125.png, https://discuss.pytorch.org/uploads/default/optimized/3X/e/c/ec601c9c3cadc2c6eff599466ca09f6e60ad898e_2_1035x187.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/e/c/ec601c9c3cadc2c6eff599466ca09f6e60ad898e_2_1380x250.png 2x" data-dominant-color="EEEEEF"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Screen Shot 2020-10-21 at 5.08.07 PM</span><span class="informations">1740×316 36.5 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>