Not displaying loaded images

Surekha_Gaikwad: <p>I am trying to load images from folder using dataloader but its giving me below error. Not sure why its happening…I am completely novice in pytorch…please suggest what is the potential solution for this.</p>
<p>folder structure:-<br>
./data/Disguised/Set/<br>
./data/Original/Set/<br>
All images are present under this hierarchy in jpg format.</p>
<p>O</p>
<pre><code class="lang-auto">SError: Traceback (most recent call last):
  File "/anaconda3/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 138, in _worker_loop
  File "/anaconda3/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 138, in &lt;listcomp&gt;
  File "/anaconda3/lib/python3.7/site-packages/torchvision/datasets/folder.py", line 132, in __getitem__
    sample = self.loader(path)
  File "/anaconda3/lib/python3.7/site-packages/torchvision/datasets/folder.py", line 178, in default_loader
    return pil_loader(path)
  File "/anaconda3/lib/python3.7/site-packages/torchvision/datasets/folder.py", line 160, in pil_loader
    img = Image.open(f)
  File "/anaconda3/lib/python3.7/site-packages/PIL/Image.py", line 2687, in open
OSError: cannot identify image file &lt;_io.BufferedReader name='./data/Disguised/set/Marisa_Tomei_h_003.jpg'&gt;
</code></pre>

ptrblck: <p>This error seems to be related to <code>PIL</code>.<br>
Is this particular image still opened in another process or did you just write to the file without a flush?<br>
Also, did you (accidentally) imported <code>Image</code> instead of <code>from PIL import Image</code>?</p>

Surekha_Gaikwad: <p>I am not using PIL for importing images in my code.<br>
No I havent opened the image in any other process I am just using dataloader</p>

Surekha_Gaikwad: <p>Below is the code…</p>
<pre><code class="lang-auto">transform = transforms.Compose([
        transforms.Resize(32),
        transforms.ToTensor(),
        transforms.Normalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5))])
 
dataset = datasets.ImageFolder(root="./data/", transform=transform)
dataloader = torch.utils.data.DataLoader(dataset, batch_size=32, shuffle=True, num_workers=2)


# functions to show an image
def imshow(img):
    npimg = img.numpy()
    plt.imshow(np.transpose(npimg, (1, 2, 0)))

    
# get some images
dataiter = iter(dataloader)
images = dataiter.next()
</code></pre>

Surekha_Gaikwad: <p>Now getting below error after resolving the previous issue.</p>
<pre><code class="lang-auto">  File "/Users/surekhagaikwad/Documents/COMP8420_Ass2/cnn_draft.py", line 37, in &lt;module&gt;
    images = dataiter.next()

  File "/anaconda3/lib/python3.6/site-packages/torch/utils/data/dataloader.py", line 582, in __next__
    return self._process_next_batch(batch)

  File "/anaconda3/lib/python3.6/site-packages/torch/utils/data/dataloader.py", line 608, in _process_next_batch
    raise batch.exc_type(batch.exc_msg)

RuntimeError: Traceback (most recent call last):
  File "/anaconda3/lib/python3.6/site-packages/torch/utils/data/_utils/worker.py", line 99, in _worker_loop
    samples = collate_fn([dataset[i] for i in batch_indices])
  File "/anaconda3/lib/python3.6/site-packages/torch/utils/data/_utils/collate.py", line 68, in default_collate
    return [default_collate(samples) for samples in transposed]
  File "/anaconda3/lib/python3.6/site-packages/torch/utils/data/_utils/collate.py", line 68, in &lt;listcomp&gt;
    return [default_collate(samples) for samples in transposed]
  File "/anaconda3/lib/python3.6/site-packages/torch/utils/data/_utils/collate.py", line 43, in default_collate
    return torch.stack(batch, 0, out=out)
RuntimeError: invalid argument 0: Sizes of tensors must match except in dimension 0. Got 48 and 42 in dimension 2 at /Users/distiller/project/conda/conda-bld/pytorch_1556653492823/work/aten/src/TH/generic/THTensor.cpp:711
</code></pre>

ptrblck: <p>It looks like the <code>Resize</code> is not being applied.<br>
Could you set <code>batch_size=1</code> and print the shape of your data?</p>

Surekha_Gaikwad: <p>yes I did that and it worked but could not understand why?</p>

ptrblck: <p>Because the size mismatch error is not thrown anymore if you are not trying to stack the tensors.<br>
That is why the shape information would be interesting to see. <img src="https://discuss.pytorch.org/images/emoji/apple/wink.png?v=9" title=":wink:" class="emoji" alt=":wink:"></p>

Surekha_Gaikwad: <p>but I could not understand the logic behind it</p>

ptrblck: <p>It’s just a debugging step to see the shapes of your data.<br>
The error seems to be thrown, since the spatial size of your image tensors differ, although you are using <code>Resize</code> in your transformation.<br>
If you run the code with <code>batch_size=1</code>, you could print the shape using:</p>
<pre><code class="lang-python">for data, target in loader:
    print(data.shape)
</code></pre>
<p>so that we can debug this issue further.</p>

Surekha_Gaikwad: <p>hmm thanks <a class="mention" href="/u/ptrblck">@ptrblck</a> <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=9" title=":slight_smile:" class="emoji" alt=":slight_smile:"></p>

ptrblck: <p>We would still need the output shapes! <img src="https://discuss.pytorch.org/images/emoji/apple/wink.png?v=9" title=":wink:" class="emoji" alt=":wink:"><br>
So could you please post them here?</p>
<p>Setting the <code>batch_size=1</code> is just a way to debug your code and is not the solution to this issue.</p>

Surekha_Gaikwad: <p>this is what I am getting.</p>
<pre><code class="lang-auto">torch.Size([1, 1, 48, 32])
torch.Size([1, 1, 42, 32])
</code></pre>

ptrblck: <p>Thanks for the output.<br>
Try to use <code>Resize((32, 32))</code> and run your code again with the original batch size.</p>

Surekha_Gaikwad: <p>thats kind of weird…now its showing me two images instead of one</p>

ptrblck: <p>Could you explain it a bit?<br>
Which shape does your data batch have now?</p>

Surekha_Gaikwad: <p><code>torch.Size([2, 1, 32, 32])</code></p>

Surekha_Gaikwad: <p>this is how I am setting batch_size and resize …</p>
<pre><code class="lang-auto">transform = transforms.Compose([transforms.Grayscale(num_output_channels=1),
        transforms.Resize((32,32)),
        transforms.ToTensor(),
        transforms.Normalize([0.5], [0.5])])
 
dataset = datasets.ImageFolder(root="./data/", transform=transform)
dataloader = torch.utils.data.DataLoader(dataset, batch_size=32, shuffle=True, num_workers=1)
</code></pre>

ptrblck: <p>How many images are stored in the subfolders in <code>./data/</code>?</p>

Surekha_Gaikwad: <p>as of now I have kept only two images for testing purpose</p>