Batch must contain tensors, numpy arrays, numbers, dicts or lists; found <class 'PIL.Image.Image'>

patrick126: <p>Hi everyone, I don’t know how to handle this.</p>
<p>I did</p>
<pre><code class="lang-auto">valid_and_test_set = torchvision.datasets.MNIST("./mnist", train=False, download=True)
dataset_valid, dataset_test = torch.utils.data.random_split(valid_and_test_set,[5000, 5000])


class Convert(object):
    def __call__(self, img):
        return torch.unsqueeze(torch.from_numpy(np.array(img)), 0).float()

class Flatten(object):
    def __call__(self, img):
        return img.view(28*28)

class OneHot(object):
    def __call__(self, label):
        #target = torch.zeros(10, dtype=torch.float)
        #target[label] = 1.0
        return label

transform = transforms.Compose(
    [Convert(),
     transforms.Normalize(mean=[mu], std=[std]),
     Flatten()])

dataset_valid.transform = transform
dataset_valid.target_transform = OneHot()

dataloader_valid = torch.utils.data.DataLoader(dataset_valid, batch_size=500, num_workers=num_workers, pin_memory=True)
</code></pre>
<p>But when i try</p>
<pre><code class="lang-auto"> for batch in dataloader_valid:
</code></pre>
<p>I get the erorr</p>
<pre><code class="lang-auto">TypeError: default_collate: batch must contain tensors, numpy arrays, numbers, dicts or lists; found &lt;class 'PIL.Image.Image'&gt;
</code></pre>
<p>Thank you guys.</p>

cskarthik7: <p>In your transform, add transforms.ToTensor() before transforms.Normalize()</p>

Nikronic: <p>Hi,<br>
Here is the problem. <code>MNIST</code> object does not have <code>transform</code> and <code>target_transform</code>. You need to call <code>dataset</code> attribute of <code>MNIST</code> then call <code>transform</code> etc.</p>
<aside class="quote no-group" data-username="patrick126" data-post="1" data-topic="85669">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/p/838e76/48.png" class="avatar"> patrick126:</div>
<blockquote>
<pre><code class="lang-auto">dataset_valid.transform = transform
dataset_valid.target_transform = OneHot()
</code></pre>
</blockquote>
</aside>
<p>Solution:</p>
<pre><code class="lang-auto">dataset_valid.dataset.transform = transform
dataset_valid.dataset.target_transform = OneHot()
</code></pre>
<p>Bests</p>

patrick126: <p>Thanks, this resolved the problem.<br>
But now i get another error<br>
On</p>
<pre><code class="lang-auto">accuracy_valid = 100. * predictions.eq(dataset_valid.dataset.targets).sum().float() / len(dataset_valid)
</code></pre>
<p>I get</p>
<pre><code class="lang-auto">RuntimeError: The size of tensor a (5000) must match the size of tensor b (10000) at non-singleton dimension 0
</code></pre>
<p>To check i did</p>
<pre><code class="lang-auto">print ("\nTrain data are %d, with shape %s" % (len(dataset_train), dataset_train.data.shape))
</code></pre>
<p>That prints</p>
<pre><code class="lang-auto">Validation data are 5000, with shape torch.Size([10000, 28, 28])
</code></pre>
<p>I am very confused, I would be grateful if you could help me.</p>

Nikronic: <p>Here you using <code>dataset_valid</code>, but you are printing shape of <code>dataset_train</code>.</p>
<aside class="quote no-group" data-username="patrick126" data-post="4" data-topic="85669">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/p/838e76/48.png" class="avatar"> patrick126:</div>
<blockquote>
<p><code>accuracy_valid = 100. * predictions.eq(dataset_valid.dataset.targets).sum().float() / len(dataset_valid)</code></p>
</blockquote>
</aside>
<p>And also, it seems for <code>predictions</code>, you have used <code>batch_size=5000</code> which lead to 5000 predictions but you are using labels of entire dataset which is 10000.<br>
And I think you are not looping over batches and want to feed entire dataset to network, in this case change <code>batch_size= dataset_valid.dataset.data.shape[0]</code> in this line:</p>
<aside class="quote no-group" data-username="patrick126" data-post="1" data-topic="85669">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/p/838e76/48.png" class="avatar"> patrick126:</div>
<blockquote>
<p><code>dataloader_valid = torch.utils.data.DataLoader(dataset_valid, batch_size=500, num_workers=num_workers, pin_memory=True)</code></p>
</blockquote>
</aside>

patrick126: <p>Yes sorry, I coiped the wrong line but the output I posted was from dataset_valid.</p>
<blockquote>
<p>And also, it seems for  <code>predictions</code> , you have used  <code>batch_size=5000</code>  which lead to 5000 predictions but you are using labels of entire dataset which is 10000.</p>
</blockquote>
<p>So, If I’ve understood well, random_split splits the input data but not the labels, right?<br>
In this case, what should i do?</p>
<p>Sorry to bother you with maybe dumb questions, I’m new at PyTorch.</p>

Nikronic: <p>No, <code>random_split</code> is doing fine. The problem is the way you obtained the <code>predictions</code>. Can you share the code that generated those predictions?</p>
<aside class="quote no-group" data-username="patrick126" data-post="4" data-topic="85669">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/p/838e76/48.png" class="avatar"> patrick126:</div>
<blockquote>
<p>dataset_valid.dataset.targets</p>
</blockquote>
</aside>
<p>Here you need to only pass targets with same size of batch.</p>
<p>For instance here is simple snippet:</p>
<pre><code class="lang-auto">for data, target in dataloader_valid:
    # this is one batch in every iteration of loop
    predictions = model(data)  # size [batch_size, ...]
    # now targets are also size [batch_size, ...]
 
</code></pre>
<p>But you are not using batch for labels, you passing entire labels out of loop</p>
<aside class="quote no-group" data-username="patrick126" data-post="4" data-topic="85669">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/p/838e76/48.png" class="avatar"> patrick126:</div>
<blockquote>
<p><code>dataset_valid.dataset.targets</code></p>
</blockquote>
</aside>
<p>This is not a batch of data, it is your whole dataset.</p>

patrick126: <p>Here is where i defined predictions (it is a multi-class classifier):</p>
<pre><code class="lang-auto">def test(dataset, dataloader):

    # switch to test mode
    net.eval()  

    # initialize predictions
    predictions = torch.zeros(len(dataset), dtype=torch.int64)
    sample_counter = 0

    # do not accumulate gradients (faster)
    with torch.no_grad():

        # test all batches
        for batch in dataloader:

            # get data from dataloader [ignore labels/targets as they are not used in test mode]
            inputs = batch[0]

            # move data to device
            inputs = inputs.to(device, non_blocking=True)

            # forward pass
            outputs = net(inputs)

            # store predictions
            outputs_max = torch.argmax(outputs, dim=1)
            for output in outputs_max:
                predictions[sample_counter] = output
                sample_counter += 1

    return predictions
</code></pre>
<p>I tought it was a problem with random_split becouse this problem has rised since I used that function.<br>
Previosly I was using the original test set from MNIST with the same code and had no issue.<br>
Thanks for helping.</p>

Nikronic: <aside class="quote no-group" data-username="patrick126" data-post="8" data-topic="85669">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/p/838e76/48.png" class="avatar"> patrick126:</div>
<blockquote>
<pre><code class="lang-auto">for output in outputs_max:
                predictions[sample_counter] = output
                sample_counter += 1
</code></pre>
</blockquote>
</aside>
<p>You are assigning the label of each sample by looping over each prediction in each batch. This is not optimal at all and although you are using batches but at each iteration, you are using <code>predictions</code> to have all predicted labels. But still it should be fine and I cannot see any issue regarding size mismatch, maybe a mistake in passing arguments to methods?</p>
<p>This will do the same job:</p>
<pre><code class="lang-auto">dataloader_valid = torch.utils.data.DataLoader(dataset_valid, 
                                               batch_size=len(dataset_valid),
                                               num_workers=0,
                                               pin_memory=True)

def test(dataset, dataloader):

    # switch to test mode
    net.eval()  

    # do not accumulate gradients (faster)
    with torch.no_grad():

        # test all batches
        for batch in dataloader:

            # get data from dataloader [ignore labels/targets as they are not used in test mode]
            inputs = batch[0]

            # move data to device
            inputs = inputs.to(device, non_blocking=True)

            # forward pass
            outputs = net(inputs)

            # store predictions
            predictions = torch.argmax(outputs, dim=1)

    return predictions
</code></pre>

patrick126: <p>Oh, I didn’t know it, thank you.<br>
I arleady posted everything involved in the line where I get the error.<br>
The problem is that <code>dataset_valid.dataset.targets</code> has a lenght of 10000, but I can’t understand why since i used <code>random_split</code> with a lenght of 5000</p>

Nikronic: <p>Ow, I am sorry, I misread one of your posts.</p>
<aside class="quote no-group" data-username="patrick126" data-post="10" data-topic="85669">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/p/838e76/48.png" class="avatar"> patrick126:</div>
<blockquote>
<p>dataset_valid.dataset.targets</p>
</blockquote>
</aside>
<p>Is not giving you the subsets you created by <code>random_split</code>. <code>.dataset</code> gives you original dataset and for using transforms you had to use it.</p>
<p>In this part of code, you just need to get the subset instead of whole dataset:</p>
<aside class="quote no-group" data-username="patrick126" data-post="4" data-topic="85669">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/p/838e76/48.png" class="avatar"> patrick126:</div>
<blockquote>
<p><code>accuracy_valid = 100. * predictions.eq(dataset_valid.dataset.targets).sum().float() / len(dataset_valid)</code></p>
</blockquote>
</aside>
<p>Fix:</p>
<pre><code class="lang-auto">accuracy_valid = 100. * predictions.eq(dataset_valid.dataset.targets[dataset_valid.indices]).sum().float() / len(dataset_valid)
</code></pre>
<p>More explanation would be when you use <code>random_split</code>, it does not create two objects that have data, it just creates a list of indices that are randomly generated from original dataset.<br>
So for instance, you can obtain original dataset by calling <code>dataset_valid.dataset</code> which is equal to first line of your code in first post. and it also have another attribute <code>dataset_valid.indices</code> which contains 5000 indices (as you provided) of randomly selected items in dataset.</p>

patrick126: <p>Ah ok, thank you I’ve finally understood.</p>