Error while resizing image via tranforms

kirk86: <p>Could someone tell me what’s wong here? I suspect that the resize functionality is not upscaling the images properly but I’m not 100%.</p>
<pre><code class="lang-auto">dataset = 'FashionMNIST'                                                                                                                                                                     
datapath = './data'                                                                                                                                                                          
ds = getattr(torchvision.datasets, dataset)                                                                                                                                                  
transforms = torchvision.transforms.Compose([                                                                                                                                                
            torchvision.transforms.Resize((32, 32)),                                                                                                                                                   
            torchvision.transforms.ToTensor(),                                                                                                                                               
 ])                                                                                                                                                                                           
train_set = ds(root=datapath, train=True, download=True, transform=transforms)                                                                                                               
train_set.data.unsqueeze_(1)                                                                                                                                                                 
train_set.data = train_set.data.repeat(1, 3, 1, 1)                                                                                                                                           
                                                                                                                                                                                             
dummy_data = torch.utils.data.DataLoader(train_set, batch_size=24, shuffle=True, num_workers=4, pin_memory=True)                                                                             
                                                                                                                                                                                             
x, y = next(iter(dummy_data))
</code></pre>
<p>I keep getting the following error</p>
<pre><code class="lang-auto">Original Traceback (most recent call last):
  File "/home/kirk/miniconda3/envs/torch/lib/python3.6/site-packages/torch/utils/data/_utils/worker.py", line 178, in _worker_loop
    data = fetcher.fetch(index)
  File "/home/kirk/miniconda3/envs/torch/lib/python3.6/site-packages/torch/utils/data/_utils/fetch.py", line 44, in fetch
    data = [self.dataset[idx] for idx in possibly_batched_index]
  File "/home/kirk/miniconda3/envs/torch/lib/python3.6/site-packages/torch/utils/data/_utils/fetch.py", line 44, in &lt;listcomp&gt;
    data = [self.dataset[idx] for idx in possibly_batched_index]
  File "/home/kirk/miniconda3/envs/torch/lib/python3.6/site-packages/torchvision/datasets/mnist.py", line 92, in __getitem__
    img = Image.fromarray(img.numpy(), mode='L')
  File "/home/kirk/miniconda3/envs/torch/lib/python3.6/site-packages/PIL/Image.py", line 2661, in fromarray
    raise ValueError("Too many dimensions: %d &gt; %d." % (ndim, ndmax))
ValueError: Too many dimensions: 3 &gt; 2.
</code></pre>

vainaijr: <p>I think if you comment these two lines and then print train_set data size</p>
<pre><code class="lang-auto"># train_set.data.unsqueeze_(1)                                                                                                                                                                 
# train_set.data = train_set.data.repeat(1, 3, 1, 1)
print(train_set.data.size())
</code></pre>
<p>gives</p>
<pre><code class="lang-auto">torch.Size([60000, 28, 28])
</code></pre>
<p>then if you</p>
<pre><code class="lang-auto">print(x.shape)
</code></pre>
<p>this gives</p>
<pre><code class="lang-auto">torch.Size([24, 1, 32, 32])
</code></pre>
<p>fashion mnist dataset has 1 channel images of size 28x28, which we resized to 32x32.<br>
Resize has no issue, it works as it should.<br>
I think if you want 3x32x32, then you will have to pass these images through</p>
<pre><code class="lang-auto">c = nn.Conv2d(in_channels=1, out_channels=3, kernel_size=1)
c(x).shape
</code></pre>
<p>gives</p>
<pre><code class="lang-auto">torch.Size([24, 3, 32, 32])
</code></pre>

kirk86: <p>Thanks a lot. I was hoping to avoid changing the model by adding an additional conv operation just to make the input sizes appropriate.</p>
<p>I tried having it in my transforms</p>
<pre><code class="lang-auto">transforms.Resize(32),
transforms.RandomHorizontalFlip(),                        
transforms.RandomCrop(32, padding=4),                     
transforms.ToTensor(),                                    
transforms.Normalize((0.485, 0.456, 0.406), (0.229, 0.224, 0.225)),
transforms.Lambda(lambda x: nn.Conv2d(                    
in_channels=1, out_channels=3, kernel_size=1)(x)      
),
</code></pre>
<p>but produces broadcast errors</p>
<pre><code class="lang-auto">Traceback (most recent call last):
  File "/home/kirk/miniconda3/envs/torch/lib/python3.6/site-packages/torch/utils/data/_utils/worker.py", line 178, in _worker_loop
    data = fetcher.fetch(index)
  File "/home/kirk/miniconda3/envs/torch/lib/python3.6/site-packages/torch/utils/data/_utils/fetch.py", line 44, in fetch
    data = [self.dataset[idx] for idx in possibly_batched_index]
  File "/home/kirk/miniconda3/envs/torch/lib/python3.6/site-packages/torch/utils/data/_utils/fetch.py", line 44, in &lt;listcomp&gt;
    data = [self.dataset[idx] for idx in possibly_batched_index]
  File "/home/kirk/miniconda3/envs/torch/lib/python3.6/site-packages/torchvision/datasets/mnist.py", line 95, in __getitem__
    img = self.transform(img)
  File "/home/kirk/miniconda3/envs/torch/lib/python3.6/site-packages/torchvision/transforms/transforms.py", line 61, in __call__
    img = t(img)
  File "/home/kirk/miniconda3/envs/torch/lib/python3.6/site-packages/torchvision/transforms/transforms.py", line 166, in __call__
    return F.normalize(tensor, self.mean, self.std, self.inplace)
  File "/home/kirk/miniconda3/envs/torch/lib/python3.6/site-packages/torchvision/transforms/functional.py", line 217, in normalize
    tensor.sub_(mean[:, None, None]).div_(std[:, None, None])
RuntimeError: output with shape [1, 32, 32] doesn't match the broadcast shape [3, 32, 32]
</code></pre>
<p>I seems that what I’m missing is the equivalent commands of</p>
<pre><code class="lang-auto">train_set.data.unsqueeze_(1)                                                                                                                                                                 
train_set.data = train_set.data.repeat(1, 3, 1, 1)  
</code></pre>
<p>but for PIL images so that I can hook them up in the transforms.</p>

mailcorahul: <p><a class="mention" href="/u/kirk86">@kirk86</a> You should ideally provide the image in the order (H, W, C to PIL for resizing(and not as &lt;C, H, W&gt; as you have provided above, hence the error). You can later transpose the tensor to get the shape = (N, C, H, W).</p>
<p>P.S: If you need the input image to be of shape (3, 32, 32), you can use your own code<br>
<code>train_set.data = train_set.data.repeat(1, 3, 1, 1)</code> to repeat the grayscale data along RGB channels(after the resize). I don’t think  <code>nn.Conv2d</code> is the right way to do this.</p>

kirk86: <aside class="quote no-group" data-username="mailcorahul" data-post="4" data-topic="58619">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/mailcorahul/48/15919_2.png" class="avatar"> mailcorahul:</div>
<blockquote>
<p>You should ideally provide the image in the order (H, W, C to PIL for resizing(and not as &lt;C, H, W&gt; as you have provided above, hence the error). You can later transpose the tensor to get the shape = (N, C, H, W).</p>
</blockquote>
</aside>
<p>These different representations of different programs are so fucking annoying and can cause unnecessary headaches.</p>

mailcorahul: <p>I think both opencv and pillow takes images in this order(rows, cols) or (rows, cols, depth).</p>

kirk86: <p>Can you give a MWE example of what you explained above in my use case. Everything that I’ve tried doesn’t seem to work. Especially because any transforms are only applied when you iterate over the loader.</p>

kirk86: <p>I’ve also found that the resize in the transforms works only if you don’t have any normalization after it, otherwise it throws errors on dimensions.</p>

mailcorahul: <p>You can write your own <code>Dataset</code> class. Basically use existing FashionMNIST with your own custom modifications</p>
<pre><code class="lang-auto">class CustomDataset(Dataset):
    def __init__(self, root_dir):
        self.train_set = FashionMNIST(root=root_dir, train=True, download=True)
        self.transforms = torchvision.transforms.Compose([                                                                                                                                                
            torchvision.transforms.Resize((32, 32)),                                                                                                                                                   
            torchvision.transforms.ToTensor(),                                                                                                                                               
 ])                                           
    def __getitem__(self, idx):
        # apply necessary transforms including resize first
        image = self.transforms(self.train_set.data[idx])
        
        # image of shape (1, 3, 32, 32)
        image = image.view((1, 1, 32, 32)).repeat(1, 3, 1, 1)
        label = self.train_set.targets[idx]
        
        # do other operations necessary
        return image, label</code></pre>

kirk86: <p>Thanks, the other only alternative that I came up with is to use the transforms as before without the normalization</p>
<pre><code class="lang-auto">transforms.Resize(32),
transforms.RandomHorizontalFlip(),                        
transforms.RandomCrop(32, padding=4),                     
transforms.ToTensor(),                                     
),
</code></pre>
<p>then when iterating over the loader checking channel dim and repeating accordingly</p>
<pre><code class="lang-auto">for x, y in loader:
  if x.shape[1] != 3:
     x = x.repeat(1, 3, 1, 1)
</code></pre>

mailcorahul: <p>Yes, that can also be done!</p>