Target size (torch.Size([3, 3, 256, 256])) must be the same as input size (torch.Size([3, 65536]))

username: ChrisRahme
cooked: <p>Hello!</p>
<p>I’m new to machine learning and PyTorch, and I’m stuck on this error which seems really simple but I can’t find where to fix it:</p>
<pre><code class="lang-auto">ValueError                                Traceback (most recent call last)
~\AppData\Local\Temp/ipykernel_52832/2153252887.py in &lt;module&gt;
      5 n_epochs  = 10
      6 
----&gt; 7 train_unet(unet, trainloader, valloader, optimizer, loss, n_epochs=n_epochs)
      8 # test_unet(mlp_net, testloader, loss)

~\AppData\Local\Temp/ipykernel_52832/3077527990.py in train_unet(net, trainloader, valloader, optimizer, loss_function, n_epochs)
     24 
     25             y_pred = net(X.float())
---&gt; 26             y_loss = loss_function(y_pred, y)
     27             y_dice = dice(y, y_pred)
     28 

~\AppData\Roaming\Python\Python39\site-packages\torch\nn\modules\module.py in _call_impl(self, *input, **kwargs)
   1108         if not (self._backward_hooks or self._forward_hooks or self._forward_pre_hooks or _global_backward_hooks
   1109                 or _global_forward_hooks or _global_forward_pre_hooks):
-&gt; 1110             return forward_call(*input, **kwargs)
   1111         # Do not call functions when jit is used
   1112         full_backward_hooks, non_full_backward_hooks = [], []

~\AppData\Roaming\Python\Python39\site-packages\torch\nn\modules\loss.py in forward(self, input, target)
    711 
    712     def forward(self, input: Tensor, target: Tensor) -&gt; Tensor:
--&gt; 713         return F.binary_cross_entropy_with_logits(input, target,
    714                                                   self.weight,
    715                                                   pos_weight=self.pos_weight,

~\AppData\Roaming\Python\Python39\site-packages\torch\nn\functional.py in binary_cross_entropy_with_logits(input, target, weight, size_average, reduce, reduction, pos_weight)
   3128 
   3129     if not (target.size() == input.size()):
-&gt; 3130         raise ValueError("Target size ({}) must be the same as input size ({})".format(target.size(), input.size()))
   3131 
   3132     return torch.binary_cross_entropy_with_logits(input, target, weight, pos_weight, reduction_enum)

ValueError: Target size (torch.Size([3, 3, 256, 256])) must be the same as input size (torch.Size([3, 65536]))
</code></pre>
<p>My model returns an output of size <code>torch.Size([3, 65536])</code>, while the labels are images of <code>torch.Size([3, 3, 256, 256])</code>.</p>
<p>The images are of size <code>torch.Size([3, 3, 256, 256])</code> and not <code>torch.Size([3, 256, 256])</code>. The first 3 is the batch size, the second one is for the 3 RGB channels, and the 256s are the image dimensions.</p>
<p>Clearly, the model returns a size of 65536, which is 256*256, converting the image to a line. But how do I convert it back to a square? And how do I get rid of the batch size’s 3?</p>
<p>Here’s a reproducible code:</p>
<pre><code class="lang-auto">import torch
import torch.nn as nn

from torch.nn                    import Linear, Module, ModuleList, ReLU, Sequential, BCEWithLogitsLoss, Conv2d, ConvTranspose2d, MaxPool2d
from torch.optim                 import SGD, Adam
from torch.utils.data.dataloader import DataLoader
from torch.utils.data.dataset    import Dataset

from skimage.io        import imread
from skimage.transform import resize

from glob import glob
import os

import matplotlib.pyplot as plt
import numpy as np



class TissueDataset(Dataset):
    def __init__(self, img_path, target_path):
        super().__init__()
        self.imgs    = glob(os.path.join(img_path, "*.jpg"))
        self.targets = glob(os.path.join(target_path, "*.jpg"))
    
    def __getitem__(self, idx):
        size = (3, 256, 256)

        image = imread(self.imgs[idx])
        label = imread(self.targets[idx])

        image = resize(image, size, order = 1, preserve_range = True)
        label = resize(label, size, order = 0, preserve_range = True).astype(int)

        return image, label
    
    def __len__(self):
        return len(self.imgs)



def makeLoader(type):
    dataloader = DataLoader(TissueDataset(
        img_path    = f'data/tissue/{type}/jpg',
        target_path = f'data/tissue/{type}/lbl'
    ), batch_size = 3, shuffle = True)

    return dataloader


trainloader = makeLoader('train')
valloader   = makeLoader('val')
testloader  = makeLoader('test')



class DoubleConv2d(Module):
    def __init__(self, in_channels, out_channels):
        super(DoubleConv2d, self).__init__()

        self.stack = Sequential(
            Conv2d(in_channels, out_channels, 3, 1, 1),
            ReLU(),
            Conv2d(out_channels, out_channels, 3, 1, 1),
            ReLU()
        )


    def forward(self, x):
        return self.stack(x)



class UNet(Module):
    def __init__(self):
        super(UNet, self).__init__()
        
        inn = 3
        out = 1
        mid = [112, 224, 448]

        self.encoder = ModuleList()
        self.bottom  = DoubleConv2d(mid[-1], 2*mid[-1]) # should both be mid[-1]?
        self.decoder = ModuleList()
        self.end     = Conv2d(mid[0], 1*out, 1)

        self.maxpool = MaxPool2d(2, 2)
        #self.linear  = Linear(65536, 10)

        for dim in mid:
            self.encoder.append(DoubleConv2d(inn, dim))
            inn = dim

        for dim in mid[::-1]:
            self.decoder.append(ConvTranspose2d(2*dim, dim, 2, 2))
            self.decoder.append(DoubleConv2d(2*dim, dim))


    def forward(self, x):
        connections = []

        for i in range(len(self.encoder)):
            module = self.encoder[i]

            x = module(x)
            connections.append(x)
            x = self.maxpool(x)

        x = self.bottom(x)

        for i in range(len(self.decoder)):
            module = self.decoder[i]
            x = module(x)

            if i % 2 == 0: # ConvTranspose2d
                connection = connections.pop()
                x = torch.cat((connection, x), dim=1)
        
        x = self.end(x)

        x = x.view(x.size(0), -1)
        #x = self.linear(x)

        return x



# print(UNet())
# unet = UNet()
# x = torch.randn(3, 3, 256, 256)
# out = unet(x)



def dice(ytrue, ypred):
    if ytrue.dim() == 2:
        inter = torch.dot(ytrue[i].reshape(-1), ypred[i].reshape(-1))

        cardA = torch.sum(ytrue)
        cardB = torch.sum(ypred)

        num = 2 * inter     + 1e-6
        den = cardA + cardB + 1e-6

        return num / den

    elif ytrue.dim() == 3:
        batch_size = ytrue.shape[0]
        total = 0

        for i in range(batch_size):
            total += dice(ytrue[i, ...], ypred[i, ...])

        return total / batch_size
    
    return None



def train_unet(net, trainloader, valloader, optimizer, loss_function, n_epochs):
    net = net.float()

    t_size = len(trainloader.dataset)
    v_size = len(valloader.dataset)

    graphs = {
        'train_loss' : [],
        'val_loss'   : [],
        'train_dice' : [],
        'val_dice'   : []
    }

    for epoch in range(1, n_epochs+1):
        net.train()
        y_loss = 0
        y_dice = 0
        w_loss = 0
        w_dice = 0

        for X_batch, (X, y) in enumerate(trainloader):
            X, y = X.to(device), y.to(device)
            
            y_pred = net(X.float())
            y_loss = loss_function(y_pred, y)
            y_dice = dice(y, y_pred)

            optimizer.zero_grad()
            y_loss.backward()
            optimizer.step()

            if X_batch % 100 == 0:
                net.eval()
                w_loss = 0
                w_dice = 0

                for V_batch, (V, w) in enumerate(valloader):
                    V, w = V.to(device), w.to(device)

                    w_pred  = net(V.float())
                    w_loss  = loss_function(w_pred, w)
                    w_dice += dice(w, w_pred)

                w_loss = w_loss / v_size
                w_dice = w_dice / v_size

            graphs['train_loss'].append(y_loss.item())
            graphs['train_dice'].append(y_dice)
            graphs['val_loss'].append(w_loss.item())
            graphs['val_dice'].append(w_dice)



unet      = UNet()

optimizer = Adam(unet.parameters(), lr=0.001)
loss      = BCEWithLogitsLoss()
n_epochs  = 10

train_unet(unet, trainloader, valloader, optimizer, loss, n_epochs=n_epochs)
</code></pre>
<p>Or if you prefer it as a file: <a href="https://we.tl/t-0oAjeZVZQ9" rel="noopener nofollow ugc">https://we.tl/t-0oAjeZVZQ9</a><br>
And here is the data: <a href="https://we.tl/t-9g3tUYCJ3d" rel="noopener nofollow ugc">https://we.tl/t-9g3tUYCJ3d</a></p>
<hr>
<p>Edit 1: Getting rid of the line <code>x = x.view(x.size(0), -1)</code> and using a batch size of 4 instead of 3 for clarity, the error becomes:</p>
<pre><code class="lang-auto">Target size (torch.Size([4, 3, 256, 256])) must be the same as input size (torch.Size([4, 1, 256, 256]))
</code></pre>
<hr>
<p>Edit 2: Using sizes <code>(3, 256, 256)</code> for images and <code>(1, 256, 256)</code> for labels, and removing <code>.astype(int)</code> from the <code>__getitem__</code> method gives this error:</p>
<pre><code class="lang-auto">TypeError: conv2d() received an invalid combination of arguments - got (builtin_function_or_method, Parameter, Parameter, tuple, tuple, tuple, int), but expected one of:
 * (Tensor input, Tensor weight, Tensor bias, tuple of ints stride, tuple of ints padding, tuple of ints dilation, int groups)
      didn't match because some of the arguments have invalid types: (!builtin_function_or_method!, !Parameter!, !Parameter!, !tuple!, !tuple!, !tuple!, int)
 * (Tensor input, Tensor weight, Tensor bias, tuple of ints stride, str padding, tuple of ints dilation, int groups)
      didn't match because some of the arguments have invalid types: (!builtin_function_or_method!, !Parameter!, !Parameter!, !tuple!, !tuple!, !tuple!, int)
</code></pre>

username: KFrank
cooked: <p>Hi Chris!</p>
<aside class="quote no-group quote-modified" data-username="ChrisRahme" data-post="1" data-topic="149359" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/chrisrahme/48/48696_2.png" class="avatar"> ChrisRahme:</div>
<blockquote>
<p>Edit 1: Getting rid of the line <code>x = x.view(x.size(0), -1)</code><br>
…<br>
Edit 2: Using sizes <code>(3, 256, 256)</code> for images and <code>(1, 256, 256)</code> for labels, and removing <code>.astype(int)</code> from the <code>__getitem__</code> method</p>
</blockquote>
</aside>
<p>Based on your edits, it looks like you are on the right track.</p>
<p>Let me make some comments based on my speculation about your use<br>
case and what you are trying to do.</p>
<p>Based on the name <code>TissueDataset</code>, the name <code>UNet</code>, and the use of<br>
<code>BCEWithLogitsLoss</code> as your loss criterion, I assume that you are<br>
performing <em>binary semantic segmentation.</em>  That is, you wish to classify<br>
each pixel in your input as being in either “class 0” (background or healthy<br>
tissue or whatever) or “class 1” (foreground or diseased tissue or whatever).</p>
<p>As a general rule (and as a requirement for convolutional networks),<br>
pytorch networks work with <em>batches</em> of inputs (and outputs and labels).<br>
If you want to process a single image, you still need to package it as a<br>
batch with a batch size of one.</p>
<p>The input to your model should have shape <code>[nBatch, nChannels, H, W]</code>,<br>
where in your case <code>nChannels = 3</code> and presumably correspond to the RGB<br>
channels of a color image, and have type <code>float</code> (or <code>double</code>).</p>
<p>Your output and labels should both have shape <code>[nBatch, 1, H, W]</code> (or just<br>
<code>[nBatch, H, W]</code>), and also be of type <code>float</code>.</p>
<p>Note, UNet does not (typically) have <code>H</code> and <code>W</code> wired into it – the same<br>
UNet can be trained on, and perform inference on images of differing<br>
shapes – but any given <em>batch</em> has to consist of images of the same<br>
shape.</p>
<p>So, yes, get rid of the final <code>x = x.view(x.size(0), -1)</code> (and the<br>
commented-out <code>x = self.linear(x)</code>), and have your model output<br>
the result of the final <code>Conv2d</code> (<code>self.end</code>) layer.  And, yes, <code>out = 1</code> is<br>
correct.</p>
<p>If your labels are indeed (batches of) images of shape <code>[3, 3, 256, 256]</code>,<br>
then you have to figure out how they are “encoded” to give <em>binary</em> class<br>
labels.  Could they be pure black-and-white images that happen to be<br>
encoded as three-channel RGB images?</p>
<p>In any event, you have to process your labels “images” to be single-channel<br>
binary labels (of type <code>float</code>).  (Your labels don’t <em>actually</em> have to be pure<br>
binary, that is, exactly zero or one – they can be probabilistic labels that<br>
run from zero to one.)</p>
<p>Best.</p>
<p>K. Frank</p>

username: ChrisRahme
cooked: <p>Thank you so much for your detailed answer! I am indeed trying to solve a segmentation problem. Here is an example of an image and its label:</p>
<p><img src="https://discuss.pytorch.org/uploads/default/original/3X/c/0/c0cac875e9e9da30711eae43e09cf186192ffbcb.jpeg" alt="17997" data-base62-sha1="rvw8mXA4syTlnBxvfxP0PquBt8L" width="270" height="134"> <img src="https://discuss.pytorch.org/uploads/default/original/3X/5/f/5f447d10e2eeb36dcffa08f8e6215ad9aa8d2a42.jpeg" alt="17997" data-base62-sha1="dAM5VUvfkWb4sOotz6Vv55eAjJM" width="270" height="134"></p>
<aside class="quote no-group" data-username="KFrank" data-post="2" data-topic="149359">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/k/ecb155/48.png" class="avatar"> KFrank:</div>
<blockquote>
<p>If your labels are indeed (batches of) images of shape <code>[3, 3, 256, 256]</code>,<br>
then you have to figure out how they are “encoded” to give <em>binary</em> class<br>
labels. Could they be pure black-and-white images that happen to be<br>
encoded as three-channel RGB images?</p>
</blockquote>
</aside>
<p>If I open a label image on Photoshop, I can see the image mode is set to Grayscale and not RGB, so the shape should be <code>[3, 1, 256, 256]</code>, right?</p>
<p>And this is what I see when I <code>print(y_pred.shape, y.shape)</code>, so it’s probably correct.</p>
<pre data-code-wrap="python"><code class="lang-python">torch.Size([4, 1, 256, 256]) torch.Size([4, 1, 256, 256])
</code></pre>
<p>All looks fine now, but I still have this error I can’t make sense of:</p>
<pre><code class="lang-auto">TypeError                                 Traceback (most recent call last)
~\AppData\Local\Temp/ipykernel_26916/903322469.py in &lt;module&gt;
      5 n_epochs  = 10
      6 
----&gt; 7 train_unet(unet, trainloader, valloader, optimizer, loss, n_epochs=n_epochs)
      8 # test_unet(unet, testloader, loss)

~\AppData\Local\Temp/ipykernel_26916/239980266.py in train_unet(net, trainloader, valloader, optimizer, loss_function, n_epochs)
     40                     V, w = V.to(device), w.to(device)
     41 
---&gt; 42                     w_pred  = net(V.float)
     43                     w_loss  = loss_function(w_pred, w)
     44                     w_dice += dice(w, w_pred)

~\AppData\Roaming\Python\Python39\site-packages\torch\nn\modules\module.py in _call_impl(self, *input, **kwargs)
   1108         if not (self._backward_hooks or self._forward_hooks or self._forward_pre_hooks or _global_backward_hooks
   1109                 or _global_forward_hooks or _global_forward_pre_hooks):
-&gt; 1110             return forward_call(*input, **kwargs)
   1111         # Do not call functions when jit is used
   1112         full_backward_hooks, non_full_backward_hooks = [], []

~\AppData\Local\Temp/ipykernel_26916/3729299638.py in forward(self, x)
     30             module = self.encoder[i]
     31 
---&gt; 32             x = module(x)
     33             connections.append(x)
     34             x = self.maxpool(x)

~\AppData\Roaming\Python\Python39\site-packages\torch\nn\modules\module.py in _call_impl(self, *input, **kwargs)
   1108         if not (self._backward_hooks or self._forward_hooks or self._forward_pre_hooks or _global_backward_hooks
   1109                 or _global_forward_hooks or _global_forward_pre_hooks):
-&gt; 1110             return forward_call(*input, **kwargs)
   1111         # Do not call functions when jit is used
   1112         full_backward_hooks, non_full_backward_hooks = [], []

~\AppData\Local\Temp/ipykernel_26916/3683377085.py in forward(self, x)
     12 
     13     def forward(self, x):
---&gt; 14         return self.stack(x)

~\AppData\Roaming\Python\Python39\site-packages\torch\nn\modules\module.py in _call_impl(self, *input, **kwargs)
   1108         if not (self._backward_hooks or self._forward_hooks or self._forward_pre_hooks or _global_backward_hooks
   1109                 or _global_forward_hooks or _global_forward_pre_hooks):
-&gt; 1110             return forward_call(*input, **kwargs)
   1111         # Do not call functions when jit is used
   1112         full_backward_hooks, non_full_backward_hooks = [], []

~\AppData\Roaming\Python\Python39\site-packages\torch\nn\modules\container.py in forward(self, input)
    139     def forward(self, input):
    140         for module in self:
--&gt; 141             input = module(input)
    142         return input
    143 

~\AppData\Roaming\Python\Python39\site-packages\torch\nn\modules\module.py in _call_impl(self, *input, **kwargs)
   1108         if not (self._backward_hooks or self._forward_hooks or self._forward_pre_hooks or _global_backward_hooks
   1109                 or _global_forward_hooks or _global_forward_pre_hooks):
-&gt; 1110             return forward_call(*input, **kwargs)
   1111         # Do not call functions when jit is used
   1112         full_backward_hooks, non_full_backward_hooks = [], []

~\AppData\Roaming\Python\Python39\site-packages\torch\nn\modules\conv.py in forward(self, input)
    445 
    446     def forward(self, input: Tensor) -&gt; Tensor:
--&gt; 447         return self._conv_forward(input, self.weight, self.bias)
    448 
    449 class Conv3d(_ConvNd):

~\AppData\Roaming\Python\Python39\site-packages\torch\nn\modules\conv.py in _conv_forward(self, input, weight, bias)
    441                             weight, bias, self.stride,
    442                             _pair(0), self.dilation, self.groups)
--&gt; 443         return F.conv2d(input, weight, bias, self.stride,
    444                         self.padding, self.dilation, self.groups)
    445 

TypeError: conv2d() received an invalid combination of arguments - got (builtin_function_or_method, Parameter, Parameter, tuple, tuple, tuple, int), but expected one of:
 * (Tensor input, Tensor weight, Tensor bias, tuple of ints stride, tuple of ints padding, tuple of ints dilation, int groups)
      didn't match because some of the arguments have invalid types: (!builtin_function_or_method!, !Parameter!, !Parameter!, !tuple!, !tuple!, !tuple!, int)
 * (Tensor input, Tensor weight, Tensor bias, tuple of ints stride, str padding, tuple of ints dilation, int groups)
      didn't match because some of the arguments have invalid types: (!builtin_function_or_method!, !Parameter!, !Parameter!, !tuple!, !tuple!, !tuple!, int)
</code></pre>

username: ChrisRahme
cooked: <p>For reference, this is the UNet I’m trying to build (ignoring the dropouts)</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/a/0/a0a53b62eca099db0ac3dae110c9eb5f4cffe9c3.png" data-download-href="https://discuss.pytorch.org/uploads/default/a0a53b62eca099db0ac3dae110c9eb5f4cffe9c3" title="UNet-repr"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/a/0/a0a53b62eca099db0ac3dae110c9eb5f4cffe9c3_2_690x437.png" alt="UNet-repr" data-base62-sha1="mV8oZIwV21To8uKqnsoKRiC5uW7" width="690" height="437" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/a/0/a0a53b62eca099db0ac3dae110c9eb5f4cffe9c3_2_690x437.png, https://discuss.pytorch.org/uploads/default/original/3X/a/0/a0a53b62eca099db0ac3dae110c9eb5f4cffe9c3.png 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/a/0/a0a53b62eca099db0ac3dae110c9eb5f4cffe9c3.png 2x" data-dominant-color="E6E9EE"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">UNet-repr</span><span class="informations">850×539 46.9 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

username: ChrisRahme
cooked: <p>Nevermind, after hours of work I found the error - stupid error, as always <img src="https://discuss.pytorch.org/images/emoji/apple/man_facepalming.png?v=12" title=":man_facepalming:" class="emoji" alt=":man_facepalming:" loading="lazy" width="20" height="20"></p>
<p>I was calling <code>w_pred  = net(V.float)</code> instead of <code>w_pred  = net(V.float())</code>.<br>
It’s written right there in the error that I kept looking at but I kept missing it until I posted it here <img src="https://discuss.pytorch.org/images/emoji/apple/smiling_face_with_tear.png?v=12" title=":smiling_face_with_tear:" class="emoji" alt=":smiling_face_with_tear:" loading="lazy" width="20" height="20"></p>
<p>Thanks a lot!</p>