ValueError: Target size (torch.Size([4, 1])) must be the same as input size (torch.Size([16, 1]))

abuadam: <p>Can someone help me, i could not pass through this error</p>
<p>import torch.optim as optim<br>
import torch.nn.functional as F</p>
<h1>
<a name="loss-function-1" class="anchor" href="#loss-function-1"></a>Loss function</h1>
<p>criterion = nn.BCEWithLogitsLoss()</p>
<h1>
<a name="optimizer-2" class="anchor" href="#optimizer-2"></a>Optimizer</h1>
<p>optimizer = optim.Adam(model.parameters(), lr=0.001)</p>
<h1>
<a name="define-number-of-training-epochs-3" class="anchor" href="#define-number-of-training-epochs-3"></a>Define number of training epochs</h1>
<p>EPOCHS = 10</p>
<h1>
<a name="train-the-model-4" class="anchor" href="#train-the-model-4"></a>train the model</h1>
<p>for epoch in range(EPOCHS):<br>
# clear gradients<br>
optimizer.zero_grad()<br>
running_loss = 0<br>
for images, labels in train_loader:<br>
# move data to device<br>
images = images.to(device)<br>
labels = labels.to(device)<br>
labels = labels.view(-1, 1) # reshape labels<br>
labels = torch.round(labels).to(torch.float).view(-1, 1)</p>
<pre><code>    # zero gradients
    optimizer.zero_grad()
    
    # forward pass
    outputs = model(images)
    outputs = outputs.view(-1, 1) #reshape output
    running_loss = criterion(outputs, labels)

    # backward pass
    running_loss.backward()
    # update model parameters
    optimizer.step()

    running_loss += running_loss.item()
    
    # print loss and accuracy
print("epoch:", epoch, "loss:", running_loss.item())
running_loss = 0

# validation loss
test_loss = 0
with torch.no_grad():
    for images, labels in val_loader:
         # move data to device
        images = images.to(device)
        labels = labels.to(device)
        labels = torch.round(labels).to(torch.float).view(-1, 1)
        # forward pass
        outputs = model(images)
        outputs = outputs.view(-1, 1) # reshape output
        test_loss += criterion(outputs, labels)
        test_loss.backward()
        optimizer.step()
        test_loss += test_loss.item()

# print loss and accuracy
print("epoch:", epoch, "loss:", running_loss.item(), "test_loss:", test_loss.item())
</code></pre>
<h1>
<a name="evaluate-model-on-test-data-5" class="anchor" href="#evaluate-model-on-test-data-5"></a>evaluate model on test data</h1>
<p>with torch.no_grad():<br>
test_loss = 0<br>
for images, labels in test_generator:<br>
# move data to device<br>
images = images.to(device)<br>
labels = labels.to(device)<br>
labels = labels.view(-1, 1) # reshape labels</p>
<pre><code>    # forward pass
    test_outputs = model(images)
    test_loss += criterion(test_outputs, labels)
    
test_loss = test_loss / len(test_generator)
print("Test loss:", test_loss.item())
</code></pre>
<hr>
<p>ValueError                                Traceback (most recent call last)<br>
~\AppData\Local\Temp\ipykernel_14840\2968328638.py in <br>
30         outputs = model(images)<br>
31         outputs = outputs.view(-1, 1) <span class="hashtag">#reshape</span> output<br>
—&gt; 32         running_loss = criterion(outputs, labels)<br>
33<br>
34         # backward pass</p>
<p>~\anaconda3\lib\site-packages\torch\nn\modules\module.py in _call_impl(self, *input, **kwargs)<br>
1192         if not (self._backward_hooks or self._forward_hooks or self._forward_pre_hooks or _global_backward_hooks<br>
1193                 or _global_forward_hooks or _global_forward_pre_hooks):<br>
 → 1194             return forward_call(*input, **kwargs)<br>
1195         # Do not call functions when jit is used<br>
1196         full_backward_hooks, non_full_backward_hooks = [], []</p>
<p>~\anaconda3\lib\site-packages\torch\nn\modules\loss.py in forward(self, input, target)<br>
718<br>
719     def forward(self, input: Tensor, target: Tensor) → Tensor:<br>
 → 720         return F.binary_cross_entropy_with_logits(input, target,<br>
721                                                   self.weight,<br>
722                                                   pos_weight=self.pos_weight,</p>
<p>~\anaconda3\lib\site-packages\torch\nn\functional.py in binary_cross_entropy_with_logits(input, target, weight, size_average, reduce, reduction, pos_weight)<br>
3158<br>
3159     if not (target.size() == input.size()):<br>
 → 3160         raise ValueError(“Target size ({}) must be the same as input size ({})”.format(target.size(), input.size()))<br>
3161<br>
3162     return torch.binary_cross_entropy_with_logits(input, target, weight, pos_weight, reduction_enum)</p>
<p>ValueError: Target size (torch.Size([4, 1])) must be the same as input size (torch.Size([16, 1]))</p>

J_Johnson: <aside class="quote no-group" data-username="abuadam" data-post="1" data-topic="170502">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/a/8e8cbc/48.png" class="avatar"> abuadam:</div>
<blockquote>
<p><code>running_loss = criterion(outputs, labels)</code></p>
</blockquote>
</aside>
<p>Before this line, add a print statement:</p>
<pre><code class="lang-auto">print(outputs.size(), labels.size())
</code></pre>
<p>BCEWithLogitsLoss expects these to be the same size and it seems they are not. Something like this works fine:</p>
<pre><code class="lang-auto">loss = nn.BCEWithLogitsLoss()
input = torch.randn(3, requires_grad=True)
target = torch.empty(3).random_(2)
output = loss(input, target)
output.backward()
</code></pre>
<p><a href="https://pytorch.org/docs/stable/generated/torch.nn.BCEWithLogitsLoss.html" class="onebox" target="_blank" rel="noopener nofollow ugc">https://pytorch.org/docs/stable/generated/torch.nn.BCEWithLogitsLoss.html</a></p>

abuadam: <p>Thank you for your reply, I have added the code as</p>
<h1>
<a name="forward-pass-1" class="anchor" href="#forward-pass-1"></a>forward pass</h1>
<pre><code>    outputs = model(images)
    outputs = outputs.view(-1, 1) #reshape output
    print(outputs.size(), labels.size())
    running_loss = criterion(outputs, labels)
</code></pre>
<p>But the error Remains the same. This link to the complete code <a href="https://drive.google.com/file/d/1oYgAF9quiWCEa2QtNH-aYSfBYVXqtEln/view?usp=sharing" rel="noopener nofollow ugc">COMPLETE CODE</a></p>

J_Johnson: <p>A <code>print</code> statement will not fix the error. But it will show you if the sizes don’t match. Scroll <em>above</em> the error message to see what printed before the error occurred.</p>

abuadam: <p>torch.Size([16, 1]) torch.Size([4, 1])</p>
<hr>
<p>ValueError                                Traceback (most recent call last)<br>
~\AppData\Local\Temp\ipykernel_11832\1478269417.py in <br>
31         outputs = outputs.view(-1, 1) <span class="hashtag">#reshape</span> output<br>
32         print(outputs.size(), labels.size())<br>
—&gt; 33         running_loss = criterion(outputs, labels)<br>
34<br>
35         # backward pass</p>
<p>~\anaconda3\lib\site-packages\torch\nn\modules\module.py in _call_impl(self, *input, **kwargs)<br>
1192         if not (self._backward_hooks or self._forward_hooks or self._forward_pre_hooks or _global_backward_hooks<br>
1193                 or _global_forward_hooks or _global_forward_pre_hooks):<br>
 → 1194             return forward_call(*input, **kwargs)<br>
1195         # Do not call functions when jit is used<br>
1196         full_backward_hooks, non_full_backward_hooks = [], []</p>
<p>~\anaconda3\lib\site-packages\torch\nn\modules\loss.py in forward(self, input, target)<br>
718<br>
719     def forward(self, input: Tensor, target: Tensor) → Tensor:<br>
 → 720         return F.binary_cross_entropy_with_logits(input, target,<br>
721                                                   self.weight,<br>
722                                                   pos_weight=self.pos_weight,</p>
<p>~\anaconda3\lib\site-packages\torch\nn\functional.py in binary_cross_entropy_with_logits(input, target, weight, size_average, reduce, reduction, pos_weight)<br>
3158<br>
3159     if not (target.size() == input.size()):<br>
 → 3160         raise ValueError(“Target size ({}) must be the same as input size ({})”.format(target.size(), input.size()))<br>
3161<br>
3162     return torch.binary_cross_entropy_with_logits(input, target, weight, pos_weight, reduction_enum)</p>
<p>ValueError: Target size (torch.Size([4, 1])) must be the same as input size (torch.Size([16, 1]))</p>

J_Johnson: <p>The model outputs don’t match the label size. It means your model is not giving out the right size.</p>

abuadam: <aside class="quote no-group" data-username="J_Johnson" data-post="6" data-topic="170502">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/j_johnson/48/55494_2.png" class="avatar"> J_Johnson:</div>
<blockquote>
<p>s don’t match the label size. It me</p>
</blockquote>
</aside>
<p>Please what can i do to make it work. I’m new to pytorch.</p>

J_Johnson: <p>Update your model code so it gives the correct output size.</p>

abuadam: <p>This is the model<br>
import torch.nn as nn<br>
import torch.nn.functional as F</p>
<h1>
<a name="model-definition-1" class="anchor" href="#model-definition-1"></a>model definition</h1>
<p>class Net(nn.Module):<br>
def <strong>init</strong>(self):<br>
super(Net, self).<strong>init</strong>()</p>
<pre><code>    #convolutional layers
    self.conv1 = nn.Conv2d(3, 32, kernel_size=3, stride=1, padding=1)
    self.pool1 = nn.MaxPool2d(kernel_size=2, stride=2, padding=0)
    
    self.conv2 = nn.Conv2d(32, 64, kernel_size=3, stride=1, padding=1)
    self.pool2 = nn.MaxPool2d(kernel_size=2, stride=2, padding=0)
    
    self.conv3 = nn.Conv2d(64, 128, kernel_size=3, stride=1, padding=1)
    self.pool3 = nn.MaxPool2d(kernel_size=2, stride=2, padding=0)
    
    self.conv4 = nn.Conv2d(128, 256, kernel_size=3, stride=1, padding=1)
    self.pool4 = nn.MaxPool2d(kernel_size=2, stride=2, padding=0)
    
    #dropout layers
    self.dropout1 = nn.Dropout2d(p=0.2)
    self.dropout2 = nn.Dropout2d(p=0.2)
    self.dropout3 = nn.Dropout2d(p=0.2)
    
    # fully connected layers
    self.fc1 = nn.Linear(256*7*7, 256)
    self.fc2 = nn.Linear(256, 1)

    # sigmoid activation
    self.sigmoid = nn.Sigmoid()

def forward(self, x):
    x = self.pool1(F.relu(self.conv1(x)))
    x = self.dropout1(x)
    x = self.pool2(F.relu(self.conv2(x)))
    x = self.dropout2(x)
    x = self.pool3(F.relu(self.conv3(x)))
    x = self.dropout3(x)
    x = self.pool4(F.relu(self.conv4(x)))
    x = x.view(-1, 256*7*7)
    x = F.relu(self.fc1(x))
    x = self.fc2(x)
    x = self.sigmoid(x)
    return x
</code></pre>
<h1>
<a name="create-an-instance-of-the-model-2" class="anchor" href="#create-an-instance-of-the-model-2"></a>create an instance of the model</h1>
<p>model = Net()<br>
model.to(device)</p>

J_Johnson: <p>Your model doesn’t deal with variable input sizes. Each of your <code>Conv2d</code> layers give the size input as output. Each <code>MaxPool2d</code> reduces both dims in half. You have 4 of those.</p>
<p>Say your input image is 32:<br>
[32, 32]/(2**4) = [2, 2]</p>
<p>That leaves 2x2 = 4 values coming out of the final <code>MaxPool2d</code>.</p>

abuadam: <p>Can you help me with the section to update?</p>

J_Johnson: <p>The problem has been identified. There are many ways to fix it. If you built that model, it should be pretty straight forward how to get the size down to 1x1 before the linear layers.</p>
<p>You could add another Conv2d and MaxPool2d, you could put a final AvgPool2d or AdaptiveAvgPool2d, you could increase the input size of your first linear layer by 4, etc.</p>

abuadam: <p>Thank you once again, this really help. this is how the model is<br>
import torch.nn as nn<br>
import torch.nn.functional as F</p>
<h1>
<a name="model-definition-1" class="anchor" href="#model-definition-1"></a>model definition</h1>
<p>class Net(nn.Module):<br>
def <strong>init</strong>(self):<br>
super(Net, self).<strong>init</strong>()<br>
# Add the AdaptiveAvgPool2d layer<br>
self.avgpool = nn.AdaptiveAvgPool2d((1,1))</p>
<pre><code>    #convolutional layers
    self.conv1 = nn.Conv2d(3, 32, kernel_size=3, stride=1, padding=1)
    self.pool1 = nn.MaxPool2d(kernel_size=2, stride=2, padding=0)
    
    self.conv2 = nn.Conv2d(32, 64, kernel_size=3, stride=1, padding=1)
    self.pool2 = nn.MaxPool2d(kernel_size=2, stride=2, padding=0)
    
    self.conv3 = nn.Conv2d(64, 128, kernel_size=3, stride=1, padding=1)
    self.pool3 = nn.MaxPool2d(kernel_size=2, stride=2, padding=0)
    
    self.conv4 = nn.Conv2d(128, 256, kernel_size=3, stride=1, padding=1)
    self.pool4 = nn.MaxPool2d(kernel_size=2, stride=2, padding=0)
    
    #self.conv5 = nn.Conv2d(256, 512, kernel_size=3, stride=1, padding=1)
    #self.pool5 = nn.MaxPool2d(kernel_size=2, stride=2, padding=0)
    
   #dropout layers
    self.dropout1 = nn.Dropout2d(p=0.2)
    self.dropout2 = nn.Dropout2d(p=0.2)
    self.dropout3 = nn.Dropout2d(p=0.2)
    
    # fully connected layers
    self.fc1 = nn.Linear(256, 256)
    self.fc2 = nn.Linear(256, 1)

    # sigmoid activation
    self.sigmoid = nn.Sigmoid()

def forward(self, x):
    x = self.pool1(F.relu(self.conv1(x)))
    x = self.dropout1(x)
    x = self.pool2(F.relu(self.conv2(x)))
    x = self.dropout2(x)
    x = self.pool3(F.relu(self.conv3(x)))
    x = self.dropout3(x)
    x = self.pool4(F.relu(self.conv4(x)))
    x = self.avgpool(x) # apply adaptive average pooling
    x = x.view(-1, 256)
    x = F.relu(self.fc1(x))
    x = self.fc2(x)
    x = self.sigmoid(x)
    return x
</code></pre>
<h1>
<a name="create-an-instance-of-the-model-2" class="anchor" href="#create-an-instance-of-the-model-2"></a>create an instance of the model</h1>
<p>model = Net()<br>
model.to(device)</p>
<p>The error is no more but it keep on printing<br>
torch.Size([4, 1]) torch.Size([4, 1])<br>
torch.Size([4, 1]) torch.Size([4, 1])<br>
torch.Size([4, 1]) torch.Size([4, 1])<br>
torch.Size([4, 1]) torch.Size([4, 1])<br>
torch.Size([4, 1]) torch.Size([4, 1])<br>
torch.Size([4, 1]) torch.Size([4, 1])<br>
torch.Size([4, 1]) torch.Size([4, 1])<br>
torch.Size([4, 1]) torch.Size([4, 1])<br>
torch.Size([4, 1]) torch.Size([4, 1])<br>
torch.Size([4, 1]) torch.Size([4, 1])<br>
torch.Size([4, 1]) torch.Size([4,</p>

J_Johnson: <p>Bravo!</p>
<p>Now you can delete that <code>print</code> statement you added earlier.</p>

abuadam: <p>I just remember that, removed it  and re-run the code, but it yielded to an error<br>
epoch: 0 loss: -4.0401434898376465</p>
<hr>
<p>RuntimeError                              Traceback (most recent call last)<br>
~\AppData\Local\Temp\ipykernel_10448\3620899133.py in <br>
56             outputs = outputs.view(-1, 1) # reshape output<br>
57             test_loss += criterion(outputs, labels)<br>
—&gt; 58             test_loss.backward()<br>
59             optimizer.step()<br>
60             test_loss += test_loss.item()</p>
<p>~\anaconda3\lib\site-packages\torch_tensor.py in backward(self, gradient, retain_graph, create_graph, inputs)<br>
486                 inputs=inputs,<br>
487             )<br>
 → 488         torch.autograd.backward(<br>
489             self, gradient, retain_graph, create_graph, inputs=inputs<br>
490         )</p>
<p>~\anaconda3\lib\site-packages\torch\autograd_<em>init</em>_.py in backward(tensors, grad_tensors, retain_graph, create_graph, grad_variables, inputs)<br>
195     # some Python versions print out the first line of a multi-line function<br>
196     # calls in the traceback and some print out the last line<br>
 → 197     Variable.<em>execution_engine.run_backward(  # Calls into the C++ engine to run the backward pass<br>
198         tensors, grad_tensors</em>, retain_graph, create_graph, inputs,<br>
199         allow_unreachable=True, accumulate_grad=True)  # Calls into the C++ engine to run the backward pass</p>
<p>RuntimeError: element 0 of tensors does not require grad and does not have a grad_fn</p>

J_Johnson: <p>Please read here:</p>
<p><a href="https://pytorch.org/docs/stable/notes/autograd.html#in-place-operations-with-autograd" class="onebox" target="_blank" rel="noopener nofollow ugc">https://pytorch.org/docs/stable/notes/autograd.html#in-place-operations-with-autograd</a></p>
<p>And if you’re not clear what an “in-place operation” is, read here first:</p>
<p><a href="https://discuss.pytorch.org/t/what-is-in-place-operation">https://discuss.pytorch.org/t/what-is-in-place-operation</a></p>

abuadam: <p>Fixed. Thank you very much. How can I optimize the parameters using GWO?</p>