ValueError: Expected input batch_size (150) to match target batch_size (50)

Sayeed_Shaiban: <blockquote>
<p>iter = 0<br>
for epoch in range(n_epoch):<br>
for i, (images, labels) in enumerate(train_loader):</p>
<pre><code>    #######################
    #  USE GPU FOR MODEL  #
    #######################
    if torch.cuda.is_available():
        images = Variable(images.view(-1, 28*28).cuda())
        labels = Variable(labels.cuda())
    else:
        images = Variable(images.view(-1, 28*28))
        labels = Variable(labels)
    
    # Clear gradients w.r.t. parameters
    optimizer.zero_grad()
    print(images.shape)
    # Forward pass to get output/logits
    outputs = model(images)
    print(outputs.shape)
    # Calculate Loss: softmax --&gt; cross entropy loss
    loss = criterion(outputs, labels)
    
    # Getting gradients w.r.t. parameters
    loss.backward()
    
    # Updating parameters
    optimizer.step()
    
    iter += 1
    
    if iter % 500 == 0:
        # Calculate Accuracy         
        correct = 0
        total = 0
        # Iterate through test dataset
        for images, labels in test_loader:
            #######################
            #  USE GPU FOR MODEL  #
            #######################
            images = Variable(images.view(-1, 12288).cuda())
            
            # Forward pass only to get logits/output
            outputs = model(images)
            
            # Get predictions from the maximum value
            _, predicted = torch.max(outputs.data, 1)
            
            # Total number of labels
            total += labels.size(0)
            
            #######################
            #  USE GPU FOR MODEL  #
            #######################
            # Total correct predictions
            correct += (predicted.cpu() == labels.cpu()).sum()
        
        accuracy = 100 * correct / total
        
        # Print Loss
        print('Iteration: {}. Loss: {}. Accuracy: {}'.format(iter, loss.data, accuracy))
</code></pre>
</blockquote>
<p>I’m new in pytorch. My model is initialized as such</p>
<blockquote>
<p>input_dim = 28*28<br>
output_dim = 2<br>
model = FeedForward_NN(input_dim,output_dim)</p>
</blockquote>
<p>I get a ValueError: Expected input batch_size (150) to match target batch_size (50)<br>
whats the reason for this?</p>

ptrblck: <p>Could you post the definition of your model so that we can have a look, please? <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=9" title=":slight_smile:" class="emoji" alt=":slight_smile:"></p>

Sayeed_Shaiban: <blockquote>
<p>class FeedForward_NN(nn.Module):<br>
def <strong>init</strong>(self,input_dim,output_dim):<br>
super(FeedForward_NN,self).<strong>init</strong>()</p>
<pre><code>    self.fc0 = nn.Linear(input_dim, output_dim)
    
def forward(self,x):

    out = self.fc0(x)
    return out
</code></pre>
</blockquote>

ptrblck: <p>Thanks for the code.<br>
It looks like this line of code is creating an invalid view:</p>
<pre><code class="lang-python">images = Variable(images.view(-1, 12288).cuda())
</code></pre>
<p>Based on the model definition and the training loop, each input should have a shape of <code>[batch_size, 28*28=784]</code>.</p>
<p>Also, <code>Variables</code> are deprecated since PyTorch <code>0.4.0</code> so you can simply use tensors now. <img src="https://discuss.pytorch.org/images/emoji/apple/wink.png?v=9" title=":wink:" class="emoji" alt=":wink:"></p>

Sayeed_Shaiban: <p>ahh. Sorry, I changed the code while trying to figure out the error. The error exists nonetheless. Even after changing it to</p>
<blockquote>
<p>images = Variable(images.view(-1, 28*28).cuda())</p>
</blockquote>

ptrblck: <p>Could you print the shapes of your input as well as the target before passing them to the model, please?</p>

Sayeed_Shaiban: <p>So these are my parameters shape</p>
<blockquote>
<p>print(model.parameters())</p>
<p>print(len(list(model.parameters())))</p>
<p>print(list(model.parameters())[0].size())</p>
<p>print(list(model.parameters())[1].size())</p>
</blockquote>
<p>output:</p>
<pre><code class="lang-auto">&lt;generator object Module.parameters at 0x7fc5eb5a6fc0&gt;
2
torch.Size([2, 784])
torch.Size([2])
</code></pre>
<p>The shape it takes after using DataLoader:</p>
<blockquote>
<p>for i, (images, labels) in enumerate(train_loader):<br>
print(images.shape)</p>
</blockquote>
<p>output:<br>
This goes on for 36 times</p>
<pre><code class="lang-auto">torch.Size([50, 3, 28, 28])
torch.Size([50, 3, 28, 28])
torch.Size([50, 3, 28, 28])
torch.Size([50, 3, 28, 28])
torch.Size([50, 3, 28, 28])
torch.Size([50, 3, 28, 28])
torch.Size([50, 3, 28, 28])
torch.Size([50, 3, 28, 28])
torch.Size([50, 3, 28, 28])
torch.Size([50, 3, 28, 28])
</code></pre>
<blockquote>
<p>images = Variable(images.view(-1, 28*28).cuda())<br>
image size after the above line<br>
output:<br>
<code>torch.Size([150, 784])</code></p>
</blockquote>
<blockquote>
<p>outputs = model(images)<br>
outputs size after the above line<br>
output:<br>
torch.Size([150, 2])</p>
</blockquote>

ptrblck: <p>Thanks for the information.<br>
The input shapes don’t match your <code>view</code> call inside the model, as you are pushing the 3 channels to the batch dimension.<br>
Could you use <code>images.view(images.shape(0), -1)</code> to create the views?<br>
This will create an input tensor of <code>[batch_size, 28*28*3 = 2352]</code>, so that you would also need to adapt the input features of your linear layer.</p>
<p>Also, based on your initial description it seems you expect to deal with single-channel images, so are you transforming them to RGB somewhere?</p>

Sayeed_Shaiban: <p>so the images where originally in torch.Size([28, 28, 3]). But as pytorch takes input in [channels, height, width]. I changed it to torch.Size([3, 28, 28]) using the permute() function.</p>
<p>And using <code>images.view(images.shape(0), -1)</code> gives the following error</p>
<blockquote>
<p>TypeError                                 Traceback (most recent call last)<br>
 in ()<br>
7         #######################<br>
8         if torch.cuda.is_available():<br>
9             images = Variable(images.view(images.shape(0),-1).cuda())<br>
10             labels = Variable(labels.cuda())<br>
11         else:</p>
<p>TypeError: ‘torch.Size’ object is not callable</p>
</blockquote>

ptrblck: <p>Oh, my bad. Use <code>images.size(0)</code> instead of <code>.shape</code>.</p>

Sayeed_Shaiban: <p>I get a cuda error if I do that <img src="https://discuss.pytorch.org/images/emoji/apple/confused.png?v=12" title=":confused:" class="emoji" alt=":confused:" loading="lazy" width="20" height="20"></p>
<blockquote>
<hr>
<p>RuntimeError                              Traceback (most recent call last)<br>
 in <br>
23<br>
24         # Getting gradients w.r.t. parameters<br>
—&gt; 25         loss.backward()<br>
26<br>
27         # Updating parameters</p>
<p>~/anaconda3/lib/python3.7/site-packages/torch/tensor.py in backward(self, gradient, retain_graph, create_graph)<br>
164                 products. Defaults to <code>False</code>.<br>
165         “”"<br>
 → 166         torch.autograd.backward(self, gradient, retain_graph, create_graph)<br>
167<br>
168     def register_hook(self, hook):</p>
<p>~/anaconda3/lib/python3.7/site-packages/torch/autograd/<strong>init</strong>.py in backward(tensors, grad_tensors, retain_graph, create_graph, grad_variables)<br>
97     Variable._execution_engine.run_backward(<br>
98         tensors, grad_tensors, retain_graph, create_graph,<br>
—&gt; 99         allow_unreachable=True)  # allow_unreachable flag<br>
100<br>
101</p>
<p>RuntimeError: CUDA error: device-side assert triggered</p>
</blockquote>
<p>Also I’m giving my input dimensions as such:</p>
<pre><code class="lang-auto">input_dim = 28*28*3
output_dim = 2
model = FeedForward_NN(input_dim,output_dim)
</code></pre>

ptrblck: <p>Could you run the code on the CPU and check the error message, please?<br>
This should give you a better error.<br>
If the code runs fine on the CPU, use <code>CUDA_LAUNCH_BLOCKING=1 python setup.py args</code> to rerun the script on the GPU again and post the stack trace here.</p>

Sayeed_Shaiban: <pre><code class="lang-auto">---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
&lt;ipython-input-21-874ab4eae012&gt; in &lt;module&gt;
     20         print(outputs.shape)
     21         # Calculate Loss: softmax --&gt; cross entropy loss
---&gt; 22         loss = criterion(outputs, labels)
     23 
     24         # Getting gradients w.r.t. parameters

~/anaconda3/lib/python3.7/site-packages/torch/nn/modules/module.py in __call__(self, *input, **kwargs)
    539             result = self._slow_forward(*input, **kwargs)
    540         else:
--&gt; 541             result = self.forward(*input, **kwargs)
    542         for hook in self._forward_hooks.values():
    543             hook_result = hook(self, input, result)

~/anaconda3/lib/python3.7/site-packages/torch/nn/modules/loss.py in forward(self, input, target)
    914     def forward(self, input, target):
    915         return F.cross_entropy(input, target, weight=self.weight,
--&gt; 916                                ignore_index=self.ignore_index, reduction=self.reduction)
    917 
    918 

~/anaconda3/lib/python3.7/site-packages/torch/nn/functional.py in cross_entropy(input, target, weight, size_average, ignore_index, reduce, reduction)
   2007     if size_average is not None or reduce is not None:
   2008         reduction = _Reduction.legacy_get_string(size_average, reduce)
-&gt; 2009     return nll_loss(log_softmax(input, 1), target, weight, None, ignore_index, None, reduction)
   2010 
   2011 

~/anaconda3/lib/python3.7/site-packages/torch/nn/functional.py in nll_loss(input, target, weight, size_average, ignore_index, reduce, reduction)
   1836                          .format(input.size(0), target.size(0)))
   1837     if dim == 2:
-&gt; 1838         ret = torch._C._nn.nll_loss(input, target, weight, _Reduction.get_enum(reduction), ignore_index)
   1839     elif dim == 4:
   1840         ret = torch._C._nn.nll_loss2d(input, target, weight, _Reduction.get_enum(reduction), ignore_index)

RuntimeError: Assertion `cur_target &gt;= 0 &amp;&amp; cur_target &lt; n_classes' failed.  at /opt/conda/conda-bld/pytorch_1573049306803/work/aten/src/THNN/generic/ClassNLLCriterion.c:97
</code></pre>
<p>I get the above error on cpu</p>

ptrblck: <p>Thanks for the error message.<br>
It seems your targets are out of bounds.<br>
For 2 output units, your target should be a <code>LongTensor</code> with the shape <code>[batch_size]</code> containing class indices in <code>[0, 1]</code>.</p>

Sayeed_Shaiban: <p>This is how my labels look</p>
<pre><code class="lang-auto">tensor([1, 2, 2, 1, 1, 2, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2, 1, 1, 2, 1, 2, 2, 2,
        2, 1, 1, 2, 2, 2, 2, 1, 2, 1, 1, 1, 1, 2, 1, 2, 1, 1, 2, 2, 2, 1, 1, 2,
        2, 2])
</code></pre>
<p>And the shapes are</p>
<blockquote>
<p>labels<br>
torch.Size([50])<br>
images<br>
torch.Size([50, 2352])<br>
outputs<br>
torch.Size([50, 2])</p>
</blockquote>
<p>Seems to have the same size as the batch_size</p>

ptrblck: <p>The shape seems to be correct, but the labels should be zeros and ones.</p>

Sayeed_Shaiban: <p>Yes It worked!! Thank you so much for all the help</p>