Not working for larger batch size

username: dpetrini
cooked: <p>Hi,<br>
I am doing transfer learning from large network based in Resnet.<br>
It is related to image classification that outputs labels.</p>
<p>When I feed input with one image and corresponding target as below:</p>
<pre><code class="lang-auto">torch.Size([1, 1, 2500, 1900]) 
tensor([[0., 1.]], device='cuda:1') torch.Size([1, 2])

</code></pre>
<p>I have correct output like:</p>
<pre><code class="lang-auto">outputs: tensor([[[-0.4243, -1.0620],
                  [-0.0913, -2.4389]]], device='cuda:1', grad_fn=&lt;SliceBackward&gt;) torch.Size([1, 2, 2])

</code></pre>
<p>When I input a 2 image batch (using Dataloader)<br>
The input/target are like that:</p>
<pre><code class="lang-auto">torch.Size([2, 1, 2500, 1900]) 
tensor([[1., 0.], [1., 0.]], device='cuda:1') torch.Size([2, 2])

</code></pre>
<p>And I get following output:</p>
<pre><code class="lang-auto">outputs: tensor([[[-0.2792, -1.4123],
                  [-0.0257, -3.6729]],

                 [[-0.2532, -1.4974],
                  [-0.0518, -2.9860]]], device='cuda:1', grad_fn=&lt;SliceBackward&gt;) torch.Size([2, 2, 2])

</code></pre>
<p>–&gt; Correct too.</p>
<p>But after that I got an error when I try input/target:</p>
<pre><code class="lang-auto">torch.Size([3, 1, 2500, 1900]) 
tensor([[0., 1.],
        [0., 1.],
        [1., 0.]], device='cuda:1') torch.Size([3, 2])

</code></pre>
<p>The output tensor becomes:</p>
<pre><code class="lang-auto">outputs: tensor([[[-0.1289, -2.1125],
                  [-0.0123, -4.4008]],

                 [[-0.2694, -1.4434],
                  [-0.0322, -3.4504]]], device='cuda:1', grad_fn=&lt;SliceBackward&gt;) torch.Size([2, 2, 2])

</code></pre>
<p>Which is not 3 in first dimension as I expected, and happens an error message below:</p>
<p>/home/dpetrini/.local/lib/python3.6/site-packages/torch/nn/modules/loss.py:498: UserWarning: Using a target size (torch.Size([3, 2])) that is different to the input size (torch.Size([2, 2])) is deprecated. Please ensure they have the same size.<br>
return F.binary_cross_entropy(input, target, weight=self.weight, reduction=self.reduction)<br>
Traceback (most recent call last):<br>
File “src/modeling/train_maca.py”, line 593, in <br>
main()<br>
File “src/modeling/train_maca.py”, line 553, in main<br>
loss_func, optimizer, device, mini_batch, num_epochs)<br>
File “src/modeling/train_maca.py”, line 97, in train_and_validate<br>
loss = loss_criterion(y_hat, labels)    # compute loss<br>
File “/home/dpetrini/.local/lib/python3.6/site-packages/torch/nn/modules/module.py”, line 547, in <strong>call</strong><br>
result = self.forward(*input, **kwargs)<br>
File “/home/dpetrini/.local/lib/python3.6/site-packages/torch/nn/modules/loss.py”, line 498, in forward<br>
return F.binary_cross_entropy(input, target, weight=self.weight, reduction=self.reduction)<br>
File “/home/dpetrini/.local/lib/python3.6/site-packages/torch/nn/functional.py”, line 2044, in binary_cross_entropy<br>
“!= input nelement ({})”.format(target.numel(), input.numel()))<br>
ValueError: Target and input must have the same number of elements. target nelement (6) != input nelement (4)</p>
<p>The model is a Resnet plus the layers below:</p>
<pre><code class="lang-auto">  (fc1): Linear(in_features=256, out_features=256, bias=True)
  (output_layer): OutputLayer(
    (fc_layer): Linear(in_features=256, out_features=4, bias=True)
  )
</code></pre>
<p>I appreciate any help or suggestion. It prevents me to better use GPU filling one or two more samples for each batch and making it faster.<br>
Thanks</p>

username: albanD
cooked: <p>Hi,</p>
<p>I think you have some extra code between the model that you give (where the last layer is a linear that outputs 4 features so batch x 4) and the size you print (which are batch x 2 x 2)<br>
You should make sure that this extra code has no part that is hardcoded that ignore part of the input.</p>

username: dpetrini
cooked: <p>Hi,<br>
I see also in model declaration:</p>
<pre><code class="lang-auto">layers.OutputLayer(256, (2, 2))
</code></pre>
<p>I may explain the (batch x 2 x 2).<br>
I got first question layers print from: “print(model)” and it actually is not very precise, as you point out.</p>
<p>Is there any way to print model with more detail?<br>
Thanks,</p>

username: albanD
cooked: <p>The user can create any model he wants so there is not one way to print the model.<br>
The best way is to look at the code of the forward function to make sure what happens <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=9" title=":slight_smile:" class="emoji" alt=":slight_smile:"></p>

username: dpetrini
cooked: <p>I see, I think I get the point:<br>
The model itself has this forward:</p>
<pre><code class="lang-auto">    def forward(self, x):
        result = self.view_resnet(x)
        h = F.relu(self.fc1(result))
        h = self.output_layer(h)[:2]
        return h
</code></pre>
<p>And output_layer by its turn has the following forward:</p>
<pre><code class="lang-auto">    def forward(self, x):
        h = self.fc_layer(x)
        if len(self.output_shape) &gt; 1:
            h = h.view(h.shape[0], *self.output_shape)  
        h = F.log_softmax(h, dim=-1)
        return h
</code></pre>
<p>So it seems that <strong>h = self.output_layer(h)[:2]</strong> in top model is preventing from getting 3 outputs or more. I changed to <strong>h = self.output_layer(h)</strong> and is just working.</p>
<p>Thank you!</p>