untimeError: The expanded size of the tensor (32) must match the existing size (8) at non-singleton dimension 1

Daniel_Joseph: <p>I have got this error while running my code, the tensor shape is (BatchXchannelsXHightXWidth) [8,204,15,15],</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/2X/e/ebc513420c5961fd24c9585a052ab8f5a0b3846c.png" data-download-href="https://discuss.pytorch.org/uploads/default/ebc513420c5961fd24c9585a052ab8f5a0b3846c" title="erreeer"><img src="https://discuss.pytorch.org/uploads/default/optimized/2X/e/ebc513420c5961fd24c9585a052ab8f5a0b3846c_2_690x193.png" alt="erreeer" data-base62-sha1="xDIr6UkXh3s1C1OOcT1YxahLfHK" width="690" height="193" srcset="https://discuss.pytorch.org/uploads/default/optimized/2X/e/ebc513420c5961fd24c9585a052ab8f5a0b3846c_2_690x193.png, https://discuss.pytorch.org/uploads/default/optimized/2X/e/ebc513420c5961fd24c9585a052ab8f5a0b3846c_2_1035x289.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/2X/e/ebc513420c5961fd24c9585a052ab8f5a0b3846c_2_1380x386.png 2x" data-dominant-color="2C2E2E"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">erreeer</span><span class="informations">1394×391 417 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>However, it worked perfectly for the same image with different height and width [8,204,11,11].<br>
Thanks in advance</p>

ptrblck: <p>Could you post the shapes of <code>target</code> and <code>pred</code> before trying to calculate <code>correct</code>?</p>

Daniel_Joseph: <p>before correct the shape of the target is [32,1] and pred is [1,128]<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/2X/2/2afd9bb6d60fc853968324fb2cf9a16a033a95d7.png" data-download-href="https://discuss.pytorch.org/uploads/default/2afd9bb6d60fc853968324fb2cf9a16a033a95d7" title="ptrck"><img src="https://discuss.pytorch.org/uploads/default/optimized/2X/2/2afd9bb6d60fc853968324fb2cf9a16a033a95d7_2_690x249.png" alt="ptrck" data-base62-sha1="68jpeiRLW9FKvL3ruAm8ZF6Hav5" width="690" height="249" srcset="https://discuss.pytorch.org/uploads/default/optimized/2X/2/2afd9bb6d60fc853968324fb2cf9a16a033a95d7_2_690x249.png, https://discuss.pytorch.org/uploads/default/optimized/2X/2/2afd9bb6d60fc853968324fb2cf9a16a033a95d7_2_1035x373.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/2X/2/2afd9bb6d60fc853968324fb2cf9a16a033a95d7_2_1380x498.png 2x" data-dominant-color="292A2A"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">ptrck</span><span class="informations">1849×669 121 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

Daniel_Joseph: <p>Any suggestion to solve it !?</p>

ptrblck: <p>I’m not sure, why the target and pred tensors have such a different shape.<br>
Could you explain your use case a bit and how the predictions correspond to the target?</p>

Daniel_Joseph: <p>The original datasets size is  [103x11x11] or [103x15x15] same datasets just different image sizes, it has 9 classes.<br>
In “Train()” you can see the size of both tensors "output [400 9] , after FC.layer  " &amp; “target [32 1] which is [batchSz 1]”<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/2X/7/73df0af97509ce3a3f23fb7b8280dcd4e24ad910.png" data-download-href="https://discuss.pytorch.org/uploads/default/73df0af97509ce3a3f23fb7b8280dcd4e24ad910" title="001"><img src="https://discuss.pytorch.org/uploads/default/optimized/2X/7/73df0af97509ce3a3f23fb7b8280dcd4e24ad910_2_690x67.png" alt="001" data-base62-sha1="gx2M7SLW3GQ7oLBa9WWYyvBkH4s" width="690" height="67" srcset="https://discuss.pytorch.org/uploads/default/optimized/2X/7/73df0af97509ce3a3f23fb7b8280dcd4e24ad910_2_690x67.png, https://discuss.pytorch.org/uploads/default/optimized/2X/7/73df0af97509ce3a3f23fb7b8280dcd4e24ad910_2_1035x100.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/2X/7/73df0af97509ce3a3f23fb7b8280dcd4e24ad910_2_1380x134.png 2x" data-dominant-color="2E2D2A"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">001</span><span class="informations">1729×169 31.4 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div><br>
The mismatching size error occurs in the highlighted line while trying to reshape both outputs, target into the same shape  :<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/2X/0/0107de4bf8bd9c5832ac943a1c316948e77494f3.png" data-download-href="https://discuss.pytorch.org/uploads/default/0107de4bf8bd9c5832ac943a1c316948e77494f3" title="002"><img src="https://discuss.pytorch.org/uploads/default/optimized/2X/0/0107de4bf8bd9c5832ac943a1c316948e77494f3_2_690x153.png" alt="002" data-base62-sha1="97kMZtsTfr3DztOQAtrQhqTBWb" width="690" height="153" srcset="https://discuss.pytorch.org/uploads/default/optimized/2X/0/0107de4bf8bd9c5832ac943a1c316948e77494f3_2_690x153.png, https://discuss.pytorch.org/uploads/default/optimized/2X/0/0107de4bf8bd9c5832ac943a1c316948e77494f3_2_1035x229.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/2X/0/0107de4bf8bd9c5832ac943a1c316948e77494f3_2_1380x306.png 2x" data-dominant-color="2B2A29"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">002</span><span class="informations">1770×395 62.7 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div><br>
So what I wanna do exactly is finding a way in which flattening step before the FC should be of the form  [batchSz -1]</p>
<pre><code class="lang-auto">
        self.ap = nn.AvgPool2d(2)
        self.fc1 = nn.Linear('***', 1024)  '***: i don't know what to set as an input here '
        self.relu = nn.ReLU(inplace=True)
        self.dropout = nn.Dropout(p=0.5)
        self.fc2 = nn.Linear(1024, 9 )

    def forward(self, x):
        x = self.model(x)
        x = self.ap(x)
        x = x.view("**", -1)  " **: i wanna make it [batch_size -1] "
        x = self.fc1(x)
        x = self.relu(x)
        x = self.dropout(x)
        x = self.fc2(x)
        return x
</code></pre>

Daniel_Joseph: <p><a class="mention" href="/u/ptrblck">@ptrblck</a> Thanks for your help <img src="https://discuss.pytorch.org/images/emoji/apple/blush.png?v=9" title=":blush:" class="emoji" alt=":blush:"></p>

ptrblck: <p>Add a print statement in your <code>forward</code> method so see the shape of <code>x</code>:</p>
<pre><code class="lang-python">    def forward(self, x):
        x = self.model(x)
        x = self.ap(x)
        x = x.view(x.size(0), -1)
        print(x.shape)
        x = self.fc1(x)
        x = self.relu(x)
        x = self.dropout(x)
        x = self.fc2(x)
        return x
</code></pre>
<p>and just set the <code>in_features</code> of <code>self.fc1</code> to a temporary number.<br>
After the first forward pass, use the printed shape to set the number of input features for your linear layer.</p>
<p>Alternatively you could also calculate the feature size using the whole model architecture.</p>
<p>Also, if you are dealing with varying sized inputs, you might want to add an adaptive pooling layer, which will always yield the same output size.</p>

Ikram_elhattab: <p>I have got this error while running my code</p>
<pre><code class="lang-auto">Traceback (most recent call last):
  File "train.py", line 622, in &lt;module&gt;
    train(hyp, opt, device, tb_writer, wandb)
  File "train.py", line 79, in train
    model = Model(opt.cfg or ckpt['model'].yaml, ch=3, nc=nc).to(device)  # create
  File "/content/drive/MyDrive/yolov3_v0/models/yolo.py", line 97, in __init__
    m.stride = torch.tensor([s / x.shape[-2] for x in self.forward(torch.zeros(1, ch, s, s),torch.zeros(1, ch, s, s))])  # forward
  File "/content/drive/MyDrive/yolov3_v0/models/yolo.py", line 174, in forward
    out = self.model[32:33](torch.unsqueeze((out6),dim=0))
  File "/usr/local/lib/python3.7/dist-packages/torch/nn/modules/module.py", line 889, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/usr/local/lib/python3.7/dist-packages/torch/nn/modules/container.py", line 119, in forward
    input = module(input)
  File "/usr/local/lib/python3.7/dist-packages/torch/nn/modules/module.py", line 889, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/content/drive/MyDrive/yolov3_v0/models/yolo.py", line 50, in forward
    out[i] = self.m[i](out[i])  # conv
RuntimeError: The expanded size of the tensor (256) must match the existing size (27) at non-singleton dimension 1.  Target sizes: [1, 256, 32, 32].  Tensor sizes: [27, 32, 32]
</code></pre>
<p>and my code :</p>
<pre><code class="lang-auto">            out_rgb = self.model[0:3](in_rgb)
            print("out_rgb", out_rgb.shape)


            out_ther = self.model[3:6](in_ther)
            print("out_ther", out_ther.shape)

            in_cat = self.model[6:7]((out_rgb,out_ther))
            print("in-cat", in_cat.shape)


            out11 = self.model[7:11](in_cat) 
            print("out11", out11.shape)

            out1 = self.model[11:13](out11) 
            print("out1", out1.shape)


            out2 = self.model[13:22](out1) 
            print("out2", out2.shape)

            out3 = self.model[22:23]((out1,out2))
            print("out3", out3.shape)

            out4 = self.model[23:29](out3)
            print("out4",out4.shape)

            out_cat3 = self.model[22:23]((out11,out4))
            print("out_cat3", out_cat3.shape)

            out5 = self.model[29:30](torch.unsqueeze((out_cat3),dim=0))
            print("out5",out5.shape)

            out6 = self.model[30:32](out5)
            print("out6",out6.shape)
          
            out = self.model[32:33](torch.unsqueeze((out6),dim=0))

            print("out",out.shape)

           return output 

</code></pre>
<p>Shapes :</p>
<pre><code class="lang-auto">out_rgb torch.Size([1, 64, 128, 128])
out_ther torch.Size([1, 64, 128, 128])
in-cat torch.Size([1, 128, 128, 128])
out11 torch.Size([1, 256, 32, 32])
out1 torch.Size([1, 512, 16, 16])
out2 torch.Size([1, 256, 16, 16])
out3 torch.Size([1, 768, 16, 16])
out4 torch.Size([1, 128, 32, 32])
out_cat3 torch.Size([1, 384, 32, 32])
out5 torch.Size([1, 384, 32, 32])
out6 torch.Size([1, 256, 32, 32])
</code></pre>
<p>Thanks in advance</p>

ptrblck: <p>Based on the error message it seems you are trying to <code>expand</code> a tensor with an invalid shape.<br>
This code snippet would reproduce the issue:</p>
<pre><code class="lang-python">x = torch.randn(27, 32, 32)
y = x.expand(1, 256, 32, 32)
&gt; RuntimeError: The expanded size of the tensor (256) must match the existing size (27) at non-singleton dimension 1.  Target sizes: [1, 256, 32, 32].  Tensor sizes: [27, 32, 32]
</code></pre>
<p>You could check your model implementation for these <code>expand</code> calls and make sure it’s using a valid shape.</p>