RuntimeError: size mismatch, m1: [8 x 73728], m2: [512 x 227] at C:/w/1/s/windows/pytorch/aten/src\THC/generic/THCTensorMathBlas.cu:290

Yaqi_CAI: <p>I am using re-resnet50 to do multi-label annottaion and the dataset is food101<br>
But when I was running</p>
<blockquote>
<p>model = senet.SENet50()</p>
<p>num_ftrs = model.linear.in_features # revising the categories classes<br>
model.linear= nn.Linear(num_ftrs, len(targets))</p>
<p>ct = 0<br>
for name, child in model.named_children():<br>
ct += 1<br>
if ct &lt; 8:<br>
for name2, params in child.named_parameters():<br>
params.requires_grad = False<br>
output = model(img_data)</p>
</blockquote>
<p>It came out the problem:<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/a/f/afecdabb0ce0684b283dde7cd6ae96394859b9e4.png" data-download-href="https://discuss.pytorch.org/uploads/default/afecdabb0ce0684b283dde7cd6ae96394859b9e4" title="image"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/a/f/afecdabb0ce0684b283dde7cd6ae96394859b9e4_2_303x500.png" alt="image" data-base62-sha1="p6j0XSaQW1C46VUnxa4Nz0hOP9G" width="303" height="500" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/a/f/afecdabb0ce0684b283dde7cd6ae96394859b9e4_2_303x500.png, https://discuss.pytorch.org/uploads/default/optimized/3X/a/f/afecdabb0ce0684b283dde7cd6ae96394859b9e4_2_454x750.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/a/f/afecdabb0ce0684b283dde7cd6ae96394859b9e4_2_606x1000.png 2x" data-dominant-color="F5F6F5"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">908×1498 38.3 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>Here is the model:<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/2/f/2f84fb78c1cc42cb6378e91af79092b6d654841d.png" data-download-href="https://discuss.pytorch.org/uploads/default/2f84fb78c1cc42cb6378e91af79092b6d654841d" title="image"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/2/f/2f84fb78c1cc42cb6378e91af79092b6d654841d_2_208x500.png" alt="image" data-base62-sha1="6MnlYjfm4tQ0QfzHLwacAitt0Tz" width="208" height="500" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/2/f/2f84fb78c1cc42cb6378e91af79092b6d654841d_2_208x500.png, https://discuss.pytorch.org/uploads/default/optimized/3X/2/f/2f84fb78c1cc42cb6378e91af79092b6d654841d_2_312x750.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/2/f/2f84fb78c1cc42cb6378e91af79092b6d654841d_2_416x1000.png 2x" data-dominant-color="F1F1F1"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">901×2158 38.1 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>Can anyone tell me how to fix the problem?</p>

Yaqi_CAI: <p>Besides, the RE-resnet original coda is here:<a href="https://github.com/kuangliu/pytorch-cifar/blob/master/models/senet.py" rel="nofollow noopener">https://github.com/kuangliu/pytorch-cifar/blob/master/models/senet.py</a></p>

ptrblck: <p>The code works fine for me:</p>
<pre><code class="lang-python">model =  SENet(PreActBlock, [2,2,2,2])

num_ftrs = model.linear.in_features # revising the categories classes
model.linear= nn.Linear(num_ftrs, 10)

out = model(torch.randn(1, 3, 32, 32))
print(out.shape)
&gt; torch.Size([1, 10])
</code></pre>
<p>PS: It’s always better to post code by wrapping it into three backticks ```, as it’s easier to debug and also the pictures are really hard to read. <img src="https://discuss.pytorch.org/images/emoji/apple/wink.png?v=9" title=":wink:" class="emoji" alt=":wink:"></p>

Yaqi_CAI: <p>Well, thanks for your reminding.<br>
But when I change the codes, the problem still exists：</p>
<blockquote>
<p>RuntimeError                              Traceback (most recent call last)<br>
 in <br>
25         img_data, target = img_data.to(device), target.to(device)<br>
26<br>
—&gt; 27         output = model(img_data) <span class="hashtag-raw">#FWD</span> prop<br>
28<br>
29         loss = criterion(output, target) <span class="hashtag-raw">#Cross</span> entropy loss</p>
<p>D:\pythonana\envs\pytorch\lib\site-packages\torch\nn\modules\module.py in <strong>call</strong>(self, *input, **kwargs)<br>
530             result = self._slow_forward(*input, **kwargs)<br>
531         else:<br>
 → 532             result = self.forward(*input, **kwargs)<br>
533         for hook in self._forward_hooks.values():<br>
534             hook_result = hook(self, input, result)</p>
<p>C:\yaqi_anaconda\senet.py in forward(self, x)<br>
102         out = F.avg_pool2d(out, 4)<br>
103         out = out.view(out.size(0), -1)<br>
 → 104         out = self.linear(out)<br>
105         return out<br>
106</p>
<p>D:\pythonana\envs\pytorch\lib\site-packages\torch\nn\modules\module.py in <strong>call</strong>(self, *input, **kwargs)<br>
530             result = self._slow_forward(*input, **kwargs)<br>
531         else:<br>
 → 532             result = self.forward(*input, **kwargs)<br>
533         for hook in self._forward_hooks.values():<br>
534             hook_result = hook(self, input, result)</p>
<p>D:\pythonana\envs\pytorch\lib\site-packages\torch\nn\modules\linear.py in forward(self, input)<br>
85<br>
86     def forward(self, input):<br>
—&gt; 87         return F.linear(input, self.weight, self.bias)<br>
88<br>
89     def extra_repr(self):</p>
<p>D:\pythonana\envs\pytorch\lib\site-packages\torch\nn\functional.py in linear(input, weight, bias)<br>
1368     if input.dim() == 2 and bias is not None:<br>
1369         # fused op is marginally faster<br>
 → 1370         ret = torch.addmm(bias, input, weight.t())<br>
1371     else:<br>
1372         output = input.matmul(weight.t())</p>
<p>RuntimeError: size mismatch, m1: [8 x 73728], m2: [512 x 10] at C:/w/1/s/windows/pytorch/aten/src\THC/generic/THCTensorMathBlas.cu:290</p>
</blockquote>

ptrblck: <p>Are you getting the error with my code snippet?<br>
If not, could you compare your current script with my examples and post the differences, so that we can have another look, please?</p>

Yaqi_CAI: <p>Yes, i just get the error with your code snippet.<br>
And here is the model now:</p>
<blockquote>
<p>SENet(<br>
(conv1): Conv2d(3, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)<br>
(bn1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(layer1): Sequential(<br>
(0): PreActBlock(<br>
(bn1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(conv1): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)<br>
(bn2): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(conv2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)<br>
(fc1): Conv2d(64, 4, kernel_size=(1, 1), stride=(1, 1))<br>
(fc2): Conv2d(4, 64, kernel_size=(1, 1), stride=(1, 1))<br>
)<br>
(1): PreActBlock(<br>
(bn1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(conv1): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)<br>
(bn2): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(conv2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)<br>
(fc1): Conv2d(64, 4, kernel_size=(1, 1), stride=(1, 1))<br>
(fc2): Conv2d(4, 64, kernel_size=(1, 1), stride=(1, 1))<br>
)<br>
)<br>
(layer2): Sequential(<br>
(0): PreActBlock(<br>
(bn1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(conv1): Conv2d(64, 128, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)<br>
(bn2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)<br>
(shortcut): Sequential(<br>
(0): Conv2d(64, 128, kernel_size=(1, 1), stride=(2, 2), bias=False)<br>
)<br>
(fc1): Conv2d(128, 8, kernel_size=(1, 1), stride=(1, 1))<br>
(fc2): Conv2d(8, 128, kernel_size=(1, 1), stride=(1, 1))<br>
)<br>
(1): PreActBlock(<br>
(bn1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(conv1): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)<br>
(bn2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)<br>
(fc1): Conv2d(128, 8, kernel_size=(1, 1), stride=(1, 1))<br>
(fc2): Conv2d(8, 128, kernel_size=(1, 1), stride=(1, 1))<br>
)<br>
)<br>
(layer3): Sequential(<br>
(0): PreActBlock(<br>
(bn1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(conv1): Conv2d(128, 256, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)<br>
(bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)<br>
(shortcut): Sequential(<br>
(0): Conv2d(128, 256, kernel_size=(1, 1), stride=(2, 2), bias=False)<br>
)<br>
(fc1): Conv2d(256, 16, kernel_size=(1, 1), stride=(1, 1))<br>
(fc2): Conv2d(16, 256, kernel_size=(1, 1), stride=(1, 1))<br>
)<br>
(1): PreActBlock(<br>
(bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(conv1): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)<br>
(bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)<br>
(fc1): Conv2d(256, 16, kernel_size=(1, 1), stride=(1, 1))<br>
(fc2): Conv2d(16, 256, kernel_size=(1, 1), stride=(1, 1))<br>
)<br>
)<br>
(layer4): Sequential(<br>
(0): PreActBlock(<br>
(bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(conv1): Conv2d(256, 512, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)<br>
(bn2): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(conv2): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)<br>
(shortcut): Sequential(<br>
(0): Conv2d(256, 512, kernel_size=(1, 1), stride=(2, 2), bias=False)<br>
)<br>
(fc1): Conv2d(512, 32, kernel_size=(1, 1), stride=(1, 1))<br>
(fc2): Conv2d(32, 512, kernel_size=(1, 1), stride=(1, 1))<br>
)<br>
(1): PreActBlock(<br>
(bn1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(conv1): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)<br>
(bn2): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(conv2): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)<br>
(fc1): Conv2d(512, 32, kernel_size=(1, 1), stride=(1, 1))<br>
(fc2): Conv2d(32, 512, kernel_size=(1, 1), stride=(1, 1))<br>
)<br>
)<br>
(linear): Linear(in_features=512, out_features=10, bias=True)<br>
)</p>
</blockquote>

Yaqi_CAI: <p>Here is my codes before:</p>
<blockquote>
<p>model = senet.SENet50()</p>
<p>num_ftrs = model.linear.in_features # revising the categories classes<br>
model.linear= nn.Linear(num_ftrs, len(targets))</p>
<p>ct = 0<br>
for name, child in model.named_children():<br>
ct += 1<br>
if ct &lt; 8:<br>
for name2, params in child.named_parameters():<br>
params.requires_grad = False<br>
output = model(img_data)</p>
</blockquote>
<p>And it came out the problem :RuntimeError: size mismatch, m1: [8 x 73728], m2: [512 x227]</p>
<p>But now after i change your codes:</p>
<blockquote>
<p>from senet import PreActBlock<br>
model =  SENet(PreActBlock, [2,2,2,2])</p>
<p>num_ftrs = model.linear.in_features # revising the categories classes<br>
model.linear= nn.Linear(num_ftrs, 10)</p>
<p>out = model(torch.randn(1, 3, 32, 32))</p>
<p>ct = 0<br>
for name, child in model.named_children(): <span class="hashtag-raw">#frozen</span> 7 layers<br>
ct += 1<br>
if ct &lt; 8:<br>
for name2, params in child.named_parameters():<br>
params.requires_grad = False</p>
</blockquote>
<blockquote>
<pre><code>for img_data, target in tqdm_notebook(train_loader, desc='Training'):    
    img_data, target = img_data.to(device), target.to(device)
    
    output = model(img_data) #FWD prop 
</code></pre>
</blockquote>
<p>it came out the same problem as I showed in the last reply.<br>
RuntimeError: size mismatch, m1: [8 x 73728], m2: [512 x 10]</p>
<p>Plus: my dataset is food101 and it has 101 classes.</p>

ptrblck: <p>Sorry, I cannot reproduce this error even with the parameter freezing.</p>

Yaqi_CAI: <p>OK. Thank you. Maybe I should change another method to optimize my result.</p>