RuntimeError: size mismatch, m1: [4 x 21168], m2: [28224 x 3] at /pytorch/aten/src/THC/generic/THCTensorMathBlas.cu:290

neerajshukla1911: <p>I m trying to write a simple custom neural network for the image classification task. While running model I am getting the runtime error. I changed linear layer input either 21168/28224 but the issue still exists.</p>
<pre><code class="lang-auto">import torch
import torch.nn as nn
import torch.nn.functional as F

class Model(nn.Module):
  def __init__(self):
        super(Model, self).__init__()
        # first layer
        self.conv1 = nn.Conv2d(3, 20, 1)
        nn.init.kaiming_uniform_(self.conv1.weight, nonlinearity='relu')
        self.act1 = nn.ReLU()
        self.pool1 = nn.MaxPool2d((2,2), stride=(2,2))

        # second layer
        self.conv2 = nn.Conv2d(3, 40, 1)
        nn.init.kaiming_uniform_(self.conv2.weight, nonlinearity='relu')
        self.act2 = nn.ReLU()
        self.pool2 = nn.MaxPool2d((2,2), stride=(2,2))

        # third layer
        self.conv3 = nn.Conv2d(3, 40, 3)
        nn.init.kaiming_uniform_(self.conv2.weight, nonlinearity='relu')
        self.act3 = nn.ReLU()
        self.pool3 = nn.MaxPool2d((2,2), stride=(2,2))

        # fourth layer
        self.conv4 = nn.Conv2d(3, 40, 5)
        nn.init.kaiming_uniform_(self.conv2.weight, nonlinearity='relu')
        self.act4 = nn.ReLU()
        self.pool4 = nn.MaxPool2d((2,2), stride=(2,2))

        # self.fc1 = nn.Linear(20*110*110, 3)
        # self.fc1 = nn.Linear(141120, 3)
        self.fc1 = nn.Linear(28224, 3)
        

  def forward(self, x):
       x1 = self.conv1(x)
       x1 = self.act1(x)
       x1 = self.pool1(x)

       x2 = self.conv2(x1)
       x2 = self.act2(x1)
       x2 = self.pool2(x1)

       x3 = self.conv3(x1)
       x3 = self.act3(x1)
       x3 = self.pool3(x1)

       x4 = self.conv4(x1)
       x4 = self.act4(x1)
       x4 = self.pool4(x1)

       x = torch.cat([x2.view(-1, 4), x3.view(-1, 4), x4.view(-1, 4)], -2)


       x = x.view(4, -1)
       x = F.relu(self.fc1(x))
       return F.log_softmax(x)

criterion = nn.CrossEntropyLoss()

# Observe that all parameters are being optimized

optimizer_ft = optim.SGD(model_ft.parameters(), lr=0.001, momentum=0.9)

# Decay LR by a factor of 0.1 every 7 epochs

exp_lr_scheduler = lr_scheduler.StepLR(optimizer_ft, step_size=7, gamma=0.1)

model = Model()
device = torch.device("cuda:0" if torch.cuda.is_available() else "cpu")
model = model.to(device)

model_tr = train_model(model, criterion, optimizer_ft, exp_lr_scheduler, 4)
</code></pre>
<p>Following is the stack trace:</p>
<pre><code class="lang-auto">RuntimeError                              Traceback (most recent call last)
&lt;ipython-input-38-8d53b5624a06&gt; in &lt;module&gt;()
----&gt; 1 model_tr = train_model(model, criterion, optimizer_ft, exp_lr_scheduler, 4)

5 frames
&lt;ipython-input-14-1a5d4f162548&gt; in train_model(model, criterion, optimizer, scheduler, num_epochs)
     30                 # track history if only in train
     31                 with torch.set_grad_enabled(phase == 'train'):
---&gt; 32                     outputs = model(inputs)
     33                     _, preds = torch.max(outputs, 1)
     34                     loss = criterion(outputs, labels)

/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py in __call__(self, *input, **kwargs)
    530             result = self._slow_forward(*input, **kwargs)
    531         else:
--&gt; 532             result = self.forward(*input, **kwargs)
    533         for hook in self._forward_hooks.values():
    534             hook_result = hook(self, input, result)

&lt;ipython-input-36-2714cf86e78a&gt; in forward(self, x)
     56 
     57        x = x.view(4, -1)
---&gt; 58        x = F.relu(self.fc1(x))
     59        return F.log_softmax(x)

/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py in __call__(self, *input, **kwargs)
    530             result = self._slow_forward(*input, **kwargs)
    531         else:
--&gt; 532             result = self.forward(*input, **kwargs)
    533         for hook in self._forward_hooks.values():
    534             hook_result = hook(self, input, result)

/usr/local/lib/python3.6/dist-packages/torch/nn/modules/linear.py in forward(self, input)
     85 
     86     def forward(self, input):
---&gt; 87         return F.linear(input, self.weight, self.bias)
     88 
     89     def extra_repr(self):

/usr/local/lib/python3.6/dist-packages/torch/nn/functional.py in linear(input, weight, bias)
   1368     if input.dim() == 2 and bias is not None:
   1369         # fused op is marginally faster
-&gt; 1370         ret = torch.addmm(bias, input, weight.t())
   1371     else:
   1372         output = input.matmul(weight.t())

RuntimeError: size mismatch, m1: [4 x 21168], m2: [28224 x 3] at /pytorch/aten/src/THC/generic/THCTensorMathBlas.cu:290
</code></pre>
<p>Can someone please help ?</p>

spacemeerkat: <aside class="quote no-group quote-modified" data-username="neerajshukla1911" data-post="1" data-topic="77066">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/neerajshukla1911/48/22932_2.png" class="avatar"> neerajshukla1911:</div>
<blockquote>
<p>x = torch.cat([x1.view(-1, 4), x2.view(-1, 4), x3.view(-1, 4)], -2)</p>
</blockquote>
</aside>
<p>Not quite answering the question you have but this line above replaces the output of the first convolution layer:</p>
<pre><code class="lang-auto">x = self.conv1(x)
x = self.act1(x)
x = self.pool1(x)
</code></pre>
<p>Is this a desired approach?</p>

neerajshukla1911: <p>Its desired approach.</p>

neerajshukla1911: <p>Updated post with some correction in code but issue stills exists</p>