RuntimeError: mat1 and mat2 shapes cannot be multiplied (5376x28 and 784x512)

username: thesekyi
cooked: <pre><code class="lang-auto">class Baseline(nn.Module):
    def __init__(self):
        super().__init__()
        # 5 Hidden Layer Network
        self.fc1 = nn.Linear(28 * 28, 512)
        self.fc2 = nn.Linear(512, 256)
        self.fc3 = nn.Linear(256, 128)
        self.fc4 = nn.Linear(128, 64)
        self.fc5 = nn.Linear(64, 3)

        # Dropout module with 0.2 probbability
        self.dropout = nn.Dropout(p=0.2)
        # Add softmax on output layer
        self.log_softmax = F.log_softmax

    def forward(self, x):
        x = self.dropout(F.relu(self.fc1(x)))
        x = self.dropout(F.relu(self.fc2(x)))
        x = self.dropout(F.relu(self.fc3(x)))
        x = self.dropout(F.relu(self.fc4(x)))

        x = self.log_softmax(self.fc5(x), dim=1)

        return x
</code></pre>
<p>Error:</p>
<pre><code class="lang-auto">---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
&lt;ipython-input-48-0030d9c3852c&gt; in &lt;module&gt;
     18         optimizer.zero_grad()
     19         # Make predictions
---&gt; 20         log_ps = model(images)
     21         loss = criterion(log_ps, labels)
     22         #backprop

~/anaconda3/envs/torch/lib/python3.8/site-packages/torch/nn/modules/module.py in _call_impl(self, *input, **kwargs)
    725             result = self._slow_forward(*input, **kwargs)
    726         else:
--&gt; 727             result = self.forward(*input, **kwargs)
    728         for hook in itertools.chain(
    729                 _global_forward_hooks.values(),

&lt;ipython-input-46-09dd06cd0a72&gt; in forward(self, x)
     15 
     16     def forward(self, x):
---&gt; 17         x = self.dropout(F.relu(self.fc1(x)))
     18         x = self.dropout(F.relu(self.fc2(x)))
     19         x = self.dropout(F.relu(self.fc3(x)))

~/anaconda3/envs/torch/lib/python3.8/site-packages/torch/nn/modules/module.py in _call_impl(self, *input, **kwargs)
    725             result = self._slow_forward(*input, **kwargs)
    726         else:
--&gt; 727             result = self.forward(*input, **kwargs)
    728         for hook in itertools.chain(
    729                 _global_forward_hooks.values(),

~/anaconda3/envs/torch/lib/python3.8/site-packages/torch/nn/modules/linear.py in forward(self, input)
     91 
     92     def forward(self, input: Tensor) -&gt; Tensor:
---&gt; 93         return F.linear(input, self.weight, self.bias)
     94 
     95     def extra_repr(self) -&gt; str:

~/anaconda3/envs/torch/lib/python3.8/site-packages/torch/nn/functional.py in linear(input, weight, bias)
   1690         ret = torch.addmm(bias, input, weight.t())
   1691     else:
-&gt; 1692         output = input.matmul(weight.t())
   1693         if bias is not None:
   1694             output += bias

RuntimeError: mat1 and mat2 shapes cannot be multiplied (5376x28 and 784x512)
</code></pre>
<p>I tried changing <em>fc1</em> to <code> self.fc1 = nn.Linear(5376, 512)</code> but I still get <strong>RuntimeError: mat1 and mat2 shapes cannot be multiplied (5376x28 and 5376x512).</strong></p>
<p>I then adjust the whole architecture as follows:</p>
<pre><code class="lang-auto">class Baseline(nn.Module):
    def __init__(self):
        super().__init__()
        # 5 Hidden Layer Network
        self.fc1 = nn.Linear(5376, 28)
        self.fc2 = nn.Linear(28, 256)
        self.fc3 = nn.Linear(256, 128)
        self.fc4 = nn.Linear(128, 64)
        self.fc5 = nn.Linear(64, 3)

        # Dropout module with 0.2 probbability
        self.dropout = nn.Dropout(p=0.2)
        # Add softmax on output layer
        self.log_softmax = F.log_softmax

    def forward(self, x):
        x = self.dropout(F.relu(self.fc1(x)))
        x = self.dropout(F.relu(self.fc2(x)))
        x = self.dropout(F.relu(self.fc3(x)))
        x = self.dropout(F.relu(self.fc4(x)))

        x = self.log_softmax(self.fc5(x), dim=1)

        return x
</code></pre>
<p>and I still get the following <strong>error:</strong></p>
<pre><code class="lang-auto">RuntimeError                              Traceback (most recent call last)
&lt;ipython-input-54-0030d9c3852c&gt; in &lt;module&gt;
     18         optimizer.zero_grad()
     19         # Make predictions
---&gt; 20         log_ps = model(images)
     21         loss = criterion(log_ps, labels)
     22         #backprop

~/anaconda3/envs/torch/lib/python3.8/site-packages/torch/nn/modules/module.py in _call_impl(self, *input, **kwargs)
    725             result = self._slow_forward(*input, **kwargs)
    726         else:
--&gt; 727             result = self.forward(*input, **kwargs)
    728         for hook in itertools.chain(
    729                 _global_forward_hooks.values(),

&lt;ipython-input-52-f98f89e15885&gt; in forward(self, x)
     15 
     16     def forward(self, x):
---&gt; 17         x = self.dropout(F.relu(self.fc1(x)))
     18         x = self.dropout(F.relu(self.fc2(x)))
     19         x = self.dropout(F.relu(self.fc3(x)))

~/anaconda3/envs/torch/lib/python3.8/site-packages/torch/nn/modules/module.py in _call_impl(self, *input, **kwargs)
    725             result = self._slow_forward(*input, **kwargs)
    726         else:
--&gt; 727             result = self.forward(*input, **kwargs)
    728         for hook in itertools.chain(
    729                 _global_forward_hooks.values(),

~/anaconda3/envs/torch/lib/python3.8/site-packages/torch/nn/modules/linear.py in forward(self, input)
     91 
     92     def forward(self, input: Tensor) -&gt; Tensor:
---&gt; 93         return F.linear(input, self.weight, self.bias)
     94 
     95     def extra_repr(self) -&gt; str:

~/anaconda3/envs/torch/lib/python3.8/site-packages/torch/nn/functional.py in linear(input, weight, bias)
   1690         ret = torch.addmm(bias, input, weight.t())
   1691     else:
-&gt; 1692         output = input.matmul(weight.t())
   1693         if bias is not None:
   1694             output += bias

RuntimeError: mat1 and mat2 shapes cannot be multiplied (5376x28 and 5376x28)
</code></pre>
<p>NB: The input data is of shape <strong>torch.Size([64, 3, 28, 28])</strong>.</p>
<p>For the purposes of replication:</p>
<pre><code class="lang-auto">model = Baseline()
X = torch.rand(64, 3, 28, 28)
model(X)
</code></pre>

username: ptrblck
cooked: <p>Based on the input shapes the <code>in_features</code> of the first linear layer should be set to <code>3*28*28=2352</code> and you would most likely want to flatten the input via:</p>
<pre><code class="lang-python">    def forward(self, x):
        x  = x.view(x.size(0), -1)
        x = self.dropout(F.relu(self.fc1(x)))
</code></pre>