Missmatch error with Pytorch CNN and Linear Layer model

Tino: <p>I have built a network and it seems that my first fully connected layer has not been set the correct input size, my problem is I don’t know how to properly set it. I keep getting the error:</p>
<blockquote>
<p> in inner(_it, _timer)</p>
<p>/usr/local/lib/python3.6/dist-packages/torch/nn/functional.py in linear(input, weight, bias)<br>
1368     if input.dim() == 2 and bias is not None:<br>
1369         # fused op is marginally faster<br>
 → 1370         ret = torch.addmm(bias, input, weight.t())<br>
1371     else:<br>
1372         output = input.matmul(weight.t())</p>
<p>RuntimeError: size mismatch, m1: [5 x 196608], m2: [26912 x 512] at /pytorch/aten/src/TH/generic/THTensorMath.cpp:136</p>
</blockquote>
<p>This is the code for my network I would appreciate any help I can get</p>
<pre><code class="lang-auto"># Convolutional neural network
class ConvNet(nn.Module):
    
    def __init__(self, num_classes=10):
        super(ConvNet, self).__init__()

  
        # Add network layers here
        # Layer 1
        self.conv1 = nn.Conv2d(3,16, (3,3))
        self.pool = nn.MaxPool2d(2,2)
        self.relu = nn.ReLU()
        self.dropout = nn.Dropout(p=0.3)


            # Layer 2
        self.conv2 = nn.Conv2d(16,24, (4,4))
        self.pool = nn.MaxPool2d(2,2)
        self.relu = nn.ReLU()
        self.dropout = nn.Dropout(p=0.3)

        conv = self.conv2
        print(conv)

            # Layer 3
        self.conv3 = nn.Conv2d(24,32, (4,4))
        self.pool = nn.MaxPool2d(2,2)
        self.relu = nn.ReLU()
        self.dropout = nn.Dropout(p=0.3)

            # Layer 4 - Fully connected
        self.fc1 = nn.Linear(32 * 29 * 29, 512)
        

        self.fc2 = nn.Linear(512, num_classes)
        self.final = nn.Softmax(dim=1)
        
    def forward(self, x):
       out = x.reshape(x.size(0), -1) # TODO what does this do? Why do we need it?
       out = self.fc1(out)

        # Complete the graph

       return out
       

net = ConvNet()
</code></pre>
<blockquote>
<p>loss_function = nn.CrossEntropyLoss()</p>
<p>optimizer = optim.SGD(net.parameters(), lr=0.0001,momentum=0.9)</p>
<p>print(len(test_loader))<br>
def train_model_epochs(num_epochs):<br>
for epoch in range(10):</p>
<pre><code>running_loss = 0.0
</code></pre>
<p>for i, data in enumerate(test_loader, 0):<br>
images, labels = data</p>
<pre><code>optimizer.zero_grad()

outputs = net(images)

loss = loss_function(outputs, labels)

loss.backward()

optimizer.step()

running_loss =+ loss.item()
if i%1000 == 999:
  print('Epoch / Batch [%d / %d] - Loss: %.3f' %
      (epoch + 1, i + 1, running_loss / 1000))
  running_loss = 0.0    
</code></pre>
<p>print(“Training finished”)</p>
<p>cpu_train_time = timeit.timeit(<br>
“train_model_epochs(num_epochs)”,<br>
setup=“num_epochs=10”,<br>
number=1,<br>
globals=globals(),<br>
)</p>
<p>print("Training time: " + cpu_train_time)</p>
</blockquote>

ptrblck: <p>In your <code>forward</code> method you are directly flattening your input <code>x</code> and passing it to the linear layer skipping all conv layers.<br>
I assume you would like to pass the input first to <code>self.conv1</code>, <code>self.pool</code> etc.<br>
You are also redefining the modules inside the <code>__init__</code> method, so use different names for the layers.</p>
<p><a href="https://pytorch.org/tutorials/beginner/examples_nn/two_layer_net_module.html">This tutorial</a> might be helpful.</p>

Tino: <p>Ahh yes that makes sense!</p>
<blockquote>
<p>I assume you would like to pass the input first to  <code>self.conv1</code> ,  <code>self.pool</code>  etc.</p>
</blockquote>
<p>For the above, I am unsure if my method is accurate in terms of syntax:</p>
<blockquote>
<p>def forward(self, x):<br>
out = x.reshape(x.size(0), -1)<br>
out = self.conv1(out)<br>
out = self.pool1(out)<br>
out = self.relu1(out)</p>
<pre><code>   out = self.conv2(out)
   out = self.pool2(out)
   out = self.relu2(out)

   out = self.conv3(out)
   out = self.pool3(out)
   out = self.relu3(out)

   out = self.fc1(out)
</code></pre>
</blockquote>
<p>With this new forward method I get the error:</p>
<blockquote>
<p>Expected 4-dimensional input for 4-dimensional weight 16 3 3 3, but got 2-dimensional input of size [5, 196608] instead</p>
</blockquote>

Tino: <blockquote>
<p>def forward(self, x):</p>
<pre><code>   out = self.conv1(out)
   out = self.conv2(out)
   out = self.conv3(out)
   
   out = x.reshape(x.size(0), -1)

   out = self.fc1(out)
</code></pre>
</blockquote>
<p>I changed the model to use sequential layers however i am getting the error:</p>
<blockquote>
<p>local variable ‘out’ referenced before assignment</p>
</blockquote>
<p>Sorry to bother you I am at a loss</p>

ptrblck: <p>Your <code>forward</code> method uses <code>x</code> as the variable name, while you are passing <code>out</code> to the first layer.</p>