Help Reading CNN Architecture Table

username: ayrts
cooked: <p>Hello,</p>
<p>I am trying to program an 8-layered toy network that accepts a 32 x 32 colored image into Pytorch by following the table, but I get an error when the input to the first f.c. layer has incorrect number of in_features (1x784 as opposed to 1x3136).</p>
<p>Can someone point out what I missed? I followed the formula:</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/b/7/b7205af48ea4089f98a89497ed8948043b030b9f.png" data-download-href="https://discuss.pytorch.org/uploads/default/b7205af48ea4089f98a89497ed8948043b030b9f" title="image"><img src="https://discuss.pytorch.org/uploads/default/original/3X/b/7/b7205af48ea4089f98a89497ed8948043b030b9f.png" alt="image" data-base62-sha1="q80Hh4Bro9iZ1zAzR4kuIMP1XDp" width="690" height="193" data-dominant-color="303030"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">1103×309 24.1 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>But in reality, torch gives me <img src="https://discuss.pytorch.org/uploads/default/original/3X/c/7/c7d12552fa857bf07908a0afea2b8bc04c75b7fb.png" alt="image" data-base62-sha1="svF6MCJAmjiTp4bgNNLna7eTBl9" width="170" height="19">, which I am confused about because it does not follow the formula.</p>
<p>Either method does not lead to the desired output, which is <img src="https://discuss.pytorch.org/uploads/default/original/3X/5/9/59fd5446790977733944b1bd98a95d6392c4f768.png" alt="image" data-base62-sha1="cQ5fvGxK1hnJ44gX4OR5Z69xO4w" width="165" height="23">, and would give me the correct <code>in_features</code> for the 1st fc layer.</p>
<p>Hoping that someone can help me find my mistake. I have attached the table and my code below. Thank you.</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/1/e/1e3fbeb39c9fc2ead6355f6a93b3806f2381647f.png" data-download-href="https://discuss.pytorch.org/uploads/default/1e3fbeb39c9fc2ead6355f6a93b3806f2381647f" title="image"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/1/e/1e3fbeb39c9fc2ead6355f6a93b3806f2381647f_2_517x270.png" alt="image" data-base62-sha1="4jATC2lU9QcOhO1N0GXVnoCidht" width="517" height="270" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/1/e/1e3fbeb39c9fc2ead6355f6a93b3806f2381647f_2_517x270.png, https://discuss.pytorch.org/uploads/default/optimized/3X/1/e/1e3fbeb39c9fc2ead6355f6a93b3806f2381647f_2_775x405.png 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/1/e/1e3fbeb39c9fc2ead6355f6a93b3806f2381647f.png 2x" data-dominant-color="EDEDEE"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">802×420 56.3 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>And here is my code:</p>
<pre data-code-wrap="python"><code class="lang-python">import torch
import torch.nn as nn
import torch.nn.functional as F

class Net(nn.Module):
  def __init__(self, max_epochs = 10, learning_rate=0.01, num_classes=10):
    super(Net, self).__init__()

    # Model
    self.conv1 = nn.Conv2d(in_channels=3, out_channels=64, kernel_size=3, stride=1) 
    self.conv1_bn = nn.BatchNorm2d(64)
    
    self.conv2 = nn.Conv2d(64, 128, 3, 1)
    self.conv2_bn = nn.BatchNorm2d(128)

    self.conv3 = nn.Conv2d(128, 196, 3, 1)
    self.conv3_bn = nn.BatchNorm2d(196)

    self.fc1 = nn.Linear(in_features=3136, out_features=256)
    self.fc1_bn = nn.BatchNorm1d(256)

    self.fc2 = nn.Linear(256, num_classes)

  def forward(self, x):
    x = self.conv1(x)
    x = self.conv1_bn(x)
    x = F.relu(x)
    x = F.max_pool2d(x, kernel_size=2, stride=2)
    # Outputs 1 x 64 x 15 x 15
    
    x = self.conv2(x)
    x = self.conv2_bn(x)
    # Outputs 1 x 128 x 13 x 13

    x = F.relu(x)
    x = F.max_pool2d(x, kernel_size=2, stride=2)
    # Outputs 1 x 128 x 6 x 6,  but shouldn't it be 1 x 128 x 8 x 8 ?

    x = self.conv3(x)
    x = self.conv3_bn(x)
    # Outputs 1 x 196 x  4 x 4

    x = F.relu(x)
    x = F.max_pool2d(x, kernel_size=2, stride=2)
    # Outputs 1 x 196 x 2 x 2
    
    x = torch.flatten(x, start_dim=1)
    # Outputs: 1 x 784; Expected: 1 x 3136 = ( 196 x 4 x 4)

    x = self.fc1(x)
    x = self.fc1_bn(x)
    x = F.relu(x)

    return F.softmax(self.fc2(x), dim=1)

# Test code
random_data = torch.rand((1, 3, 32, 32))

my_nn = Net()
result = my_nn(random_data)
print (result)
</code></pre>

username: AbdulsalamBande
cooked: <p>The problem is that your batch size is 1. Make it greater than 1</p>
<p>The calculation you made is right. So change <code>self.fc1 = nn.Linear(in_features=3136, out_features=256)</code> to  <code>self.fc1 = nn.Linear(in_features=784, out_features=256)</code>.</p>
<p>Refer to <a href="https://discuss.pytorch.org/t/error-expected-more-than-1-value-per-channel-when-training/26274" class="inline-onebox">Error: Expected more than 1 value per channel when training</a> to know why the batch size has to be  greater than 1.</p>

username: ayrts
cooked: <p>Hello, I just realized that the expected output size of each layer on the table does not reflect the output sizes I observed.</p>
<p>I overlooked the padding and assumed it was 0 because the table did not specify it. Setting the padding = 1 resolves the main problem.</p>
<p>I then got the problem on the batch_size that <a class="mention" href="/u/abdulsalambande">@AbdulsalamBande</a> mentioned. However, this is only a problem for training and setting the model to <code>eval</code> mode allows my model to test the dummy tensor.</p>
<p>Thank you for the help!</p>