Linear layer input neurons number calculation after conv2d

andreiliphd: <p>I will be really thankful to those who can explain me this. I know the formula for calculation but after some iterations, I haven’t got to the answer yet.</p>
<p>The formula for output neuron:<br>
Output = ((I-K+2P)/S + 1), where<br>
I - a size of input neuron,<br>
K - kernel size,<br>
P - padding,<br>
S - stride.</p>
<p>Input tensor shape:</p>
<pre><code class="lang-auto">torch.Size([36, 200, 150, 3])
</code></pre>
<p>I have the following model:</p>
<pre><code class="lang-auto">class Flatten(torch.nn.Module):
    def forward(self, x):
        return x.view(x.size()[0], -1)

model = torch.nn.Sequential(
        torch.nn.Conv2d(32, 64, kernel_size=(3, 3)),
        torch.nn.ReLU(),
        torch.nn.Conv2d(64, 128, kernel_size=(3, 3)),
        torch.nn.ReLU(),
        torch.nn.MaxPool2d(kernel_size=(2, 2)),
        torch.nn.Dropout(0.25),
        Flatten(),
        torch.nn.Linear(128, 128),
        torch.nn.ReLU(),
        torch.nn.Linear(128, 2),
        torch.nn.Softmax()
        )
</code></pre>
<p>I can’t calculate the number of neurons in Linear layer.<br>
Could you kindly help and explain me the calculations of Linear input neuron size behind this network?</p>

ptrblck: <p>Your input shape seems to be a bit wrong, as it looks like the channels are in the last dimension.<br>
In PyTorch, image data is expected to have the shape <code>[batch_size, channel, height, width]</code>.<br>
Based on your shape, I guess 36 is the <code>batch_size</code>, while 3 seems to be the number channels.</p>
<p>However, as your model expects 32 input channels, your input won’t work at all currently.</p>
<p>Let’s just assume we are using an input of <code>[1, 32, 200, 150]</code> and walk through the model and the shapes.<br>
Since your <code>nn.Conv2d</code> layers don’t use padding and a default stride of 1, your activation will lose one pixel in both spatial dimensions.<br>
After the first conv layer your activation will be <code>[1, 64, 198, 148]</code>, after the second <code>[1, 128, 196, 146]</code>.<br>
<code>nnMaxPool2d(2)</code> will halve the activation to <code>[1, 128, 98, 73]</code>.</p>
<p>If you set the number of <code>in_features</code> for the first linear layer to <code>128*98*73</code> your model will work for my input.</p>
<p>I also recommend to just print out the shape of your activation before the linear layer, if the shape calculation is too cumbersome, and set the input features according to this.<br>
For your Sequential model you can just create a print layer with:</p>
<pre><code class="lang-python">class Print(nn.Module):
    def forward(self, x):
        print(x.size())
        return x
</code></pre>

andreiliphd: <p>Thank you very much <a href="https://discuss.pytorch.org/u/ptrblck">ptrblck</a>. It was really helpful.<br>
It seems that I manage to make a function out off it for those who struggle with the issue.</p>
<pre><code class="lang-auto">def count_input_neuron(model, image_dim):
    return model(torch.rand(1, *(image_dim))).data.view(1, -1).size(1)
</code></pre>
<aside class="quote no-group" data-username="ptrblck" data-post="2" data-topic="28659">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/48/1823_2.png" class="avatar"> ptrblck:</div>
<blockquote>
<p>However, as your model expects 32 input channels, your input won’t work at all currently.</p>
</blockquote>
</aside>
<p>So, I have to change it to 3? The number of features or number of color channels?</p>

ptrblck: <p>The <code>in_channels</code> of the first conv layer correspond to the channels of your input.<br>
In case you are using a color image tensor, i.e. 3 channels, you would have to set <code>in_channels=3</code>.</p>

AbdulsalamBande: <p>First of all there is a problem with your input shape. The shape should be BATCH_SIZE * CHANNEL * HEIGHT *WIDTH. So lets correct your size and I assume you BATCH_SIZE = 36, CHANNEL = 3, HEIGHT = 200 , WIDTH = 150.</p>
<pre><code class="lang-auto">images = image.permute(0,3,1,2)
</code></pre>
<p>Next lets change your first Conv2d code. IT should be</p>
<pre><code class="lang-auto">torch.nn.Conv2d(3, 64, kernel_size=(3, 3))
</code></pre>
<p>So after the first convolution using your formular, we will have</p>
<pre><code class="lang-auto">[3, 64, 198, 148]
</code></pre>
<p>After the second Conv2d operation, we will have</p>
<pre><code class="lang-auto">[3, 128, 196, 146].
</code></pre>
<p>The maxpooling which halves the activations we will have</p>
<pre><code class="lang-auto">[3, 128, 98, 73]
</code></pre>
<p>And finally the input of the fully connected layer will be 128×98×73 = 915712</p>

Rui_Li: <p>Hi, I have some questions here:<br>
"<br>
So after the first convolution using your formular, we will have</p>
<pre><code class="lang-auto">[3, 64, 198, 148]
</code></pre>
<p>After the second Conv2d operation, we will have</p>
<pre><code class="lang-auto">[3, 128, 196, 146].
</code></pre>
<p>"<br>
Since we assume batch_size=36, maybe it should be that after the first convolution layer, we will have</p>
<pre><code class="lang-auto">[36, 64, 198, 148].
</code></pre>
<p>And after the second convolution layer, we will have</p>
<pre><code class="lang-auto">[36, 128, 196, 146].
</code></pre>

AbdulsalamBande: <p>First of all welcome to the Pytorch Community Rui_Li <img src="https://discuss.pytorch.org/images/emoji/apple/blush.png?v=9" title=":blush:" class="emoji" alt=":blush:"><br>
The first dimension of Pytorch Convolution should always be the <strong>the number of channels</strong> (3) for the input image , While on the other and the first dimension of the inputed image should be the <strong>batch_size</strong>(36). So do not let this confuse you . Furthermore the convolution operation will be done 36 number of times for all of the 36(batch size) images in parallel.</p>

Ripley: <p>Speaking about the topic of calculating the number of neurons for your linear layer input, what about the rest of your linear layers? Is there a specific theory or formula we can use to determine the number of layers to use and the number to put for our input and output for the linear layers?</p>
<p>I 've read quite abit online for this problem, but it seems like most people does not have a specific solution or answer to this. It is mostly based on empirical methods and taking that as a reference to increase or decrease layers. What are your thoughts about this? Thank you!</p>

ptrblck: <p>I would try to start with a known good baseline and try to adapt the architecture to your use case.<br>
Also, the number of neurons defines e.g. if your model has a bottleneck, which might be useful for certain use cases.<br>
That being said, I think a lot of architectures are developed empirically and the reasoning about why the model is working find is created afterwards. <img src="https://discuss.pytorch.org/images/emoji/apple/wink.png?v=9" title=":wink:" class="emoji" alt=":wink:"></p>

tdbr: <p>You could also use torch.nn.LazyLinear, a module where in_features is inferred.<br>
Please refer to <a href="https://discuss.pytorch.org/t/inferring-shape-via-flatten-operator/138/20" class="inline-onebox">Inferring shape via flatten operator - #20 by iacob</a></p>