Can anyone explain how this output of cnn less than inp

username: Archy_dragon
cooked: <p>Hi i got a code to study but when i look at the code it doesnt make sense special for out<br>
this is code for the module</p>
<pre><code class="lang-auto">class Bottleneck(nn.Module):
    def __init__(self, inp, oup, stride, expansion):
        super(Bottleneck, self).__init__()
        self.connect = stride == 1 and inp == oup
        #
        self.conv = nn.Sequential(
            # pw
            nn.Conv2d(inp, inp * expansion, 1, 1, 0, bias=False),
            nn.BatchNorm2d(inp * expansion),
            nn.PReLU(inp * expansion),
            # nn.ReLU(inplace=True),

            # dw
            nn.Conv2d(inp * expansion, inp * expansion, 3, stride, 1, groups=inp * expansion, bias=False),
            nn.BatchNorm2d(inp * expansion),
            nn.PReLU(inp * expansion),
            # nn.ReLU(inplace=True),

            # pw-linear
            nn.Conv2d(inp * expansion, oup, 1, 1, 0, bias=False),
            nn.BatchNorm2d(oup),
        )

    def forward(self, x):
        if self.connect:
            return x + self.conv(x)
        else:
            return self.conv(x)
</code></pre>
<pre><code class="lang-auto">MobiFace_bottleneck_setting = [
    # t, c , n, s
    [2, 64, 1, 2],
    [2, 64, 2, 1],
    [4, 128, 1, 2],
    [2, 128, 3, 1],
    [4, 256, 1, 2],
    [2, 256, 6, 1]
]
</code></pre>
<p>this is when i run this code</p>
<pre><code class="lang-auto">class MobiFace(nn.Module):
    def __init__(self, bottleneck_setting=MobiFace_bottleneck_setting, final_linear=False):
        super(MobiFace, self).__init__()
        self.final_linear = final_linear

        self.conv1 = ConvBlock(3, 64, 3, 2, 1)

        self.dw_conv1 = ConvBlock(64, 64, 3, 1, 1, dw=True)

        

        self.inplanes = 64
        block = Bottleneck
        #set_trace()

        self.blocks = self._make_layer(block, bottleneck_setting)
        
        self.conv2 = ConvBlock(256, 512, 1, 1, 0, linear=True)

        self.linear1 = nn.Linear(7*7*512, 512)
        
        self.prelu1 = nn.PReLU()

        for m in self.modules():
            if isinstance(m, nn.Conv2d):
                n = m.kernel_size[0] * m.kernel_size[1] * m.out_channels
                m.weight.data.normal_(0, math.sqrt(2. / n))
            elif isinstance(m, nn.BatchNorm2d):
                m.weight.data.fill_(1)
                m.bias.data.zero_()

    def _make_layer(self, block, setting):
        layers = []
        for t, c, n, s in setting:
            for i in range(n):
                if i == 0:
                    layers.append(block(self.inplanes, c, s, t))
                else:
                    layers.append(block(self.inplanes, c, 1, t))
                self.inplanes = c

        return nn.Sequential(*layers)

    def forward(self, x):
        x = self.conv1(x)
        x = self.dw_conv1(x)
        
        x = self.blocks(x)
        x = self.conv2(x)
        x = x.view(x.size(0), -1)
        x = self.linear1(x)
        if self.final_linear is False:
            x = self.prelu1(x)

        return x
</code></pre>
<p>so print the layer</p>
<pre><code class="lang-auto">if __name__ == "__main__":
    input = Variable(torch.FloatTensor(2, 3, 112, 96))
    net = MobiFace()
    print(net)
</code></pre>
<pre><code class="lang-auto">MobiFace(
  (conv1): ConvBlock(
    (conv): Conv2d(3, 64, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
    (bn): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    (prelu): PReLU(num_parameters=64)
  )
  (dw_conv1): ConvBlock(
    (conv): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=64, bias=False)
    (bn): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    (prelu): PReLU(num_parameters=64)
  )
  (blocks): Sequential(
    (0): Bottleneck(
      (conv): Sequential(
        (0): Conv2d(64, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (2): PReLU(num_parameters=128)
        (3): Conv2d(128, 128, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), groups=128, bias=False)
        (4): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (5): PReLU(num_parameters=128)
        (6): **Conv2d(128, 64, kernel_size=(1, 1), stride=(1, 1), bias=False)**
        (7): **BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)**
</code></pre>
<p>can anyone explain to me how does the conv2d input is 128 in (6) it reduce to 64 at the end how is this happen?</p>

username: ptrblck
cooked: <p>Your <code>Bottleneck</code> layer defines:</p>
<pre><code class="lang-python"># pw-linear
nn.Conv2d(inp * expansion, oup, 1, 1, 0, bias=False),
nn.BatchNorm2d(oup),
</code></pre>
<p>where at the <code>oup</code> is set to 64, which creates the last conv and batchnorm layer in the first <code>Bottleneck</code> as:</p>
<pre><code class="lang-python">(6): Conv2d(128, 64, kernel_size=(1, 1), stride=(1, 1), bias=False)
(7): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
</code></pre>

username: Archy_dragon
cooked: <p>ok thank you that make sense now but just wonder how does oup be less than inp is this mean the layer deleter channel?</p>

username: ptrblck
cooked: <p>The number of output channels is defined by the number of filters in the layer.<br>
In the default setup each filter will use <em>all</em> input channels and create a single output channel.<br>
Multiple filters define the complete activation output.<br>
<a href="https://cs231n.github.io/convolutional-networks/#conv">CS231n - Convolution layer</a> explains the logic pretty well. <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=9" title=":slight_smile:" class="emoji" alt=":slight_smile:"></p>

username: Archy_dragon
cooked: <p>thnk you so much will check it out <img src="https://discuss.pytorch.org/images/emoji/apple/smile.png?v=9" title=":smile:" class="emoji" alt=":smile:"></p>

username: Archy_dragon
cooked: <p>and can you explain what is the n value in # t, c , n, s been try to understand it but only that part im not quite sure what is it for</p>

username: ptrblck
cooked: <p><code>n</code> is used in the loop:</p>
<pre><code class="lang-python">for i in range(n):
</code></pre>
<p>to create multiple layers.<br>
As you can see in the final model <code>blocks</code> will contain 14 <code>Bottleneck</code> modules, which corresponds to the values of <code>n</code> in the setting (1+2+1+3+1+6=14).</p>