VGG with 1x1 convolution

username: ricardborras
cooked: <p>Hi,</p>
<p>given the first 5 layers of a VGG network:</p>
<p>Sequential(<br>
(0): Conv2d(3, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))<br>
(1): ReLU(inplace=True)<br>
(2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))<br>
(3): ReLU(inplace=True)<br>
(4): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)<br>
(5): Conv2d(64, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))<br>
(6): ReLU(inplace=True)<br>
(7): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))<br>
(8): ReLU(inplace=True)<br>
(9): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)<br>
(10): Conv2d(128, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))<br>
(11): ReLU(inplace=True)<br>
(12): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))<br>
(13): ReLU(inplace=True)<br>
(14): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))<br>
(15): ReLU(inplace=True)<br>
(16): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)<br>
(17): Conv2d(256, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))<br>
(18): ReLU(inplace=True)<br>
(19): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))<br>
(20): ReLU(inplace=True)<br>
(21): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))<br>
(22): ReLU(inplace=True)<br>
(23): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)<br>
(24): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))<br>
(25): ReLU(inplace=True)<br>
(26): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))<br>
(27): ReLU(inplace=True)<br>
(28): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))<br>
(29): ReLU(inplace=True)<br>
(30): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)<br>
)</p>
<p>How to add a layer to classify each activation into n_classes? I am trying with a 1x1 convolution from 512 (number of filters in last layer) to n_classes, something like this:</p>
<p>nn.Sequential(<br>
nn.Conv2d(in_channels=512, out_channels=n_classes, kernel_size=1),<br>
nn.Softmax(dim=1)<br>
)<br>
This network gives (batch_size, n_classes H // 32, W // 32) outputs. However, all outputs for each class are equal for a given image, I mean that, for a given batch, B and  class C, all values are the same.</p>
<p>I have tried training with CrossEntropyLoss and even converting to logits and applying BCELoss, but without success. I also have tried other networks in my train loop and this do not happens.</p>
<p>Any clue?</p>
<p>Thanks</p>

username: mailcorahul
cooked: <p>You can use <code>nn.AdaptiveAvgPool2d</code> to reduce the spatial dimensions to 1x1(HxW) and then use 1x1 convolution to bring num_filters = num_classes. Use this after the last conv output from vgg.</p>
<pre><code class="lang-auto">self.classifier = nn.Sequential(
          nn.AdaptiveAvgPool2d((1, 1)), 
          nn.Conv2d(in_channels=512, out_channels=n_classes, kernel_size=1),
          nn.ReLU(inplace=True)
         )
</code></pre>
<p>Given what you have done above, I am not sure how you were able to calculate loss since the output will be num_classes, H//32, W//32. However one important thing to note is, do not use <code>nn.Softmax</code> at the end of the network when using <code>nn.CrossEntropy</code>(it combines softmax and nll loss). That can cause problems.</p>

username: ricardborras
cooked: <p>Hi,</p>
<p>thanks for the tip, I have removed Softmax from classification layer. Also, I found my mistake: it was a problem of weight initialization of last 1x1 convolution layer. Now the network is learning properly.</p>
<p>many thanks</p>