How can i perform Global average pooling before the last fully connected layer

username: Muhammad_Izaz
cooked: <p>class ConvNet(nn.Module):<br>
def <strong>init</strong>(self):<br>
super(ConvNet, self).<strong>init</strong>()<br>
self.layer1 = nn.Sequential(<br>
nn.Conv1d(1, 16, kernel_size=5, stride=1 , padding=2),<br>
nn.BatchNorm1d(16),<br>
nn.ReLU(),<br>
nn.MaxPool1d(kernel_size=2, stride=2))<br>
self.layer2 = nn.Sequential(<br>
nn.Conv1d(16, 32 ,kernel_size=5, stride=1, padding=2),<br>
nn.BatchNorm1d(32),<br>
nn.ReLU(),<br>
nn.MaxPool1d(kernel_size=2, stride=2))</p>
<pre><code>    self.drop_out = nn.Dropout()
    
    #self.fc1 = nn.Linear(32*25, 1000)
    self.fc2 = nn.Linear(64, 2)
</code></pre>
<h1>this step is how the data flow through these layers in forward pass</h1>
<pre><code>def forward(self, x):
    
    
    out = self.layer1(x)
    out = self.layer2(out)
    #out = F.adaptive_avg_pool2d(x, (1, 1))
    out = F.avg_pool1d(out,1)
    #out = self.layer3(out)
    #out = out.reshape(-1, 12)
    out = self.drop_out(out)
    #out = self.fc1(out)
    
    out = self.fc2(out)
    return out</code></pre>

username: ptrblck
cooked: <p>Based on the <a href="https://arxiv.org/pdf/1312.4400.pdf">Network in Network paper</a> global average pooling is described as:</p>
<blockquote>
<p>Instead of adding fully connected layers on top of the feature maps, we take the average of each feature map, and the resulting vector is fed directly into the softmax layer. One advantage of global average pooling over the fully connected layers is that it is more native to the convolution structure by enforcing correspondences between feature maps and categories. Thus the feature maps can be easily interpreted as categories confidence maps. Another advantage is that there is no parameter to optimize in the global average pooling thus overfitting is avoided at this layer. Futhermore, global average pooling sums out the spatial information, thus it is more robust to spatial translations of the input.</p>
</blockquote>
<p>which can be achieved by using <code>out = out.mean([2, 3])</code>.</p>
<p>However, since you are using linear layers, I’m not sure, if you are really looking for the global average pooling layers.</p>

username: Muhammad_Izaz
cooked: <p>i want to remove those fully connected layers and replace them with average pooling layer.</p>

username: ptrblck
cooked: <p>In that case the <code>mean</code> operation should work.</p>

username: Muhammad_Izaz
cooked: <p>how will i implement that please. i am searching for it for long time.</p>

username: ptrblck
cooked: <p>I’ve given the code in my previous code:</p>
<pre><code class="lang-python">out = out.mean([2, 3])
</code></pre>
<p>This will calculate the mean in dim2 and dim2 and return a 2-dimensional tensor, which should then (based on the paper desciption) contain the class logits in the channel dimension (dim1).</p>

username: Muhammad_Izaz
cooked: <p>Dimension out of range (expected to be in range of [-2, 1], but got 2)</p>

username: ptrblck
cooked: <p>Oh, my bad. I missed that you are using <code>nn.Conv1d</code>. In that case, you should only calculate the mean in the temporal dimension: <code>out = out.mean(2)</code>.<br>
Also, note that you would have to apply it before feeding the activation to your linear layers and before reshaping the tensor.<br>
Based on the error message, it seems you are trying to apply it on a flattened tensor.</p>