Channel Max Pooling

martinodonnell: <p>I am trying to replicate a technique from a paper which adds a channel max pooling layer in-between the last max-pooling layer and the first FC layer of the VGG16 model. The paper can be found at <a href="https://ieeexplore.ieee.org/document/8648206/citations#citations" rel="noopener nofollow ugc">https://ieeexplore.ieee.org/document/8648206/citations#citations</a></p>
<p>You may not be able to read it but the important lines are:</p>
<blockquote>
<p>‘VGG16 with CMP (VGG16-CMP): Similar as DenseNet161-CMP, we applied the CMP operation to the VGG16 by implementing the CMP layer between the last max-pooling layer and the first FC layer. The dimension of the pooled features was changed from 512 × 7 × 7 to c × 7 × 7. All the other components remained unchanged’</p>
</blockquote>
<p>Notes:</p>
<p>CMP is channel max pooling layer<br>
c is the channel number of the feature maps</p>
<p>Questions<br>
1)From the statement above, is the pooling layer the newly layer addd or the max pooling layer already present?<br>
2) Is the code below even right?</p>
<p>I have created a custom maxpooling layer and added this to the model but I have no clue if it is what the paper is talking about</p>
<pre><code class="lang-auto">import torch.nn as nn
import torch
import torchvision
class ChannelPoolingNetwork(nn.Module):
    def __init__(self, num_classes):
        super().__init__()

        self.base = torchvision.models.vgg16(pretrained=True, progress=True)   
        
        self.base.features = nn.Sequential(
            self.base.features,
            ChannelPool(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
        )
        print(self.base.features)
        self.base.classifier = nn.Sequential(
            nn.Linear(25088, 4096),
            nn.ReLU(),
            nn.Dropout(0.5),
            nn.Linear(4096, 4096),
            nn.ReLU(),
            nn.Dropout(0.5),
            nn.Linear(4096, num_classes)
        )

        

    def forward(self, x):
        fc = self.base(x)
        return fc
# Code from https://stackoverflow.com/questions/46562612/pytorch-maxpooling-over-channels-dimension
class ChannelPool(nn.MaxPool1d):
    def forward(self, input):
        n, c, w, h = input.size()
        input = input.view(n,c,w*h).permute(0,2,1)
        pooled =  nn.MaxPool1d(input)
        _, _, c = input.size()
        input = input.permute(0,2,1)
        return input.view(n,c,w,h)

</code></pre>

ptrblck: <p>I’m not sure which paper you are linking to, but could you explain, how channel-max-pooling is supposed to work?<br>
Would it be a topk-operation on the channel-dimension?</p>

martinodonnell: <p>CMP does max pooling across the dimensions of the feature map. The image below is a visualisation representation given in the paper</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/b/8/b80816cefc4aa94cd473ec918cb1e6515ec50901.png" data-download-href="https://discuss.pytorch.org/uploads/default/b80816cefc4aa94cd473ec918cb1e6515ec50901" title="Screenshot 2020-03-18 at 09.16.22"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/b/8/b80816cefc4aa94cd473ec918cb1e6515ec50901_2_690x324.png" alt="Screenshot 2020-03-18 at 09.16.22" data-base62-sha1="qg1buyZyFfIctFNzHpof725fvrz" width="690" height="324" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/b/8/b80816cefc4aa94cd473ec918cb1e6515ec50901_2_690x324.png, https://discuss.pytorch.org/uploads/default/optimized/3X/b/8/b80816cefc4aa94cd473ec918cb1e6515ec50901_2_1035x486.png 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/b/8/b80816cefc4aa94cd473ec918cb1e6515ec50901.png 2x" data-dominant-color="DEDDDC"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Screenshot 2020-03-18 at 09.16.22</span><span class="informations">1158×544 124 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

martinodonnell: <p>Turns out I made some mistakes in the code above but I think I have it correct now. My main problem is that it takes a long time to process. Is there anyway to speed it up?<br>
The input array has 4 dimensions which are batch_index, channel dimension, kernel weight and height. I have to go through each image(input[x]) and do max pooling across the channels with a kernel size of 7 and stride 2. The input is [32,512,7,7] and have hard-coded these hyper parameters to work on the data.</p>
<p>I don’t know much about tensors but I am sure there is a better way than one monolithic for loop.</p>
<pre><code class="lang-auto">class ChannelPool(nn.Module):

    def __init__(self, kernel_size=7, stride=2, padding=3, dilation=1,
                 return_indices=False, ceil_mode=False):
        super().__init__()
        self.kernel_size = kernel_size
        self.stride = stride or kernel_size
        self.padding = padding
        self.dilation = dilation
        self.return_indices = return_indices
        self.ceil_mode = ceil_mode
        self.compression = 2
        self.output = None


    def forward(self, input):

        n, c, w, h = input.size()
        #Add padding to input so work with kernal size
        input = torch.nn.functional.pad(input, (0, 0, 0, 0, self.padding, self.padding), "constant", 0)
        
        #Get output
        output = torch.stack([ 
                        torch.stack(
                            [torch.max(input[x][index:index+self.kernel_size-1],axis=0)[0] #Get max at each position in  kernal size
                            for index in range(0,input.size()[1]-self.kernel_size,self.stride)]) #Move stride
                            for x in range(n)]) #Do work for each image in batch

        return output.cuda()
</code></pre>

efirdc: <p>I haven’t looked at your code, but it sounds like you could just use a 3D maxpool to do this. Just size the kernel so it covers all channels.</p>

hrishikesh_khade: <p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/a/2/a2392159f131facbe26b6954480bbe037e561c3b.jpeg" data-download-href="https://discuss.pytorch.org/uploads/default/a2392159f131facbe26b6954480bbe037e561c3b" title="now"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/a/2/a2392159f131facbe26b6954480bbe037e561c3b_2_690x126.jpeg" alt="now" data-base62-sha1="n95KzSJRqR33iYmrf8NO8La0zF1" width="690" height="126" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/a/2/a2392159f131facbe26b6954480bbe037e561c3b_2_690x126.jpeg, https://discuss.pytorch.org/uploads/default/original/3X/a/2/a2392159f131facbe26b6954480bbe037e561c3b.jpeg 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/a/2/a2392159f131facbe26b6954480bbe037e561c3b.jpeg 2x" data-dominant-color="E7E7E6"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">now</span><span class="informations">903×165 43.4 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

NAVEEN_PALURU: <h1>
<a name="using-3d-maxpool-for-2d-channel-wise-polling-1" class="anchor" href="#using-3d-maxpool-for-2d-channel-wise-polling-1"></a>using 3D MaxPool for 2D channel wise Polling</h1>
<p>ff = torch.rand(32,256,256,256)<br>
b = torch.nn.MaxPool3d((8,1,1),stride=(8,1,1))<br>
gg = b(ff)<br>
gg.shape<br>
torch.Size([32, 32, 256, 256])</p>