CNN Computationally expensive

Alymostafa: <pre><code class="lang-auto">class ConvNet(nn.Module):
    def __init__(self, num_classes=2):
        super(ConvNet, self).__init__()
        self.layer1 = nn.Sequential(
            nn.Conv2d(1, 16, kernel_size=5, stride=1, padding=2),
            nn.ReLU(),
            nn.MaxPool2d(kernel_size=2, stride=2))
        self.layer2 = nn.Sequential(
            nn.Conv2d(16, 32, kernel_size=5, stride=1, padding=2),
            #nn.BatchNorm2d(64),
            nn.ReLU(),
            nn.MaxPool2d(kernel_size=5, stride=0))
        
        self.drop_out = nn.Dropout()

        self.fc1 = nn.Linear(15488, 152)
        self.fc2 = nn.Linear(152,2)
        
    def forward(self, x):
        x.float()
        out = self.layer1(x)
        out = self.layer2(out)
        out = out.reshape(out.size(0), -1)
        out = self.drop_out(out)
        out = self.fc1(out)
        out = self.fc2(out)
       
        return out
</code></pre>
<p>This architecture is so Computationally expensive and requires much memory , any recommendation to   Minimize this computation and save the performance.<br>
[ my model is take 224*224 picture and return 2 classes : P , N ]<br>
thanks in advance</p>

JuanFMontesinos: <p>Dont use a fc 15000-200 but get a single 1d vector of fetures (32x1x1)using a adaptative max pool</p>

Alymostafa: <p>sorry but i dont understand you , can you write your changes in the architecture and post it<br>
thanks in advance</p>

JuanFMontesinos: <pre><code class="lang-auto">class ConvNet(nn.Module):
    def __init__(self, num_classes=2):
        super(ConvNet, self).__init__()
        self.layer1 = nn.Sequential(
            nn.Conv2d(1, 16, kernel_size=5, stride=1, padding=2),
            nn.ReLU(),
            nn.MaxPool2d(kernel_size=2, stride=2))
        self.layer2 = nn.Sequential(
            nn.Conv2d(16, 32, kernel_size=5, stride=1, padding=2),
            #nn.BatchNorm2d(64),
            nn.ReLU(),
            # nn.MaxPool2d(kernel_size=5, stride=0)) # Too much parameters after this to use a relu
            nn.AdaptiveMaxPool2d(1)
        
        self.drop_out = nn.Dropout()

        self.fc1 = nn.Linear(32, 16)
        self.relu = nn.ReLu()
        self.fc2 = nn.Linear(16,2)
        
    def forward(self, x):
        x.float()
        out = self.layer1(x)
        out = self.layer2(out)
        out = out.reshape(out.size(0), -1)
        out = self.drop_out(out)
        out = self.fc1(out)
        out = self.relu(out)
        out = self.fc2(out)
       
        return out</code></pre>

Alymostafa: <aside class="quote no-group quote-modified" data-username="JuanFMontesinos" data-post="4" data-topic="76099">
<div class="title">
<div class="quote-controls"></div>
<img alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/juanfmontesinos/48/76115_2.png" class="avatar"> JuanFMontesinos:</div>
<blockquote>
<p>nn.nn.AdaptiveMaxPool2d(1)</p>
</blockquote>
</aside>
<p>is this right syntax not a typo ?</p>

JuanFMontesinos: <p>Yep, Basically you are making a pooling to get spatial size = 1. This way instead of having 15k parameters you will have just 1 parameter per channel. Then,  fully connected is feasible. Besides you need a non-linearity between both FC. Otherwise it’s like having a single FC (as FC are linear layers)</p>

Alymostafa: <p>ok i will try it and i will tell you about the performance<br>
thanks</p>

JuanFMontesinos: <p>Ypu can try adaptative average pooling instead of max pooling, depending on the final size of your tensor.</p>

Alymostafa: <p>can you explain more details about that ?</p>

JuanFMontesinos: <p>Hmmm. Max pooling propagates the maximum activation of each channel. Average pooling propagate a mean channel.<br>
You can Google advantages and disadvantages for example this qa<br>
<a href="https://www.quora.com/What-is-the-benefit-of-using-average-pooling-rather-than-max-pooling" class="onebox" target="_blank" rel="nofollow noopener">https://www.quora.com/What-is-the-benefit-of-using-average-pooling-rather-than-max-pooling</a></p>

Alymostafa: <p>can you give me a syntax for average pooling or a source to it ?<br>
thanks for your interest</p>