Inception Network in Pytorch (Code Check)

username: sgaur
cooked: <p>Hi Guys,</p>
<p>I need to create Inception network in pytorch for one of my project from scratch so I am digging some tutorials most of the tutorials advice to use pre trained weight. But with some research I build small block of inception model on MNIST dataset.</p>
<p>Just need to know is it correct or not.</p>
<p>**<strong>Input size 128,1,28,28</strong> **<br>
<strong>(128 is batch size)</strong></p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/4/e/4e80fb44b63526504c590e75671772d85142cc67.png" data-download-href="https://discuss.pytorch.org/uploads/default/4e80fb44b63526504c590e75671772d85142cc67" title="Screenshot from 2020-04-05 18-31-03"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/4/e/4e80fb44b63526504c590e75671772d85142cc67_2_690x335.png" alt="Screenshot from 2020-04-05 18-31-03" data-base62-sha1="bctAe85WKvFwHtF4zjY4dHYuxpB" width="690" height="335" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/4/e/4e80fb44b63526504c590e75671772d85142cc67_2_690x335.png, https://discuss.pytorch.org/uploads/default/original/3X/4/e/4e80fb44b63526504c590e75671772d85142cc67.png 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/4/e/4e80fb44b63526504c590e75671772d85142cc67.png 2x" data-dominant-color="F7F8F8"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Screenshot from 2020-04-05 18-31-03</span><span class="informations">759×369 41.5 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<pre><code class="lang-auto">class InceptionA(nn.Module):
    def __init__(self):
        super().__init__()
        ##Branch 1
        self.conv1x1   =  nn.Conv2d(in_channels=1,out_channels= 32 , kernel_size = 1)

       
        ##Branch 2
        self.conv1x1_1 =  nn.Conv2d(in_channels=1,out_channels= 32 , kernel_size = 1)        
        self.conv5x5   =  nn.Conv2d(in_channels=32,out_channels= 64 , kernel_size = 5,padding =2)
        
        ##Branch 3
        self.conv1x1_2 =  nn.Conv2d(in_channels=1,out_channels= 64 , kernel_size = 1)
        self.conv3x3   =  nn.Conv2d(in_channels=64,out_channels= 128 , kernel_size = 3)
        self.conv3x3_1 =  nn.Conv2d(in_channels=128,out_channels= 196 , kernel_size = 3,padding=2)
        
        ##Branch 4        
        self.pooling   =  nn.MaxPool2d(kernel_size=3, stride=1, padding=1)
        self.conv1x1_3 =  nn.Conv2d(in_channels=1,out_channels= 64 , kernel_size = 1 ,padding = 0)
        
        ## Linear Layer
        self.fc1 = nn.Linear(356*28*28,32)
        self.fc2 = nn.Linear(32,10)
        
        
        
    
    def forward(self,x):
        ## 1st branch
        conv1x1   = torch.relu(self.conv1x1(x))
        
        ## 2nd Branch
        conv1x1_1 = torch.relu(self.conv1x1_1(x))
        conv5x5 = torch.relu(self.conv5x5(conv1x1_1))
        
        ## 3rd Branch
        conv1x1_2 = torch.relu(self.conv1x1_2(x))
        conv3x3   = torch.relu(self.conv3x3(conv1x1_2))
        conv3x3_1 = torch.relu(self.conv3x3_1(conv3x3))
        
        ## 4th Branch
        pooling = torch.relu(self.pooling(x))
        conv1x1_3  = torch.relu(self.conv1x1_3(pooling))
        
        output = [conv1x1,conv5x5,conv3x3_1,conv1x1_3]
        out = torch.cat(output,1)
        
        out = out.view(-1,356*28*28)
        
        x = self.fc1(out)
        x = self.fc2(x)
        
        return x
</code></pre>
<p>Please let me know if anyone  need more info.</p>
<p>Stay safe.!!!</p>

username: ptrblck
cooked: <p>Your implementation is almost identical to the <a href="https://github.com/pytorch/vision/blob/e61538cba036c42bab23ce8f9d205da9889977ae/torchvision/models/inception.py#L200-L235">torchvision implementation</a> bedsides:</p>
<ul>
<li>branch3 misses <code>paddin=1</code> for <code>self.conv3x3</code>
</li>
<li>you are adding linear layers, which is not part of the inception block, but might fit your use case.</li>
</ul>