# Improvement for Convolutional Neural Networks in Image Classification Using Long Skip Connection
## 3 Methodology
### M-DenseNet Architecture  
Figure 3 illustrates the simple M-DenseNet architecture. Similar to DenseNet, we create dense connection between layers in each block, then add transition layer(s) to perform downsampling. However, we change all \(3\times 3\) conv layers in DenseNet to s-conv layer, then add Add block as shown in Table 1.  
Figure 2: Connection between 2 blocks with upstream and pooling layers (called Add block). \(BN\) denotes Batch Normalization and the normal line represents long skip connection between blocks while dash line states that there could be additional block(s) between block F and K. \begin{table}
\begin{tabular}{c c c c} \hline \hline
**Name** & **Size** & **M-DenseNet121** & **M-DenseNet169** & **M-DenseNet201** \\ \hline Separable-Conv2D & \(112\times 112\) & & \(7\times 7\) s-conv, stride = 2 \\ MaxPool2D & \(56\times 56\) & & \(3\times 3\) max pool, stride = 2 \\ Block 1 & \(56\times 56\) & & \(\left[\begin{array}{c}1\times 1\ conv\\ 3\times 3s-conv\end{array}\right]\times 6\) \\ Transition Layer 1 & \(56\times 56\) & & \(1\times\ 1\) conv \\ \(28\times 28\) & & \(2\times 2\) average pool, stride 2 \\ Add block (0–1) & \(56\times 56\) & upsampling2d(size = 2), \(1\times 1\) conv, maxpool2d \\ Block 2 & \(28\times 28\) & & \(1\times\ 1\) conv \\ Transition Layer 2 & \(14\times 14\) & & \(2\times\ 2\) average pool, stride 2 \\ Add block (0-2) & \(56\times 56\) & upsampling2d(size = 4), \(1\times 1\) conv, maxpool2d \\ Add block (1-2) & \(28\times 28\) & upsampling2d(size = 2), \(1\times 1\) conv, maxpool2d \\ Block 3 & \(14\times 14\) & \(\left[\begin{array}{c}1\times 1\ conv\\ 3\times 3s-conv\end{array}\right]\times 3\) & \(\left[\begin{array}{c}1\times 1\ conv\\ 3\times 3s-conv\end{array}\right]\times 3\) \\ Transition Layer 3 & \(14\times 14\) & & \(1\times\ 1\) conv \\ \(2\times 2\) average pool, stride 2 \\ Add block (0-3) & \(56\times 56\) & upsampling2d(size = 8), \(1\times 1\) conv, maxpool2d \\ Add block (1-3) & \(28\times 28\) & upsampling2d(size = 4), \(1\times 1\) conv, maxpool2d \\ Add block (2-3) & \(14\times 14\) & upsampling2d(size = 2), \(1\times 1\) conv, maxpool2d \\ Block 4 & \(7\times 7\) & \(\left[\begin{array}{c}1\times 1\ conv\\ 3\times 3s-conv\end{array}\right]\times\left[\begin{array}{c}1\times 1\ conv\\ 3\times 3s-conv\end{array}\right]\times 3\) & \(\left[\begin{array}{c}1\times 1\ conv\\ 3\times 3s-conv\end{array}\right]\times 16\) \\ \(24\times 14\) & & \(1\times\ 1\) conv \\ Transition Layer 3 & \(7\times 7\) & & \(1\times\ 1\) conv \\ Add block (0-3) & \(56\times 56\) & upsampling2d(size = 8), \(1\times 1\) conv, maxpool2d \\ Add block (1-3) & \(28\times 28\) & upsampling2d(size = 4), \(1\times 1\) conv, maxpool2d \\ Add block (2-3) & \(14\times 14\) & upsampling2d(size = 2), \(1\times 1\) conv, maxpool2d \\ Block 4 & \(7\times 7\) & \(\left[\begin{array}{c}1\times 1\ conv\\ 3\times 3s-conv\end{array}\right]\times\left[\begin{array}{c}1\times 1\ conv\\ 3\times 3s-conv\end{array}\right]\times 32\) & \(\left[\begin{array}{c}1\times 1\ conv\\ 3\times 3s-conv\end{array}\right]\times 32\) \\ \(16\) & & Global AvgPool2D \\ \hline \hline \end{tabular}
\end{table} 
Table 1: MDenseNet’s architectures. Figure 3: An overview of model architecture. Note that the skip connections are created between blocks (layers with different shapes). Our model can be considered as a larger version of connection of DenseNet with separable convolution instead of normal 2D convolution. Table 1 shows different configuration of the M-DenseNet. Blocks are shown in bracket with number of repetitions. Upsampling2d with \(1\times 1\) conv2d helps add blocks match dimension. Introduced initially before the Block 1, a convolution with 64 output channels, \(7\times 7\) kernel size and 2 strides are applied on the input image. The block architecture, as shown in Figure 2, follows the idea of Deeper Bottleneck Architectures [11].