## 2 Baseline
### DenseNet  
DenseNet [18] is a dense block structure consisting of composite functions as shown in Figure 1. A composite function consists of three consecutive operations: batch normalization (BN) [26], followed by a rectified linear unit (ReLU) [27], and a \(3\times 3\) convolution (Conv), as shown in Figure 2.  
In a typical deep neural network, the output of \(\ell\)-th layer can be expressed as:  
\[x_{\ell}=H_{\ell}(x_{\ell-1}) \tag{1}\]  
In Equation (1), where \(x_{\ell}\) is the output of the \(\ell\)-th layer, \(x_{\ell-1}\) is the output of the \((\ell-1)\)-th layer, and it is the input of the \(\ell\)-th layer. \(H_{\ell}(\cdot)\) is a composite function that is a non-linear transformation.  
Deep neural networks have a disadvantage in that they do not learn well when the layer is deep. To overcome this drawback, ResNet [28] used a skip connection whose input is added to the output of the same layer as:  
\[x_{\ell}=H_{\ell}(x_{\ell-1})+x_{\ell-1} \tag{2}\]  
The skip connection of ResNet allows the gradient to be propagated directly to the previous layers during learning, helping to learn well in deeper architectures. However, as the input and output of the layer are summed, there is a disadvantage that the information in the previous layers becomes weaker.  
To overcome the disadvantages of ResNet skip connection above, DenseNet [18] proposes a way to concatenate the feature maps of all preceding layers. Output \(x_{\ell}\) of the \(\ell\)-th layer is expressed as:  
\[x_{\ell}=H_{\ell}([x_{0},x_{1},\ldots,x_{\ell-1}]) \tag{3}\]  
where \([x_{0},x_{1},\ldots,x_{\ell-1}]\) refers to concatenation of the output feature map of layers 0 to \((\ell-1)\). This concatenation scheme is effective for training because the gradient is propagated directly to the previous layers and the input is fed forward directly to the following layers. In the feed-forward process, all of the previous layer outputs are used as input, so that it can compensate that the information of the previous layer becomes weaker as the layer passes. The number of output feature maps for each composite function is denoted by growth rate \(k\). In Figure 1, The number of final feature maps of a dense block can be expressed as \(m_{0}+k\times L\). Here, \(m_{0}\) is the number of feature maps for an input of a dense block, and \(L\) is the number of composite functions. When passing through the layer, feature maps are increased by k.  
Due to the concatenation characteristics of DenseNet, the number of feature maps increases very much depending on the growth rate, the number of composite functions and dense blocks. To reduce the number of feature maps, compression blocks are added at the back of the dense block. The compression block consists of BN-ReLU-1 \(\times\) 1 Conv as Figure 3. If the number of the output feature maps of the dense block is \(m\), then the number of feature maps created from compression becomes an  
Figure 1: Dense block.  
Figure 2: Composite function.  
\(\lfloor\theta m\rfloor\). Compression rate \(\theta\) is a value in the range \(0<\theta\leq 1\), and when \(\theta=1\), the number of feature maps for input and output is the same.