# Single Image Super-Resolution Method Using CNN-Based Lightweight Neural Networks
## 3 Proposed Method
### Architecture of SR-ILLNN  
Figure 2 shows the proposed SR-ILLNN, which consists of two inputs, 15 convolution layers and two deconvolution layers. The two inputs are denoted as a low-resolution (LR) image \(X_{\text{LR}}\) and a bi-cubic interpolated low-resolution (ILR) image \(X_{ILR}\) where N and M denote the width and height of the input image \(X_{LR}\), respectively. The reason why we deployed the two inputs is to compensate the dense LR features of SR-DenseNet with high-resolution (HR) features of \(X_{ILR}\), while reducing the number of convolutional layers as many as possible. As depicted in Figure 2, it consists of three parts, which are LR feature layers from convolutional layer 1 (Conv1) to Conv8, HR feature layers from Conv9 to Conv12, and shared feature layers from Conv13 to Conv15. Each convolution is operated as in (1), where \(W_{i}\), \(B_{i}\), and '\(\otimes\)' represent the kernels, biases, and convolution operation of the \(i\)th layer, respectively. In this paper, we notate a kernel as \([F_{w}\times F_{h}\times F_{c}]\), where \(F_{w}\times F_{h}\) and \(F_{c}\) are the spatial size of filter and the number of channels, respectively:  
\[F_{i}(X_{LR})\ =\ max(0,\ W_{i}\ \otimes\ F_{i-1}(X_{LR})\ +\ B_{i}), \tag{1}\]  
Figure 1: The framework of SR-DenseNet [18]. In the process of convolution operation, we applied rectified linear unit (ReLU, \(max(0,\ x)\)) on the filter responses and used a partial convolution-based padding scheme [23] to avoid the loss of boundary information. The padding sizes is defined so that the feature maps between different convolution layers can have the same spatial resolution as follows:  
\[\textit{Padding Size = Floor(F_{w}/2)}, \tag{2}\]  
where Floor(x) means the rounding down operation. Note that the convolutional layers of Conv1-4 and Conv9-12 of Figure 2 are conducted to generate output feature maps with dense connections for more flexible and richer feature representations, which are concatenated the feature maps of the previous layer with those of the current layer. So, convolution operations with dense connections are calculated as in (3):  
\[ \begin{array}{l}F_{i}(X_{LR})\ =\ max(0,\ W_{i}\ \otimes\ [F_{1}(X_{LR}),\ \dots,\ F_{i-1}(X_{LR})]\ +\ B_{i}),\\ F_{j}(X_{ILR})\ =\ max(0,\ W_{j}\ \otimes\ [F_{9}(X_{ILR}),\ \dots,\ F_{j-1}(X_{ILR}) ]\ +\ B_{j})\end{array}  \tag{3}\]  
A ResNet scheme [11] with skip connections can provide a high-speed training and prevent the gradient vanishing effect, so we deployed a local and a global skip connection to train the residual at the output feature maps of Conv4 and Conv15. Because the output feature maps \(F_{4}\) and \(X_{LR}\) have the different number of channels in local skip connection, \(X_{LR}\) is copied up to the number of channels of \(F_{4}\) before operating Conv5. It should be noted that the number of feature maps has a strong effect on the inference speed. Therefore, the proposed SR-LNNs is designed to reduce the number of feature maps from 192 to 32, before deconvolution operation. Then, Deconv1 and Deconv2 are operated for image up-sampling as follows:  
\[ \begin{array}{l}F_{deconv}(X_{LR})\ =\ max(0,\ W_{deconv}\ \odot\ F_{i-1}(X_{LR})\ +\ B_{deconv}),\end{array}  \tag{4}\]  
where \(W_{deconv}\), \(B_{deconv}\) are the kernels and biases of the deconvolution layer, respectively, and the symbol \({}^{\prime}\odot^{\prime}\) denotes the deconvolution operation. As each deconvolution layer has different kernel weights and biases, it is superior to the conventional SR methods like pixel-wise interpolation methods. In the stage of the shared feature layers, the output feature maps of the LR feature layers \(F_{8}(X_{LR})\) are concatenated with those of HR feature layers \(F_{12}(X_{ILR})\). Then, the  
Figure 2: The framework of the proposed SR-ILLNN.