Speedy and accurate image super‐resolution via deeply recursive CNN with skip connection and network in network

IET Image ProcessingVolume 13, Issue 7 p. 1201-1209 Research ArticleFree Access Speedy and accurate image super-resolution via deeply recursive CNN with skip connection and network in network Dan Guo, Dan Guo School of Instrumentation and OptoelectronicEngineering, Beihang University, Beijing, 100191 People's Republic of ChinaSearch for more papers by this authorYanxiong Niu, Corresponding Author Yanxiong Niu niuyx@buaa.edu.cn School of Instrumentation and OptoelectronicEngineering, Beihang University, Beijing, 100191 People's Republic of ChinaSearch for more papers by this authorPengyan Xie, Pengyan Xie School of Instrumentation and OptoelectronicEngineering, Beihang University, Beijing, 100191 People's Republic of ChinaSearch for more papers by this author Dan Guo, Dan Guo School of Instrumentation and OptoelectronicEngineering, Beihang University, Beijing, 100191 People's Republic of ChinaSearch for more papers by this authorYanxiong Niu, Corresponding Author Yanxiong Niu niuyx@buaa.edu.cn School of Instrumentation and OptoelectronicEngineering, Beihang University, Beijing, 100191 People's Republic of ChinaSearch for more papers by this authorPengyan Xie, Pengyan Xie School of Instrumentation and OptoelectronicEngineering, Beihang University, Beijing, 100191 People's Republic of ChinaSearch for more papers by this author First published: 07 May 2019 https://doi.org/10.1049/iet-ipr.2018.5907Citations: 4AboutSectionsPDF ToolsRequest permissionExport citationAdd to favoritesTrack citation ShareShare Give accessShare full text accessShare full-text accessPlease review our Terms and Conditions of Use and check box below to share full-text version of article.I have read and accept the Wiley Online Library Terms and Conditions of UseShareable LinkUse the link below to share a full-text version of this article with your friends and colleagues. Learn more.Copy URL Share a linkShare onFacebookTwitterLinkedInRedditWechat Abstract The single image super-resolution (SISR) methods based on the deep convolutional neural network (CNN) have recently achieved significant improvements in accuracy, advancing the state of the art. However, these deeper models are computationally expensive and require a large number of parameters. Accordingly, they demand more memory and are unsuitable for on-chip devices. In this study, a novel SISR method using a deeply recursive CNN with skip connections and a network in network structure is proposed. The deeply recursive CNN with skip connections is adopted for the image feature extraction at both local and global levels. Parallelised 1 × 1 CNNs, usually called a network in network structure, are adopted for image reconstruction. Specifically, recursive learning is utilised to control the number of model parameters needed and residual learning is used to ease the difficulty of training. The proposed method performs favourably against the state-of-the-art methods in terms of computational speed and accuracy. It significantly outperforms the previous methods by a large margin, while demanding far fewer parameters. This model requires less memory and is friendly to on-chip devices. 1 Introduction In the imaging process, there are some degradation phenomena such as optical and motion blur, insufficient sampling and additional noise in the image, which can lead to a low-resolution (LR) image. The goal of single image super-resolution (SISR) is to restore high-resolution (HR) images from LR images. As SISR recovers high-frequency information, it has been widely applied in various fields, including remote sensing detection, military reconnaissance, security video surveillance, medical diagnosis, and pattern recognition, where high-frequency information in the images is prevalent. Many SISR methods have been proposed in the computer vision community. Interpolation, including bicubic interpolation and Lanczos resampling [1], was widely adopted in the early time. Recently, using large image databases, a number of super-resolution (SR) methods have been proposed that achieve a great performance. These approaches learn a mapping from LR to HR image patches. The neighbour embedding methods [2, 3] interpolate the patch subspace. The sparse coding methods [4-7] use a learned compact dictionary based on a sparse signal representation. The random forest [8] and convolutional neural network (CNN) [9] methods have also been applied with excellent improvements in accuracy. Among them, due to their powerful learning ability, CNN methods have been widely employed to solve the ill-posed inverse problem of SR; these have outperformed the fundamental reconstruction methods and other learning algorithms. Dong et al. [10] first proposed an SR convolutional neural network (SRCNN) to learn a non-linear LR-to-HR mapping. The network more accurately predicts the non-linear mapping via a fully convolutional network; the methods are superior to the traditional non-deep learning (DL) methods. While the SRCNN method successfully introduces a DL method into the SR problem and demonstrates promising results, there are a number of limitations with the approach. First, as a pre-processing step, the original LR image needs to be up-sampled to the desired size; this can be accomplished using a bicubic interpolation method. This pre-processing step increases the computational cost and often results in visible reconstruction artefacts. Extremely slow convergence of training arises from this pre-processing step. The second restriction lies with the costly non-linear mapping step. This step increases the number of layers in the network but does not significantly improve the accuracy of the SR reconstruction. To solve these issues, Dong et al. propose the fast SRCNN (FSRCNN) [11] method. The FSRCNN method exploits a deconvolution layer to replace the bicubic interpolation computation to resize the images. To further reduce the computation cost, the FSRCNN method places the deconvolution layer at the end of the network. Shi et al. also directly use LR images of the original size as the network input, adding a sub-pixel convolution layer at the end of the network. The features of the HR image that have been obtained in the previous convolution operation are periodically inserted into the corresponding HR image, according to their specific positions. Kim et al. [12] propose the very deep SR (VDSR), which is a convolutional network utilising 20 layers. The increased network depth shows a significant improvement in the SISR accuracy. However, increasing the network depth by adding new convolutional layers requires additional parameters in the model. Two problems arise with this approach. First, overfitting can occur which decreases the accuracy of the model. Second, the size of the model increases which can exceed the memory available. Accordingly, Kim et al. [13] subsequently propose a model with a deeply recursive convolutional network (DRCN). The DRCN method repeatedly applies the same convolutional layer as many times as desired. The number of parameters does not increase while more recursions are performed. When the deeper networks are able to converge, this model exposes a degradation problem: as the depth of the network increases, the accuracy reaches saturation and then degenerates rapidly. Unexpectedly, this degradation is not caused by overfitting. Rather, the addition of more layers to a model of reasonable depth leads to a higher training error rate. He et al. propose a deep residual learning framework to solve this degradation problem [14]. Tai et al. [15] introduce the idea of simultaneously adopting residual learning and recursion into the design of the SR network in a new method deep recursive residual network (DRRN). In this work, the authors effectively build a very deep network structure, which achieves better performance. Ledig et al. [16] lead to the use of a generative adversarial network (GAN) for the SR problem and propose the SRGAN method. The SRGAN method adopts the GAN approach to generate details in the image, making the details of the image more realistic. The network has obtained good accuracy in processing general images. However, it is not very effective for processing images of human faces [17]. In addition, a very large number of parameters negatively affect the training time and convergence speed of the network. Mao et al. [18] propose a 30-layer convolutional auto-encoder network named RED30 for image restoration. Utilising symmetrically linked convolutional and deconvolutional layers with skip-layer connections, the training converges much faster and attains a higher quality local optimum. Given the valuable contributions and progress achieved in the area of SISR, the following research question is investigated in this work: Can a SISR method be defined that provides further improvements in both computational efficiency and accuracy while remaining friendly for on-chip devices? To address this issue, a novel network named DRCSCN is proposed. The DRCSCN method has three major features. First, the LR image of the original size is chosen as the input instead of the bicubic interpolation image in the pre-processing stage. This dramatically reduces the computational complexity. Second, residual learning and skip connection are introduced in the DRCSCN method. This reduces the problem of network degradation and improves the accuracy of the method. Third, recursive learning is adopted, so that the DRCSCN method can improve accuracy by increasing the network depth without adding weight parameters. As shown in Fig. 1, compared with several recent CNN models, DRCSCN achieves excellent performance with fewer parameters. Fig. 1Open in figure viewerPowerPoint Parameters versus performance of recent CNN models for the scale factor×3 on the dataset Set5 2 Related work This section focuses on three methods available in the literature that have been adopted in our model: residual network, recursive structure, and network in network. 2.1 Residual network The residual learning method has been proposed by He et al. [14]. The depth of a neural network is very important to its performance. However, as the depth of a network increases, the accuracy of the network model degrades rapidly. Consequently, training deeper neural networks becomes more difficult. The goal of the residual network is to solve this training problem. The ResNet method exploits a residual learning framework to ease the training of very deep neural networks. The layers are reformulated explicitly as learning residual functions that reference the layer inputs, instead of unreferenced functions. These layers fit a residual mapping, which is easier for optimisation. Denoting the input as x and the expected output as , the goal of network learning is (1)The residuals are equivalent to changing the learning goals; hence, the learning goals no longer reflect the complete output . The differences between the output and the input are the residuals. Fig. 2 clearly shows the input, output, and residual image. Fig. 2Open in figure viewerPowerPoint Input, output, and residual image (a) Input image, (b) Output image, (c) Residual image Problems, such as information loss, exist in the traditional convolution layers (i.e. full connection layers). The residuals network can address this problem by directly transmitting the input information to the output, which protects the integrity of the information. The whole network only needs to learn the difference between the input and the output to simplify the learning goal. Thus, residual learning has been widely and successfully used in SR networks. Fig. 3 illustrates simplified network structures using the ResNet, [14] VDSR, [12] and DRCSCN methods. These networks adopt the use of residual units. Fig. 3a is a basic residual learning structure. Fig. 3b is the structure of VDSR, which use the interpolated LR image as input. Note that Fig. 3c is the structure of our proposed DRCSCN. Different from previous networks, DRCSCN uses the original LR image as input, which greatly reduces the computational complexity of subsequent networks. Fig. 3Open in figure viewerPowerPoint Simplified network structures of ResNet, VDSR, and DRCSCN (a) ResNet, (b) VDSR, (c) DRCSCN 2.2 Recursive structure A recursive learning method is applied to control the number of model parameters while increasing the network depth. Increasing the recursion depth can improve performance without introducing new parameters for additional convolutions. Consequently, the recursive structure is widely used in SISR methods. Here, we focus on two methods that are adopted in our model: DRCN [13] and DRRN [15]. 2.2.1 Deeply recursive convolutional network The DRCN method has a very deep recursive layer (up to 16 recursions). This model has been proposed by J.K et al. [13]. A recursive layer is introduced into the network, which shares parameters, rather than increasing the number of model parameters for the recursions performed. The DRCN method consists of three sub-networks: the embedding network, the inference network, and the reconstruction network. The embedding network processes the input image and represents it as a set of feature maps. The inference network is the main component to solve the task of SR. A single recursive layer analyses a large image region. Each recursion applies the same convolution function followed by a rectified linear unit. While feature maps from the final application of the recursive layer represent the HR image, the reconstruction network transforms them back into the original image space [13]. More precisely, the embedding network represents a given image x as feature maps . The inference network stacks T recursions in a recursive layer, with shared weights among these recursions. Finally, the reconstruction network generates the intermediate HR image, where is the output of the inference network [13]. For the tth intermediate recursion in the recursive layer, the output produced by the reconstruction network is formulated as (2)where represents weight and it is learned during training. 2.2.2 Deep recursive residual network The DRRN method has been proposed by Tai et al. [15]. The recursive learning of residual units is proposed in the DRRN method to keep the model compact. The DRRN method is different from the DRCN method. First, DRCN shares weights among the convolutional layers, while the DRRN method has a recursive block consisting of several residual units, and the weight set is shared among these residual units. Second, to address the vanishing/exploding gradients problem of very deep models, the DRRN method designs a recursive block with a multi-path structure while the DRCN method supervises every recursion. The DRRN method has two key parameters: the recursive block number B and the residual unit number U in each recursive block. Specifically, the depth of the DRRN method, represented as d, is calculated as (3)Computing the output of the bth recursive block is represented by (b = 1; 2; …; B). Denoting and y to be the input and output of the DRRN method, and to be the function of the bth recursive block, then (4)where can be formulated as (5)where , , and denote the functions of the different convolutional layers. When b equals 1, we define as Then, the DRRN method can be formulated as (6)where is a reconstruction function for the last convolutional layer in the DRRN method. Adopting the recursive structure, both the DRCN and DRRN methods repeatedly apply the same convolutional layer as many times as desired. Therefore, the parameters do not increase when the depth of the convolution layer increases. However, as the input of the VDSR and DRCN networks is an interpolated LR image, the size of the interpolated image is large, and more filters are needed to extract additional information from the image, which increases the number of network parameters and the computational complexity. Fig. 4 shows a simple recursive framework for the DRCN DRRN and DRCSCN models. The lilac part in the figure is a recursive structure. Differently, DRCSCN applies the original size of the image as input, greatly reducing the network parameters and computational complexity. Fig. 4Open in figure viewerPowerPoint Simple recursive structures of (a) DRCN, (b) DRRN, (c) DRCSCN 2.3 Network in network The convolution layer mainly depends on increasing the number of output channels to improve its ability to extract features. However, the side effect is to increase the computational complexity and the introduction of over-fitting. Each output channel corresponds to one filter. Each filter can extract only one kind of feature; thus, the representability of output channels is limited. In contrast, a network in network structure [19] has the ability to combine information between output channels, so the effect is obvious. Here, we introduce the inception module [20] for a network in network structure. The inception module consists of four branches. The first branch can organise the information across channels, improve the network expression ability, and increase or reduce the dimension of the output channel. The second branch uses a 1 × 1 convolution layer and then connects to a 3 × 3 convolution layer, which is equivalent to two feature transformations. The third branch is similar; first, a 1 × 1 convolution layer is used, then connects to a 5 × 5 convolution layer. The last branch applies a 3 × 3 maximum pool layer and then connects to a 1 × 1 convolution layer. The four branches are finally merged on the output channel dimension through aggregation operations. The inception module consists of three different sizes convolution layers and one maximum pool layer, which increases the adaptability of the network to different scales. We improve the inception module and design our own structures for image SR. In SR networks, we remove the pool layer as it is not necessary. In order to reduce the computational complexity and speed up the performance of the network, we also remove the branch that contains a 5 × 5 convolutional layer. Fig. 5 shows the inception module and the modified structure of the proposed DRCSCN. Fig. 5Open in figure viewerPowerPoint Structures of (a) Inception module and (b) Our own structures DRCSCN 3 Proposed method 3.1 Model overview As shown in Fig. 6, the proposed network consists of two sub-networks: feature extraction and image reconstruction. In the feature extraction network, the recursive convolutional layers are used to extract the features of the input LR image. The extracted features are sent directly to the image reconstruction network via skip connections [13]. After this processing, the dimensions of the extracted features are high. Therefore, in the image reconstruction network, a parallelised 1 × 1 convolutional layer is adopted to reduce the dimensions of the concatenated features. The LR feature map is obtained by the last convolution layer. Subsequently, the feature map is upscaled into the HR residual image by using a periodic shuffling (PS) method [21]. As a result, the residual image is obtained. Finally, the HR image output is estimated by adding the HR residual image to the bicubic interpolated LR image. Fig. 6Open in figure viewerPowerPoint Structure of our model (DCSCN) 3.2 Feature extraction network In the feature extraction network, we design a ten-layer structure with nine recursive layers. Each layer consists of convolution layers, bias, and parametric ReLU units. Each output of a layer is passed to the next layer and simultaneously skipped to the reconstruction network. In particular, we use an original image as an input of our model (not a resized image) so that the network can extract features efficiently. We also optimise the number of filters of each CNN layer and send these features directly to the image reconstruction network via skip connections. It is important to use an appropriate number of training parameters to optimise the network. It is recognised that increasing the number of convolutional layers and filters can result in the extraction of a larger number of features. However, the computational demands also increase accordingly and training the model becomes more difficult. Considering the performance and difficulty of training, the number of recursive layers is set to 9 and the number of filters is set to 128. Furthermore, the filter size of the recursive layer is set to 3 × 3. 3.3 Image reconstruction network The final HR image is reconstructed in the last half of the model; the network structure is like the network in network [19]. In the reconstruction network, all of the extracted features are concatenated at the input layer. As a result, the dimensions of the input data are rather large. If these features are directly used to reconstruct the HR images, a heavy computational load would be incurred. Therefore, we use a 1 × 1 CNN layer to reduce the input dimensions before generating the HR pixels. Remarkably, the 1 × 1 CNN layer not only reduces the dimensions of the previous layer, resulting in a faster computation with less information loss but also adds more non-linearity to enhance the performance of the network. In Fig. 6, the filter size of the CNN layer A and CNN layer B1 is set to 1 × 1, respectively, while CNN layer B2 is set to 3 × 3. The number of filters of the CNN layer A is set to 64. The number of filters of the CNN layer B1 and B2 is both set to 32. Finally, the sub-pixel CNN [21], represented by the green colour in Fig. 6, is used to transform the final features into the HR residual image. In this CNN, the outputs are s2 channels where s denotes the scale factor, and each channel represents each corner-pixel of an up-sampled pixel. The reconstruction network reshapes the LR image into an HR image; then the HR residual image is added to the bicubic interpolated image. As with typical residual learning networks, our conjecture is the learning performance of the model is greatly improved. 4 Experiments 4.1 Datasets For training, we use a training dataset of 291 images, where 91 images are from the Yang et al. dataset [4] and the remaining 200 images are from the Berkeley Segmentation Dataset [22]. In addition, data augmentation [23] is performed on the training images. The images are flipped horizontally, vertically, and both horizontally and vertically. Hence, we can obtain three additional images for each original image. Compared with simply increasing the number of images in the training set, dataset augmentation can improve the generalisation performance evaluation of the network. After the datasets are augmented, colour (RGB) images are converted to YCbCr images and only the Y-channel is processed. A Gaussian fuzzy filter is used to obtain a corresponding LR image. The scale factor is set to 2, 3, and 4, respectively, in our experiments. For testing, we utilise four widely used benchmark datasets identified as SET5, SET14, BSDS100 [24], and Urban which have 5, 14, 100, and 100 images, respectively. 4.2 Training setup The mean squared error (MSE) between the estimated output and ground truth is used as a basic loss value. MSE loss is adopted for training and the weight decay is set to 0.0001. The method proposed by He et al. [25] is used for the weight initialisation; all biases and PReLUs are initialised to 0. During training, a dropout [26] with p = 0.8 is adopted to each output of a PReLU layer. During testing, each output of a PReLU layer is multiplied by (1–p). Adam [27] with an initial learning rate 0.002 is used for the optimisation to minimise loss. The learning rate is decreased by a factor of 2 when the loss still does not reduce for 5 epochs. If the learning rate is <2 × 10−5, the procedure is terminated. The training roughly takes 10 h using one GTX 1080Ti GPU. 4.3 Study of deep recursions Increasing the recursion depth can improve performance; however, when the number of network layers reaches a certain number, the performance of the network begins to degenerate, which makes training difficult. In this subsection, we study the effect of increasing the recursion depth. Using different scale factors (2, 3, and 4), we trained models, respectively, with different numbers of recursions (1 to 15). These 45 models use the same parameters. We determine the best value of the recursion depth d as follows. The peak signal-to-noise ratio (PSNR) is used as the evaluation criterion to compare the reconstruction performance under the different values of d. The results, shown in Fig. 7, show that all the PSNR values reach a maximum value, when d equals 9 in the different scale factors. Consequently, we determine that the optimal depth of recursion in this study is 9. Fig. 7Open in figure viewerPowerPoint Recursion versus performance for the scale factor ×2, ×3, and ×4 on the dataset Set5 4.4 Comparisons with state-of-the-art methods We provide quantitative and qualitative experimental results in this section on comparative studies. For these studies, we use the publicly available code for the SRCNN [10], RED30 [18], VDSR [12], and DRCN [13] methods. 4.4.1 Comparisons with accuracy Table 1 summarises the quantitative results obtained using four testing sets and representative SR methods from SRCNN, RED30, VDSR, and DRCN. Our DRCSCN model has either the best or second-best performance across all datasets and scale factors, in both PSNR and Structural Similarity Index (SSIM). The PSNR of our model is approximately 1 dB higher than that of SRCNN on datasets Set5. Table 1. Benchmark results Dataset Scale SRCNN PSNR/SSIM RED30 PSNR/SSIM VDSR PSNR/SSIM DRCN PSNR/SSIM DRCSCN PSNR/SSIM Set5 ×2 36.66/0.9542 37.66/0.9599 37.53/0.9587 37.63/0.9588 37.64/0.9592 ×3 32.75/0.9090 33.82/0.9230 33.66/0.9213 33.82/0.9226 33.83/0.9233 ×4 30.48/0.8628 31.51/0.8869 31.35/0.8838 31.53/0.8854 31.55/0.8871 Set14 ×2 32.42/0.9063 32.94/0.9144 33.03/0.9124 33.04/0.9118 33.05/0.9127 ×3 29.28/0.8209 29.61/0.8341 29.77/0.8314 29.76/0.8311 29.78/0.8348 ×4 27.49/0.7503 27.86/0.7718 28.01/0.7674 28.02/0.7670 28.04/0.7720 BSD100 ×2 31.36/0.8879 31.99/0.8974 31.90/0.8960 31.85/0.8942 31.91/0.8962 ×3 28.41/0.7863 28.93/0.7994 28.82/0.7976 28.80/0.7963 28.95/0.8011 ×4 26.90/0.7101 27.40/0.7290 27.29/0.7251 27.23/0.7233 27.31/0.7262 Urban100 ×2 29.50/0.8946 30.85/0.9151 30.76/0.9140 30.75/0.9133 30.86/0.9205 ×3 26.24/0.7989 27.24/0.8296 27.14/0.8279 27.15/0.8276 27.25/0.8299 ×4 24.52/0.7221 25.29/0.7565 25.18/0.7524 25.14/0.7510 25.23/0.7544 Average PSNR/SSIMs for scale factor ×2, ×3 and ×4 on datasets Set5, Set14, BSD100, and Urban100. Italic values indicate the best performance and bold italic values indicate the second-best performance. 4.4.2 Comparisons with parameters quantity and computation complexity For CNN layers, the number of parameters is calculated as the square of filters size times input filters times output filters. The bias, PReLU, and the number of adding or multiplying layers are filters times output filters. The bias, PReLU, and the number of adding or multiplying layers are calculated as the number of filters. The calculation method of the computational complexity is similar to the calculation used for the parameters. The difference is the parameters can be shared while the computational complexity cannot be shared. In addition, the computational complexity is related to the size of the input image. For bicubic up-sampling, the computational complexity is calculated as the square of the scale factors times the number of input filters. The approximate number of parameters and the computational complexity for each method is shown in Table 2. We can see the number of parameters needed in our method is at least ten times lower than the DRCN method and five times lower than the RED30 method. In addition, the computational complexity of our method is much smaller than the deep convolution network such as VDSR, RED30, and DRCN methods. Table 2. Comparisons of approximate parameters quantity and computation complexity (scale factor ×4) Method CNN layers CNN filters CNN bias Activation layers Input image size Parameters Complexity (k) SRCNN 3 3264 3 2 ×16 57.4K 918.4K RED30 30 64 0 36 ×16 1038.2K 16,611.2K VDSR 20 64 20 19 ×16 667.1K 10,673.6K DRCN 20 256 20 19 ×16 1952.0K 312,332.6K DRCSCN 14 32,64,128 14 13 ×1 186.7K 2474.4K 4.4.3 Trade-off between performance and computation speed As each model is implemented under different platforms and libraries, it is not fair to measure the execution time to compare these methods. However, the computational complexity is proportional to the execution time. Therefore, we calculate the computational complexity of these methods instead. The computational complexity of 1 pixel is applied to estimate the computational speed because the DL computation is difficult to parallelise. Fig. 8 shows the trade-off between the computational complexity and performance (in terms of PSNR) on Set14 with the scale factor set to 4. We can see that our method has a state-of-the-art reconstruction performance, while the computation complexity is at least 120 times smaller than DRCN. Moreover, it is approximately seven times smaller than RED30 and five times smaller than VDSR. It is obvious that our method is more accurate and speedier. Fig. 8Open in figure viewerPowerPoint Computation complexity versus performance of recent CNN models for the scale factor ×4 on the dataset Set14 4.4.4 Comparisons with visual effect The qualitative comparisons among SRCNN, RED30, VDSR, and DRCN are illustrated in Figs. 9-11. There are some example images for different scale factors on the datasets identified as SET5, SET14, and BSD100. As we can see, the contours are clean and vivid in our method, whereas they are severely blurred or distorted in the other methods. Fig. 9Open in figure viewerPowerPoint Visual comparison for ×2 super-resolution on Set5 Fig. 10Open in figure viewerPowerPoint Visual comparison for ×3 super-resolution on Set14 Fig. 11Open in figure viewerPowerPoint Visual comparison for ×4 super-resolution on BSD100 5 Conclusions In this paper, a DRCSCN is proposed. LR images of the original size are utilised as input, which dramatically reduces the computational complexity. Moreover, a residual learning method and a recursion structure are introduced in DRCSCN. The residual learning method can reduce the network degradation problem. Increasing the recursion depth can improve the performance without introducing new parameters for additional convolutions. We optimise the number of recursions in the feature extraction network; consequently, the SR reconstruction performance is maximised while the network structure is simplified. The experimental results show that our DRCSCN method is more accurate and speedier. First, the DRCSCN method has either a best or second-best performance across all of the datasets and scale factors. Second, the number of parameters in the proposed method is far fewer than other deep convolution networks. In addition, the computational complexity is much smaller than currently available methods. This method is superior to the state-of-the-art methods in terms of computation speed, memory space, and accuracy, which indicates that it is a more suitable choice for on-chip devices. 6 References 1Duchon C.E.: ‘Lanczos filtering in one and two dimensions’, J. Appl. Meteorol., 1979, 18, (8), pp. 1016– 1022 2Chang H., Yeung D.Y., and Xiong Y.: ‘ Super-Resolution through neighbor embedding’. Proc. IEEE Conf. Computer Vision and Pattern Recognition, Washington, DC, June/July 2004, pp. 275– 282 3Bevilacqua M., Roumy A., and Guillemot C. et al: ‘ Low-Complexity single image super-resolution based on nonnegative neighbor embedding’. Proc. British Machine Vision Conf., Surrey, Guildford, England, September 2012, pp. 135.1– 135.10 4Yang J., Wright J., and Huang T.S. et al: ‘Image super-resolution via sparse representation’, IEEE Trans. Image Process., 2010, 19, (11), pp. 2861– 2873 5Roman Z., Michael E., and Matan P.: ‘ On single image scale-up using sparse-representations’. Proc. Int. Conf. Curves and Surfaces, Avignon, France, June 2010, pp. 711– 730 6Timofte R., Smet V.D., and Gool L.V.: ‘ Anchored neighborhood regression for fast example-based super-resolution’. Proc. IEEE Int. Conf. on Computer Vision, Sydney, Australia, December 2013, pp. 1920– 1927 7Timofte R., Smet V.D., and Gool L.V.: ‘ A + : adjusted anchored neighborhood regression for fast super-resolution’. Proc. Asian Conf. on Computer Vision, Singapore, Singapore, November 2014, pp. 111– 126 8Schulter S., Leistner C., and Bischof H.: ‘ Fast and accurate image upscaling with super-resolution forests’. Proc. IEEE Conf. Computer Vision and Pattern Recognition, Boston, MA, June 2015, pp. 3791– 3799 9Dong C., Loy C.C., and He K. et al: ‘Image super-resolution using deep convolutional networks’, IEEE Trans. Pattern Anal. Mach. Intell., 2016, 38, (2), pp. 295– 307 10Dong C., Chen C.L., and He K. et al: ‘ Learning a deep convolutional network for image super-resolution’. Proc. European Conf. on Computer Vision, Zurich, Switzerland, September 2014, pp. 184– 199 11Dong C., Chen C.L., and Tang X.: ‘ Accelerating the super-resolution convolutional neural network’. Proc. European Conf. on Computer Vision, Amsterdam, Netherlands, October 2016, pp. 391– 407 12Kim J., Lee J.K., and Lee K.M.: ‘ Accurate image super-resolution using very deep convolutional networks’. Proc. IEEE Conf. Computer Vision and Pattern Recognition, Seattle, WA, June 2016, pp. 1646– 1654 13Kim J., Lee J.K., and Lee K.M.: ‘ Deeply-Recursive convolutional network for image super-resolution’. Proc. IEEE Conf. Computer Vision and Pattern Recognition, Seattle, WA, June 2016, pp. 1637– 1645 14He K., Zhang X., and Ren S. et al: ‘ Deep residual learning for image recognition’. Proc. IEEE Conf. Computer Vision and Pattern Recognition, Seattle, WA, June 2016, pp. 770– 778 15Tai Y., Yang J., and Liu X.: ‘ Image super-resolution via deep recursive residual network’. Proc. IEEE Conf. Computer Vision and Pattern Recognition, Honolulu, Hawaii, US, July 2017, pp. 2790– 2798 16Ledig C., Theis L., and Huszar F. et al: ‘ Photo-Realistic single image super-resolution using a generative adversarial network’. Proc. IEEE Conf. Computer Vision and Pattern Recognition, Honolulu, Hawaii, July 2017, pp. 105– 114 17Chen Z., and Tong Y.: ‘ Face super-resolution through Wasserstein GANs’. arXiv: 1705.02438, May 2017 18Mao X.J., Shen C., and Yang Y.B.: ‘ Image restoration using very deep convolutional encoder-decoder networks with symmetric skip connections’. Proc. Adv. Neural Information Processing Systems, Barcelona, Spain, December 2016, pp. 2802– 2810 19Lin M., Chen Q., and Yan S.: ‘ Network in network’. arXiv: 1312.4400, March 2014 20Szegedy C., Liu W., and Jia Y. et al: ‘ Going deeper with convolutions’. Proc. IEEE Conf. Computer Vision and Pattern Recognition, Boston, MA, June 2015, pp. 1– 9 21Shi W., Caballero J., and Huszár F. et al: ‘ Real-Time single image and video super-resolution using an efficient sub-pixel convolutional neural network’. Proc. IEEE Conf. Computer Vision and Pattern Recognition, Seattle, WA, June 2016, pp. 1874– 1883 22Martin D., Fowlkes C., and Tal D. et al: ‘ A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics’. Proc. IEEE Int. Conf. on Computer Vision, Vancouver, Canada, July 2001, pp. 416– 423 23Timofte R., Rothe R., and Gool L.V.: ‘ Seven ways to improve example-based single image super resolution’. Proc. IEEE Conf. Computer Vision and Pattern Recognition, Seattle, WA, June 2016, pp. 1865– 1873 24Arbeláez P., Maire M., and Fowlkes C. et al: ‘Contour detection and hierarchical image segmentation’, IEEE Trans. Pattern Anal. Mach. Intell., 2011, 33, (5), pp. 898– 916 25He K., Zhang X., and Ren S. et al: ‘ Delving deep into rectifiers: surpassing human-level performance on ImageNet classification’. Proc. IEEE Int. Conf. on Computer Vision, Santiago, Chile, December 2015, pp. 1026– 1034 26Hinton G.E., Srivastava N., and Krizhevsky A. et al: ‘Improving neural networks by preventing co-adaptation of feature detectors’, Comput. Sci., 2012, 3, (4), pp. 212– 223 27Kingma D., and Ba J.: ‘ Adam: A method for stochastic optimization’. arXiv: 1412.6980, January 2017 Citing Literature Volume13, Issue7May 2019Pages 1201-1209 FiguresReferencesRelatedInformation