CRNET: IMAGE SUPER-RESOLUTION USING A CON-
Year: 2019

Abstract:
Convolutional Sparse Coding (CSC) has been attracting more and more attention in recent years, for making full use of image global correlation to improve performance on various computer vision applications. However, very few studies focus on solving CSC based image Super-Resolution (SR) problem. As a consequence, there is no significant progress in this area over a period of time. In this paper, we exploit the natural connection between CSC and Convolutional Neural Networks (CNN) to address CSC based image SR. Specifically, Convolutional Iterative Soft Thresholding Algorithm (CISTA) is introduced to solve CSC problem and it can be implemented using CNN architectures. Then we develop a novel CSC based SR framework analogy to the traditional SC based SR methods. Two models inspired by this framework are proposed for pre-/post-upsampling SR, respectively. Compared with recent state-of-the-art SR methods, both of our proposed models show superior performance in terms of both quantitative and qualitative measurements.

heading: 1 INTRODUCTION
text: Single Image Super-Resolution (SISR), which aims to restore a visually pleasing High-Resolution (HR) image from its Low-Resolution (LR) version, is still a challenging task within computer vision research community (Timofte et al., 2017; 2018). Since multiple solutions exist for the mapping from LR to HR space, SISR is highly ill-posed. To regularize the solution of SISR, various priors of natural images have been exploited, especially the current leading learning-based methods (Wang et al., 2015; Dong et al., 2016; Mao et al., 2016; Kim et al., 2016a;b; Tai et al., 2017a;b; Lim et al., 2017; Ahn et al., 2018; Haris et al., 2018; Li et al., 2018; Zhang et al., 2018) are proposed to directly learn the non-linear LR-HR mapping.
By modeling the sparse prior in natural images, the Sparse Coding (SC) based methods for SR (Yang et al., 2008; 2010; 2014) with strong theoretical support are widely used owing to their excellent performance. Considering the complexity in images, these methods divide the image into overlapping patches and aim to jointly train two over-complete dictionaries for LR/HR patches. There are usually three steps in these methods’ framework. First, overlapping patches are extracted from input image. Then to reconstruct the HR patch, the sparse representation of LR patch can be applied to the HR dictionary with the assumption that LR/HR patch pair shares similar sparse representation. The final HR image is produced by aggregating the recovered HR patches.
Recently, with the development of Deep Learning (DL), many researchers attempt to combine the advantages of DL and SC for image SR. Dong et al. (Dong et al., 2016) firstly proposed the seminal CNN model for SR termed as SRCNN, which exploits a shallow convolutional neural network to learn a nonlinear LR-HR mapping in an end-to-end manner and dramatically overshadows conventional methods (Yang et al., 2010; Timofte et al., 2014). However, sparse prior is ignored to a large extent in SRCNN for it adopts a generic architecture without considering the domain expertise. To address this issue, Wang et al. (Wang et al., 2015) implemented a Sparse Coding based Network (SCN) for image SR, by combining the merits of sparse coding and deep learning, which fully exploits the approximation of sparse coding learned from the LISTA (Gregor & LeCun, 2010) based sub-network.
It’s worth to note that most of SC based methods utilize the sparse prior locally (Papyan et al., 2017b), i.e., coping with overlapping image patches. Thus the consistency of pixels in overlapped
patches has been ignored (Gu et al., 2015; Papyan et al., 2017b). To address this issue, CSC is proposed to serve sparse prior as a global prior (Zeiler et al., 2010; Papyan et al., 2017b; 2018) and it furnishes a way to fill the local-global gap by working directly on the entire image by convolution operation. Consequently, CSC has attained much attention from researchers (Zeiler et al., 2010; Bristow et al., 2013; Heide et al., 2015; Gu et al., 2015; Sreter & Giryes, 2018; Garcia-Cardona & Wohlberg, 2018a). However, very few studies focus on the validation of CSC for image SR (Gu et al., 2015), resulting in no work been reported that CSC based image SR can achieve state-of-theart performance. Can CSC based image SR show highly competitive results with recent state-of-theart methods (Dong et al., 2016; Mao et al., 2016; Kim et al., 2016a;b; Tai et al., 2017a;b; Tong et al., 2017; Lim et al., 2017; Li et al., 2018; Zhang et al., 2018)? To answer this question, the following issues need to be considered:
Framework Issue. Compared with SC based image SR methods (Yang et al., 2008; 2010), the lack of a unified framework has hindered progress towards improving the performance of CSC based image SR.
Optimization Issue. The previous CSC based image SR method (Gu et al., 2015) contains several steps and they are optimized independently. Hundreds of iterations are required to solve the CSC problem in each step.
Memory Issue. To solve the CSC problem, ADMM (Boyd et al., 2011) is commonly employed (Bristow et al., 2013; Wohlberg, 2014; Heide et al., 2015; Wohlberg, 2016; Garcia-Cardona & Wohlberg, 2018a), where the whole training set needs to be loaded in memory. As a consequence, it is not applicable to improve the performance by enlarging the training set.
Multi-Scale Issue. Training a single model for multiple scales is difficult for the previous CSC based image SR method (Gu et al., 2015).
Based on these considerations, in this paper, we attempt to answer the aforementioned question. Specifically, we exploit the advantages of CSC and the powerful learning ability of deep learning to address image SR problem. Moreover, massive theoretical foundations for CSC (Papyan et al., 2017b; 2018; Garcia-Cardona & Wohlberg, 2018b) make our proposed architectures interpretable and also enable to theoretically analyze our SR performance. In the rest of this paper, we first introduce CISTA, which can be naturally implemented using CNN architectures for solving the CSC problem. Then we develop a framework for CSC based image SR, which can address the Framework Issue. Subsequently, CRNet-A (CSC and Residual learning based Network) and CRNet-B inspired by this framework are proposed for image SR. They are classified as pre- and post-upsampling models (Wang et al., 2019) respectively, as the former takes Interpolated LR (ILR) images as input while the latter processes LR images directly. By adopting CNN architectures, Optimization Issue and Memory Issue would be mitigated to some extent. For Multi-Scale Issue, with the help of the recently introduced scale augmentation (Kim et al., 2016a;b) or scale-specific multi-path learning (Lim et al., 2017; Wang et al., 2019) strategies, both of our models are capable of handling multi-
scale SR problem effectively, and achieve favorable performance against state-of-the-arts, as shown in Fig. 1. The main contributions of this paper include:
• We introduce CISTA, which can be naturally implemented using CNN architectures for solving the CSC problem. • A novel framework for CSC based image SR is developed. Two models, CRNet-A and CRNet-B, inspired by this framework are proposed for image SR. • The differences between our proposed models and several SR models with recursive learning strategy, e.g., DRRN (Tai et al., 2017a), SCN (Wang et al., 2015), DRCN (Kim et al., 2016b), are discussed.

heading: 2 RELATED WORK
text: Sparse coding has been widely used in a variety of applications (Zhang et al., 2015). As for SISR, Yang et al. (Yang et al., 2008) proposed a representative Sparse coding based Super-Resolution (ScSR) method. In the training stage, ScSR attempts to learn the LR/HR overcomplete dictionary pair Dl/Dh jointly by given a group of LR/HR training patch pairs xl/xh. In the test stage, the HR patch xh is reconstructed from its LR version xl by assuming they share the same sparse code. Specifically, the optimal sparse code is obainted by minimizing the following sparsity-inducing `1- norm regularized objective function
z∗ = argmin z ‖xl −Dlz‖22 + λ‖z‖1, (1)
then the HR patch is obtained by xh = Dhz∗. Finally, the HR image can be estimated by aggregating all the reconstructed HR patches. Inspired by ScSR, many SC based methods have been proposed with various constraints on sparse code or dictionary Yang et al. (2012); Wang et al. (2012).
Traditional SC based SR algorithms usually process images in a patch based manner to reduce the burden of modeling and computation, resulting in the inconsistency problem (Papyan et al., 2017b). As a special case of SC, CSC is inherently suitable (Zeiler et al., 2010) and proposed to avoid the inconsistency problem by representing the whole image directly. Specifically, an image y ∈ Rnr×nc can be represented as the summation of m feature maps zi ∈ Rnr×nc convolved with the corresponding filters fi ∈ Rs×s: y = ∑m i=1 fi ⊗ zi, where ⊗ is the convolution operation.
Gu et al. (Gu et al., 2015) proposed the CSC-SR method and revealed the potential of CSC for image SR. In (Gu et al., 2015), CSC-SR requires to solve the following CSC based optimization problem in both the training and testing phase:
min f ,z
1
2 ∥∥∥∥∥y − m∑ i=1 fi ⊗ zi ∥∥∥∥∥ 2
2
+ λ m∑ i=1 ‖zi‖1 . (2)
which is solved by alternatively optimizing the z and f subproblems (Wohlberg, 2014). The z subproblem is a standard CSC problem. Hundreds of iterations are required to solve the CSC problem and the aforementioned Optimization Issue and Memory Issue cannot be completely avoided. Inspired by the success of deep learning based sparse coding (Gregor & LeCun, 2010), we exploit the natural connection between CSC and CNN to solve the CSC problem efficiently.

heading: 3 CISTA FOR SOLVING CSC PROBLEM
text: CSC can be considered as a special case of conventional SC, due to the fact that convolution operation can be replaced with matrix multiplication, so the objective function of CSC can be formulated as:
min z ∥∥∥∥∥y − m∑ i=1 Fizi ∥∥∥∥∥ 2
2
+ λ m∑ i=1 ‖zi‖1 . (3)
y, zi are in vectorized form and Fi is a sparse convolution matrix with the following attributes:
Fizi ≡ fi ⊗ zi F Ti zi ≡ flipud(fliplr(fi))⊗ zi
≡ flip(fi)⊗ zi (4)
where fliplr(·) and flipud(·) are following the notations of Zeiler et al.(Zeiler et al., 2010), representing that array is flipped in left/right or up/down direction.
Iterative Soft Thresholding Algorithm (ISTA) (Daubechies et al., 2004) can be utilized to solve (3), at the kth iteration:
zk+1 = hθ ( zk + 1
L F T (y − Fzk)
) (5)
where L is the Lipschitz constant, F = [F1,F2, . . . ,Fm] and Fz = ∑m i=1 Fizi. Using the relation in (4) to replace the matrix multiplication with convolution operator, we can reformulate (5) as:
zk+1 = hθ ( Izk + 1
L flip(f)⊗ (y − f ⊗ zk)
) (6)
where f = [f1,f2, . . . ,fm], flip(f) = [flip(f1),flip(f2), . . . ,flip(fm)] and I is the identity matrix. Note that identity matrix I is also a sparse convolution matrix, so according to (4), there existing a filter n satisfies:
Iz = n⊗ z, (7)
so (6) becomes:
zk+1 = hθ (W ⊗ y + S ⊗ zk) , (8)
whereW = 1Lflip(f) and S = n− 1 Lflip(f)⊗ f . Even though (3) is for a single image with one channel, the extension to multiple channels (for both image and filters) and multiple images is mathematically straightforward. Thus for y ∈ Rb×c×nr×nc representing b images of size nr × nc with c channels, (8) is still true withW ∈ Rm×c×s×s and S ∈ Rm×m×s×s. As for hθ, (Papyan et al., 2017a) reveals two important facts: (1) the expressiveness of the sparsity inspired model is not affected even by restricting the coefficients to be nonnegative; (2) the ReLU (Nair & Hinton, 2010) activation function and the soft nonnegative thresholding operator are equal, that is:
h+θ (α) = max(α− θ, 0) = ReLU(α− θ). (9)
We set θ = 0 for simplicity. So the final form of (8) is:
zk+1 = ReLU (W ⊗ y + S ⊗ zk) . (10)
One can see that (10) is a convolutional form of (5), so we name it as CISTA. It provides the solution of (3) with theoretical guarantees (Daubechies et al., 2004). Furthermore, this convolutional form can be implemented employing CNN architectures. SoW and S in (10) would be trainable.

heading: 4 PROPOSED METHOD
text: In this work, exploiting the basic CISTA block, we implement the SR using CNN techniques. With more recursions used in CISTA, the network becomes deeper and tends to be bothered by the gradient vanishing/exploding problems. Residual learning (Kim et al., 2016a;b; Tai et al., 2017a) is such a useful tool that not only mitigates these difficulties, but helps network converge faster (A.1).

heading: 4.1 CRNET-A MODEL FOR PRE-UPSAMPLING
text: As shown in Fig. 2, CRNet-A takes the ILR image Iy with c channels as input, and predicts the output HR image as Ix. Two convolution layers, F0 ∈ Rn0×c×s×s consisting of n0 filters of spatial size c× s× s and F1 ∈ Rn0×n0×s×s containing n0 filters of spatial size n0 × s× s are utilized for hierarchical features extraction from ILR image:
y = ReLU ( F1 ⊗ReLU(F0 ⊗ Iy) ) . (11)
The ILR features are then fed into a CISTA block to learn the convolutional sparse codes. As stated in (10), two convolutional layersWl ∈ Rm0×n0×s×s and S ∈ Rm0×m0×s×s are needed:
zk+1 = ReLU(Wl ⊗ y + S ⊗ zk), (12) where z0 is initialized to ReLU(Wl ⊗ y). The convolutional sparse codes z are learned after K recursions with S shared across every recursion. When the convolutional sparse codes z are obtained, it is then passed through a convolution layer Wh ∈ Rn0×m0×s×s to recover the HR feature maps. The last convolution layerH ∈ Rc×n0×s×s is used as HR filters:
R =H ⊗ReLU(Wh ⊗ z). (13) Note that we pad zeros before all convolution operations to keep all the feature maps to have the same size, which is a common strategy used in a variety of methods (Kim et al., 2016a;b; Tai et al., 2017a). So the residual image R has the same size as the input ILR image Iy , and the final HR image Ix would be reconstructed by:
Ix = Iy +R. (14)

heading: 4.2 CRNET-B MODEL FOR POST-UPSAMPLING
text: We extend CRNet-A to its post-upsampling version to further mine its potential. Notice that most post-upsampling models (Ledig et al., 2017; Tong et al., 2017; Li et al., 2018; Zhang et al., 2018) need to train and store many scale-dependent models for various scales without fully using the interscale correlation, so we adopt the scale-specific multi-path learning strategy (Wang et al., 2019) presented in MDSR (Lim et al., 2017) with minor modifications to address this issue. The complete model is shown in A.2. The main branch is our CRNet-A module. The pre-processing modules are used for reducing the variance from input images of different scales and only one residual unit with 3 × 3 kernels is used in each of the pre-processing module. At the end of CRNet-B, upsampling modules are used for multi-scale reconstruction.

heading: 5 EXPERIMENTAL RESULTS

heading: 5.1 SETTINGS
text: Datasets and Metrics Being fair to (Kim et al., 2016a; Tai et al., 2017a; Zhang et al., 2018), 291(Yang et al., 2010; Martin et al., 2001) images are used for CRNet-A, while 800 (Timofte et al., 2017) images for CRNet-B. During testing, Set5 (Bevilacqua et al., 2012), Set14 (Zeyde et al., 2010), B100 (Martin et al., 2001), Urban100 (Huang et al., 2015) and Manga109 (Matsui et al., 2017) are employed. Both PSNR and SSIM (Wang et al., 2004) on Y channel of transformed YCbCr space are calculated for evaluation.
Parameter Settings We set n0 = 128, m0 = 256 for CRNet-A (i.e. every convolution layer contains 128 filters whileWl and S have 256 filters), while n0 = 64, m0 = 1024 for CRNet-B. We choose K = 25 in both of our models. We implement our models using the PyTorch (Paszke et al., 2017) framework with NVIDIA Titan Xp. More training details and parameter study could be found in A.4 and A.5.

heading: 5.2 COMPARISON WITH CSC-SR
text: We first compare our proposed models with the existing CSC based image SR method, i.e., CSCSR (Gu et al., 2015). Since CSC-SR utilizes LR images as input image, it can be considered as a post-upsampling method, thus CRNet-B is used for comparison. Tab. 1 presents that our CRNet-B clearly outperforms CSC-SR by a large margin.

heading: 5.3 COMPARISON WITH STATE OF THE ARTS
text: We now compare the proposed models with other state-of-the-arts in recent years. We compare CRNet-A with pre-upsampling models (i.e., VDSR (Kim et al., 2016a), DRCN (Kim et al., 2016b), DRRN (Tai et al., 2017a), MemNet (Tai et al., 2017b)) while CRNet-B with post-upsampling architectures (i.e., SRDenseNet (Tong et al., 2017), MSRN (Li et al., 2018), D-DBPN (Haris et al., 2018), EDSR (Lim et al., 2017), RDN (Zhang et al., 2018)). Similar to (Lim et al., 2017; Zhang et al., 2018), self-ensemble strategy (Lim et al., 2017) is also adopted to further improve the performance of CRNet-B, and we denote the self-ensembled version as CRNet-B+.
Tab. 2 and Tab. 3 show the quantitative comparisons on the benchmark testing sets. Both of our models achieve superior performance against the state-of-the-arts, which indicates the effectiveness of our models. Qualitative results are provided in Fig. 3. Our methods tend to produce shaper edges and more correct textures, while other images may be blurred or distorted. More visual comparisons are available in A.6.
Fig. 4(a) and 4(b) shows the performance versus the number of parameters, our CRNet-B and CRNet-B+ achieve better results with fewer parameters than EDSR (Lim et al., 2017) and RDN (Zhang et al., 2018). It’s worth noting that EDSR/MDSR and RDN are far deeper than CRNetB (e.g., 169 vs. 36), but CRNet-B is quite wider (Wl and S have 1, 024 filters). As reported in (Lim et al., 2017), when increasing the number of filters to a certain level, e.g., 256, the training procedure of EDSR (for ×2) without residual scaling (Szegedy et al., 2018; Lim et al., 2017) is numerically unstable, as shown in Fig. 4(c). However, CRNet-B is relieved from the residual scaling
trick. The training loss of CRNet-B is depicted in Fig. 4(d), it converges fast at the begining, then keeps decreasing and finally fluctuates at a certain range.

heading: 5.4 ABLATION STUDY
text: The proposed networks combine the merits of CSC and Residual learning. Particularly, CSC problem is solved in a learnable manner, namely CISTA, where three techniques are naturally inferred, i.e. Local Residual Learning (LRL), Pre-Activation (PA) and Consistency Constraint (CC). Ablation studies are implemented to address the superiority of the proposed networks with the elementary blocks of CISTA by combining LRL, PA and CC. It is shown in Tab. 1 that the network simultaneously containing LRL, PA and CC gives the best performance.

heading: 6 DISCUSSIONS
text: We discuss the differences between our proposed models and several recent CNN models for SR with recursive learning strategy, i.e., DRRN (Tai et al., 2017a), SCN (Wang et al., 2015) and DRCN (Kim et al., 2016b). Due to the fact that CRNet-B is an extension of CRNet-A, i.e., the main part of CRNet-B has the same structure as CRNet-A, so we use CRNet-A here for comparison. The simplified structures of these models are shown in A.3, where the digits on the left of the recursion line represent the number of recursions.
Difference to DRRN. The main part of DRRN (Tai et al., 2017a) is the recursive block structure, where several residual units with BN layers are stacked. On the other hand, guided by (10), CRNetA contains no BN layers. Coinciding with EDSR/MDSR (Lim et al., 2017), by normalizing features, BN layers get rid of range flexibility from networks. Furthermore, BN consumes much amount of GPU memory and increases computational complexity. Experimental results on benchmark datasets under common-used assessments demonstrate the superiority of CRNet-A.
Difference to SCN. There are two main differences between CRNet-A and SCN (Wang et al., 2015): CISTA block and residual learning. Specifically, CRNet-A takes consistency constraint into consideration with the help of CISTA block, while SCN uses linear layers and ignores the information from the consistency prior. On the other hand, CRNet-A adopts residual learning, which is a powerful tool for training deeper networks. CRNet-A (30 layers) is much deeper than SCN (5 layers). As indicated in (Kim et al., 2016a), a deeper network has larger receptive fileds, so more contextual information in an image would be utilized to infer high-frequency details. In A.1, we show that more recursions, e.g., 48, can be used to achieve better performance.
Difference to DRCN. CRNet-A differs with DRCN (Kim et al., 2016b) in two aspects: recursive block and training techniques. In the recursive block, both local residual learning (Tai et al., 2017a) and pre-activation (He et al., 2016; Tai et al., 2017a) are utilized in CRNet-A, which are demonstrated to be effective in (Tai et al., 2017a). As for training techniques, DRCN is not easy to train, so recursive-supervision is introduced to facilitate the network to converge. Moreover, an ensemble strategy (the final output is the weighted average of all intermediate predictions) is used to further improve the performance. CRNet-A is relieved from these techniques and can be easily trained with more recursions.

heading: 7 CONCLUSIONS
text: In this work, we propose two effective CSC based image SR models, i.e., CRNet-A and CRNet-B, for pre-/post-upsampling SR, respectively. By combining the merits of CSC and CNN, we achieve superior performance against recent state-of-the-arts. Furthermore, our framework and CISTA block are expected to be applicable in various CSC based tasks, though in this paper we focus on CSC based image SR.

heading: A APPENDIX
text: A.1 RESIDUAL vs. NON-RESIDUAL
A.2 CRNET-B
A.3 GRAPHICAL COMPARISON TO OTHER RECURRENT CNNS IN SR
A.4 IMPLEMENTATION DETAILS
CRNet-A To enlarge the training set, data augmentation, which includes flipping (horizontally and vertically), rotating (90, 180, and 270 degrees), scaling (0.7, 0.5 and 0.4), is performed on each image of 291 dataset, where 91 of these images are from Yang et al. (Yang et al., 2010) with the
addition of 200 images from Berkeley Segmentation Dataset (Martin et al., 2001). Scale augmentation is exploited by combining images of different scales (×2, ×3 and ×4) into one training set. Furthermore, all training images are partitioned into 33 × 33 patches with the stride of 33, providing a total of 1, 929, 728 ILR-HR training pairs. Every convolution layer in CRNet-A contains 128 filters (n0 = 128) of size 3 × 3 while Wl and S have 256 filters (m0 = 256). We follow the same strategy as He et al. (He et al., 2015) for weight initialization where all weights are drawn from a normal distribution with zero mean and variance 2/nout, where nout is the number of output units. The network is optimized using SGD with mini-batch size of 128, momentum parameter of 0.9 and weight decay of 10−4. The learning rate is initially set to 0.1 and then decreased by a factor of 10 every 10 epochs. We train a total of 35 epochs as no further decrease of the loss can be observed. For maximal convergence speed, we utilize the adjustable gradient clipping strategy stated in (Kim et al., 2016a), with gradients clipped to [−θ, θ], where θ = 0.4 is the gradient clipping parameter. CRNet-B We use the 800 training images of the DIV2K (Timofte et al., 2017) dataset to train CRNet-B, and all the images are pre-processed by substracting the mean RGB value of the DIV2K dataset. Data augmentation includes random horizontal flips and rotations. Every weight layer in CRNet-B has 64 filters (n0 = 64) with the size of 3 × 3 except Wl and S have 1, 024 filters (m0 = 1, 024). CRNet-B is updated using Adam (Kingma & Ba, 2014) with β1 = 0.9, β2 = 0.999, and = 10−8. The mini-batch size is 16. We use RGB LR patches of size 54 × 54 as inputs. For each epoch, there are 103 iterations. The initial learning rate is 10−4 and halved every 200 epochs. We train CRNet-B for 800 epochs. Unlike CRNet-A, CRNet-B is trained using L1 loss for better convergence speed.
Recursion We choose K = 25 in both of our models. We implement our models using the PyTorch (Paszke et al., 2017) framework with NVIDIA Titan Xp. It takes approximately 4.5 days to train CRNet-A, and 15 days to train CRNet-B.
A.5 PARAMETER STUDY
The key parameters in both of our models are the number of filters (n0,m0) and recursions K.
Number of Filters We set n0 = 128,m0 = 256,K = 25 for CRNet-A as stated in Section A.4. In Fig. 8(a), CRNet-A with different number of filters are tested (DRCN (Kim et al., 2016b) is used for reference). We find that even n0 is decreased from 128 to 64, the performance is not affected greatly. On the other hand, if we decrease m0 from 256 to 128, the performance would suffer an obvious drop, but still better than DRCN (Kim et al., 2016b). Based on these observations, we set the parameters of CRNet-B by making m0 larger and n0 smaller for the trade off between model size and performance. Specifically, we use n0 = 64,m0 = 1024,K = 25 for CRNet-B. As shown in Fig. 8(b), the performance of CRNet-B can be significantly boosted with larger m0 (MDSR (Lim et al., 2017) and MSRN (Li et al., 2018) are used for reference). Even with small m0, i.e., 256, CRNet-B still outperforms MSRN (Li et al., 2018) with fewer parameters (2.0M vs. 6.1M).
Number of Recursions We also have trained and tested CRNet-A with 15, 20, 25, 48 recursions, so the depth of the these models are 20, 25, 30, 53 respectively. The results are presented in Fig. 9(a). It’s clear that CRNet-A with 20 layers still outperforms DRCN with the same depth and increasing K can promote the final performance. The results of using different recursions in CRNet-B are shown in Fig. 9(b), which demonstrate that more recursions facilitate the performance improved.
A.6 MORE QUALITATIVE RESULTS
In Fig. 10-15, we provide additional visual results on benchmark datasets to clearly show the superiority of our proposed models. The pre-upsampling models (SRCNN (Dong et al., 2016), VDSR (Kim et al., 2016a), DRCN (Kim et al., 2016b), DRRN (Tai et al., 2017a), MemNet (Tai et al., 2017b)) and the post-upsampling models (MSRN (Li et al., 2018), EDSR/MDSR (Lim et al., 2017), RDN (Zhang et al., 2018)) are used for comparisons. Red indicates the best performance.

Review summaries:
This paper presents a single image super-resolution method. The discussion refers to sparse coding as motivation and how such coding can be achieved within a neural network using activation functions, and then it slides into iterative soft thresholding and ends up with the observation that ReLU and the soft nonnegative thresholding operator are equal (paper should have pointed that they are equal with slight difference around the bias term). This argument is not new, this conclusion has already been known and analyzed extensively in (Papyan et al., 2017a).