Denoising convolutional neural network with mask for salt and pepper noise

IET Image ProcessingVolume 13, Issue 13 p. 2604-2613 Research ArticleFree Access Denoising convolutional neural network with mask for salt and pepper noise Jiuning Chen, Jiuning Chen School of Mathematical Sciences, East China Normal University, Shanghai, People's Republic of ChinaSearch for more papers by this authorFang Li, Corresponding Author Fang Li fli@math.ecnu.edu.cn School of Mathematical Sciences, East China Normal University, Shanghai, People's Republic of China School of Mathematical Sciences and Shanghai Key Laboratory of PMMP, East China Normal University, Shanghai, People's Republic of ChinaSearch for more papers by this author Jiuning Chen, Jiuning Chen School of Mathematical Sciences, East China Normal University, Shanghai, People's Republic of ChinaSearch for more papers by this authorFang Li, Corresponding Author Fang Li fli@math.ecnu.edu.cn School of Mathematical Sciences, East China Normal University, Shanghai, People's Republic of China School of Mathematical Sciences and Shanghai Key Laboratory of PMMP, East China Normal University, Shanghai, People's Republic of ChinaSearch for more papers by this author First published: 15 October 2019 https://doi.org/10.1049/iet-ipr.2019.0096Citations: 11AboutSectionsPDF ToolsRequest permissionExport citationAdd to favoritesTrack citation ShareShare Give accessShare full text accessShare full-text accessPlease review our Terms and Conditions of Use and check box below to share full-text version of article.I have read and accept the Wiley Online Library Terms and Conditions of UseShareable LinkUse the link below to share a full-text version of this article with your friends and colleagues. Learn more.Copy URL Share a linkShare onFacebookTwitterLinkedInRedditWechat Abstract In this study, the authors propose a new loss function for denoising convolutional neural network (DnCNN) for salt-and-pepper noise (SPN). Based on the motivation of utilising the mask of SPN, firstly from the usual SPN-denoising restoration equation, the authors establish a perfect restoration condition; the restored image is precisely the clean image if this condition holds. Then they design a mask-involved loss function to encourage the network to satisfy this condition in training progress. Experimental results demonstrate that compared with general DnCNN and other state-of-the-art SPN denoising methods, DnCNN equipped with the proposed loss function involving mask (MaskDnCNN) is more effective, robust and efficient. 1 Introduction Images are usually corrupted by impulse noise due to reasons such as malfunctioning pixels in camera sensors and faulty memory locations in hardware. The impulse noise includes two common types: salt-and-pepper noise (SPN) and random-valued noise. When corrupted by SPN, noisy pixels take only the maximum value or the minimum value, contributing to white and black dots on images. Existing methods for denoising SPN-corrupted images can be roughly classified into two categories. The first category is filter-based methods. They are approaches based on the standard median filter, for instance, the adaptive median filter [1], the progressive switching median filter [2], the multi-state median filter [3], the noise adaptive soft-switching median filter [4], the new detector for switching median filter [5], the noise adaptive fuzzy switching median filter [6], the modified decision based unsymmetric trimmed median filter [7], the fast and efficient median filter [8], the adaptive weighted mean filter [9] and the adaptive fuzzy switching weighted mean filter [10]. The second category is the variational model methods. These variational models usually contain one or two regularisation term and a data fitting term. The widely considered regularisation terms include the total variation [11-14], the wavelet tight frame [15, 16] and the norm [13] for the estimated noise mask. The widely studied data fitting term is norm error [11, 16], norm error [13, 15] and norm error [12, 14]. A new adaptive denoising technique based on autoencoders is proposed for blind denoising [17]. It does not require any training data, and it learns the autoencoder model while denoising, the results of which are quite promising. The profound progress of deep learning methods [18-24] suggests the power of network-based methods in image denoising. Methods in [18, 19] are typical denoising methods on Gaussian noise by training the convolutional neural network (CNN). Methods in [20-24] are delightful attempts at combining model-based methods and network-based methods for general denoising tasks. A non-local switching filter CNN denoising algorithm for SPN is proposed in [25]. It consists of two steps, i.e. a filter processing step and a CNN training step. The CNN is shallow, which contains three convolution layers. In this paper, we consider a deep CNN method for SPN denoising based on the effective denoising CNN (DnCNN) proposed by Zhang et al. [18]. As a type of network-based denoising method, DnCNN is general in the sense of being able to denoise any specific noisy image if trained by that type of noisy images. However, DnCNN fails to utilise essential information of noise directly if trained in this way. For example, the mask of SPN as prior knowledge can be used to guide SPN denoising. With the guidance of prior knowledge, it is expected to get better denoising performance. For this aim, we propose a new type of loss function specific to SPN in this paper, which involves the noise mask information as prior. Starting from the usual restoration procedure for the SPN-corrupted image, we establish a perfect restoration condition and then obtain a loss function that satisfies the condition. The main contribution of our paper is that we propose a new SPN-specific loss function involving a noise mask based on DnCNN, named as MaskDnCNN. The proposed method has two advantages: (i) promising denoising performance; (ii) reducing weights in the network with similar performance as DnCNN. The outline of this paper is as follows. The general frameworks of DnCNN, including non-residual learning and residual learning networks, are reviewed in Section 2. Our proposed mask-involved loss function is presented in Section 3.1. Experiments and comparisons with general DnCNN methods and traditional methods are presented in Section 4. Finally, we conclude the work in Section 5. 2 Denoising convolutional neural networks Generally, the deep neural network learns the most appropriate composite function that maps the problem set onto the solution set. Through parametrisation of functions, the learning progress can be regarded as a regression problem. For image denoising problem, let be the training set, where is the noisy image, is the clean image and K is the number of image pairs. Then the regression problem has the form of (1)where is the composition of simple parametric functions whose weights w are learned by networks. Loss functions in (1) for image restoration tasks are naturally deduced from loss criteria , common choices of which are - and - norm. Different choices of training labels contribute to different networks. Let f represent the learned mapping of CNN. If labels are clean images Y, then the commonly used loss function is (2)Trained by this loss function, the network output tends to be close to clean images Y. This type of network is called a non-residual learning network. If labels are image residuals , then the loss function is (3)Trained by this loss function, the network output tends to be close to image residuals N. This type of network is called a residual learning network. For example, the residual learning network is used in DnCNN [18] for Gaussian noise. In the next section, we propose an SPN-specific loss function for the non-residual learning networks, and equip our DnCNN network with this loss function. 3 Proposed method 3.1 Loss function Let be the corrupted image and the clean image. One commonly used SPN model is (4)where defines the noise level. For simplicity, we set , , in this paper. To derive our loss function specific to SPN, an equivalent additive model is needed. Let and be the masks of salt noise and pepper noise, respectively, that is (5)and let be the mask of SPN where denotes the corrupted pixels and denotes the uncorrupted pixels. With these notations, we use the following equivalent SPN model: (6)Unless explicitly stated, element-wise multiplication is used throughout this paper. Under this additive noise model, we have (7)Note that the last equality holds since both and are all-zero arrays. According to [26], when the image-corruption mapping is approximately an identity mapping, the performance of residual learning network would be better than that of non-residual learning network. For SPN, the image-corruption mapping is far from identity. Hence, we choose non-residual learning network, i.e. labels are clean images. We then deduce our loss function for a non-residual learning network. Assume f is our desired non-residual learning network and its output is an estimation of the clean image, in the testing stage, we use the following procedure to recover the image: (8)where is the recovered image. The idea behind this restoration procedure is that we use the network to recover only corrupted pixels since the clean pixels are already known. The network output at uncorrupted pixels is not used in the testing stage since in (8) where . According to (7) and (8), to make perfect restoration, i.e. to let , the following condition must hold: (9)We name (9) as the perfect restoration condition. According to this condition, we need to minimise to acquire an optimal restoration. Thus, the loss function for non-residual learning network is designed as (10)where is the mask of . Remark that in loss function (10), only the corrupted pixels are considered. Hence in the training stage, it cannot be guaranteed that the output values of the uncorrupted pixels are close to the original values. However, as we have discussed below (8), this does not affect the restoration since in the testing stage only values of corrupted pixels are used to recover the image. In summary, instead of using the standard loss function (2), we use loss function (10) to train network with given architecture, and use restoration procedure (8) to restore the corrupted image. The difference between (2) and (10) is the noise mask . With this mask, our network focuses on minimising the errors of the noisy pixels such that it can better denoise the noisy pixels than (2). For noisy free pixel, the correct information is given, no denoising is needed. 3.2 Network architecture To validate the effectiveness of our proposed loss function, we use the VGG-like network [18] as the implementation of our network (MaskDnCNN), with loss function being a masked version of mean squared error, i.e. is -norm in (10). The main reason for this choice is its simplicity; better performance is possible if using the state-of-art network architecture. We train our network as a non-residual learning network. The training workflow of MaskDnCNN is shown in Fig. 1. Fig. 1Open in figure viewerPowerPoint Proposed non-residual MaskDnCNN for SPN In Fig. 1, Conv + BN + PReLU is the basic building block of this VGG-like networks. Batch normalisation layer at the beginning is removed since training data will be normalised before training. The last block consists of a single convolutional layer since the last layer is expected to be trainable. ReLU in [18] is replaced by PReLU [27] to prevent dead neurons. For engineering reasons, we treat the SPN mask as an accessory training label and pile it onto the clean image, resulting in a two-channel image label. Besides, an additional matrix replication layer Repmat is used to create a two-channel restored image so that the number of channels of the label and that of the network output are equal; the second channel of network output is not used in computing loss and gradient. Each convolutional layer consists of c filters of size , where c is a fixed integer representing the network width. The stride of the convolution filter is set to 1, and zero padding is used to ensure image size not shrink. We initialise the weights by the method in [27]. 4 Experimental results We generate our training set from 400 images in BSD500 image set [28] with mini-batch size 128. Rotation and flipping are used for data augmentation. The optimisation method used for training is Adam [29]. The learning rate drops exponentially from to . Learning rate drop periods are 20 and 10 for training epochs 50 and 25, respectively. In the training stage, we use the exact noise mask. While in the testing stage, following [14], we predicted the SPN mask M as (11)Remark that the noise detection mask given in (11) is rough. It will detect the clean pixels with a value of Smin or Smax as noise pixels. However, the drawback can be compensated. There are two reasons. Firstly, the ratio of purely white or black pixels in the natural greyscale image is averagely small. For example, in the widely used BSD300 grey images https://www2.eecs.berkeley.edu/Research/Projects/CS/vision/bsds/, the ratio of white and black pixels are 0.0055 and 0.0004, respectively. Secondly, even if the clean pixel is misclassified as noisy pixel, the proposed method can also restore the information from the network output. All networks are implemented in MATLAB R2018a with its built-in Neural Network Toolbox. We use one Nvidia GTX 1080ti GPU card and i7-6800K CPU (3.40 GHz), it takes about 8 h to train 50 epochs. Codes are available on the author's homepage http://math.ecnu.edu.cn/∼fli/Publication/index.html. We use three test sets: Set12 [18] which includes 12 classic test images with size or ; BSD68 [30] which includes 68 images with size or ; DIV2K [31] which includes 100 high-resolution images from DIV2K with size around . All the colour images are transformed to grey image by Matlab routine 'rgb2gray'. 4.1 Performance evaluation In this subsection, we compare the proposed MaskDnCNN method with several state-of-the-art model-based denoising methods, including TV-PADMM [14], TV-SBM [32], TV-PDA [14] and TV-AOP [13]. We have tuned the parameters of the four model-based algorithms to get the best results for fair comparison. We also compare our method with the closely related deep learning method DnCNN [18] which is retrained with clean and SPN noisy image pairs. The implementation codes of the model-based methods are collected from the author website of [14]. Parameter settings are slightly adjusted to gain higher denoising performances. All networks are of depth 17 and width 64 and are trained for 50 epochs with drop rate period 20. For quantitative evaluation, we report the peak signal-to-noise ratio (PSNR)/structural similarity (SSIM) values of different methods on each dataset in Tables 1–3, respectively. The results with higher PSNR and SSIM are preferred. The best values are marked in bold. In the three tables, our MaskDnCNN achieves the best PSNR and SSIM results at all noise levels among all the competing methods. The deep learning methods including DnCNN and our MaskDnCNN are much better than the four model-based methods including TV-PADMM [14], TV-SBM [32], TV-PDA [14] and TV-AOP [13]. For example, DnCNN and our MaskDnCNN gain about 6 dB higher PSNR on Set 12, and about 3.5 dB higher PSNR on BSD68 and DIV2K than all the other methods on average. Table 1. Denoising result on Set12 Set12 TV-PADMM [14] TV-SBM [32] TV-PDA [14] TV-AOP [13] DnCNN [18] MaskDnCNN Noise level PSNR SSIM PSNR SSIM PSNR SSIM PSNR SSIM PSNR SSIM PSNR SSIM 10 38.448 0.987 32.089 0.920 37.716 0.986 35.204 0.968 41.849 0.989 42.391 0.991 20 35.174 0.974 29.240 0.870 34.428 0.973 33.499 0.959 38.554 0.978 39.231 0.983 30 32.995 0.960 27.465 0.827 32.179 0.956 32.149 0.950 36.774 0.972 36.962 0.974 40 31.183 0.941 25.813 0.778 30.343 0.935 30.748 0.934 34.966 0.961 35.249 0.964 50 29.551 0.918 23.941 0.714 28.669 0.908 29.352 0.912 33.262 0.948 33.570 0.951 60 27.976 0.887 21.423 0.632 27.031 0.871 27.820 0.881 31.644 0.930 31.901 0.933 70 26.309 0.845 18.490 0.551 25.322 0.816 26.253 0.841 29.766 0.902 30.129 0.909 80 24.455 0.783 16.439 0.492 23.325 0.723 24.471 0.779 27.744 0.861 27.960 0.867 90 21.863 0.666 14.612 0.440 19.545 0.528 21.724 0.664 24.841 0.775 25.010 0.785 95 20.100 0.577 13.935 0.426 16.581 0.432 19.452 0.548 22.676 0.689 22.749 0.694 average 28.805 0.854 22.345 0.665 27.514 0.813 28.067 0.844 32.208 0.901 32.515 0.905 Table 2. Denoising result on BSD68 BSD68 TV-PADMM [14] TV-SBM [32] TV-PDA [14] TV-AOP [13] DnCNN [18] MaskDnCNN Noise level PSNR SSIM PSNR SSIM PSNR SSIM PSNR SSIM PSNR SSIM PSNR SSIM 10 38.193 0.986 30.668 0.885 37.189 0.984 34.716 0.964 41.418 0.989 42.231 0.992 20 34.594 0.97 27.947 0.807 33.704 0.966 32.874 0.949 37.872 0.977 38.493 0.982 30 32.255 0.950 26.288 0.747 31.323 0.944 31.525 0.934 35.888 0.968 36.037 0.97 40 30.417 0.925 24.786 0.685 29.471 0.916 30.211 0.912 33.954 0.952 34.141 0.955 50 28.816 0.894 23.134 0.614 27.884 0.881 28.952 0.884 32.192 0.931 32.411 0.934 60 27.265 0.852 20.849 0.535 26.363 0.834 27.526 0.845 30.52 0.903 30.679 0.905 70 25.761 0.799 18.106 0.468 24.822 0.769 26.126 0.791 28.83 0.862 29.024 0.867 80 24.138 0.725 15.939 0.419 22.987 0.665 24.533 0.716 27.086 0.806 27.214 0.811 90 21.953 0.605 14.096 0.384 19.120 0.460 21.971 0.589 24.742 0.703 24.862 0.710 95 20.413 0.526 13.354 0.369 16.027 0.376 19.396 0.476 22.994 0.615 23.035 0.619 average 28.381 0.823 21.517 0.591 26.889 0.78 27.783 0.806 31.550 0.871 31.813 0.875 Table 3. Denoising result on DIV2K DIV2K TV-PADMM [14] TV-SBM [32] TV-PDA [14] TV-AOP [13] DnCNN [18] MaskDnCNN Noise level PSNR SSIM PSNR SSIM PSNR SSIM PSNR SSIM PSNR SSIM PSNR SSIM 10 40.958 0.985 32.295 0.896 39.783 0.982 37.705 0.975 44.447 0.991 45.487 0.986 20 37.385 0.974 30.209 0.859 36.299 0.969 35.988 0.966 40.855 0.977 42.150 0.982 30 35.004 0.960 28.313 0.81 33.87 0.953 34.479 0.955 38.734 0.969 39.173 0.971 40 33.071 0.943 26.467 0.751 31.900 0.933 33.001 0.940 36.336 0.957 37.048 0.961 50 31.356 0.920 24.280 0.674 30.176 0.907 31.532 0.919 34.122 0.942 34.587 0.947 60 29.714 0.891 21.300 0.601 28.544 0.872 30.006 0.890 32.357 0.925 32.616 0.929 70 28.039 0.850 18.028 0.54 26.816 0.820 28.37 0.849 30.197 0.896 30.497 0.902 80 26.148 0.791 15.283 0.485 24.389 0.716 26.396 0.787 28.452 0.856 28.505 0.861 90 23.536 0.687 13.123 0.445 19.128 0.479 23.663 0.681 25.682 0.772 25.878 0.782 95 21.764 0.613 12.291 0.43 15.302 0.407 20.816 0.567 23.905 0.697 24.048 0.704 average 30.697 0.861 22.159 0.649 28.621 0.804 30.196 0.853 33.509 0.898 33.999 0.903 Our method also performs better than DnCNN on the three datasets. Averagely, our method gains about 0.207, 0.363 and 0.490 dB higher PSNR than DnCNN on Set12, BSD68 and DIV2K, respectively. Meanwhile, the SSIM values of our method are also higher than those of DnCNN. Particularly, the advantage of our method over DnCNN is more significant for low noise level SPN. We note that combining DnCNN with our detection mask can only enhance a little bit of PSNR over DnCNN, which are 0.02, 0.03, and 0.09 on Set12, BSD68 and DIV2K, respectively. In Table 4, we compare the computational cost on Set12. The shortest execution time for each noise level is marked in bold. We provide both CPU and GPU times for DnCNN [18] and MaskDnCNN, since the CNN layers in Neural Network Toolbox have not been optimised to run CPU codes efficiently. According to Table 4, the additional restoration procedure in (8) does not introduce GPU time. To remove the hardware starting time, for each algorithm, we run it three times and take the last one. Table 4. Computational cost on Set12 (CPU time: seconds) Noise level DnCNN [18] (GPU) MaskDnCNN (GPU) DnCNN [18] (CPU) MaskDnCNN (CPU) 50 7.5534 7.3307 23.3495 34.9497 90 7.1857 7.2291 21.2196 28.0901 Noise level TV-PADMM [14] TV-SBM [32] TV-PDA [14] TV-AOP [13] 50 21.1366 14.2786 39.5443 45.2698 90 22.5568 14.6109 32.8494 128.1553 For qualitative evaluation, we show some of the results on the image 'Lena' in Set12 and the image 'train' in BSD68. The two clean test images are shown in Fig. 2. We choose a median noise level equals 50% and an extremely high noise level equals 95%, which are representative. The noisy tested images are shown in Figs. 3 and 4. When the noise level is 50%, we can still see some details of the images (see Figs. 3b and 4b). However, when the noise level is as high as 95%, no useful information can be observed in the noisy images by the human eyes (see Figs. 3d and 4d). In Figs. 3c, 3e, 4c and 4e, we show the noise residue images which are calculated by subtracting the clean image from the noisy images. It is obvious that for 95% noise level, the useful information is contained in the residue images. Fig. 2Open in figure viewerPowerPoint Two test images (a) 'Lena' in Set12, size , (b) 'Train' in BSD68, size Fig. 3Open in figure viewerPowerPoint Test image 'Lena' in Set12 (a) Noisy (50%), (b) Residue (50%), (c) Mask (50%), (d) Noisy (95%), (e) Residue (95%), (f) Mask (95%) Fig. 4Open in figure viewerPowerPoint Test image 'train' in BSD68 (a) Noisy (50%), (b) Residue (50%), (c) Mask (50%), (d) Noisy (95%), (e) Residue (95%), (f) Mask (95%) In Fig. 5, we show the denoising results on the image 'Lena' with 50% SPN by six methods including TV-PADMM, TV-SBM, TV-PDA, TV-AOP, DnCNN and the proposed method. The PSNR and SSIM values are also reported below each result image. Visually, the result of TV-SBM is the worst in which the details are smoothed out. The results of the other five methods seem quite good. However, by carefully comparing the hair part, we find that DnCNN and our MaskDnCNN can better preserve the details than the others. The visual difference of our method and DnCNN is not apparent, however, our method gains higher PSNR and SSIM values. Fig. 5Open in figure viewerPowerPoint Qualitative evaluation on 'Lena' in Set12 with 50% noise (a) Result of TV-PADMM [14], PSNR = 34.627, SSIM = 0.941, (b) Result of TV-SBM [32], PSNR = 27.644, SSIM = 0.802, (c) Result of TV-PDA [14], PSNR = 33.513, SSIM = 0.935, (d) Result of TV-AOP [13], PSNR = 34.403, SSIM = 0.938, (e) Result of DnCNN [18], PSNR = 38.134, SSIM = 0.955, (f) Result of our MaskDnCNN, PSNR = 38.380, SSIM = 0.958 For a better comparison of details, in Fig. 6, we show the residue images of each method by subtracting the clean image from the denoised image. It is obvious that the residue of TV-SBM in Fig. 6b contains more edges and details than others. In the residue images – Figs. 6a, c and d, there are many details of the hair, the hat and the face, including eyes, nose and mouth. While in the residue images – Figs. 6e and f, the details of the hair are weaker than others, and there are almost no details of hat and face. It shows that DnCNN and our MaskDnCNN lose less detail information in the denoising process. Fig. 6Open in figure viewerPowerPoint Qualitative evaluation on Lena in Set12 with 50% noise (a) Residue of TV-PADMM, (b) Residue of TV-SBM, (c) Residue of TV-PDA, (d) Residue of TV-AOP, (e) Residue of DnCNN, (f) Residue of our MaskDnCNN In Fig. 7, we show the denoising results on the image 'Lena' with 95% SPN by six methods. TV-SBM cannot recover the image (see Fig. 7b). The result of TV-PDA in Fig. 7c is better than the result of TV-SBM, but worse than others. As shown in Figs. 7a and d, TV-PADMM and TV-AOP can almost recover the cartoon part without details. It is obvious that DnCNN and our MaskDnCNN give much better results than others since many details are recovered (see Figs. 7e and f). Meanwhile, in the residue images shown in Fig. 8, we also find that DnCNN and our MaskDnCNN lose much fewer details than others. In terms of PSNR and SSIM, our method is slightly better than DnCNN. Fig. 7Open in figure viewerPowerPoint Qualitative evaluation on Lena in Set12 with 95% noise (a) Result of TV-PADMM [14], PSNR = 23.508, SSIM = 0.696, (b) Result of TV-SBM [32], PSNR = 15.168, SSIM = 0.557, (c) Result of TV-PDA [14], PSNR = 18.348, SSIM = 0.550, (d) Result of TV-AOP [13], PSNR = 22.793, SSIM = 0.681, (e) Result of DnCNN [18], PSNR = 26.960, SSIM = 0.784, (f) Result of our MaskDnCNN, PSNR = 26.996, SSIM = 0.787 Fig. 8Open in figure viewerPowerPoint Qualitative evaluation on Lena in Set12 with 95% noise (a) Residue of TV-PADMM, (b) Residue of TV-SBM, (c) Residue of TV-PDA, (d) Residue of TV-AOP, (e) Residue of DnCNN, (f) Residue of our MaskDnCNN In Fig. 9, we show the denoising results on the image 'train' with 50% SPN by six methods. The PSNR and SSIM values are also reported. The result of TV-SBM is over smooth and is the worst among all. The results of the other five methods seem similar. However, by carefully comparing the train and the bridge, we can see that DnCNN and our MaskDnCNN can better preserve the details than others. Meanwhile, the difference is more apparent by observing the residue images in Fig. 10. Both DnCNN and our method can preserve the details of the train and bridge, which are better than others. Again, our method achieves higher PSNR and SSIM than DnCNN. Fig. 9Open in figure viewerPowerPoint Qualitative evaluation on the train image in BSD68 with 50% noise (a) Result of TV-PADMM [14], PSNR = 26.399, SSIM = 0.907, (b) Result of TV-SBM [32], PSNR = 21.282, SSIM = 0.661, (c) Result of TV-PDA [14], PSNR = 25.720, SSIM = 0.893, (d) Result of TV-AOP [13], PSNR = 25.982, SSIM = 0.897, (e) Result of DnCNN [18], PSNR = 28.301, SSIM = 0.936, (f) Result of our MaskDnCNN, PSNR = 28.423, SSIM = 0.938 Fig. 10Open in figure viewerPowerPoint Qualitative evaluation on the train image in BSD68 with 50% noise (a) Residue of TV-PADMM, (b) Residue of TV-SBM, (c) Residue of TV-PDA, (d) Residue of TV-AOP, (e) Residue of DnCNN, (f) Residue of our MaskDnCNN The denoising results by the six methods on the image 'train' with 95% SPN are displayed in Fig. 11. Similar as Fig. 7, TV-SBM cannot recover the image in Fig. 11b. TV-PDA gives better result in Fig. 11c than TV-SBM, but worse than others. TV-PADMM and TV-AOP can almost recover the cartoon part, but lose all the details (see Figs. 11a and d), while in the results of DnCNN and our MaskDnCNN (see Figs. 11e and f), the details are much better preserved than others. Moreover, the residue images shown in Fig. 12 also agree with the above analysis. In terms of PSNR and SSIM measures, our method is better than DnCNN. Fig. 11Open in figure viewerPowerPoint Qualitative evaluation on the train image in BSD68 with 95% noise (a) Result of TV-PADMM [14], PSNR = 18.816, SSIM = 0.558, (b) Result of TV-SBM [32], PSNR = 14.087, SSIM = 0.449, (c) Result of TV-PDA [14], PSNR = 16.219, SSIM = 0.476, (d) Result of TV-AOP [13], PSNR = 17.897, SSIM = 0.518, (e) Result of DnCNN [18], PSNR = 20.841, SSIM = 0.657, (f) Result of our MaskDnCNN, PSNR = 20.890, SSIM = 0.658 Fig. 12Open in figure viewerPowerPoint Qualitative evaluation on the train image in BSD68 with 95% noise (a) Residue of TV-PADMM, (b) Residue of TV-SBM, (c) Residue of TV-PDA, (d) Residue of TV-AOP, (e) Residue of DnCNN, (f) Residue of our MaskDnCNN 4.2 More comparison with DnCNN [18] To validate the superiority of MaskDnCNN over DnCNN [18] on SPN, we compare the PSNR values of each method with different noise level r, network width c, network depth d and training epochs. Both networks are implemented in the same code framework with the same configuration; without otherwise specified, we fixed noise level 30, network depth 11 and width 32. In Fig. 13, we show the PSNR performance curves of the two networks on Set12 dataset [18], trained by 25 and 50 epochs, respectively. Fig. 13Open in figure viewerPowerPoint PSNR comparison of DnCNNs [18] and MaskDnCNNs. The default network width is 32 and depth is 11. For each network, two curves for 25 and 50 training epoches are shown (a) PSNR versus noise level, (b) PSNR versus network width (noise level = 30), (c) PSNR versus network depth (noise level = 30) Fig. 13a shows the PSNR curves of DnCNN and MaskDnCNN at different noise levels. We observe a significant superiority of MaskDnCNN over DnCNN for low-level and middle-level SPN. When the noise level becomes too high since the mask of SPN gives little information to guide the denoising procedure, MaskDnCNN degenerates into DnCNN; this complies with the observation that PSNR gaps between DnCNN and MaskDnCNN decrease. Nevertheless, MaskDnCNNs always perform better than DnCNN at each noise level. When networks are trained for 25 epochs, this difference becomes more significant. The comparisons in Figs. 13b and c demonstrate how network width and depth affect the denoising performance. It can be seen that both networks are limited by their depth and width. However, compared with DnCNN, MaskDnCNN is more stable and robust to this limitation; the PSNR curves of MaskDnCNN are smoother than that of DnCNN. For instance, with the width decreasing from 64 to 32, or with the depth decreasing from 21 to 11, there is no drastic performance loss for MaskDnCNN. When networks are trained for 25 epochs, this difference becomes more significant as well. In Table 5, we show numerical results of DnCNN [18] and MaskDnCNN with full size (depth 17 and width 64), and with compressed size (depth 11 and width 32), respec