NHNet: A non‐local hierarchical network for image denoising

IET Image ProcessingEarly View ORIGINAL RESEARCHOpen Access NHNet: A non-local hierarchical network for image denoising Jiahong Zhang, Jiahong Zhang orcid.org/0000-0002-5687-1839 State Key Laboratory of Media Convergence and Communication, Communication University of China, Beijing, China Neuroscience and Intelligent Media Institute, Communication University of China, Beijing, ChinaSearch for more papers by this authorLihong Cao, Corresponding Author Lihong Cao lihong.cao@cuc.edu.cn State Key Laboratory of Media Convergence and Communication, Communication University of China, Beijing, China Neuroscience and Intelligent Media Institute, Communication University of China, Beijing, China State Key Laboratory of Mathematical Engineering and Advanced Computing, Wuxi, China Correspondence Lihong Cao, State Key Laboratory of Media Convergence and Communication, Communication University of China, Beijing, China. Email: lihong.cao@cuc.edu.cnSearch for more papers by this authorTian Wang, Tian Wang State Key Laboratory of Media Convergence and Communication, Communication University of China, Beijing, China Neuroscience and Intelligent Media Institute, Communication University of China, Beijing, ChinaSearch for more papers by this authorWenlong Fu, Wenlong Fu State Key Laboratory of Media Convergence and Communication, Communication University of China, Beijing, China Neuroscience and Intelligent Media Institute, Communication University of China, Beijing, ChinaSearch for more papers by this authorWeiheng Shen, Weiheng Shen Data Science and Media Intelligence, Communication University of China, Beijing, ChinaSearch for more papers by this author Jiahong Zhang, Jiahong Zhang orcid.org/0000-0002-5687-1839 State Key Laboratory of Media Convergence and Communication, Communication University of China, Beijing, China Neuroscience and Intelligent Media Institute, Communication University of China, Beijing, ChinaSearch for more papers by this authorLihong Cao, Corresponding Author Lihong Cao lihong.cao@cuc.edu.cn State Key Laboratory of Media Convergence and Communication, Communication University of China, Beijing, China Neuroscience and Intelligent Media Institute, Communication University of China, Beijing, China State Key Laboratory of Mathematical Engineering and Advanced Computing, Wuxi, China Correspondence Lihong Cao, State Key Laboratory of Media Convergence and Communication, Communication University of China, Beijing, China. Email: lihong.cao@cuc.edu.cnSearch for more papers by this authorTian Wang, Tian Wang State Key Laboratory of Media Convergence and Communication, Communication University of China, Beijing, China Neuroscience and Intelligent Media Institute, Communication University of China, Beijing, ChinaSearch for more papers by this authorWenlong Fu, Wenlong Fu State Key Laboratory of Media Convergence and Communication, Communication University of China, Beijing, China Neuroscience and Intelligent Media Institute, Communication University of China, Beijing, ChinaSearch for more papers by this authorWeiheng Shen, Weiheng Shen Data Science and Media Intelligence, Communication University of China, Beijing, ChinaSearch for more papers by this author First published: 15 April 2022 https://doi.org/10.1049/ipr2.12499AboutSectionsPDF ToolsRequest permissionExport citationAdd to favoritesTrack citation ShareShare Give accessShare full text accessShare full-text accessPlease review our Terms and Conditions of Use and check box below to share full-text version of article.I have read and accept the Wiley Online Library Terms and Conditions of UseShareable LinkUse the link below to share a full-text version of this article with your friends and colleagues. Learn more.Copy URL Share a linkShare onFacebookTwitterLinked InRedditWechat Abstract With the fast development of deep learning models, hierarchical convolutional neural networks have achieved great success in image denoising tasks. To further boost the performance of image denoising, a novel non-local hierarchical network (NHNet) is proposed. Unlike existing U-Net-based hierarchical methods, which mainly focus on downsampling operations, NHNet adopts an initial resolution path and a high resolution path. Specifically, the high-resolution features are obtained through upsampling, where the non-local mechanism is adopted to capture the self-similarity properties, which contribute to a better denoising performance. Cross connections and channel attention layers are added between the two paths to integrate features in different resolutions. Compared with other U-Net-based hierarchical networks, NHNet requires fewer parameters. Experiments show that NHNet achieves state-of-the-art performance in Gaussian denoising tasks and gets competitive results when dealing with real image denoising. 1 INTRODUCTION Image denoising is a fundamental and long-lasting image processing topic, which aims to remove the external noises and reconstruct high-quality images [1]. As an important prerequisite for high-level vision tasks and practical application, the research of image-denoising techniques have attracted considerable attentions from both academic and industry. Current research mainly focuses on two kinds of noise images: synthetic noise images and real noise images. The former is artificially generated by adding the synthetic noise to the clear image, where the additive white Gaussian noise (AWGN) is commonly used. The latter is directly obtained in real-life scenarios, such as the smartphone. Generally, for a given noisy image y, the problem of image denoising can be modeled as y = x + v $y=x+v$ , where x denotes the desired clean signal and v represents the noise. Various classical mathematical methods like block-matching and 3D filtering (BM3D) [2] and weighted nuclear norm minimization (WNNM) [3] have been adopted and demonstrated satisfactory performance in image denoising. However, their complex manual tuning of parameters and complicated optimization algorithms make them hard to be widely used [4]. Fortunately, recent dramatic progress on deep learning significantly promotes the development of image-denoising techniques. Motivated by the outstanding performance of Convolutional Neural Network (CNN) in a great number of computer vision tasks, CNN has been widely concerned with solving these problems. Zhang et al. presented a denoising convolutional neural network (DnCNN), where residual learning was adopted to remove noise [4]. Singh et al. proposed a deep convolution neural network with added benefits of residual learning (ResDNN) [5] to solve the problem of gradient vanishing. Then, Ren et al. presented a depth-wise separable DN-ResNet (DS-DN-ResNet), which utilized the proposed Depthwise Separable ResBlock (DS-ResBlock) instead of standard ResBlock [6] to improve performance. To better obtain the critical information, Tian et al. proposed an attention-guided CNN(ADNet) [7], which demonstrated the ability of attention mechanism in improving a CNN denoiser. While the above CNN-based denoisers can benefit from their deep-network architecture, they may face the difficulty of training deep networks and the problems like gradient vanishing or exploding. To better avoid these problems caused by the increased network depth, some studies have tried to expand the width of the network to form a hierarchical network. Tian et al. proposed double-path networks BRDNet [8] and DudeNet [9], which outperformed single-path networks. For treble-path, networks based on the hierarchical structure of the U-Net [10] were adopted. Liu et al. presented a multi-level wavelet CNN (MWCNN) [11] based on modified U-Net architecture and wavelet transform. Bumjun Park et al. replaced the convolution block in U-Net with densely connected residual (DCR) block and proposed a densely connected hierarchical image denoising network (DHDN) [12]. Long Bao et al. presented a multi-scale residual dense cascaded U-Net with block-connection (MCU-Net) by improving DCR block to multi-scale residual dense block (MRDB) [13]. Although current U-Net-based hierarchical denoising networks have achieved great success, they still face two limitations. The first is they do not explicitly take advantage of the self-similarity properties in images, which can effectively capture the correlation among non-local patches to improve the performance [14]. The second is that they do not consider the effect of upsampling and instead directly focus on the process of downsampling when adopting the U-Net architecture. To overcome the two above-mentioned limitations, we propose a non-local hierarchical network (NHNet), as shown in Figure 1. It consists of a high-resolution path (HRP), an initial resolution path (IRP), and a Tail. HRP and IRP are responsible for feature extraction at high resolution and original resolution, respectively. Tail performs feature integration and outputs the processed image. Non-local operations are adopted in the upsampling process, offering HRP reliable feature correlations for self-similarity measurements. FIGURE 1Open in figure viewerPowerPoint Overall architecture of the proposed NHNet and structure of key components. NHNet consists of a high-resolution path (HRP), an initial resolution path (IRP), and a Tail. C represents convolutional layer, BN stands for Batch Normalization, R denotes ReLU activation function and PR denotes parametric rectified linear unit [15] The main contributions of this work can be concluded as: 1) To the best of our knowledge, we are the first to employ upsampling path in a hierarchical network for the image denoising. A novel high resolution path, HRP, is designed for rich, high-resolution feature extraction. 2) Non-local mechanism is used during the upsampling process in HRP, providing features related to self-similarity measurements. 3) NHNet is proposed to perform image denoising task. NHNet consists of a HRP for high-resolution feature extraction through upsampling operations, an IRP to extract initial-resolution level feature, and a Tail to integrate features and generate the output image. 4) NHNet achieves state-of-the-art performance on denoising tasks for color images with Gaussian noise and obtains competitive performance on real image denoising tasks. Compared with other existing U-Net-based image denoising models, NHNet has fewer parameters and moderate computation consumption. 2 RELATED WORK 2.1 Hierarchical network With the fast development of deep learning, CNNs have been widely applied to various computer vision tasks. While increasing the network's depth can improve the performance to some extent, the increased depth makes it difficult to train the model efficiently and may suffer from gradient vanishing or exploding. To alleviate the problem caused by increased network depth, researchers have applied the hierarchical network in various image processing tasks such as image segmentation [10], object detection [16], human pose estimation [17], traffic sign detection [18]. In image denoising, hierarchical networks have been employed and achieved good performance by offering abundant features from different image resolution paths. DHDN [12] and MCU-Net [13] adopted three different resolution paths: original resolution, 1/2 resolution, and 1/4 resolution. Both of them achieved outstanding performance in the New Trends in Image Restoration and Enhancement (NTIRE) Real Image Denoising Challenge. Furthermore, the adoption of hierarchical architecture is not limited to generating paths for different resolutions but also can be used for different image transforms. MWCNN [11] employed hierarchical paths by using discrete wavelet transform (DWT) and performed well on several image restoration tasks. However, they do not consider the effect of upsampling when adopting the U-Net architecture. In this study, we adopt the hierarchical network architecture and explore the role of upsampling path. 2.2 Non-local mechanism Non-local self-similarity in natural images has been proven to be an effective prior for image restoration tasks [14]. Nonlocally centralized sparse representation (NCSR) [19] was proposed to use image self-similarity for image restoration. Liu et al. proposed a non-local recurrent network (NLRN) to incorporate non-local operations into a recurrent neural network (RNN), which achieved good results with fewer parameters [14]. Li et al. presented a novel hyperspectral and multispectral image fusion method based on non-local low-rank tensor approximation and sparse representation to obtain high-resolution HS images [20]. For image denoising tasks, image self-similarity is also helpful when extracting features. The non-local mechanism can be used to obtain the non-local self-similarity in the image and contribute to a better denoising performance by removing the noise considering the neighbor information of the pixel. Ma et al. combined non-local with CNN and proposed an enhanced non-local cascading network with the attention mechanism (ENCAM) to remove noise on hyperspectral remote sensing images (HSIs) [21]. These studies demonstrate the potential of non-local mechanism in capturing self-similarity information, contributing to better performance in image restoration tasks. Inspired by these studies, we adopt non-local mechanism during the upsampling process to introduce self-similarity measurements. 3 PROPOSED METHOD In this section, we illustrates the proposed NHNet architecture, as shown in Figure 1. NHNet takes the input of gray image or color image, removes the noise and outputs a clear image. For a given image, it is first processed by a convolution layer with 1 × $\ensuremath{\times{}}$ 1 convolution kernels to generate 64 feature maps that share the same shape as the input image. These feature maps are then duplicated and sent to IRP and HRP, respectively. Specifically, IRP extracts features on the initial resolution, and HRP receives the upsampled feature map and then extracts high-resolution features. HRP is responsible for high-resolution feature extraction, which is obtained through upsampling. IRP is adopted to extract features on the initial resolution. Both IRP and HRP rely on the main processing unit, which adopts the improved dense connected block and is denoted as D _ $\_$ Block (see Figure 1b). Convolution layers in D _ $\_$ Block are set as kernel size 3 × 3 $3\ensuremath{\times{}}3$ , stride 1, and padding 1. Cross connections between IRP and HRP are added to promote the interaction among features for different resolutions. Each cross connection is followed by efficient channel attention (ECA) [22]. Outputs of IRP and HRP are concatenated and further processed by the Tail to get the output image more smoothly. During training, NHNet receives the image input, y = x + v $y=x+v$ , where x represents the desired clean image and v is the additive noise. As NHNet adopts residual learning, the essence of the network is to learn the residual mapping F ( y ) ≈ v . F ( x ) $F(y)\approx v.F(x)$ is mainly composed of three parts: IRP, HRP, and Tail. According to Figure 1, the mapping function F HRP ${F}_{\textit{HRP}}$ of HRP can be derived as follows. F hrp 1 = f ( x up 1 , w irp 1 ) F hrp 2 = f ( Conc at ( F hrp 1 , x up 2 ) , w hrp 2 ) F HRP = f ( Conc at ( F hrp 2 , x up 3 ) , w hrp 3 ) , \begin{equation} \def\eqcellsep{&}\begin{array}{rl}{{F}_{\textit{hrp}1}}& {=f({x}_{\textit{up}1},{w}_{\textit{irp}1})}\\ {{F}_{\textit{hrp}2}}& {=f(\textit{Conc}\textit{at}({F}_{\textit{hrp}1},{x}_{\textit{up}2}),{w}_{\textit{hrp}2})}\\ {{F}_{\textit{HRP}}}& {=f(\textit{Conc}\textit{at}({F}_{\textit{hrp}2},{x}_{\textit{up}3}),{w}_{\textit{hrp}3})}\end{array} , \end{equation} (1)where x upi ${x}_{\textit{upi}}$ denotes the ith upsampled feature map, w hrpi ${w}_{\textit{hrpi}}$ is the parameter set, and f is the mapping function of the ith combination of two D _ $\_$ Blocks. Conc at $\textit{Conc}\textit{at}$ stands for the matrix splicing operation. Similarly, we can get the mapping function of IRP, F IRP ${F}_{\textit{IRP}}$ . Taking the Tail into account, the final F ( y ) $F(y)$ can be represented as: F ( y ) = f t ( Conc at ( F HRP , F IRP ) , w t ) . \begin{equation} F(y)={f}_{t}(\textit{Conc}\textit{at}({F}_{\textit{HRP}},{F}_{\textit{IRP}}),{w}_{t}). \end{equation} (2) w t ${w}_{t}$ and f t ${f}_{t}$ are the parameter set and mapping function of the Tail, respectively. Then we have x ′ = y − F ( y ) ${x}^{{}^{\prime}}=y-F(y)$ , and our goal is to make x ′ ${x}^{{}^{\prime}}$ as close to x as possible. Mean square error is used as the loss function: L ( Θ ) = 1 2 N ∑ n ( ( y i − F ( y i , Θ ) ) − x i ) 2 , i ∈ [ 1 , N ] , \begin{equation} L(\mathrm{\Theta})=\frac{1}{2N}\sum _{n}{(({y}_{i}-F({y}_{i},\mathrm{\Theta}))-{x}_{i})}^{2},i\in [1,N], \end{equation} (3)where Θ is the trainable parameter. 3.1 Upsamping and downsampling operations Since noise is not evenly distributed in every pixel in the picture, it can be efficiently removed with the help of the self-similarity measurements in the image. In NHNet, the non-local mechanism is used during the upsampling process in HRP to provide self-similarity information. As shown in Figure 1, the upsampling operation consists of a PixelShuffer layer, a non-local module, and a 1 × 1 $1\ensuremath{\times{}}1$ convolution layer. Specifically, PixelShuffer is an upsampling method based on efficient sub-pixel convolutional [23]. We set the upscale factor 2 to get x2 resolution features. After upsampling, the number of channels becomes a quarter of the previous. We adopt the non-local module proposed in [14], illustrated in Figure 2. The neighborhood pixels q is set to 4 and 2 for synthetic noise and real noise, respectively. The smaller q for real noise is a consideration of saving computing resources. FIGURE 2Open in figure viewerPowerPoint The illustration of non-local module is from [14]. The white box denotes the deep feature representation of an entire image. The dark blue point is the feature of this location, and the light blue square denotes the neighborhood features. m is the number of channels of the input feature. The output of this module on each location only depends on its q × q $q\ensuremath{\times{}}q$ neighborhood. l is the number of channels of the intermediate feature. The multiplication used here is matrix multiplication. This non-local module is pluggable and does not change the size of the input feature For downsampling operations, we use a convolution layer with kernel size 4 × 4 $4\ensuremath{\times{}}4$ , stride 2, and padding 1. After downsampling, the number of channels is increased by 4 times. 3.2 Densely connected block The Densely connected convolutional network (DenseNet) [24] is adopted in NHNet to establish the connection between different layers, making full use of the feature to reduce the gradient vanishing problem further. As shown in Figure 1, our D _ $\_$ Block consists of a densely connected block and an extra C+BN+R. The added C+BN+R is responsible for alleviating the internal covariate shift magnified by the cross connections. In addition, BN can prevent gradient explosion or dispersion and improve the robustness of the training model to different super parameters such as learning rate and initialization. 4 EXPERIMENTS We evaluate the performance of the proposed NHNet on synthetic and real datasets and compare it with previous methods. 4.1 Implementation details The models were implemented using python 3.5, pytorch 1.5.1 with cuda 10.1. NHNet is trained end-to-end on NVIDIA Tesla P100. For synthetic noise training, we use Adam to optimize the Equation (3). The initial learning rate is set as 0.0001, and other settings of Adam are set as default. Before training, we flip the train images horizontally and vertically and randomly divide them into 64 × $\ensuremath{\times{}}$ 64 patches. The mini-batch size is set to 64. 4.2 Datasets 4.2.1 Synthetic noise datasets The training set includes 400 images (Set400) in [25], 400 images from the validation set of ImageNet [26] and 4,744 images from the Waterloo Exploration Database [27]. This setting refers to [28]. AWGN is added to these images to form the synthetic noise dataset. We use the same Gaussian noise generation algorithm as [4], in which the noise level is determined by the standard deviation σ. NHNet is trained with three noise levels, 15, 25, and 50. Both gray and color images are used for testing. We choose commonly used Set12 [29] and BSD68 [29] datasets to test the performance on gray images. Set12 includes 12 images such as C.man, House, Peppers, which are shown in Figure 5. Set5 [30], CBSD68 [29], Kodak24 [31] and McMaster [32] are used as color image testing sets. FIGURE 3Open in figure viewerPowerPoint The weight of channels from IRP and DRP in NHNet FIGURE 4Open in figure viewerPowerPoint The weight of channels from IRP and DRP in NHNet without non _ $\_$ local module FIGURE 5Open in figure viewerPowerPoint The image in Set12 are C.man, House, Peppers, Starfish, Monarch, Airplane, Parrot, Lena, Barbara, Boat, Man and Couple in order 4.2.2 Real noise datasets The sRGB track of the Smartphone Image Denoising DATA set (SIDD) [40] is used as the training set. These training images come from 160 different scene instances, which were captured by five smartphone cameras under different lighting conditions and camera settings. Each scene instance has two pairs of high-resolution images, and each pair includes one noisy image and its corresponding ground-truth image. In total, there are 320 training image pairs. We use the benchmarks of SIDD [40] and the Darmstadt Noise Data set (DND) [41] as the testing sets. It should be noted that for SIDD dataset, the validation data and benchmark data are used as two testing sets. As for DND dataset, it does not provide any training data, models tested on DND are trained on SIDD training set. PSNR/SSIM results are obtained through the online submission system. 4.3 Results 4.3.1 Synthetic noise results Tables 2 and 3 demonstrate the gray image denoising performance of NHNet and other popular methods on Set12 and BSD68, respectively. The best and second-best results are highlighted in red and blue. It can be seen that NHNet outperforms other popular methods on all three noise levels in both Set12 and BSD58, demonstrating the effectiveness of NHNet. Compared with the double-path model BRDNet [8] on Set12, the average PSNR of NHNet is 0.16 dB higher for σ= 25, indicating the effectiveness of NHNet. The performance of different denoisers on Set12 with noise level 50 is visualized in Figure 7, where NHNet shows the best visual effect. The performance of NHNet on different noise levels is further presented in Figure 8. Among all images in Set12, NHNet performed best on House and worst on Airplane, demonstrating that NHNet's better ability in dealing with simple texture images. TABLE 1. Comparison of the indicators between NHNet and the popular U-Net-based methods. Total MAdd represents the theoretical total number of multiply and add operations, and total MemR+W denotes the amount of reading and writing operations. The best and second-best results are highlighted in red and blue, respectively Methods Indicators ( 3 × 128 × 128 ) $(3\ensuremath{\times{}}128\ensuremath{\times{}}128)$ ( 3 × 256 × 256 ) $(3\ensuremath{\times{}}256\ensuremath{\times{}}256)$ ( 3 × 512 × 512 ) $(3\ensuremath{\times{}}512\ensuremath{\times{}}512)$ U-Net [10] Total params 34,527,041 34,527,041 34,527,041 Total memory 129.19 MB 516.75 MB 2067.00 MB Total MAdd 32.72GMAdd 130.89GMAdd 523.58GMAdd Total Flops 16.38GFlops 65.51GFlops 262.03GFlops Total MemR+W 373.71 MB 1.07 GB 3.91 GB DHDN [12] Total params 168,195,125 168,195,125 168,195,125 Total memory 419.88 MB 1679.50 MB 6718.00 MB Total MAdd 509.8GMAdd 2.04TMAdd 8.16TMAdd Total Flops 255.01GFlops 1.02TFlops 4.08TFlops Total MemR+W 1.64 GB 4.7 GB 6.9 GB NHNet Total params 8,167,906 8,167,906 8,167,906 Total memory 1008.20 MB 4032.76 MB 16,131.01 MB Total MAdd 308.46GMAdd 1.23TMAdd 4.94TMAdd Total Flops 154.43GFlops 617.71GFlops 2.47TFlops Total MemR+W 2.27 GB 8.99 GB 35.88 GB TABLE 2. PSNR (dB) results for different networks on Set12 with noise levels of 15, 25, 50 Images C.man House Peppers Starfish Monarch Airplane Parrot Lena Barbara Boat Man Couple Average Noise level σ = 15 $\sigma =15$ BM3D [2] 31.91 34.93 32.69 31.14 31.85 31.07 31.37 34.26 33.10 32.13 31.92 32.10 32.37 DnCNN [4] 32.61 34.97 33.30 32.20 33.09 31.70 31.83 34.62 32.64 32.42 32.46 32.47 32.86 FFDNet [28] 32.43 35.07 33.25 31.99 32.66 31.57 31.81 34.62 32.54 32.38 32.41 32.46 32.77 ResDNN [5] 32.73 34.99 33.23 32.11 33.20 31.65 31.87 34.57 32.56 32.39 32.42 32.43 32.85 U-Net [10] 32.33 34.79 33.16 32.00 32.94 31.64 31.84 34.46 32.43 32.30 32.34 32.31 32.71 ADNet [7] 32.81 35.22 33.49 32.17 33.17 31.86 31.96 34.71 32.80 32.57 32.47 32.58 32.98 DudeNet [9] 32.71 35.13 33.38 32.29 33.28 31.78 31.93 34.66 32.73 32.46 32.46 32.49 32.94 BRDNet [8] 32.80 35.27 33.47 32.24 33.35 31.82 32.00 34.75 32.93 32.55 32.50 32.62 33.03 NHNet 32.95 35.40 33.60 32.36 33.55 31.98 32.10 34.80 33.14 32.65 32.57 32.69 33.15 Noise level σ = 25 $\sigma =25$ BM3D [2] 29.45 32.85 30.16 28.56 29.25 28.42 28.93 32.07 30.71 29.90 29.61 29.71 29.97 DnCNN [4] 30.18 33.06 30.87 29.41 30.28 29.13 29.43 32.44 30.00 30.21 30.10 30.12 30.43 FFDNet [28] 30.10 33.28 30.93 29.32 30.08 29.04 29.44 32.57 30.01 30.25 30.11 30.20 30.44 ResDNN [5] 30.17 32.99 30.73 29.24 30.30 29.00 29.38 32.31 29.70 30.11 30.04 29.96 30.33 U-Net [10] 30.18 33.18 30.91 29.38 30.41 29.18 29.57 32.59 30.19 30.25 30.10 30.14 30.51 ADNet [7] 30.34 33.41 31.14 29.41 30.39 29.17 29.49 32.61 30.25 30.37 30.08 30.24 30.58 DudeNet [9] 30.23 33.24 30.98 29.53 30.44 29.14 29.48 32.52 30.15 30.24 30.08 30.15 30.52 BRDNet [8] 31.39 33.41 31.04 29.46 30.50 29.20 29.55 32.65 30.34 30.33 30.14 30.28 30.61 NHNet 30.49 33.65 31.20 29.72 30.68 29.34 29.65 32.76 30.70 30.44 30.20 30.40 30.77 Noise level σ = 50 $\sigma =50$ BM3D [2] 26.13 29.69 26.68 25.04 25.82 25.10 25.90 29.05 27.22 26.78 26.81 26.46 26.72 DnCNN [4] 27.03 30.00 27.32 25.70 26.78 25.87 26.48 29.39 26.22 27.20 27.24 26.90 27.18 FFDNet [28] 27.05 30.37 27.54 25.75 26.81 25.89 26.57 29.66 26.45 27.33 27.29 27.08 27.32 ResDNN [5] 26.63 29.27 26.68 25.31 26.27 25.35 26.01 28.80 24.48 26.72 26.90 26.25 26.56 U-Net [10] 27.42 30.48 27.67 25.92 26.94 25.89 26.66 29.84 27.02 27.42 27.30 27.17 27.48 ADNet [7] 27.31 30.59 27.69 25.70 26.90 25.88 26.56 29.59 26.64 27.35 27.17 27.07 27.37 DudeNet [9] 27.22 30.27 27.51 25.88 26.93 25.88 26.50 29.45 26.49 27.26 27.19 26.97 27.30 BRDNet [8] 27.44 30.53 27.67 25.77 26.97 25.93 26.66 29.73 26.85 27.38 27.27 27.17 27.45 NHNet 27.54 30.85 27.84 26.24 27.10 26.00 26.76 29.83 27.19 27.46 27.32 27.28 27.62 TABLE 3. Results for different networks on BSD68 Network BM3D [2] DnCNN [4] FFDNet [28] ADNet [7] DudeNet [9] BRDNet [8] U-Net [10] RIDNet [33] DHDN [12] NHNet σ = 15 $\sigma =15$ 31.07 31.72 31.62 31.74 31.78 31.79 31.54 31.81 − − $--$ 31.85 σ = 25 $\sigma =25$ 28.57 29.23 29.19 29.25 29.29 29.29 29.13 29.34 − − $--$ 29.37 σ = 50 $\sigma =50$ 25.62 26.23 26.30 26.29 26.31 26.36 26.39 26.40 26.43 26.43 FIGURE 6Open in figure viewerPowerPoint Denoising result on butterfly in Set5. (a) clear image, (b)noisy image, (c) FFDNet, (d) VDN, (e) NHNet FIGURE 7Open in figure viewerPowerPoint Denoising results of image Monarch from Set12 with noise level 50 using different methods: (a) original image, (b) noisy image/14.71 dB, (c) DnCNN/26.78 dB, (d) BM3D/25.82 dB, (e) BRDNet/26.97 dB, and (f) NHNet/27.10 dB FIGURE 8Open in figure viewerPowerPoint Denoising results of of NHNet on image Monarch from Set12 with different noise level. (a), (b), and (c) are images with noise level 15, 25, and 50, respectively. (d), (e), (f) are the denoising results of NHNet, and the PSNR is 33.55 dB, 30.68 dB, and 27.10 dB, respectively To demonstrate NHNet's ability on color image denoising, we compare it to several state-of-the-art methods, including BRDNet [8] and VDN [34]. According to the results in Table 4, NHNet achieves outstanding performance in different color image datasets. Figures 6 and 9 demonstrate the performance of different denoisers on Set5 and Kodak24, respectively. It can be seen that NHNet can successfully remove the noise and retain important details. TABLE 4. Color image denoising results of different networks Datasets Methods σ = 15 $\sigma =15$ σ = 25 $\sigma =25$ σ = 50 $\sigma =50$ Set5 CBM3D [2] 33.42 30.92 28.16 FFDNet [28] 34.30 32.10 29.25 VDN [34] 34.34 32.24 29.47 NHNet 34.80 32.56 29.64 CBSD68 CBM3D [2] 33.52 30.71 27.38 FFDNet [28] 33.80 31.18 27.96 DnCNN [4] 33.98 31.31 28.01 ADNet [7] 33.99 31.31 28.04 DudeNet [9] 34.01 31.34 28.09 VDN [34] 33.90 31.35 28.19 BRDNet [8] 34.10 31.43 28.16 NHNet 34.20 31.54 28.31 Kodak24 CBM3D [2] 34.28 31.68 28.46 FFDNet [28] 34.55 32.11 28.99 DnCNN [4] 34.73 32.23 29.02 ADNet [7] 34.76 32.26 29.10 DudeNet [9] 34.81 32.26 29.10 BRDNet [8] 34.88 32.41 29.22 NHNet 35.02 32.54 29.41 FIGURE 9Open in figure viewerPowerPoint Denoising results of different methods on one image from the Kodak24 with σ = 50 $\sigma =50$ : (a) noisy image,