# Non-uniform Noise Image Denoising Based on Non-local Means

Jiaxin Li, Jing Hu, Min Wei, Bin Zhang, Yanfang Wang

► To cite this version:

Jiaxin Li, Jing Hu, Min Wei, Bin Zhang, Yanfang Wang. Non-uniform Noise Image Denoising Based on Non-local Means. 10th International Conference on Intelligent Information Processing (IIP), Oct 2018, Nanning, China. pp.437-445, 10.1007/978-3-030-00828-4_45. hal-02197770

HAL Id: hal-02197770

https://inria.hal.science/hal-02197770

Submitted on 30 Jul 2019

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
Non-uniform Noise Image Denoising Based on Non-local
Means

Jiaxin Li¹, Jing Hu¹, Min Wei¹, Bin Zhang¹ and Yanfang Wang²

¹ Chengdu University of Information Technology, College of Computer, Chengdu, China{1351825273@qq.com, jing_hu09@163.com, weimin@cuit.edu.cn, bynnzhang@foxmail.com}
² Hohai University, College of Computer and Information, Nanjing, China yanfang_wang08@163.com

**Abstract.** Restricted by camera hardware, digital images captured by digital cameras are noisy, and the noise content of each color channel of the digital image is not balanced. However, most of the existing denoising algorithms assume that the entire image noise is constant, causing errors in the denoising of color images (non-uniform noise images), affecting image noise removal and texture detail protection. To solve this problem, we propose an evaluation operator that can describe the noise content and texture content in the local area of the image. According to the description value, the image pixels are classified, and heuristic denoising parameters are selected for each class to achieve a balance between noise removal effect and texture retention effect. Experimental results of multiple denoising methods show that the proposed algorithm has better denoising effect on color images.

**Keywords:** Non-uniform Noise Image, Non-local Means, Image Denoising.

# 1 Introduction

Digital images help us learn. It can transmit and store information. Digital image is often polluted during the process of acquisition and transmission that the image quality is reduced. Therefore, image noise removal is an important research direction of image processing that has been extensively studied in the past several decades [1]-[8]. Most existing denoising methods are concentrated in additive white Gaussian noise (AWGN) [1]-[16], in which the observed noisy image is modeled as a composition of clean image and AWGN noise: $z(i) = x(i) + n(i)$. It is important to note that most of these methods assume that the noise variance of the entire image is fixed so that will inevitably bias the denoising result in the subsequent experiments, which will also have a certain impact on the subsequent application.

As a matter of fact, the noise level in the real noisy image is often non-uniform. In other words, the noise variance is not fixed and is randomly distributed over the entire image. Nam et al. [17] pointed out that the real color image can be modeled as mixed
Gaussian noise among different color channels, and a Bayesian non-local mean denoising algorithm is employed in their paper to denoise images with non-uniform noise. Other related denoising methods are also proposed recently [18]-[22]. For example, Xu et al. [18] proposed a combined method which leverages a guided external prior and internal prior learning for non-uniform noise image denoising. A multi-channel (MC) denoising model is proposed in Xu et al. [19] to use the redundancy between color channels to distinguish different noise statistics among color channels for real color image denoising. Tian et al. [20] proposed a new Direction-of-arrival (DOA) estimation algorithm, which is suitable for dealing with unknown non-uniform noise. In Chen et al. [21], an adaptive BM3D filter was proposed to deal with non-uniform noise images, and in Plötz and Roth [22], a noise reduction algorithm for real photos was proposed. In this paper, a new image denoising method within the framework of non-local means (NLM) regarding non-uniform noise is proposed. Compare with the above algorithm, the proposed algorithm does not need to learn from the image and is computational efficient. More specifically, an evaluation operator is leveraged to measure local patch's noise level and texture strength. After that, image pixel classification is carried out according to the evaluation value. And parameters of non-local means are heuristically selected accordingly. The main contributions of the proposed method are:

(1) The proposed algorithm is devised for non-uniform noise images.

(2) An evaluation operator is used to roughly obtain pixel noise level and texture strength, and then a voting strategy is used to distinguish smooth and texture image areas for more accurate denoising.

(3) For regions containing different image texture degree, the inner parameters of NLM are adaptively selected according to patch property, leading to better denoising results.

The rest of the paper is organized as follows: In the next section, the proposed algorithm is described in detail. Experimental results are provided in Section3 which compares the proposed algorithm with other state-of-the-art image denoising algorithms. Section4 concludes this paper.

# 2 The Proposed Method

## 2.1 Non-uniform Noise Model

Non-uniform noise images can be expressed as in Nam et al. [17]:

$$z(i) = x(i) + n(i) \quad n(i) \sim N(0, \delta(i)) \quad (1)$$

Where $x(i)$ refers to the intensity of a noise-free image at pixel $i$, $n(i)$ is the non-uniform white noise, and $\delta(i)$ is the noise standard deviation, $z(i)$ is the non-uniform noisy image. As for color images, non-uniform noise is added respectively to the R, G, B color channels.
## 2.2 Framework of NLM

In 2005, Buades et al. [5] proposed a non-local means (NLM) denoising algorithm. Its basic idea is that the estimated value of the current pixel value is calculated by weighted average of the pixels in the image that have a similar neighbourhood structure, and the weight function is determined according to the similarity between pixels. NLM is defined as follows:

$$NL[x](i) = \sum_{j \in I} \omega(i, j)z(j), \quad \omega(i, j) = \frac{1}{C(i)} \exp\left(-\frac{\|z(N_i) - z(N_j)\|_{2,a}^2}{h^2}\right) \quad (2)$$

Where $x(i)$ is the denoised pixel value, $\omega(i, j)$ is the weight value between pixels, $z(j)$ is the pixel value of the noise image, $C(i)$ is the normalization parameter, and $h$ is the filter parameter. The algorithm makes full use of the self-similarity of the image and the redundancy of the structure information and achieves a good denoising effect.

## 2.3 Evaluation Operator for Noise Level and Texture Strength

From the traditional NLM, we can draw a conclusion that a large size of image patch works well on a smooth area, while a small patch size is suitable for a texture area. Furthermore, if bandwidth parameter is large, it is not suitable for retaining details, but a small bandwidth leads to a poor denoising effect. Therefore, the image can be divided into the texture and the flat area, so that we can set parameters suitable for the area for different regions, in order to obtain better denoising effects and preserve the details of the image. First, we use an evaluation operator to roughly obtain the noise distribution of the image and the structure of the image, and then a voting strategy is embedded to accurately distinguish different areas of the image.

Our evaluation operator follows the way illustrated in our previous work Hu and Luo [10] to perform rough image pixel classification. $R$ is a combination of noise level indicator $H$ and texture descriptor $F$. And the mechanism for $H$ is that noisy image's eigenvalues $\tilde{S}_{i,1}^2, \tilde{S}_{i,2}^2$ (in descending order) are increasing with the local noise variance $\delta_i^2$ when the original clean image patch is corrupted by noise.

$$H(i) = \frac{\tilde{S}_{i,1}^2 - \tilde{S}_{i,2}^2}{\tilde{S}_{i,1}^2 + \tilde{S}_{i,2}^2} = \frac{\mathbf{S}_{i,1}^2 - \mathbf{S}_{i,2}^2}{\mathbf{S}_{i,1}^2 + \mathbf{S}_{i,2}^2 + 2\xi N^2 \delta_i^2}, \quad F(i) = 1 - \frac{1}{1 + \xi_i / 255^2} \quad (3)$$

$$R(i) = \left(1 - \frac{1}{1 + \xi_i / 255^2}\right) \frac{\tilde{S}_{i,1}^2 - \tilde{S}_{i,2}^2}{\tilde{S}_{i,1}^2 + \tilde{S}_{i,2}^2} \quad (4)$$

The $\xi_i$ in formula (3) refers to the second order moment of the grayscale cumulative histogram with pixel $i$ as the neighbourhood (9×9 in the experiment), and $\tilde{S}_{i,1}^2, \tilde{S}_{i,2}^2$ is the structural tensor of the neighbourhood. It can be seen from Eq. (3) that the value of $H(i)$ is determined by the local nature of the image (flatness and texture) and the local noise criteria. Because neither $\delta$ nor the eigenvalues are fixed, the value of $H(i)$ cannot effectively discriminate the comparative strength between noise and texture, which
would leads to classification error in the texture area. Therefore, R(i) can only obtain a
rough pixel classification result.

2.4 Vote Strategy for Image Pixel Classification

Based on the cumulative histogram of R, it was determined that classification thresholds
T₁, T₂, T₃ are determined by the 30%, 70%, and 90% of the R cumulative histogram.
Hence, the whole image is divided into 4 parts by way of voting. The texture area with
a small noise variance (c₁), the medium texture area (c₂), the texture area with a large
noise variance (c₃) and the flat area (c₄). We take a patch centered on pixel i in R (the
patch size in the experiment is 5×5). The r(j) is the corresponding value in the patch.
count(r) represents the number of r(j) that satisfied the condition. f₁, f₂, f₃, f₄ corre-
spond to the count value of area c₁, c₂, c₃, c₄ respectively.

$$
\begin{equation}
\left\{
\begin{array}{ll}
\text{if } r(j) \ge T_i & \text{then } \sum \text{count}(r) = f_1 \\
\text{if } T_2 \le r(j) \le T_3 & \text{then } \sum \text{count}(r) = f_2, i \in \left\{ \begin{array}{ll} c_1 & \text{if } \max(f_1, f_2, f_3, f_4) = f_1 \\ c_2 & \text{if } \max(f_1, f_2, f_3, f_4) = f_2 \end{array} \right. \\
\text{if } T_i \le r(j) \le T_2 & \text{then } \sum \text{count}(r) = f_3 \\
\text{if } r(j) \le T_i & \text{then } \sum \text{count}(r) = f_4
\end{array}
\right.
\tag{5}
\end{equation}
$$

This is where the last area of pixel *i* belongs. We take a patch centered on pixel *i* in *R*, and compare the values in the patch one by one with *T*₁, *T*₂, *T*₃, and count the value that satisfies the condition. Which finally the count value of the area is the biggest, which area the pixel belongs to. Fig. 1 demonstrates the classification results on noisy image and clean image respectively. The noise level is between 1 and 30. Fig. 1. (c) has slight distortions in some places. These areas are smooth areas (dark blue) in Figure 1(b) but are divided into sub-textured areas (light blue) in Fig. 1. (c). This is due to the fact that a simple texture descriptor *F* is very sensitive to noise leading to an incorrect classification. In addition, we also presented the classification results based on the classification method in Hu and Luo [10]. Compared with (b), we can see that there is the same problem as in (c), but its distortion is more serious.

<figure><img src="image_2.png" /><figcaption>**Fig. 1.** The color image with a noise variance of 1-40 is based on the block diagram of R. c₁ is orange, c₂ is green, c₃ is light blue, c₄ is dark blue. (a) original image; (b) original image classification; (c) noise image classification; (d) classification of literature [10].</figcaption></figure>
## 2.5 Adaptive Setting of Denoising Parameters

Patch size and bandwidth are two important parameters in the NLM denoising algorithms. Taking into the consideration that a large patch size is more suitable for smooth regions and a small patch size is more suitable for texture regions, we choose the patch size for the area types $c_1, c_2$ and $c_4$ to be 7×7,9×9 and 13×13 respectively. Moreover, since $c_3$ is a texture area with a large noise variance, which implies the noise variance exceeds the ones in texture area $c_1$ and $c_2$. In order to make a better denoising effect and a better texture preservation for this area, the neighbourhood block of this area should be small enough. In our experiments, it is chosen to be a size of 5×5. As for the bandwidth parameter, it is devised as in Zeng et al. [13]:

$$h(i) = \begin{cases} a_1 \times \delta \times \exp(-(D_1 / \beta_1)^2) & \text{if } i \in c_1 \\ a_2 \times \delta \times \exp(-(D_2 / \beta_2)^2) & \text{if } i \in c_2 \\ a_3 \times \delta \times \exp(-(D_3 / \beta_3)^2) & \text{if } i \in c_3 \\ a_4 \times \delta \times \exp(-(D_4 / \beta_4)^2) & \text{if } i \in c_4 \end{cases} \quad (6)$$

Where $a_1, a_2, a_3, a_4$ are constant values 2.4, 2.5, 2.6, and 2 respectively, $\delta$ is the estimated noise variance by Chen et al. [23]. Note that Chen's method assumes that the noise level of the entire image is fixed. Therefore, this method cannot accurately estimate the noise level in the non-uniform noisy image, $D_i$ is the average R value for the region, $\beta$ is computed according to the MAD estimator,

$$\beta_j = b * C * \text{median}(|R_j - \text{median}(R_j)|), \quad R_j = \begin{cases} (R > T_3) * R & \text{if } j=1 \\ (R > T_2) * R & \text{if } j=2 \\ (R > T_1) * R & \text{if } j=3 \\ R & \text{if } j=4 \end{cases} \quad (7)$$

Where $|\cdot|$ and $\text{median}(\cdot)$ denote norm and median operator, respectively. $C = 1.4826 \times \nu$, where $\nu$ is the variance of $R_j$.

## 3 Experimental Results and Analysis

In the following experiments, RGB image is transformed into the YCbCr space first. The proposed method is only performed on the Y channel for computational efficiency, and the other two channels are denoised using a Gaussian filter. Both the synthetic non-uniform noise images and real noise color images are used in our experiments. As shown Fig. 2, six images including bikes, man-fishing, coin-fountain, ocean, building2 and woman are used. Three different non-uniform noise are added to the image, within the range of [1, 20], [1, 30] and [1, 40], respectively.

The traditional NLM algorithm, the algorithm of Hu and Luo [10] are used for comparison. The median of the noise variance is used as the guidance for the bandwidth parameters for the compared algorithms. Peak signal-to-noise ratio (PSNR) and structural similarity (SSIM) are used for quantitative comparison.
<figure><img src="image_3.png" /><figcaption>**Fig. 2.** Natural images for simulation experiments from the Kodak PhotoCD Dataset. (a) bikes, (b) man-fishing, (c) coin-fountain, (d) ocean, (e) building2, (f) woman</figcaption></figure>

<table><caption>**Table 1.** PSNR/SSIM results for different denoising methods on Fig.2 (a) and (b).</caption><thead><tr><th rowspan="2"></th><th colspan="2">δ=[1,20]</th><th colspan="2">δ=[1,30]</th><th colspan="2">δ=[1,40]</th></tr><tr><th>(a)</th><th>(b)</th><th>(a)</th><th>(b)</th><th>(a)</th><th>(b)</th></tr></thead><tbody><tr><td>image</td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>NLM</td><td>28.5631/0.914</td><td>30.2644/0.917</td><td>27.0786/0.8169</td><td>27.5628/0.8262</td><td>24.5086/0.7633</td><td>25.8374/0.7519</td></tr><tr><td>[10]</td><td>30.5522/0.9623</td><td>31.7479/0.9607</td><td><b>29.2278/0.9141</b></td><td><b>29.6065/0.9275</b></td><td><b>27.1558/0.9044</b></td><td><b>28.0344/0.8888</b></td></tr><tr><td>ours</td><td><b>30.5912/0.9581</b></td><td><b>31.8431/0.9605</b></td><td>28.8453/0.9153</td><td>29.3094/0.9312</td><td>26.4557/0.9045</td><td>27.1822/0.8964</td></tr></tbody></table>

<figure><img src="image_4.png" /><figcaption>**Fig. 3.** Denoised images on image bike contained by Gaussian noise within the noise standard deviation [1, 40]. Zoom for better comparison</figcaption></figure>

<figure><img src="image_5.png" /><figcaption>**Fig. 4.** Seven cropped noiseless images used in the experiment.</figcaption></figure>

<table><caption>**Table 2.** PSNR/SSIM results for different denoising methods on natural images.</caption><thead><tr><th>Camera Settings</th><th>NLM</th><th>[10]</th><th>ours</th></tr></thead><tbody><tr><td>Canon 5D Mark</td><td>38.4817/0.9689</td><td>38.5712/0.9657</td><td><b>41.3221/0.9893</b></td></tr><tr><td rowspan="2">III ISO = 3200</td><td>35.5861/0.9394</td><td><b>36.0402/0.9449</b></td><td>29.6237/0.9689</td></tr><tr><td>34.8774/0.9388</td><td>34.9745/0.9348</td><td><b>37.4064/0.9838</b></td></tr><tr><td>Nikon D600</td><td>35.6644/0.9423</td><td>35.9615/0.9486</td><td><b>37.8382/0.9806</b></td></tr><tr><td>ISO = 3200</td><td></td><td></td><td></td></tr><tr><td>Nikon D800</td><td>40.0291/0.9682</td><td>39.862/0.9654</td><td><b>41.4639/0.9798</b></td></tr><tr><td>ISO = 1600</td><td><b>36.4954/0.9153</b></td><td>36.3499/0.9111</td><td>23.313/0.9181</td></tr><tr><td>Nikon D800</td><td>33.5755/0.9006</td><td><b>34.0995/0.9098</b></td><td>32.919/0.9256</td></tr><tr><td>ISO = 6400</td><td></td><td></td><td></td></tr></tbody></table>
<figure><img src="image_6.png" /><figcaption>**Fig. 5.** Denoised images of a region cropped from the real noisy image "Canon 5D Mark 3 ISO 3200 1" [18] by different methods. The images are better to be zoomed in on screen.</figcaption></figure>

Table 1 shows the PSNR and SSIM results for different denoising methods on Fig.2 (a) and (b). It can be seen from Table 1 that when the image noise is small, our method has a better effect on noise removal. When the noise is large, our method is better on the texture protection of the image. Fig.3 shows the denoised results on noisy image that is contained by noise range [1, 40]. It can be clearly seen that our algorithm is better for noise removal and texture protection. Next, our method is evaluated on real images from the dataset provided by Xu et al. [18], where images are captured either indoor or outdoor lighting conditions with different types of camera and camera settings. Each noise image in this dataset has an average image, which can be regarded as a “ground truth”. Fig.4 shows the selected real noise images in our experiment. Table 2 shows the PSNR and SSIM results for different denoising methods on this dataset. We can see that our method is more capable of real image denoising. When our method is applied to a real natural image, the protection of the texture of the image and the effect of denoising are superior to other methods. Fig.5 shows the denoised images of a scene captured by Canon 5D Mark III at ISO = 3200. We can see that our method is better at texture preservation and noise removal.

**Acknowledgments.** This work was supported in part by the National Natural Science Foundation of China under Grant 61602065, Sichuan province Key Technology Research and Development project under Grant 2017RZ0013, Scientific Research Foundation of the Education Department of Sichuan Province under Grant No.17ZA0062; J201608 supported by Chengdu University of Information and Technology (CUIT) Foundation for Leader of Disciplines in Science, project KYTZ201610 supported by the Scientific Research Foundation of CUIT.

**References**

1. S. G. Chang, B. Yu, and M. Vetterli. Adaptive wavelet thresholding for image denoising and compression. IEEE Transactions on Image Processing, 9(9):1532–1546, 2000.

2. J. L. Starck, E. J. Cand`es, and D. L. Donoho. The curvelet transform for image denoising. IEEE Transactions on Image Processing, 11(6):670–684, 2002.

3. K. Dabov, A. Foi, V. Katkovnik, and K. Egiazarian. Color image denoising via sparse 3D collaborative filtering with grouping constraint in luminance-chrominance space. IEEE International Conference on Image Processing (ICIP), pages 313–316, 2007.

4. C.TomasiandR.Manduchi. Bilateralfilteringforgrayandcolorimages. IEEE International Conference on Computer Vision (ICCV), pages 839–846, 1998.
5. A. Buades, B. Coll, and J. M. Morel. A non-local algorithm for image denoising. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 60–65, 2005.

6. S. Gu, L. Zhang, W. Zuo, and X. Feng. Weighted nuclear norm minimization with application to image denoising. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 2862–2869, 2014.

7. D. Zoran and Y. Weiss. From learning models of natural image patches to whole image restoration. IEEE International Conference on Computer Vision (ICCV), pages 479–486, 2011.

8. Y. Chen, W. Yu, and T. Pock. On learning optimized reaction diffusion processes for effective image restoration. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 5261–5269, 2015.

9. W. L. Zeng, X. B. Lu. Region-based non-local means algorithm for noise removal, lectronics Letters, 2011.

10. Jing Hu, Yu-Pin Luo. Non-local means algorithm with adaptive patch size and bandwidth, Optik 124 (2013) 5639–5645.

11. Hongjun Li, Ching Y. Suen, A novel Non-local means image denoising method based on grey theory, Pattern Recognition, http://dx.doi.org/10.1016/j.patcog.2015.05.028.

12. Rajiv Verma, Rajoo Pandey. Non Local Means Algorithm with Adaptive Isotropic Search Window Size for Image Denoising, IEEE INDICON 2015 15701706.

13. Weili Zeng, Yijun Du, Changhui Hu. Noise Suppression by Discontinuity Indicator Controlled Non-local Means Method, 2017.

14. Kaiqun Leng. An Improved Non-Local Means Algorithm for Image Denoising, 2017 IEEE 2nd International Conference on Signal and Image Processing.

15. Sanjay Ghosh, Amit K. Mandal, Kunal N. Chaudhury. Pruned non-local means, IET Image Processing, 2017.

16. Liuyun Zhang, Chao Hu, Shuangqing Wu, Tian Wang, Jialin Cui, Jun Qiu. An improved non-local means image denoising algorithm, 2017 IEEE International Conference on Information and Automation (ICIA).

17. Seonghyeon Nam, Youngbae Hwang, Yasuyuki Matsushita, Seon Joo Kim. A Holistic Approach to Cross-Channel Image Noise Modeling and its Application to Image Denoising, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

18. Jun Xu, Lei Zhang, David Zhang. External Prior Guided Internal Prior Learning for Real Noisy Image Denoising, Computer Vision and Pattern Recognition, 12 May 2017.

19. Jun Xu, Lei Zhang, David Zhang, Xiangchu Feng. Multi-channel Weighted Nuclear Norm Minimization for Real Color Image Denoising, Computer Vision and Pattern Recognition, 28 May 2017.

20. Ye Tian, Hongyin Shi and He Xu. DOA estimation in the presence of unknown non-uniform noise with coprime array, lectronics Letters, 2017.

21. Guangyi Chen, Guangchun Luo, Ling Tian, Aiguo Chen. Noise Reduction for Images with Non-uniform Noise Using Adaptive Block Matching 3D Filtering, Chinese Journal of Electronics, 2017.

22. Tobias Plötz, Stefan Roth. Benchmarking Denoising Algorithms with Real Photographs, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

23. G. Chen, F. Zhu, and A. H. Pheng. An efficient statistical method for image noise level estimation. IEEE International Conference on Computer Vision (ICCV), December 2015.