# NAMF: A Nonlocal Adaptive Mean Filter for Removal of Salt-and-Pepper Noise

Authors: Houwang Zhang; Yuan Zhu; Hanying Zheng
Journal: Mathematical Problems in Engineering
 (2021)

---

## Abstract

In this paper, a novel algorithm called a Nonlocal Adaptive Mean Filter (NAMF) for removing salt-and-pepper (SAP) noise from corrupted images is presented. We employ an efficient window detector with adaptive size to detect the noise. The noisy pixel is then replaced by the combination of its neighboring pixels, and finally, a SAP noise based nonlocal mean filter is used to reconstruct the intensity values of noisy pixels. Extensive experimental results demonstrate that NAMF can obtain better performance in terms of quality for restoring images at all levels of SAP noise.


---

## Body

## 1. Introduction

Digital images are often corrupted by noises in the process of image acquisition and transmission [1, 2]. Image noise is a random change in the brightness or color information in an image, which will bring troubles for image processing and computer vision, even causing some seriously ill-posed problems [3]. Hence, image denoising has been taken as one of the most important tasks in image processing [4, 5]. As a preprocessing step in image processing, image denoising can protect edges, textures, and other details in the image [6]. Generally, image noises can be divided into Gaussian noise, Poisson noise, and Salt-and-Pepper (SAP) noise in terms of discreteness and randomness. SAP noise commonly exists in natural images, and the pixels contaminated by SAP noise take the maximum or minimum value and can be represented as black or white points [7, 8]. Considering the universality of SAP noise, we focus our attention on the SAP noise and develop a denoising method that provides some help for further image analysis.To remove SAP noise, lots of computational methods have been proposed. Among them, median filter (MF) and adaptive median filter (AMF) [9] are the two most popular methods in the early stage. MF can restore image details well under low noise intensity, but it performs poorly when noise intensity is high [10]. AMF adopts the measure of the window with adaptive size, which makes it perform well in high noise intensity [11]. In addition, based on AMF, noise adaptive fuzzy switching median filter (NAFSMF) recognizes SAP noise by analyzing the histogram of noisy images and takes a fuzzy method to denoise [12]. Adaptive weighted mean filter (AWMF) uses two successive windows to detect noisy pixels and processes them with a weighted mean filter [13]. The method proposed in [14] is based on NAFSMF and AWMF, and it uses a new adaptive fuzzy switching weighted mean filter to remove SAP noise. In [15], researchers proposed a method, based on pixel density filter (BPDF), to remove SAP noise through searching the repeated numbers of the pixels, and achieves a good performance under low SAP noise intensity. Different applied median filter (DAMF) is proposed for removing SAP noise at all densities [16]. In [17], adaptive frequency median filter (AFMF) is proposed to remove the SAP noise, which employs frequency median to restore gray values of the corrupted pixels. The adaptive switching weight mean filter (ASWMF) proposed in [18] repairs noisy pixels by assigning a value of a switching weight mean to the gray value of the center pixel of an adaptive window.The processes of the above denoising methods can be divided into two stages, the detection of noisy pixels and the restoration of noisy pixels. For the latter one, the restoration of SAP noise is using the rest information (uncontaminated pixels) to repair the absent information (contaminated pixels). As Figure1(b) shows, the limitation of existing state-of-the-art methods is that under high SAP noise level, the boundary of the restored image is jagged and the details are blurred. When SAP noise intensity is too high, the image loses too much information. The consequence is that less information is available to be used for restoring the image.Figure 1
(a) “Lena” corrupted by SAP noise ratio of 10%, and the gray values of noisy candidates inE and F are both 0. (b) The result of the method proposed [14] for “Lena” corrupted by SAP noise ratio of 90%.
(a)(b)To solve the above problem, in this paper, we propose a nonlocal adaptive mean filter (NAMF) to remove SAP noise. NAMF can be divided into two stages: (1) SAP noise detection; (2) SAP noise elimination. Firstly, in the stage of SAP noise detection, we find out pixels whose gray values are equal to the global maximum or minimum gray value of the image. We then take them as noisy candidates and use a window with adaptive size to confirm them. For the possible noisy pixel, we calculate the proportion of the pixels with the same value of candidate in the window and then filter it by a threshold. If it is smaller than the threshold, the candidate is regarded as a noisy pixel. Otherwise, it is noiseless and will not be processed. Secondly, in the stage of SAP noise elimination, the noisy pixel will be replaced by the mean of its neighboring pixels. Finally, we use a SAP noise based nonlocal mean filter to further restore it. The main contributions of this paper can be concluded as follows:(i)
A proportion based method is proposed to distinguish the noise pixels and texture pixels in the SAP noise detection, which can effectively avoid noiseless texture pixels being processed as noisy pixels.(ii)
An improved nonlocal mean method based on the characteristics of SAP noise is developed to further restore noisy images, which can further enhance the quality of restored images.In the experiments, we use 16 typical images and 40 test images collected from the TEST IMAGES Database [19] to evaluate our method. The experimental results demonstrate that NAMF outperforms the existing state-of-the-art methods at both high SAP noise levels and low SAP noise levels.


## 2. Related Works

In this section, we will firstly review traditional MF based methods used for SAP noise removal and then we introduce the classical nonlocal means method.


### 2.1. MF Based Methods for SAP Noise Elimination

MF is the most commonly used algorithm to remove pulse interference and SAP noise. The main idea is that all pixels are replaced by neighborhood median pixels whether pixels are noiseless or not. Detailedly, letX be an original 8-bit gray-level image with a size of M×N, and xi,j be the gray value of the pixel at the location i,j. Then, for MF with a filter size of 3×3, the new gray value x˜i,j of the pixel at the location i,j can be calculated by the following equation:(1)x˜i,j=medianxi−1,j−1,xi−1,j,xi−1,j+1,xi,j−1,xi,j,xi,j+1,xi+1,j−1,xi+1,j,xi+1,j+1.Motived by this idea, many MF methods have been proposed. However, the large distance between the noisy pixel and its neighborhood may result in a large difference between the median value pixel and the original noiseless pixel, which would cause the denoised image to be degraded [14]. In [13], the median value of neighborhoods is equipped with different weights, which are calculated by the distance from the center noisy pixel. Besides, when noise density is too high, for some noisy pixels, there are no noiseless neighborhoods for restoring. In [14], to tackle the problem, the researchers provide a method by using neighborhoods in the restored image to calculate the median value.


### 2.2. Nonlocal Mean (NLM) Method

The NLM method combined the idea of the Yaroslavsky filter [20] and bilateral filter [21]. For the noisy pixel, NLM uses the intensity patch feature instead of single pixel feature to compute the denoised mean value [22].Given a pixelx∈I, let Nx denote an image block centered at x. Then, NLM calculates a weighted average of all the pixels in the searching window B by the following equations:(2)x˜=∑y∈By∗ux,y∑y∈Bux,y,(3)ux,y=e−dx,y/h2,(4)dx,y=Nx−Ny2,a2,where x˜ is the restored gray value of x, while Nx denotes a block centered at x, and h is the smoothing parameter. ux,y denotes the similarity between pixels x and y which depends on the weighted Euclidean distance between two image blocks Nx and Ny, ⋅2 is the Euclidean norm, and a is the standard deviation of the Gaussian kernel with zero mean and standard deviation, for details about these, please refer to [22].


## 2.1. MF Based Methods for SAP Noise Elimination

MF is the most commonly used algorithm to remove pulse interference and SAP noise. The main idea is that all pixels are replaced by neighborhood median pixels whether pixels are noiseless or not. Detailedly, letX be an original 8-bit gray-level image with a size of M×N, and xi,j be the gray value of the pixel at the location i,j. Then, for MF with a filter size of 3×3, the new gray value x˜i,j of the pixel at the location i,j can be calculated by the following equation:(1)x˜i,j=medianxi−1,j−1,xi−1,j,xi−1,j+1,xi,j−1,xi,j,xi,j+1,xi+1,j−1,xi+1,j,xi+1,j+1.Motived by this idea, many MF methods have been proposed. However, the large distance between the noisy pixel and its neighborhood may result in a large difference between the median value pixel and the original noiseless pixel, which would cause the denoised image to be degraded [14]. In [13], the median value of neighborhoods is equipped with different weights, which are calculated by the distance from the center noisy pixel. Besides, when noise density is too high, for some noisy pixels, there are no noiseless neighborhoods for restoring. In [14], to tackle the problem, the researchers provide a method by using neighborhoods in the restored image to calculate the median value.


## 2.2. Nonlocal Mean (NLM) Method

The NLM method combined the idea of the Yaroslavsky filter [20] and bilateral filter [21]. For the noisy pixel, NLM uses the intensity patch feature instead of single pixel feature to compute the denoised mean value [22].Given a pixelx∈I, let Nx denote an image block centered at x. Then, NLM calculates a weighted average of all the pixels in the searching window B by the following equations:(2)x˜=∑y∈By∗ux,y∑y∈Bux,y,(3)ux,y=e−dx,y/h2,(4)dx,y=Nx−Ny2,a2,where x˜ is the restored gray value of x, while Nx denotes a block centered at x, and h is the smoothing parameter. ux,y denotes the similarity between pixels x and y which depends on the weighted Euclidean distance between two image blocks Nx and Ny, ⋅2 is the Euclidean norm, and a is the standard deviation of the Gaussian kernel with zero mean and standard deviation, for details about these, please refer to [22].


## 3. The Proposed Nonlocal Adaptive Mean (NAMF) Filter

In this section, we will present the details of our proposed nonlocal adaptive mean (NAMF) filter. Same as the settings of many traditional SAP noise removal filters, NAMF can be divided into two stages: (1) SAP noise detection; (2) SAP noise elimination. Details are presented as follows.


### 3.1. SAP Noise Detection

In accordance with the mathematical notation as the previous section,X=xi,j∈ℝM×N and Y=xi,j∈ℝM×N represent the original 8-bit gray-level image and noisy image corrupted by SAP noise, respectively. xi,j and yi,j represent the gray value of the pixel at the location i,j of X and Y, respectively, where i,j∈Λ≡1,…,M×1,…,N.In a corrupted image, the value of a “salt” pixel equals the maximum gray value 255, and the value of a “pepper” pixel equals the minimum value 0. Thus,yi,j is defined by the following equation:(5)yi,j=255or0,with probabilityα,xi,j,with  probability1−α,where α is the density of SAP noise of Y, α is unknown in practical problems, here we just use α to represent the degree of noise. And in the process of image denoising, we use Si,jw to represent a 2w+1×2w+1 window centered at i,j with the radius w.Considering the characteristics of SAP noise, pixelyi,j corrupted by SAP noise is 0 or 255. That is to say, noisy pixel candidate yi,j only has 2 possible values: ymin=0 and ymax=255. Following is a prior decision condition in noise detection.(6)oi,j=1,yi,j=0or255,0,otherwise,where O=oi,j∈ℝM×N is an indicator matrix with binary value. oi,j=1 means that pixel yi,j is the noisy pixel candidate, while oi,j=0 that means pixel yi,j is noiseless. For a natural image, the pixels with high or low value are also possible to be the texture of the image. For example, as shown in Figure 1(a), E and F are both candidates with a gray value of 0 (black). However, F has a larger possibility to be a noisy pixel, and E has a larger possibility to be a texture pixel. Therefore, it is very necessary to further confirm these pixels with maximum or minimum value.In fact, in natural image, pixels are supposed to change smoothly, and adjacent pixels tend to have similar values. The value of pixels in an image with SAP noise may change suddenly. Motivated by these facts, we propose an adaptive SAP noise detection method by considering the neighborhood to identify noise candidates as noisy pixels or not. Its procedures can be divided into two steps:(i)
Step 1. For the pixel yi,j with oi,j=1, we calculate Si,jsumw within an adaptive searching window Si,jw. The radius w of window Si,jw is initially set to 1. If the condition is met (w==wmax or Si,jsumw>0), the w is just what we need. If not, then w+1 and continue to compute, where Si,jsumw is the number of pixels within Si,jw which are not equal to ymin and ymax, wmax is the maximum size of window. If Si,jsumw>0, we take the pixel yi,j as a noisy pixel, the detection for yi,j ends. Otherwise, we go to Step 2.(ii)
Step 2. If Si,jsumw=0, yi,j maybe considered as a texture pixel. Then, we calculate the proportion ρ=Si,jnumw/2w+1×2w+1, where Si,jnumw is the number of pixels owning the same value as the candidate pixel yi,j in the window Si,jw. After that, we set a threshold T to identify the noisy pixel. If ρ<T, the candidate pixel yi,j is regarded as a noisy pixel, else it is noiseless.For the pixelyi,j, if it is finally detected as noisy, we mark it with the discriminant matrix L=li,j∈ℝM×N, and li,j=1, else li,j=0 and will not be processed.


### 3.2. SAP Noise Elimination

We restore noisy pixels in two steps. LetZ represent the initially restored image, Z^ represent the final output image. Before processing, we initialize Z with Z=Y.Firstly, when a pixelyi,j is detected as a noisy pixel, we use Si,jmeanw to restore it. The calculation of Si,jmeanw is based on Si,jsumw as shown in equation (7). When Si,jsumw≠0, Si,jmeanw is the mean of the noiseless pixels in Si,jw, otherwise it is the mean of three processed neighboring pixels of yi,j in Z. Unlike the four neighbors adopted in [14], the utilization of four neighboring pixels will lead to residual noisy pixels on the boundary, as shown in Figure 1(b) (even if the image boundary is expanded during the process, some noisy pixels located in boundary can still not be restored). Hence, we select three processed neighboring pixels in Z, i.e., Si,jmeanw=zi−1,j−1+zi−1,j+zi,j−1/3, and its illustration is shown in Figure 2.(7)Si,jmeanw=∑e,f∈Si,jw1−le,f∗ye,f∑e,f∈Si,jw1−le,f,Si,jsumw≠0,zi−1,j−1+zi−1,j+zi,j−1/3,otherwise,Figure 2
Y is a noisy image, Z is an image being restored, the red pixel in Z is prepared to process, the yellow pixels are pixels that have been restored, other blue pixels in Z are pixels that remained to be restored. When processing noisy pixels yi,j, neighborhoods in the window are all noisy, i.e., Si,jsumw=0. So here we select three neighboring pixels zi−1,j−1,zi−1,j and zi,j−1 in the restored image Z for restoring pixel yi,j in the original image Y.
(a)(b)Secondly, considering the problem that restoration will be hard under high noise intensity, we introduce the nonlocal mean (NLM) method [22, 23] to further restore noisy pixels. NLM can restore noisy pixels by using all neighbors instead of part of neighbors, which are detected as noiseless. In this way, we can use NLM to further enhance the restoration effect. Considering that the original NLM is designed for Gaussian noise or Speckle noise, which are quite different from SAP noise, we modify it based on the characteristics of SAP noise and make it applicable for our method.In the noise detection stage, a pixel is identified and marked by the discriminant matrixL. li,j=1 means the pixel yi,j is identified as a noisy pixel, otherwise noiseless. After the restoration of the noisy pixel yi,j, we get zi,j. The modified NLM method is used to further restore zi,j. We use z^i,j to represent the value of the further restored zi,j, and it can be calculated as follows:(8)z^i,j=1C∑ze,f∈Bzi,j,rze,f∗uzi,j,ze,f,li,j=1,zi,j,li,j=0,(9)C=∑ze,f∈Bzi,j,ruzi,j,ze,f,…,uzi,j,ze,f=e−dzi,j,ze,f/h2,e≠iorf≠j,0,e=iandf=j,(10)dzi,j,ze,f=Nzi,j−Nze,f2,a2,…,h=L¯M×N2∗β2+L¯M×N∗β1+β0,where Bzi,j,r represents a searching window of size 2r+1×2r+1 centered at zi,j, and uzi,j,ze,f represents the weight of pixel ze,f in Bzi,j,r. Nzi,j, also called similarity window, is a square block centered at zi,j, so is Nze,f. As shown in equation (10), the similarity between zi,j and ze,f is measured by the Gaussian weighted Euclidean distance dzi,j,ze,f between Nzi,j and Nze,f, where a is the standard deviation of the Gaussian kernel, and h is the smoothing parameter for NLM.When processing the noisy pixelzi,j, the original NLM assigns the weight based on the similarity. That is to say, the weight of pixel zi,j itself is the largest. Different from the original NLM, in our method, the noisy pixel to be processed will not participate in the process of NLM. Thus the weight of the pixel zi,j should be set as 0, as shown in equation (9).In the NLM algorithm, the higher the noise intensity is, the larger the smoothing parameterhshould be. But the intensity of noise is not easy to be confirmed. Considering that SAP noise can be significantly detected, we use the intensity of SAP noise to confirm h. As shown in equation (10), L¯ represents the total number of non-zero elements in the discriminant matrix L, that is, the more noisy pixels are detected, the larger h should be; β0, β1 and β2 are the parameters used to fit h.Due to the high computational cost of the NLM algorithm, here we introduce a kind of fast implementation of the NLM algorithm [24] based on the computation of patch distances using sums of lines to accelerate our NAMF algorithm. The details of the proposed NAMF are shown in Algorithm 1.Algorithm 1: NAMF.
/∗STAGE 1 ∗/Compute the indicator matrixOFor each pixeli,j∈∧ in the noisy image Y and the initially restored image Z, do(1)
Ifoi,j==0,li,j=0,zi,j=yi,j, break;Otherwise, go to step 2).(2)
Initializew=1,h=1,wmax=7.(3)
ComputeSi,jsumwuntilw==wmaxorSi,jsumw>0;Otherwise,w=w+h and repea step 3).(4)
IfSi,jsumw>0,li,j=1,zi,j=Si,jmeanw,break;Otherwise, go to step 5).(5)
Computeρ If ρ<T,li,j=1,zi,j=Si,jmeanw;Otherwiseli,j=0,zi,j=yi,j./∗STAGE 2∗/Computeh.For each pixelzi,j in image Z and z˜i,j in output image Z˜, do(6)
Ifli,j==1,z^i,j=1/c∑ze,f∈Bzi,j,rze,f∗uzi,j,ze,f;Otherwisez˜i,j=zi,j.


## 3.1. SAP Noise Detection

In accordance with the mathematical notation as the previous section,X=xi,j∈ℝM×N and Y=xi,j∈ℝM×N represent the original 8-bit gray-level image and noisy image corrupted by SAP noise, respectively. xi,j and yi,j represent the gray value of the pixel at the location i,j of X and Y, respectively, where i,j∈Λ≡1,…,M×1,…,N.In a corrupted image, the value of a “salt” pixel equals the maximum gray value 255, and the value of a “pepper” pixel equals the minimum value 0. Thus,yi,j is defined by the following equation:(5)yi,j=255or0,with probabilityα,xi,j,with  probability1−α,where α is the density of SAP noise of Y, α is unknown in practical problems, here we just use α to represent the degree of noise. And in the process of image denoising, we use Si,jw to represent a 2w+1×2w+1 window centered at i,j with the radius w.Considering the characteristics of SAP noise, pixelyi,j corrupted by SAP noise is 0 or 255. That is to say, noisy pixel candidate yi,j only has 2 possible values: ymin=0 and ymax=255. Following is a prior decision condition in noise detection.(6)oi,j=1,yi,j=0or255,0,otherwise,where O=oi,j∈ℝM×N is an indicator matrix with binary value. oi,j=1 means that pixel yi,j is the noisy pixel candidate, while oi,j=0 that means pixel yi,j is noiseless. For a natural image, the pixels with high or low value are also possible to be the texture of the image. For example, as shown in Figure 1(a), E and F are both candidates with a gray value of 0 (black). However, F has a larger possibility to be a noisy pixel, and E has a larger possibility to be a texture pixel. Therefore, it is very necessary to further confirm these pixels with maximum or minimum value.In fact, in natural image, pixels are supposed to change smoothly, and adjacent pixels tend to have similar values. The value of pixels in an image with SAP noise may change suddenly. Motivated by these facts, we propose an adaptive SAP noise detection method by considering the neighborhood to identify noise candidates as noisy pixels or not. Its procedures can be divided into two steps:(i)
Step 1. For the pixel yi,j with oi,j=1, we calculate Si,jsumw within an adaptive searching window Si,jw. The radius w of window Si,jw is initially set to 1. If the condition is met (w==wmax or Si,jsumw>0), the w is just what we need. If not, then w+1 and continue to compute, where Si,jsumw is the number of pixels within Si,jw which are not equal to ymin and ymax, wmax is the maximum size of window. If Si,jsumw>0, we take the pixel yi,j as a noisy pixel, the detection for yi,j ends. Otherwise, we go to Step 2.(ii)
Step 2. If Si,jsumw=0, yi,j maybe considered as a texture pixel. Then, we calculate the proportion ρ=Si,jnumw/2w+1×2w+1, where Si,jnumw is the number of pixels owning the same value as the candidate pixel yi,j in the window Si,jw. After that, we set a threshold T to identify the noisy pixel. If ρ<T, the candidate pixel yi,j is regarded as a noisy pixel, else it is noiseless.For the pixelyi,j, if it is finally detected as noisy, we mark it with the discriminant matrix L=li,j∈ℝM×N, and li,j=1, else li,j=0 and will not be processed.


## 3.2. SAP Noise Elimination

We restore noisy pixels in two steps. LetZ represent the initially restored image, Z^ represent the final output image. Before processing, we initialize Z with Z=Y.Firstly, when a pixelyi,j is detected as a noisy pixel, we use Si,jmeanw to restore it. The calculation of Si,jmeanw is based on Si,jsumw as shown in equation (7). When Si,jsumw≠0, Si,jmeanw is the mean of the noiseless pixels in Si,jw, otherwise it is the mean of three processed neighboring pixels of yi,j in Z. Unlike the four neighbors adopted in [14], the utilization of four neighboring pixels will lead to residual noisy pixels on the boundary, as shown in Figure 1(b) (even if the image boundary is expanded during the process, some noisy pixels located in boundary can still not be restored). Hence, we select three processed neighboring pixels in Z, i.e., Si,jmeanw=zi−1,j−1+zi−1,j+zi,j−1/3, and its illustration is shown in Figure 2.(7)Si,jmeanw=∑e,f∈Si,jw1−le,f∗ye,f∑e,f∈Si,jw1−le,f,Si,jsumw≠0,zi−1,j−1+zi−1,j+zi,j−1/3,otherwise,Figure 2
Y is a noisy image, Z is an image being restored, the red pixel in Z is prepared to process, the yellow pixels are pixels that have been restored, other blue pixels in Z are pixels that remained to be restored. When processing noisy pixels yi,j, neighborhoods in the window are all noisy, i.e., Si,jsumw=0. So here we select three neighboring pixels zi−1,j−1,zi−1,j and zi,j−1 in the restored image Z for restoring pixel yi,j in the original image Y.
(a)(b)Secondly, considering the problem that restoration will be hard under high noise intensity, we introduce the nonlocal mean (NLM) method [22, 23] to further restore noisy pixels. NLM can restore noisy pixels by using all neighbors instead of part of neighbors, which are detected as noiseless. In this way, we can use NLM to further enhance the restoration effect. Considering that the original NLM is designed for Gaussian noise or Speckle noise, which are quite different from SAP noise, we modify it based on the characteristics of SAP noise and make it applicable for our method.In the noise detection stage, a pixel is identified and marked by the discriminant matrixL. li,j=1 means the pixel yi,j is identified as a noisy pixel, otherwise noiseless. After the restoration of the noisy pixel yi,j, we get zi,j. The modified NLM method is used to further restore zi,j. We use z^i,j to represent the value of the further restored zi,j, and it can be calculated as follows:(8)z^i,j=1C∑ze,f∈Bzi,j,rze,f∗uzi,j,ze,f,li,j=1,zi,j,li,j=0,(9)C=∑ze,f∈Bzi,j,ruzi,j,ze,f,…,uzi,j,ze,f=e−dzi,j,ze,f/h2,e≠iorf≠j,0,e=iandf=j,(10)dzi,j,ze,f=Nzi,j−Nze,f2,a2,…,h=L¯M×N2∗β2+L¯M×N∗β1+β0,where Bzi,j,r represents a searching window of size 2r+1×2r+1 centered at zi,j, and uzi,j,ze,f represents the weight of pixel ze,f in Bzi,j,r. Nzi,j, also called similarity window, is a square block centered at zi,j, so is Nze,f. As shown in equation (10), the similarity between zi,j and ze,f is measured by the Gaussian weighted Euclidean distance dzi,j,ze,f between Nzi,j and Nze,f, where a is the standard deviation of the Gaussian kernel, and h is the smoothing parameter for NLM.When processing the noisy pixelzi,j, the original NLM assigns the weight based on the similarity. That is to say, the weight of pixel zi,j itself is the largest. Different from the original NLM, in our method, the noisy pixel to be processed will not participate in the process of NLM. Thus the weight of the pixel zi,j should be set as 0, as shown in equation (9).In the NLM algorithm, the higher the noise intensity is, the larger the smoothing parameterhshould be. But the intensity of noise is not easy to be confirmed. Considering that SAP noise can be significantly detected, we use the intensity of SAP noise to confirm h. As shown in equation (10), L¯ represents the total number of non-zero elements in the discriminant matrix L, that is, the more noisy pixels are detected, the larger h should be; β0, β1 and β2 are the parameters used to fit h.Due to the high computational cost of the NLM algorithm, here we introduce a kind of fast implementation of the NLM algorithm [24] based on the computation of patch distances using sums of lines to accelerate our NAMF algorithm. The details of the proposed NAMF are shown in Algorithm 1.Algorithm 1: NAMF.
/∗STAGE 1 ∗/Compute the indicator matrixOFor each pixeli,j∈∧ in the noisy image Y and the initially restored image Z, do(1)
Ifoi,j==0,li,j=0,zi,j=yi,j, break;Otherwise, go to step 2).(2)
Initializew=1,h=1,wmax=7.(3)
ComputeSi,jsumwuntilw==wmaxorSi,jsumw>0;Otherwise,w=w+h and repea step 3).(4)
IfSi,jsumw>0,li,j=1,zi,j=Si,jmeanw,break;Otherwise, go to step 5).(5)
Computeρ If ρ<T,li,j=1,zi,j=Si,jmeanw;Otherwiseli,j=0,zi,j=yi,j./∗STAGE 2∗/Computeh.For each pixelzi,j in image Z and z˜i,j in output image Z˜, do(6)
Ifli,j==1,z^i,j=1/c∑ze,f∈Bzi,j,rze,f∗uzi,j,ze,f;Otherwisez˜i,j=zi,j.


## 4. Experimental Results

In the experiments, NAMF are compared with 8 state-of-the-art methods: AMF [9], NAFSMF [12], AWMF [13], the method proposed in [14], BPDF [15], DAMF [16], AFWM [17], and ASWMF [18]. Sixteen typical images (Barbara, Elaine, Goldhill, Lena, Man, Peppers, Yacht, and Zelda, Baboon, House, Boat, Cameraman, Einstein, Face, Straw, and Couple) and 40 test images in the TEST IMAGES Database [19] are chosen for the experiments.We use 4 typical image quality metrics, peak signal-to-noise ratio (PSNR) [25], structural similarity (SSIM) [26], mean absolute error (MAE) [27], and image enhancement factor (IEF) [28] to evaluate the experimental results. For an image U and an image V with the same size of M×N , PSNR can be calculated by equation (11).(11)PSNRU,V=10∗log102552/MSE,MSE=1M×N∑iM∑jNui,j−vi,j2,where MSE is the mean square error of two images, andui,j, vi,j are pixels of image U and V, respectively. SSIM for image U and V can be defined as follows:(12)SSIMU,V=2μuμv+c1∗2σu,v+c2μu2+μv2+c1∗σu2+σv2+c2,where μu and μv are the average intensities of image U and V, respectively. σu and σv are standard deviations, σu,v is the covariance, c1 and c2 are some constants. Here c1 and c2 are set to be 0.01∗2552 and 0.03∗2552 as in [26], respectively. MAE can be computed as shown in the following equation:(13)MAEU,V=1M×N∑iM∑jNui,j−vi,j,Different from traditional image quality metrics, IEF is designed for evaluating the respiring effects of the denoising method. IEF can be obtained by the following equation:(14)IEF=∑iM∑jNI¨i,j−I˙i,j2∑iM∑jNI˙i,j−Ii,j∘,where I˙, I¨, I∘ denote the original image, the noisy image, and the repaired image, respectively. In this paper, we set β2=2.2186, β2=2.2186, and β0=4.5595 to fit h. The size of the searching window is 5×5. And the size of the similarity window is 41×41. Through the test, we finally take the threshold T=0.8for our method. Other methods keep the default parameters. The experiments are performed on a personal computer with Intel Core i7 2.2 GHz processor and 16 GB RAM.Figure3 shows the restored results of all methods for “Barbara” with SAP noise ratio of 10%. By observing the enlarged area in Figure 3, we can see that AMF, AWMF, BPDF, and ASWMF cannot restore the details of the original image very well. And the result of our proposed NAMF is nearly the same as the original image.Figure 3
Experimental results of different methods for “Barbara” with SAP noise ratio of 10%. (a) Noisy image. (b) AMF. (c) NAFSMF. (d) AWMF. (e) [14]. (f) BPDF. (g) DAMF. (h) AFMF. (i) ASWMF. (j) NAMF. (k) Original image.
(a)(b)(c)(d)(e)(f)(g)(h)(i)(j)(k)Figure4 shows the restored results for “Lena” with SAP noise ratio of 90%. In the enlarged area in Figure 4, it can be found that the performances of AMF, NAFSMF, BPDF, AFMF, and ASWMF are very poor. And the restored images by other methods are also very blurred, while the result of our method looks more natural and smooth.Figure 4
Experimental results of different methods for “Lena” with SAP noise ratio of 90%. (a) Noisy image. (b) AMF. (c) NAFSMF. (d) AWMF. (e) [14]. (f) BPDF. (g) DAMF. (h) AFMF. (i) ASWMF. (j) NAMF. (k) Original image.
(a)(b)(c)(d)(e)(f)(g)(h)(i)(j)(k)The curves of average PSNR, SSIM, MAE, and IEF are shown in Figure5. Figure 5(a) illustrates that NAMF obtains the highest PSNR under both low and high noise intensity, and the PSNR of our method is much higher than the results of other methods. Figure 5(b) shows that the SSIM curves of most methods are basically the same under low SAP noise intensity. However, with the increase of noise level, the superiority of NAMF is getting more obvious. When the noise ratio exceeds 30%, the SSIM obtained by NAMF is significantly higher than other methods. Figure 5(c) gives the MAE curves of all methods. We find that NAMF owns the lowest MAE at all SAP noise levels. The IEF curves of all methods are given in Figure 5(d). It is clear that our method can obtain the highest IEF score at all SAP noise levels. Furthermore, our NAMF can significantly outperform other methods in the low SAP noise level.Figure 5
Comparison of all methods in terms of PSNR, SSIM, MAE, and IEF at all SAP noise. (a) Average PSNR of different methods at all SAP noise levels. (b) Average SSIM of different methods at all SAP noise levels. (c) Average MAE of different methods at all SAP noise levels. (d) Average IEF of different methods at all SAP noise levels.
(a)(b)(c)(d)Figure6 illustrates the average running time of different methods at all noise levels. With the increase of noise intensity, the running time of all methods except AWMF and AFMF increases. Clearly, the average running time of the method proposed in [14] is the shortest. As shown in the enlarged area of Figure 6, although the rank of NAMF is in the middle, its processing speed is superior to NAFSMF and similar to DAMF under high noise intensity.Figure 6
Average running time of different methods at all SAP noise levels.Besides, to further illustrate the function of our proposed SAP based NLM method, we make the ablation experiments to show its effectiveness. As shown in Figure7, we compare the NAMF without the SAP based NLM and the original NAMF with the SAP based NLM. The former one uses a traditional denoising mechanism, and the latter one adopts the SAP based NLM to further repair the noisy images. Figure 7 shows that the performances of the NAMF without the SAP based NLM are significantly weakened compared with the original NAMF.Figure 7
Experimental results of the original NAMF and the NAMF without the SAP based NLM. (a) PSNR of NAMF and NAMF without the SAP based NLM. (b) SSIM of NAMF and NAMF without the SAP based NLM. (c) MAE of NAMF and NAMF without the SAP based NLM. (d) IEF of NAMF and NAMF without the SAP based NLM.
(a)(b)(c)(d)


## 5. Conclusion

In this paper, a method called NAMF for SAP noise denoising is proposed, which adopts a SAP noise based nonlocal mean method. NAMF can get much higher restoring quality than state-of-the-art methods at all SAP noise levels. The processing time of NAMF is comparable to most state-of-the-art methods. The experimental results show that NAMF can get much better PSNR and SSIM at all SAP noise levels. Moreover, NAMF can preserve more details even at a noise level as high as 90%.



---

*Source: 4127679-2021-03-10.xml*