# A Modified Iterative Alternating Direction Minimization Algorithm for Impulse Noise Removal in Images

Authors: Di Guo; Xiaobo Qu; Meng Wu; Keshou Wu
Journal: Journal of Applied Mathematics
 (2014)

---

## Abstract

Images are often corrupted by impulse noise. In this paper, an alternating direction minimization with continuation algorithm is modified and iteratively used to remove the impulse noise in images by exploring its self-similarity. A patch-based nonlocal operator and sparse representation are married in thel1-l1 optimization model to be solved. Simulation results demonstrate that the proposed algorithm outperforms typical denoising methods in terms of preserving edges and textures for both salt-and-pepper noise and random-valued impulse noise. It can be also applied to suppress impulse noise-like artifacts in real mural images.


---

## Body

## 1. Introduction

Image denoising is a fundamental problem in image processing. Besides the commonly processed Gaussian noise, impulse noise (Figure1(a)) is another typical type of noise generated due to noisy sensors or channel transmission errors [1]. In mural images, the impulse noise-like artifacts are also observed because of the bacterial plaque (Figure 1(b)) or missing objects (Figure 1(c)). A classical impulse denoising method is median filtering, which replaces the central pixel with a median-value within a local window. Median filtering is with low computation but often loses image structures when noise is heavy. The denoising performance is expected to be greatly improved if the image structures are extracted and preserved in denoising.Images with impulse noise. (a) House image with 10% salt-and-pepper noise; (b) and (c) are two mural images.
(a)
(b)
(c)In recent years, sparse representation method has attracted great attention to provide effective representation of image structures and found promising applications in image denoising [2–5], image decomposition [6, 7], image fusion [8, 9], pattern recognition [10–13], and so forth. In impulse noise removal, sparse representation is remarkably effective at smoothing away noise [14–16], but it may lose image structures which are not sparsely represented by its predefined basis such as wavelet [3, 14]. Although the adaptive dictionary training [15, 16] improves the denoising performance, its computation cost of training dictionaries while removing noise is relatively high. Besides, the commonly used total variation [17–20] is highly related to enforcing the sparsity of finite difference of images. However, total variation assumes that images are piecewise smooth; thus, textures would be easily lost after denoising [15, 16, 21, 22]. Therefore, exploring the sparse representation for impulse noise removal is still challenging.Similarity is often rooted in repeated patterns of images [23–25] and greatly improves image quality in deblurring [26] and medical image reconstruction [27]. Similarity information was previously investigated in block matching 3D frames [26]. Patch-based nonlocal operator (PANO) [27] is a linear operator to model the sparse representation of similar patches, which can be viewed as an alternative form of the block matching 3D frames. Unlike the traditional sparsifying methods, for example, conventional wavelets or finite difference, PANO provides an adaptive sparse representation of image with a very fast training phase. Besides, the linearity of PANO allows setting up explicitly a sparsity-based denoising model.The contributions of this paper are summarized as follows.(1)
The impulse noise is removed by making use of similarity-motivated sparse representation of images. Mathematically, we model the denoising problem as anl1-l1 minimization problem with PANO which provides the sparse representation of similar image patches.(2)
To solve the optimization problem, a fast alternating direction minimization with continuation algorithm is developed by taking the patch-based representation property into account.(3)
Besides removing salt-and-pepper noise and random-value impulse noise in images, the proposed method is also applied in recovering mural images from corrupted data. Experimental results on simulated data and real mural images verify that the proposed method outperforms other methods in removing impulse noise or impulse noise-like artifacts.


## 2. Model

In this section, the impulse noise model and typical denoising methods are briefly summarized. Then, the proposed denoising model is presented.


### 2.1. Impulse Noise

Image contaminated with impulse noise is usually modeled as(1)f=η⊙u+(1-η)⊙ε,
where u is the noiseless image, f is the observation, ⊙ denotes an element-wise product, η∈RN is a sample drawn from an i.i.d. multivariate Bernoulli distribution with success probability 1-p, 1∈RN represents a vector with all elements being equal to one, ε∈RN is either salt-and-pepper noise or random-valued impulse noise.For the salt-and-pepper noise, the elementεn of ε satisfies
(2)εn={cmin⁡withprobabilityp1cmax⁡withprobabilityp2,
where p1+p2=p. For the random-valued impulse noise, εn is drawn from uniform distribution in [cmin⁡,cmax⁡]. In this paper, cmin⁡=0, cmax⁡=1, and p1=p2 are set.


### 2.2. Proposed Denoising Model

With a maximum a posteriori estimator on the impulse noise and prior knowledge of image, the noiseless imageu is estimated by minimizing
(3)min⁡uλ∥f-u∥1+φ(u),
where φ can be chosen as total variation assuming that the image is piecewise constant [19, 28] or l1 norm of wavelet coefficients assuming that the image is sparse in the wavelet domain [14]. However, total variation is observed to easily lose textures [16] and wavelet may lose image structures that are not sparsely represented by its predefined basis. Thus, how to preserve the edges and textures remains open for the impulse denoising problem.In this paper, image self-similarity is employed to provide an optimized sparse representation of images. Assuming that the similarity information of image is known with PANO, anl1-l1 regularization model is proposed to remove the impulse noise,
(4)min⁡u∑j=1J∥Aju∥1+λ∥f-u∥1,
where ∑j=1J∥Aju∥1 promotes the sparsity using PANO, ∥f-u∥1 removes the outliers in images, and λ balances between the sparsity and outliers removing.PANO is a linear operator to model the sparse representation of similar patches [27, 29]. For a given image u, PANO is defined as
(5)Aj=Ψ3DRvjPi,
where Pi denotes the patch extraction, Rvjmeans grouping some of the patches into a 3D cube, and Ψ3D is 3D Haar transform. Figure 2 illustrates the process of grouping patches into a cube. First, an image is extracted into patches. Then, some similar patches are grouped together and stacked into 3D cubes. Finally, a sparse representation will be achieved by performing a 3D Haar transform on each cube,
(6)αj=Aju,
where αj is the sparse vector for the jth cube [27, 29]. The adjoint operator of Aj is AjT=PiTRvjTΨ3DT, and ∑jJAjTAj=O, where O is a diagonal matrix whose nth diagonal entry is a counter indicating the repetitions that the nth pixel is grouped into the 3D cubes. Therefore, by taking this patch-based representation property into account, the traditional alternating direction algorithm can be derived to achieve fast computation.Group image patches. (a) An image; (b) four groups of similar patches; (c) the patch and group dimension.
(a)
(b)
(c)


## 2.1. Impulse Noise

Image contaminated with impulse noise is usually modeled as(1)f=η⊙u+(1-η)⊙ε,
where u is the noiseless image, f is the observation, ⊙ denotes an element-wise product, η∈RN is a sample drawn from an i.i.d. multivariate Bernoulli distribution with success probability 1-p, 1∈RN represents a vector with all elements being equal to one, ε∈RN is either salt-and-pepper noise or random-valued impulse noise.For the salt-and-pepper noise, the elementεn of ε satisfies
(2)εn={cmin⁡withprobabilityp1cmax⁡withprobabilityp2,
where p1+p2=p. For the random-valued impulse noise, εn is drawn from uniform distribution in [cmin⁡,cmax⁡]. In this paper, cmin⁡=0, cmax⁡=1, and p1=p2 are set.


## 2.2. Proposed Denoising Model

With a maximum a posteriori estimator on the impulse noise and prior knowledge of image, the noiseless imageu is estimated by minimizing
(3)min⁡uλ∥f-u∥1+φ(u),
where φ can be chosen as total variation assuming that the image is piecewise constant [19, 28] or l1 norm of wavelet coefficients assuming that the image is sparse in the wavelet domain [14]. However, total variation is observed to easily lose textures [16] and wavelet may lose image structures that are not sparsely represented by its predefined basis. Thus, how to preserve the edges and textures remains open for the impulse denoising problem.In this paper, image self-similarity is employed to provide an optimized sparse representation of images. Assuming that the similarity information of image is known with PANO, anl1-l1 regularization model is proposed to remove the impulse noise,
(4)min⁡u∑j=1J∥Aju∥1+λ∥f-u∥1,
where ∑j=1J∥Aju∥1 promotes the sparsity using PANO, ∥f-u∥1 removes the outliers in images, and λ balances between the sparsity and outliers removing.PANO is a linear operator to model the sparse representation of similar patches [27, 29]. For a given image u, PANO is defined as
(5)Aj=Ψ3DRvjPi,
where Pi denotes the patch extraction, Rvjmeans grouping some of the patches into a 3D cube, and Ψ3D is 3D Haar transform. Figure 2 illustrates the process of grouping patches into a cube. First, an image is extracted into patches. Then, some similar patches are grouped together and stacked into 3D cubes. Finally, a sparse representation will be achieved by performing a 3D Haar transform on each cube,
(6)αj=Aju,
where αj is the sparse vector for the jth cube [27, 29]. The adjoint operator of Aj is AjT=PiTRvjTΨ3DT, and ∑jJAjTAj=O, where O is a diagonal matrix whose nth diagonal entry is a counter indicating the repetitions that the nth pixel is grouped into the 3D cubes. Therefore, by taking this patch-based representation property into account, the traditional alternating direction algorithm can be derived to achieve fast computation.Group image patches. (a) An image; (b) four groups of similar patches; (c) the patch and group dimension.
(a)
(b)
(c)


## 3. Algorithm

To numerically solve the optimization problem in (4), an alternating direction minimization with continuation (ADMC) algorithm [30, 31] is modified to remove the impulse noise. ADMC algorithm is also used for image deblurring [32, 33]. Here, by introducing auxiliary variables αj=Aju and z=f-u to (6), one has
(7)min⁡u,α,z∑j=1J(∥αj∥1+β2∥αj-Aju∥22)+λ(∥z∥1+β2∥z-(f-u)∥22).Asβ→∞, the solution of (7) approaches that of (6). In practice, we use the previous solution as a “warm start” for the next alternating optimization as β gradually increased. β does not need to be very large for image denoising. Simulation results show that increasing it will not improve the quality of denoised image but need more computation time. Thus β≤214 is suggested. When β is fixed, (7) can be solved in an alternating fashion between αj(j=1,…,J), z, and u. Consider the following.(1)
Fixu and z; solve min⁡αj∥αj∥1+(β/2)∥αj-Aju∥22(j=1,2,…,J), whose solution is
(8)α^j=S(Aju,1β),(j=1,2,…,J),
where S(v,t) denotes the soft thresholding operation on v with threshold t [30, 31].(2)
Fixu and α; solve min⁡z∥z∥1+(β/2)∥z-(f-u)∥22, whose solution is
(9)z^=S(f-u,1β).(3)
Fixz and α; solve min⁡u∑j=1J∥αj-Aju∥22+λ∥z-(f-u)∥22, whose solution is
(10)u^=(λI+O)-1[∑j=1JAjTαj+λ(f-z)].To construct PANO, similarity needs to be learnt so that some similar patches are grouped together and stacked into 3D cubes. However, the similarity is corrupted for a noisy image. To obtain a reasonable similarity, we need to have an initial denoised image as a guide image. The similarity is first learnt from a denoised image using conventional methods such as median filtering and then learnt twice from the denoised image using PANO. The process of the modified algorithm is summarized as Algorithm1.Algorithm 1:Remove impulse noise using PANO.
Initialization:
Input the noise contaminated imagef, the PANO Aj(j=1,…,J),
regularization parameterλ, tolerance of inner loop μ=10-3, and times of
learning similarity is 3. Initializeu=f, z=0, αj=0 for j=1,2,…,J,β=24 and ulast=0.
Main:
Step 1. Obtain the initial guide image r from median filtering [1];
While  t≤3,
Step 2.  Learn similarity Aj(j=1,…,J)  in PANO from the guide image r [27];
Whileβ≤214,
Step 3.  Solve (8);
Step 4.  Solve (9);
Step 5.  Solve (10);
Step 6.  If ∥ulast-u∥>μ∥ulast∥,  ulast←u, go to Step 3;
otherwise, go to Step 7;
Step 7.  β←2β, u^←u, go to Step 3.
End While
Step 8.  r←u^, t←t+1, β=24, Go to Step 2;
End While
Output:  u^


## 4. Results

In simulations, the proposed PANO-based denoising method is compared with median filtering [1], wavelet-based sparse representation denoising, and total variation denoising. The dual-tree complex wavelet, which was used in impulse denoising [14], is adopted as the wavelet transform. PANO is applied with the typical parameter settings: the number of similar patches is 16, the patch size is 8 × 8, and the search region is 39 × 39.Objective image quality assessments, peak signal-to-noise ratio (PSNR), and mean measure of structural similarity (MSSIM) [34] are evaluated. The PSNR is defined as
(11)PSNR(dB)=20log10N∥u-u^∥,
where N is the number of pixels in an image and u and u^ are the original and denoised images, respectively. MSSIM is defined as
(12)MSSIM=1N∑n=1NSSIM(an,bn),
where an and bn are the image content at the nth local window and SSIM evaluates local reconstruction errors by measuring the similarity between two images in a local window [34]. A higher MSSIM value means that structural information is better preserved and more suitable for human visual system.The experiments run on 4 Cores 2.6 GHz CPU laptop computer with 12 GB RAM. The computational time of the proposed method is approximately 25 (or 70) seconds for image with size 256 × 256 (or 512 × 512). For the simulated noisy images, the window size of the median filtering and the regularization parameters of the rest methods are optimized to maximize MSSIMs while removing most of the impulse noise.


### 4.1. Choose Guide Images

The similarity is assumed to be known in the proposed denoising model. However, the similarity is also corrupted for a noisy image. To obtain a reasonable similarity, we need to have an initial denoised image as a guide image. In this section, we will discuss how to choose a guide image.Since conventional denoising methods are available, one may consider taking the denoising result by median filtering, total variation, or wavelet denoising as the guide image to learn the similarity. When a better denoised image is obtained using the proposed approach, one may further utilize the result as an updated guide image.As shown in Figure3, different initial guide images lead to comparable denoising results with 30% noise. So median filtering is chosen to produce the initial guide image since its computation is the fastest. After the similarity is learnt for 3 times, both the PSNR and MSSIM of the proposed method become stable, and more similarity learning will cost more computation time. Hence, learning the similarity 3 times is sufficient.Effect of initial guide images on the denoising performance of the proposed method. (a) Original House image; (b) noisy image with 30% salt-and-pepper noise; (c), (d), and (e) are denoised images using median filtering, total variation, and wavelet, respectively; (f), (g), and (h) are denoised images using the proposed method with (c), (d), and (e) as the initial guide image, respectively; (i) and (j) are the PSNR and MSSIM performance curves.
(a)
(b)
(c)
(d)
(e)
(f)
(g)
(h)
(i)
(j)


### 4.2. Remove Simulated Impulse Noise

Images (Figures3(a), 4(a), and 4(d)), which are usually used in image processing, are treated as ground truth images; salt-and-pepper noise or random-valued impulse noise is manually added in simulations.Images used in simulation. (a) and (d) are original images,Barbara (size 512 × 512) andBoat (size 512 × 512), respectively; (b) and (e) are the images corrupted with 30% salt-and-pepper noise; (c) and (f) are the images corrupted with 30% random-valued impulse noise.
(a)
(b)
(c)
(d)
(e)
(f)


#### 4.2.1. Salt-and-Pepper Noise Removal

The performances of the four denoising methods for salt-and-pepper noise are compared. As shown in Figure5, repeated image patterns, for example, textures inBarbara, are obviously lost using median filtering and total variation. Although wavelet recovers the textures much better than the above two methods, it suffers from losing edges. The proposed method preserves these features best, with the clearest textures inBarbara and edges in bothHouse andBoat.Figure 5
Denoised images with 30% salt-and-pepper noise. (a1), (a2), and (a3) are original images; (b1), (b2), and (b3) are results of median filtering; (c1), (c2), and (c3) are results of total variation denoising; (d1), (d2), and (d3) are results of the wavelet-based sparse representation denoising; (e1), (e2), and (e3) are results of the proposed method.As shown in Figure6, the proposed method achieves the highest PSNRs when the noise level is below 40% and the highest SSIMs for all noise levels. When the noise becomes heavier (e.g., 50%), the performance of the proposed method gradually dropped. As Figure 7 shows, the qualities of the initial guide images using median filtering, total variation, and wavelet denoising methods are all unsatisfactory; thus, similarity cannot be learnt well so that the denoising results of the proposed method are also not satisfactory. Table 1 shows the performance of the proposed method using different initial guide images under heavy noise. Suppose we have the original image as the initial guide image so that similarity information can be exactly leant; the proposed method can remove the noise very well and preserve the image structures. In summary, the proposed method leads to comparable denoising results when ground truth image is not available.Table 1
Effect of initial guide image with 50% salt-and-pepper noise inBarbara.
Initial guide image
Median filter
Total variation
Wavelet
Original
PSNR
32.42
32.40
32.45
33.09
MSSIM
0.7288
0.7280
0.7376
0.8298Denoising performance for salt-and-pepper noise. (a), (b), and (c) are the PSNRs of imageBarbara,House, andBoat; (d), (e), and (f) are the MSSIMs of these images.
(a)
(b)
(c)
(d)
(e)
(f)Figure 7
DenoisedBarbara images with 50% salt-and-pepper noise. (a1), (b1), and (c1) are initial guide image using median filtering, total variation, and wavelet, respectively; (d1) original image; (a2), (b2), (c2), and (d2) are denoising results of the proposed method using (a1), (b1), (c1), and (d1) as the initial guide image, respectively.


#### 4.2.2. Random-Valued Impulse Noise Removal

The performances of the four denoising methods for random-valued impulse noise are also compared. As shown in Figure8, edges and textures are best preserved by the proposed method. Under different noise levels, proposed method achieves the highest MSSIMs (Figure 9). These observations imply that the proposed method preserves the best image structures during removing the random-valued impulse noise.Figure 8
Denoised images with 30% random-valued impulse noise. (a1), (a2), and (a3) are original images; (b1), (b2), and (b3) are results of median filtering; (c1), (c2), and (c3) are results of total variation denoising; (d1), (d2), and (d3) results of the wavelet-based sparse representation denoising; (e1), (e2), and (e3) are results of the proposed method.Denoising performance for random-valued impulse noise. (a), (b), and (c) are the PSNRs of imageBarbara,House, andBoat; (d), (e), and (f) are the MSSIMs of these images.
(a)
(b)
(c)
(d)
(e)
(f)


### 4.3. Remove Impulse Noise-Like Artifacts in Mural Images

In the section, the impulse noise-like artifacts in realistic mural images are suppressed using these denoising methods. As shown in Figure10, both median filtering and total variation will easily blur the images. On the contrary, the proposed method preserves the image details much better and suppresses most the impulse noise-like artifacts as marked. We can observe that the lost image structures are minimal among these methods (Figure 11). Therefore, the proposed method can also be used to remove these impulse noise-like artifacts of real dataset.Figure 10
Mural images after removing impulse noise-like artifacts. (a1), (a2) are original images; (b1), (b2) are results of median filtering; (c1), (c2) are results of total variation denoising; (d1), (d2) are results of the wavelet-based sparse representation denoising; (e1), (e2) are results of the proposed method.Figure 11
Difference between original and denoised images. (a1), (b1), (c1), and (d1) are lost structures in denoising Figure1(b) using median filtering, total variation, wavelet, and the proposed method, respectively; (a2), (b2), (c2), and (d2) are lost structures in denoising Figure 1(c) using median filtering, total variation, wavelet, and the proposed method, respectively.


### 4.4. The Effect of Noise Detectors

Noise detectors are very useful to improve the denoising performance of salt-and-pepper noise. Chan et al. [18] proposed to use a noise detector first and apply a weighted regularization model with less penalization on the noise-free pixels. Following this idea, we use a noise detector to distinguish the noisy pixels from the rest of the pixels in the preliminary results. The locations of detected noise candidates are stored in a set Ω, whose complementary set Ω- stores locations of the rest of the pixels. We tried a weighted l1-l1 regularization model as
(13)min⁡u{∥Ψu∥1+λ∥W(f-u)∥1},
where W is a diagonal matrix, whose mth diagonal entry is wm. A small weight wi(i∈Ω) is assigned for noise candidates and a large weight wk(k∈Ω¯) is set for the rest of pixels. Ψ is chosen to be total variation or PANO.Simulation results listed in Table2 show that noise detectors can significantly improve the denoising performance. When both methods are equipped with noise detectors, the proposed method still outperforms total variation with higher MSSIMs and comparable PSNRs. This implies that the proposed method can preserve image structures better, as shown in Figure 12.Table 2
Denoising performance ofBarbara at high salt-and-pepper noise levels.
Noise levels
Proposed
Total variation with detectors
Proposed with detectors
PSNR
MSSIM
PSNR
MSSIM
PSNR
MSSIM
50%
32.47
0.7486
35.35
0.8647
36.18
0.9262
70%
30.36
0.6205
33.61
0.7790
33.49
0.8214
90%
26.09
0.4619
31.65
0.6310
30.54
0.6322Denoising results ofBarbara image with 70% salt-and-pepper noise. (a) Ground truth image, (b) noisy image, and (c), (d), and (e) are denoised images using the proposed method, total variation with noise detectors, and the proposed method with noise detectors, respectively.
(a)
(b)
(c)
(d)
(e)An efficient method to suppress the noise via combining PANO with noise detectors can be found in [35]. However, the noise detectors proposed in [18] only work for salt-and-pepper noise. How to detect the noise candidates in random-valued noisy image is still unknown. Besides, how to solve the proposed weighted regularization model efficiently and setting the weight optimally need comprehensive investigation. We leave these as the future work.


## 4.1. Choose Guide Images

The similarity is assumed to be known in the proposed denoising model. However, the similarity is also corrupted for a noisy image. To obtain a reasonable similarity, we need to have an initial denoised image as a guide image. In this section, we will discuss how to choose a guide image.Since conventional denoising methods are available, one may consider taking the denoising result by median filtering, total variation, or wavelet denoising as the guide image to learn the similarity. When a better denoised image is obtained using the proposed approach, one may further utilize the result as an updated guide image.As shown in Figure3, different initial guide images lead to comparable denoising results with 30% noise. So median filtering is chosen to produce the initial guide image since its computation is the fastest. After the similarity is learnt for 3 times, both the PSNR and MSSIM of the proposed method become stable, and more similarity learning will cost more computation time. Hence, learning the similarity 3 times is sufficient.Effect of initial guide images on the denoising performance of the proposed method. (a) Original House image; (b) noisy image with 30% salt-and-pepper noise; (c), (d), and (e) are denoised images using median filtering, total variation, and wavelet, respectively; (f), (g), and (h) are denoised images using the proposed method with (c), (d), and (e) as the initial guide image, respectively; (i) and (j) are the PSNR and MSSIM performance curves.
(a)
(b)
(c)
(d)
(e)
(f)
(g)
(h)
(i)
(j)


## 4.2. Remove Simulated Impulse Noise

Images (Figures3(a), 4(a), and 4(d)), which are usually used in image processing, are treated as ground truth images; salt-and-pepper noise or random-valued impulse noise is manually added in simulations.Images used in simulation. (a) and (d) are original images,Barbara (size 512 × 512) andBoat (size 512 × 512), respectively; (b) and (e) are the images corrupted with 30% salt-and-pepper noise; (c) and (f) are the images corrupted with 30% random-valued impulse noise.
(a)
(b)
(c)
(d)
(e)
(f)


### 4.2.1. Salt-and-Pepper Noise Removal

The performances of the four denoising methods for salt-and-pepper noise are compared. As shown in Figure5, repeated image patterns, for example, textures inBarbara, are obviously lost using median filtering and total variation. Although wavelet recovers the textures much better than the above two methods, it suffers from losing edges. The proposed method preserves these features best, with the clearest textures inBarbara and edges in bothHouse andBoat.Figure 5
Denoised images with 30% salt-and-pepper noise. (a1), (a2), and (a3) are original images; (b1), (b2), and (b3) are results of median filtering; (c1), (c2), and (c3) are results of total variation denoising; (d1), (d2), and (d3) are results of the wavelet-based sparse representation denoising; (e1), (e2), and (e3) are results of the proposed method.As shown in Figure6, the proposed method achieves the highest PSNRs when the noise level is below 40% and the highest SSIMs for all noise levels. When the noise becomes heavier (e.g., 50%), the performance of the proposed method gradually dropped. As Figure 7 shows, the qualities of the initial guide images using median filtering, total variation, and wavelet denoising methods are all unsatisfactory; thus, similarity cannot be learnt well so that the denoising results of the proposed method are also not satisfactory. Table 1 shows the performance of the proposed method using different initial guide images under heavy noise. Suppose we have the original image as the initial guide image so that similarity information can be exactly leant; the proposed method can remove the noise very well and preserve the image structures. In summary, the proposed method leads to comparable denoising results when ground truth image is not available.Table 1
Effect of initial guide image with 50% salt-and-pepper noise inBarbara.
Initial guide image
Median filter
Total variation
Wavelet
Original
PSNR
32.42
32.40
32.45
33.09
MSSIM
0.7288
0.7280
0.7376
0.8298Denoising performance for salt-and-pepper noise. (a), (b), and (c) are the PSNRs of imageBarbara,House, andBoat; (d), (e), and (f) are the MSSIMs of these images.
(a)
(b)
(c)
(d)
(e)
(f)Figure 7
DenoisedBarbara images with 50% salt-and-pepper noise. (a1), (b1), and (c1) are initial guide image using median filtering, total variation, and wavelet, respectively; (d1) original image; (a2), (b2), (c2), and (d2) are denoising results of the proposed method using (a1), (b1), (c1), and (d1) as the initial guide image, respectively.


### 4.2.2. Random-Valued Impulse Noise Removal

The performances of the four denoising methods for random-valued impulse noise are also compared. As shown in Figure8, edges and textures are best preserved by the proposed method. Under different noise levels, proposed method achieves the highest MSSIMs (Figure 9). These observations imply that the proposed method preserves the best image structures during removing the random-valued impulse noise.Figure 8
Denoised images with 30% random-valued impulse noise. (a1), (a2), and (a3) are original images; (b1), (b2), and (b3) are results of median filtering; (c1), (c2), and (c3) are results of total variation denoising; (d1), (d2), and (d3) results of the wavelet-based sparse representation denoising; (e1), (e2), and (e3) are results of the proposed method.Denoising performance for random-valued impulse noise. (a), (b), and (c) are the PSNRs of imageBarbara,House, andBoat; (d), (e), and (f) are the MSSIMs of these images.
(a)
(b)
(c)
(d)
(e)
(f)


## 4.2.1. Salt-and-Pepper Noise Removal

The performances of the four denoising methods for salt-and-pepper noise are compared. As shown in Figure5, repeated image patterns, for example, textures inBarbara, are obviously lost using median filtering and total variation. Although wavelet recovers the textures much better than the above two methods, it suffers from losing edges. The proposed method preserves these features best, with the clearest textures inBarbara and edges in bothHouse andBoat.Figure 5
Denoised images with 30% salt-and-pepper noise. (a1), (a2), and (a3) are original images; (b1), (b2), and (b3) are results of median filtering; (c1), (c2), and (c3) are results of total variation denoising; (d1), (d2), and (d3) are results of the wavelet-based sparse representation denoising; (e1), (e2), and (e3) are results of the proposed method.As shown in Figure6, the proposed method achieves the highest PSNRs when the noise level is below 40% and the highest SSIMs for all noise levels. When the noise becomes heavier (e.g., 50%), the performance of the proposed method gradually dropped. As Figure 7 shows, the qualities of the initial guide images using median filtering, total variation, and wavelet denoising methods are all unsatisfactory; thus, similarity cannot be learnt well so that the denoising results of the proposed method are also not satisfactory. Table 1 shows the performance of the proposed method using different initial guide images under heavy noise. Suppose we have the original image as the initial guide image so that similarity information can be exactly leant; the proposed method can remove the noise very well and preserve the image structures. In summary, the proposed method leads to comparable denoising results when ground truth image is not available.Table 1
Effect of initial guide image with 50% salt-and-pepper noise inBarbara.
Initial guide image
Median filter
Total variation
Wavelet
Original
PSNR
32.42
32.40
32.45
33.09
MSSIM
0.7288
0.7280
0.7376
0.8298Denoising performance for salt-and-pepper noise. (a), (b), and (c) are the PSNRs of imageBarbara,House, andBoat; (d), (e), and (f) are the MSSIMs of these images.
(a)
(b)
(c)
(d)
(e)
(f)Figure 7
DenoisedBarbara images with 50% salt-and-pepper noise. (a1), (b1), and (c1) are initial guide image using median filtering, total variation, and wavelet, respectively; (d1) original image; (a2), (b2), (c2), and (d2) are denoising results of the proposed method using (a1), (b1), (c1), and (d1) as the initial guide image, respectively.


## 4.2.2. Random-Valued Impulse Noise Removal

The performances of the four denoising methods for random-valued impulse noise are also compared. As shown in Figure8, edges and textures are best preserved by the proposed method. Under different noise levels, proposed method achieves the highest MSSIMs (Figure 9). These observations imply that the proposed method preserves the best image structures during removing the random-valued impulse noise.Figure 8
Denoised images with 30% random-valued impulse noise. (a1), (a2), and (a3) are original images; (b1), (b2), and (b3) are results of median filtering; (c1), (c2), and (c3) are results of total variation denoising; (d1), (d2), and (d3) results of the wavelet-based sparse representation denoising; (e1), (e2), and (e3) are results of the proposed method.Denoising performance for random-valued impulse noise. (a), (b), and (c) are the PSNRs of imageBarbara,House, andBoat; (d), (e), and (f) are the MSSIMs of these images.
(a)
(b)
(c)
(d)
(e)
(f)


## 4.3. Remove Impulse Noise-Like Artifacts in Mural Images

In the section, the impulse noise-like artifacts in realistic mural images are suppressed using these denoising methods. As shown in Figure10, both median filtering and total variation will easily blur the images. On the contrary, the proposed method preserves the image details much better and suppresses most the impulse noise-like artifacts as marked. We can observe that the lost image structures are minimal among these methods (Figure 11). Therefore, the proposed method can also be used to remove these impulse noise-like artifacts of real dataset.Figure 10
Mural images after removing impulse noise-like artifacts. (a1), (a2) are original images; (b1), (b2) are results of median filtering; (c1), (c2) are results of total variation denoising; (d1), (d2) are results of the wavelet-based sparse representation denoising; (e1), (e2) are results of the proposed method.Figure 11
Difference between original and denoised images. (a1), (b1), (c1), and (d1) are lost structures in denoising Figure1(b) using median filtering, total variation, wavelet, and the proposed method, respectively; (a2), (b2), (c2), and (d2) are lost structures in denoising Figure 1(c) using median filtering, total variation, wavelet, and the proposed method, respectively.


## 4.4. The Effect of Noise Detectors

Noise detectors are very useful to improve the denoising performance of salt-and-pepper noise. Chan et al. [18] proposed to use a noise detector first and apply a weighted regularization model with less penalization on the noise-free pixels. Following this idea, we use a noise detector to distinguish the noisy pixels from the rest of the pixels in the preliminary results. The locations of detected noise candidates are stored in a set Ω, whose complementary set Ω- stores locations of the rest of the pixels. We tried a weighted l1-l1 regularization model as
(13)min⁡u{∥Ψu∥1+λ∥W(f-u)∥1},
where W is a diagonal matrix, whose mth diagonal entry is wm. A small weight wi(i∈Ω) is assigned for noise candidates and a large weight wk(k∈Ω¯) is set for the rest of pixels. Ψ is chosen to be total variation or PANO.Simulation results listed in Table2 show that noise detectors can significantly improve the denoising performance. When both methods are equipped with noise detectors, the proposed method still outperforms total variation with higher MSSIMs and comparable PSNRs. This implies that the proposed method can preserve image structures better, as shown in Figure 12.Table 2
Denoising performance ofBarbara at high salt-and-pepper noise levels.
Noise levels
Proposed
Total variation with detectors
Proposed with detectors
PSNR
MSSIM
PSNR
MSSIM
PSNR
MSSIM
50%
32.47
0.7486
35.35
0.8647
36.18
0.9262
70%
30.36
0.6205
33.61
0.7790
33.49
0.8214
90%
26.09
0.4619
31.65
0.6310
30.54
0.6322Denoising results ofBarbara image with 70% salt-and-pepper noise. (a) Ground truth image, (b) noisy image, and (c), (d), and (e) are denoised images using the proposed method, total variation with noise detectors, and the proposed method with noise detectors, respectively.
(a)
(b)
(c)
(d)
(e)An efficient method to suppress the noise via combining PANO with noise detectors can be found in [35]. However, the noise detectors proposed in [18] only work for salt-and-pepper noise. How to detect the noise candidates in random-valued noisy image is still unknown. Besides, how to solve the proposed weighted regularization model efficiently and setting the weight optimally need comprehensive investigation. We leave these as the future work.


## 5. Conclusions

Impulse noise removal with similarity-motivated sparse representation is proposed. With a patch-based nonlocal operator to model sparse representation of similar image patches, thel1-l1 optimization problem is solved by a modified iterative alternating direction minimization algorithm.Experiments on simulated data demonstrate that the proposed method significantly improve the recovery of textures and edges while removing both salt-and-pepper noise and random-valued impulse noise. It can be also used to remove the impulse noise-like artifacts in mural images well. Denoising performance of the proposed method may be further improved if one can learn a proper similarity from serious noisy image.



---

*Source: 595782-2014-07-06.xml*