# Improved Denoising with Robust Fitting in the Wavelet Transform Domain

Adrienn Dineva, Annamária R. Várkonyi-Kóczy, József K. Tar

► To cite this version:

Adrienn Dineva, Annamária R. Várkonyi-Kóczy, József K. Tar. Improved Denoising with Robust Fitting in the Wavelet Transform Domain. 6th Doctoral Conference on Computing, Electrical and Industrial Systems (DoCEIS), Apr 2015, Costa de Caparica, Portugal. pp.179-187, 10.1007/978-3-319-16766-4_19. [https://inria.hal.science/hal-01343481](https://inria.hal.science/hal-01343481)

HAL Id: hal-01343481

https://inria.hal.science/hal-01343481

Submitted on 8 Jul 2016

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Improved Denoising with Robust Fitting in the Wavelet Transform Domain

Adrienn Dineva¹, Annamária R. Várkonyi-Kóczy², József K. Tar³,

¹ Doctoral School of Applied Informatics and Applied Mathematics Óbuda University,
Budapest, Hungary

² Bánki Donát Faculty of Mechanical and Safety Engineering,
Institute of Mechatronics and Vehicle Engineering Óbuda University, Budapest, Hungary

³ John von Neumann Faculty of Informatics, Óbuda University, Budapest, Hungary
{¹dineva.adrienn@bgk.uni-obuda.hu, ²varkonyi-koczy@uni-obuda.hu,
³tar.jozsef@nik.uni-obuda.hu}

**Abstract.** In this paper we present a new method for thresholding the coefficients in the wavelet transform domain based on the robust local polynomial regression technique. It is proven that the robust locally-weighted smoother excellently removes the outliers or extreme values by performing iterative reweighting. The proposed method combines the main advantages of multiresolution analysis and robust fitting. Simulation results show efficient denoising at low resolution levels. Besides, it provides simultaneously high density impulse noise removal in contrast to other adaptive shrinkage procedures. Performance has been determined by using quantitative measures, such as signal to noise ratio and root mean square error.

**Keywords:** wavelet shrinkage, robust fitting, nonparametric regression

## 1 Introduction

Many applications of signal representation, adaptive coding, image enhancement, radio astronomy, etc., require the development of highly efficient data processing techniques. Traditional approaches, such as linear filtering, can smooth the corrupted signal, but with weak feature localization and incomplete noise suppression. Nonlinear filters have been proposed to overcome these limitations. Among the classical signal processing methods, wavelet-based noise reduction has been successfully applied to filter data, because it provides information at a level of detail, which is not available with Fourier-based methods [1]. The discrete wavelet transform analyses the signal at different frequency scales with different resolutions by reducing the signal into approximate and detail information [1][2]. For removing noise, wavelet shrinkage employs nonlinear soft thresholding functions in the wavelet domain [3]. The popularity of this nonparametric method is due to the excellent localization and feature extracting behavior. The fast signal transform algorithms provide significant reduction in the computation time [1][2]. The effectiveness of wavelet shrinkage relies on that the wavelet transforms the additive white noise into white noise in the
coefficient domain. Thus, fewer coefficients represent the signal which allows proper separation of the noise. Further, the wavelet-based technique requires only less assumption about the properties of the signal [3]. On the other hand, by using higher decomposition levels, the signal loses more of its important features which degrade the result significantly. The specific choice of the wavelet function, decomposition level, and thresholding rule allows to construct many different shrinkage procedures. A number of advanced concepts have been introduced on defining the threshold estimator rule (see, e.g. [3][4][5][6][7][8]). It is desirable to construct shrinkage procedures that are robust to impulse noises, which can lead to data loss. The main motivation for the work presented in this paper is to show that by applying robust fitting technique in the wavelet domain, highly efficient denoising effect can be achieved. The proposed method is robust to high density impulse-type noises and requires only low resolution levels.

## 1.1 Basics on Wavelet Shrinkage

The two major approaches of the noise removal task are the denoising in the time or space domain and the denoising in the transform domain. The transform-domain denoising procedures assume that the original signal can be well approximated by a linear combination of some basis functions. The wavelet transform preserves the true signal in few high-magnitude atoms and the others can be associated with noise. Consider the classical noise suppression problem:

$$y_i = f(t) + \varepsilon_i, \quad i = 1, \dots, s \qquad (1)$$

where $y_i$ denotes the observed noisy data and $\varepsilon_i$ represents the random noise, which is an independent and identically distributed (iid) process, and (t) stands for time. Let $f$ denote the unknown function. The sampling points are equally spaced $s=2^n$ in order to allow to perform the discrete wavelet transform (DWT) [1]. The issue is to estimate $f$ on $y_i = [y_1, \dots, y_s]$ with minimum risk in least squares sense, i.e. to find

$$\hat{f} = \min \|f - \hat{f}\|^2. \qquad (2)$$

The first step of wavelet shrinkage is the decomposition of Eq. (1), as follows

$$y_{ij} = \omega_{ij} + \varepsilon_i, \quad j=1, \dots, n \qquad (3)$$

where $\omega_{ij}$ are the wavelet (detail and approximate) coefficients on $j^{th}$ scale. The general idea behind wavelet shrinkage is to replace the coefficients with small magnitude to zero (hard thresholding) [3],

$$\eta(\omega_{ij}, \lambda) = \begin{cases} \omega_{ij} & \text{if } \omega_{ij} > \lambda \\ 0 & \text{otherwise} \end{cases} \qquad (4)$$

or set their value to the threshold level [3], the rule is given by
$$ \eta(\omega_{ij}, \lambda) = \operatorname{sgn}(\omega_{ij}), \max(|\omega_{ij}| - \lambda, 0) \quad (5) $$

where $\eta(\omega_{ij}, \lambda)$ is a soft thresholding function applied to the $\omega_{ij}$ transform coefficients and $\lambda$ is the threshold value. Then, the reconstruction process performs the inverse discrete wavelet transform (IDWT).

## 1.2 Thresholding Operations in the Wavelet Domain

In this section we briefly summarize the prevalent shrinking concepts. The importance and advantages of wavelet shrinkage is discussed in several papers (see, e.g. [9]). Most of them construct nonlinear threshold functions based on statistical considerations. Shrinkage of empirical wavelet coefficient (RiskShrink) is firstly introduced in [3]. Another effective and widely used level-dependent and smoothness-adaptive method (SureShrink) is proposed to thresholds each dyadic resolution level using the principle of Stein's Unbiased Estimate of Risk [4][10]. The universal bound thresholding rule also provides good results with low computational complexity [5]; the rule is defined, as follows

$$ \eta = \sigma_{\text{MAD}} \sqrt{2 \log s_j}, \quad (6) $$

where $\sigma_{\text{MAD}} = \frac{\text{median}(\omega_j)}{0.6745}$ denotes the absolute median deviation, and $\omega_j = [\omega_1, \dots, \omega_n]$. Later, the Heuristic Sure thresholding rule is introduced by a heuristic combination of the SureShrink and the universal bound [5];

$$ \eta = \begin{cases} \eta_{\text{UB}} & \text{if } p \ge q \\ \min(\eta_{\text{Sure}}, \eta_{\text{UB}}) & \text{if } p < q \end{cases} \quad (7) $$

where $p = \frac{m-k}{k}$ and $q = (\log_2 k)^2$, $m = \sum_{i=1}^k \omega_i^2$. The Minimax estimator is also a preferred technique [6], the rule is given by

$$ \eta = \begin{cases} \sigma_{\text{MAD}} (0.3936 + 0.1829 \log_2 s) & \text{if } s > 32 \\ 0 & \text{if } s < 32 \end{cases} \quad (8) $$

Regarding the shrinkage approaches concentrating on white noise modelling, another effective data-adaptive procedure for eliminating Gaussian noise from images is BayesShrink [7]. It performs the noise variance estimation on the sub-bands with median estimator. NeighShrink employs an overlapped window on the neighboring coefficients, thus this method takes into account the relation between the atoms [8]. The threshold value for each coefficient is determined by the principle, that when in the block the neighboring atoms represent the signal it is probable that the current element is also a part of the signal.

The other major approach in noise reducing applications is minimizing the effects of outliers (extreme values or elements that deviate from the observation pattern). Recent studies deal with eliminating the outliers during the signal pre-processing.
Several areas of engineering practice benefit from such algorithms, for instance monitoring and fault detection applications, data mining, feature identification in satellite images, etc., (see, e.g. [11][12]). It is proven, that the robust local polynomial regression technique detects outliers excellently [13]. With this in mind, the present paper proposes an improved denoising method based on robust fitting in wavelet domain.

## 2 Contribution to Cloud-based Engineering

Cloud-based engineering has been a major discussion in the recent years [14]. The basic concept is that by sharing dynamic and quickly expandable resources among users more computational power, storage, etc., are available at lower cost. The challenges of outsourced applications involve the development of advanced signal processing techniques. On the other hand, the increasing amount of data, and transmission processes, generate a lot more noise. Typically, in a wider sense, collaborative or adaptive filtering solutions maintain big databases including many different kinds of data, such as sensing and monitoring data, signals of environmental sensing over large areas or multiple sensors. Additionally, several industrial processes depend on outsourced adaptive controllers [15]. In order to guarantee the proper performance, the signals need special care. Dealing with very sensitive data it is essential to ensure lossless processing even in nonstationary environments. The proposed denoising method can be applied in various scenarios where efficient noise removal is required especially when impulse-type disturbances may occur. Recent researches on cloud-based applications of multiple access communication highlight the construction of low probability of intercept signals [15][16]. Wavelet functions are proposed to employ as building blocks or atoms for the construction of communication signals. The theoretical and technical approaches and results presented in [17], show that the wavelet communication scheme satisfies the main objectives of secure, high throughput and low bit error. Based on the proposed robust thresholding method further improvements can be achieved, e.g. in energy threshold detection and recovery problems in the transformed domain.

## 3 Improved Denoising with Robust Fitting in the Wavelet Transform Domain

### 3.1 Local Polynomial Regression

The local polynomial regression (loess) procedure originally described in [18], is also called locally weighted running line smoother. Extensions of the original method can be found in, e.g [19]. The principle of the method can be summarized as follows. Let's consider Eq.(1). Function $f(t)$ can be approximated by fitting a regression surface to the data by determining a local neighbourhood of an arbitrary ($t_0$). These neighbouring points are weighted depending on their distance from ($t_0$). The closer points get larger $w_i$ weights. The estimate $\hat{f}$ is obtained by fitting a linear or quadratic polynomial using the weighted values from the neighbourhood. Detailed
description of the procedure and the loss curve construction can be found in [13][18][20]. Loss relies on least squares regression and it is known that this is vulnerable to outliers that can significantly degrade the result. In order to introduce robustness in the procedure an iterative reweighting is proposed with bisquare method [21]. The brief description of the simple bisquare method following the notation of [20], is the next. The residuals can be obtained $\hat{\epsilon} = y_i - \hat{f}_i$ after fitting a linear regression. The weights are obtained by the bisquare function, the formula is given by

$$ B(x) = \begin{cases} (1 - x_i^2)^2 & |x_i| < 1 \\ 0 & \text{otherwise} \end{cases} \tag{9} $$

and the robust weights are calculated as

$$ r_i = B\left(\frac{\hat{\epsilon}_i}{6\hat{q}_{0.5}}\right) \tag{10} $$

where $\hat{q}_{0.5}$ denotes the median of $|\hat{\epsilon}_i|$.

## 3.2. Wavelet Shrinkage with Robust Fitting

The proposed shrinkage approach includes the following steps. Firstly, the corrupted signal is decomposed with orthogonal wavelet functions, which separates correctly the detail and approximate coefficients of the signal. Then, the robust fitting is applied on the coefficients on each level. Afterwards, the signal is reconstructed with inverse discrete wavelet transform. Thus, the realization of the new shrinkage procedure is the following: 1.) perform the discrete wavelet transform, 2.) fit the local polynomial regression curve on the coefficients with the $w_i$ weights; 3.) get the residuals; 4.) get the median absolute value of the residuals; 5.) calculate the robust weights $r_i$ with Eq. (20.); 6.) repeat step 1 with $r_i w_i$; 7.) repeat step 3 to 6 until it converges; 8.) perform the IDWT.

# 4 Simulation Results

The performance of the proposed procedure has been tested on a one-dimensional signal corrupted with additive white Gaussian noise and impulse noises (Fig.1.). The results have been compared with two other shrinkage algorithms. The simulation has been built by using Matlab7. The performance is measured by the root mean square error (RMSE) and the signal to noise ratio (SNR), calculated by the formula below

$$ \text{SNR}_{\text{dB}} = 10 \times \log_{10} \frac{\sigma_s^2}{\sigma_n^2} \tag{11} $$

where $\sigma_s^2$ is the variation of the signal after denoising and $\sigma_n^2$ is the variation of the eliminated noise. For the decomposition an orthogonal wavelet has been chosen from the Symlet family [2]. The results are summarized in Table1.
<table><caption>**Table 1.** Simulation results.</caption><thead><tr><th></th><th>Shrinkage with<br>Robust Fitting</th><th>HeurSure</th><th>HeurSure</th><th>Minimax</th></tr></thead><tbody><tr><td>SNR [dB] before</td><td>2.2294</td><td>2.2294</td><td>2.2294</td><td>2.2294</td></tr><tr><td>SNR [dB] after</td><td>23.5412</td><td>8.7601</td><td>3.6653</td><td>5.9276</td></tr><tr><td>RMSE</td><td>0.04159</td><td>0.5372</td><td>0.7621</td><td>0.3293</td></tr><tr><td>Level of decomposition</td><td>1</td><td>4</td><td>8</td><td>4</td></tr><tr><td>Abs. max error</td><td>0.2289</td><td>0.8621</td><td>0.9535</td><td>1.1107</td></tr><tr><td>Elapsed Time [s]</td><td>0.4196</td><td>0.0184</td><td>0.0127</td><td>0.0312</td></tr></tbody></table>

The performance of the robust fitting-based method can be seen in Fig. 2. The procedure precisely removes the noise and smoothes the signal. Though, the HeurSure and the Minimax rule are faster (see Table 1.) and eliminate additive noise, but can not cope with impulse noise (Figs. 3-4). Since the reconstruction is not sufficient, further smoothing and impulse-eliminating processes are desired, which increase the total elapsed time. With this in mind, the speed of the proposed procedure is acceptable. Fig. 5. shows the denoising effect of the HeurSure method at 8 decomposition levels. Comparing Fig. 4. with Fig 5. and the SNR and RMSE values, it can be seen, that in case of denoising with smoothness-adaptive shrinkage, the presence of impulse noises and the increase of resolution levels lead to data loss.

<figure><img src="image_2.png" /><figcaption>**Fig. 1.** The original signal corrupted with additive white Gaussian noise and impulse noises</figcaption></figure>

<figure><img src="image_3.png" /><figcaption>**Fig. 2.** Denoising effect of shrinkage with robust fitting. black (dotted) line –original signal, blue (solid) line –denoised signal</figcaption></figure>
<figure><img src="image_4.png" /><figcaption>**Fig. 3.** Denoising effect of minimax method. *black (dotted)* line –original signal, *blue (solid)* line – denoised signal</figcaption></figure>

<figure><img src="image_5.png" /><figcaption>**Fig. 4.** Denoising effect of heuristic Sure method. *black (dotted)* line –original signal, *blue (solid)* line – denoised signal, level of decomposition: 4</figcaption></figure>

<figure><img src="image_6.png" /><figcaption>**Fig. 5.** Denoising effect of heuristic Sure method. *black (dotted)* line –original signal, *blue (solid)* line – denoised signal, level of decomposition: 8</figcaption></figure>

# 5 Conclusions

In this paper a new approach is proposed for shrinking the wavelet coefficients. There are various sources of perturbations that may occur in the coefficient domain. This procedure excellently removes both additive noise and impulse noise with retaining the important parts of the signal. The method requires only low resolution levels and is able to avoid data loss. Simulation results support, that the denoising efficiency can be significantly improved. Due to fitting a polynomial and calculating the robust weights on each point the performance is time-consuming. However, it is not necessary to use all the data points. Research into solving this limitation by
introducing further data adaptive law in the proposed method is already underway.
Future work also aims to reduce complexity, and to extend the procedure to
nonorthogonal representation and investigate other types of signals.

**Acknowledgments.** This work was supported by the Hungarian National Scientific Research Fund (OTKA 105846).

References

1. Mallat, S.: Multiresolution Approximations and Wavelet Orthonormal Bases of $L^2$, Trans. Am. Math. Soc. 315, 69--78 (1989)

2. Daubechies, I.: The Wavelet Transform, Time Frequency Localization and Signal Analysis, IEEE Trans. Inform. Theory 36, 961--1005 (1990)

3. Donoho, D.L.: De-noising by Soft-thresholding, IEEE Trans. on Inf. Theory 41(3), 613--627 (1995)

4. Donoho D.L., Johnstone, I.M.: Ideal Spatial Adaptation via Wavelet Shrinkage, Biometrika 81, 425--445 (1994)

5. Donoho, D.L., Johnstone, I.M.: Adapting to Unknown Smoothness via Wavelet Shrinkage, Journal of the American Statistical Association 90, 1200--1224 (1995)

6. Donoho D., Johnstone, I.M.: Minimax Estimation via Wavelet Shrinkage, Annals of Statistics 26, 879--921 (1998)

7. Chang, S. G., Yu, B., Vetterli, M.: Spatially Adaptive Wavelet Thresholding with Context Modeling for Image Designing, IEEE Trans. on Image Processing 9(9), 1522--1531 (2000)

8. Cai, T.T., Silverman, B.W: Incorporating Information on Neighboring Coefficients into Wavelet Estimation, Sankhya: The Indian Journal of Statistics B, 127--148 (2001)

9. Taswell, C.: The What, How, and Why of Wavelet Shrinkage Denoising, IEEE Computing in Science and Engineering 2(3), 12--19, (2000)

10. Stein, Charles M.: Estimation of the Mean of a Multivariate Normal Distribution, The Annals of Statistics 9(6), 1135--1151 (1981)

11. Mastriani, M.: Fuzzy Thresholding in Wavelet Domain for Speckle Reduction in Synthetic Aperture Radar Images, Int. Journal of Intelligent Technology 1(3), 252--265, (2006)

12. Ivera-Azcárate, A. et al.: Outlier Detection in Satellite Data Using Spatial Coherence, Remote Sensing of Environment 119, 84--91, (2012)

13. Rousseeuw, P. J., Leroy, A. M.: Robust Regression and Outlier Detection, John Wiley & Sons (2005)

14. Rudas, I.J.: Cloud Computing in Intelligent Robotics, In: IEEE 10th Jubilee International Symposium on Intelligent Systems and Informatics, pp. 15--17, IEEE Press, (2012)

15. Tronsco-Pastoriza, J.R. et al.: Secure Signal Processing in the Cloud: Enabling Technologies for Privacy-preserving Multimedia Cloud Processing, IEEE Signal Processing Magazine 30(2), 29--41, (2013)

16. Valsan, S.P., Swarup, K. S.: Wavelet Transform Based Digital Protection for Transmission Lines, Int. Journal of Electrical Power and Energy Systems 31(7-8), 379--388, (2009)

17. Teolis, A.: Computational Signal Processing with Wavelets, Birkhauser Boston (1998)

18. Cleveland, W. S.: Robust Locally Weighted Regression and Smoothing Scatterplots, Journal of the Am. Stat. Ass. 74, 829--836, (1979)

19. Cleveland, W. S. McGill, R.: The Many Faces of Scatterplot, Journal of the Am. Stat. Ass. 79, 829--836 (1984)

20. Martinez, W. L., Martinez, A. R.: Computational Statistics Handbook with MATLAB, Boca Raton, FL: Chapman & Hall/CRC, (2008)

21. Fox, J.: Nonparametric Simple Regression, Thousand Oaks, CA: Sage Publication Inc., (2000)