# An Overview of Composite Standard Elastic-Net Distribution Based on Complex Wavelet Coefficients

Authors: Tahani A. Aloafi; Hassan M. Aljohani
Journal: Journal of Mathematics
 (2022)

---

## Abstract

Statistical tools have changed significantly in the past decade; when the maximum likelihood method is usually applied, it provides an inaccurate solution due to its unsuitable properties and causes problems in fit. However, the current Ordinary Least Square (OLS) Model is more reliable in specific situations where the estimation is based on the slope only. On the other hand, some methods depend on the slope, and the intercept is recommended. These methods can be described as thresholding rules. Another often preferred method is the posterior mean (PM) technique. This procedure depends on two parts, the first part is the likelihood, and the other is the prior distribution, where the second part plays a significant role in the estimation. In this article, the standard elastic-net distribution is assumed as the prior part, which consists of two parts, the first being the normal distribution and the second being the double exponential distribution. The reason why this model is used is that wavelet tools have different levels of resolution. Thus, this model may provide a more accurate estimation for the wavelet coefficients, which might be estimated using a normal or double exponential distribution. In the past, some properties of elastic-net penalized were introduced and discussed. However, more properties are introduced for this distribution. In addition, two models based on the elastic-net method are demonstrated, involving the point mass prior. The first model combines normal as likelihood, and elastic-net distributions as prior, while the other combines the double exponential distribution as likelihood with the elastic-net distribution as prior. Moreover, the level-dependent components are estimated at each resolutions. A simulated investigation is studied using the Markov Chain Monte Carlo (MCMC) tool to estimate the underlying features, where real data are involved and modelled using the proposed methods. A stationary wavelet basis is also applied. As a result, the proposed procedure reduces noise levels which may be helpful since noise levels often corrupt real data, usually a significant cause of most numerical estimation problems.


---

## Body

## 1. Introduction

The traditional statistics tools start with the ordinary least squares method, which is used to estimate the unknown parameters. However, this method provides inaccurate results because noise levels corrupt the real data. The technique has improved to reduce the problem and improve reconstruction; thus, more complex statistical tools are available. The reconstruction of unknown parameters becomes more accurate after the posterior distribution of Ridge, Lasso, and other penalized forms is implemented. One of these methods of penalized form was the elastic-net technique, which was introduced by the study of [1]; subsequently, in 2017, Hassan, with his supervisor, Robert, wrote about the penalty of implementing elastic-net as distribution. The prior parameters were fixed based on knowledge or given information preceding that. Nowadays, the distribution of elastic-net is available with some properties. This distribution allows estimation of the prior parameters using a flexible method such as the MCMC method. There are two reasons for using the penalized form as a probability distribution. The first is that the distribution form allows estimation of the distribution parameters rather than keeping them fixed. The second is that the wavelet coefficients are centered and peaked at zero [2]; this statement is true for some wavelet coefficients, but not all. Therefore, it is believed that wavelet coefficients need to thresh at some levels and shrink at other levels. The SElastic-net ζ,γ has two parts. The first is normal, which allows the shrinkage of wavelet coefficients, and the second is double exponential distribution, which allows them to thresh. In this article, the two previously mentioned methods will be used. The first has earlier been used by Hassan with his supervisor Robert [3, 4]. The other is a newly proposed method, where the difference between these methods is that, in the first one, the variance is estimated using the wavelet coefficients at their finest scale, see [5] for more details. The use of the normal distribution and the prior distribution gives a quadratic equation, meaning that there are two solutions to an estimation of the wavelet coefficients. However, in the second method, where the variance is proposed using exponential distribution, the prior changes the likelihood from normal distribution to double exponential distribution [6]. The procedure provides a cubic equation, where the solution becomes more complicated. However, the resolution of this combination between double exponential and standard elastic net ζ,γ can be provided using the following numerical method. Now, suppose the problem is given by the following equation:(1)yi=βi+εi,i=1,2,3,…,n,where yi is a general observation, βi is the parameter, which is unknown, and εi is the noise, which is assumed to be independent N0,σ2 distributed. The relation between yi and βi is linear, adding a small value that is almost normal; this describes the impact of noise. The use of normal distribution as a likelihood function provides an inaccurate solution since the maximum of exp−1/σ2yi−βi2 equals yi=βi, which shows no difference between the observation and the estimation. Moreover, the included normal as likelihood provides a solution based on the slope only. On the other hand, the necessary proposed prior becomes more fixable, such as the standard elastic net ζ,γ, which has two parts normal as well as double exponential distributions, and the normal is used as likelihood. Figure 1 shows the plot of smooth data and the corresponding wavelet coefficients where the magnitude of wavelet coefficients at their finest level are small and can be centered around the zero. Note that, it can be threshed. However, the wavelet coefficients at a high level are large and seem to be away from zero and these coefficients can be shrunk. Several authors have published many articles about the Bayesian approach to the wavelet method, such as [7] who solved the model using single prior Weibull distribution based on two methods, where the model is compared with various mixture priors. Reference [8] used the wavelet method in nonparametric regression with kernel function based on the normal distribution. The monographs of [9, 10] can be referred to for extensive details. Vidakovic and Ruggeri used double experiential distribution as a probability function after proposing the noise variance using exponential distribution and a prior. This article aims to generalise the Bayesian Adaptive Multiresolution Shrinker (BAMS) method of Vidakovic and Ruggeri, where level independent densities are applied. Wavelet bases are introduced by Daubechies in [11] and it is clearly explained by [12]. Nondecimated technique is studied by [13, 14]. Lawton studied the complex valued wavelet transforms to subband decomposition and provided an application [15]. The MCMC can be found in [16, 17]. This article is structured as follows: introduction to the standard elastic-net distribution in Section 2. Some properties are referred to in Sections 3. Modelling is provided in Section 4. Section 5 gives all technical arguments are referred to. Section 6 explains the complex-valued wavelet basis. Simulation study to investigate estimation properties are provided in Section 7. Section 8 gives the result of the proposed rule to real data. The final summary and conclusions are presented in Section 9.Figure 1
Plots of the artificial data (a), wavelet coefficients of the artificial data at 32 equally spaced point.
(a)(b)


## 2. Standard Elastic-Net Distribution

Definition 1.
LetΘ be a random variable with probability density function fβ and defined as follows:(2)fβ;ζ,γ=1Zζ,γexp−gβ;ζ,γ,β∈Θ⊂R;ζ>0,0<γ<1,where gβ;ζ,γ is a function based on ζ and γ such that gβ;ζ,γ=ζγβ2+1−γβ, g¯β;ζ,γ=ζγβ2−1−γβ and(3)Zζ,γ=4πζγexp14γζ1−γ21−Φζ1−γ2ζγ,where the parameters ζ and γ are the scale parameters. This distribution is called the standard elastic net distribution with mean 0 and where we suppose A=ζγ and B=ζ1−γ, then variance is given as follows:(4)2exp1/4AB22π/2A2AB/2Aϕ2AB/2A+1−Φ2AB/2A+2πB/2A21−Φ2AB/2A22πA/Aexp1/4AB21−Φ2AB/2A,where ϕ⋅ and Φ⋅ are the standard normal and the standard normal cumulative distributions, respectively.
Figure2 shows the plot of the elastic-net distribution with different values of γ. It can be seen that as γ⟶1, the shape of the distribution takes a normal curve, which is a bell-curve. This standard elastic net can be symbolized by SElastic-net ζ,γ.Figure 2
Plots of the standard elastic net withζ=1 and different values of γ.Remark 1.
(1)
Settingγ=0 in (1), then Zζ,γ=2/ζ, the results for the double exponential distribution with mean zero and variance 2/ζ2(2)
Settingγ=1 in (1), then Zζ,γ=π/ζ, the results for the normal distribution with mean zero and variance 1/ζ
Figure3 shows the cumulative function, which has an S-shape and is increasing and continuous. As γ⟶0 more wavelet coefficients are threshold.Figure 3
Plots the cumulative function of SElastic-netζ,γ with ζ=1 and different values of γ.


## 3. Some Properties

The cumulative distribution is given as follows:(5)Fu;ζ,γ=1Zζ,γ∫−∞uexp−gβ;ζ,γdθ=1Zζ,γπζγexpζ1−γ24γΦ2ζγu−1−γγ,u<0,1Zζ,γπζγexpζ1−γ24γΦ−2ζγ1−γγ+Φ2ζγu+1−γγ−Φ2ζγ1−γγ,u≥0.Forγ⟶1 then(6)Fu;ζ,γ=1=1Zζ,γ=1πζΦ2ζγu,−∞<u<∞.The SElastic-netζ,γ hazard rate function is given by the following equation:(7)hu;ζ,γ=1/Zζ,γexp−gβ;ζ,γ1−1/Zζ,γπ/ζγexpζ1−γ2/4γΦ2ζγu−1−γ/γ,u<0,1/Zζ,γexp−gβ;ζ,γ1−1/Zζ,γπ/ζγexpζ1−γ2/4γΦ−2ζγ1−γ/γ+Φ2ζγu+1−γ/γ−Φ2ζγ1−γ/γ,u≥0.For more details—see Appendix A. Forγ⟶1 then(8)hu;ζ,γ=1=1/Zζ,γexp−gβ;ζ,γ1−π/ζΦ2ζγu,−∞<u<∞.Figure4 shows the plots of the hazard rate function with different values of γ, where it changes, as well as the products that failed early. Hence, SElastic-net ζ,γ distribution is a continuous distribution, symmetric about its zero and is nonzero over the entire line. The moment generating function of the SElastic-net ζ,γ distribution is given by the following equation:(9)Mt=2πZζ,γexpγζ4tζγ+1−γγ2Φ−2γζ2tζγ+1−γγ+expγζ4tζγ−1−γγ2Φ¯−2γζ2tζγ−1−γγ,where Φ¯=1−Φ.Figure 4
Plots the hazard rate function of SElastic-netζ,γ with different values of γ and; ζ=1 (a), ζ=3 (b).
(a)(b)


## 4. Modelling

Let the model be defined as follows:(10)yi=βi+εi,where y=yi;i=1,2,…,n represents the observation, β=βi;i=1,2,…,n and refers to the true signal, and where ε is represented by an additive Gaussian error model with mean zero and variance σ2; the likelihood takes the following form:(11)PY|β=∏i=1n12πσ2exp−12σ2yi−βi2;σ>0,βi,yi∈R.Hence, this means that the relationship betweenyi and βi is linear, which is affected by white noise. More precisely, the values of the observed data, y, are known and recorded with the level of noise, this procedure of changing can lead to different estimates. Let prior of the wavelet coefficients be defined as follows:(12)Pβi=1Zζ,γexp−ζγβ2+1−γβi,βi∈R;ζ>0,0<γ<1,where(13)Zζ,γ=4πζγexp14γζ1−γ21−Φζ1−γ2ζγ.The main reason for choosing this prior is that the elastic net distribution has flexibility and reliability since the wavelet coefficients have different degrees of shrinkage or thresholding. For more details see [3].


## 5. Bayesian Approaches

### 5.1. Model I

Here the posterior distribution can be written as follows:(14)Pβ|Y≈12πσ2Zζ,γn∏i=1nexp−12σ2yi−βi2exp−ζγβi2+1−γβi,where the denominate is negligible because it does not contain any information about βi. The predictive distribution of βi, when yi=0, is defined as follows:(15)Pyi;σ,γ,ζ=12πσ2Zζ,γ∫exp−12σ2βi2exp−ζγβi2+1−γβidβi=221/2σ2+ζγexpζ21−γ2/41/2σ2+ζγσZζ,γΦ−ζ1−γ21/2σ2+ζγ.The predictive ofβi, when yi<0, is denoted as follows:(16)Pyi;σ,γ,ζ=12πσ2Zζ,γ∫exp−12σ2yi−βi2exp−ζγβi2+1−γβidβi=exp−1/2σ2yi2σZζ,γ121/2σ2+ζγexp12σ2+ζγ1/σ2yi+ζ1−γ21/2σ2+ζγ2×Φ−2/2σ2yi+ζ1−γ21/2σ2+ζγ+exp12σ2+ζγ1/σ2yi−ζ1−γ21/2σ2+ζγ2×Φ−1/σ2yi−ζ1−γ21/2σ2+ζγ.The distribution of the parameterβi when yi>0 is(17)Pyi;σ,γ,ζ=12πσ2Zζ,γ∫exp−12σ2yi−βi2exp−ζγβi2+1−γβidβi=exp−1/2σ2yi2σZζ,γ121/2σ2+ζγexp12σ2+ζγ1/σ2yi+ζ1−γ21/2σ2+ζγ2×Φ−2/2σ2yi+ζ1−γ21/2σ2+ζγ+exp12σ2+ζγ1/σ2yi−ζ1−γ21/2σ2+ζγ2×Φ212σ2+ζγyi−1/σ2yi−ζ1−γ21/2σ2+ζγ−Φ−1/σ2yi−ζ1−γ21/2σ2+ζγ+exp12σ2+ζγ1/σ2yi−ζ1−γ21/2σ2+ζγ2×Φ−212σ2+ζγyi−1/σ2yi−ζ1−γ21/2σ2+ζγ.For more details, see Appendix B. Figure5 shows the wavelet coefficient distribution with different values of γ, and its flat shape as γ⟶zero. This procedure demonstrates the behavior of the distribution when the distribution is normal. As γ increases towards 1, the shape becomes flat owing to the variance of the wavelet coefficients, while the shape of the distribution becomes narrow as γ increases towards zero since the wavelet coefficients center around and peak at zero. The expectation of βi, when yi<0, is denoted as follows:(18)Eβi;yi,σ,γ,ζ=12πσ2Zζ,γ×∫βiexp−12σ2yi−βi2exp−ζγβi2+1−γβidβi=exp−1/2σ2yi22πσ2Zζ,γ121/2σ2+ζγ×exp12σ2+ζγζ1−γ+2/2σ2yi21/2σ2+ζγ2×−exp−21/2σ2+ζγyi−ζ1−γ+2/2σ2yi/21/2σ2+ζγ2/221/2σ2+ζγ+2πζ1−γ+2/2σ2yi21/2σ2+ζγ×Φ212σ2+ζγyi−ζ1−γ+2/2σ2yi21/2σ2+ζγ+exp−21/2σ2+ζγyi−ζ1−γ+2/2σ2yi/21/2σ2+ζγ2/221/2σ2+ζγ−exp−21/2σ2+ζγζ1−γ+2/2σ2yi/21/2σ2+ζγ2/221/2σ2+ζγ+2πζ1−γ+2/2σ2yi21/2σ2+ζγΦ−212σ2+ζγζ1−γ+2/2σ2yi21/2σ2+ζγ−Φ212σ2+ζγyi−ζ1−γ+2/2σ2yi21/2σ2+ζγ+exp12σ2−ζγ1/σ2yi−ζ1−γ21/2σ2+ζγ2×exp−21/2σ2+ζγ1/σ2yi−ζ1−γ/21/2σ2+ζγ2/221/2σ2+ζγ−1/σ2yi−ζ1−γ21/2σ2+ζγ×Φ−212σ2+ζγ1/σ2yi−ζ1−γ21/2σ2+ζγ.Figure 5
Plots of wavelet coefficientsβ distribution with different values of γ.For more details—see Appendix C. The Bayes estimator ofβi, when yi<0, is denoted as follows:(19)Eβiyi,σ,γ,ζ=Eβi|yi,σ,γ,ζPyi|σ,γ,ζ.Rules with a more desirable shape result from the prior onθ that contain a point mass at zero; that is given by the following equation:(20)βi|yi∼γδ0+1−γEla−net0,ζ,γ.The marginal is(21)Pyiδ1=γN0,σ2+1−γPβi|yi,σ,γ,ζ.The corresponding Bayes rule is given by the following equation:(22)Πβi|yiδ1=1−γPβi|yi,σ,γ,ζEβyi,σ,γ,ζγN0,σ2+1−γPβi|yi,σ,γ,ζ.Figure6 shows the rule in (22), where its shrinkage rule is based on the shape of the rule. However, it is seen that the rule can shrink the wavelet coefficients sharply as γ increases. The process behavior happens, since the normal distribution appears. More precisely, the posterior distribution is based on the likelihood and the prior, which are based on normal distribution, and it provides a smooth shape. This procedure can be seen in the ordinary least squares and ridge rules. Despite this, the denominator and numerator depend on the elastic net.Figure 6
Plots the Bayes rule of with different values ofγ and; ζ=0.4,σ2=15 and γ=0.1.


### 5.2. Model II

Assuming that, the difference between the truth wavelet coefficients and the unknown wavelet coefficients can be interpreted as a Gaussian error model, it is given as follows:(23)y|θ∼Nθ,σ2,with density is given by the following equation:(24)Py|θ,σ2=12πσ2exp−12σ2y−θ2,−∞<y,θ<∞;σ>0.This expression can be found in many existing articles such as, [18], where the variance can be computed from the highest level of wavelet coefficients [12]. However, the prior variance is assumed as exponential distribution given by the following equation:(25)σ2∼Eμ,with the parameter μ and the density of variance defined as follows:(26)pσ2=μexp−μσ2,σ>0;μ>0.The marginal likelihood obtained as follows:(27)y|θ∼DEθ,12μ,and density defined as follows:(28)py|θ,μ=2μ2exp−2μy−θ,−∞<y,θ<∞;μ>0,which was proven by [6, 19]. The prior distribution of the unknown coefficients θ, is assumed as the standard elastic-net distribution is given by the following equation:(29)θ|ζ,γ∼Ela−net0,ζ,γ.Then, the posterior distribution can be written as follows:(30)PY|β=2μ2Zζ,γn∏i=1nexp−2μyi−βiexp−ζγβi2+1−γβi.The predictive distribution ofβi, when yi=0, is defined as follows:(31)Pyi=2μ2Zζ,γ∫exp−2μβiexp−ζγβi2+1−γβidβi=4μπ2ζγZζ,γexpζγ2μ+ζ1−γ22ζγ2Φ−2ζγ2μ+ζ1−γ2ζγ.The predictive ofβi, when yi<0, is denoted as follows:(32)Pβi|yi,μ,γ,ζ=2μ2Zζ,γ∫exp−2μyi−βiexp−ζγβi2+1−γβidβi=4μπ22ζγZζ,γexp−2μyi+ζγ2μ+ζ1−γ22ζγ2×Φ2ζγyi−2μ+ζ1−γ2ζγ+exp2μyi+ζγ2μ−ζ1−γ22ζγ2×Φ2ζγ2μ−ζ1−γ2ζγ−Φ2ζγyi+2μ−ζ1−γ2ζγ+exp2μyi+ζγ2μ+ζ1−γ22ζγ2Φ−2ζγ2μ+ζ1−γ2ζγ.The predictive ofβi, when yi>0, is denoted as follows:(33)Pβi|yi,μ,γ,ζ=2μ2Zζ,γ∫exp−2μyi−βiexp−ζγβi2+1−γβidβi=4μπ22ζγZζ,γexp−2μyi+ζγ2μ+ζ1−γ22ζγ2×Φ−2ζγ2μ+ζ1−γ2ζγ+exp−2μyi+ζγ2μ−ζ1−γ22ζγ2×Φ2ζγyi−2μ−ζ1−γ2ζγ−Φ−2ζγ2μ−ζ1−γ2ζγ+exp2μyi+ζγ2μ+ζ1−γ22ζγ2×Φ−2ζγyi+2μ+ζ1−γ2ζγ.For more details, see Appendix D. Figure7 shows the plot of the wavelet coefficients β using Model II, with its flat shape, as γ towards zero. The expectation of βi, when yi<0, is denoted as follows:(34)Eβi|yi,σ,γ,ζ=2μ2Zζ,γ∫βiexp−2μyi−βiexp−ζγβi2+1−γβidβi=2μ22ζγZζ,γexp−2μyi+ζγ2μ+ζ1−γ22ζγ2×−12ζγexp−2ζγyi−2μ+ζ1−γ/2ζγ22+2π2μ+ζ1−γ2ζγΦ2ζγyi−2μ+ζ1−γ2ζγ+exp−2μyi+ζγ2μ+ζ1−γ22ζγ2×12ζγexp−2ζγyi−2μ+ζ1−γ/2ζγ22−exp−2ζγ2μ+ζ1−γ/2ζγ22+2π2μ+ζ1−γ2ζγ×Φ−2ζγ2μ+ζ1−γ2ζγ−Φ2ζγyi−2μ+ζ1−γ2ζγ+exp2μyi+ζγ2μ+ζ1−γ22ζγ2×12ζγexp−2ζγ2μ+ζ1−γ/2ζγ22−2π2μ+ζ1−γ2ζγ×Φ2ζγ2μ+ζ1−γ2ζγ.Figure 7
Plots the distribution ofβ with different values of γ and; ζ=0.5, and μ=4.Similarly, the expectation ofβi, when yi>0, is denoted as follows:(35)Eβi|yi,σ,γ,ζ=2μ2Zζ,γ∫βiexp−2μyi−βiexp−ζγβi2+1−γβidβi=2μ22ζγZζ,γexp−2μyi+ζγ2μ+ζ1−γ22ζγ2×−12ζγexp−2ζγ2μ+ζ1−γ/2ζγ22+2π2μ+ζ1−γ2ζγ×Φ2ζγ2μ+ζ1−γ2ζγ+exp−2μyi+ζγ2μ−ζ1−γ22ζγ2×12ζγ−exp−2ζγyi−2μ−ζ1−γ/2ζγ22+exp−2ζγ2μ−ζ1−γ/2ζγ22+2π2μ−ζ1−γ2ζγ×Φ2ζγyi−2μ−ζ1−γ2ζγ−Φ−2ζγ2μ−ζ1−γ2ζγ+exp2μyi+ζγ2μ+ζ1−γ22ζγ2×12ζγexp−2ζγyi+2μ+ζ1−γ/2ζγ22−2π2μ+ζ1−γ2ζγΦ−2ζγyi+2μ+ζ1−γ2ζγ.For more details—see Appendix E. The posterior mean ofβi, when yi<0, is denoted as follows:(36)Pβi|yi,μ,γ,ζ=Eyi|yi,μ,γ,ζPyi|yi,μ,γ,ζ.Rules with a more desirable shape result from the prior onθ that contain a point mass at zero; that is given by the following equation:(37)βi|yi∼γδ0+1−γEla−net0,ζ,γ.The marginal is(38)Pyiδ2=γDE0,12μ+1−γPβi|yi,μ,γ,ζ.The corresponding Bayes rule is given by the following equation:(39)Πβi|yiδ2=1−γpβi|yi,μ,γ,ζEβi|yi,μ,γ,ζγDE0,1/2μ+1−γpβi|yi,μ,γ,ζ.Figure8 shows the plots of the rule in (39), It provides a thresholding rule since the wavelet coefficients around zero increase toward zero because the denominator and numerator depend on the elastic net. In addition, the posterior includes likelihood, which is the double exponential distribution. According to [2, 20], the thresholding rule is recommended, owing to the empirical.Figure 8
Plots the ruleΠβi|yiδ2 of β with different values of γ= and; ζ=0.6, μ=9 and γ=0.9.


## 5.1. Model I

Here the posterior distribution can be written as follows:(14)Pβ|Y≈12πσ2Zζ,γn∏i=1nexp−12σ2yi−βi2exp−ζγβi2+1−γβi,where the denominate is negligible because it does not contain any information about βi. The predictive distribution of βi, when yi=0, is defined as follows:(15)Pyi;σ,γ,ζ=12πσ2Zζ,γ∫exp−12σ2βi2exp−ζγβi2+1−γβidβi=221/2σ2+ζγexpζ21−γ2/41/2σ2+ζγσZζ,γΦ−ζ1−γ21/2σ2+ζγ.The predictive ofβi, when yi<0, is denoted as follows:(16)Pyi;σ,γ,ζ=12πσ2Zζ,γ∫exp−12σ2yi−βi2exp−ζγβi2+1−γβidβi=exp−1/2σ2yi2σZζ,γ121/2σ2+ζγexp12σ2+ζγ1/σ2yi+ζ1−γ21/2σ2+ζγ2×Φ−2/2σ2yi+ζ1−γ21/2σ2+ζγ+exp12σ2+ζγ1/σ2yi−ζ1−γ21/2σ2+ζγ2×Φ−1/σ2yi−ζ1−γ21/2σ2+ζγ.The distribution of the parameterβi when yi>0 is(17)Pyi;σ,γ,ζ=12πσ2Zζ,γ∫exp−12σ2yi−βi2exp−ζγβi2+1−γβidβi=exp−1/2σ2yi2σZζ,γ121/2σ2+ζγexp12σ2+ζγ1/σ2yi+ζ1−γ21/2σ2+ζγ2×Φ−2/2σ2yi+ζ1−γ21/2σ2+ζγ+exp12σ2+ζγ1/σ2yi−ζ1−γ21/2σ2+ζγ2×Φ212σ2+ζγyi−1/σ2yi−ζ1−γ21/2σ2+ζγ−Φ−1/σ2yi−ζ1−γ21/2σ2+ζγ+exp12σ2+ζγ1/σ2yi−ζ1−γ21/2σ2+ζγ2×Φ−212σ2+ζγyi−1/σ2yi−ζ1−γ21/2σ2+ζγ.For more details, see Appendix B. Figure5 shows the wavelet coefficient distribution with different values of γ, and its flat shape as γ⟶zero. This procedure demonstrates the behavior of the distribution when the distribution is normal. As γ increases towards 1, the shape becomes flat owing to the variance of the wavelet coefficients, while the shape of the distribution becomes narrow as γ increases towards zero since the wavelet coefficients center around and peak at zero. The expectation of βi, when yi<0, is denoted as follows:(18)Eβi;yi,σ,γ,ζ=12πσ2Zζ,γ×∫βiexp−12σ2yi−βi2exp−ζγβi2+1−γβidβi=exp−1/2σ2yi22πσ2Zζ,γ121/2σ2+ζγ×exp12σ2+ζγζ1−γ+2/2σ2yi21/2σ2+ζγ2×−exp−21/2σ2+ζγyi−ζ1−γ+2/2σ2yi/21/2σ2+ζγ2/221/2σ2+ζγ+2πζ1−γ+2/2σ2yi21/2σ2+ζγ×Φ212σ2+ζγyi−ζ1−γ+2/2σ2yi21/2σ2+ζγ+exp−21/2σ2+ζγyi−ζ1−γ+2/2σ2yi/21/2σ2+ζγ2/221/2σ2+ζγ−exp−21/2σ2+ζγζ1−γ+2/2σ2yi/21/2σ2+ζγ2/221/2σ2+ζγ+2πζ1−γ+2/2σ2yi21/2σ2+ζγΦ−212σ2+ζγζ1−γ+2/2σ2yi21/2σ2+ζγ−Φ212σ2+ζγyi−ζ1−γ+2/2σ2yi21/2σ2+ζγ+exp12σ2−ζγ1/σ2yi−ζ1−γ21/2σ2+ζγ2×exp−21/2σ2+ζγ1/σ2yi−ζ1−γ/21/2σ2+ζγ2/221/2σ2+ζγ−1/σ2yi−ζ1−γ21/2σ2+ζγ×Φ−212σ2+ζγ1/σ2yi−ζ1−γ21/2σ2+ζγ.Figure 5
Plots of wavelet coefficientsβ distribution with different values of γ.For more details—see Appendix C. The Bayes estimator ofβi, when yi<0, is denoted as follows:(19)Eβiyi,σ,γ,ζ=Eβi|yi,σ,γ,ζPyi|σ,γ,ζ.Rules with a more desirable shape result from the prior onθ that contain a point mass at zero; that is given by the following equation:(20)βi|yi∼γδ0+1−γEla−net0,ζ,γ.The marginal is(21)Pyiδ1=γN0,σ2+1−γPβi|yi,σ,γ,ζ.The corresponding Bayes rule is given by the following equation:(22)Πβi|yiδ1=1−γPβi|yi,σ,γ,ζEβyi,σ,γ,ζγN0,σ2+1−γPβi|yi,σ,γ,ζ.Figure6 shows the rule in (22), where its shrinkage rule is based on the shape of the rule. However, it is seen that the rule can shrink the wavelet coefficients sharply as γ increases. The process behavior happens, since the normal distribution appears. More precisely, the posterior distribution is based on the likelihood and the prior, which are based on normal distribution, and it provides a smooth shape. This procedure can be seen in the ordinary least squares and ridge rules. Despite this, the denominator and numerator depend on the elastic net.Figure 6
Plots the Bayes rule of with different values ofγ and; ζ=0.4,σ2=15 and γ=0.1.


## 5.2. Model II

Assuming that, the difference between the truth wavelet coefficients and the unknown wavelet coefficients can be interpreted as a Gaussian error model, it is given as follows:(23)y|θ∼Nθ,σ2,with density is given by the following equation:(24)Py|θ,σ2=12πσ2exp−12σ2y−θ2,−∞<y,θ<∞;σ>0.This expression can be found in many existing articles such as, [18], where the variance can be computed from the highest level of wavelet coefficients [12]. However, the prior variance is assumed as exponential distribution given by the following equation:(25)σ2∼Eμ,with the parameter μ and the density of variance defined as follows:(26)pσ2=μexp−μσ2,σ>0;μ>0.The marginal likelihood obtained as follows:(27)y|θ∼DEθ,12μ,and density defined as follows:(28)py|θ,μ=2μ2exp−2μy−θ,−∞<y,θ<∞;μ>0,which was proven by [6, 19]. The prior distribution of the unknown coefficients θ, is assumed as the standard elastic-net distribution is given by the following equation:(29)θ|ζ,γ∼Ela−net0,ζ,γ.Then, the posterior distribution can be written as follows:(30)PY|β=2μ2Zζ,γn∏i=1nexp−2μyi−βiexp−ζγβi2+1−γβi.The predictive distribution ofβi, when yi=0, is defined as follows:(31)Pyi=2μ2Zζ,γ∫exp−2μβiexp−ζγβi2+1−γβidβi=4μπ2ζγZζ,γexpζγ2μ+ζ1−γ22ζγ2Φ−2ζγ2μ+ζ1−γ2ζγ.The predictive ofβi, when yi<0, is denoted as follows:(32)Pβi|yi,μ,γ,ζ=2μ2Zζ,γ∫exp−2μyi−βiexp−ζγβi2+1−γβidβi=4μπ22ζγZζ,γexp−2μyi+ζγ2μ+ζ1−γ22ζγ2×Φ2ζγyi−2μ+ζ1−γ2ζγ+exp2μyi+ζγ2μ−ζ1−γ22ζγ2×Φ2ζγ2μ−ζ1−γ2ζγ−Φ2ζγyi+2μ−ζ1−γ2ζγ+exp2μyi+ζγ2μ+ζ1−γ22ζγ2Φ−2ζγ2μ+ζ1−γ2ζγ.The predictive ofβi, when yi>0, is denoted as follows:(33)Pβi|yi,μ,γ,ζ=2μ2Zζ,γ∫exp−2μyi−βiexp−ζγβi2+1−γβidβi=4μπ22ζγZζ,γexp−2μyi+ζγ2μ+ζ1−γ22ζγ2×Φ−2ζγ2μ+ζ1−γ2ζγ+exp−2μyi+ζγ2μ−ζ1−γ22ζγ2×Φ2ζγyi−2μ−ζ1−γ2ζγ−Φ−2ζγ2μ−ζ1−γ2ζγ+exp2μyi+ζγ2μ+ζ1−γ22ζγ2×Φ−2ζγyi+2μ+ζ1−γ2ζγ.For more details, see Appendix D. Figure7 shows the plot of the wavelet coefficients β using Model II, with its flat shape, as γ towards zero. The expectation of βi, when yi<0, is denoted as follows:(34)Eβi|yi,σ,γ,ζ=2μ2Zζ,γ∫βiexp−2μyi−βiexp−ζγβi2+1−γβidβi=2μ22ζγZζ,γexp−2μyi+ζγ2μ+ζ1−γ22ζγ2×−12ζγexp−2ζγyi−2μ+ζ1−γ/2ζγ22+2π2μ+ζ1−γ2ζγΦ2ζγyi−2μ+ζ1−γ2ζγ+exp−2μyi+ζγ2μ+ζ1−γ22ζγ2×12ζγexp−2ζγyi−2μ+ζ1−γ/2ζγ22−exp−2ζγ2μ+ζ1−γ/2ζγ22+2π2μ+ζ1−γ2ζγ×Φ−2ζγ2μ+ζ1−γ2ζγ−Φ2ζγyi−2μ+ζ1−γ2ζγ+exp2μyi+ζγ2μ+ζ1−γ22ζγ2×12ζγexp−2ζγ2μ+ζ1−γ/2ζγ22−2π2μ+ζ1−γ2ζγ×Φ2ζγ2μ+ζ1−γ2ζγ.Figure 7
Plots the distribution ofβ with different values of γ and; ζ=0.5, and μ=4.Similarly, the expectation ofβi, when yi>0, is denoted as follows:(35)Eβi|yi,σ,γ,ζ=2μ2Zζ,γ∫βiexp−2μyi−βiexp−ζγβi2+1−γβidβi=2μ22ζγZζ,γexp−2μyi+ζγ2μ+ζ1−γ22ζγ2×−12ζγexp−2ζγ2μ+ζ1−γ/2ζγ22+2π2μ+ζ1−γ2ζγ×Φ2ζγ2μ+ζ1−γ2ζγ+exp−2μyi+ζγ2μ−ζ1−γ22ζγ2×12ζγ−exp−2ζγyi−2μ−ζ1−γ/2ζγ22+exp−2ζγ2μ−ζ1−γ/2ζγ22+2π2μ−ζ1−γ2ζγ×Φ2ζγyi−2μ−ζ1−γ2ζγ−Φ−2ζγ2μ−ζ1−γ2ζγ+exp2μyi+ζγ2μ+ζ1−γ22ζγ2×12ζγexp−2ζγyi+2μ+ζ1−γ/2ζγ22−2π2μ+ζ1−γ2ζγΦ−2ζγyi+2μ+ζ1−γ2ζγ.For more details—see Appendix E. The posterior mean ofβi, when yi<0, is denoted as follows:(36)Pβi|yi,μ,γ,ζ=Eyi|yi,μ,γ,ζPyi|yi,μ,γ,ζ.Rules with a more desirable shape result from the prior onθ that contain a point mass at zero; that is given by the following equation:(37)βi|yi∼γδ0+1−γEla−net0,ζ,γ.The marginal is(38)Pyiδ2=γDE0,12μ+1−γPβi|yi,μ,γ,ζ.The corresponding Bayes rule is given by the following equation:(39)Πβi|yiδ2=1−γpβi|yi,μ,γ,ζEβi|yi,μ,γ,ζγDE0,1/2μ+1−γpβi|yi,μ,γ,ζ.Figure8 shows the plots of the rule in (39), It provides a thresholding rule since the wavelet coefficients around zero increase toward zero because the denominator and numerator depend on the elastic net. In addition, the posterior includes likelihood, which is the double exponential distribution. According to [2, 20], the thresholding rule is recommended, owing to the empirical.Figure 8
Plots the ruleΠβi|yiδ2 of β with different values of γ= and; ζ=0.6, μ=9 and γ=0.9.


## 6. Complex-Valued Wavelets Basis

Wavelet bases have been used widely in past decades, according to the results of an extensive simulation. Wavelet methods treat the wavelet coefficients term by term, where the wavelet coefficients have threshed or shrunk. There are many base wavelets with the sample basis being the Haar wavelet.In this article, a complex wavelet basis is involved, where the real wavelet coefficients are treated and the imaginary wavelet coefficients are negligible. The complex wavelet technique was introduced by [15], who explained the complex-valued wavelets of the Daubechies wavelets. Subsequently, complex wavelets have become more popular in the past two decades. There have been several published articles considering complex-valued wavelets, such as [21–24]. A simple explanation of the method can be found in [25] where the model shows the complex-valued wavelet by a bivariate shrinkage rule leaving the phase undamaged. The method of the complex-valued wavelet can be explained briefly in the following: suppose y1,y2,…,yn have random value, which is known as the observation and β1,β2,…,βn are the parameters, which are unknown, the model can thus be written as follows:(40)y1=βi+εi,where εi is a random variable, such as εi∼N0,σ2. The corresponding wavelet model is given by the following equation:(41)di∗=di+ηi,where D∗=Wy is a vector of wavelet coefficients of the observation, D=Wβ is a vector of the wavelet coefficients of the unknown parameters, η=Wε is a vector of the wavelet coefficients of the noise and η∼Nn0,Inσ2, where W is a matrix transform of the wavelet, based on the choice of the wavelet basis. Barber and Nason assumed the complex-valued wavelet coefficients as follows:(42)di∼N2di∗,Σ,where Σ is a covariance matrix, including the real and the imaginary wavelet coefficients-see [25] for more detail. The complex-valued wavelet method requires a wavelet basis with N being equal to or more than 3 vanishing moments.In Complex-valued “Daubechies” basis, IfN=3, there are four solutions but two are negligible because there are two real extremal-phase wavelets and the other is the complex-valued conjugate pair. Figure 9(a) shows the plot of the wavelet basis has N=3 vanishing moments, while Figure 9(b) shows the real and imaginary wavelet coefficients. It can be seen that the real and imaginary wavelet coefficients are slightly smoother than the wavelet’s function for Daubechies’ wavelet with N=3 vanishing moments. The procedure of the comparison can be explained by the following stepwise technique:(1)
Compute the wavelet coefficients: the real and the imaginary wavelet coefficients(2)
Choose the parameters in models I and II, which give the small minimum mean squared-error (MMSE) by comparing the estimated signal with the truth test function(3)
Run step (2) 1000 iterations(4)
This procedure is replicated till it equals 1000 to calculate the average mean squared error (AMSE)Figure 9
The wavelets function for Daubechies’ waveletN = 3 (a) and complex Daubechies’ wavelet equivalent (b). The real part is drawn as a solid black line and the imaginary wavelet coefficients as a solid red.
(a)(b)Hence, the real wavelet coefficients are estimated by the proposed methods, while the imaginary wavelet coefficients remained. Furthermore, the “LinaMayrand” wavelet basis, with numbers equaling wavelet basis with number equaling 5.1, is used to recover the test function. The proposed method when compared with the complex multiwavelet style (CMWS) using universal hard and soft thresholding and nondecimated transforms is applied.


## 7. Simulation Results

This section implements experiments on simulation to assess the finite sample performance of the proposed methods. The simulated data sets consist of the standard test signal, Heavisine of (Donoho and Johnstone, 1994) and (Nason and Silverman, 1994), corrupted by independent Gaussian noise. The test signal is rescaled to have unit variance. The degree of noise is measured by the ratio of the standard deviations of the signal and noise, referred to as the root signal to noise ratio (SNR). Nondecimated wavelet transform has also been considered. Also, the real and imaginary wavelet coefficients and the real coefficients are estimated, while the imaginary wavelet coefficients are reminded. Then, the underlying signal is calculated from the real coefficients. Tables1 and 2 show the AMSE and associated standard errors obtained when denoising signals using two of the new methods: Models I and II are compared with CMWS universal hard and soft thresholding of [25]. The results in Tables 1 and 2 are based on 1000 simulated data set lengths with n=64 and 128 equally spaced points with different levels of noise σ=0.1,0.2,0.5. In each case, no thresholding was done below level 3. We have compared all of our methods to a range of techniques using real-valued wavelets and implemented the same simulated data sets as in Tables 1 and 2. For all of the results of simulations, we have found the method of [25] has been found to give the smallest AMSE. The first is that the distribution form allows estimation of the distribution parameters rather than keeping them fixed.Table 1
Comparison of nondecimated complex wavelet shrinkage methods for 64 wavelet coefficients: CMWS-hard and CMWS-soft. Results are AMSE and standard errors replicated 1000 for estimation of standard test functions. The bold font indicates the smallest AMSE result.
Signalσ64MWS-hard (SE)MWS-soft (SE)Model 1Model 2HeaviSine0.10.0050 (0.0012)0.0047 (0.0011)0.1010 (0.0109)0.0930 (0.0059)0.20.0192 (0.0050)0.0179 (0.0047)0.1112 (0.0084)0.1066 (0.0053)0.50.1089 (0.0303)0.1079 (0.0325)0.2025 (0.0320)0.2120 (0.0388)0.90.3252 (0.0994)0.3158 (0.0836)0.5326 (0.1041)0.5310 (0.1277)Table 2
Comparison of nondecimated complex wavelet shrinkage methods for 128 wavelet coefficients: CMWS-hard and CMWS-soft. Results are AMSE and standard errors replicated 1000 for estimation of standard test functions. The bold font indicates the smallest AMSE result.
Signalσ128MWS-hard (SE)MWS-soft (SE)Model 1Model 2HeaviSine0.10.0030 (0.0007)0.0028 (0.0006)0.1018 (0.0065)0.0917 (0.0030)0.20.0115 (0.0027)0.0108 (0.0024)0.1303 (0.0117)0.1279 (0.0093)0.50.0678 (0.0171)0.0664 (0.0174)0.2873 (0.0586)0.1610 (0.0523)0.90.2039 (0.0562)0.2023 (0.0544)0.4699 (0.0712)0.4518 (0.0986)A selection of results is presented in Tables1 and 2, where the results are shown for CMWS-hard, CMWS-soft, and the proposed methods, Models I and II. For all these competitive procedures, there has been considerable effort to tune their parameters to give as effective results as possible. Real valued multiwavelet shrinkage and, to a lesser extent, cross-validation, were found to be highly sensitive to the choice of primary resolution. All other methods were found to be robust to the choice of primary resolution. As the level of the noise increases, the result of the AMSE increases and the method CMWS-soft provides a better result than the other method. More precisely, the method of [25], CMWS reduces the AMSE. Tables 1 and 2 show the results of the AMSE when the nondecimated decomposed wavelet coefficients were implemented. Both tables show the results of the proposed methods compared with CMWS-hard and CMWS-soft with five vanishing moments. The CMWS-soft method provides better results when the size of data equals 64 and 128, however, as the size of data increases, the AMSE decreases. Since large data contains more information about the signal. This pattern has been seen in [26].This article considers three ways of estimating the unknown signal: CMWS-Hard; CMWS-soft; model 1, and model 2. More precisely, the simulation was replicated byR=1000 times, and the run for estimating the underlying signal equals 1000 times. the average mean squared-error (AMSE) was calculated with(43)AMSE=1Rm∑j=1R∑i=1mβ^ii−βi2,where β^ji,j=1,…,R is the estimate of the underlying signal from the jth replicate. It is important to know that R is the number of replicates, and m is the length of β. Algorithm 1 shows the main idea of calculating AMSE. The underlying signal is estimated by using (cthresh. The algorithm of the cthresh method has been applied using the R package “wavethresh.” For models 1 and 2, the following Algorithm 2 can be used to explain the idea.Algorithm 1: AMSE algorithm.
Result: AMSELetR=1000forj = 1 to RdoGenerateε∼Nn0,σ2Iny=β+εComputeβ^ComputeMSE=1/n∑i=1nβ^ij−βi2endCompute theAMSE=1/R∑j=1RMSEAlgorithm 2: MSE algorithm for level-dependent prior distributions.
Result: MSEInitializationβ=0, d=0, τ∗=0.01, τ21=0.01,0.01,…,0.01, τ22=0.01,0.01,…,0.01, γk=0.01,0.01,…,0.01μ=0.01,ζk=0.01,…,0.01, and k=1,2,…,J−1,J=log2nComputeΠβi|yiδ2 and MSEOLDforr = 1 to Rdoform = 1 to Mdoforj = 0 to J − 1doCompute the real coefficientsd of y at level j, and M=lengthdfori = 1 to MdoGenerate fromεi a Gaussian distribution N0,τ1rd∗=dj+εjComputeβ∗ComputeΠβi∗|yiδ2∗ and MSENEW=0.5ifMSEOLD>MSENEWthendij = di∗,ζj=ζ∗, γj=γ∗, and μ=μ∗elsedij = dij−1, ζj=ζj−1, and γj=γj−1, μ=μj−1endendendζ∗=ζj+τ21,j,γ∗=γj+τ22,j, μ∗=μ+τUpdataτ, τ1 and τ2, (Robert’s)endCompute MSEendTables1 and 2 show the results of different methods for various levels of noise (0.1, 0.2, 0.5, 0.9), the CMWS-soft method provides a better result than other methods. However, model 2 gives a better result than model 1, where the data is used to estimate the variance. Hence, the variance is estimated by an exponential distribution in model 2.


## 8. Application to Real Data

Within this section, real data is simulated to show investigation of the performance of the methodology. Figure9 shows the popular motorcycle crash data introduced by [27], this set of data is available in R under the package MASS. The location “Time” points are not regularly spaced. Figure 10 shows the plot of 128 points of the motorcycle crash data.Figure 10
Plots the motorcycle crash data introduced by [27].And, Figure11 shows the plot of the reconstructions of different methods; the panel (a) shows the thresholding of the method CMWS-hard where it is seen that this method works well. However, there is a misfit in the top of the panel. The problems in the fitting can be seen in panel (b), which is provided by CMWS-soft. Panels (c) and (d) are recovered by Model I and II, respectively, whereas in Model I the parameters are fixed by using the result in Tables 1 and 2. In this method, the parameters are γ=0.2938, ζ=25.6356 and γ=0.6964, while the parameters are fixed by γ=0.5237,ζ=36.2542,μ=36.8896 and γ=0.2309 in Model II. These parameters are chosen using the MMSE method, where the test function was recovered. As seen, there is a slight difference between the reconstruction of Model I and II.Figure 11
Plots of the reconstructions of thresholding: the panels show reconstruction using; (a) CMWS-hard; (b) CMWS-soft; (c) Model I; and (d) Model II.
(a)(b)(c)(d)


## 9. Conclusion

This article has investigated two new models for real-valued denoising signals using the elastic-net distribution with complex-valued wavelets. Two models are proposed, the first model has normal distribution as likelihood, while the second model has double exponential distribution as likelihood. In the first model, the varianceσ is calculated from the wavelet coefficient at the finest level, while in the second model, the variance was estimated using the exponential distribution. The wavelet coefficients are used to compute the complex wavelet coefficients, where the imagery coefficients are negligible and the underlying signal is estimated using the real coefficients. Extensive simulation results have shown in Tables 1 and 2 where different methods are compared by using AMSE, and the methods of [25] work well and provide a reasonable restriction than the other proposed methods and it reducing the result of the AMSE. As much as 0.0028, when the variance of noise equals 0.1. However, these methods do not offer smooth solutions to real data as they are not regularly spaced. Our method provides slightly more excellent resolution than other methods. However, the CMWS method is simple and easy to use and is also much faster. The proposed methods show smoothing features in the reconstruction when the data are not equally spaced.



---

*Source: 9005413-2022-08-13.xml*