# Kernel Least Logarithmic Absolute Difference Algorithm

Authors: Dongliang Fu; Wei Gao; Wentao Shi; Qunfei Zhang
Journal: Mathematical Problems in Engineering
 (2022)

---

## Abstract

Kernel adaptive filtering (KAF) algorithms derived from the second moment of error criterion perform very well in nonlinear system identification under assumption of the Gaussian observation noise; however, they inevitably suffer from severe performance degradation in the presence of non-Gaussian impulsive noise and interference. To resolve this dilemma, we propose a novel robust kernel least logarithmic absolute difference (KLLAD) algorithm based on logarithmic error cost function in reproducing kernel Hilbert spaces, taking into account of the non-Gaussian impulsive noise. The KLLAD algorithm shows considerable improvement over the existing KAF algorithms without restraining impulsive interference in terms of robustness and convergence speed. Moreover, the convergence condition of KLLAD algorithm with Gaussian kernel and fixed dictionary is presented in the mean sense. The superior performance of KLLAD algorithm is confirmed by the simulation results.


---

## Body

## 1. Introduction

Kernel adaptive filters as a tremendous breakthrough of the conventional linear adaptive filters have been widely used in many practical nonlinear applications including time series prediction [1], acoustic echo cancellation [2], channel equalization [3], abnormal event detection [4], etc. The scheme of kernel adaptive filtering (KAF) is to map the original input data into high or infinite dimensional feature space via kernel function and then apply the framework of typical linear adaptive filtering to the transformed data in the reproducing kernel Hilbert spaces (RKHS) leading to various KAF algorithms [5–8]. The kernel least-mean-square (KLMS) algorithm, as the benchmark among of KAF algorithms, is developed from the cost function of second-order statistic of the error between the desired signal and instantaneous estimate under the assumption of Gaussian noise for its mathematical simplicity and convenience [9]. Therefore, it can be ensured that the performances of KLMS-type algorithms only for Gaussian disturbance noise severely degrade, when the desired signals are corrupted by the impulsive interferences. In practical applications, e.g., underwater acoustic signal processing [10], wireless communication environments [11], and radar cluster elimination [12], the impulsive noises with the statistical characteristics consisting of infrequency, short duration, and high amplitude are more rational than the ideal Gaussian noises.For conventional linear robust adaptive filtering, the signed adaptive filters and their theoretical analyses have been extensively studied in [13–15]. The generalized maximum correntropy criterion (GMCC) algorithm with generalized Gaussian density function was proposed in [16], and its stability and steady-state mean square performance were also investigated. The authors of [17] proposed the least logarithmic absolute difference (LLAD) with the correntropy-induced metric (CIM) constraint in order to exploit system sparsity and suppress the interferences. The minimum kernel risk-sensitive loss (MKRSL) algorithm was developed to achieve fast convergence speed by applying risk-sensitive loss while yielding the robust performance to outliers [18]. When the shape parameter is equal to 2 and the risk-sensitive parameter gradually tends to trivial, both GMCC and MKRSL algorithms reduce to the ordinary maximum correntropy criterion (MCC) algorithm. The constrained least mean logarithmic square (CLMLS) based on a relative logarithmic cost function and its variants were proposed in [19], and they were used in the application of sparse sensor array synthesis achieving the desired beam pattern with much less senor elements. In [20], a robust least mean logarithmic square (RLMLS) algorithm and its variable step-size variant were presented to combat impulsive noises, and its theoretical mean square performance was also analyzed with the stationary white Gaussian inputs.Meanwhile, the topic of nonlinear system identification corrupted by the impulsive noise also attracted large numbers of research interests [21]. It is thus important and necessary to study the robust KAF algorithm in order to resolve the divergence problem of performance caused by non-Gaussian impulsive noise. In [22], the kernel maximum correntropy criterion (KMCC) algorithm was developed by introducing the framework of MCC into RKHS in the presence of impulsive noises. By combining the kernel method and the affine projection sign algorithm, the kernel affine projection sign algorithm (KAPSA) was proposed in [23] to combat the non-Gaussian impulse interference. An improved variable forgetting factor recursive logarithmic least mean p th (IVFF-RLLMP) algorithm was developed in [24] for the Volterra system identification against the impulsive interference modeled by α-stable distribution. As a counterpart of GMCC algorithm, the generalized kernel maximum correntropy (GKMC) and the quantized GKMC (QGKMC) algorithm were developed in [25] for robust nonlinear adaptive filtering. In [26], the quantized minimum kernel risk-sensitive loss (QMKRSL) algorithm was proposed to achieve better and robust performance of nonlinear filtering for outliers. Motivated by the studies in [27, 28] on the Cauchy loss which has been successfully used in various robust learning applications, the multikernel minimum Cauchy kernel loss (MKMCKL) algorithm was reported in [29] showing the improved nonlinear filtering performance over counterpart single algorithm in the presence of extreme outliers. Recently, the kernel affine projection-like (KAPL) algorithm in RKHS was proposed and investigated for nonlinear channel equalization in scenarios of non-Gaussian noises [30]. The kernel least mean p-power (KLMP) algorithm was proposed to alleviate the adverse impact of impulsive noise in [31, 32], independently. More recently, Nyström kernel recursive generalized maximum correntropy (NKRGMC) with probability density rank-based quantization sampling algorithm was proposed to improve the convergence performance for impulsive noises [33]. The tracking analysis of kernel signed error algorithm (KSEA) with Gaussian kernel for time-variant nonlinear system was analyzed thoroughly under the assumption of impulsive noise [34]. More recently, the logarithmic hyperbolic cosine-based adaptive filter (LHCAF) was proposed in [35] to address the issue of instability of its prototype algorithm, and the transient and steady-state analyses were also provided. Subsequently, the authors of [36] proposed the multiple random Fourier features Cauchy-loss conjugate gradient (MRFGCG) algorithm which has better performance than the classical KAF algorithms in terms of computational complexity and filtering accuracy.Therefore, the cost functions adopting the frameworks of fractional order statistics of error or the distinct types of error measures are able to provide effective ways to reveal the robust performance against impulsive noises. Inspired by the family of linear adaptive filtering algorithms based on the logarithmic cost proposed in [37], our purpose is to extend this scheme into RKHS to obtain the robustness of KAF algorithm particularly within non-Gaussian impulsive noise environment. In this paper, the kernel least logarithmic absolute difference algorithm based on the logarithmic error cost framework is proposed to achieve the nonlinear system identification in the impulsive interference environments, which are more frequently encountered in practical applications. Simulation results illustrate the proposed KLLAD algorithm can consistently decrease the drastic perturbation of recursive weight coefficients caused by the large amplitude of instantaneous estimation error with low probability.Notation: We use normal font small letters x for scalar variables, boldface small letters x for column vectors, and boldface capital letters X for matrices. The superscript ·⊤ represents the transpose of a vector or a matrix. The expectation is denoted by E·, and matrix trace is denoted by tr·. The Gaussian distribution with mean μ and variance σ2 is denoted by Nμ,σ2. The notation ·2 is the ℓ2-norm of its matrix or vector argument; xA2 is the weighted square value x⊤Ax. Notation sgn· is the signum function. The operator eigmaxX denotes the maximum eigenvalue of matrix X. Identity matrix of size N×N is denoted by IN.


## 2. Preliminaries of KAF Algorithms

Letℋ denote a Hilbert space of real-valued function ψ from a compact subspace U⊂ℝL. Function κ:U×U⟶ℝ is a reproducing kernel, and ℋ,·,·ℋ is the induced RKHS with its inner product. The following unknown nonlinear system is considered:(1)yn=f⋆xn+zn.The scalaryn and the vector xn∈ℝL denote the desired output and the input signal, respectively. Here, f⋆ represents the optimum functional to be identified, and zn stands for the non-Gaussian impulsive noise modeled as the contaminated-Gaussian (CG) noise as follows [13, 38]:(2)zn=vn+bnϵn,where both vn and ϵn are statistically independent zero-mean white Gaussian noises with the variances σv2 and σϵ2=Kσv2, with parameter 1≪K. Moreover, the random sequence bn is from a Bernoulli random process with the probability of Prbn=1=pr and Prbn=0=1−pr. Notice that vn and bnϵn represent the common Gaussian noise and particular impulsive interference component, respectively. Subsequently, the probability density function (pdf) of the impulsive CG noise zn is given by(3)pz=1−prN0,σv2+prN0,K+1σv2.Whenpr=0, the impulsive CG noise zn deteriorates to a common white Gaussian noise with zero-mean and variance σv2, namely, impulsive interference free. Correspondingly, the variance of impulsive noise zn is given by(4)σz2=Ezn2=1−prσv2+prK+1σv2.Hence,zn is statistically independent of any other signals. The reason why we adopt the CG model is that it makes the analysis of mean stability of KLLAD algorithm mathematically tractable.Given a sample set of pairs of input vectors and desired output scalars, i.e.,xn,ynn=1N, we aim at estimating a nonlinear regression function ψ that relates input data xn and output data yn corrupted by impulsive noise zn. Let ℋ be a RKHS with kernel κ. By virtue of the representer theorem [39], the function ψ in ℋ that minimizes the regularized least-squares problem:(5)minψ∈ℋ∑n=1Nyn−ψxn2+εψℋ2,with ε a nonnegative regularization constant, can be written as the kernel expansion in terms of available training data in form of(6)ψ=∑n=1Nwnκ·,xn.Whenε=0, it will not affect the derivation of algorithm. It can be seen from (5) and (6) that the functional representation is formulated as the parametric vector form. However, solution (6) is infeasible to be performed in an online manner, because the algorithm can not cope with the linear increase of the size N of the model as the latest input data available. A commonly used strategy is to adopt a finite-order model of the form [5](7)ψ¯=∑m=1Mnwmκ·,xω,m.The set of selected input dataDn=xω,mm=1Mn is so-called online dictionary with the length Mn and is generated by the informative criteria from input vectors xn in an online manner, e.g., coherence criterion [1], surprise criterion [5], and approximate linear dependency [40]. Note that (6) seems to be the same as (7) in form, whereas the length Mn determined by the online sparsification criterion is analogue to the order of transversal filter. Without loss of generality, we chose the coherence criterion which allows the candidate xn to be inserted into the dictionary if its maximum coherence remains below the given threshold δκ, namely,(8)maxm=1,2,…,Mnκxn,xω,m≤δκ,where 0<δκ≤1 determines both the level of sparsity and the coherence of the dictionary. Based on the stochastic gradient of cost functions Jen with respect to w, using the parametric finite-order model (7), we can then obtain the corresponding recursive update equation of the KAF algorithms:(9)wn+1=wn+η∂Jen∂wκω,n.With the positive step-sizeη and the weight coefficients vector wn=wn1,wn2,…,wnMn⊤, the instantaneous estimation error en is given by(10)en=yn−wn⊤κω,n.Moreover, the kernelized input vectorκω,n is defined by(11)κω,n=κxn,xω,1,κxn,xω,2,…,κxn,xω,Mn⊤.Adopting the mean-squared error (MSE) criterion for (9) leads to the scheme of KLMS algorithm described as follows:(12)wn+1=wn+ηenκω,n,where the complete KLMS algorithm with online sparsification criterion is not present for clarity.It can be easily observed from (12) that the KLMS-type algorithms suffer from sever performance degradation and even divergence caused by the instantaneous estimation error given in (10), which is contaminated by the non-Gaussian impulsive noise defined in (2). It is thus critical for the design of robust KAF algorithm to effectively suppress the adverse effects of impulsive interference noise.


## 3. KLLAD Algorithm

In this section, we shall derive the KLLAD algorithm based on the logarithmic cost function.Logarithmic cost as one type of relative cost measures is capable of providing relatively legitimate amplification for the ordinary error fluctuation and significant attenuation for very large error value induced by impulsive interferences. It has been illustrated in [37] that the logarithmic cost function can proportionally adjust the weight coefficients for small and large error values depending on the combination weights varying with time. Therefore, we introduce the differentiable combined logarithmic error cost function as follows:(13)Jen=Fen−1λln1+λFen,where λ is a design parameter, and Fen is the conventional cost function of instantaneous estimation error. Note that the universal adaptive filtering algorithm obtained from (13) is able to update the coefficients by small error and suppress the drastic perturbation of error simultaneously.Taking the derivative of (13) with respective to weight vector w, we obtain(14)∇wJen=∇wFenλFen1+λFen.Since the impulsive CG noise exists, the kernel adaptive filters based on quadratic cost function have to face the severe performance degradation. Nevertheless, the signed adaptive filtering algorithms based on theℓ1-norm of estimation error are robust to the impulsive noise because of its recursive update equation only utilizing the sign of instantaneous estimation error. In order to mitigate the adverse impact of impulsive noise on KAF, let the conventional cost function of error Fen be Een using (10), namely,(15)Fen=Eyn−wn⊤κω,n.It should be pointed out that cost function (13) using (15) can perform less attenuation for the small estimation errors and more attenuation for relatively large estimation errors simultaneously. Substituting (15) into (14) leads to(16)∂Jen∂w=∂Fen∂wλFen1+λFen.Applying the steepest-decent method to minimize the logarithmic cost function (13), thus the weights vector of KLLAD algorithm can be solved iteratively by(17)wn+1=wn+η−∂Jen∂w.By (16), the recursive update equation of KLLAD algorithm can be reformulated as(18)wn+1=wn−η∂Fen∂wλFen1+λFen,where the subgradient in the second term on the right hand side of (18) is calculated as(19)∂Fen∂w≈sgnen.Note that the above approximation notation implies that the subgradient of conventional cost functionFen is replaced by its instantaneous estimate. Substituting (15) and (19) into (18) and then removing the expectations, the stochastic update equation of KLLAD algorithm can be rewritten as(20)wn+1=wn+ηλen1+λensgnenκω,n=wn+ηλen1+λenκω,n,where κω,n denotes the Mn×1 dimensional kernelized input vector as in (11). For universal nonlinear filtering performance, we only focus on using the Gaussian kernel function κx,x′=exp−x−x′22/2ξ2, with ξ>0 the kernel bandwidth in this paper. Moreover, the Gaussian kernel function has been successfully used in the theoretical analysis of KAF algorithms due to its mathematical convenience and tractability in the derivation [34, 41]. Based on (20), the recursive update equations of KLLAD algorithm based on the finite-order model using the coherence criterion are presented in the following.At each time instantn, the input xn will be decided into the case of rejection or reception according to the coherence criterion (8) for online dictionary Dn as follows:(i)
Rejection: ifmaxm=1,2,…,Mnκxn,xω,m>δκ(21)Dn+1=Dn,Mn+1=Mn,wn+1=wn+ηλen1+λ|en|κω,n.Note that the dimensions of kernelized input vectorκω,n and weight vector wn remain still in this case.(ii)
Reception: ifmaxm=1,2,…,Mnκxn,xω,m≤δκ(22)Dn+1=Dn∪xn,Mn+1=Mn+1,wn+1=wn0+ηλen1+λ|en|κω,n.Note that the dimensions of kernelized input vectorκω,n and weight vector wn are augmented to make the update equation of weight vector valid.It should be pointed that the thresholdδκ is selected by grid search over the interval 0,1 to determine the sparsity of online dictionary Dn as in [1, 5–7].The KLLAD algorithm described by (21) and (22) combines the advantages of KLMS and KSEA algorithms leading to the improved convergence performance taking into account Gaussian noise and non-Gaussian impulsive noise existing simultaneously. When the design parameter λ is set to large value, the KLLAD algorithm can achieve the robust convergence performance against the impulsive noise. In addition, the KLLAD algorithm has faster convergence rate than the KSEA algorithm especially for highly correlated input signals. The scheme of KLLAD algorithm is summarized as Algorithm 1.Algorithm 1: KLLAD algorithm.
(1)
Input: xn,yn, n=1,2,…(2)
Initialization: select the step-size η>0, the Gaussian kernel bandwidth ξ>0, the threshold δκ>0, the parameter λ>0, the dictionary D0=x1, and weight coefficient w01=0.(3)
forn=1,2,…, do(4)
ifmaxm=1,2,…,Mnκxn,xω,m>δκ Update wn+1 via (21);(5)
elseifmaxm=1,2,…,Mnκxn,xω,m≤δκ Update wn+1 via (22);(6)
end if Obtain solution ψ¯xn=∑m=1Mnwnmκxn,xω,m.(7)
end forBy (9), the recursive update equation of KAF algorithms can be expressed in a general form of(23)wn+1=wn+ηfenenκω,n,where fen is defined as the nonlinear error function which performs like the generalized variable step-size against non-Gaussian impulsive noise. According to (23), various nonlinear KAF algorithms can be readily obtained by solving distinct cost functions in the RKHS. Hence, the error functions of some KAF algorithms mentioned previously are listed in Table 1.Table 1
Error functions of several KAF algorithms.
AlgorithmError functionfenKLMSfen=1KMCCfen=κyn,ϕxn=exp−en22/2ξf2KLMPfen=enp−2KLLADfen=λ/1+λ|en|It should be pointed out that the error function of MKRSL algorithm reduces to that of KMCC algorithm as the risk-sensitive parameterλf⟶0+ or that of KLMS algorithm as the kernel bandwidth ξf⟶∞ [18]. This means that the KMCC algorithm is a particular case of MKRSL algorithm in fact. Thus, we are only concerned with the KMCC algorithm with the characteristic of robustness for impulsive noises in the following. Correspondingly, the curves of error functions listed in Table 1 with different parameters are plotted in Figure 1. First of all, it can be observed from Figure 1(a) that the value of error function of KLMS algorithm is constant one without any ability of suppressing the impulsive noise due to its quadratic cost function for Gaussian noise. We can then see from Figures 1(b)–1(d) that the output values of nonlinear error functions of KMCC, KLMP, and KLLAD algorithms are effectively attenuated even for the very large instantaneous estimation errors caused by impulsive interferences. Specifically, the attenuation rate of error function of KMCC with small kernel bandwidth ξf is very fast, whereas its maximum output values corresponding to the relatively small error inputs particularly near zero are always less than 1. Consequently, the KMCC algorithm has to take much larger step-size η than the KLMP and KLLAD algorithms. Although both error functions have similar shape of output values for KLMP algorithm with small p and KLLAD algorithm with large λ, the error function of KLLAD consistently gives much smaller outputs for larger error inputs than that of KLMP.Figure 1
Curves of error functionsfen with en∈−10,10. (a) KLMS algorithm. (b) KMCC algorithm. (c) KLMP algorithm. (d) KLLAD algorithm.
(a)(b)(c)(d)


## 4. Convergence Condition on Mean Stability of the KLLAD Algorithm

In this section, we shall investigate the sufficient convergence condition of KLLAD algorithm via Gaussian kernel function with the fixed dictionary in the mean sense.For mathematical tractability of theoretical analysis, the CG modeled impulsive noisezn given in (1), instead of the standard symmetric α-stable distribution, has been successfully used in the theoretical analysis of robust adaptive filters [13, 14, 34].One the one hand, the coherence criterion guarantees the lengthMn is infinite [1]. On the other hand, it is true that the length Mn of online dictionary Dn gradually tends to be invariant in the steady-state phase. Therefore, we only need to consider the recursive update equation (21) of KLLAD algorithm with the steady fixed dictionary D with constant length M in the context of the derivation of convergence condition on mean stability.We start with introducing the weight error vector defined by(24)vn=wn−w⋆,where w⋆=w1⋆,w2⋆,…,wM⋆⊤ represents the optimal weight vector of vector-valued form of nonlinear system. It should be emphasized that w⋆ corresponds to the functional f⋆ based on the specific dictionary elements. From (1) and (24), the instantaneous estimation error (10) can be rewritten as(25)en=zn−vn⊤κ˜ω,n.With the kernelized input vector with fixed dictionaryD(26)κ˜ω,n=κxn,xω,1,κxn,xω,2,…,κxn,xω,M⊤.Subtractingw⋆ from both sides of (21), and using (25), we obtain(27)vn+1=vn+ηλzn−vn⊤κ˜ω,n1+λ|en|κ˜ω,n.Taking the expected values of both sides of (27), it follows that(28)Evn+1=Evn+ηλEznκ˜ω,n1+λen−ηλEκ˜ω,nκ˜ω,n⊤1+λenvn.Since the impulsive CG noisezn with zero-mean is assumed to be statistically independent of any other signals as mentioned above, and κ˜ω,nκ˜ω,n⊤ is assumed to be statistically independent of weight error vector vn, i.e., the modified independence assumption (MIA) widely used in [34, 41, 42], then (28) can be reformulated as(29)Evn+1=IM−ηλEκ˜ω,nκ˜ω,n⊤1+λ|en|Evn.For theoretical analysis tractability, the second term of (29) can be approximated as(30)Eκ˜ω,nκ˜ω,n⊤1+λ|en|≈Eκ˜ω,nκ˜ω,n⊤1+λEen.On the one hand, the numerator of (30) is the autocorrelation matrix of the kernelized input vector defined by(31)Rκκ=Eκ˜ω,nκ˜ω,n⊤∈ℝM×M.It has been determined in the theoretical analysis of KAF algorithms [34, 41]. The i,j-th element of matrix Rκκ can be computed by(32)Rκκij=IL+2ξ2Rxx−1/2×exp−14ξ22xω,i22+xω,j22−xω,i+xω,jIL+ξ2Rxx−1/2−12,with the input covariance matrix Rxx=Exnxn⊤∈ℝL×L. On the other hand, using (25) and the MIA assumption, the expression of MSE for KLLAD algorithm is given by(33)Een2≈σz2+trRκκVn,with the autocorrelation matrix of weight error vector Vn=Evnvn⊤∈ℝM×M. Although the recursion of Vn is not explicitly provided, it has trivial influence on the convergence condition on the mean stability of KLLAD algorithm as shown below. By the result of (4), (33) can be expressed as(34)Een2=1−prσv2+prK+1σv2+trRκκVn.The expectation in denominator of (30) can be roughly approximated as(35)Een≈1−prσv2+prK+1σv2+trRκκVn.(36)Evn+1=IM−ηλRκκ1+λ1−prσv2+prK+1σv2+trRκκVnEvn.Then, (29) can be further determined by (36), as shown at the top of next page. By (36), thus the sufficient convergence condition on mean stability is given by(37)0<η<2+2λ1−prσv2+prK+1σv2+trRκκVneigmaxRκκ.Thus, (37) can be rigorously reformulated as(38)0<η<2+2λσv1+prKeigmaxRκκ,where the calculation of autocorrelation matrix of weight error vector Vn is not used as explained before.


## 5. Simulation Results

In this section, we evaluated the performance of the proposed KLLAD algorithm in the context of impulsive noise by the simulation results. All the curves are obtained by averaging over 200 independent Monte Carlo trails.


### 5.1. Example I

In general, the optimal weight vectors of KAF algorithms, which correspond to the elements of online dictionary built by an online manner, are unavailable a priori. As a consequence, it is difficult to exhibit the convergence performance, particularly the mean-square-deviation (MSD). In order to obviously demonstrate the excellent mean and mean square convergence performance of KLLAD algorithm, the desired output of an ideal nonlinear synthesis system consisting of the optimum weight vector and the kernelized inputs is given by(39)yn=∑m=1Mwm⋆exp−xω,m−xn222ξ2+zn,where the kernel bandwidth is set to ξ=0.65, and zn is the non-Gaussian impulsive CG noise with parameters K=1×104, σv2=1, and pr=0.05.Furthermore, the CG noise model defined by (2) can be generalized into the mixture Gaussian noise model by changing the distribution of the random variable ϵn into the uniform, binary, Rayleigh, Laplacian distributions, etc. Then, we are able to thoroughly investigate the variations of nonlinear filtering performance of KLLAD algorithm in the presence of distinct impulsive noises with large outliers.Meanwhile, the preselected dictionary with 5 elements utilized for the kernelized nonlinear mapping is given by(40)D=xω,1,xω,2,xω,3,xω,4,xω,5=0.721.44,3.311.28,−3.03−2.75,1.48−1.66,−1.28−0.32,which is generated by the coherence criterion from the input signals a priori. Correspondingly, the learning curve of the MSD is defined as(41)MSDn=1T∑t=1Twt,n−w⋆22,where T is the total number of Monte Carlo runs. The input signal was assumed to be a sequence of statistically independent vectors xn=x1,n,x2,n⊤ with correlated samples satisfying x1,n=0.5x2,n+τx,n, where x2,n is a white Gaussian noise sequence with variance σx22=1 and τx,n is a white Gaussian so that x1,n has variance σx12=1. The optimum weight vector w⋆ was supposed to be abruptly changed from 0.3,0.1,−0.15,−0.3,−0.1⊤ to 0.05,0.25,−0.05,−0.2,−0.35⊤ at time instant n=2×104.The set of parameters of all used algorithms for Example I is listed in Table2. It should be pointed that the step-size of KLLAD algorithm is selected by satisfying the convergence condition presented by (38).Table 2
Parameter settings of used algorithms for Example I.
AlgorithmParametersKLMSξ=0.65; η=0.05;KMCCξ=0.65; η=3; ξf=0.05;KLMPξ=0.65; η=0.05; p=1.1KLLADξ=0.65; η=0.08; λ=15The convergence curves of weight coefficients obtained by implementing KLMS, KMCC, KLMP, and KLLAD algorithms are illustrated in Figures2–5, respectively. As illustrated in Figure 2, the averaged curves of weight coefficients of KLMS algorithm are not able to tend to the optimal weight coefficients during the two stages due to the interference effect of impulsive CG noise. In contrast, the averaged curves of weight coefficients of KMCC, KLMP, and KLLAD algorithms are smooth and all converge to two optimal weight coefficients within the two stages, as shown in Figures 3–5. In addition, the effectiveness of convergence condition on the mean stability (38) is validated by Figure 5. More importantly, Figures 6(a)–6(f) show that the filtering performance of KLLAD algorithm consistently outperforms those of KLMS, KMCC, and KLMP algorithms in terms of robustness, convergence rate, and accuracy of the learning curves of MSD during the two stages using the mixture Gaussian noise model based on the normal, uniform, binary, Rayleigh, Laplacian, and symmetric α-stable distributions, respectively. Therefore, the robust performance of KLLAD algorithm is validated by the simulation results of nonstationary nonlinear system identification in the presence of non-Gaussian impulsive noise.Figure 2
Convergence of the coefficientswnm for KLMS algorithm (the dotted lines are for the optimal weight coefficients, and solid lines are for weight coefficients of KLMS algorithm).Figure 3
Convergence of the coefficientswnm for KMCC algorithm (the dotted lines are for the optimal weight coefficients, and solid lines are for weight coefficients of KMCC algorithm).Figure 4
Convergence of the coefficientswnm for KLMP algorithm (the dotted lines are for the optimal weight coefficients, and solid lines are for weight coefficients of KLMP algorithm).Figure 5
Convergence of the coefficientswnm for KLLAD algorithm (the dotted lines are for the optimal weight coefficients, and solid lines are for weight coefficients of KLLAD algorithm).Figure 6
Comparisons of learning curves of MSD for KLMS, KMCC, KLMP, and KLLAD algorithms with different non-Gaussian distribution noises. (a) Normal distribution noise. (b) Uniform distribution noise with−1,1. (c) Binary distribution noise with −1,1. (d) Rayleigh distribution noise with σ2=4. (e) Laplacian distribution noise with μ=0 and σ2=1. (f) Symmetric α-stable distribution noise with α=0.9, γ=0.5, and δ=0.
(a)(b)(c)(d)(e)(f)


### 5.2. Example II

As the second example, we consider the input random sequence generated from the following relation:(42)un=ρun−1+σu1−ρ2ζn,where ζn is a random noise following the i.i.d. standard normal distribution. Here, the correlation factor ρ and the standard deviation σu of random sequence un were all chosen as 0.5. The desired output of nonlinear system is generated as follows:(43)tn=0.5un−0.3un−1,yn=tn−1.25tn2+0.25tn3+zn,where zn is the non-Gaussian impulsive CG noise with parameters K=1×104, σv2=1, and pr=0.1. At each time instant n, the input vectors xn=un,un−1⊤ are used to estimate the nonlinear desired response signal yn contaminated by the impulsive noise. In order to clearly investigate the characteristic of convergence of weight coefficients, the 6 elements of dictionary were chosen by the coherence criterion from the input vectors a priori as follows:(44)D=xω,1,xω,2,xω,3,xω,4,xω,5,xω,6=0.17−1.92,−1.62−0.18,0.521.55,2.901.92,−2.0−2.47,2.65−0.82.The set of parameters of used algorithms for Example II is listed in Table3. Likewise, the step-size of KLLAD algorithm in second example is also determined from the range of convergence condition according to (38).Table 3
Parameter settings of used algorithms for Example II.
AlgorithmParametersKMCCξ=0.75; η=1; ξf=0.5;KLMPξ=0.75; η=0.1; p=0.7KLLADξ=0.75; η=0.1; λ=15Although the fixed dictionary with 6 elements is used to compare the differences of convergent performance among used algorithms, the corresponding optimal weight coefficients are still unknown in fact. The KLMS algorithm is ignored due to its severe instability of convergence. Figure7 shows that the mean value curves of weight coefficients of KLLAD algorithm converge much smoother and faster compared with the KLMP and KMCC algorithms, which verified the superiorities of KLLAD algorithm.Figure 7
Convergence of the coefficientswnm for KMCC, KLMP, and KLLAD algorithms.


## 5.1. Example I

In general, the optimal weight vectors of KAF algorithms, which correspond to the elements of online dictionary built by an online manner, are unavailable a priori. As a consequence, it is difficult to exhibit the convergence performance, particularly the mean-square-deviation (MSD). In order to obviously demonstrate the excellent mean and mean square convergence performance of KLLAD algorithm, the desired output of an ideal nonlinear synthesis system consisting of the optimum weight vector and the kernelized inputs is given by(39)yn=∑m=1Mwm⋆exp−xω,m−xn222ξ2+zn,where the kernel bandwidth is set to ξ=0.65, and zn is the non-Gaussian impulsive CG noise with parameters K=1×104, σv2=1, and pr=0.05.Furthermore, the CG noise model defined by (2) can be generalized into the mixture Gaussian noise model by changing the distribution of the random variable ϵn into the uniform, binary, Rayleigh, Laplacian distributions, etc. Then, we are able to thoroughly investigate the variations of nonlinear filtering performance of KLLAD algorithm in the presence of distinct impulsive noises with large outliers.Meanwhile, the preselected dictionary with 5 elements utilized for the kernelized nonlinear mapping is given by(40)D=xω,1,xω,2,xω,3,xω,4,xω,5=0.721.44,3.311.28,−3.03−2.75,1.48−1.66,−1.28−0.32,which is generated by the coherence criterion from the input signals a priori. Correspondingly, the learning curve of the MSD is defined as(41)MSDn=1T∑t=1Twt,n−w⋆22,where T is the total number of Monte Carlo runs. The input signal was assumed to be a sequence of statistically independent vectors xn=x1,n,x2,n⊤ with correlated samples satisfying x1,n=0.5x2,n+τx,n, where x2,n is a white Gaussian noise sequence with variance σx22=1 and τx,n is a white Gaussian so that x1,n has variance σx12=1. The optimum weight vector w⋆ was supposed to be abruptly changed from 0.3,0.1,−0.15,−0.3,−0.1⊤ to 0.05,0.25,−0.05,−0.2,−0.35⊤ at time instant n=2×104.The set of parameters of all used algorithms for Example I is listed in Table2. It should be pointed that the step-size of KLLAD algorithm is selected by satisfying the convergence condition presented by (38).Table 2
Parameter settings of used algorithms for Example I.
AlgorithmParametersKLMSξ=0.65; η=0.05;KMCCξ=0.65; η=3; ξf=0.05;KLMPξ=0.65; η=0.05; p=1.1KLLADξ=0.65; η=0.08; λ=15The convergence curves of weight coefficients obtained by implementing KLMS, KMCC, KLMP, and KLLAD algorithms are illustrated in Figures2–5, respectively. As illustrated in Figure 2, the averaged curves of weight coefficients of KLMS algorithm are not able to tend to the optimal weight coefficients during the two stages due to the interference effect of impulsive CG noise. In contrast, the averaged curves of weight coefficients of KMCC, KLMP, and KLLAD algorithms are smooth and all converge to two optimal weight coefficients within the two stages, as shown in Figures 3–5. In addition, the effectiveness of convergence condition on the mean stability (38) is validated by Figure 5. More importantly, Figures 6(a)–6(f) show that the filtering performance of KLLAD algorithm consistently outperforms those of KLMS, KMCC, and KLMP algorithms in terms of robustness, convergence rate, and accuracy of the learning curves of MSD during the two stages using the mixture Gaussian noise model based on the normal, uniform, binary, Rayleigh, Laplacian, and symmetric α-stable distributions, respectively. Therefore, the robust performance of KLLAD algorithm is validated by the simulation results of nonstationary nonlinear system identification in the presence of non-Gaussian impulsive noise.Figure 2
Convergence of the coefficientswnm for KLMS algorithm (the dotted lines are for the optimal weight coefficients, and solid lines are for weight coefficients of KLMS algorithm).Figure 3
Convergence of the coefficientswnm for KMCC algorithm (the dotted lines are for the optimal weight coefficients, and solid lines are for weight coefficients of KMCC algorithm).Figure 4
Convergence of the coefficientswnm for KLMP algorithm (the dotted lines are for the optimal weight coefficients, and solid lines are for weight coefficients of KLMP algorithm).Figure 5
Convergence of the coefficientswnm for KLLAD algorithm (the dotted lines are for the optimal weight coefficients, and solid lines are for weight coefficients of KLLAD algorithm).Figure 6
Comparisons of learning curves of MSD for KLMS, KMCC, KLMP, and KLLAD algorithms with different non-Gaussian distribution noises. (a) Normal distribution noise. (b) Uniform distribution noise with−1,1. (c) Binary distribution noise with −1,1. (d) Rayleigh distribution noise with σ2=4. (e) Laplacian distribution noise with μ=0 and σ2=1. (f) Symmetric α-stable distribution noise with α=0.9, γ=0.5, and δ=0.
(a)(b)(c)(d)(e)(f)


## 5.2. Example II

As the second example, we consider the input random sequence generated from the following relation:(42)un=ρun−1+σu1−ρ2ζn,where ζn is a random noise following the i.i.d. standard normal distribution. Here, the correlation factor ρ and the standard deviation σu of random sequence un were all chosen as 0.5. The desired output of nonlinear system is generated as follows:(43)tn=0.5un−0.3un−1,yn=tn−1.25tn2+0.25tn3+zn,where zn is the non-Gaussian impulsive CG noise with parameters K=1×104, σv2=1, and pr=0.1. At each time instant n, the input vectors xn=un,un−1⊤ are used to estimate the nonlinear desired response signal yn contaminated by the impulsive noise. In order to clearly investigate the characteristic of convergence of weight coefficients, the 6 elements of dictionary were chosen by the coherence criterion from the input vectors a priori as follows:(44)D=xω,1,xω,2,xω,3,xω,4,xω,5,xω,6=0.17−1.92,−1.62−0.18,0.521.55,2.901.92,−2.0−2.47,2.65−0.82.The set of parameters of used algorithms for Example II is listed in Table3. Likewise, the step-size of KLLAD algorithm in second example is also determined from the range of convergence condition according to (38).Table 3
Parameter settings of used algorithms for Example II.
AlgorithmParametersKMCCξ=0.75; η=1; ξf=0.5;KLMPξ=0.75; η=0.1; p=0.7KLLADξ=0.75; η=0.1; λ=15Although the fixed dictionary with 6 elements is used to compare the differences of convergent performance among used algorithms, the corresponding optimal weight coefficients are still unknown in fact. The KLMS algorithm is ignored due to its severe instability of convergence. Figure7 shows that the mean value curves of weight coefficients of KLLAD algorithm converge much smoother and faster compared with the KLMP and KMCC algorithms, which verified the superiorities of KLLAD algorithm.Figure 7
Convergence of the coefficientswnm for KMCC, KLMP, and KLLAD algorithms.


## 6. Conclusion

In this paper, we presented a novel KLLAD algorithm based on the logarithmic error cost criterion under the assumption of non-Gaussian impulsive CG noise. The KLLAD algorithm can effectively mitigate the instability of convergence learning curves caused by impulsive noise. The simulation results demonstrated that the proposed KLLAD algorithm has an excellent performance compared to the KLMS, KMCC, and KLMP algorithms in the presence of impulsive noise.



---

*Source: 9092663-2022-03-08.xml*