# Statistical Tracking Behavior Analysis for the Affine Projection Algorithm Based on Direction Error

Authors: Yongfeng Zhi; Wenyan Guo
Journal: Wireless Communications and Mobile Computing
 (2021)

---

## Abstract

Under the condition that the step size is less than one, a statistical tracking behavior analysis for the affine projection algorithm based on direction error is discussed. When the unknown true weight vector is modeled by the stochastic walk model, the mean weight error is derived under the four assumptions based on the deterministic recursive equation. Furthermore, the statistical tracking behavior of the steady state is analyzed for the affine projection algorithm based on direction error. Simulation analysis is shown to suppniort the mathematical results.


---

## Body

## 1. Introduction

Since the normalized least mean square Wen algorithm is computational simplicity, this algorithm is widely put into use by the adaptation algorithm. And the normalized least mean square algorithm is also robust to the length of the finite word effects and ease of implementation in the signal processing. However, the highly colored input signals will cause the normalized least mean square algorithm converge slowly [1]. Compared to the normalized least mean square algorithm, the affine projection (AP) algorithm is a better alternative. The AP algorithm was firstly given by Ozeki and Umeda, and it improves the convergence speed by reusing the input signal [2]. Based on the idea that the successive vectors of the input signal are orthogonal with each other, the best improvement convergence will be obtained; the normalized least mean square based on orthogonal correction factors (NLMS-OCF) was shown in [3]. When we set the step size to be equal one, according to the input direction vectors of the input signal, a novel definition was given for the affine projection algorithm [4]. A pseudo-AP (PAP) algorithm is a simplified definition form to the affine projection algorithm, and this algorithm is ease of implementation [5]. Since the weights update direction and the direction that is caused by the adaptive error are not the same for the PAP algorithm, an AP algorithm based on direction error (AP-DE) was given to resolve the nonconformity problem [6]. Fast AP algorithms have been given as well [7]. These algorithms, including the AP, PAP, and NLMS-OCF, can be considered a class of the AP algorithms, which updates the adaptive weights according to the multiple vectors of the input signal. Some recent studies are as follows. A new class of AP algorithm is proposed based on the high-order error power criterion [8] and can achieve reliable performance under Gaussian interference. The effect of the noise vector on the weight-error vector is considered. The edified filtered-x affine projection algorithm is effective for active noise control owing to its good convergence behavior and medium computational burden [9]. The proposed method of sparsity aware affine-projection-like robust set membership M-estimate (SAPL-RSM) filtering has been utilized for alleviating the impact of impulsive noise on the adaptation of feedback canceler’s weights [10]. A time-varying parameter PAPLM (TV-PAPLM) algorithm is proposed, which uses a modified exponential function to adjust the time-varying parameter according to the ratio of the mean square score function to the system noise variance [11].There are lots of work that has been done to analyze the statistical convergence and tracking behavior of the AP and NLMS-OCF algorithms. Based on the assumption that the input signal is the identically and independent distributed form, the statistical analysis of the class of the AP algorithms was shown, in which the mean weight error (MWE) and the mean-square error (MSE) were shown to study the convergence behavior for the NLMS-OCF algorithm [12, 13]. When we set the step size to be one, the closed-form expression is given for the convergence behavior of the MWE and MSE, appropriate for autoregressive-moving average (ARMA) input signal models of the AP algorithm, as in [14]. In the adaptive direction of the weight update, when we set each weight error to be zero, the optimal step size for the PAP algorithm was given; then using this model, the MWE and MSE were derived based on the deterministic recursive equations [15]. The statistical analysis of convergence model for the AP algorithm has been shown by using the (AR) input model [16–18]. Under the assumption given by [12, 13], the convergence model was shown for the kind of AP-DE algorithm in [19]. When the step size was equal to one, the statistical tracking behavior was given for the AP-DE algorithm [20], in which the MWE and MSE behaviors are analyzed.In this paper, four assumptions are made for the input vectors and the direction vectors. When the unknown true weight vector is modeled by the stochastic walk model, the tracking MWE and MSE are obtained for the AP-DE algorithm. At last, the simulation analysis is given to show the proposed statistical tracking behavior.


## 2. AP-DE Algorithm

An affine projection algorithm with direction error (AP-DE) is presented to solve the nonconformity between the iterated direction of the adaptive filter and the direction caused by the iteration error. In the adaptive filtering system identification model, the colored input signal is transformed into the successive input vectorxn, and then, we have obtained
(1)xn=xnxn−1⋯xn−N+1T.According to the most recent successivem input vectors for the input signal, we can obtain the input matrix Xn−1,m as
(2)Xn−1,m=xn−1xn−2⋯xn−m.The AP-DE algorithm updates of the weight vector are implemented by the adaptive filter [6], as follows:
(3a)en=dn−∑k=1ma^n,kdn−k−wnHφn,(3b)wn+1=wn+μen∗φnHφnφn,where H denotes a transposed matrix or vector and the direction vector φn of the input signal is estimated based on
(3c)φn=xn−Xn−1,mα^n.And from the least-squares formulation, the vectorα^n=a∧n,1a∧n,2⋯a∧n,mT is found as
(3d)α^n=Xn−1,mHXn−1,m−1Xn−1,mHxn.The iterated direction of AP is the input vectorxn, which causes the iteration error. Compared with AP algorithms, if the measurement noise is absent, the iteration error of the AP-DE algorithm is caused only by the direction vector φn, which is also the iterated direction of the adaptive filter. Thus, the AP-DE algorithm improves the convergence rate compared with the PAP and AP algorithms.


## 3. Statistical Properties of the Direction and Input Vectors

In order to study the tracking performance of the AP-DE algorithm, the four assumptions are as follows:(A1)
The successive input vectorxn is zero mean, identically and independent distributed, and then the covariance matrix can be obtained as [12, 13](4)R=ExnxnH=VΛVH,where Λ=diagλ1λ2⋯λN are the eigenvalues of R and the corresponding eigenvectors V=υ1υ2⋯υN are orthonormal to each other, i.e., VHV=Ι(A2)
Using assumption A1 and assuming three different independent stochastic variables constitute the direction vectorxn. That is,(5)xn=snrnvn,(6)Psn=±1=12,rn~xnPvn=υi=λitrR,1≤i≤N,where the matrix trace is denoted by the expression tr•, rn~xn means that the stochastic variable rn, and the input vectors have the same distribution. Similar assumption has also been given in [12, 13]
(A3)
Based on assumption A2, make the following assumption. The direction vectorsφn of the successive input signal are independent on the weight vector wn, and they are the Gaussian stochastic vectors with variance σφ2 and zero mean. Therefore, the direction vectors are also three independent stochastic variables [20], i.e.,(7)φn=s^nr^nv^n,(8)Ps^n=±1=12,r^n~φnPv^n=υi=1N,1≤i≤N,where the stochastic variable r^n~φn means that r^n has the same distribution as the direction vectors of the input signal, and then, the square of the stochastic variable is denoted by r∧2=Nσφ2(A4)
Based on assumption A3, make assumption about the vectorωn. The desire output signal dn is given by the mathematical model as follows:(9)dn=wnoHxn+εn,where the measurement noise εn is identically distributed, independent, and stationary with variance σε2 and zero mean and the time-variant vectors wno are the system model parameters. The unknown weight vector wno is modeled by the stochastic walk model given,
(10)wno=wn−1o+ωn−1,where the vector ωn−1 is an independent, stationary zero mean vector process with the variance σω2 and it is also the three independent stochastic variables that are identically and independent distributed. That is,
(11)ωn=s~nr~nv~n,(12)Ps~n=±1=12,r~n~ωn,Pv~n=υi=1N,1≤i≤N,where the independent stochastic variable r~n~ωn means that it has the same distribution as the vector ωn and the square of the independent stochastic variable is denoted by r~2=Nσω2Therefore, based on (3a) and (9), the iteration error en of the adaptive filtering can be obtained as
(13)en=wnoHxn−∑k=1ma^n,kwn−koHxn−k−wnHφn+εn−∑k=1ma^n,kεn−k.From (10) and (13), we have
(14)en=wnoHxn−∑k=1ma^n,kwnoHxn−k+∑k=1ma^n,k∑j=1kωn−jHxn−k−wnHφn+εn−∑k=1ma^n,kεn−k.From (3c) and (14), it yields
(15)en=w~nHφn+∑k=1ma^n,k∑j=1kωn−jHxn−k+εn−∑k=1ma^n,kεn−k,where
(16)w~n=wno−wn.


## 4. Behavior of Mean Weight Error

In order to study the tracking performance of (3b), according to (15), we have
(17)wn+1=wn+μφnφnHφnHφnw~n+μφnφnHφn∑k=1ma^n,k∗xn−kH∑j=1kωn−j+μφnφnHφnεn∗−∑k=1ma^n,k∗εn−k∗.Combining (10), (16), and (17), the adaptation equation is shown as
(18)w~n+1=I−μφnφnHφnHφnw~n−μφnφnHφn∑k=1ma^n,k∗xn−kH∑j=1kωn−j−μφnφnHφnεn∗−∑k=1ma^n,k∗εn−k∗+ωn.Since the parameterεn and the stochastic vector ωn are both white noise with zero mean, we can take expectation on both sides of (18), and the last three terms in (18) becomes zero, so they can be obtained as
(19)Ew~n+1=EI−μφnφnHφnHφnw~n.Based on the vectorsυ1,υ2,⋯,υN which are orthonormal to each other, the representation of Ew~n can be defined as the vector ρn. That is,
(20)ρn≡VHEw~n.Therefore,(21)ρn,i=υiHEw~n=EυiHw~n.Using this result, according to (7), premultiplication on both sides of (19) by υiH, we have
(22)ρn+1,i=EυiHI−μv^nv^nHw~n.For the orthonormality of the vectorυi. That is,
(23)υiHυjυjH=υiH,ifi=j,0,ifi≠j.Based on (23), (22) becomes
(24)ρn+1,i=1−μNρn,i.


## 5. Statistical MSE Behavior

Under assumption A3 where the direction vectorsφn of the input signal are independent on the vector wn, according to (18), the covariance of the weight error is proposed as
(25)covw~n+1=EI−μφnφnHφnHφncovw~nI−μφnφnHφnHφn−μEI−μφnφnHφnHφnw~n∑k=1ma^n,k∑j=1kωn−jHxn−kφnHφnHφn−μEφnφnHφn∑k=1ma^n,k∗xn−kH∑j=1kωn−jw~nHI−μφnφnHφnHφn−EI−μφnφnHφnHφnw~nεn−∑k=1ma^n,kεn−kμφnHφnHφn−ωnH−Eεn∗−∑k=1ma^n,k∗εn−k∗μφnφnHφn−ωnw~nHI−μφnφnHφnHφn+μ2EφnφnHφn∑k=1ma^n,k∗xn−kH∑j=1kωn−j∑k=1ma^n,k∑j=1kωn−jHxn−kφnHφnHφn+μEφnφnHφn∑k=1ma^n,k∗xn−kH∑j=1kωn−jεn−∑k=1ma^n,kεn−kμφnHφnHφn−ωnH+μEεn∗−∑k=1ma^n,k∗εn−k∗μφnφnHφn−ωn∑k=1ma^n,k∑j=1kωn−jHxn−kφnHφnHφn+Eεn∗−∑k=1ma^n,k∗εn−k∗μφnφnHφn−ωnεn−∑k=1ma^n,kεn−kμφnHφnHφn−ωnH.Under the assumption thatεn is white noise with zero mean and the vector ωn is an independent, stationary, zero mean white noise vector, so the six different polynomials, including the second, third, fourth, fifth, seventh, and eighth items, all turn into zero. Because the direction vector φn of the input signal is independent of the vector wn, so we obtain the following:
(26)covw~n+1=EI−μφnφnHφnHφncovw~nI−μφnφnHφnHφn+μ2tr∑k=1mEa^n,ka^n,k∗Exn−kxn−kHE∑j=1kωn−jωn−jHEφnφnHφnHφn2+μ2Eεn−∑k=1ma∧n,kεn−k∗εn−∑k=1ma^n,kεn−kEφnφnHφnHφn2+EωnωnH.Sinceεn is identically distributed, independent with variance σε2 and zero mean, under assumptions A2, A3, and A4, using (5), (7), and (11), (26) becomes as follows:
(27)covw~n+1=EI−μv^nv^nHcovw~nI−μv^nv^nH+μ2rn2σω2r^n2∑k=1mkEa^n,ka^n,k∗Ev^nv^nH+μ2σε2r^n21+∑k=1mEa^n,ka^n,k∗Ev^nv^nH+r~n2Ev~nv~nH.For the covariance matrix, we define the diagonal elements asλ~n+1,i. So we have
(28)λ~n+1,i≡VHcovw~n+1Vii≡υiHcovw~n+1υi.We premultiply and postmultiply on both sides of (27) by υiH and υi, respectively; under assumption A3, using (29), we can obtain
(29)EυiHI−μv^nv^nHAI−μv^nv^nHυi=1+μ2−2μNυiHAυi,where A is a N×N dimension matrix; we can obtain
(30)λ~n+1,i=1+μ2−2μNλ~n,i+μ2rn2σω2r^n2∑k=1mkEa^n,ka^n,k∗EυiHv^nv^nHυi+μ2σε2r^n21+∑k=1mEa^n,ka^n,k∗EυiHv^nv^nHυi+r~n2EυiHv~nv~nHυi.Based on assumptions A3 and A4, we can obtain that the vectorsv^n and v~n both have the probability of 1/N to be υi, respectively. From (30), we can obtain
(31)λ~n+1,i=1+μ2−2μNλ~n,i+μ2rn2σω2Nr^n2∑k=1mkEa^n,ka^n,k∗+μ2σε2Nr^n21+∑k=1mEa^n,ka^n,k∗+r~n2N.According to (15), we have the statistical MSE of tracking performance for the AP-DE algorithm as
(32)Eenen∗=Ew~nHφnφnHw~n+Ew~nHφn∑k=1ma^n,k∗xn−kH∑j=1kωn−j+E∑k=1ma^n,k∑j=1kωn−jHxn−kφnHw~n+Ew~nHφnεn∗−∑k=1ma^n,k∗εn−k∗+Eεn−∑k=1ma^n,kεn−kφnHw~n+E∑k=1ma^n,k∑j=1kωn−jHxn−k∑k=1ma^n,k∗xn−kH∑j=1kωn−j+E∑k=1ma^n,k∑j=1kωn−jHxn−kεn∗−∑k=1ma^n,k∗εn−k∗+Eεn−∑k=1ma^n,kεn−k∑k=1ma^n,k∗xn−kH∑j=1kωn−j+Eεn−∑k=1ma^n,kεn−kεn∗−∑k=1ma^n,k∗εn−k∗.Forεn and ωn which are independent, stationary white noise vectors with zero mean, so the second, third, fourth, fifth, seventh, and eighth terms all become zero. According to assumption A3 that the direction vector φn is independent on the weight vector wn, we can obtain
(33)Eenen∗=trEw~nw~nHEφnφnH+∑k=1mEa^n,ka^n,k∗trExn−kxn−kH∑j=1kEωn−jωn−jH+σε21+∑k=1mEa^n,ka^n,k∗.Under assumptions A3 and A4, based on the definition ofλ~n,i, from (33), the statistical MSE of the tracking performance for the AP-DE algorithm can be rewritten as
(34)Eenen∗=σφ2∑i=1Nλ~n,i+rn2σω2∑k=1mkEa^n,ka^n,k∗+σε21+∑k=1mEa^n,ka^n,k∗.


## 6. Steady-State Behavior

Assuming convergence, asn⟶∞, (31) can be transformed into
(35)λ~∞,i=12μ−μ2μ2rn2σω2r^n2∑k=1mkEa^n,ka^n,k∗+μ2σε2r^n21+∑k=1mEa^n,ka^n,k∗+r~n2.Combining with (34) and (35), based on r∧2=Nσφ2, the mean square error of the steady-state tracking performance can be written as
(36)Eenen∗=12μ−μ2μ2rn2σω2∑k=1mkEa^n,ka^n,k∗+μ2σε21+∑k=1mEa^n,ka^n,k∗+r^n2r~n2+rn2σω2∑k=1mkEa^n,ka^n,k∗+σε21+∑k=1mEa^n,ka^n,k∗.


## 7. Simulation

We give the statistical MSE of the tracking performance learning curves from the simulations in this section, and the derived models are given by (34) and (36). The initial system true weight w0o is stochastic produced by 32 taps. The length of the adaptive filtering is given with the same as the derived model. The weight of the initial estimated values is given as w0=0. The 100 independent simulations are averaged for each experiment, and each step size is equal to 0.1. The variance of the measurement noise and the vector ωn are set to be σε2=0.01 and 2.5×10−5, respectively.Case 1.
Consider that the input signal model defined byxn=−0.95xn−1+zn is an AR (1) model and the parameter zn is the white Gaussian noise with the zero mean. The parameter m is set to be one. The statistical MSE for tracking behavior predicted by the model given by (34) and (36) are shown in Figure 1, together with the simulation results. We observe that the derived models are almost the same simulation results in this case.Case 2.
When the parameterzn is assumed to be the white Gaussian noise with the zero mean, the input signal is an ARMA (2, 1) model defined byxn=0.7−0.3ixn−1−0.4−0.6ixn−2+zn+0.5zn−1. The parameter is set to be m=3. The mean square error of the tracking behavior predicted by the derived model given with (34) and (36) is shown in Figure 2. We find that the derived model cooperates with the simulation results well for the input ARMA (2, 1) model.Figure 1
Learning MSE curves of the statistical tracking performance for AR (1) input.Figure 2
Learning MSE curves of the statistical tracking performance for ARMA (2, 1) input.


## 8. Conclusion

Under the condition that the unknown true weight vector is given by the stochastic walk model, a statistical tracking model for the AP-DE algorithm is analyzed. We give four assumptions that show the properties for both the input and direction vectors. A statistical tracking model of the MWE and the MSE is derived based on these four assumptions. A prediction tracking model of the statistical steady-state MSE of the AP-DE algorithm is also proposed. According to the derived models in this paper, the simulation results for the input given by AR (1) and ARMA (2, 1) show the affine projection algorithm based on direction error, the better statistical tracking behavior.



---

*Source: 5536455-2021-06-08.xml*