# Affine Projection Algorithm Using Regressive Estimated Error

Authors: Shu Zhang; Yongfeng Zhi
Journal: ISRN Signal Processing
 (2011)

---

## Abstract

An affine projection algorithm using regressive estimated error (APA-REE) is presented in this paper. By redefining the iterated error of the affine projection algorithm (APA), a new algorithm is obtained, and it improves the adaptive filtering convergence rate. We analyze the iterated error signal and the stability for the APA-REE algorithm. The steady-state weights of the APA-REE algorithm are proved to be unbiased and consist. The simulation results show that the proposed algorithm has a fast convergence rate compared with the APA algorithm.


---

## Body

## 1. Introduction

Adaptive filtering techniques are widely used in noise and echo cancellation, equalization, and beamforming. The normalized least mean square (NLMS) algorithm is a widely used adaptation algorithm due to its computational simplicity and ease of implementation [1]. Unfortunately, for highly colored input signals—with a covariance matrix that exhibits a large dynamic range of eigenvalues—this algorithm deteriorates its convergence rate. To overcome this problem, Ozeki and Umeda [2] discovered the affine projection algorithm (APA) using affine subspace projections. Rupp [3], based on the direction vector, gave a definition for the APA algorithm, and it shows the desirable decorrelation properties. Sankaran and Beex [4], based on the idea that the best improvement in weights occurred while the successive input vectors were orthogonal to each other, proposed the NLMS with orthogonal correction factors (NLMS-OCF). The entire class of algorithms can be referred to as the APA algorithms. In this paper, we will use the formulation by [3] to derive a new adaptive filtering algorithm using regressive estimated error and analyze its convergence rate.The iterated error of the APA algorithm, which is the same with the estimated output error, is not only caused by the iterated direction, but also caused by the other directions that do not contribute to the iterated direction. In this paper, the regressive estimated error is used as the iterated error to solve this problem, and then we obtain the affine projection algorithm using regressive estimated error (APA-REE). Under the measurement noise-free condition, the iterated error, which is different from the estimated output error, is almost only caused by the iterated direction. The stability for the APA-REE algorithm is also analyzed, and the steady-state weights are proved to be unbiased and consist. The simulation results show that the proposed algorithm has a fast convergence rate compared with the APA algorithm.


## 2. APA-REE Algorithm

Figure1 shows an adaptive filter used in the system identification mode, the wide-sense stationary input process {xn}, which is zero mean, and the corresponding measured output {dn}, possibly contaminated with the measurement noise {εn} that is zero-mean white noise, is measurable. The input process is converted into input vectors {xn}, via a tapped delay line (TDL), and is defined as(1)xn=[xnxn-1⋯xn-N+1]T.
The objective is to estimate an N-dimensional weight vector {wn} using the most recent (m+1) input vectors available at the nth instant; the latter define the input matrix {Xn,m+1} as follows(2)Xn,m+1=[xnxn-1⋯xn-m].
The estimated output {d̂n} is obtained according to(3)d̂n=wnHxn.
The weight vector {wn} is adjusted so that the estimated output {d̂n} is as close as possible to the measured output {dn} in the mean square error (MSE) sense. The APA-REE algorithm update of the weight vector {wn} is given as follows:(4a)ẽn=dn-d̂n,(4b)en=ẽn-∑i=1mân,iẽn-i,(4c)wn+1=wn+μen*φnHφnφn,
where the notations (·)* and (·)H are adopted to denote the complex conjugate for the scalar and the Hermitian transpose of a vector or matrix, respectively. ẽn denotes the estimated output error, and en, used as the iterated error of the APA-REE algorithm, is the regressive estimated error. The direction vector {φn} is evaluated according to (4d)φn=xn-Xn-1,mα̂n=xn-∑i=1mân,ixn-i.
The vector α̂n=[ân,1ân,2⋯ân,m]T, which is also used in (4b), is found according to the following least squares formulation:(4e)α̂n=[Xn-1,mHXn-1,m]-1Xn-1,mHxn.Figure 1
Adaptive system identification scenario.When the iterated error of formulation (4c) is directly driven by the estimated output error, this algorithm is called APA algorithm, which is firstly introduced by [2]. But the iterated error of the APA-REE algorithm is driven by the regressive estimated error, and the latter will certify that the iterated error is almost only caused by the iterated direction. From (4c), we can also find that the iterated direction of the APA-REE algorithm is the direction vector {φn}.


## 3. Regressive Estimated Error Analysis

In order to make the analysis becoming tractable, a true adaptive filter weight vectorw0 of dimension N is supposed to exist such that the estimated weight error(5)w̃n=w0-wn
and the corresponding measured output signal(6)dn=w0Hxn+εn,
where {εn} denotes the measurement noise, which is zero mean white noise. Using (4a) into (4b), we obtain(7)en=(dn-∑i=1mân,idn-i)-(d̂n-∑i=1mân,id̂n-i),
where ân,i, 1≤i≤m is an element of the vector α̂n. Using (6) and (3) into (7) yields(8)en=(w0Hxn-∑i=1mân,iw0Hxn-i)-(wnHxn-∑i=1mân,iwn-iHxn-i)+(εn-∑i=1mân,iεn-i).
The estimated weight vector changes inapparently during the most recent past m steps, especially for steady-state. So we can consider that the estimated weight vectors wn≈wn-i, 1≤i≤m, and then, from (8), we obtain(9)en=w0H(xn-∑i=1mân,ixn-i)-wnH(xn-∑i=1mân,ixn-i)+(εn-∑i=1mân,iεn-i).
Combining (4d) and (9) yields(10)en=w0Hφn-wnHφn+(εn-∑i=1mân,iεn-i).
Based on (5) and (10), it follows that(11)en=w̃nHφn+(εn-∑i=1mân,iεn-i).
From (11), we observe that, if the measurement noise is assumed to be absent, the iterated error of the APA-REE algorithm is only caused by the direction vector, which is the same as the iterated direction.Now we consider the APA algorithm. Under the assumption that there exists a true adaptive filter weight vectorw0 of dimension N, based on (5) and [5], the iterated error of the APA algorithm can be written as(12)ên=w̃nHxn+εn.
Under the measurement noise-free condition, from (4c) and [3], we can find that both of the iterated directions of the APA-REE and APA algorithms are the direction vector {φn}. Based on (11), it can be concluded that the iterated error {en} of the APA-REE algorithm is only caused by the direction {φn}, which is the same as the iterated direction {φn}. But from (12), we can find that the iterated error {ên} of the APA algorithm is caused by the direction {xn}, which is different from the iterated direction {φn}. It shows that the iterated error {ên} of the APA algorithm is not only caused by the iterated direction, but also caused by the other directions that do not contribute the iterated direction. So the APA-REE algorithm should have a faster convergence rate compared with the APA algorithm, which will be proved by the latter simulation results.


## 4. Stability Analysis

To analyze the convergence behavior of (4c), first, substitute (11) into (4c)(13)wn+1=wn+μφnφnHφnHφnw̃n+μφnφnHφn(εn-∑i=1mân,iεn-i)*.
Combining (13) and (5), the adaptation equation in error form is obtained as(14)w̃n+1=w̃n-μφnφnHφnHφnw̃n-μφnφnHφn(εn-∑i=1mân,iεn-i)*.
Under the assumption that the measurement noise {εn} is zero-mean white noise, it is independent of the input process {xn} and therefore of the direction vector {φn} as well. So the expected value of the last term of (14) becomes(15)E[μφnφnHφn(εn-∑i=1mân,iεn-i)*]=0.
From (14), it can be found that the weight error {w̃n} does not depend on the direction vector {φn}, but it does depend on the direction vector {φn-1}. So based on (15), taking expectation on both sides of (14) yields(16)E[w̃n+1]=(I-μE[φnφnHφnHφn])E[w̃n],
where I is an identity matrix. Based on [5], under the situation that the input process {xn} is assumed to be zero-mean wide-sense stationary autoregressive inputs of order p and the number of the most recent input vectors m≥p, the following result can be obtained: (17)E[φnφnHφnHφn]=1N-m-2I.
From (16) and (17), we obtain(18)E[w̃n+1]=(1-μ1N-m-2)E[w̃n].
The mean of the weight error will converge to zero if(19)|1-μ1N-m-2|<1
which leads to(20)-1<1-μ1N-m-2<1.
The right inequality in (20) is unrestrictive; the left inequality produces(21)μ<2(N-m-2).


## 5. Steady-State Weights

Substituting (7) into (4c), we obtain(22)wn+1=wn+μφnφnHφn×[(dn-∑i=1mân,idn-i)-(d̂n-∑i=1mân,id̂n-i)]*.
Letting n→∞, the result wn+1=wn=wn-i=w∞, 1≤i≤m can be obtained. Based on it, (22) can be rewritten as(23)φn(d̂n-∑i=1mân,id̂n-i)*=φn(dn-∑i=1mân,idn-i)*.
Using (3) into (23) yields(24)φn(xn-∑i=1mân,ixn-i)Hw∞=φn(dn-∑i=1mân,idn-i)*.
Combining (4d) and (24), we obtain(25)φnφnHw∞=φn(dn-∑i=1mân,idn-i)*.
When w∞ is independent of the direction vector {φn}, taking the expectation on both sides of (25) yields (26)w∞=(E[φnφnH])-1E[φn(dn-∑i=1mân,idn-i)*].
Under the assumption that there exist a true adaptive filter weight vector w0 of dimension N, and from (6), we obtain(27)w∞=(E[φnφnH])-1×E[φn(xn-∑i=1mân,ixn-i)Hw0+φn(εn-∑i=1mân,iεn-i)*].
From (4d) and (27), the following result can be obtained:(28)w∞=(E[φnφnH])-1×{E[φnφnH]w0+E[φn(εn-∑i=1mân,iεn-i)*]}.
From (28), it yields(29)w∞=w0+(E[φnφnH])-1E[φn(εn-∑i=1mân,iεn-i)*].
When the measurement noise is assumed to be zero-mean white noise, the last term of (29) is equal to zero. So we can obtain the result w∞=w0. And it can be concluded that the steady-state weights of the APA-REE algorithm are unbiased and consist. Furthermore, based on [5], if the input process {xn} is assumed to be zero-mean wide-sense stationary autoregressive inputs of order p and the number of the most recent input vectors m≥p, the direction vector {φn} and the estimated weight vector {wn} are statistically independent, and the direction vector {φn} is a zero-mean Gaussian random vector, so the results of (15) and (29) can also be obtained to be equal to zeros.


## 6. Comparison with Simulation Results

In this section, we compare the MSE learning curves between the APA-REE and APA algorithms. The estimated output error of the APA-REE and APA algorithms is defined as in (4a). Simulation results corresponding to two different types of signals, namely, reasonably colored and highly colored, are shown. The reasonably and highly colored signals are generated as a Gaussian first-order autoregressive process with poles at 0.25 and 0.95, respectively. The initial estimate for the weight vector is w0=0. The system to be identified has a 32-point long impulse. The simulation results shown are obtained by ensemble averaging over 100 independent trials for each experiment. And we use the maximum entropy assumption for the optimal true weight vector w0. That is, w0 has equal components along all eigenvectors of E[φnφnH]=VΛVH:(30)w0=V1,
where the vector 1=[11⋯1]T. And the measurement noise {εn} is assumed to be zero-mean white noise.Using a reasonably colored input signal, Figure2 gives the comparison result between the APA-REE and APA algorithms. The measurement noise is assumed to be absent. We use the parameter m=1. The parameter step size μ is set to be equal to 0.05. We observe that the APA-REE algorithm improves the adaptive filtering convergence rate than the APA algorithm.Figure 2
Reasonably colored input without measurement noise.Figure3 considers the same input and parameters as Figure 2, but with the signal-to-noise ratio 60 dB. We observe that the convergence rate of the APA-REE algorithm is slightly better compared with the APA algorithm.Figure 3
Reasonably colored input with measurement noise.Under the situation that the input is a highly colored input signal without the measurement noise, Figure4 gives the comparative simulation results between the APA-REE and APA algorithms. We use the parameter m=3 and step size μ=0.2. We find that, for the convergence rate, the APA-REE algorithm is much better than the APA algorithm.Figure 4
Highly colored input without measurement noise.Figure5 considers the same input and parameters as Figure 4, but the signal-to-noise ratio is set to be equal to 60 dB.  From Figure 5, we observe that the APA-REE algorithm converges faster than the APA algorithm.Figure 5
Highly colored input with measurement noise.


## 7. Conclusions

The iterated error of the APA algorithm is not only caused by the iterated direction{φn}, but also caused by the other directions that do not contribute to the iterated direction. In this paper, the APA-REE algorithm is presented to ameliorate this problem. Under the measurement noise-free condition, the iterated error of the APA-REE algorithm is almost only caused by the iterated direction {φn}. The stability for the APA-REE algorithm has also been analyzed and shown that the range of the step size μ<2(N-m-2) is much wider than the step size range μ<2 of the APA algorithm given by [6]. And the steady-state weights of the APA-REE algorithm are proved to be unbiasedness and consistency. The simulation results show that the APA-REE algorithm improves the adaptive filtering convergence rate compared with the APA algorithm.



---

*Source: 180624-2011-04-27.xml*