# AR Model-Based Direction-of-Arrival Estimation of Coherent Signals in the Presence of Unknown Mutual Coupling

Authors: Zhi-Chao Sha; Zhang-Meng Liu; Zhi-Tao Huang; Yi-Yu Zhou
Journal: International Journal of Antennas and Propagation
 (2013)

---

## Abstract

This paper addresses the problem of direction-of-arrival (DOA) estimation of coherent signals in the presence of unknown mutual coupling, and an autoregression (AR) model-based method is proposed. The effects of mutual coupling can be eliminated by the inherent mechanism of the proposed algorithm, so the DOAs can be accurately estimated without any calibration sources. After the mixing matrix is estimated by independent component analysis (ICA), several parameter equations are established upon the mixing matrix. Finally, all DOAs of coherent signals are estimated by solving these equations. Compared with traditional methods, the proposed method has higher angle resolution and estimation accuracy. Simulation results demonstrate the effectiveness of the algorithm.


---

## Body

## 1. Introduction

Direction-of-arrival (DOA) estimation is very important in a variety of wireless communication applications, such as mobile communication, radar, and distributed sensor networks. In particular, many effective high-resolution DOA estimation algorithms have been developed and deeply investigated in the last decades [1]. Since then, the attention of the signal processing community has focused on the factors that block the practical application of those algorithms. The first factor is the unknown mutual coupling, which will affect the array manifold of the array and result in poor accuracy of DOA estimation [2]. The other factor is that there may be highly correlated or coherent signals because of multipath propagation [3, 4]. When the incident signals are highly correlated or coherent in the presence of unknown mutual coupling, the performance of conventional high-resolution DOA estimation methods will deteriorate significantly.In the last years, many array calibration algorithms have been proposed with respect to the mutual coupling effect [5–13]. Hung [5] uses an iterative least mean-square approach to estimate the calibration matrix, but it requires a preliminary calibration. The above algorithms may not be easily carried out in practice, because of the additional calibration sources or sensors. An iterative algorithm is given to compensate the mutual coupling and perturbation of gain and phase in [6]. However, the convergence rate is slow, and computational cost is very expensive. In [7], a novel online mutual coupling compensation algorithm is presented to estimate coupling parameters through an alternating minimization technique, but the convergence is not well guaranteed. In [8], an algorithm that applies a group of auxiliary sensors in uniform linear arrays (ULAs) has been proposed to estimate the DOAs, but the algorithm requires a large number of sensors, and it is difficult to be satisfied in practice. References [9, 10] present a unified framework and sparse Bayesian perspective for array calibration and DOA estimation. Moreover, Dai et al. proposed a sparse representation method to eliminate the effect of mutual coupling by its inherent mechanism [11]. However, the computational cost is expensive. Many studies have been made to reduce the computational complexity of the calculations by using a certain unitary transformation that converts complex-valued manifold matrices of uniform linear arrays (ULAs) into real ones [12, 13].On the other hand, many techniques have been proposed to deal with the correlated or coherent situation. A forward/backward spatial smoothing (FBSS) method that can solve the coherent problem is presented in [3]. Malioutov et al. propose the method of L1-SVD to address the general DOA estimation problem [14]. L1-SVD first decomposes the array output and extracts the signal energy into K (the signal number) singular vectors and then represents them under sparsity constraint to estimate the signal directions. The method in [15] estimates the uncorrelated and coherent signals separately but encounters the differencing matrix power loss and needs extra processing to recover the rank. Recently, independent component analysis (ICA) has been utilized to solve the DOA estimation problem [16, 17]. These methods can estimate the real steering vectors with unknown mutual coupling. However, owing to the complex structure of the MCM, all the methods taking care of the correlated or coherent situation cannot be utilised to estimate the DOAs in the presence of unknown mutual coupling.However, it is more difficult to estimate DOAs of coherent signals in the presence of unknown mutual coupling. Dai and Ye [18] propose an improved spatial smoothing algorithm for DOA estimation of coherent signals in the presence of unknown mutual coupling, but it significantly deteriorates while angle interval is not large enough or several groups of coherent signals coexist. Inspired by [19] and based on the estimation of the real steering vectors by ICA, we develop a spatial AR model-based algorithm for coherent DOA estimation of ULA in the presence of mutual coupling. Simulations illustrate that the DOA estimation accuracy of our approach is higher than the improved spatial smoothing algorithm in [18].The paper is organized as follows. The data model of the ULA is given in Section2. The spatial AR model algorithm is described in detail in Section 3. Computer simulations and conclusions follow in Sections 4 and 5.


## 2. Data Model

ConsiderK narrowband non-Gaussian signals impinging on a uniform linear array (ULA) with M array elements, where the distance d between adjacent sensors is equal to half of the wavelength. Assume that there exist P groups of coherent signals because of multipath propagation, and the signals within the same group are coherent and independent in different groups. In the ith group, suppose the coherent signal coming from the direction θil is corresponding to the lth multipath propagation of the source si(t), and l=1,…,Ki. The total number of coherent signals can be denoted as K=∑i=1PKi. The array output vector is expressed as
(1)x(t)=As(t)+n(t)=∑i=1P∑lKia(θil)γilϕilsi(t)+n(t),
where a(θil)=[1,e-j2πdsin(θil)/λ,…,e-j(M-1)2πdsin(θil)/λ]T is the steering vector of the direction θil, λ is the wavelength of the signal, T is the transpose operator, γil and ϕil are corresponding to the amplitude and phase fading coefficients of the lth signal in the ith group, ρi=[γi1ϕi1,…,γiKiϕiKi]T, Ai=[a(θi1),…,a(θiKi)], s(t)=[s1(t),…,sP(t)]T, and n(t) is zero mean additive white Gaussian noise vector. The real steering vector of the ith group coherent signals is given by τi=Aiρi.In the presence of mutual coupling, the true steering vector should be modified as(2)x(t)=CAs(t)+n(t),
where C is the mutual coupling matrix (MCM) and can be expressed as a banded symmetric Toeplitz matrix with just a few nonzero coefficients
(3)C=Toeplitz{1,c1,…,cP-1,01×(M-P)}.The covariance matrix of the received signals is defined by(4)Rx=E{x(t)x(t)H}=CARsAHCH+σ2IM,
where E{·} is the expectation operator, the superscript H denotes transpose complex conjugate operation, Rs=E{s(t)s(t)H} is the sources covariance matrix, σ2 is the variance of the additive noise, and IM is an identity matrix.


## 3. Novel AR Model-Based DOA Estimation Algorithm

We assumeL denotes the number of coherent signals in one group. Referring to [19], the mappings ejωk(k=1,…,L) of the signal directions on the unit circle are distinct roots of an Lth order equation if no ambiguity occurs. The relationship between ωk and the direction of the kth signal θk is ωk=2πdsin(θk)/λ. Assuming that the coefficients of the unified equation are κ0,…,κL-1 and the unknown parameter is β, the equation is then given by
(5)f(β)=∏k=1L(β-ejωk)=βL+κL-1βL-1+⋯+κ1β+κ0=0.
Equation (5) presents the relationship of the coherent signals directions. For one group of coherent signals without mutual coupling, the real steering vector is a linear mixture ofLideal steering vectors. The special relationship is then given by
(6)τ=[τ0,…,τM-1]T=ε[∑k=1Lρkej0ωk,…,∑k=1Lρkej(M-1)ωk]T,
where ρk is the corresponding fading coefficient in the kth multipath propagation and ε is the corresponding scaling ambiguity coefficient caused by ICA processing.According to (6), the Jth element of τ is
(7)τJ=∑k=1LερkejJωk,J=0,1,…,M-1.Becauseejωk(k=1,…,L) are the roots of (5), the following L equations hold:
(8)ejLωk+κL-1ej(L-1)ωk+⋯+κ1ejωk+κ0=0,k=1,…,L.
According to the idea of [19], multiplying both sides of (8) with ερkejLωk(k=1,…,L) and then summing up the L equations yield
(9)∑k=1Lερkej(L+ξ)ωk+κL-1∑k=1Lερkej(L+ξ-1)ωk+⋯+κ1∑k=1Lερkej(1+ξ)ωk+κ0∑k=1Lερkejξωk=0;ξ=0,1,…,M-L-1.Substituting (7) into (9), we obtain the relationship between the equation coefficients and vector τ(10)τL+ξ+κL-1τL+ξ-1+⋯+κ1τ1+ξ+κ0τξ=0,ξ=0,1,…,M-L-1.The real steering vectors in the presence of unknown mutual coupling can be given by(11)G=CA=[1c1⋯cNc⋯0c11c1⋯⋱0⋮c11⋱⋯cNccNc⋯⋱⋱c1⋮0⋱⋯c11c10⋯cNc⋯c11]M×M×[τ1,…,τP]M×P.
Each of gi contains all the spatial information of one group of coherent signals. The real steering vectors G can be estimated by ICA [16, 17].Then we will introduce the proposed method to solve the problem of DOA estimation of coherent signals in the presence of unknown mutual coupling.


### 3.1. The Case ofNc=1

For any given column vectorg, we get
(12)g0=τ0+c1τ1g1=c1τ0+τ1+c1τ2g2=c1τ1+τ2+c1τ3⋮gM-2=c1τM-3+τM-2+c1τM-1gi,M-1=c1tM-2+tM-1.For the second equation to the (L+2)th equation, multiplying both sides of (8) with {κ0,…,κL-1,1} and then summing up the L equations yield
(13)gL+1+κL-1gL+⋯+κ1g2+κ0g1=0.Using the same principle to process all the adjacentL+1 equations, we get
(14)[g1g2⋯gLg2g3⋯gL+1⋮⋮⋱⋮gM-L-2gM-L-1⋯gM-3][κ0κ1⋮κL-1]=-[gL+1gL+2⋮gM-2].In addition, as the signals are of complex value, we utilize their conjugate information to improve the precision of the proposed method as [19]. Similarly, the following equations hold:
(15)gξ*+κL-1gξ+1*+⋯+g1τξ+L-1*+κ0gξ+L*=0,ξ=1,…,M-L-2.Combining (10) and (15), we establish the following equation set:
(16)[g1g2⋯gLg2g3⋯gL+1⋮⋮⋱⋮gM-L-2gM-L-1⋯gM-3gL+1*gL*⋯g2*gL+2*gL+1*⋯g3*⋮⋮⋱⋮gM-2*gM-3*⋯gM-L-1*][κ0κ1⋮κL-1]=-[gL+1gL+2⋮gM-2g1*g2*⋮gM-L-2*]
in which the superscript * denotes the conjugate operation.


### 3.2. The Case ofNc>1

From the case ofNc=1, we note that the elements of real steering vectors can be utilized adequately due to the mutual coupling. If Nc>1, the elements {g0,…gNc-1} and {gM-Nc,…gM-1} are useless to estimate the DOAs. The number of useful elements is
(17)Ng=M-2*Nc.According to (16), the new formulation is given by
(18)[gNcgNc+1⋯gNc+L-1gNc+1gNc+2⋯gNc+L⋮⋮⋱⋮gM-L-Nc-1gM-L-Nc⋯gM-Nc-2gNc+L*gNc+L-1*⋯gNc+1*gNc+L+1*gNc+L*⋯gNc+2*⋮⋮⋱⋮gM-Nc-1*gM-Nc-2*⋯gM-L-Nc*][κ0κ1⋮κL-1]=-[gNc+LgNc+L+1⋮gM-Nc-1gNc*gNc+1*⋮gM-L-Nc-1*].By comparing (16) with (18), we know that (16) is a special case of (18). The right vectors g of (18) can be estimated by ICA. So, the coefficient vector κ can be estimated under the least mean-square error criterion. The least square solution is then given by
(19)κ=-[gNcgNc+1⋯gNc+L-1gNc+1gNc+2⋯gNc+L⋮⋮⋱⋮gM-L-Nc-1gM-L-Nc⋯gM-Nc-2gNc+L*gNc+L-1*⋯gNc+1*gNc+L+1*gNc+L*⋯gNc+2*⋮⋮⋱⋮gM-Nc-1*gM-Nc-2*⋯gM-L-Nc*]†×[gNc+LgNc+L+1⋮gM-Nc-1gNc*gNc+1*⋮gM-L-Nc-1*],
where [·]† denotes Moore-Penrose inverse.All roots of (8) can be estimated with the known equation coefficients κ0,…,κL-1. The DOAs of coherent signals in one group are given by
(20)θk=sin-1(Arg(βk)λ/(2πd)),k=1,…,L,
where β1,…,βL are the roots of (8) and Arg(·)denotes the phase angle of complex value. Thus, the DOAs of coherent signals in one group are estimated according to (20).


### 3.3. Discussion

Due to (18), in order to make sure the equation has a unique solution, the left matrix of (18) must be of full rank. Therefore, the following inequality must be satisfied:
(21)2(M-L-2Nc)≥L.It is not difficult to see that the number of coherent signals in one groupL≤⌊(2/3)(M-2Nc)⌋. As we know, the maximum number of independent sources resolved by ICA is equal to the number of sensors [17]. So, the maximum detectable number of source signals by the proposed method is
(22)Kmax=⌊23(M-2Nc)⌋·M.For the improved FBSS, the detectable number of sourceK and the length of subarrays M0 must satisfy
(23)K<M0,K-P<M-M0-2Nc+1.WhenM0=K+1, (23) can be rewrite as
(24)K<(M+P-2Nc)2.


## 3.1. The Case ofNc=1

For any given column vectorg, we get
(12)g0=τ0+c1τ1g1=c1τ0+τ1+c1τ2g2=c1τ1+τ2+c1τ3⋮gM-2=c1τM-3+τM-2+c1τM-1gi,M-1=c1tM-2+tM-1.For the second equation to the (L+2)th equation, multiplying both sides of (8) with {κ0,…,κL-1,1} and then summing up the L equations yield
(13)gL+1+κL-1gL+⋯+κ1g2+κ0g1=0.Using the same principle to process all the adjacentL+1 equations, we get
(14)[g1g2⋯gLg2g3⋯gL+1⋮⋮⋱⋮gM-L-2gM-L-1⋯gM-3][κ0κ1⋮κL-1]=-[gL+1gL+2⋮gM-2].In addition, as the signals are of complex value, we utilize their conjugate information to improve the precision of the proposed method as [19]. Similarly, the following equations hold:
(15)gξ*+κL-1gξ+1*+⋯+g1τξ+L-1*+κ0gξ+L*=0,ξ=1,…,M-L-2.Combining (10) and (15), we establish the following equation set:
(16)[g1g2⋯gLg2g3⋯gL+1⋮⋮⋱⋮gM-L-2gM-L-1⋯gM-3gL+1*gL*⋯g2*gL+2*gL+1*⋯g3*⋮⋮⋱⋮gM-2*gM-3*⋯gM-L-1*][κ0κ1⋮κL-1]=-[gL+1gL+2⋮gM-2g1*g2*⋮gM-L-2*]
in which the superscript * denotes the conjugate operation.


## 3.2. The Case ofNc>1

From the case ofNc=1, we note that the elements of real steering vectors can be utilized adequately due to the mutual coupling. If Nc>1, the elements {g0,…gNc-1} and {gM-Nc,…gM-1} are useless to estimate the DOAs. The number of useful elements is
(17)Ng=M-2*Nc.According to (16), the new formulation is given by
(18)[gNcgNc+1⋯gNc+L-1gNc+1gNc+2⋯gNc+L⋮⋮⋱⋮gM-L-Nc-1gM-L-Nc⋯gM-Nc-2gNc+L*gNc+L-1*⋯gNc+1*gNc+L+1*gNc+L*⋯gNc+2*⋮⋮⋱⋮gM-Nc-1*gM-Nc-2*⋯gM-L-Nc*][κ0κ1⋮κL-1]=-[gNc+LgNc+L+1⋮gM-Nc-1gNc*gNc+1*⋮gM-L-Nc-1*].By comparing (16) with (18), we know that (16) is a special case of (18). The right vectors g of (18) can be estimated by ICA. So, the coefficient vector κ can be estimated under the least mean-square error criterion. The least square solution is then given by
(19)κ=-[gNcgNc+1⋯gNc+L-1gNc+1gNc+2⋯gNc+L⋮⋮⋱⋮gM-L-Nc-1gM-L-Nc⋯gM-Nc-2gNc+L*gNc+L-1*⋯gNc+1*gNc+L+1*gNc+L*⋯gNc+2*⋮⋮⋱⋮gM-Nc-1*gM-Nc-2*⋯gM-L-Nc*]†×[gNc+LgNc+L+1⋮gM-Nc-1gNc*gNc+1*⋮gM-L-Nc-1*],
where [·]† denotes Moore-Penrose inverse.All roots of (8) can be estimated with the known equation coefficients κ0,…,κL-1. The DOAs of coherent signals in one group are given by
(20)θk=sin-1(Arg(βk)λ/(2πd)),k=1,…,L,
where β1,…,βL are the roots of (8) and Arg(·)denotes the phase angle of complex value. Thus, the DOAs of coherent signals in one group are estimated according to (20).


## 3.3. Discussion

Due to (18), in order to make sure the equation has a unique solution, the left matrix of (18) must be of full rank. Therefore, the following inequality must be satisfied:
(21)2(M-L-2Nc)≥L.It is not difficult to see that the number of coherent signals in one groupL≤⌊(2/3)(M-2Nc)⌋. As we know, the maximum number of independent sources resolved by ICA is equal to the number of sensors [17]. So, the maximum detectable number of source signals by the proposed method is
(22)Kmax=⌊23(M-2Nc)⌋·M.For the improved FBSS, the detectable number of sourceK and the length of subarrays M0 must satisfy
(23)K<M0,K-P<M-M0-2Nc+1.WhenM0=K+1, (23) can be rewrite as
(24)K<(M+P-2Nc)2.


## 4. Simulation Experiment

In this section, some computer simulations are reported to illustrate the performance of our proposed method. In the following simulations, we will compare the proposed method to FBSS [3] and the improved FBSS algorithm in [18] for DOA estimation.In the first simulation, we consider one group of two coherent signals impinging on an 8-element ULA from the directions[-10∘,20∘], and the number of the mutual coupling coefficients is Nc=1 with c1=0.3844-0.3476i [18]. The amplitude fading factor is [1,0.8]. Figure 1 shows the root mean square error (RMSE) of each DOA estimate against input SNR computed via 200 Monte Carlo runs for each SNR and 500 snapshots of data for each run. As shown in Figure 1, the proposed method outperforms the improved FBSS method, and achieves similar performance as the original FBSS method with known mutual coupling.Figure 1
RMSE of DOA estimates against SNR (Nc=P=1).In the second simulation, we consider the more complicated situation: two groups of two coherent signals impinge on an 8-element ULA from the directions[-32∘,-8∘] and [15∘,42∘], and the number of the mutual coupling coefficients is Nc=2 with c1=0.3844-0.3476i and c2=0.24+0.1i. The amplitude fading factors are [1,0.9] and [1,0.8]. Figure 2 shows the RMSE of each DOA estimate against input SNR computed via 200 Monte Carlo runs for each SNR and 500 snapshots of data for each run. The results illustrate that our method can get high-resolution in the complex structure of the MCM when the level of SNR is large enough. However, the improved FBSS maintains a coarse accuracy no matter what the levels of SNR are.Figure 2
RMSE of DOA estimates against SNR (Nc=P=2).In the third simulation, we will validate the high spatial resolution of the proposed method. Consider one group of two coherent signals with SNR of 10 dB impinging on an 8-element ULA, where the amplitude factors are the same as simulation 1. Assume that the directions of two coherent signals are-10∘ and -9∘+Δθ, where 1+Δθ denotes the angle interval between the two source signals. We have 200 Monte Carlo trials with 500 snapshots to demonstrate the performance of the proposed method. Figure 3 shows the RMSE of DOA estimation using different algorithms versus the angle interval Δθ. It can be seen from Figure 3 that the proposed method outperforms the other two methods when the angle interval is not large enough.Figure 3
The RMSE of DOA estimation versus the angle interval.To verify the maximum detectable number of source signals by the proposed method, in the forth simulation, we consider five groups of two coherent signals impinging on a 5-element ULA from the directions[-40∘,10∘], [-30∘,20∘], [-20∘,30∘], [-10∘,40∘], and [0∘,50∘], and the number of the mutual coupling coefficients is Nc=1 with c1=0.3844-0.3476i. According to (22), the number of source signals is the maximum detectable number of our method ⌊(2/3)(M-2Nc)⌋·M=10. However, the improved FBSS and other sparse representation methods are incapable of processing the 10 sources with 5-element ULA due to their conditions. Figure 4 shows the RMSE of each DOA estimate against input SNR computed via 500 Monte Carlo runs for each SNR and 500 snapshots of data for each run. The result illustrates that the maximum detectable number of sources by the proposed method is consistent with (24).Figure 4
The RMSE of DOA estimation against SNR (Nc=1, P=5, K=10).


## 5. Conclusion

In this paper, an AR model-based DOA estimation algorithm is proposed for coherent signals in the presence of unknown mutual coupling. The effects of mutual coupling can be eliminated by solving a mathematical equation. Simulation results demonstrate that the proposed method has high spatial resolution and DOA estimation accuracy compared to the improved FBSS algorithm. Furthermore, the number of signals resolved by our method is larger than that of others.



---

*Source: 386367-2013-11-11.xml*