# An Efficient DOA Estimation Method for Passive Surveillance System Based on Troposcatter

Authors: Xinping Mi; Zan Liu; Xihong Chen; Qiang Liu
Journal: Mathematical Problems in Engineering
 (2021)

---

## Abstract

Direction of arrival (DOA) estimation plays an important role in the passive surveillance system based on troposcatter. Rank deficiency and subspace leakage resulting from multipath propagation can deteriorate the performance of the DOA estimator. In this paper, characteristics of signals propagated by troposcatter are analyzed, and an efficient DOA estimation method is proposed. According to our new method, the invariance property of noise subspace (IPNS) is introduced as the main method. To provide precise noise subspace for INPS, forward and backward spatial smoothing (FBSS) is carried out to overcome rank deficiency. Subspace leakage is eliminated by a two-step scheme, and this process can also largely reduce the computational load of INPS. Numerical simulation results validate that our method has not only good resolution in condition of closely spaced signals but also superior performance in case of power difference.


---

## Body

## 1. Introduction

Troposcatter as a promising beyond-line-of-sight (b-LOS) communication link has been well studied [1, 2]. On the basis of electromagnetic (EM) wave radiated by hostile radiator and propagated by troposcatter, passive surveillance can realize b-LOS location [3, 4]. Direction of arrival (DOA) estimation with high performance is the prerequisite of a passive surveillance system [5–12]. Intricate scattering procedure can generate multipath propagation, and scattered signals can be treated as closely spaced sources and own respective power; some signals with low signal to noise ratio (SNR) may be flooded by noises. Therefore, the multipath effect of troposcatter can bring rank deficiency and subspace leakage to pivotal covariance matrix, and prevalent DOA estimation methods, including multiple signal classification (MUSIC) and estimation signal parameters via rotational invariance technique (ESPRIT), will suffer from serious performance deterioration.The DOA estimation method employed by a passive surveillance system must confront coherent signals owning different power and close DOAs. The maximum likelihood method [7] can overcome the multipath effect. Since multidimensional solution searching, its application is restricted. Forward and backward spatial smoothing (FBSS) can also solve the rank deficiency [10]. To improve the performance of the passive surveillance system, a novel DOA estimation method is developed in this paper. The invariance property of noise subspace (IPNS), which has strong robustness for close DOAs and different power, is employed as the main method. Meanwhile, to improve the performance of INPS, rank deficiency is overcome by FBSS, and subspace leakage is eliminated on the basis of a two-step scheme.The rest of this paper is organized as follows. Section2 discusses characteristics of signals received by the passive surveillance system. In Section 3, the novel DOA estimation method is introduced. In Section 4, several examples are described. Some conclusions are drawn in Section 5. The notations of tr⋅, ()H, ()T, and ⋅∗ denote the trace, conjugation-transpose, transpose, and conjugation of the matrix, respectively.


## 2. Signal Model

Figure1 shows a troposcatter link suffering from multipath propagation, and segmental power can arrive at our passive surveillance system. Multipath propagation can make the covariance matrix rank-deficient. Various excitation sources existing in the scattering cross section can bring different propagation loss and close DOAs to received signals. Some signals with low SNR may be treated as noises.Figure 1
Troposcatter b-LOS paths.The correlation coefficient can effectively describe the multipath effect. When frequency coherent characteristics are considered, correlation coefficient can be finally simplified as [11–13](1)ρΔf=exp−1+s2πHΔfs1ψv12+ψv222c2ln2,where ρΔf denotes the correlation coefficient, Δf is the frequency difference, and c is the light speed. s2=1/s1=θt/θr, where θt and θr are the transmitter and receiver horizon angles, respectively. H denotes the height of lowest scattering point (km), H=10−3Θ0L/4. L refers to the path length between the receiving and transmitting antennas (km). Θ0 refers to the least scatter angle, and Θ0=θt+θr+L/ae, where ae is the median effective earth radius (km). ψ denotes the beamwidth; for a parabolic antenna, ψ≈1.2λ/D, and for an array antenna, ψ≈0.866λ/Md [10, 11], where D represents the antenna diameter, M is the array element number, d is the interelement space, and λ is the wavelength. Figure 2 shows the correlation coefficient of a troposcatter link with parameters: Dt=10m, M=30, f=4GHz, d=λ/2, and θt=θr=1∘. Larger frequency difference and longer distance can reduce correlation. Doppler shifts of signals propagated by troposcatter are relatively low and can be neglected, so scattered signals coming from identical radiation are coherent or highly correlated.Figure 2
Correlation coefficient for separation.Assume that there exist several scatterers in the scatter cross section. Under the worst condition, allKc signals received by passive surveillance system are coherent, and each signal has an individual SNR. Undetected signals with low SNR are treated as noises. Uniform linear frequency array (ULA) with M sensors is employed as the received antenna, and KK<M narrowband signals impinge on the ULA. The output of receiver can be represented as [5, 6, 14](2)Xt=AθSt+Nt,t=1,2,…,n,where Aθ=aθ1,aθ2,…,aθN, Nt is the noise vector, and aθk(k=1,2,…,M) is the steering vector. Covariance matrix R∈CM×M can be given by [14](3)R=EXtXHt=ARSAH+RN,where RS is the source covariance matrix and RN is the noise covariance matrix. When all received signals are uncorrelated, the eigenvalues of R can be expressed as(4)λ1≥λ2≥⋯≥λK≥λK+1=⋯=λM.Due to multipath propagation, coherent signals exist,R is a rank-deficient matrix, two subspaces cannot be accurately estimated, and the DOA methods-based subspace will become invalid. FBSS technique can output a full-rank covariance matrix. L denotes the number of subarrays, and m refers to the elements of each subarray. The full-rank covariance matrix can be given by [15](5)RFB=12L∑k=1LFkR+JR∗JFkT,where Fk=0m×k−1Im0m×N−k−m+1 and J denotes the m×m exchange matrix; the elements on its inverse diagonal are 1, and the other elements are 0. In practice, the estimated covariance matrix can be expressed as(6)R^=1T∑t=1TXtXHt,where T denotes the snapshot number. After operating FBSS, eigenvectors of R^FB can be written as(7)λ^1≥⋯≥λ^K≥λ^K+1≥⋯≥λ^M.Several methods can precisely estimate the source number based on eigenvectors. Those signals flooded by noises cannot be detected by our system and are treated as noises. After running FBSS and estimating source number, MUSIC can efficiently estimate DOAs. However, the ability of MUSIC to resolve sources with different power and close DOAs is limited. Literatures [16–19] have demonstrated that INPS performs better than MUSIC under these adverse conditions. INPS depends on the noise subspace which cannot be precisely got from a rank-deficient covariance matrix. Therefore, FBSS is still necessary.


## 3. DOA Estimation Model

### 3.1. IPNS Method

The IPNS method is based on the property that noise subspace keeps invariant when only power of source changes. The new matrix can be defined as [16](8)Dφ=R+haφaφH,where h is a constant and aφ is the M×1 steering vector for direction φ. When φ in (8) is set to one of the source directions, RN keeps invariant and the last (M-K) eigenvalues of Dφ and R are equal, i.e.,(9)μkφ=λk,k=K+1,…,M,where μkφ denotes the eigenvalue of Dφ. Except the actual DOAs, no other values of φ have this unique property. Therefore, the objective function can be given by [17](10)Fφ=1∑k=K+1Mμ^kφ−λ^k.DOAs correspond to maximums ofFφ. As shown in (8), parameter h affects the performance of INPS. The suitable value has been experimentally obtained and can be written as [20–22](11)h=trRM.As analyzed above, eigenvalue decomposition for eachφ is required, so running the INPS method needs much computational cost. Computational load can be further reduced with no performance degradation when approximate DOAs are known in advance. If exact eigenvectors of noise subspace can be acquired by decomposing covariance matrix, precise DOAs will be obtained according to the INPS method. FBSS can only make R full-rank, and some signals flooded in noises must be eliminated. Therefore, the covariance matrix must be further modified before eigenvalue decomposition.


### 3.2. Our Method Based on Modified Covariance Matrix

Substituting (2) into (6), we can expand R^ as(12)R^=1NAst+ntAst+ntH=A1N∑t=1NstsHtAH+1N∑t=1NntnHt+A1N∑t=1NstnHt+1N∑t=1NntsHtAH.According to (12), R^ consists of four terms. The first two terms represent RS and RN, respectively. The last two terms can be treated as the correlation between signals and noises, which can decrease the performance of DOA estimation. The fourth term is equal to the Hermitian of the third term. For the passive surveillance system based on troposcatter, this undesirable subspace leakage results from two aspects. On the one hand, some true signals with relatively low SNR are flooded by noises. On the other hand, the system can only get finite snapshots. To provide precise eigenvectors for the INPS method, portion of true signals residing in RN must be eliminated. Literatures [23, 24] employ the least square technique to obtain St, i.e.,(13)S^t=A^HA^−1A^HXt.Noise component can be estimated as the difference betweenSt and Xt, i.e.,(14)N^t=Xt−A^S^t.On the basis of (13) and (14), the third term of (12) can be given by(15)T=A^1N∑t=1NS^tN^Ht=P^AR^P^A⊥,where P^A⊥=I−P^A, and P^A can be expressed as(16)P^A=A^A^HA^−1A^H.Generally,A^ cannot be known in advance. Similar to literature [19], we introduce a two-step scheme to estimate A^. Approximate DOAs are initially acquired by FBSS-MUSIC. Then, substituting consequences into (5), we can estimate A^. So, the value of subspace leakage can be further estimated on the basis of (15) and (16). Finally, the modified covariance matrix can be acquired as(17)R^2=R^−γT+TH.The scaling factorγ in (17) denotes a real number between 0 and 1. Ideally, the value of γ should be equal to 1. Unfortunately, estimation errors are inevitable. Literatures [23, 24] use the stochastic ML (SML) objective function to estimate suitable γ. Because cross covariance matrixes of signal and noise parts can be partly eliminated, the noise subspace can be estimated more precisely. Procedure of our method is shown in Figure 3 and can be performed according to the following steps:Inputs:M, λ, N, L, d, and XtOutputs:θ^12,θ^22,…,θ^K2Steps:(1)
Employ FBSS-MUSIC to get approximate DOAs (θ^11,θ^21,…,θ^K1).(2)
EstimateA^ based on θ^11,θ^21,…,θ^K1 and modify R^ according to (13)–(17).(3)
Conduct FBSS to make the modified covariance matrix full-rank.(4)
Operate the INPS method based on output of step 3 to getθ^12,θ^22,…,θ^K2. While running the INPS method, find the peaks only around θ^11,θ^21,…,θ^K1.Figure 3
Procedure of our method.Because FBSS is employed twice, for an array withM elements, maximum sources that our method can effectively identify are 2M/3. We may know the possible position of target in advance; step 1 can also estimate approximate DOAs. Therefore, INPS can only search peaks around approximate DOAs, and the computational load is highly reduced. Meanwhile, targets of the passive surveillance system based on troposcatter are motionless or move with relatively low speed including ground-based radars or ship-borne radars, and the surveillance system has relatively high computation tolerance.


## 3.1. IPNS Method

The IPNS method is based on the property that noise subspace keeps invariant when only power of source changes. The new matrix can be defined as [16](8)Dφ=R+haφaφH,where h is a constant and aφ is the M×1 steering vector for direction φ. When φ in (8) is set to one of the source directions, RN keeps invariant and the last (M-K) eigenvalues of Dφ and R are equal, i.e.,(9)μkφ=λk,k=K+1,…,M,where μkφ denotes the eigenvalue of Dφ. Except the actual DOAs, no other values of φ have this unique property. Therefore, the objective function can be given by [17](10)Fφ=1∑k=K+1Mμ^kφ−λ^k.DOAs correspond to maximums ofFφ. As shown in (8), parameter h affects the performance of INPS. The suitable value has been experimentally obtained and can be written as [20–22](11)h=trRM.As analyzed above, eigenvalue decomposition for eachφ is required, so running the INPS method needs much computational cost. Computational load can be further reduced with no performance degradation when approximate DOAs are known in advance. If exact eigenvectors of noise subspace can be acquired by decomposing covariance matrix, precise DOAs will be obtained according to the INPS method. FBSS can only make R full-rank, and some signals flooded in noises must be eliminated. Therefore, the covariance matrix must be further modified before eigenvalue decomposition.


## 3.2. Our Method Based on Modified Covariance Matrix

Substituting (2) into (6), we can expand R^ as(12)R^=1NAst+ntAst+ntH=A1N∑t=1NstsHtAH+1N∑t=1NntnHt+A1N∑t=1NstnHt+1N∑t=1NntsHtAH.According to (12), R^ consists of four terms. The first two terms represent RS and RN, respectively. The last two terms can be treated as the correlation between signals and noises, which can decrease the performance of DOA estimation. The fourth term is equal to the Hermitian of the third term. For the passive surveillance system based on troposcatter, this undesirable subspace leakage results from two aspects. On the one hand, some true signals with relatively low SNR are flooded by noises. On the other hand, the system can only get finite snapshots. To provide precise eigenvectors for the INPS method, portion of true signals residing in RN must be eliminated. Literatures [23, 24] employ the least square technique to obtain St, i.e.,(13)S^t=A^HA^−1A^HXt.Noise component can be estimated as the difference betweenSt and Xt, i.e.,(14)N^t=Xt−A^S^t.On the basis of (13) and (14), the third term of (12) can be given by(15)T=A^1N∑t=1NS^tN^Ht=P^AR^P^A⊥,where P^A⊥=I−P^A, and P^A can be expressed as(16)P^A=A^A^HA^−1A^H.Generally,A^ cannot be known in advance. Similar to literature [19], we introduce a two-step scheme to estimate A^. Approximate DOAs are initially acquired by FBSS-MUSIC. Then, substituting consequences into (5), we can estimate A^. So, the value of subspace leakage can be further estimated on the basis of (15) and (16). Finally, the modified covariance matrix can be acquired as(17)R^2=R^−γT+TH.The scaling factorγ in (17) denotes a real number between 0 and 1. Ideally, the value of γ should be equal to 1. Unfortunately, estimation errors are inevitable. Literatures [23, 24] use the stochastic ML (SML) objective function to estimate suitable γ. Because cross covariance matrixes of signal and noise parts can be partly eliminated, the noise subspace can be estimated more precisely. Procedure of our method is shown in Figure 3 and can be performed according to the following steps:Inputs:M, λ, N, L, d, and XtOutputs:θ^12,θ^22,…,θ^K2Steps:(1)
Employ FBSS-MUSIC to get approximate DOAs (θ^11,θ^21,…,θ^K1).(2)
EstimateA^ based on θ^11,θ^21,…,θ^K1 and modify R^ according to (13)–(17).(3)
Conduct FBSS to make the modified covariance matrix full-rank.(4)
Operate the INPS method based on output of step 3 to getθ^12,θ^22,…,θ^K2. While running the INPS method, find the peaks only around θ^11,θ^21,…,θ^K1.Figure 3
Procedure of our method.Because FBSS is employed twice, for an array withM elements, maximum sources that our method can effectively identify are 2M/3. We may know the possible position of target in advance; step 1 can also estimate approximate DOAs. Therefore, INPS can only search peaks around approximate DOAs, and the computational load is highly reduced. Meanwhile, targets of the passive surveillance system based on troposcatter are motionless or move with relatively low speed including ground-based radars or ship-borne radars, and the surveillance system has relatively high computation tolerance.


## 4. Simulations and Results

To verify the effectiveness of our DOA estimation method, several simulations are conducted. Parameters involved in simulations can be defined as follows:M=30, L=3, m=28, d=λ/2, N=500, and searching step is 0.01∘. Five scattered signals impinge on our array, and the passive system can recognize three of them; two residuals with lower SNR are treated as noises. In the first experiment, three detected signals coming from the directions of θ1=−5∘, θ2=0∘, and θ3=5∘ are considered, their SNR is −3 dB, 2 dB, and −3 dB, respectively. Figure 4 shows the corresponding estimation curve of different methods.Figure 4
Estimation performance for different methods.Figure4 indicates that signals flooded in noises can obviously lower spatial spectrum of detected signals and deteriorate estimation performance. Apparently, INPS can weaken the disadvantageous effect caused by power difference. Because the subspace leakage is partly eliminated, our method has the best performance.In the second experiment, DOAs are same as the first experiment, power difference is 5 dB, and 500 Monte Carlo trials are carried out. Figure5 shows the root mean square error (RMSE) versus strongest SNR. When highest SNR is 0 dB, Figure 6 shows the RMSE versus snapshot number.Figure 5
RMSE versus SNR (N = 200).Figure 6
RMSE versus snapshot number.From Figures5 and 6, we can find that with the increase of SNR and N, the orthogonality of noise subspace and signal subspace turns to be more obvious, subspace leakage reduces, and RMSE of all methods gets smaller. Compared with other methods, our method which eliminates the subspace leakage can estimate DOAs more precisely. Because INPS has a good resolution in condition of power difference and closely spaced sources, FBSS-INPS also performs better than traditional FBSS-MUSIC; this conclusion is similar to literatures [14–16].In the third experiment, detected signals coming from the directions ofθ1=−2.5∘, θ2=0∘, and θ3=2.5∘ are considered, the power of signal with θ2=0∘ is constant, and 500 Monte Carlo trials are carried out. The trial is regarded as a successful estimation if the consequence satisfies(18)θ^1−θ1≤θ1−θ22,θ^2−θ2≤θ1−θ22,ifθ^2−θ2≤0,θ^2−θ2≤θ3−θ22,ifθ^2−θ2>0,θ^3−θ3≤θ3−θ22.Probability of resolution versus power difference is depicted in Figure7. When SNR of signals is −5 dB, 0 dB, and −5 dB, respectively, DOA of the signal whose SNR is 0 dB remains constant, and DOAs of other two signals change. Figure 8 shows the probability of resolution versus angle difference.Figure 7
Probability of resolution versus power ratio.Figure 8
Probability of resolution versus angle difference.From Figures7 and 8, probability of resolution suffers from degradation as power difference increases, and decreasing angle difference can lead to similar consequence. Moreover, our method has the best resolution probability compared with other two methods. Two figures clearly indicate the superior capability of our method in resolving closely spaced signals with power difference.To compare the computational complexity of these methods, 500 Monte Carlo simulations under different peak searching steps are carried out, and the total time consumption is listed in Table1. It can be found that smaller searching step can bring more running time. Because our method searches peaks only around approximate values, it saves more time than FBSS-INPS. Compared with estimation methods proposed in literatures [14–16], our method can efficiently process coherent signals, modified covariance matrix can offer more accurate eigenvalues, and running time is largely reduced on the basis of approximate DOAs estimated in priori steps. Moreover, we replace MUSIC with INPS, and our method has better resolution than literature [19] to process signals with adjacent DOAs and power difference.Table 1
Time consumption (unit: second).
Searching stepFBSS + MUSICFBSS + IPNSOur method0.1°17.8562.6332.990.01°58.99507.23101.290.001°509.987023.25918.11


## 5. Conclusions

The passive surveillance system based on troostite must process coherent signals with different power and adjacent DOAs. In this work, to enhance the robustness of the DOA estimator, INPS is carried out to estimate DOAs, the covariance matrix is preprocessed by FBSS, and subspace leakage is eliminated by a two-step scheme. After these procedures, more precise eigenvalues corresponding to noises can be provided for INPS. The approximate DOAs known in advance can largely reduce computational load without performance degradation. Simulation results indicate the validity and superiority of our method.



---

*Source: 6650278-2021-03-05.xml*