# Infrared Fast-Maneuvering Target Tracking Based on Robust Exact Differentiator with Improved Particle Filter

Authors: Wanxin Zhang; Bingrui Huang; Sijie Meng; Jihong Zhu
Journal: Journal of Robotics
 (2022)

---

## Abstract

This paper deals with the problem of nonlinear uncertainties when tracking an infrared dim small target with fast maneuvers. The particle filter (PF)-based methods are mostly considered. The existing improvement methods for the PF can handle the infrared dim small target with the conventional maneuver, the motion state of which changes slowly and in most cases is assumed to be linear. However, when fast maneuvers appear on the target, the PF-based method will soon suffer from particle degeneracy, or even loss of the target. In this paper, a robust exact differentiator (RED)-based particle generating method is proposed to improve the PF. New birth particles are produced by the proposed method, which can keep up with the fast maneuvers, and ensure the particle diversity of the PF, so as to avoid the particle degeneracy and depletion, meanwhile the number of particles required can be decreased. Numerical simulations are conducted, showing that the proposed algorithm has more advantages than the state-of-the-art method.


---

## Body

## 1. Introduction

The target tracking technology estimates and predicts the motion state of the target by establishing a motion model [1, 2] and integrating filtering methods [3], based on the data information obtained from radar, infrared imaging system, or other sensors [4–6]. Traditional target tracking technology adopts the detect-before-track (DBT) method, which directly uses the original observation date obtained from the sensors with low thresholds and mines the data information. Through the integration of observations over time, the signal-to-noise ratio can be improved, so as to achieve the tracking of the target. The DBT method works well under high signal-to-noise ratio, however, cannot handle the target well under low signal-to-noise ratio. The track-before-detect (TBD) method is then introduced by scholars, which can handle weak target with good effects through noncoherent integration [7]. The TBD method searches for the target trajectory from multiple frames of data and accumulates the energy along the trajectory. The key problem of the method is how to obtain the prior information about the target.Fast maneuvering targets, such as vehicles, aircrafts, and missiles, are able to change the motion states with quick speed, or big acceleration in a short time [8], so as to realize the maneuvering motivation, which will bring nonlinear uncertainties to the information about the target [9]. Due to the nonlinear uncertainty of the target model [10, 11], the traditional Kalman filter (KF) [12] is not applicable. Besides, an infrared dim small target in a complex background will also bring difficulties to filters. Infrared image is obtained by making use of the intensity of infrared light of the object. Different objects or different parts of the same object in most cases have different thermal radiation characteristics, such as temperature and emissivity. Through the infrared imaging system, the differences are converted into electrical signals, and then the electrical signals are converted into visible light images. Infrared image is generally dim, with low contrast resolution between the target image and background, and the noise, meanwhile is large, which affects the subsequent processing of the image.In recent years, the particle filter (PF)-based methods have been used to detect and track targets, [13–16] and have performed good results on tracking dim small targets [16, 17]. Different from the KF, the PF can get rid of the limitations of linear and Gaussian assumptions [18]. However, the PF has two major problems. One is the large number of samples required to approximate the posteriori probability density. When the background is complex, the sample size should be huge, which consequently increases the algorithm complexity. The other is the sample depletion. The resampling process abandons particles with low weights, which will cause the loss of diversity and effectiveness in particles after iterations for times, resulting in particle degeneracy and sample depletion. Studies [19–21] show that these two problems of the PF are particular serious on tracking non-Gaussian targets with strong nonlinearities.Aiming at solving the two problems of the PF, this paper proposed a new target tracking method, with which the PF can be ensured to keep up with the change of the target motion when the target is highly maneuvered, and meanwhile the target motion trajectory can be estimated accurately.The rest of this paper is organized as follows: Section2 reviews related works on the PF method and the KF method. Section 3 develops the robust exact differentiator (RED)-based particle generating method and improved PF algorithm. Section 4 discusses experimental results and, finally, Section 5 concludes the paper.


## 2. Related Work

The tracking ability of the filter is usually determined by the reasonable mathematical model of the target motion. However, in the real situation, the motion law of the target will be affected by many factors [22, 23], which causes a mismatch between the target motion and the filtering model. The state of the target, consequently, cannot be estimated accurately. Denote the state vector of the target at time k as xk,k∈N, which consists of the position, the speed, and the acceleration of the target, with size of nx. Denote the observation vector of the target at time k as zk,k∈N with size of nz. Then, the state model and observation model can be described as follows:(1)xk=fkxk−1,uk−1,zk=gkxk,vk,where fk and gk are the nonlinear functions of the state vector, representing the state transition function and the observation function, respectively, uk−1 is the process noise vector of the discrete system, and vk is the observation noise vector.


### 2.1. PF

The PF is a sequential Monte Carlo method, with the idea which makes use of the weighted random samples and the estimations based on these samples to achieve the posterior probability densitypxk|z1:k. zk corresponds to the infrared image measurements obtained from infrared sensors at frame k.Denote the random sample set of the posterior probability densitypxk|z1:k as xki,wkii=1Nk, where xki is the i th particle at frame k, wki is the weight of the particle xki, and Nk is the number of particles. Normalization processing is usually made that ∑iwki=1. The posterior probability density is given as follows:(2)pxk|z1:k≈∑i=1Nkwkiδxk−xki.wki is updated by the following recursive equations [24]:(3)wk|k−1i∝wk−1ipxki|xk−1iqxki|xk−1i,zk,(4)wki∝wk|k−1ipzk|xki,where qxki|xk−1i,zk is the importance density, of which three forms are commonly used [25]:(i)
the prior probabilityqxki|xk−1i,zk=pxki|xk−1i;(ii)
the likelihoodqxki|xk−1i,zk=pzk|xki; and(iii)
the optimal by minimizing the weight varianceqxki|xk−1i,zk=pzk|xkipxki|xk−1i.Denote the weighted particle set with joint state vector asxki,eki,wi=1Nk, where eki=1 indicates that the target exists, and eki=0 indicates that the target is absent. Resample from the previous particle set and newly weighted particle set. Finally, the position of the target can be estimated by the following equation:(5)xk=∑i=1Nkxki⋅eki∑i=1Nkeki.


### 2.2. KF

The KF always makes an assumption that the system can be regarded to be linear and Gaussian, so the model in (1) needs to be simplified as follows:(6)xk=Fk−1xk−1+uk−1,zk=Gkxk+vk,where F is the linearized form of f in (1), representing the state transition matrix, and G is the linearized form of g in (1), representing the observation matrix.u and v are assumed to be mutually independent, with a zero mean and covariance matrices Q and R, respectively,(7)uk∼N0,Qk,vk∼N0,Rk.The KF searches for the optimal solution by minimizing the mean square error,(8)x^opt=argminx^Ex^−xTx^−x.Denote the estimation ofx at frame n given the observations of frame k that k≤m as x^n|m. Denote the error covariance matrix as Pn|m with Pn|m=covxn−x^n|m. The predictions of x^ and P are given as follows:(9)x^k+1|k=Fkx^k|k,Pk+1|k=FkPk|kFkT+Qk.Then(10)x^k+1|k+1=x^k+1|k+Kk+1zk+1−Gkx^k+1|k,Pk+1|k+1=I−Kk+1Gk+1Pk+1|k,where the gain matrix K is given by(11)Kk+1=Pk+1|kGkTGkPk+1|kGkT+Rk−1.


## 2.1. PF

The PF is a sequential Monte Carlo method, with the idea which makes use of the weighted random samples and the estimations based on these samples to achieve the posterior probability densitypxk|z1:k. zk corresponds to the infrared image measurements obtained from infrared sensors at frame k.Denote the random sample set of the posterior probability densitypxk|z1:k as xki,wkii=1Nk, where xki is the i th particle at frame k, wki is the weight of the particle xki, and Nk is the number of particles. Normalization processing is usually made that ∑iwki=1. The posterior probability density is given as follows:(2)pxk|z1:k≈∑i=1Nkwkiδxk−xki.wki is updated by the following recursive equations [24]:(3)wk|k−1i∝wk−1ipxki|xk−1iqxki|xk−1i,zk,(4)wki∝wk|k−1ipzk|xki,where qxki|xk−1i,zk is the importance density, of which three forms are commonly used [25]:(i)
the prior probabilityqxki|xk−1i,zk=pxki|xk−1i;(ii)
the likelihoodqxki|xk−1i,zk=pzk|xki; and(iii)
the optimal by minimizing the weight varianceqxki|xk−1i,zk=pzk|xkipxki|xk−1i.Denote the weighted particle set with joint state vector asxki,eki,wi=1Nk, where eki=1 indicates that the target exists, and eki=0 indicates that the target is absent. Resample from the previous particle set and newly weighted particle set. Finally, the position of the target can be estimated by the following equation:(5)xk=∑i=1Nkxki⋅eki∑i=1Nkeki.


## 2.2. KF

The KF always makes an assumption that the system can be regarded to be linear and Gaussian, so the model in (1) needs to be simplified as follows:(6)xk=Fk−1xk−1+uk−1,zk=Gkxk+vk,where F is the linearized form of f in (1), representing the state transition matrix, and G is the linearized form of g in (1), representing the observation matrix.u and v are assumed to be mutually independent, with a zero mean and covariance matrices Q and R, respectively,(7)uk∼N0,Qk,vk∼N0,Rk.The KF searches for the optimal solution by minimizing the mean square error,(8)x^opt=argminx^Ex^−xTx^−x.Denote the estimation ofx at frame n given the observations of frame k that k≤m as x^n|m. Denote the error covariance matrix as Pn|m with Pn|m=covxn−x^n|m. The predictions of x^ and P are given as follows:(9)x^k+1|k=Fkx^k|k,Pk+1|k=FkPk|kFkT+Qk.Then(10)x^k+1|k+1=x^k+1|k+Kk+1zk+1−Gkx^k+1|k,Pk+1|k+1=I−Kk+1Gk+1Pk+1|k,where the gain matrix K is given by(11)Kk+1=Pk+1|kGkTGkPk+1|kGkT+Rk−1.


## 3. Approach

To handle the maneuvering target, the KF usually makes the assumption of constant acceleration. The state vectorxk is defined as follows:(12)xk=xk,x˙k,x¨k,yk,y˙k,y¨kT,where xk is the x position of the target in the infrared image at frame k, yk is the y position of the target in the infrared image at frame k, x˙k and y˙k are speeds along x-axis and y-axis, and x¨k and y¨k are accelerations along x-axis and y-axis.Based on the assumption of constant acceleration that the acceleration of the target is constant in small time interval [26, 27], the state and observation equations can be obtained as follows:(13)xk=1ΔtΔt2/201Δt001001ΔtΔt2/201Δt001xk−1+uk−1,zk=001000000001xk+vk,where Δt is the time interval between adjacent frames.Considering the situations that the target maneuvers at a fast-varying speed, the assumption of constant acceleration is no longer suitable. This paper uses the robust exact differentiation (RED), proposed by Levant, to deal with the practical real-time differentiation problem and obtain the real-time acceleration of the target [28]. We use x0 to represent the estimation of x-position of the target in the image, x1 is the estimation of speed along x-axis, and x2 is the estimation of acceleration along x-axis. The same goes for y. According to the RED based on the supertwisting algorithm [29], the tracking observation model of x1,x2,x3,y1,y2,y3 is given as follows:(14)x˙0=−λ2Lx1/3x0−xt2/3signx0−xt+x1,x˙1=−λ1Lx2/3x0−xt1/3signx0−xt+x2,x˙2=−λ0Lxsignx0−xt,y˙0=−λ2Ly1/3y0−yt2/3signy0−yt+y1,y˙1=−λ1Ly2/3y0−yt1/3signy0−yt+y2,y˙2=−λ0Lysigny0−yt,where xt,yt are measurable locally bounded functions, having derivatives with Lipschitz constants Lx>0, and Ly>0, λ0,λ1,λ2>0 are the gain parameters.Algorithm 1: Calculating the center of newly generated particles by the proposed method.
Inputs:zk,ΔTOutputs:x^k,x^k+1(1)
Initialization: declare and initialize variablesx0,x1,x2,y0,y1,y2,x^0,P0.(2)
Declare differentiator parameters and filter parametersλ0=1.1, λ1=2.12, λ2=2, η, and ε, Lipschitz constants Lx, Ly, and the covariance matrices Q and R.(3)
while not convergent do(4)
ifzk−x^k∞<η then {Case 1 KF}(5)
According to (10)-(14), updatex^k+1|k, Pk+1|k, x^k+1|k+1, Pk+1|k+1, and Kk+1 in sequence based on the constant acceleration model in (6)(6)
else {Case 2 RED}(7)
x˙0=−λ2Lx1/3x0−zxk2/3signx0−zxk+x1(8)
x˙1=−λ1Lx2/3|x0−zxk|1/3signx0−zxk+x2(9)
x˙2=−λ0Lxsignx0−zxk(10)
y˙0=−λ2Ly1/3y0−zyk2/3signy0−zyk+y1(11)
y˙1=−λ1Ly2/3y0−zyk1/3signy0−zyk+y2(12)
y˙2=−λ0Lysigny0−zyk(13)
Integrate variablesx˙0, x˙1, x˙2, y˙0, y˙1, and y˙2 over ΔT to obtain x0,x1,x2,y0,y1,y2(14)
end if(15)
Check the convergence conditions:x^k−zk∞<ε(16)
end whileApart from the persisting particles, we generate new birth particles according to the estimations obtained from Algorithm1, with the importance density of βx|x^. The weights of the new birth particles are given by [30].(15)wki=bkxkiβkxki|x^k1N^k,i=Nk+1,…,Nk+N^k,where N^k is the number of new birth particles and bkxki is given by the uniform distribution over the state space around the target predictions of Algorithm 1.The flow chart of the proposed algorithm as a whole is shown in Figure1. At the initial stage, the tracking target is specified manually. Then, the features of the target are calculated. At the searching stage, the particles are uniformly assigned. Then, the weights of the particles are updated according to the results of similarity calculation. Based on the updated weights, the efficient number of particles is calculated.(16)Neffk=1∑i=1Nwki2.Figure 1
Flow chart of the proposed algorithm.The efficient number is the indicator to determine whether to generate new particles. Particle resampling process is then conducted. By repeating the abovementioned processes for several times, the most likely place for the target will be calculated. Algorithm2 describes the iteration of the improved PF.Algorithm 2: The improved PF for the fast-maneuvering target.
Inputs:zk, x^k+1Outputs:xk+1(1)
Initialization: declare the particle representationx1,ei,1,w1,eii=1N1e,x1,ai,0,w1,aii=1N1a.(2)
xki,wkii=1Nk=xk,ei,wk,eii=1Nke∪xk,ai,wk,aii=1Nka(3)
fork = 1:n do(4)
xk+1|ki∼ρk+1x|xki,zk+1i=1,…,Nk(5)
xk+1|ki∼βk+1x|x^k+1i=Nk+1,…,Nk+N^k(6)
end for(7)
Computewki, i=1,…,Nk using (3) and (4).(8)
Computewki, i=Nk+1,…,Nk+N^k using (15).(9)
Computep=Nke/Nk, p^=N^ke/N^k.(10)
Calculate the number of persistent particles in the next stepN˜k+1=⌈Nk+1p/p+p^⌉.(11)
ResampleN˜k+1 times from xki,wkii=1Nk, N˜k+1−N˜k+1 times from xki,wkii=Nk+1Nk+N^k.(12)
Normalize the weights of theNK+1 particles.(13)
Computexk+1 using (5).


## 4. Experiments

The experiments adopt the dataset download fromhttps://www.dx.doi.org/10.11922/sciencedb.902 [31]. The dataset contains image sequences for tracking low altitude flying dim small targets. Fast maneuvers are also included in some of the image sequences.Image preprocessing is firstly made. To reduce the noise, the images are improved by enhancing the information required and weakening the information unwanted. The original small range of gray value is broadened. Then, image segmentation is carried out to make the target separated from the background.We compare the proposed method with the basic PF and KF-PF. For the convenience of intuitive understanding, we name our method as RED-KF-PF. The three methods give the same parameter settings for KF and PF in the initialization step so as to guarantee the fairness. The comparison results of RMSE are listed in Table1. The RMSE is the square root of the mean square error, which is given as follows:(17)RMSEx=1Nk∑i=1Nkxki−x^k2.Table 1
Comparison of RMSE.
ParticlePFKF-PFRED-KF-PFN = 100x0.34540.16660.1437y0.14870.07190.0530N = 500x0.25620.12360.0971y0.08850.04270.0359N = 1000x0.10140.08490.0661y0.03730.02310.0124The comparison results of running time are listed in Table2. According to Tables 1 and 2, higher accuracy with less particles and less time is achieved by the proposed method.Table 2
Comparison of running time.
ParticlePFKF-PFRED-KF-PFN = 1006.4831s12.5712s12.4358sN = 50025.0614s48.5951s45.9103sN = 100050.5887s95.1003s94.1379sBy choosing the dataset with fast-maneuvering test, Figure2 demonstrates the tracking process with the proposed algorithm. By choosing the close-range test and the distant-range test, Figures 3 and 4 demonstrate the detection results and particle distributions. The target takes slow motions, so the particle distributions did not show apparent characteristics of the proposed idea. As in Figure 5, in which a quick maneuver is taken by the target, the particles with great importance mainly come from the RED-based particle generating method. If not, the green dot in Figure 5(a) calculated from the persistent particles would influence the tracking result heavily.Figure 2
Tracking results of the fast-maneuvering target. The white dot is the target. The blue dot is the estimation of the trajectory. The red dots are particles. (a) The 400th frame. (b) The 410th frame. (c) The 420th frame. (d) The 430th frame. (e) The 440th frame. (f) The 450th frame. (g) The 460th frame. (h) The 470th frame. (i) The 480th frame.
(a)(b)(c)(d)(e)(f)(g)(h)(i)Figure 3
Infrared target characteristics detection. (a) Close target detection. (b) Particle distributions before and after resampling.
(a)(b)Figure 4
Dim-small object detection. (a) Distant target detection. (b) Particle distributions before and after resampling.
(a)(b)Figure 5
Fast-maneuvering dim-small target tracking. (a) Fast-maneuvering target detection. (b) Particle distributions before and after resampling.
(a)(b)


## 5. Conclusion

To improve the distribution of the particles, the KF with the constant acceleration model is considered to generate particles around the estimation center. In the cases the KF is not capable to track the motions of the target, the RED is constructed to update the estimation center. The generated particles not only improve the diversity of particles but also make the resampled particles much closer to the real target state, so that the PF can approximate the posterior probability density better. Method for modification of the Lipschitz constantsLx and Ly in real time is considered for future work.



---

*Source: 6406672-2022-10-21.xml*