# Learning to Track Multiple Radar Targets with Long Short-Term Memory Networks

Authors: Fei Song; Yong Li; Wei Cheng; Limeng Dong
Journal: Wireless Communications and Mobile Computing
 (2023)

---

## Abstract

Radar multitarget tracking in a dense clutter environment remains a complex problem to be solved. Most existing solutions still rely on complex motion models and prior distribution knowledge. In this paper, a new online tracking method based on a long short-term memory (LSTM) network is proposed. It combines state prediction, measurement association, and trajectory management functions in an end-to-end manner. We employ LSTM networks to model target motion and trajectory associations, relying on their strong learning ability to learn target motion properties and long-term dependence of trajectory associations from noisy data. Moreover, to address the problem of missing appearance information of radar targets, we propose an architecture based on the LSTM network to calculate similarity function by extracting long-term motion features. And the similarity is applied to trajectory associations to improve their robustness. Our proposed method is validated in simulation scenarios and achieves good results.


---

## Body

## 1. Introduction

In an environment with much clutter, multitarget tracking based on radar detection results is a challenging task. An important branch of its solution is the multitarget tracking algorithm based on data association, which can be generally decomposed into the following three aspects: modeling the motion of the tracked target to estimate trajectory parameters and filter measurements, associating the measurements with the motion trajectory to distinguish the measurements from different targets or background noise and clutter interference, and effectively managing different motion trajectories to determine the birth, retention identity, and termination of different target trajectories.For modeling the motion of the tracked target, the traditional method is based on the Bayesian filtering theory [1, 2]. Earlier, the Kalman filter is used to obtain unbiased optimal estimation of Gaussian linear targets. For nonlinear motion targets, the following developed methods consist of extended Kalman filter (EKF) [3], unsensitive Kalman filter (UKF) [4], interactive multiple model (IMM) [5] algorithm, and so on. All of these methods require limited motion models in advance. Particle filtering (PF) [6], which does not need the aforementioned assumptions, however, has low computational efficiency and sample poverty problem.For tracking trajectory association and trajectory management, it can theoretically be regarded as a maximum matching problem of a bipartite graph. For this problem, the Hungarian algorithm (HA) [7] and Kuhn and Munkres algorithm (KM) are first used. Later, many more complex algorithms have appeared, including multiple hypothesis tracker (MHT) [8], which establishes a potential tracking hypothesis tree for each candidate target and calculates the tracking probability to select the most likely tracking combination. Another popular technique is joint probabilistic data association (JPDA) [9]. By building a validation matrix, it can calculate all feasible joint association event probabilities and associate targets based on the obtained scores. However, these traditional methods not only require knowing the number of targets, starting positions, and clutter distribution but also result in “combination explosions” increase of calculation when the number of targets or clutter is enlarged. This will not only reduce the tracking accuracy but also may lead to trace association errors.Recently, deep learning has made great progress in classification and detection applications in many fields. For example, it is widely used in identifying interference in wireless communication systems [10] and lightweight radio frequency fingerprint identification (RFFID) systems [11]. Especially in computer vision, the target tracking methods in video based on deep learning have also improved, such as pedestrian monitoring [12, 13], car driving monitoring [14, 15], and biological sequence tracking [16]. However, deep learning-based multitarget tracking algorithms that are applied to radar targets are relatively rare. This may be caused by several reasons. First, deep learning requires a large amount of annotated training data, but real data is difficult to obtain in radar multitarget scenarios. Secondly, the measurement of radar targets nearly has no apparent information, which prevents the appearance detection information commonly used in video target tracking. Thirdly, radar target tracking and detection have real-time requirements, and the tracking algorithm generally needs to be used online.The main contributions of this paper are as follows:(1)
To overcome these problems in traditional radar multitarget tracking based on data association, inspired by deep learning ideas, we propose an end-to-end multitarget tracking structure based on recurrent neural networks. We use LSTM networks to model the state prediction and measurement association parts of the multitarget tracking algorithm, respectively. Relying on the powerful learning ability of LSTM networks, it is able to obtain the long-term dependence of multiple tracking target motion states and measurement association by learning from a large amount of training data. Then, it combines status update and tracking management into a unified network structure and finally realizes multitarget tracking. The advantages of this method are that the target motion models and clutter distribution are not required to be known in advance, and the combination explosion problem in the traditional data association method is alleviated. Meanwhile, the tracking process runs online without caching any future frame information(2)
To solve the problem of missing appearance information when the neural network is applied to the radar multitarget tracking algorithm, we creatively design a motion feature extraction LSTM to extract motion features from the speed of the targets and calculate the similarity scores, which are used to learn long-dependent measurement associations(3)
To address the problem of insufficient training data, we propose a method to obtain extensive training data from simulation models. Our multitarget tracking architecture is also confirmed by a simulation analysis of the simulated data


## 2. Related Work

Lately, target tracking in the video application field has been extended from a single target to multitarget tracking [17]. And the specific methods have also been extended from constructing complex appearance models to focusing on the motion model and interaction of the targets simultaneously [18]. In these applications, deep learning methods play an important role, mainly reflected in the following three aspects:
(1)
Using the deep neural network to extract high-order features of the tracking targets, especially for the appearance model, this can effectively improve the target tracking performance. Earlier, Reid [8] use AlexNet for extracting deep features from targets for the use of the MHT framework. Leal-Taixe et al. [19] use the Siamese network structure to extract deep features. By combining the depth features and motion information with a gradient enhancement algorithm, the tracking problem is expressed as linear programming and solved effectively. However, there is no apparent information in the radar signal, so these methods are not suitable for extracting the depth features of the radar signal(2)
Deep neural networks are used to directly learn key components of a multitarget tracking framework. Chen et al. [20] construct two CNN-based classifiers using the features of fasterRCNN [21] from the VGG-16 model [22] as input and combining the confidence degree of the classifier as particle weights to obtain tracking results by particle filtering. Xiang et al. [23] propose a CNN based on triplet state loss to learn the distance measure between tracker and detection and thus constitute the cost of the bipartite graph, which can be effectively solved by the Hungarian algorithm. However, radar targets are different from video targets. It often has a poor performance to extract radar target features and classify them using the CNN network(3)
Deep neural networks are designed directly in an end-to-end manner to obtain multitarget tracking results. The multitarget tracking task involves many interwoven components and is difficult to be modeled as a whole for learning. There have been some preliminary researches recently. Milan et al. [24] unify target state prediction, state update, and existence probability calculation under a whole RNN. Also, they design a set of LSTM networks for data matching matrix calculation and act on the state update process. These architectures achieved good results. Sadeghian et al. [25] design a more complex RNN structure, which is divided into three subnetworks to extract multiframe appearance features, motion features, and interaction features, respectively, and integrate them into a top-level RNN for time-dependent comprehensive inferring, so as to obtain the final matching probability. Experimental results show that this method is more robustOur architecture is similar to that of [24] but different in three key aspects: First, RNN is used in [24] to deal with motion state prediction. If the time interval is too long, the time dependence will be weakened. Instead, we use the LSTM network for motion state prediction, which enhances the long-term dependence learning ability of target states. Secondly, in the measurement association part, we use the motion feature extraction network based on LSTM to calculate the similarities for data associations, which has stronger anti-interference ability than the method in [24] only using the Euclidean distance between the target and each measurement to carry out data associations. Thirdly, we use different loss functions due to different network architectures.


## 3. Our Approach

For addressing the aforementioned radar target tracking problem, we propose an online end-to-end radar multitarget tracking network architecture based on LSTM (see Figure1). Our architecture implements target state prediction, measurement association, state update, and tracking path management under a unified LSTM network architecture, and all model learning is completed in an online end-to-end manner.Figure 1
Overview of our architecture. In a time stept, the architecture is composed of two parts as target motion module and target association module (see Section 3.2 for details). The input can be expressed as the following: at the current moment t, the target motion state xt, the hidden state ht, and the probability vector ηt (see below) come from the learning of the previous moment t−1. After the processing of our architecture, the output is the estimated target motion state xt+1, the hidden state ht+1, and the probability vector ηt+1 of the next moment t+1.


### 3.1. Notations

In our application scenario, the target state vector we care about isxt∈RN⋅D, including x coordinate, y coordinate, x axis velocity vx, and y axis velocity vy, as represented x,y,vx,vy, D=4, N represents the maximum number of targets tracked simultaneously in a frame. The measurement value vector is zt∈RM⋅D, whose dimension D is the same as the target state vector, and M represents the maximum number of detected values in a frame, including the targets and clutter. The association probability matrix of the measurement association part is expressed as At∈0,1N⋅M+1, each row of which represents the probability that each measurement value belongs to a certain target. That is, Atij presents the probability of the jth measurement value belonging to the ith target. The added column indicates the probability that the target lacks measurement, so that each row is satisfied ∀i:∑jAtij=1. The path management section uses a probability vector ηt∈0,1N to express the probability of each target’s existence, which can accurately describe the target generation and termination processes.


### 3.2. Motion-Association Multitarget Tracking with LSTMs (MA-LSTM)

In our architectural design (see Figure1), we partition the state prediction, state update, and path management into a target motion module and measurement association into another target association module. This primarily deals with the diversity of measurement association algorithms and facilitates the replacement of this part for tracking effectiveness comparison. At the same time, the two modules are trained separately for easy convergence.


#### 3.2.1. State Prediction in Motion Module

State prediction is the first submodule of the motion module. The task of this module is to learn a complex motion pattern, which can be noisy and nonlinear, and predict the future motion states based on the trend of the past motion states. This is actually a time series prediction problem, which we designed to achieve by using a LSTM network structure, as shown in Figure2. We train the network on noisy motion trajectories so that the network can completely learn the long-term dependent motion pattern from the training data without prior knowledge.Figure 2
Prediction submodule of motion module on the basis of LSTM is used to estimate target state without the help of measurement.More specifically, at the current momentt, both the hidden state ht and the cell state ct of the prediction LSTM come from the learning of the previous target motion state, and the input xt is the target motion state at the current moment. Through the internal processing of the LSTM network, as described below, the predicted motion state x^t+1 at the next moment as well as the hidden state ht+1 and cell state ct+1 can be obtained to continue the transmission.The formula is expressed as follows:(1)h′=Whh′ht+Wxh′xt,ct+1=σh′ct+σh′tanhh′,ht+1=σh′tanhct+1,x^t+1=Whoht+1,where σ is the sigmoid and tanh is the hyperbolic tangent activation functions, and the learnable parameters are expressed as Whh′Wxh′Who.


#### 3.2.2. State Update and Track Management in Motion Module

Status update and track management is the second submodule of the motion module. The task of this module is to update the motion state of the target and identify and judge the start and end of the target trajectory on the basis of the obtained target prediction, measured values, and association matrix. This is an important step in the multitarget tracking architecture, for taking into account the clutter interference in the multitarget measurements and the association of the measured data. The network structure we designed is shown in Figure3.Figure 3
Update and management submodule of motion module is used to update target state and identify and manage the target trajectory with the help of the known target state prediction, measured values, and association matrix.In detail, the measured valueszt+1 at t+1 moment are combined with the predicted values x^t+1 output by the state prediction submodule, as x^t+1;zt+1. Then, using the dot product with the association matrix At+1 output by the target association module, the possible state of the target is x^t+1;zt+1⋅At+1. Then, it is multiplied by the target existence probability ηt. With the multiplied result and the hidden state of the prediction module ht+1, the updated motion state xt+1∗ is obtained after a nonlinear transformation, and the target existence probability ηt+1∗ is also calculated.The formula is expressed as follows:(2)x~t+1=x^t+1;zt+1⋅At+1,h~t+1=ht+1+Wxh~x~t+1ηt,xt+1∗=Wx∗otanhh~t+1,ηt+1∗=σWη∗otanhh~t+1,where σ is the sigmoid and tanh is the hyperbolic tangent activation functions, and the learnable parameters are expressed as Wxh~Wx∗oWη∗o.


#### 3.2.3. Loss of Motion Module

In our architecture, we are interested in tracking performance-related losses and propose the following loss functions that meet our needs:(3)Lx^t+1,xt+1∗,ηt+1∗,xt+1,ηt+1=α∑x∧t+1−xt+12+β∑xt+1∗−xt+12+μLη+ψη¯t+1,(4)Lηηt+1∗,ηt+1=ηt+1logηt+1∗+1−ηt+1log1−ηt+1∗,(5)η¯t+1=absηt+1∗−ηt,where x^t+1 is the predicted value of the prediction module, xt+1∗ is the predicted value of the update module, and ηt+1∗ is the predicted value of the existence probability. xt+1, ηt+1 are true values.This loss function in formula (3) we designed consists of four parts. The first part is to predict the motion state x^t+1 of the targets without measured values, and we take the mean square error (MSE) of the predicted values and the real values. The second part is the state prediction xt+1∗ after the measurement update. We take the mean square error of the updated values and the real values. The third part is to predict the probability of the target’s existence ηt+1∗ at each moment, and we use binary cross entropy loss function, as shown in formula (4). The fourth part is the additional smoothing variable η¯t+1, in order to smooth the absolute difference between two consecutive η values, so as to prevent the loss of the target in the tracking process.


#### 3.2.4. Target Association Module

The target association module is the most challenging and creative part of the multitarget tracking architecture. We design it as an independent association module and further subdivide it into two parts: motion extraction LSTM and association LSTM.The fundamental task of target association is to uniquely partition the corresponding measurement values for each tracking target in a clutter interference environment, which is essentially a maximum matching problem of a binary graph. Different from the traditional solution mentioned above, we construct the association LSTM in an end-to-end manner. Relying on the powerful memory learning ability of LSTM, by learning and remembering the long-term dependence of target and measurement associations from abundant data, the association probability of each target to all measurement values is predicted, and finally, the association matrix of all targets is obtained. Our association method satisfies the one-to-one association constraint and is online without viewing future frames. This network structure is shown in Figure4.Figure 4
The association LSTM submodule of association module is used to learn the long-term dependence of targets and measurement association. The inputpM are motion eigenvectors from the extraction LSTM.To be more specific, at timet, its input is feature vector pM from the extraction LSTM (see below), and the hidden state ht and cell state ct both come from the learning of the previous target assignment. Through the LSTM network, we can get a target distribution probability vector At+1i for all available measurements at the last softmax layer, and multiple targets constitute the association matrix At+1.The input of the association LSTM requires eigenvectors representing similarity measures. The traditional similarity measure function solutions are handmade, and as aforementioned, radar target appearance feature information is absent. In order to solve these problems, we put forward the motion extraction LSTM, an architecture of computing similarity function based on the LSTM network (see Figure5). Through the long-term time-dependent learning of the target motion velocity features, the similarity score of the target and the measurement can be calculated end-to-end without manually specifying parameters or weights, so as to determine whether the new measurement value is similar to the target motion features in the previous period of time.Figure 5
The extraction LSTM submodule of association module is used to extract the velocity features of the target’s long-term motion and calculate the similarity between the new measurement and the existing trajectory.As shown in Figure5, we designed the motion extraction LSTM. The input v1i⋯vti are the motion velocity features of the ith tracked target at the specified time step 1⋯t, producing the H-dimensional output pi after processing through LSTM networks. The other input vt+1j is the jth measured velocity vector at the t+1 time step, which we map to an H-dimensional vector pj via an FC layer. The two vectors are then connected and passed to another FC layer, which transforms the 2H-dimensional vector into a K-dimensional eigenvector pM. In the pretraining process, we use the softmax classifier at the end to train our model parameters to judge whether the measured velocity feature vt+1j corresponds to the real trajectory’s velocity feature v1i⋯vti.


#### 3.2.5. Loss of Association Module

For motion extraction LSTM, we use the classifier and cross-entropy loss function for pretraining.For association LSTM, in order to measure the cost of inappropriate associations, we adopt the common negative log likelihood loss, as below:(6)LAt+1i,mt+1i=−logAt+1im,where mt+1i is the correct assignment of target i and At+1ij is the probability of measurement j assigned to target i at time t+1.


### 3.3. Implementation Details

We use the TensorFlow architecture to implement our design. We designed the state prediction LSTM with a single layer of 256 hidden cells. The measurement of association task requires more representation ability. The association LSTM uses 256 hidden units in two layers, and the motion extraction LSTM uses 128 hidden units in one layer. The output feature vector is 100 dimensions, and the length of the specific extracted sequence is 10. We use grid search to select the optimal network hyperparameters [26]. We use Adam to update and optimize our framework. The learning rate is initially set at 0.001 decreasing by 10% for every 10 periods. We set the maximum number of iterations to 10,000, which is enough to achieve convergence. Training for these network architectures takes about 10 hours on a GPU.


#### 3.3.1. Training Data

As mentioned above, deep learning network learning requires a large amount of training data. However, there are very few open marker datasets for multitarget tracking of radar signals. Therefore, we generate training data from radar motion simulation models. The training trajectories we generated are set with multiple targets, whose birth time and lifetime are random, and incorporate a large number of random uniformly distributed clutter. The initial position and velocity of each target are randomly distributed within a certain range, and the conventional constant-velocity motion model and constant acceleration motion model of the radar target are adopted. The radar measurements are sampled every 2 seconds, and a Gaussian measurement noise is added, while the annotations for multitarget associations are manually added.


## 3.1. Notations

In our application scenario, the target state vector we care about isxt∈RN⋅D, including x coordinate, y coordinate, x axis velocity vx, and y axis velocity vy, as represented x,y,vx,vy, D=4, N represents the maximum number of targets tracked simultaneously in a frame. The measurement value vector is zt∈RM⋅D, whose dimension D is the same as the target state vector, and M represents the maximum number of detected values in a frame, including the targets and clutter. The association probability matrix of the measurement association part is expressed as At∈0,1N⋅M+1, each row of which represents the probability that each measurement value belongs to a certain target. That is, Atij presents the probability of the jth measurement value belonging to the ith target. The added column indicates the probability that the target lacks measurement, so that each row is satisfied ∀i:∑jAtij=1. The path management section uses a probability vector ηt∈0,1N to express the probability of each target’s existence, which can accurately describe the target generation and termination processes.


## 3.2. Motion-Association Multitarget Tracking with LSTMs (MA-LSTM)

In our architectural design (see Figure1), we partition the state prediction, state update, and path management into a target motion module and measurement association into another target association module. This primarily deals with the diversity of measurement association algorithms and facilitates the replacement of this part for tracking effectiveness comparison. At the same time, the two modules are trained separately for easy convergence.


### 3.2.1. State Prediction in Motion Module

State prediction is the first submodule of the motion module. The task of this module is to learn a complex motion pattern, which can be noisy and nonlinear, and predict the future motion states based on the trend of the past motion states. This is actually a time series prediction problem, which we designed to achieve by using a LSTM network structure, as shown in Figure2. We train the network on noisy motion trajectories so that the network can completely learn the long-term dependent motion pattern from the training data without prior knowledge.Figure 2
Prediction submodule of motion module on the basis of LSTM is used to estimate target state without the help of measurement.More specifically, at the current momentt, both the hidden state ht and the cell state ct of the prediction LSTM come from the learning of the previous target motion state, and the input xt is the target motion state at the current moment. Through the internal processing of the LSTM network, as described below, the predicted motion state x^t+1 at the next moment as well as the hidden state ht+1 and cell state ct+1 can be obtained to continue the transmission.The formula is expressed as follows:(1)h′=Whh′ht+Wxh′xt,ct+1=σh′ct+σh′tanhh′,ht+1=σh′tanhct+1,x^t+1=Whoht+1,where σ is the sigmoid and tanh is the hyperbolic tangent activation functions, and the learnable parameters are expressed as Whh′Wxh′Who.


### 3.2.2. State Update and Track Management in Motion Module

Status update and track management is the second submodule of the motion module. The task of this module is to update the motion state of the target and identify and judge the start and end of the target trajectory on the basis of the obtained target prediction, measured values, and association matrix. This is an important step in the multitarget tracking architecture, for taking into account the clutter interference in the multitarget measurements and the association of the measured data. The network structure we designed is shown in Figure3.Figure 3
Update and management submodule of motion module is used to update target state and identify and manage the target trajectory with the help of the known target state prediction, measured values, and association matrix.In detail, the measured valueszt+1 at t+1 moment are combined with the predicted values x^t+1 output by the state prediction submodule, as x^t+1;zt+1. Then, using the dot product with the association matrix At+1 output by the target association module, the possible state of the target is x^t+1;zt+1⋅At+1. Then, it is multiplied by the target existence probability ηt. With the multiplied result and the hidden state of the prediction module ht+1, the updated motion state xt+1∗ is obtained after a nonlinear transformation, and the target existence probability ηt+1∗ is also calculated.The formula is expressed as follows:(2)x~t+1=x^t+1;zt+1⋅At+1,h~t+1=ht+1+Wxh~x~t+1ηt,xt+1∗=Wx∗otanhh~t+1,ηt+1∗=σWη∗otanhh~t+1,where σ is the sigmoid and tanh is the hyperbolic tangent activation functions, and the learnable parameters are expressed as Wxh~Wx∗oWη∗o.


### 3.2.3. Loss of Motion Module

In our architecture, we are interested in tracking performance-related losses and propose the following loss functions that meet our needs:(3)Lx^t+1,xt+1∗,ηt+1∗,xt+1,ηt+1=α∑x∧t+1−xt+12+β∑xt+1∗−xt+12+μLη+ψη¯t+1,(4)Lηηt+1∗,ηt+1=ηt+1logηt+1∗+1−ηt+1log1−ηt+1∗,(5)η¯t+1=absηt+1∗−ηt,where x^t+1 is the predicted value of the prediction module, xt+1∗ is the predicted value of the update module, and ηt+1∗ is the predicted value of the existence probability. xt+1, ηt+1 are true values.This loss function in formula (3) we designed consists of four parts. The first part is to predict the motion state x^t+1 of the targets without measured values, and we take the mean square error (MSE) of the predicted values and the real values. The second part is the state prediction xt+1∗ after the measurement update. We take the mean square error of the updated values and the real values. The third part is to predict the probability of the target’s existence ηt+1∗ at each moment, and we use binary cross entropy loss function, as shown in formula (4). The fourth part is the additional smoothing variable η¯t+1, in order to smooth the absolute difference between two consecutive η values, so as to prevent the loss of the target in the tracking process.


### 3.2.4. Target Association Module

The target association module is the most challenging and creative part of the multitarget tracking architecture. We design it as an independent association module and further subdivide it into two parts: motion extraction LSTM and association LSTM.The fundamental task of target association is to uniquely partition the corresponding measurement values for each tracking target in a clutter interference environment, which is essentially a maximum matching problem of a binary graph. Different from the traditional solution mentioned above, we construct the association LSTM in an end-to-end manner. Relying on the powerful memory learning ability of LSTM, by learning and remembering the long-term dependence of target and measurement associations from abundant data, the association probability of each target to all measurement values is predicted, and finally, the association matrix of all targets is obtained. Our association method satisfies the one-to-one association constraint and is online without viewing future frames. This network structure is shown in Figure4.Figure 4
The association LSTM submodule of association module is used to learn the long-term dependence of targets and measurement association. The inputpM are motion eigenvectors from the extraction LSTM.To be more specific, at timet, its input is feature vector pM from the extraction LSTM (see below), and the hidden state ht and cell state ct both come from the learning of the previous target assignment. Through the LSTM network, we can get a target distribution probability vector At+1i for all available measurements at the last softmax layer, and multiple targets constitute the association matrix At+1.The input of the association LSTM requires eigenvectors representing similarity measures. The traditional similarity measure function solutions are handmade, and as aforementioned, radar target appearance feature information is absent. In order to solve these problems, we put forward the motion extraction LSTM, an architecture of computing similarity function based on the LSTM network (see Figure5). Through the long-term time-dependent learning of the target motion velocity features, the similarity score of the target and the measurement can be calculated end-to-end without manually specifying parameters or weights, so as to determine whether the new measurement value is similar to the target motion features in the previous period of time.Figure 5
The extraction LSTM submodule of association module is used to extract the velocity features of the target’s long-term motion and calculate the similarity between the new measurement and the existing trajectory.As shown in Figure5, we designed the motion extraction LSTM. The input v1i⋯vti are the motion velocity features of the ith tracked target at the specified time step 1⋯t, producing the H-dimensional output pi after processing through LSTM networks. The other input vt+1j is the jth measured velocity vector at the t+1 time step, which we map to an H-dimensional vector pj via an FC layer. The two vectors are then connected and passed to another FC layer, which transforms the 2H-dimensional vector into a K-dimensional eigenvector pM. In the pretraining process, we use the softmax classifier at the end to train our model parameters to judge whether the measured velocity feature vt+1j corresponds to the real trajectory’s velocity feature v1i⋯vti.


### 3.2.5. Loss of Association Module

For motion extraction LSTM, we use the classifier and cross-entropy loss function for pretraining.For association LSTM, in order to measure the cost of inappropriate associations, we adopt the common negative log likelihood loss, as below:(6)LAt+1i,mt+1i=−logAt+1im,where mt+1i is the correct assignment of target i and At+1ij is the probability of measurement j assigned to target i at time t+1.


## 3.2.1. State Prediction in Motion Module

State prediction is the first submodule of the motion module. The task of this module is to learn a complex motion pattern, which can be noisy and nonlinear, and predict the future motion states based on the trend of the past motion states. This is actually a time series prediction problem, which we designed to achieve by using a LSTM network structure, as shown in Figure2. We train the network on noisy motion trajectories so that the network can completely learn the long-term dependent motion pattern from the training data without prior knowledge.Figure 2
Prediction submodule of motion module on the basis of LSTM is used to estimate target state without the help of measurement.More specifically, at the current momentt, both the hidden state ht and the cell state ct of the prediction LSTM come from the learning of the previous target motion state, and the input xt is the target motion state at the current moment. Through the internal processing of the LSTM network, as described below, the predicted motion state x^t+1 at the next moment as well as the hidden state ht+1 and cell state ct+1 can be obtained to continue the transmission.The formula is expressed as follows:(1)h′=Whh′ht+Wxh′xt,ct+1=σh′ct+σh′tanhh′,ht+1=σh′tanhct+1,x^t+1=Whoht+1,where σ is the sigmoid and tanh is the hyperbolic tangent activation functions, and the learnable parameters are expressed as Whh′Wxh′Who.


## 3.2.2. State Update and Track Management in Motion Module

Status update and track management is the second submodule of the motion module. The task of this module is to update the motion state of the target and identify and judge the start and end of the target trajectory on the basis of the obtained target prediction, measured values, and association matrix. This is an important step in the multitarget tracking architecture, for taking into account the clutter interference in the multitarget measurements and the association of the measured data. The network structure we designed is shown in Figure3.Figure 3
Update and management submodule of motion module is used to update target state and identify and manage the target trajectory with the help of the known target state prediction, measured values, and association matrix.In detail, the measured valueszt+1 at t+1 moment are combined with the predicted values x^t+1 output by the state prediction submodule, as x^t+1;zt+1. Then, using the dot product with the association matrix At+1 output by the target association module, the possible state of the target is x^t+1;zt+1⋅At+1. Then, it is multiplied by the target existence probability ηt. With the multiplied result and the hidden state of the prediction module ht+1, the updated motion state xt+1∗ is obtained after a nonlinear transformation, and the target existence probability ηt+1∗ is also calculated.The formula is expressed as follows:(2)x~t+1=x^t+1;zt+1⋅At+1,h~t+1=ht+1+Wxh~x~t+1ηt,xt+1∗=Wx∗otanhh~t+1,ηt+1∗=σWη∗otanhh~t+1,where σ is the sigmoid and tanh is the hyperbolic tangent activation functions, and the learnable parameters are expressed as Wxh~Wx∗oWη∗o.


## 3.2.3. Loss of Motion Module

In our architecture, we are interested in tracking performance-related losses and propose the following loss functions that meet our needs:(3)Lx^t+1,xt+1∗,ηt+1∗,xt+1,ηt+1=α∑x∧t+1−xt+12+β∑xt+1∗−xt+12+μLη+ψη¯t+1,(4)Lηηt+1∗,ηt+1=ηt+1logηt+1∗+1−ηt+1log1−ηt+1∗,(5)η¯t+1=absηt+1∗−ηt,where x^t+1 is the predicted value of the prediction module, xt+1∗ is the predicted value of the update module, and ηt+1∗ is the predicted value of the existence probability. xt+1, ηt+1 are true values.This loss function in formula (3) we designed consists of four parts. The first part is to predict the motion state x^t+1 of the targets without measured values, and we take the mean square error (MSE) of the predicted values and the real values. The second part is the state prediction xt+1∗ after the measurement update. We take the mean square error of the updated values and the real values. The third part is to predict the probability of the target’s existence ηt+1∗ at each moment, and we use binary cross entropy loss function, as shown in formula (4). The fourth part is the additional smoothing variable η¯t+1, in order to smooth the absolute difference between two consecutive η values, so as to prevent the loss of the target in the tracking process.


## 3.2.4. Target Association Module

The target association module is the most challenging and creative part of the multitarget tracking architecture. We design it as an independent association module and further subdivide it into two parts: motion extraction LSTM and association LSTM.The fundamental task of target association is to uniquely partition the corresponding measurement values for each tracking target in a clutter interference environment, which is essentially a maximum matching problem of a binary graph. Different from the traditional solution mentioned above, we construct the association LSTM in an end-to-end manner. Relying on the powerful memory learning ability of LSTM, by learning and remembering the long-term dependence of target and measurement associations from abundant data, the association probability of each target to all measurement values is predicted, and finally, the association matrix of all targets is obtained. Our association method satisfies the one-to-one association constraint and is online without viewing future frames. This network structure is shown in Figure4.Figure 4
The association LSTM submodule of association module is used to learn the long-term dependence of targets and measurement association. The inputpM are motion eigenvectors from the extraction LSTM.To be more specific, at timet, its input is feature vector pM from the extraction LSTM (see below), and the hidden state ht and cell state ct both come from the learning of the previous target assignment. Through the LSTM network, we can get a target distribution probability vector At+1i for all available measurements at the last softmax layer, and multiple targets constitute the association matrix At+1.The input of the association LSTM requires eigenvectors representing similarity measures. The traditional similarity measure function solutions are handmade, and as aforementioned, radar target appearance feature information is absent. In order to solve these problems, we put forward the motion extraction LSTM, an architecture of computing similarity function based on the LSTM network (see Figure5). Through the long-term time-dependent learning of the target motion velocity features, the similarity score of the target and the measurement can be calculated end-to-end without manually specifying parameters or weights, so as to determine whether the new measurement value is similar to the target motion features in the previous period of time.Figure 5
The extraction LSTM submodule of association module is used to extract the velocity features of the target’s long-term motion and calculate the similarity between the new measurement and the existing trajectory.As shown in Figure5, we designed the motion extraction LSTM. The input v1i⋯vti are the motion velocity features of the ith tracked target at the specified time step 1⋯t, producing the H-dimensional output pi after processing through LSTM networks. The other input vt+1j is the jth measured velocity vector at the t+1 time step, which we map to an H-dimensional vector pj via an FC layer. The two vectors are then connected and passed to another FC layer, which transforms the 2H-dimensional vector into a K-dimensional eigenvector pM. In the pretraining process, we use the softmax classifier at the end to train our model parameters to judge whether the measured velocity feature vt+1j corresponds to the real trajectory’s velocity feature v1i⋯vti.


## 3.2.5. Loss of Association Module

For motion extraction LSTM, we use the classifier and cross-entropy loss function for pretraining.For association LSTM, in order to measure the cost of inappropriate associations, we adopt the common negative log likelihood loss, as below:(6)LAt+1i,mt+1i=−logAt+1im,where mt+1i is the correct assignment of target i and At+1ij is the probability of measurement j assigned to target i at time t+1.


## 3.3. Implementation Details

We use the TensorFlow architecture to implement our design. We designed the state prediction LSTM with a single layer of 256 hidden cells. The measurement of association task requires more representation ability. The association LSTM uses 256 hidden units in two layers, and the motion extraction LSTM uses 128 hidden units in one layer. The output feature vector is 100 dimensions, and the length of the specific extracted sequence is 10. We use grid search to select the optimal network hyperparameters [26]. We use Adam to update and optimize our framework. The learning rate is initially set at 0.001 decreasing by 10% for every 10 periods. We set the maximum number of iterations to 10,000, which is enough to achieve convergence. Training for these network architectures takes about 10 hours on a GPU.


### 3.3.1. Training Data

As mentioned above, deep learning network learning requires a large amount of training data. However, there are very few open marker datasets for multitarget tracking of radar signals. Therefore, we generate training data from radar motion simulation models. The training trajectories we generated are set with multiple targets, whose birth time and lifetime are random, and incorporate a large number of random uniformly distributed clutter. The initial position and velocity of each target are randomly distributed within a certain range, and the conventional constant-velocity motion model and constant acceleration motion model of the radar target are adopted. The radar measurements are sampled every 2 seconds, and a Gaussian measurement noise is added, while the annotations for multitarget associations are manually added.


## 3.3.1. Training Data

As mentioned above, deep learning network learning requires a large amount of training data. However, there are very few open marker datasets for multitarget tracking of radar signals. Therefore, we generate training data from radar motion simulation models. The training trajectories we generated are set with multiple targets, whose birth time and lifetime are random, and incorporate a large number of random uniformly distributed clutter. The initial position and velocity of each target are randomly distributed within a certain range, and the conventional constant-velocity motion model and constant acceleration motion model of the radar target are adopted. The radar measurements are sampled every 2 seconds, and a Gaussian measurement noise is added, while the annotations for multitarget associations are manually added.


## 4. Experiments

We have proposed a radar multitarget tracking network architecture. To demonstrate its functionality, we first present experiments on simulated data and then give more insights and analysis of our performance.In the simulation experiment scene, we choose 2D radar to track multiple moving targets, assuming that the target movements are independent of each other, and trajectory crossing may occur. The equation of motion for each target is(7)Xk=Fk−1Xk−1+Gk−1wk−1,where Xk=x,y,vx,vy,ax,ay is the target state. Wk−1 is process noise representing random acceleration in the x and y axis direction, wk∼N0,σw2I2,σw=10m/s2.We assume that the target follows the constant acceleration (CA) model and the constant velocity (CV) model [27]. Their process equation F and process noise G are
(8)FCA=10T0T220010T0T220010T000010T000010000001,GCA=T220T0100T220T01,FCT=10T0000100000010T0000100000010000001,GCT=T220T0100T220T01,where T is the sampling period.The target observation equation is(9)Zk=HkXk+vk,where Hk is the observation matrix; the measurement noise vk is the Gaussian white noise with zero mean and covariance matrix Rk.The observation vector of radar isZk=θk,rk, where θk is the azimuth of the sensor observation target and rk is the distance of the sensor observation target. The target observation model is
(10)θkrk=arctanyk−yxk−xxk−x2+yk−y2+vθvr,where vθ∼N0,σθ2,σθ=1×π/180rad/s and vr∼N0,σr2,σr=10m.On this basis, we set an observation area of1km×1km, 5 radar tracking targets with random birth and death time. Each target has detected probability PD=0.99. In the observed region, the clutter follows a uniform Poisson distribution, and the clutter intensity is set asλk=30.To generate large amounts of training data, the initial position of the target is randomly distributed within the observation area, the initial linear velocity is randomly distributed from 100 m/s to 300 m/s, and the acceleration is randomly distributed from0m/s2 to 8m/s2. The times of birth and death for each target are randomly distributed between 3 and 18 s. At the same time, the number of targets at each sampling time is manually annotated. From this, we produce 10,000 random paths and their observation data, respectively. These are sufficient to train our network architecture.The results of the tracking are shown in Figures6 and 7. From this, we can qualitatively conclude that our method can track multiple moving radar targets in a large number of clutter environments, although the trajectories of these targets may be crossed. Furthermore, we also see that the predicted trajectories always start and end one or two points later than the real trajectories, which is difficult to avoid with the online method.Figure 6
The tracking results of our method on 5 targets. The different color curves represent the different predicted target trajectories. The black curve is the ground truth.Figure 7
The predicted value of the existence probabilityη of the 5 targets in the time frame. The threshold is set as 0.6.For quantitative analysis, we compare our proposed method (MA-LSTM) with three baseline methods. The first baseline method is the traditional KF-HA, which employs a combination of Kalman filter and data association via the Hungarian algorithm. The second baseline approach is JPDA mentioned above. The third comparison is comprised of our motion module and the data association using the Hungarian algorithm (M-HA). The optimal submode allocation (OSPA) [28] distance for each method is calculated. OSPA is a consistency measurement method for the overall performance evaluation of the target tracking system, which can be used to measure the error between real track and estimated track and can separate the total error into distance errors and correlation errors. For the tunable parameter distance sensitivity parameter p and associated sensitivity parameter c, we set p=20 and c=350, respectively.The calculation results are shown in Table1. MA-LSTM outperforms the other methods in all three aspects. It has great advantages in overall OSPA and associated OSPA and is slightly better than JPDA in terms of distance OSPA. It is worth noting that compared with the HA method for data association, the proposed method of using LSTM for data association and using motion velocity features to calculate similarity has a positive impact on the overall tracking effect, especially on the reduction of association error.Table 1
OSPA distance of the four methods.
MethodOSPA ↓OSPA Loc ↓OSPA card ↓KF-HA83.7977.488.09JPDA79.8675.398.22M-HA79.1273.1512.44MA-LSTM70.0865.787.86As shown in Figure8, this intuitively shows the trend of different method OSPA values over time under a long period of test data. It can be seen that in the early stage of the test, the OSPA value of the learning method is high and then decreases rapidly after that, because of the learning of motion properties in the early stages of tracking. The OSPA value of our method is significantly reduced from the 10th frame, because the speed feature extraction sequence length in the data association module is set to 10. That is, the role of data association can be fully played after 10 frames; after that, the performance of our method is better than other methods.Figure 8
The OSPA values for different methods.


## 5. Conclusion

This paper presents an LSTM-based network architecture for radar multitarget tracking. This architecture can effectively solve the problems of state prediction, measurement association, and trajectory management for radar multitarget tracking under much clutter. In addition, we propose to use the motion extraction LSTM to extract motion features to calculate similarity scores and use it to learn long-dependent target associations, which achieves good results. Our architecture is able to accomplish tracking tasks online and has been verified in simulation scenarios. In future work, we plan to expand it into the video tracking field and make association strategies more robust by combining more clues to achieve better performance.



---

*Source: 1033371-2023-02-15.xml*