# An Improved Q-Learning-Based Sensor-Scheduling Algorithm for Multi-Target Tracking
## 2 Related Work  
Most of the recent research in tracking-sensor networks focuses on improving the performance of sensors performing collaborative tracking tasks. Generally, the existing works are designed for single-target tracking or tracking of multiple homogeneous targets. In [7], a dynamic chain-based collaboration (DCBC) method was proposed for efficient target tracking and data gathering. In DCBC, sensors form a dynamic tracking chain around the target, which can adjust its structure as the target moves. The sensing data collected by tracking nodes is aggregated locally at each time step. In this way, DCBC can reduce the energy consumption of target tracking and prolong the network lifetime. In [8], an energy-aware task-scheduling approach (TOTD) was proposed to achieve a trade-off between energy efficiency and tracking performance. The authors utilized a true online reinforcement learning framework to obtain the optimal task scheduling scheme for sensors to track targets. Simulation results showed that TOTD outperforms other methods for target-tracking applications. [PERSON] et al. proposed a chain-cluster based routing protocol in [9], in which the combination of clustering and ant colony optimization (ACO) was used to reduce transmission delay and obtain the optimal routing path. In the simulation, this protocol was proved superior in residual energy, latency, and load balancing, making it suitable for target tracking. To manage the problem of selecting the optimal sensors for target tracking, [PERSON] et al. proposed a reliable sensor selection approach in [10]. They proposed an updated unscented Kalman filter (U2 KF) to improve the efficiency of target tracking by sensor selection. Moreover, multi-objective optimization was utilized to select a sensor set without knowing the number of sensors needed to be selected. Sigma points probability (SP) and target trajectory (TT) approaches were utilized to eliminate the effects of noise while selecting. The superiority of [PERSON] et al. 's method in effectiveness and utility was proven in extensive experiments. A dynamic cluster establishment scheme and a distributed consensus-based adaptive Kalman estimation method were proposed by [PERSON] et al. [11] to reduce energy consumption and improve estimation accuracy in target tracking. In [12], a dynamic cluster member scheduling (DCMS) algorithm was proposed by [PERSON] et al. to achieve the trade-off between energy efficiency and tracking performance; this method constructed a reward function by exploiting the weighted value. Then, a point-based online value iteration algorithm was designed to schedule sensors to collaboratively track targets. In simulations, DCMS was shown to improve tracking accuracy and extend network lifetime. Combined with the particle swarm optimization (PSO) algorithm, [PERSON] and [PERSON] [13] proposed an improved bee colony algorithm based on a double-roulette scheme to obtain the optimal sensor-scheduling scheme. Additionally, to reduce the target-missing risk and the sensor radiation interception risk, they created the sensor-scheduling objective function which takes risk factors into account. The authors in [14] proposed a dynamic-clustering protocol (EEDC) for target tracking and continuous event monitoring. In EEDC, the overlapping cluster structure was dynamically reconstructed to adapt to the changing location of targets, and candidate cluster heads were selected according to rough fuzzy C-means (RFCM) and genetic algorithm (GA). EEDC can significantly reduce system energy consumption as compared to other clustering protocols. The authors in [15] proposed a hierarchical-tracking structure based on edge intelligence (EI) technology. The structure can integrate the computing resources of sensors and edge servers to provide efficient computing for real-time tracking. Moreover, the authors designed a long-term dynamic resource allocation algorithm to obtain the optimal sensor-scheduling scheme.