# An Improved Q-Learning-Based Sensor-Scheduling Algorithm for Multi-Target Tracking
## 2 Related Work
Simulation results demonstrated that the proposed scheduling method can enhance tracking accuracy and reduce energy consumption. In [16], [PERSON] et al. proposed a message-pruning tree with shortcuts, which can reduce the total cost of updating the database and querying targets compared with the message-pruning tree. A tree-based multi-cast routing protocol was proposed in [17]. Based on residual energy and minimum hop count paths, this method can enhance multiple service quality parameters, including energy consumption, end-to-end delay, and packet delivery rate, by updating the multi-cast tree structure. Overall, most of the existing sensor-scheduling algorithms focus on improving the performance of tracking networks by studying how to organize and schedule nodes to perform tracking tasks in a collaborative manner. Their goal is to select an optimal set of tracking nodes based on metrics such as sensor energy and distance to the target. However, in multi-target-tracking scenarios, it is insufficient to consider these metrics in the scheduling process. Owing to the lack of consideration for the difference in target properties and the difference in network status where targets are located, the existing sensor-scheduling algorithms may face the problem of unbalanced allocation of tracking resources for the targets when managing multi-target tracking. For example, Figure 1 shows the scheduling results of sensors using the classical nearest-neighbor method. It can be seen that, according to the nearest-neighbor principle, node \(i\) should work as a tracking node for target 1. However, target 2 has only two tracking nodes, and the system lacks sufficient localization information for target 2, so the tracking accuracy is seriously affected. Moreover, the two tracking nodes of target 2 have to undertake heavy tasks, which may cause unbalanced energy consumption. In this case, target 2 should be given higher priority than target 1 when scheduling the tracking task for node \(i\). Besides the nearest-neighbor method, other algorithms that do not consider the balance of resource allocation in scheduling will encounter similar problems. It can be seen from the example in Figure 1 that once such an unbalanced task allocation occurs in the sensor-scheduling process, it leads to the degradation of the overall system performance, such as tracking accuracy and energy efficiency. Therefore, to address this issue, we propose a sensor-scheduling algorithm with consideration of the balance of tracking-task allocation for multi-target tracking (MTT-SS). The contributions of this paper are summarized as follows:  
Taking advantage of the entropy-weight method (EWM), MTT-SS calculates the aggregated weight for each target according to four decision criteria, including target type, distance to the cluster center, node density, and energy level. After that, linear weighting is conducted to obtain the overall priority function to evaluate target priority. Prioritizing  
Figure 1: Traditional nearest-neighbor-scheduling method. targets can help sensors select appropriate tasks to execute based on the current status of targets and the network. To obtain a balanced resource scheduling result, a Q-learning-based task allocation mechanism is developed, in which nodes are mapped as intelligent agents. Each agent selects the appropriate task to execute by using an improved exploration-exploitation policy. At each time step, agents update their Q value according to the environment state and the delayed reward. Extensive experiments and analyses for different solutions are presented in the simulation. Simulation results indicate that MTT-SS significantly outperforms other existing sensor-scheduling algorithms in terms of tracing accuracy and energy efficiency. Following the related work given in Section 2, Section 3 describes the network model and the energy consumption model. The proposed sensor-scheduling algorithm is presented in Section 4. Section 5 provides the simulation results. Finally, Section 6 concludes this paper.