# An Improved Q-Learning-Based Sensor-Scheduling Algorithm for Multi-Target Tracking
## 5 Simulation Results
### Performance Analysis
It can be seen from Figure 6 that neither the traditional random allocation mechanism nor the minimum distance rule can achieve a smaller localization error in the tracking process because their rules do not consider the priority of tracking resource allocation or set a reasonable reward for the sensors. In contrast, MTT-SS first uses a target-prioritizing strategy to determine the importance of each target, which provides an indication when scheduling sensors later using the Q-learning framework. Then, the reward function in Q-learning used by MTT-SS encourages the sensors to select a target to track to achieve a balance of resources. Therefore, compared with other scheduling algorithms, MTT-SS can reduce the occurrence of unbalanced allocation of tracking resources for multiple targets and keep the average localization errors in the tracking process to a minimum standard, which is feasible for many practical tracking scenarios. \begin{table}
\begin{tabular}{c c} \hline \hline
**Algorithms** & **Standard Deviation(m)** \\ \hline DPT [23] & 0.0702 \\ DIRL [24] & 0.0562 \\ TOTD [8] & 0.0472 \\ DCBC [7] & 0.0646 \\ MTT-SS & 0.0202 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Standard deviation of localization errors. Figure 6: Average localization error vs. number of targets. In the second set of simulations, we evaluate the energy efficiency of the scheduling algorithms by comparing the average residual energy of sensors and the system energy consumption. Figure 7 provides the comparison of average residual energy based on different sensor-scheduling algorithms. At the 1000-th time step, the average residual energy is 0.257 joules for DPT, 0.526 joules for DIRL, 0.363 joules for TOTD, 0.593 joules for DCBC, and 0.628 joules for MTT-SS. Moreover, the comparison of different sensor-scheduling algorithms in terms of energy consumption is shown in Figure 8. At the 1000-th time step, the energy consumption is 0.257 joules for DPT, 0.526 joules for DIRL, 0.363 joules for TOTD, 0.593 joules for DCBC and, 158.5 joules for MTT-SS. It can be seen that in the proposed MTT-SS, the sensors can use less energy to perform the target-tracking task compared to other algorithms. The reason is that, in the scheduling process of MTT-SS, the setting of the reward function in Equation (14) encourages the sensors to enter the sleep state after all the targets meet the tracking requirements, which reduces the transmission and processing of redundant data. In this way, the transmission and processing of redundant data in the network can be effectively reduced, so the average residual energy of the sensors in the MTT-SS can be kept at a high level. In contrast, DCBC achieves high energy efficiency as well. The reason is that the tracking chain structure of DCBC can ensure that sensors transmit the sensing data to their nearest neighbors for data aggregation, which decentralize the tasks of the cluster head and achieve better energy balance. Moreover, redundant nodes can go to sleep in DCBC in time, so that energy can be further saved. DPT is a traditional dynamic-tracking scheme without an efficient network structure and an effective sensor-scheduling framework, so it performs poorly compared to other methods in terms of energy efficiency. Although DIRL and TOTD have similar sensor-scheduling mechanisms based on reinforcement learning, they lack comprehensive consideration of target attributes, network status, and node capabilities, which makes it difficult for them to allocate resources reasonably in multi-target scenarios. Therefore, they are still inferior to MTT-SS in terms of energy efficiency. Figure 7: Average residual energy. Figure 9 shows a comparison of time steps for the first node dead (FND) as the number of targets increases. For target-tracking applications, the tracking performance of the network is greatly reduced once a node in a critical surveillance area dies, thus FND is used to measure the network lifetime. In addition, the time step of FND occurrence can also reflect the load balance of nodes in the network.