# An Improved Q-Learning-Based Sensor-Scheduling Algorithm for Multi-Target Tracking
## 4 Detailed Description of the Proposed Algorithm
### Target Prioritizing
#### (4) Energy Level  
Energy level \(\mathbb{E}\) indicates the residual energy of the nodes around the target. As the tracking system works, there may be nodes that take on the tracking task multiple times, resulting in greater energy consumption than others. To ensure the reliability of the tracking process, the target, whose surrounding nodes have low residual energy, should be set as a high tracking priority.  
To determine the weights of different indicators, we use EWM to calculate the comprehensive score for target priority. EWM is an objective weighting method based on the degree of data dispersion for each indicator [21]. EWM first uses the information entropy principle to calculate the entropy value of each indicator, and then corrects the entropy weight according to the data distribution of each indicator, so as to obtain the objective index weight. Compared with subjective methods such as analytic hierarchy process (AHP), EWM has better accuracy and adaptability. EWM includes the following steps:  
First, we normalize all the indicators by using the extreme-value processing method. When the indicator has positive effects, we have:  
\[Y_{ij}=\frac{X_{ij}-X_{i,\text{min}}}{X_{i,\text{max}}-X_{i,\text{min}}}. \tag{6}\]When the indicator has negative effects, we have:  
\[Y_{ij}=\frac{X_{i,\max}-X_{ij}}{X_{i,\max}-X_{i,\min}}. \tag{7}\]  
where \(Y_{ij}\) is the normalized value; \(X_{ij}\) is the value of the \(j^{th}\) indicator of the \(i^{th}\) target; and \(X_{i,\max}\) and \(X_{i,\min}\) are the maximum and minimum values of the corresponding indicator, respectively. After normalization, the entropy \(E_{j}\) of the \(j^{th}\) indicator is given by:  
\[E_{j}=-k\sum\limits_{i=1}^{n}f_{ij}\ln f_{ij},\quad j=1,2,\ldots,m \tag{8}\]  
\[k=1/\ln n, \tag{9}\]  
where \(m\) is the number of indicators; \(n\) is the number of targets; \(f_{ij}=Y_{ij}/\sum\limits_{i=1}^{n}Y_{ij}\) and assuming that when \(f_{ij}=0,f_{ij}\ln f_{ij}=0\). The function of \(k\) in Equation (9) is to make the entropy value fall within (0, 1). We then define the information utility value, which can measure the amount of useful information contained in the indicator. It can be learned from Equation (8) that, when values of indicators of all samples are equal, \(E\) obtains a maximum value, which means that data do not change, that is, there is little information. Thus, we reverse it to calculate the information utility value \(H_{j}\) of the \(j^{th}\) indicator:  
\[H_{j}=1-E_{j}. \tag{10}\]  
Then, we determine the weight of each indicator by normalizing the information utility value:  
\[\omega_{j}=H_{j}/\sum\limits_{j=1}^{m}H_{j}, \tag{11}\]  
where \(\omega_{i}\) is the weight of the \(j^{th}\) indicator. Finally, the comprehensive evaluation of target priority \(P\) is obtained:  
\[P(o_{i})=\omega_{1}\eta+\omega_{2}\mathfrak{D}+\omega_{3}\rho+\omega_{4} \mathbb{E}, \tag{12}\]  
where \(o_{j}\) denotes the \(i^{th}\) target.  
When the value difference between the evaluating targets of the same indicator is large and the entropy is small, it indicates that this indicator provides more useful information than other indicators. Accordingly, the weight value of this indicator is higher. By carrying out EWM, we can avoid errors caused by the prejudice of decision-makers and thus have high accuracy in weight identification.