## 3 Proposed Algorithm
### Reward Mechanism  
The establishment of the reward function in this section considers two factors: the immediate reward of real-time data allocated to the time slot and the influence of other data not allocated to the time slot. Instant reward \(r\) reflects the current priority of the data by considering the remaining time of the node and the number of hops remaining for all the data allocated by the time slot. The function \(R_{L}\) is defined to represent the impact of other data flows that are not allocated to time slots at the current time. When the current time slots are allocated, the more packets of other data flows are lost, the closer the data distance to the discarded state is, and the greater \(R_{L}\) is. The reward function is shown as  
\[R=r+R_{L} \tag{9}\]  
where \(r\) represents the instant reward value of all packets allocated to time slots. The instant reward value of each data packet allocated to time slots consists of the remaining time of the packet and the remaining hops. The smaller the remaining time is, the longer the remaining hops are, and the larger the \(r\) is, the higher the priority of the current packet is. Immediate rewards are defined as follows:  
\[r=\sum_{i=1}^{n}\beta_{i}\cdot(k_{1}\cdot\frac{h_{i}}{t_{i}}+k_{2}\cdot\frac{ 1}{t_{i}-h_{i}+1}) \tag{10}\]  
where \(t_{i}\geq h_{i}\), and \(n\) represents the number of packets to obtain slot assignments. If data \(i\) are transmitted in the current time slot and arrives at the destination node, the reward is enhanced, \(\beta_{i}\in[1,1.5)\), otherwise \(\beta_{i}=1\). \(k_{1},k_{2}\) satisfies \(k_{1}>0,k_{2}<1\), and \(k_{1}+k_{2}=1\). Obviously, \(r\) is inversely proportional to the remaining packet cutoff time \(t_{i}\) and is directly proportional to the remaining hop number \(h_{i}\) of the packet. Both \(h_{i}/t_{i}\) and \(t_{i}-h_{i}\) can reflect the degree of urgency of data. \(h_{i}/t_{i}\) reflects the degree of urgency of data through ratio relationship without considering the influence of actual remaining time slots. \(t_{i}-h_{i}\) reflects the influence of actual remaining time slots.  
\(R_{L}\) is the reward function of behavior, which reflects the negative reward. When the system is in state \(s_{i}\) and performs an action to enter the next state \(s_{i+1}\), it is assumed that among all data packets in the system, \(L_{i0}\) data packets meet the transmission state of \(t_{i}-h_{i}=-1\), \(L_{i1}\) data packets meet the transmission state of \(t_{i}-h_{i}=0\), and \(L_{i2}\) data packets meet the transmission state of \(t_{i}-h_{i}=1\).  
\[R_{L}=-(\rho_{1}L_{i0}+\rho_{2}L_{i1}+\rho_{3}L_{i2}) \tag{11}\]  
where \(L_{i0}\), \(L_{i1}\), and \(L_{i2}\) are defined as above, \(\rho_{1}\), \(\rho_{2}\), and \(\rho_{3}\) are relevant discount parameters, satisfying \(0<\rho_{1},\rho_{2},\rho_{3}<1,\rho_{1}>\rho_{2}>\rho_{3}\), and \(\rho_{1}+\rho_{2}+\rho_{3}=1\). The final reward function is expressed as follows:  
\[R=\sum_{i=1}^{n}\beta_{i}\cdot(k_{1}\cdot\frac{h_{i}}{t_{i}}+k_{2}\cdot\frac{ 1}{t_{i}-h_{i}+1})-(\rho_{1}L_{i0}+\rho_{2}L_{i1}+\rho_{3}L_{i2}) \tag{12}\]  
The partial separation and combination of reward parameters and reward factors allow the reward function to adjust the external weight. The behavior of the whole system is determined by the initial state of the data flow and the reward function of the behavior. The system can converge to the ideal equilibrium point in a given environment.