## 3 Proposed Algorithm
### Optimal Action Selection Strategy  
The optimal action selection strategy is used to determine the node-set to send. Firstly, the strategy selects the best action for the current time slot by exploring the development strategy. Secondly, based on the node where the action is located, the most urgent and non-conflicting node-set is constructed. (1) Explore development strategies  
In the process of systematic learning and trial and error, it is necessary to balance the relationship between exploration and development. The general \(\varepsilon-greddy\) strategy is prone to the problem of too fast convergence. Development under the condition of insufficient exploration will cause the learning process to be too short and the learning results to be seriously deviated. Based on the \(\varepsilon-greddy\) strategy, this algorithm introduces the Metropolis criterion in the simulated annealing algorithm into the execution action selection of the exploration and development strategy. Meanwhile, it can be seen from the Q learning model that the state space of the whole system explosively expands with the increase of the number of nodes, which requires relatively long learning times to achieve the ideal training effect. Therefore, segmented exploration and development processes are adopted to acquire state and behavior. In the early stages of DQN network learning, actions are randomly selected and saved to the experience pool before the experience pool is full. Then, based on the exploration probability \(\varepsilon_{t}\), the action selection begins to gradually balance the exploration and development process, which can better solve the problem of too fast convergence. The exploration probability \(\varepsilon_{t}\) is defined as follows:  
\[\varepsilon_{p}=\exp[\frac{-\left|\max_{a_{0}\in A}\_value(s,a_{0})\right|}{KT _{k}}] \tag{6}\]  
\[\varepsilon_{t}=\max\{\varepsilon_{\min},\varepsilon_{p}\} \tag{7}\]  
\(\varepsilon_{p}\) is the exploration probability after the simulated annealing algorithm is introduced, where \(\max_{a}\_value(s,a_{0})\) is the maximum output value after the state-action mapping in the deep neural network, and \(a_{0}\) is the action corresponding to the maximum output. \(T\) is a fixed value, \(K\) is a coefficient, satisfying \(K=\lambda^{\varepsilon}\), \(\lambda\in(0,1)\). \(e\) is the number of learning times. As the number of learning times increases, the value of \(KT_{k}\) will become smaller and smaller, and the value of \(\varepsilon_{p}\) will also become smaller and smaller, and the entire exploration process will tend to be stable. \(\varepsilon_{\min}\) is the given minimum exploration probability, which is the lower bound of exploration, \(\varepsilon_{t}\) is the final exploration probability of the current time slot, and it is the maximum value of \(\varepsilon_{p}\) and \(\varepsilon_{\min}\). When the best action is selected, to utilize the network's transmission capacity as much as possible, other nodes are selected for concurrent transmission. The process is mainly based on the conflict interference matrix and the urgency of the data. (2) Concurrent node sets based on the most urgent data Although each time slot allows multiple nodes to transmit data, there are transmission conflicts between child nodes with the same parent node, and two sensor nodes with interference edges cannot perform transmission tasks simultaneously. In this algorithm, the deep neural network mapping relation is used to determine the most urgent data, and the node where the data resides is the most urgent node. Then, other transmission nodes are dynamically selected to construct the most urgent non-conflicting node-set and transmit the selected data from the node. **Definition 1**. : _A conflict interference matrix is used to represent the matrix of the conflicted relationship between nodes, where the row number and column number respectively represent the number of the corresponding sensor node.