## 3 Proposed Algorithm
### State-Behavior Description Network  
The deep neural network is a kind of neural network containing multiple hidden layers, each of which can perform the nonlinear transformation on the output of the previous layer. Therefore, compared with a shallow layer network, a deep neural network has more excellent expression ability and can learn more complex function relations in a more compact and concise way.  
The Stacked Auto Encoder (SAE) deep neural network consists of multiple layers of sparse autoencoder neural networks. The general idea of the training process is unsupervised pre-training and supervised fine-tuning. In this network, in the unsupervised training stage, the hidden feature representation learned by the previous layer of autoencoders is used as the input of the latter layer of autoencoders. The training process of the parameters of each layer will keep the parameters of other layers fixed. After the above-mentioned pre-training process is completed, in the supervised fine-tuning stage, using the previously trained parameters as the initial value of the network, the parameters can also be adjusted, and then continue to train the neural network.  
This algorithm adopts the multi-layer stacked self-encoding deep neural network model to train and realize the mapping relationship between the system state and behavior, which can quickly obtain the optimal decision-making behavior. The structure of the SAE model is shown in Figure 4. The input layer of the model corresponds to the state information of the system, and the number of neurons in the input layer is \(3\cdot M\). The input vector is composed of the current transmission state of the data generated by all nodes of the WSNs, including the node where the data are currently located, the remaining hops, and the remaining deadline. All input vectors are denoted as \(input=[c_{1}\dots c_{M}h_{1}\dots h_{M}t_{1}\dots t_{M}]\). The output layer represents the action selection information of the model, and each output corresponds to each node to generate data in the system as the evaluation of the most urgent data in the next time slot, so the number of neurons in the output layer is \(M\). The output vector is \(output=[a_{1}\dots a_{M}]\). The hidden layer is multi-layered, and the number of neurons in each layer is related to the number of sensor nodes in the network.  
The neurons of the hidden layer in the SAE model are activation functions used for the nonlinear transformation of the input information. Common activation functions include ReLU function, sigmoid function, and tanh function. The nonlinear sigmoid function has a large signal gain in the central area and relatively small signal gain on both sides, which has a good effect on the feature space mapping of the signal. The activation function of this  
Figure 4: SAE network topology.  
algorithm selects the sigmoid function. The loss function of the overall sample during the training process is denoted as \(L(W,b)\).  
\[L(W,b)=\frac{1}{N}\sum_{i=1}^{N}l(W,b) \tag{13}\]  
where \(N\) is the total number of input samples, \(l(W,b)\) is the loss function of a single sample, and the calculation expression of \(l(W,b)\) is as follows:  
\[l(W,b)=\left\{\begin{array}{c}\frac{1}{2}(q_{eval}-q_{tgt})^{2}\left|q_{eval} -q_{tgt}\right|\leq 1\\ \left|q_{eval}-q_{tgt}\right|-\frac{1}{2},\qquad otherwise\end{array}\right. \tag{14}\]  
where \(q_{eval}\) represents the calculated Q value, and \(q_{tgt}\) represents the target Q value. When the forward propagation process is over, the parameter \(W\) and the bias \(b\) are updated using the gradient descent method.