## 5 Adaptive Data Transmission Algorithm Based on Reinforcement Learning
### Reinforcement Learning Model
\(\overline{j}_{\rm receive}^{j}\) is the average transmission delay from neighbor node \(\rm n_{j}\) to the current node \(\rm n_{i}\), which can be calculated by the time stamp of the packets from \(\rm n_{j}\). \(\varphi_{t}\) is the sensitivity coefficient of the transmission channel delay cost. \(\beta_{1}\) and \(\beta_{2}\) are used to adjust the weight of the two delay parameters, respectively. \(\overline{t_{total}^{i\to j}}\) is the actual delay of packet transmission. \(\overline{t_{receive}^{j}}\) can help nodes to optimize transmission policies without forwarding the packets. Part IV: \(\rm co(1)\) is the cost related to data transmission accuracy. The data transmission accuracy is determined by the number of correction data transmitted. \(\rm co(1)\) can be expressed as follows:_  
\[N=\{n_{1},n_{2},n_{3},\cdots,n_{m}\}, \tag{8}\]2  
\(\rm l_{correct}\) _indicates the number of correction data sent. The value range of \(l_{correct}\) is \([\rm threshold,\cdots,\rm l_{correct}^{max}]\). \(\rm l_{correct}^{max}\) is the number of correction data received or generated by the node \(\rm n_{i}\). \(\varphi_{1}\) is the sensitivity coefficient of the correction accuracy cost. Selecting higher \(\varphi_{1}\) can reduce the accuracy of the monitoring data. However, the greater length of the transmitted data can reduce the packet-sending success rate, and increase the transmission delay. Although selecting lower \(\varphi_{1}\) will reduce the accuracy of data transmission, it can improve the reliability of data transmission. Therefore, selecting the appropriate \(\varphi_{1}\) will realize the win-win of transmission efficiency and transmission accuracy._  
**Definition 4**. : _At time \(t\), if the current packet is in node \(\rm n_{i}\), the transition probability of node \(\rm n_{i}\) to node \(\rm n_{j}\) the accuracy 'acc' is defined as follows:_  
\[N=\{n_{1},n_{2},n_{3},\cdots,n_{m}\}, \tag{8}\]3  
_In order to reflect the impact of the future status on the current status, the overall reward at time \(t\) is defined as shown in [5]:_  
\[N=\{n_{1},n_{2},n_{3},\cdots,n_{m}\}, \tag{8}\]4  
_when \(\gamma=0\), the system will only consider the reward \(r\) of the current action, not the future status. When \(\gamma\to 1\), the system will consider both the current and future action rewards \(r\). According to \(Q\)-learning, the state action function under policy \(\pi\) is defined as shown in [5]:_  
\[N=\{n_{1},n_{2},n_{3},\cdots,n_{m}\}, \tag{8}\]5_Assuming that_ a _tand_ s _t are the next action and state, the optimal solution \(Q^{*}(s,a)\) in the state of_\(Q^{\pi}(s,a)\) _can be expressed as an iterative equation, as shown in_ [5]_:_  
\[N=\{n_{1},n_{2},n_{3},\cdots,n_{m}\}, \tag{8}\]6  
_According to the state action function, the node with sufficient residual energy and smaller transmission channel delay in the virtual pipeline will be selected as the relay node. The packets will be packaged according to the appropriate transmission accuracy meeting the threshold requirements and sent by the current node._  
_At time \(t\), the value function \(V\) will select the state action function that can obtain the maximum benefit, which is defined as shown in [5]:_  
\[N=\{n_{1},n_{2},n_{3},\cdots,n_{m}\}, \tag{8}\]7  
_At the initial stage of reinforcement learning, the initial \(V\) value of each agent is 0. With the accumulation of learning, the \(V\) value will be continuously updated according to Formulas (20) and (21) and gradually converge. A good data forwarding strategy will finally emerge._