# Deep-Reinforcement-Learning-Based Joint Energy Replenishment and Data Collection Scheme for WRSN
## 3 System Model and Problem Formulation
### System Model
#### 3.1.4 Transmission Model  
The wireless communication link between the UAV and the nodes mainly consists of a line-of-sight (LoS) link and a non-line-of-sight (NLoS) link, similar to many existing works [20; 39; 40]. Let the coordinate of the target CH node \(ch_{i}^{tar}\) be \([x_{ch_{i}^{tar}},y_{ch_{i}^{tar}},0]\); the free path loss of the LoS link between the UAV and \(ch_{i}^{tar}\) at \(t\) is modeled as follows:  
\[\begin{split} h_{u,ch_{i}^{tar}}(t)&=\gamma_{0} \Delta d_{u,ch_{i}^{tar}}(t)^{-2}\\ &=\frac{\gamma_{0}}{H^{2}+(x_{u}(t)-x_{ch_{i}^{tar}})^{2}+(y_{u}(t )-y_{ch_{i}^{tar}})^{2}}\end{split} \tag{9}\]  
where the channel's power gain at a reference distance of \(\Delta d_{u,ch_{i}^{tar}}=1\) m is denoted by \(\gamma_{0}\), and \(\Delta d_{u,ch_{i}^{tar}}(t)\) denotes the Euclidean distance between the UAV and \(ch_{i}^{tar}\) at \(t\), where the path loss of the NLoS link is given as \(\eta^{NLoS}\gamma_{0}\Delta d_{u,ch_{i}^{tar}}(t)^{-2}\), and \(\eta^{NLoS}\) is the attenuation coefficient of the NLoS links.  
At \(t\), the LoS link probability between the UAV and \(ch_{i}^{tar}\) is modeled as follows:  
\[P_{u,ch_{i}^{tar}}^{LoS}(\theta_{ch_{i}^{tar}}(t))=\frac{1}{1+\alpha\exp(- \beta(\theta_{u,ch_{i}^{tar}}(t)-\alpha))} \tag{10}\]  
where \(\alpha\) and \(\beta\) are constant sigmoid parameters that depend on the signal propagation environment and propagation distance. \(\theta_{ch_{i}^{tar}}(t)\) is the elevation angle of the UAV and \(ch_{i}^{tar}\) in degree; it is given as \(\theta_{ch_{i}^{tar}}(t)=\frac{180}{\pi}\text{sin}^{-1}(\frac{H}{\Delta d_{u, ch_{i}^{tar}}})\). The NLoS link probability is given as follows:  
\[P_{u,ch_{i}^{tar}}^{NLoS}(\theta_{ch_{i}^{tar}}(t))=1-P_{ch_{i}^{tar}}^{LoS}( \theta_{ch_{i}^{tar}}(t)) \tag{11}\]  
let \(g_{u,ch_{i}^{tar}}(t)\) be the wireless channel gain between the UAV and \(ch_{i}^{tar}\); it is modeled as follows:  
\[\begin{split} g_{u,ch_{i}^{tar}}(t)&=(P_{u,ch_{i}^{ tar}}^{LoS}(\theta_{u,ch_{i}^{tar}}(t))+\eta^{NLoS}P_{u,ch_{i}^{tar}}^{NLoS}( \theta_{u,ch_{i}^{tar}}(t)))h_{u,ch_{i}^{tar}}(t)\end{split} \tag{12}\]  
in this paper, we assume that the uplink and downlink channels are approximately equal. As a result, the channel power gain between the UAV and \(ch_{i}^{tar}\) is given as follows:  
\[\begin{split} h_{u,ch_{i}^{tar}}(t)&\approx g_{u, ch_{i}^{tar}}(t)=(P_{u,ch_{i}^{tar}}^{LoS}(\theta_{u,ch_{i}^{tar}}(t))\\ &\hskip 14.226378 pt+\eta^{NLoS}P_{u,ch_{i}^{tar}}^{NLoS}(\theta_{u, ch_{i}^{tar}}(t)))h_{u,ch_{i}^{tar}}(t)\end{split} \tag{13}\]  
Assuming that the UAV establishes a communication link with \(ch_{i}^{tar}\) at \(t\) and \(ch_{i}^{tar}\) maintains a constant transmit power \(P_{t}\) to upload data to the UAV, according to [PERSON]'s formula [44], the data transmission rate between the UAV and \(ch_{i}^{tar}\) can be given as follows:  
\[R_{ch_{i}^{tar}}^{t}(t)=B\text{log}_{2}(1+\frac{P_{t}\Big{|}g_{u,ch_{i}^{tar}}( t)\Big{|}^{2}}{\sigma^{2}}) \tag{14}\]  
where \(B\) and \(\sigma^{2}\) represent the channel bandwidth and noise power, respectively, and \(P_{t}\) denotes the transmit power of the node; hovering time (upload data time) is given as follows:  
\[t_{hover}=\frac{D_{ch_{i}^{tar}}(t)}{R_{ch_{i}^{tar}}^{t}(t)} \tag{15}\]  
the energy consumption for \(ch_{i}^{tar}\) to upload data at time \(t\) can be given as follows:  
\[E_{ch_{i}^{tar}}^{t}(t)=P_{t}t_{hover} \tag{16}\]