# Deep-Reinforcement-Learning-Based Joint Energy Replenishment and Data Collection Scheme for WRSN
## 3 System Model and Problem Formulation
### System Model
#### 3.1.3 UAV Model  
The UAV maintains a fixed flight altitude of \(H\). It acquires the positions of all nodes before taking off from the base station and maintains communication with the base station throughout the operation. At \(t\), the three-dimensional coordinates of the UAV can be represented as \([x_{u}(t),y_{u}(t),H]\). The distance between the UAV and the node plays a crucial role in determining the link quality and the ability of the node to upload data. In our scenario, we assume that the UAV has limited communication and charging ranges, denoted as \(R_{d}\) (maximum data transmission radius) and \(R_{c}\) (maximum charging radius), respectively. \(\Delta d_{u,ch_{i}^{thr}}\) represents the distance between the UAV and the target CH node if \(\Delta d_{u,ch_{i}^{thr}}\leq R_{d}\); the UAV enters the hovering state to collect data from the target CH node and charge all nodes within the charging range. The instantaneous speed \(v_{u}(t)\) and the instantaneous heading angle \(\theta_{u}(t)\) describe the flight control of the UAV. For safety reasons, \(v_{u}(t)\) must be within the minimum and maximum speed limits.  
\[V_{\text{min}}\leq v_{u}(t)\leq V_{\text{max}} \tag{5}\]  
This paper employs a rotor-wing UAV, and the energy consumption of the UAV can be divided into propulsion energy consumption and communication energy consumption. The propulsion energy consumption is further divided into flight energy consumption and hovering energy consumption [20]. When the UAV has an instantaneous speed \(v_{u}(t)\), the propulsion energy consumption model of the UAV can be given as follows:  
\[\begin{split}& E_{u}^{pro}(v_{u}(t))=P_{0}(1+\frac{3v_{u}(t)^{2}}{ Ul_{\text{tip}}^{2}})+P_{f}(\sqrt{1+\frac{v_{u}(t)^{4}}{4v_{0}^{4}}}-\frac{v_{u}(t) ^{2}}{2v_{0}^{2}})^{1/2}\\ &+\frac{1}{2}d_{0}\rho_{\text{air}}s_{\text{rotor}}A_{\text{rotor }}v_{u}(t)^{3}\end{split} \tag{6}\]  
where \(P_{0}\) and \(P_{f}\) are constants that represent the blade profile power and induced power in hovering status, respectively; \(Ul_{tip}\) indicates the tip speed of the rotor blade; \(v_{0}\) denotes the mean rotor-induced speed; and \(d_{0}\), \(\rho_{air}\), \(s_{rotor}\), and \(A_{rotor}\), respectively, denote the fuselage drag ratio, air density, rotor solidity, and rotor disc area. At t, UAV speed is \(v_{u}(t)\), and the flight energy consumption is expressed as \(E_{u}^{fly}=E_{u}^{pro}(v_{u}(t))\). It is worth noting that the UAV is in hovering status with flight speed \(v_{u}(t)=0\) and hovering energy consumption \(E_{hover}=E_{u}^{pro}(v_{u}(t)=0)\). To facilitate the expression, we divide the time domain \(T\) into \(n\) time steps, where each time step is denoted as \(t_{0},t_{1},t_{2},\ldots t_{n}\); thus, the total flight energy consumption of the UAV at each time step \(t_{n}\) is given as follows:  
\[E_{u}^{total}(t_{n})=\int_{0}^{t_{n}}E_{u}^{pro}(v_{u}(t))dt \tag{7}\]  
the UAV's average flight energy consumption is given as follows:  
\[E_{u}^{new}=\frac{E_{u}^{total}(T)}{T} \tag{8}\]