# Deep Reinforcement Learning for UAV-Based SDWSN Data Collection
## 3 System Model  
In a Wireless Sensor Network (WSN), some sensor nodes can serve dual roles as both leaf nodes and cluster heads. These nodes play a critical role in optimizing network performance by efficiently managing communication and energy consumption. To enhance the efficiency of cluster heads, we propose designing a bounded fuzzy range based on Line-of-Sight (LoS) coverage. By considering the fuzzy region around each cluster head coverage area, we can optimize communication paths and minimize energy consumption. We aim to find the optimal path within the fuzzy region of the WSN. This involves selecting routes that minimize the propulsion of energy usage, reduce the distance between ground gateway nodes and the UAV, and stabilize ground network energy levels. The proposed path should balance these factors effectively. Hence, based on the proposed optimal path, adjustments can be made to select the optimal cluster heads out of potential cluster heads. This decision considers factors such as UAV versus ground network energy, communication quality, and overall network stability. When evaluating different scenarios within the UAV's operational range, the proposed method's performance metrics include the percentage of sensor nodes served, energy consumption of the ground network, and average UAV energy consumption. There is a notable trade-off between the UAV's propulsion energy and the ground network's energy costs. A heuristic smooth path design for the UAV can create an energy-efficient route from the UAV's standpoint but leads to an energy-inefficient data collection model for the ground sensor nodes [25]. On the other hand, the Bezier curve path design [25] for the UAV reduces its energy consumption per mission and decreases the percentage of served sensor nodes, while improving the ground network's energy efficiency. Therefore, it is necessary to formulate an optimization problem to find the optimal solution that simultaneously minimizes UAV propulsion energy usage and ground sensor nodes' energy consumption, while maximizing the UAV's communication bit rate. To enhance the bit rate in air-to-ground communication for the UAV path (q(t)), it is necessary to develop a statistical model that accounts for the communication throughput between the UAV and SNs under LoS conditions. This paper does not consider the air-to-ground connectivity model for occasional link blockages due to NLoS links. Therefore, the total amount of information bits transferred to the UAV over the duration T is a function of the UAV's trajectory, as expressed in (1) [34]:  
\[\overline{\mathrm{R}}(\mathrm{q}(\mathrm{t}))_{\mathrm{air-to-Ground}}=\int_{0 }^{\mathrm{T}}\mathrm{B}\log_{2}\left(1+\frac{\gamma_{0}}{\mathrm{H}^{2}+ \left\|\mathrm{q}(\mathrm{t})\right\|^{2}}\right) \tag{1}\]  
where B stands for the channel bandwidth, and \(\gamma_{0}\) is the reference received signal-to-noise ratio (SNR) at \(\mathrm{d}_{0}=1\) m. H is the altitude of the UAV while flying over the ground SNs. Also, the UAV energy consumption for a fixed-wing UAV considering variable velocity \(\mathrm{v}(\mathrm{t})\) and acceleration vectors \(\mathrm{a}(\mathrm{t})\) is expressed in (2) [34]:  
\[\overline{\mathrm{E}}(\mathrm{q}(\mathrm{t}))=\int_{0}^{\mathrm{T}}[\mathrm{c }_{1}\|\mathrm{v}(\mathrm{t})\|^{3}+\frac{\mathrm{c}_{2}}{\|\mathrm{v}( \mathrm{t})\|}(1+\frac{\left\|\mathrm{a}(\mathrm{t})\right\|^{2}-\frac{\left( \mathrm{a}^{\mathrm{T}}(\mathrm{t})\mathrm{v}(\mathrm{t})\right)^{2}}{\left\| \mathrm{v}(\mathrm{t})\right\|^{2}}}{\mathrm{g}^{2}})]\mathrm{dt}+\frac{1}{2} \mathrm{m}\left(\left\|\mathrm{v}(\mathrm{t})\right\|^{2}-\left\|\mathrm{v}( 0)\right\|^{2}\right) \tag{2}\]  
in which \(\mathrm{c}_{1}\) and \(\mathrm{c}_{2}\) are two parameters related to the aircraft's weight, wing, air density, etc., \(\mathrm{g}\) is the gravitational acceleration with nominal value 9.8 m/s\({}^{2}\), m is the mass of the UAV including all its payload.