# Deep Reinforcement Learning for UAV-Based SDWSN Data Collection
## 2 Related Work
[19] and [PERSON] [20] explored the use of UAV as a mobile agent for collecting data from SNs. Their research considers the impact of clustering parameters on WSN, making their methods relevant for environmental monitoring applications. The findings suggest that the collaborative approach of UAV-assisted WSNs improves performance in both precision agriculture and ecological agriculture use cases. The challenge in WSN data gathering using UAVs as mobile sinks lies in achieving a balanced trade-off between the length of the UAV path and the energy consumption of SNs. In their work, [PERSON] et al. [21] propose a novel projection-based method for CH selection and outline a forwarding tree structure. This method efficiently collects and transfers data from SNs and to the CHs, which are responsible for aggregating data from all other nodes within the same cluster. The aggregated data are then transmitted to the UAV acting as a mobile sink, ultimately delivering it to the base station. The UAV follows a trajectory that minimizes the total distance by passing through all the selected CHs once and arriving at the destination. The clustering problem used in the research is based on an updated version of _K-means_ clustering [22]. In their proposed algorithm, after applying the K-means clustering method, the clusters are adjusted to achieve nearly equal-sized cells. Simulation results demonstrate that utilizing UAV for data collection at CHs instead of relying solely on the sink significantly reduces the number of relaying transmissions and substantially lowers energy consumption. However, it is worth noting that the approach described in this paper does not align with a realistic ground network formation scenario. Addressing the challenge of data collection from multiple points involves employing dynamic clustering methods, where only the updated CHs communicate with the UAV. Therefore, developing a novel dynamic network clustering architecture that allows for network re-orchestration based on the UAV's path, and proposing an energy-efficient data communication strategy among ground SNs, are key research areas in UAV-assisted WSN data collection. To evaluate the communication link before real-world implementation, various methods can be used to virtualize network behavior within the cloud. Efficient utilization of WSN deployments allows multiple applications to coexist on the same virtualized WSN [23]. Virtualization has been recognized as an effective method for conducting software-based simulation testing and adapting these simulations for physical networks. Software-driven virtualization provides a platform for testing and analyzing network scenarios, including dynamic behaviors. This parallel co-simulation, executed in the cloud backend, can streamline the network configuration process by eliminating the need for hardware during testing. For instance, the network simulator Contiki-Cooja is used as a virtualization platform for specific target hardware, such as the TI CC2538 Evaluation Module [24]. In our previous work [6], we explored the virtualization approach for vehicular networks, introducing the innovative concept of controlling WSN functions via the remote cloud. This approach was used to test potential re-orchestration scenarios within the vehicular network. By decoupling the functions of vehicular nodes and loading them as software components in the remote cloud, we enabled the re-organization and integration of the network. This allows for dynamic re-orchestration, where a node's functional role can be altered through software reconfigurations at the virtual level. Consequently, the operational behavior of the vehicular network can be adjusted to meet service requirements. Also, in our previous work [25], we proposed a point-by-point air-to-ground communication system that leverages a clustering structure to partition the ground network into small clusters of sensor nodes distributed over large areas. This structure supports efficient communication between sensor nodes and the UAV.