# Deep Reinforcement Learning for UAV-Based SDWSN Data Collection
## 2 Related Work  
Intelligent unmanned aerial vehicle (UAV) path planning is a critical prerequisite for a successful and real-time operation. As a result, optimizing UAV flight missions can be a solution for offering the fastest and most efficient path. With a focus on mission planning, researchers investigated several optimization techniques such as the multi-objective particle swarm optimization (PSO) algorithm. However, with the advancement of technology and recent advances in intelligent computation, virtualisation, softwarisation, artificial intelligence, and learning methods, prediction strategies and decision-making for the optimal UAV path are required. In our related work section, we explore various aspects to achieve an optimal UAV path for collecting and analysing data from Wireless Sensor Networks (WSN) situated in both sparse and dense fields. Specifically, we delve into areas related to clustering in WSN, Software-Defined Wireless Sensor Networks (SDWSN), virtualization, UAV path trajectories, and Deep Reinforcement Learning (DRL) algorithms for UAV path optimization. Considering the significant flight altitude of UAVs, establishing direct connections between the UAV and all nodes in a wireless sensor network during the data-gathering process is inefficient in terms of energy. To address this challenge, grouping the nodes into groups/clusters is looked at based on different communication parameters allowing only the cluster heads (CH) to communicate with the drone [14]. These include factors like node location, the distance with respect to other nodes that are within the transmission range, communication protocol such as ZigBee, node energy consumption, and the size of the cluster. The clustering process involves two main steps: CH selection and cluster construction. In existing literature, two critical elements stand out for research in this area:topology and data transmission routes wherein developing state-of-the-art topology and efficient data transmission routes within these clusters is essential. This ensures effective communication while minimizing energy consumption [15]. An efficient CH selection algorithm that optimally selects cluster heads is crucial. The CHs play a pivotal role in coordinating the data exchange between the UAV and the WSN nodes. Herein, in the context of ground-based WSN, data gathering often leads to faster battery depletion in sensors located near the sink due to concentrated data traffic. To mitigate this issue, the authors of [16; 17] have explored the use of mobile sinks within the network. However, employing a mobile data ferry for data collection introduces additional challenges. Frequent link changes or failures can render the network topology unstable. To address this, [PERSON] et al. [16] propose a mobile clustering data collection protocol that establishes a highly stable and reliable routing path for data transmission. In their approach, fuzzy logic-based CHs are selected, considering factors such as node speed, the number of neighboring nodes, and average connection time. Notably, this algorithm significantly enhances key performance metrics, including packet delivery ratio, throughput, energy consumption, and end-to-end delay. In the context of employing UAV as a data ferry to assist in ground network data collection, the authors of [18] focus on clustering methods for the ground Sensor Nodes (SNs) in a single UAV scenario for environmental monitoring and data gathering. The energy model for the SNs considers the energy required for packet reporting and packet forwarding in the context of mesh networks, where each node can serve as a router for neighboring nodes toward the sink. UAV path planning involves a heuristic path, which is a local search algorithm used to solve the Traveling Salesman Problem (TSP) for visiting the WSN clusters. The optimization problem revolves around balancing UAV and sensor energy consumption versus the maximum number of network hops. [PERSON] et al.