# Deep Reinforcement Learning for UAV-Based SDWSN Data Collection
###### Abstract  
Recent advancements in Unmanned Aerial Vehicle (UAV) technology have made them effective platforms for data capture in applications like environmental monitoring. UAVs, acting as mobile data ferries, can significantly improve ground network performance by involving ground network representatives in data collection. These representatives communicate opportunistically with accessible UAVs. Emerging technologies such as Software Defined Wireless Sensor Networks (SDWSN), wherein the role/function of sensor nodes is defined via software, can offer a flexible operation for UAV data-gathering approaches. In this paper, we introduce the \"UAV Fuzzy Travel Path\", a novel approach that utilizes Deep Reinforcement Learning (DRL) algorithms, which is a subfield of machine learning, for optimal UAV trajectory planning. The approach also involves the integration between UAV and SDWSN wherein nodes acting as gateways (GWs) receive data from the flexibly formulated group members via software definition. A UAV is then dispatched to capture data from GWs along a planned trajectory within a fuzzy span. Our dual objectives are to minimize the total energy consumption of the UAV system during each data collection round and to enhance the communication bit rate on the UAV-Ground connectivity. We formulate this problem as a constrained combinatorial optimization problem, jointly planning the UAV path with improved communication performance. To tackle the NP-hard nature of this problem, we propose a novel DRL technique based on Deep Q-Learning. By learning from UAV path policy experiences, our approach efficiently reduces energy consumption while maximizing packet delivery. unmanned aerial vehicle (UAV); software-defined wireless sensor networks (SDWSN); fuzzy UAV route; deep reinforcement learning (DRL) +
Footnote †: journal:  
roles: efficient data monitoring and collection from a wireless network, i.e., WSN, and data computation and decision-making. To enable flexible UAV-WSN operations, the WSN structure plays a crucial role in facilitating efficient data dissemination. Various grouping and clustering methods can organize wireless sensor nodes into star or tree structures, with representative nodes responsible for data gathering. Additionally, WSN topology can benefit from the latest technologies such as network virtualization and software-defined networking (SDN) [4; 5]. In this context, wireless sensor nodes can be configured with a given function through software definition encouraging the conceptual development of software-defined wireless sensor networks (SDWSN). A function can be represented by the terms \"Leaf Sensor Node\", \"Router Node\", or/and \"IoT Gateway Node\", which can be modeled and tested on a virtual platform before physical deployment [6]. Herein, simulators like the Contiki-Cooja offer a virtual environment for testing and have the capability to simulate those functions and test their performance prior to physical implementation using hardware boards like Texas Instruments CC2538. Therefore, SDWSN and virtualization are significant components of the IoT architecture as they contribute to the flexible operation of the sensor network and hence offer a degree of freedom to the UAV flight path. For integrating the communication between configured WSN and UAV, simulators like CupCarbon can offer smooth communication testing between UAV and WSN. From the UAV path planning point of view, the algorithms of path planning design are crucial for creating adaptable flight paths that enable efficient data gathering by gateway nodes while minimizing energy use. Researchers have examined UAV path design considering factors like speed, communication duration, energy expenditure, and communication protocols [7]. However, factors related to the UAV path, which can be efficiently adjusted based on the reconfigurability of the nodes in the WSN, need to be considered to ensure smooth UAV-WSN communication.