# Deep Reinforcement Learning for UAV-Based SDWSN Data Collection
###### Abstract
Contemporary advancements have introduced methodologies that augment decision-making capabilities in UAV flight path determination and network communication, particularly through the application of deep learning (DL) algorithms [8]. DL, a neural network-centric subset of machine learning, is characterized by its multilayered structure, enabling the processing of extensive datasets and the deciphering of complex interdependencies. Complementarily, reinforcement learning (RL) represents another machine learning strategy, distinguished by its dynamic adaptation through action modification in response to ongoing feedback, with the aim of optimizing a reward function. This technique is optimally suited for scenarios where pre-emptive rule formulation is impractical, as it facilitates experiential learning from errors and subsequent behavioral adjustments. Consequently, RL is adept at navigating multifaceted decision-making processes and addressing pragmatic challenges. Recent scholarly advancements have introduced Deep Reinforcement Learning (DRL), an integrative approach that combines DL and RL. This empowers systems to handle more complex tasks by autonomously extracting features from raw data. DRL has become a prominent method for strategically addressing UAV flight path planning challenges [9; 10; 11; 12]. However, optimizing a UAV path using DRL in conjunction with SDWSN-UAV communication has not been considered in the literature. It is worth mentioning that the UAV path can be formulated based on data gathered from the WSN, where the multiple roles/functions of some nodes can enhance system performance by reducing energy consumption and delay, especially in scenarios requiring network restructuring. This paper intends to introduce the \"UAV Fuzzy Travel Path\", a novel approach that leverages DRL algorithms for optimal UAV flight path planning to meet the requirements of various environmental applications. Herein, we also shed light on the integration between SDWSN and UAV communication, emphasizing an optimized flight path. This integration provides flexibility in network structure, allowing sensor nodes to be configured with one or multiple functions through software definitions to meet various network scenarios. For instance, the functional role of a node can be changed at the virtual level by implementing any of the three functions, such as reconfiguring a leaf node to act as a router node and vice versa. Consequently, the network's operational behavior can be adjusted to meet service requirements. In our previous work [13], we detailed the configuration of SDWSN functions on a virtual platform using the Contiki-Cooja network simulator, where each function was coded in C language to perform tasks such as data processing and computation. The proposed UAV-assisted SDWSN model features nodes configured with a gateway function (GW) that receives data from dynamically formed group members via software definition. The node configuration using the Contiki Operating System (OS) mirrors the hardware configuration, utilizing hardware libraries. Concurrently, a UAV is deployed to collect data from GWs along a pre-planned trajectory within a fuzzy span. This approach will be utilized in this paper to investigate the design of an optimized UAV path using DRL that aligns with the data-gathering approach from SDWSN. Our primary aim is to highlight the benefits of integrating SDWSN for applications such as environmental monitoring and UAV path design using DRL. The main contributions of this work are summarized as follows:  
* Utilizing SDWSN Functions: We leverage the flexibility of software-defined wireless sensor networks (SDWSN) to configure network functions. Specifically, we formulate SDWSN to facilitate UAV communication with the wireless sensor network (WSN) and efficient data collection. Notably, nodes within the network can serve as gateway functions, enabling seamless data gathering by the UAV.