# Deep Reinforcement Learning Based Resource Management in UAV-Assisted IoT Networks
## 1 Introduction  
Recently, the development of IoT applications, the consideration of efficient wireless service for IoT devices, and the issue of resource management (RM) become vital. The overall performance of RM involves the efficient and dynamic use of resources such as times, bandwidth, and frequency [1]. Therefore, higher throughput, higher data rate, lower interference, and better coverage are appropriate considerations for RM in the IoT-based wireless networks. The unmanned aerial vehicles (UAVs) are used in large-scale applications such as security inspection, aerial patrol, and traffic assessment [2]. Hence, UAV-assisted resource management becomes vital for the advancement of the fifth-generation networks. The reasons for the use of UAVs to assist the resource management include: (i) it can be used for rapid management of resource requests from the overloaded IoT users; (ii) it is easy to accommodate the further growth of system coverage and capacity; (iii) it can be operated at different altitudes; and (iv) it can provide rapid and on-demand service for IoT users [3, 4]. Due to the higher altitude and greater coverage, aerial base stations have a higher chance of having line-of-sight (LOS) connections with ground users. Therefore, to provide reliable services to IoT users, UAV based resource management is an essential and hot issue for enhancing the resource utilization of IoT networks [5, 6]. In order to improve the accuracy of regression and classification tasks in the RM scheme, deep learning methods are more preferable over conventional machine learning methods [7]. Additionally, deep learning methods allow extracting features automatically from large datasets [8; 9]. Specifically, deep reinforcement learning (DRL) has made a substantial improvement in RM that is difficult to model with conventional approaches [5]. Mainly, conventional methods have faced a significant challenge due to the wide range and complexity of wireless networks [10]. To this end, the gaps of traditional machine learning-based RM schemes, such as the model complexity, costly use training, and the generalization from test workloads to the actual application of user workloads are identified. Therefore, to solve the most complicated RM problems in IoT networks, this paper proposes a DRL method, in which the state, action, and reward are important parameters that should be designed to generate an optimal policy [11]. The DRL-based approach has been applied to various fields, such as resource management and allocation [12; 13], dynamic channel access [12], mobile offloading [13], mobile and unified edge computing caching, and communication (3C) [14; 15; 16], fog radio access networks [17; 18]. The implementation of DRL is more successful than the single agent Q-learning approach [19]. However, most of them were limited to designing and analyzing the DRL-based method in fixed base stations for solving the joint resource allocation problems. As a result, our proposed method can solve the joint RM optimization by using the DRL approach. According to Reference [20], the DRL agents consider the maximum long-term rewards rather than simply obtaining the current optimal rewards. This is critical for time-changing and dynamic systems. This paper focuses on UAV-based RM with the application of the DRL approach. Our main initiatives are applying a multi-agent-based DRL model, round robin with K-means for user request queue, and clustering tasks. To the best of our knowledge, these issues are open ones that have not yet been fully investigated. To jointly manage user data, as well as UAV-based orthogonal frequency-division multiplexing (OFDM) signal values in a real environment, the DRL approach is applied. We considered one UAV connected through wireless backhauls with the core network. Under this layout, the objective is to optimize the management of resources for IoT users based on the A2G access links. Different from previous studies, our study focuses on the actual data collection environment.