# Deep Reinforcement Learning Based Resource Management in UAV-Assisted IoT Networks
## 1 Introduction
Additionally, this paper uses a large dataset with the integration of DRL and K-means. The real data collection includes both the signal variations and their LOS/ non-line of sight (NLOS) views. Thus, the DRL can reduce the computational times and is a better way to test a large number of datasets, and produce an optimal strategy with a range of real environments relative to conventional RM approaches [21]. Finally, it inspired us to investigate the issues based on the application of the DRL approach. Figure 1 illustrates the general architecture of the DRL algorithm for our system design. The architecture displays both the primary and target networks, which contains all the obtained dataset values and the final results for allocating resources, respectively. Then, the weights of DQN are outlined to update the loss values based on the primary and target networks. Additionally, the action, state, and reward section values are considered in the environment. Then, all these are updated in the next action, next state and next reward, respectively. In the end, all values are stored into replay memory and used for making policies and decisions. Therefore, the proposed DRL learning architecture can learn user data from the grid points, clusters, and UAV altitudes. As the proposed learning architecture can automatically learn the characteristics of the environment based on the learning input sequences with different time scales. The major contributions of our work can be summarized as follows: We consider a UAV-assisted wireless IoT network and assess the RM schemes. Then, we propose a multi-agent DRL with a round robin resource-scheduling algorithm for the optimization of joint RM. A joint RM optimization method is proposed to minimize the power consumption and maximize the user throughput and the signal-to-interference-plus-noise ratio (SINR)1. We design a system model that is connected to IoT users, UAV-BS, and A2G channel access links. This framework contains a DRL based RM problem with multiple constraints, such as the number of users, channel gains, signal noise ratio (SNR) issues, and power consumption levels. These variant parameters are used to characterize the heterogeneous and dynamic nature of the environment at each time slot. 2. The DRL is applied for the development of the main system model with K-means as a clustering approach. Then, the round-robin algorithm is used to handle the service request queue for the IoT users. The IoT devices are clustered into cluster 1 (urban) and cluster 2 (sub-urban) based on their location and signal distribution. This makes our system more computationally efficient and stable. The proposed DRL framework is therefore used to perform the optimal RM for the UAV assisted IoT devices. Additionally, the DRL techniques with neuron activation mechanisms are used to compare and evaluate the impact of neuron activation on the convergence of the proposed system. 3. Ultimately, based on the proposed system, two scenarios (cluster 1 and cluster 2) are used for system evaluation. It is important to handle mobile users within the transmission range of the A2G based UAV access link. In the rest of the paper, Section 2 provides the related works, and Section 3 explains the system model architecture. In the Methods and Materials Section, such as experimental setup and data collection procedures, are discussed in Section 4. Section 5 provides the proposed system design, such as the round robin scheduling-algorithm for resource management, multi-agent DRL for joint resource management, proposed system architecture, and evaluation metrics. Section 6 reports the experimental results and discussions. Finally, the conclusion and future works are outlined in Section 7. Lastly, references are listed.