# Deep Reinforcement Learning Based Resource Management in UAV-Assisted IoT Networks
## 7 Conclusions  
By applying the DRL, this study focuses on UAV-based resource management on cellular and IoT networks. Initially, we started by identifying the challenges of resource management in IoT networks assisted by UAV-BSs. Then, we reviewed the traditional resource management mechanisms for IoT networks and assessed the usage of DRL techniques for resource management. Subsequently, a multi-agent DRL approach was proposed in order to obtain an efficient resource management method for UAV-assisted IoT communication systems. The resource management algorithm is used to manage the bandwidth, throughput, interference, and power usage issues. First, we looked at the actual data collection setting for joint RM issues. Then, we used a DRL for system development with K-means for clustering as well as round robin for service request queue. To improve the RM scheme, our proposed approach allows for allocating the available resources with UAV  
\begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Year** & **Algorithm Used** & **Accuracy** & **RMSE** & **Testing Time(s)** \\ \hline
2018 [22] & DRL & 87\% & 5.2\% & 2.5 \\ \hline
2019 [29] & Q-learning + D3 QN & 81.3\% & 7.33\% & 2.01 \\ Our proposed method & DRL + K-means + & 94\% & 2.40\% & 1.05 \\  & Round-robin & & & \\ \hline \hline \end{tabular}
\end{table}
Table 7: Comparison of our proposed system with previous works.  
Figure 10: Actual and predicted value evaluation of throughput distributions.  
to IoT users. Then, with the measurement of accuracy, RMSE and testing time(s), our proposed method was compared to previous works. The proposed system was found to have a 94% RM accuracy with respect to the classification scheme. It achieved RMSE and testing times(s) levels of 2.40% and 1.05s, respectively, for the selected scenarios. Additionally, our method was found to have a better evaluation of precision and recall estimation than the Q-learning approach as 88.05%, 93.04% and 92%, 90.01% for cluster 1 and 2, respectively. Thus, the result demonstrates the well-trained multi-agent DRL learned from sequential features. The proposed method is better for generating optimal policy with low computational complexity than a single-agent-based Q-learning method. Therefore, the proposed DRL model can manage the resource management scheme in dynamic IoT environments. For future work, this study will be expanded to take into account the mobility of users in dynamic environments with a focus on additional resource considerations.  
Conceptualization, H.-P.L., and D.-B.L.; Formal analysis, R.-T.J.; Funding acquisition, H.-P.L., and D.-B.L.; Methodology, Y.Y.M.; Project administration, H.-P.L.; Software, Y.Y.M., and G.B.T.; Supervision, R.-T.J., and D.-B.L.; Validation, G.B.T.; Writing--original draft, Y.Y.M.; Writing-review & editing, R.-T.J. All authors have read and agreed to the published version of the manuscript.  
This work was partially supported by the Ministry of Science and Technology (MOST), Taiwan, under Grants 109-2222-E-035-003-MY2.  
The data and codes presented in this study are available from the corresponding author by request.  
The authors declare no conflict of interest.