# Deep Reinforcement Learning Based Resource Management in UAV-Assisted IoT Networks
## 6 Experimental Results and Discussions  
The experimental outcomes are illustrated in this section. The real data collected from the outdoor environment were used for the evaluation as well as analysis of the results. Prepossessing was performed to label each signal record and minimize the number of irrelevant features. Then, an appropriate and structured training dataset for machine training formats was built. The performance of the proposed method is analyzed through different measurement metrics. Then, different parameters, such as the number of epochs, hidden layers and units, types of optimization, and the activation function were adjusted. Our proposed approach was trained using adam optimizer because it is computationally efficient, requires less memory, and minimize noise. Additionally, specific activation functions such as sigmoid, reLU, tanH, and softmax were used to compress the output of the system. As a result, the optimum values were obtained with a batch size of 500, 200 and 250 epochs/iterations, four hidden layers, and 824 hidden units. The classification and testing schemes were executed through the scenarios, which prove the effectiveness of the proposed approach. Accordingly, the comparison of our system performances with other related algorithms based on clusters 1 and 2 scenarios with UAV altitudes of 30 and 60 m were conducted. Table 4 shows the hardware and software experimental setups for implementing the proposed system. The Keras library of a TensorFlow 3.7 backend with a combination of a variety of deep learning frameworks has been chosen. This is because it is easy to use, has better visualization and has optimized numerical computations. Figure 7a shows the DRL-based classification evaluation of the real distribution OFDM signal values. Then, it is illustrated through the training and testing evaluation. The OFDM_data values are represented in blue color and range from \(-40\) to \(-85\); the training_set and testing_set values are represented in yellow and green colors, respectively. Figure 7b shows the clustering of signal distribution. It shows the real distribution of OFDM signal values in terms of user density. The green squares indicate the signal distribution of cluster 1. The black squares show cluster 2's plotting results. The plotting is performed through the consideration of signal distribution ranges and the LOS/NLOS estimation. \begin{table}
\begin{tabular}{c c} \hline \hline
**Category** & **Tools** \\ \hline Programming language & Python 3.7 \\ Library & Keras libraryTensorFlow \\ CPU & Intel(R) Core(TM) i7-7700 CPU@ 3.60 GHz \\ System Type & 64-bit operating system, \(\times\)64-based processor \\ GPU & NVIDIA 1080 TI \\ RAM & 16 GB \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Hardware and software resources used. Table 5 shows the performance evaluation of single-agent Q-learning and DRL approaches for clusters 1 and 2. As measurement metrics, accuracy, precision, and recall are used. The evaluation is concerned with the average allocation of throughput, interference management, and power consumption issues. In the Q-learning method, 87.03%, 75.00%, and 80% and 85.00%, 80.01%, and 85.02% are the values for accuracy, precision, and recall cluster 1 and cluster 2, respectively. In the DRL method, 94.06%, 88.05%, and 90.01% and 91.2%, 92.00%, and 93.04% are the values for accuracy, precision, and recall in cluster 1 and cluster 2, respectively. Therefore, the evaluation result shows that the performance accuracy is better in DRL evaluation for both clusters for multi-action-based RM optimization in a real and dynamic environment. Thus, multi-agent DRL is better in retrieving relevant features than single-agent Q-learning methods in complex and dynamic environments. Using pie charts, Figure 8 shows each resource management scheme and calculates the QoS satisfaction ratio. The figure provides a detailed performance comparison among the resource management parameters, where the results for the DRL method are obtained with different learning updates.