# Deep Reinforcement Learning Based Resource Management in UAV-Assisted IoT Networks
## 4 Methods and Materials
### Data Collection Procedures  
For the management of resources, we conducted a real experiment involving data measurement at the NTUT, Taiwan. To collect our data, UAVs were used as Tx. The data measurements were performed at both Tx-Rx sides as a base station and user equipment (UE) features. Following Reference [5], the location pair of horizontal and vertical (elevation) angles were enabled to estimate the direction of all resources. In each cluster, square grid areas were used for OFDM signal collection. We used a variety of devices to represent ecological characteristics because it can be used to reduce the signal variations and processing time of the dataset. Hence, smartphones such as Samsung, Huawei, and Apple were used. At each grid level, the 35 OFDM signal was composed within a 10 second interval from one reference point to the next. Three separate days were used for data collection. Hence, to match a real situation with the environment, we measured UAV elevations at 30 m and 60 m with the distributed mobile users. The dipole antennas on the UAV and a USRP unit were used. A power amplifier and processor were also used to measure the actual OFDM signal for each Tx to Rx. The USRP device was used on the Tx side to shape the software-defined radio (SDR) scheme. Additionally, the output power of the Tx antenna was set to 29.5 dBm. Table 1 illustrates the sample parameter values with the UAV-BS. The remaining parameters are as follows. For 30 m and 60 m altitudes, the  
Figure 3: (**a**) Actual NTUT data collection layout. (**b**) Architecture of orthogonal frequency-division multiplexing (OFDM) signal collection strategy from UAV-base stations (Bs). cluster index is 1 (C\({}_{1}\)) and cluster 2 (C\({}_{2}\)). The Res1 signal values are used as (Rss1, Rss2,... Rssn), signal-noise ratio (SNR) values are used as (SNR1, SNR2,... SNRn) for training. The throughput values are expressed by the values of T\({}_{1}\), T\({}_{2}\)... T\({}_{n}\) for UAV-BS at 30 m and 60 m and each cluster index of 1 and 2, respectively. The power transmission values are expressed as Pt\({}_{1}\), Pt\({}_{2}\)... Pt\({}_{n}\) for UAV-BS for both 30 m and 60 m.  
Table 2 contains the parameters for data collection requirements. Specifically, we applied a clustering approach (i.e., K-means) to cluster the dataset. Then, cluster 1 and cluster 2 are obtained based on their LOS/NLOS view, signal strength values, and fluctuation features. Figure 4 illustrates the methods of data collection, preprocessing, training, and regression phases after the dataset are obtained. Initially, users are randomly distributed to the environment (Stage 1). Stage 2: the clustering phase algorithm is applied to cluster based on user locations, signal variations, and environmental features. Stage 3: preprocessing of the dataset is done. After composing the required dataset, the training process is carried out at Stage 4. Bandwidth, throughput, power gain, and SINR values are selected as target parameters for the RM optimization model. Then, a round robin scheduling-algorithm is applied for sorting the resource request queue. The DRL is precisely trained based on user service requests. In stage 5, testing and regression tasks are carried out with a