# Deep Reinforcement Learning Based Resource Management in UAV-Assisted IoT Networks
## 3 System Model  
Figure 2 displays the UAV-assisted IoT system model. It consists of an A2G signal access link transmission, LOS/NLOS views, and the altitude between the UAV and the user (\(D_{um}\)). Assume that UAV is used by ground users to provide access-sharing information. The UAV as a base station is used and deployed at altitudes of 30 and 60 m and reach all the users randomly distributed on the ground inside the actual area. The OFDM signal, as the received signal strength (RSS) and SNR values, which could influence any user equipment, has been collected. The transmitter (Tx), i.e., the UAV, is represented as M and the receiver (Rx), i.e., the mobile user, is represented as U. The system architecture configuration covers the network location with 35 users, denoted by _U1_, _U2_, _U3_,..., _U35_, serviced by UAV. Figure 2 shows the relation between the M-U A2G link; clusters are used as target areas. All mobile users operate on the same frequency band. Therefore, all users are expected to receive the same OFDM signals. The control station is used to monitor and access the network. The defined clusters are presumed to be urban and sub-urban for cluster 1 and cluster 2, respectively. Then, the deployed UAV reaches both clusters based on their altitude and user interest for resource usage. The path loss (\(P_{L}\)) model in the states as S=LOS or S=NLOS is shown in Equation (1) according to References [37; 38],  
\[P_{L}s(D_{um})=29.5\;log(D_{um})+3.5+20\;log(f_{c})+\;\epsilon s, \tag{1}\]  
where \(D_{um}\) is the altitude between UAV and the user (in m), \(\alpha\) is the path loss exponent, \(\epsilon\) models the shadowing effect, which is a Gaussian random variable, N(0, \(\sigma_{s}^{2}\)), while \(f_{c}\) is the carrier frequency. Then, the channel gain is composed of the \(P_{L}s(D_{um})\) and \(S(D_{um})\), which are the path loss and small-scale fading coefficient between UAV to a user, respectively. Then, the channel gain can be expressed, as in Equation (2),  
\[C(D_{um})=P_{L}s(D_{um})S(D_{um}), \tag{2}\]  
To estimate the interference, it is assumed that all N channel resources are occupied by M to users [26; 27]. The UAV at the altitude of 30 m and 60 m has an estimation of SINR on the channel n, which is given by:  
\[SINR_{\text{C}}^{n}=\frac{Pt^{1}C(D_{um})1P_{L}s(D_{um})}{\sigma_{s}^{2}+ \sum_{n}D_{um}Pt^{1}\;C(D_{um})1+\;I_{n,\;N}}, \tag{3}\]  
Figure 2: System model: UAV-Bs: unmanned aerial vehicle as base stations; A2G: air-to-ground. where \(Pt^{1}\) and \(C(D_{um})1\) are the power transmission and the channel gain ratio from M to U1 to the control station, respectively; \(I_{n,\ N}\) is the noise power, \(D_{um}n\) is the altitude from M to Un. Therefore, the overall SINR is expressed as follows:  
\[SINR_{Mu}^{n}=\frac{Pt^{1}C(D_{um})}{\sigma_{s}^{2}+\sum_{n}D_{um}Pt^{1}C(D_{um })n\ +\ Pt^{D_{um}}G^{D_{um}}\ +\ I_{n,\ N}}\,, \tag{4}\]  
where \(C(D_{um})\) and \(C(D_{um})n\) are the channel gains between M to U and M to Un based on the altitude of the Tx and Rx, respectively; \(Pt^{D_{um}}\) is the power transmission between M to U. Assuming that there is a physical capacity of \(Phy_{u}\) and user throughput \(T_{U}\). Then, there is a user throughput capacity, where \(T_{U}\) has an effective throughput of \(T_{eu}\), \(M_{U}\) is the number of users who interact with UAV. Then, \(T_{s}\) is the time slot of a user communicates with \((D_{um})\) to use the A2G link. The resource request (\(R_{U}\)) ratio of the user, then the maximum \(T_{U}\), are analyzed in Equation (5). Therefore, the total throughput is measured at a grid level with clusters.