## 3 System Model  
In this paper, we considered \(M-1\) multiple ABSs accompanied by a single antenna and one ground base station (GBS) (\(m=0\)) in a downlink heterogeneous network model, with the total base stations represented by the set \(\mathcal{M}\). We randomly distributed the total \(U\) of users inside the coverage area where the total users are represented by the set \(\mathcal{U}\). The ABSs were deployed with the GBS where both used the same frequency, which may cause intercell interference. Resources were allocated orthogonally within the cell to avoid intracell interference. The coordinates for the users are represented as \((x_{u},y_{u})\) and the 3D coordinates of the ABSs are represented as \((x_{m},y_{m},h_{m})\) for the base stations, as shown in Figure 3. However, to model the emergency communication scenario, we considered that the GBS did not exist and the remaining users were connected with the ABSs based on their priority. The key notations and symbols utilized in the paper are summarized in Table 2.  
Figure 2: RL categories; model-free and model-based approaches. The channel model from ABS to ground user communication can be line-of-sight (LoS) or non-line-of-sight (NLoS). LoS communication depends on factors such as building density, users' location, and placement of the ABS as well as the angle between the ABS and the user. The LoS channel is modeled as Rician fading whereas the NLoS is modeled as Rayleigh fading. Therefore, the channel distribution of the channel fading, \(l\) is given by:  
\[f_{l}(l)= \begin{cases}f_{\text{LoS}}(l)&\text{for LoS case}\\ f_{\text{NLoS}}(l)&\text{for NLoS case}\end{cases} \]  
where \(f_{\text{NLoS}}(l)\) and \(f_{\text{NLoS}}(l)\) follow a noncentral Chi-squared distribution and exponential distribution, respectively, and are given by [28] \begin{table}
\begin{tabular}{c c} \hline \hline
**Notations** & **Description** \\ \hline ABS & Aerial base station \\ GBS & Ground base station \\ N & Number of users connected with ABS \\ \((x_{m},y_{m},h_{m})\) & ABS coordinates \\ \((x_{u},y_{u})\) & Users coordinates \\ \(P_{m}\) & ABS transmit power \\ \(\theta_{u}\) & Elevation angle \\ \(a,b\) & Environmental parameters \\ \(f_{c}\) & Carrier frequency \\ \(d\) & Distance between ABS and ground user \\ \(\eta_{LoS}\) & Mean additional loss for LoS \\ \(\eta_{NLoS}\) & Mean additional loss for NLoS \\ \(\gamma_{th}\) & SINR threshold \\ \(r_{t},s_{t},a_{t}\) & Reward, state, action at time \(t\) \\ \(\hat{s_{t}},\hat{a_{t}}\) & Next state and action \\ \(\beta\) & Discount factor \\ \(\alpha\) & Learning rate \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Notations and description. Figure 3: System model. \[f_{\text{LoS}}(l)= \frac{1+K(\theta_{u})}{\overline{H_{\text{L}}}}\exp\biggl{(}-K( \theta_{u})-\frac{1+K(\theta_{u})}{\overline{H_{\text{LoS}}}}l\biggr{)}\] \[\times I_{0}\biggl{(}2\sqrt{\frac{K(\theta_{u})(1+K(\theta_{u}))}{ \overline{H_{\text{LoS}}}}}l\biggr{)}\] \[= \frac{1}{2}\exp\biggl{(}-K(\theta_{u})-\frac{l}{2}\biggr{)}I_{0} \biggl{(}\sqrt{2K(\theta_{u})l}\biggr{)}\] \[f_{\text{NLoS}}(l)= \frac{1}{\overline{H_{\text{NLoS}}}}\exp\biggl{(}-\frac{l}{ \overline{H_{\text{NLoS}}}}\biggr{)}=\exp(-l).\]  
where \(I_{0}(\cdot)\) is the first kind with zero order Bessel function, and \(\overline{H_{\text{LoS}}}=2+2K(\theta_{u})\) and \(\overline{H_{\text{NLoS}}}=1\) are the mean of the LoS and NLoS fading gains, respectively. The probability of LoS is defined as:  
\[P_{LoS}=\frac{1}{1+e^{-b(\theta_{u}-a)}} \tag{1}\]  
Here, \(a\) and \(b\) are environmental constant, \(\theta_{u}\) is the elevation angle that depends on the ABS height and also on the distance between users and ABSs.