## 4 Simulations
### Setup  
_Parameter Setting of Environment_: We set the location of seven base stations in the cell center, and four users wererandomly distributed in each cell. The uplink user power was limited to \(P_{\min}=-40\) dBm and \(P_{\max}=23\) dBm [40]. The total bandwidth of the system was \(W=20\) MHz. The minimum throughput requirement of all the users was \(TH^{\text{th}}=0.15\) bit/s, and the power spectral density \(n_{f}\) was \(-174\) dBm/Hz. The size of the cells and channel model change according to different scenarios [39], which are referenced from the test senior in the 3 GPP protocol, such as UMa, RMa, RMi. By default, the outsider scenario of the non-line-of-sight of the RMa was selected to evaluate the proposed method. The RMa stipulates the radius of cell \(r\), and the pathloss is defined as  
\[PL_{l_{m,n}}=\max\big{(}PL_{l_{m,n},1},PL_{l_{m,n},2}\big{)}, \tag{20}\]  
where \(PL_{l_{m,n},1}\) and \(PL_{l_{m,n},2}\) denote the line-of-sight and non-line-of-sight pathloss, respectively, which are written as  
\[PL_{l_{m,n},1}=\left\{ \begin{array}{ll}PL_{l_{m,n},11},&10\,\text{m}<d_{h}< d_{\text{BP}},\\ PL_{l_{m,n},12},&d_{\text{BP}}<d_{h}<5\text{ km},\end{array} \right. \tag{21}\]  
where  
\[PL_{l_{m,n},11} =\min(0.03h_{b}^{c},10)\,\text{lg}(d_{s})-\min(0.044h_{\text{b}}^ {c},14.77) \tag{22}\] \[+0.002\,\text{lg}(h_{\text{b}})\,d_{s}+201\text{lg}(40\pi d_{s}f _{c}),\]  
\[PL_{l_{m,n},12}=PL_{l_{m,n},11}+40\,\text{lg}(\frac{d_{s}}{2\pi h_{\text{a}1} h_{\text{a}2}f_{c}/v}), \tag{23}\]  
and  
\[PL_{l_{m,n},2} =161.4-7.11\,\text{lg}(l_{\text{w}})+7.51\,\text{lg}(l_{\text{b} })-\left(24.37-3.7\frac{h_{\text{b}}^{2}}{h_{\text{a}1}^{2}}\right)\,\text{lg }(h_{\text{a}1}) \tag{24}\] \[+(43.42-3.1\,\text{lg}(h_{\text{a}1}))(\text{lg}\,d_{s}-3)+201\, \text{lg}(f_{c})-10.24\,(\text{lg}(11.75h_{\text{a}2}))^{2}+4.97.\]Here, \(d_{\rm s}=\sqrt{d_{\rm h}^{2}+\left(h_{\rm a1}-h_{\rm a2}\right)^{2}}\) and \(d_{\rm h}=d_{l_{\rm u},n,m}\) denote the straight distance and horizontal distance between BS and user respectively, where \(h_{\rm a1}\) and \(h_{\rm a2}\) are the heights of the antenna in the BS and user, respectively. \(h_{\rm b}\) is the building height, \(l_{\rm w}\) is the average width of the road, and \(\varepsilon\) is the excitation factor. For the long distance line-of-light pathloss \(PL_{l_{\rm u},n,12}\), \(f_{\rm c}\) is the central frequency, and \(v\) denotes the propagation velocity. These parameter settings are listed in Table 1. In this paper, the five benchmarks were considered:  
1. DDQN and DDPG: The existing DDQN for bandwidth assignment and the DDPG for allocating the power. The architecture with a one-layer fully connected network was used in the DDQN, and the DDPG deployed two-layer fully connected networks in the actor and critic networks. Both of them adopted the uniform sampling-based experience replay. 2. DDQN andP-DDPG: The settings were the same as (1), except that the DDPG used the prioritized experience replay. 3. MADDQN and MADDPG(ct): We treated each user as an agent and deployed the DDQN and DDPG on each agent. Centralized training was adopted. 4. MADDQN and MADDPG(dt): The MADDQN and MADDPG with distributed training. Note that each agents had the exclusive reward and loss. 5. Genetic algorithm (GA): The GA framework in the DEAP was used to realize this benchmark [45]. The bandwidth and power allocation schemes were encoded into the chromosome of each individual, which is the action sequence about the bandwidth and power allocation of all the users. We set the population size to 200. The crossover rate and mutation rate were set as 0.8 and 0.05, respectively. Note that the GA depends on the fitness rather than the learning-based reward; thus it is appropriate to compare the results after final convergence instead of comparing the entire optimization process with the learning-based approach. Hyperparameter Setting of JPRL: The JPRL method contains an MADDQN algorithm and a P-MADDPG algorithm.