## 5 Simulation Results  
In this section, we investigate the performance of uplink NOMA systems using our proposed scheme. The simulation results are compared with other myopic schemes [48] (Myopic-UP, Myopic-Random, and Myopic-OMA) in terms of average data transmission rate and energy efficiency. In the myopic schemes, the system only maximizes the reward in the current time slot, and the system bandwidth is allocated to the group only if it has at least one active CU in the current time slot. In particular, with the Myopic-UP scheme, the CBS arranges the CUs into different pairs based on Algorithm 1. In the Myopic-Random scheme, the CBS randomly decouples the CUs into pairs. In the Myopic-OMA scheme, the total system bandwidth is divided equally into sub-channels in order to assign them to each active CU without applying user pairing. In the following, we analyze the influence of the network parameters on the schemes through the numerical results. In this paper, we used Python 3.7 with the TensorFlow deep learning library to implement the DACRL algorithm. Herein, we consider a network based on different channel gain values between the CUs and the CBS, such as \(h_{1}=-20\) dB, \(h_{2}=-25\) dB, \(h_{3}=-30\) dB, \(h_{4}=-35\) dB, \(h_{5}=-40\) dB, \(h_{6}=-45\) dB, where \(h_{1},h_{2},h_{3},h_{4},h_{5},h_{6}\) are the channel gains between \(CU_{1},CU_{2},CU_{3},CU_{4},CU_{5},CU_{6}\) and the CBS, respectively. Two sequential DNNs are utilized to model the value function and the policy function in the proposed algorithm. Each DNN is designed with an input layer, two hidden layers and an output layer as described in Section 4. The number of neurons in each hidden layer of the value function DNN in the critic, and the policy function in the actor, are set at \(H_{C}=24\) and \(H_{A}=24\), respectively. For the training process, value function parameter \(\mathbf{\omega}\) and the policy parameter \(\mathbf{\theta}\) are stochastically initialized by using uniform Xavier initialization [49]. The other simulation parameters for the system are shown in Table 1.  
We first examine the average transmission rates of the the DACRL scheme under different training iterations, \(T\), while the number of episodes, \(L\), increases from 1 to 400. We achieved the results by calculating the average transmission rate after separately running the simulation 20 times, as shown in Figure 8. The curves sharply increase in the first 50 training episodes, and then gradually converge to the optimal value. We can see that the agent needs more than 350 episodes to learn the optimal policy at \(T=1000\) iterations per episode. However, with the increment in \(T\), the algorithm begins to converge faster. For instance, the proposed scheme learns the optimal policy in less than 300 episodes when \(T=2000\). Nevertheless, it might take a very long time for the training process if each episode uses too many iterations, and the algorithm evenly converges to a locally optimal policy. As a result, the number of training iterations per episode and the number of training episodes should not be too large or too small. In the rest of the simulations, we set training episodes at 300 and training iterations at 2000. Figure 7: The deep neural network in the actor. Figure 9 shows the convergence rate of the proposed scheme according to various values of actor learning rate \(\alpha_{a}\) and critic learning rate \(\alpha_{c}\). The figure shows that the reward converges faster with increments in the learning rates. In addition, we can observe that the proposed scheme with actor learning rate \(\alpha_{a}=0.001\) and critic learning rate \(\alpha_{c}=0.005\) provides the best performance after 300 episodes. When the learning rates of the actor and the critic increase to \(\alpha_{a}=0.01\) and \(\alpha_{c}=0.005\), respectively, the algorithm converges very fast, but does not bring a good reward due to underfitting.