## 5 Simulation Results
Therefore, we set the actor and critic learning rates at \(\alpha_{a}=0.001\) and \(\alpha_{c}=0.005\), respectively, for the rest of the simulations. \begin{table}
\begin{tabular}{l l l} \hline \hline
**Parameter** & **Description** & **Value** \\ \hline \(M\) & Number of groups & 3 \\ \(T_{tot}\) & Time slot duration & 200 ms \\ \(\tau_{ss}\) & Sensing duration & 2 ms \\ \(W\) & Total system bandwidth & 1 Hz \\ \(E_{bat}\) & Battery capacity & 30 ul \\ \(e_{ss}\) & Sensing cost & 1 ul \\ \(e^{tr}\) & Transmission energy & \(0,10,20\) ul \\ \(\xi_{avg}\) & Mean value of harvested energy & 5 ul \\ \(\mu\) & Initial belief that the primary channel is free & 0.5 \\ \(P_{FF}\) & Transition probability of the primary channel from state F to itself & 0.8 \\ \(P_{BF}\) & Transition probability of the primary channel from state B to state F & 0.2 \\ \(P_{d}\) & Probability of detection & 0.9 \\ \(P_{f}\) & Probability of false alarm & 0.1 \\ \(\sigma^{2}\) & Noise variance & \(-80\) dB \\ \(\gamma\) & Discount factor & 0.9 \\ \(\alpha_{a}\) & Learning rate of the actor & 0.001 \\ \(\alpha_{c}\) & Learning rate of the critic & 0.005 \\ \(\epsilon\) & Epsilon rate & 1 \(\rightarrow\) 0.01 \\ \(\epsilon_{d}\) & Epsilon decay & 0.9999 \\ \(L\) & Number of episodes & 300 \\ \(T\) & Number of iterations per episode & 2000 \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Simulation Parameters. Figure 8: The convergence rate of the proposed actor-critic deep reinforcement learning with different training steps in each episode. Figure 10 illustrates the average transmission rates under the influence of mean harvested energy. We can see that the average transmission rate of the system increases when the mean value of harvested energy grows. The reason is that with an increase in \(\xi_{avg}\), the CUs can harvest more solar energy, and thus, the CUs have a greater chance to transmit data to the CBS. In addition, the average transmission rate of the proposed scheme dominates the conventional schemes because the conventional schemes focus on maximizing the current reward, and they ignore the impact of the current decision on the future reward. Thus, whenever the primary channel is free, these conventional schemes allow all CUs to transmit their data by consuming most of the energy in the battery in order to maximize the instant reward. This makes the CUs stay silent in the future due to energy shortages. Although the Myopic-Random scheme had lower performance than the Myopic-UP scheme, it still had greater rewards than Myopic-OMA. This outcome demonstrates the efficiency of the hybrid NOMA/OMA approach, compared with the OMA approach, in terms of average transmission rate. Figure 10: Average transmission rate according to different values for mean harvested energy. Figure 9: The convergence rate of the proposed actor-critic deep reinforcement learning according to different learning rate values. In Figure 11, the energy efficiency of the schemes was compared with respect to the mean value of the harvested energy. In this paper, we define energy efficiency as the transmission data rate obtained at the CBS over the total energy consumption of the CUs during the operations. We can see that the energy efficiency declines as \(\xi_{avg}\) rises. The reason is that when the harvested energy goes up, the CUs can gather more energy for their operations; however, the amount of energy overflowing the CUs' batteries also increases. The curves show that the performance of the proposed scheme outperforms the other conventional schemes because the DACRL agent can learn about the dynamic arrival of harvested energy from the environment. Thus, the proposed scheme can make proper decision in each time slot. In Figures 12 and 13, we plot the average transmission rate and the energy efficiency, respectively, based on differing noise variance at the CBS. The curves show that system performance notably degrades when noise variance increases. To explain this, noise variance will degrade the data transmission rate, as shown in Equation (3).