## 9 Simulation Results and Discussion
These findings verify the results obtained for BER in Figure 13, which indicate that the FPA scheme is more adequate for user devices with high channel gains. Figure 12: BER vs. Power (Q-learning, Conventional NOMA (MMSE)—Rician channel). In Figure 15, attainable rates for user devices are simulated against power transmitted when OPS and FPA schemes are applied in conjunction with the proposed Q-learning algorithm that is applied as a channel estimator. Results for far and middle devices point out that OPS provides 1 b/s/Hz improvement compared to the FPA scheme. This limited improvement might be clarified where the management of the power allocation for devices is not necessarily sufficient enough to alleviate the influence of interference particularly for the device. Figure 14: Outage Prob. vs. Power (\(Q\)-learning, Optimization, FPA). Figure 5:0 BER vs. Power (\(Q\)-learning, Optimization, FPA). far and middle devices that mainly experience unstable links environments. As expected, results for near user device reveal superiority in achieved rate with respect to middle and far devices with at least 10 b/s/Hz. Furthermore, the results for the near user with FPA indicate a noticeable improvement compared to OPS, which validates the results obtained in Figures 13 and 14. In the end, we can further provide the analysis of the computational complexity as follows: The complexity of the reinforcement learning algorithm mainly depends on the size of the state space and the size of the action space [51]. According to [51], we can approximate the computational complexity of the Q-learning algorithm as \(\mathcal{O}(SAH)\) per iteration, where \(S\) is the number of states, \(A\) is the number of actions, and \(H\) is the number of steps per episode. According to the state space and action space defined in our simulation environment, the amount of work per iteration can be approximated as \(\mathcal{O}(NMK)\)[51; 52], where \(N\) represents a number of antennas in BS, \(M\) represent a number of user devices in the cell, and \(K\) represents the size of channel coefficients. On the other hand, the computational complexity for the benchmark scheme implemented in [15], is described as follows: the sizes of the input layer, the first hidden layer, the second hidden layer, and the output layer for each network implemented in [15] is denoted as \(I\), \(h_{1}\), \(h_{2}\), and \(U\) respectively. Thus, the total number of parameters in each network can be denoted as \(\theta=I+h_{1}+h_{2}+U\), therefore, the complexity of this scheme regarding the channel estimation task can be approximated as \(\mathcal{O}(MN_{A}(I+h_{1}+h_{2}+U)\)[15], where \(M\) represents the number of user terminals and \(N_{A}\) represent the number of antennas at BS. According to [53], the corresponding computational complexity for the traditional channel estimation method based MMSE can achieve a relatively low complexity, \(\mathcal{O}\big{(}M^{2.37}\big{)}\)[45; 53], but, at the cost of performance degradation. Based on the aforementioned analysis, it can be shown that the complexity of the developed RL based Q-learning algorithm is competitive compared to other procedures. Figure 5:1 Individual rate vs. Power (\(Q\)-learning, Optimization, FPA).