## 8 Simulation Environment  
Characterization of the simulation parameters and settings is discussed in this section. The examined downlink MISO-NOMA system contains three distinct user devices and one BS in which the BS is supplied with two antennas and every user device in the cell is provided with a single antenna. In the examined NOMA structure, the modulated signals in downlink transmission are superimposed and transferred by BS to user devices via independent Rayleigh or Rician fading channels that are influenced by AWGN with noise power density assigned as \(N_{0}=-174\) dBm/Hz and the path loss is set to 3.5. MATLAB software is utilized as a simulation tool to satisfy the following aims, (1) inspect, characterize, and evaluate the performance of the developed RL based Q-learning algorithm when implemented as a channel estimator in the considered MISO-NOMA system, (2) investigate the reliability of incorporating the developed Q-algorithm as channel estimator scheme with the optimized power scheme in the examined MISO-NOMA network, and performance metrics are considered to explore the impact of this integration. (3) optimized power allocation scheme and fixed power allocation scheme are both compared when the developed Q-learning scheme is utilized as a channel estimator in the cell. Monte-Carlo simulations are performed with \(N=10^{5}\) iterations, and at the outset of each iteration, pilot symbols are randomly generated and recognized at the BS and each device. The main simulation parameters are summarized in Table 1.  
The presented Simulation figures are generated based on the assumption that the channel coefficients are not available at each user device. Thus, in order to examine the effectuality of the developed RL based Q-learning procedure, and for the sake of comparison, four further simulation environments are established, (1) standard minimum mean square error (MMSE) based channel prediction scheme [45]; (2) DL algorithm based on LSTM network for channel prediction applied in [17], (3) RL based actor-critic procedure for channel prediction applied in [15], (4) the fourth simulation environment is dependent on applying RL based State-Action-Reward-State-Action (SARSA) procedure ([PERSON] et al., [PERSON] et al. and [PERSON] et al.). Throughout the simulations, we point out to MMSE technique as conventional NOMA, to denote that user devices are applying the MMSE technique for predicting the channel state information (CSI) prior to reconstructing the desired signal. In the simulation environment, NOMA parameters are generated on the basis of the LTE standard [46; 47], and channel parameters are created to initially model the Rayleigh fading channels based on the ITU models. In our developed \(Q\)-learning algorithm, at the end of the training episode, or if the terminal state is reached, the updated \(Q(s,a)\) values in the \(Q\)-table will be employed as a practical channel coefficients for the user devices. Different power percentages are initially assigned for every user device according to channel gain and based on the existing distance from the BS. Power factors \(\eta_{n}\), \(\eta_{m}\), and \begin{table}
\begin{tabular}{c c} \hline \hline
**Parameter** & **Value** \\ \hline Simulation Tool & MATLAB \\ \hline Modulation type & QPSK \\ \hline Number of Users & 3, [2; 3; 4; 5; 6; 7; 8; 9; 10] \\ \hline System Bandwidth \(B\) & 1000 kHz \\ \hline Fading channel & (Rayleigh, Rician) \\ \hline Path loss exponent & 3.5 \\ \hline Number of Iterations & \(10^{5}\) \\ \hline Noise PSD \(N_{0}\) & \(-174\) dBm/Hz \\ \hline Learning Rate \(\alpha\) & 0.1 \\ \hline Discount factor \(\gamma\) & 0.9 \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Summary of Simulation Parameters. \(\eta_{f}\) are specified for near, middle, and far users respectively. In a fixed power allocation setup, we designate \(\eta_{f}=0.65\), \(\eta_{m}=0.25\), and \(\eta_{n}=0.1\).