## 2 System Model  
### Multiuser Environment  
In this work, a multiuser environment with a single Base Station (BS) and multiple user devices (UDs) is considered. The BS is supplied with \(N\) antennas and all the UDs are supplied with a single antenna. The network is assumed to work with equal length time intervals and each time interval includes one transmission, which contains either uplink or downlink transmissions. The pilot-assisted channel prediction is considered in this work, where pilot symbols can be identified by BS and UDs [15; 17]. Each user device initially transmits its pilot symbols to BS via an uplink channel. Then, prior to data transmission, the BS can inspect the pilot symbols and the available network information to facilitate estimating the downlink CSI. The main aim of this work is to model the channel prediction task and to manage the power allocation scheme. We can refer to the matrix of downlink channel coefficients from BS with \(N\) antennas to UD \(i\) as:  
\[\mathbf{H}_{i}=[\mathbf{h}_{1i};\ \mathbf{h}_{2i};\ldots;\mathbf{h}_{Ni}] \tag{1}\]  
where \(\mathbf{h}_{ji}\) represents the vector channel parameters from \(j\)th antenna at BS to the \(i\)th UD, with \(j\in[1,\ 2,\ \ldots,N]\) and \(i\in[1,\ 2,\ \ldots,M]\), where \(N\) is the number of antennas at BS and \(M\) is the number of users in MISO-NOMA cell. Furthermore, we can denote the data signal transmitted to UD \(i\) as  
\[\mathbf{s}_{i}=[s_{i1},\ s_{i2},\ \ldots,s_{iK}] \tag{2}\]  
where \(K\) is the length of the signal. Then, the matrix of all the UD's sequences can be expressed as  
\[\mathbf{S}=[\mathbf{s}_{1};\mathbf{s}_{2};\ldots;\mathbf{s}_{M}] \tag{3}\]  
The received \(k\)th signal at \(j\)th UD can be denoted as:  
\[\mathbf{y}_{kj}=\sum_{i=1}^{N}\mathbf{h}_{ij}s_{ki}+z_{kj} \tag{4}\]  
where \(z_{kj}\) denotes the AWGN with zero mean and variance \(\sigma^{2}\) at \(j\)th UD through \(k\)th signal duration. The received \(k\)th symbol at all UDs is:  
\[\mathbf{Y}_{k}=\sum_{i=1}^{N}\mathbf{h}_{i}s_{ki}+\mathbf{z}_{k} \tag{5}\]  
where  
\[\mathbf{Y}_{k}=[\mathbf{y}_{k1};\mathbf{y}_{k2};\ \ldots\ ;\mathbf{y}_{kM}] \tag{6}\]  
\[\mathbf{z}_{k}=[z_{k1};\ z_{k2};\ \ldots;z_{kM}] \tag{7}\]  
Many of the current works depend on pilot symbols to approximate the uplink channel parameters and then utilize channel reciprocity to realize the prediction of downlink channel weights [15; 18]. These schemes for CSI prediction may not be reliable, especially in cases of inadequate channel reciprocity owing to hardware constraints. Furthermore, this kind of estimator may introduce estimation errors in case the uplink and downlink channel parameters are not stationary within a certain transmission.  
In the developed Q-learning procedure, we plan to get assistance from the pilot symbols, and network information to explicitly predict the downlink channel parameters. The set of estimated channel coefficients among BS and \(M\) UDs can be indicated as  
\[\mathbf{\hat{H}}=\big{[}\mathbf{\hat{H}}_{1};\ \mathbf{\hat{H}}_{2};\ldots;\mathbf{\hat{H}}_{M} \big{]} \tag{8}\]where \(\hat{\mathbf{H}}_{i}\) is the predicted matrix channel coefficients between BS that contains \(N\) antennas and \(i\)th UD, and can be expressed as follows:  
\[\hat{\mathbf{H}}_{i}=[\hat{\mathbf{\emph{h}}}_{1i};\ \hat{\mathbf{\emph{h}}}_{2i}; \ldots;\hat{\mathbf{\emph{h}}}_{Ni}] \tag{9}\]  
where \(\hat{\mathbf{\emph{h}}}_{ji}\) represents the predicted channel parameters between \(j\)th antenna at BS and the \(i\)th UD.