# Neighbor-Aware Non-Orthogonal Multiple Access Scheme for Energy Harvesting Internet of Things
## 4 Constrained Stochastic Game
### Transition Probability  
\(P[S_{i}^{\prime}|S_{i},A_{i}]\) denotes the probability of transiting a specific state \(S_{i}\) to another state \(S_{i}^{\prime}\) when IoT device \(i\) conducts the action \(A_{i}\).  
Even though IoT device \(i\) has an energy harvesting capability, it cannot always harvest energy because of the dynamicity of the energy source (e.g., wind and solar). That is, only when the energy source provides sufficient energy can IoT device \(i\) harvest energy. In this context, the harvesting process of IoT device \(i\) can be assumed as a Bernoulli random process that takes binary values (i.e., 0 and 1). That is, IoT device \(i\) can harvest a unit of energy with the probability \(P_{i}^{H}\)[26]. Naturally, it cannot harvest any energy with the probability \(1-P_{i}^{H}\). Thus, when IoT device \(i\) does not conduct a data transmission (i.e., \(A_{i}=0\)) and has room in its battery to store the harvested energy (i.e., \(S_{i}\
eq E_{\max}\)), its energy \(S_{i}\) can increase by a unit with probability \(P_{i}^{H}\). On the other hand, IoT device \(i\) does not have any room in its battery (i.e., \(S_{i}=E_{\max}\)), its energy level does not change. To summarize, \(P[S_{i}^{\prime}|S_{i}\
eq E_{\max},A_{i}=0]\) and \(P[S_{i}^{\prime}|S_{i}=E_{\max},A_{i}=0]\) can be represented as  
\begin{table}
\begin{tabular}{p{142.3 pt} p{142.3 pt}} \hline \hline
**Notation** & **Description** \\ \hline \(\mathbf{S_{i}}\) & Local state space of IoT device \(i\) \\ \hline \(\mathbf{S_{-i}}\) & State space of all IoT devices except IoT device \(i\) \\ \hline \(\mathbf{S}\) & Global state space \\ \hline \(E_{\max}\) & Maximum energy level of IoT device \(i\) \\ \hline \(\mathbf{A_{i}}\) & Local action space of IoT device \(i\) \\ \hline \(\mathbf{A}\) & Global action space \\ \hline \(A_{\max}\) & Maximum transmission power of IoT device \(i\) \\ \hline \(P_{i}^{H}\) & Probability that IoT device \(i\) harvests one-unit of energy \\ \hline \(\gamma_{i}\) & Expected SINR of IoT device \(i\) \\ \hline \(\overline{A_{i}}\) & Expected power of IoT device \(i\) \\ \hline \(|h_{i}|^{2}\) & Channel gain of IoT device \(i\) \\ \hline \(\sigma^{2}\) & Noise power \\ \hline \(\xi_{C}\) & Average aggregated data rate of IoT devices \\ \hline \(\xi_{E}\) & Average energy outage probability \\ \hline \hline \end{tabular}
\end{table}
Table 1: Summary of notations.  
\[P[S^{\prime}_{i}|S_{i}\
eq E_{\max},A_{i}=0]=\left\{\begin{array}{ll}P_{i}^{H}& \text{if }S^{\prime}_{i}=S_{i}+1\\ 1-P_{i}^{H}&\text{if }S^{\prime}_{i}=S_{i}\\ 0,&\text{otherwise}\end{array}\right. \tag{5}\]  
and  
\[P[S^{\prime}_{i}|S_{i}=E_{\max},A_{i}=0]=\left\{\begin{array}{ll}1,&\text{if }S^{ \prime}_{i}=S_{i}\\ 0,&\text{otherwise}.\end{array}\right. \tag{6}\]  
When IoT device \(i\) transmits the data with power \(A_{i}\) during one time epoch (i.e., a unit time), it is assumed that it consumes \(A_{i}\) units of energy. When there is no sufficient energy in the battery of IoT device \(i\) (i.e., \(S_{i}<A_{i}\)), it does not transmit the data, which indicates that the energy of IoT device \(i\) does not decrease and can increase by a unit with probability \(P_{i}^{H}\). Thus, \(P[S^{\prime}_{i}|S_{i}\geq A_{i},A_{i}\
eq 0]\) and \(P[S^{\prime}_{i}|S_{i}<A_{i},A_{i}\
eq 0]\) can be represented as  
\[P[S^{\prime}_{i}|S_{i}\geq A_{i},A_{i}\
eq 0]=\left\{\begin{array}{ll}P_{i}^{H}& \text{if }S^{\prime}_{i}=S_{i}+1-A_{i}\\ 1-P_{i}^{H}&\text{if }S^{\prime}_{i}=S_{i}-A_{i}\\ 0,&\text{otherwise}\end{array}\right. \tag{7}\]  
and  
\[P[S^{\prime}_{i}|S_{i}<A_{i},A_{i}\
eq 0]=\left\{\begin{array}{ll}P_{i}^{H}& \text{if }S^{\prime}_{i}=S_{i}+1\\ 1-P_{i}^{H}&\text{if }S^{\prime}_{i}=S_{i}\\ 0,&\text{otherwise}.\end{array}\right. \tag{8}\]