## 4 System Model
\begin{table}
\begin{tabular}{c l} \hline
**Symbol** & **Description** \\ \hline \(RU_{n}\) & Relay user equipment \\ W & System Bandwidth \\ K & Sub-channels’ number \\ \(G_{n}\), \(G_{RUm}\) & Channel gain for \(nth\) D2D pair, \(R_{n}\) to \(mth\) CU, respectively \\ \(G_{nm}\), \(G_{nm}\) & Channel gain across \(nth\) D2D pair and \(mth\) CU, \(mth\) CU \\  & and \(nth\) D2D pair, respectively \\ \(P_{n}\), \(P_{C}\), \(P_{Z}\) & Power transmission of the D2D pair, CU, and gNB, respectively \\ \(G_{TR}\), \(G_{TRU}\), \(G_{TZ}\) & Channel gain between \(T_{n}\) to \(R_{n}\), \(RU_{n}\), and gNB, respectively \\ \(G_{RUR}\), \(G_{ZR}\) & Channel gain across \(RU_{n}\), and gNB to \(R_{n}\) \\ \(\sigma\) & Noise power \\ \(\mathrm{Y}_{n,k}\), \(\mathrm{Y}_{m,k}\) & Received SINR for D2D pair \(n\) and cellular user \(m\) utilizing sub-sub-channel \(k\), respectively \\ \(\mathcal{J}_{D}\), \(\mathcal{J}_{R}\), \(\mathcal{J}_{L}\) & Interference impact of CUs, other D2D transmitters, and D2D relays on the \(nth\) DM, RM, and LM, respectively \\ \(G_{T^{\prime}R}\), \(G_{T^{\prime}RU}\), \(G_{T^{\prime}Z}\) & Channel gain between other D2D transmitters to \(R_{n}\), \(RU_{n}\), and gNB, respectively \\ \(G_{RUR}\), \(G_{RU^{\prime}RU^{\prime}}\), \(G_{RUIZ}\) & Channel gain across other D2D relays to \(R_{n}\), \(RU_{n}\), and gNB, respectively \\ \(\Pi_{n,k}^{1}\), \(\Pi_{n,k}^{2}\), \(\Pi_{n,k}^{3}\) & Binary mode sub-channel assignment indicator of DM, RM, and LM respectively \\ \(R_{n,k}\), \(R_{m,k}\) & Data rate for \(nth\) D2D pair and \(mth\) CU, respectively \\ \(\mathcal{M}_{D}\), \(\mathcal{M}_{R}\), \(\mathcal{M}_{L}\) & Mode selection sets of DM, RM, and LM respectively \\ \(\mathcal{E}^{D}\), \(\mathcal{E}^{R}\), \(\mathcal{G}^{L}\) & DM, RM, and LM, each has respective quotas \\ \(\ell^{p}\), \(\ell^{s}\) & Primary and secondary preference lists, respectively \\ \(\mathcal{A}_{t}\), \(\mathcal{A}_{t}\), \(\mathcal{B}_{t}\) & State, action, and reword of the SARSA algorithm \\ \(\mathcal{A}_{t+1}\), \(\mathcal{A}_{t+1}\) & Next state and next action of the SARSA algorithm \\ \(\alpha\) & The learning rate \\ \(\mathcal{Y}\) & The discount factor \\ \(\mathcal{Y}^{*}(\beta)\) & The value function \\ \hline \end{tabular}
\end{table} 
Table 1: _Cont._where \(\mathcal{J}_{D}\), \(\mathcal{J}_{R}\), and \(\mathcal{J}_{L}\) demonstrate the impact of the interferences on the \(nth\) D2D pair in DM, RM, and LM caused by gNB, other D2D \(T_{n}\), and other \(RU_{n}\). The mathematical representation of \(\mathcal{J}_{D}\), \(\mathcal{J}_{R}\), and \(\mathcal{J}_{L}\) can be shown as follows:  
\[\mathcal{J}_{D}=P_{Z}G_{ZR}(t)\Pi_{n,k}^{3}+\sum\limits_{n=1}^{N}P_{n}G_{T^{ \prime}R}\Pi_{n,k}^{1}+\sum\limits_{n=1}^{N}P_{n}G_{RUR}(t)\Pi_{n,k}^{2}, \tag{3}\]  
\[\mathcal{J}_{R}=\left(P_{Z}G_{ZRU}(t)+P_{Z}G_{ZR}(t)\right)\Pi_{n,k}^{3}+\sum \limits_{n=1}^{N}\left(P_{n}G_{T^{\prime}RU}(t)+P_{n}G_{T^{\prime}R}(t)\right) \Pi_{n,k}^{1} \tag{4}\]  
\[+\sum\limits_{n=1}^{N}\left(P_{n}G_{RU^{\prime}RU}(t)+P_{n}G_{RU^{\prime}R}(t )\right)\Pi_{n,k}^{2},\]  
\[\mathcal{J}_{L}=\sum\limits_{n=1}^{N}\left(P_{n}G_{T^{\prime}Z}(t)+P_{n}G_{T^{ \prime}R}(t)\right)\Pi_{n,k}^{1}+\sum\limits_{n=1}^{N}\left(P_{n}G_{RU^{ \prime}Z}(t)\!+\!P_{n}G_{RU^{\prime}R}(t)\right)\Pi_{n,k}^{2}, \tag{5}\]  
where \(G_{T^{\prime}R}\), \(G_{T^{\prime}RU}\), and \(G_{T^{\prime}Z}\) are the channel gain between other D2D transmitters to \(R_{n}\), \(RU_{n}\), and gNB, respectively. Moreover, \(G_{RUR}\), \(G_{RU^{\prime}RU}\), and \(G_{RUIZ}\) represent the channel gain between other D2D relays to \(R_{n}\), \(RU_{n}\), and gNB, respectively.