## 5 Controllable Queuing Model
### Reward Function to Consider Three Principles  
In accordance with the previously described principles of reallocation, the reward function received per unit time during the stay in state \(\mathbf{s}\in\mathcal{S}\) is defined by the following formula:  
\[R(\mathbf{s})=-\Bigg{(}w_{1}\cdot\alpha(\mathbf{s})\cdot 1(\mathbf{s}\in \overline{\mathcal{S}}_{\delta}^{1}(m))\cdot\Big{(}1(m<\frac{N}{2},\ \ \mathbf{s}\in\mathcal{S}_{\delta}^{1}(m))+1(m>\frac{N}{2},\ \ \mathbf{s}\in \mathcal{S}_{\delta}^{2}(m))\Big{)}\] \[\qquad\qquad+w_{2}\cdot\beta(\mathbf{s})\cdot 1(\mathbf{s}\in \overline{\mathcal{S}}_{\delta})+w_{3}\cdot\gamma(\mathbf{s})\cdot 1(\mathbf{s}\in \mathcal{S}_{\delta})\Bigg{)}.\]  
Let us describe each component in detail. The principle of maximum matching for equal resource partitioning is determined by the number of users waiting in the buffer due to unequal resource reallocation and is defined as  
\[\alpha(\mathbf{s})= \begin{cases}\frac{N}{2}-m,&m<\frac{N}{2},\ \ n_{1}>m,\ \ n_{1}-m>\frac{N}{2}-m,\\ n_{1}-m,&m<\frac{N}{2},\ \ n_{1}>m,\ \ n_{1}-m\leq\frac{N}{2}-m,\\ \frac{N}{2}-N+m,&m>\frac{N}{2},\ \ n_{2}>N-m,\ \ n_{2}-N+m>\frac{N}{2}-N+m,\\ n_{2}-N+m,&m>\frac{N}{2},\ \ n_{2}>N-m,\ \ n_{2}-N+m\leq\frac{N}{2}-N+m.\end{cases} \]  
The principle of maximum share of signals resulting in resource reallocation is determined by the probability of non-reallocation after a signal is received and is given by the following formula:  
\begin{table}
\begin{tabular}{l l l l} \hline \hline
**Description** & **Condition on s** & **Condition on a** & **State s\({}^{\prime}\)** \\ \hline \(1\)-Slice is occupied with waiting 1-users, and 2-slice is available for all 1-users in 1-buffer & \(n_{1}>\ m,\ n_{2}<N-m,\ \ n_{1}-m\leq N-m-n_{2}\) & \((a,n_{1},n_{2})\) \\ \hline \(1\)-Slice is occupied with waiting 1-users, and 2-slice is not available for all 1-users in 1-buffer & \(n_{1}>\ m,\ n_{2}<N-m,\ \ n_{1}-m>N-m-n_{2}\) & \((a,n_{1},n_{2})\) \\ \hline \(2\)-Slice is occupied with waiting 2-users, 1-slice is available for all 2-users in 2-buffer & \(n_{2}>N-m,\ n_{1}<m,\ \ n_{1}-m\leq n-m-n_{1}\) & \(N-n_{2}\leq a\leq m\) & \((a,n_{1},n_{2})\) \\ \hline \(2\)-Slice is occupied with waiting 2-  
\[\beta(\mathbf{s})= \delta\Big{(}\delta+\lambda_{1}\cdot 1(n_{1}+1\leq R_{1}+m)+ \lambda_{2}\cdot 1(n_{2}+1\leq R_{2}+N-m)\] \[+\frac{m}{N}C\mu_{1}\cdot 1(m>0,\;\;n_{1}>0)+\frac{N-m}{N}C\mu_{2} \cdot 1(N-m>0,\;\;n_{2}>0)\] \[+(n_{1}-m)\varepsilon_{1}\cdot 1(n_{1}>m)+(n_{2}-N+m)\varepsilon_{2} \cdot 1(n_{2}>N-m)\Big{)}^{-1}.\]  
The principle of maximum resource utilization is determined by the number of users waiting in the buffer due to idle resources, expressed as  
\[\gamma(\mathbf{s})= \begin{cases}N-m-n_{2},&\mathbf{s}\in\mathcal{S}_{\delta}^ {12}(m),\\ n_{1}-m,&\mathbf{s}\in\mathcal{S}_{\delta}^{11}(m),\\ m-n_{1},&\mathbf{s}\in\mathcal{S}_{\delta}^{22}(m),\\ n_{2}-N+m,&\mathbf{s}\in\mathcal{S}_{\delta}^{21}(m).\end{cases} \]  
To elucidate this principle with greater clarity, consider the initial equation within system \(\mathcal{S}_{\delta}^{12}(m)\). This equation signifies that the 1-slice possesses a surplus of users in its 1-buffer (where \(n_{1}>m\)), and concurrently, the 2-slice has available resources (\(n_{2}<N-m\)). Additionally, the combined total \(n_{1}+n_{2}>N\), indicating that the quantity of users awaiting the 1-slice surpasses the idle resources of the 2-slice, resulting in an inability to serve all waiting users. Consequently, we allocate the utmost number of idle resources from the 2-slice to attend to the greatest possible number of waiting users. In this scenario, the resource volume equates to the entirety of idle resources from the 2-slice, expressed as \(N-m-n_{2}\) (where \(n_{2}\) represents the count of occupied resources in the 2-slice). Moving on to the second equation, \(\mathcal{S}_{\delta}^{11}(m)\) implies that there are users awaiting service in the 1-buffer for the 1-slice (\(n_{1}>m\)), while there are unutilized resources in the 2-slice (\(n_{2}<N-m\)), and \(n_{1}+n_{2}<=N\).