## 3 System Model
### Preliminary Considerations  
Our system model is predicated on a set of simplifying assumptions. We focus on a scenario involving just two network slice service providers, between whom resources are dynamically reallocated. The arrival patterns of both controller signals and user requests are inherently stochastic. Resource reallocation is contingent upon the receipt of a signal from the controller. Within each slice, we posit an egalitarian distribution of resources among all users, irrespective of their remaining service duration, adhering to what is known as the Egalitarian Process Sharing discipline in queueing theory. All system events are assumed to follow an exponential distribution. For our numerical analysis, we chose to examine two distinct services: web browsing and bulk data transfer.  
The controller is the pivotal element in our network slicing model, dictating when resource reallocation should occur. The output of the controller's operation is the allocation of resources to the first slice--knowing that this allows us to deduce the resource allocation for the second slice, given that the total resource pool remains constant. This allocation is part of what we refer to as the control strategy. The controller's signal rate is a critical factor influencing system performance and resource reallocation; hence, it is incorporated into the reward function discussed in Section 5.4.  
Our system's self-optimization mechanisms operate by adjusting weights across three key components that collectively enhance system efficiency. This self-optimization entails  
\begin{table}
\begin{tabular}{p{14.2 pt} p{142.3 pt} p{142.3 pt} p{142.3 pt}} \hline \hline
**Ref.** & **Approaches Involving MDP** & **Reward Function** & **Action** \\ \hline \multirow{4}{*}{[45]} & Exponential weight algorithm for exploration and exploitation, multi-agent DQN & Two reward functions: (i) total achievable bit rate of URLLC and eMBB users; (ii) whether or not the agent has successfully allocated the needed resource blocks to its associated user & Every gNodeB schedules pre-allocated resource blocks assigned by the SDN controller to its associated end-users \\ \hline \multirow{4}{*}{[46]} & DQN, RL & Maximization of slice tenant’s profit & Tenants autonomously control resource allocation and engage in negotiations with the provider to effectively manage and optimize resource utilization \\ \hline \multirow{4}{*}{[47]} & Dynamic mixed-integer linear programming, DRL & Minimization of the total cost for VNF orchestration, backup, and mapping with delay constraints, reliability requirements, and limited resources & The resource management agent selects a random action with a certain probability for exploration and selects the best action that follows the greedy policy \\ \hline \hline \end{tabular}
\end{table}
Table 2: _Cont._fine-tuning policy computation based on the reward function's configuration, thereby directly influencing the selection and nature of the optimal policy. During the optimization process, we determine new resource allocation operations in response to the reward function, ensuring that the optimal policy is shaped by this function. In scenarios where only a baseline algorithm exists, we establish the policy analytically. Policies influenced by the reward function are presented in tabular form. We identify three primary risks: underutilization of resources, non-impactful controller signals, and deviations from QoS allocation. These risks are integrated into the reward function, and by modulating their respective weights, we can prioritize certain risks over others according to their significance within our framework.