## 3 Proposed Work
### Deep Contextual Bandit Network Slicing Scheme
This value also helps determine the remaining computing capacity for the network's physical infrastructure at the specified time step. \[C_{\delta}=\frac{\sum\limits_{i=1}^{I}C_{NSR_{i}}}{\sum\limits_{n=1}^{N}C_{n}}\] (13) where \(\sum\limits_{n=1}^{N}C_{n}\) refers to the total remaining computing capacity for all physical infrastructure nodes. 3. The bandwidth utilization \(B_{\delta}\) reflects the network's total bandwidth utilized at time \(t\). This bandwidth utilization is also affected by other NSRs served at the specified time step. \[B_{\delta}=\frac{\sum\limits_{i=1}^{I}B_{NSR_{i}}}{\sum\limits_{i=1}^{L}B_{i}}\] (14) where \(\sum\limits_{i=1}^{I}B_{NSR_{i}}\) is the allocated link capacity for \(NSR_{i}\) and \(\sum\limits_{i=1}^{L}B_{i}\) is the total remaining link capacity for all nodes from the physical infrastructure. 4. Node utilization \(N_{\rho}\) represents the percentage of network nodes serving all existing NSRs at time \(t\). It also verifies whether the network can allocate the nodes needed by the NSR. \[N_{\rho}=\frac{\sum\limits_{i=1}^{I}N_{NSR_{i}}}{\sum\limits_{n=1}^{N}N_{n}}\] (15)where \(\sum\limits_{i=1}^{I}N_{NSR_{i}}\) is the number of allocated nodes for \(NSR_{i}\) at time \(t\) and \(\sum\limits_{n=1}^{N}N_{n}\) is the total remaining number of physical infrastructure nodes. Using RIFs as weights whose values are determined by the network state, the reward \(R_{i}\) for action \(a_{i}\) is calculated as  
\[R_{i}=\sigma(a)\cdot\left(\frac{(w_{1})L_{hops}+(w_{2})C_{\delta}+(w_{3})B_{ \delta}+(w_{4})N_{\rho}}{\left(\sum_{n=1}^{N}C_{n}+\sum_{i=1}^{L}B_{i}+\sum_{n =1}^{N}N_{n}\right)*L_{hops}}\right) \tag{16}\]  
where \(w\in[0,1]\) is a weight factor defined by the network's current state. We have defined the following network states \(s_{t}\) to find for reward \(R_{i}\) as follows:  
1. **Network State 1 (\(S_{1}\))** represents a normal network state, meaning that the total available computing capacity \(\sum\limits_{n=1}^{N}C_{n}\), link capacity \(\sum\limits_{i=1}^{L}B_{i}\), the number of usable nodes \(\sum\limits_{n=1}^{N}N_{n}\), and total path length \(L_{hops}\) are at 81-100%. 2. **Network State 2 (\(S_{2}\))** indicates that \(\sum\limits_{n=1}^{N}C_{n}\), \(\sum\limits_{i=1}^{L}B_{i}\), \(\sum\limits_{n=1}^{N}N_{n}\), and \(L_{hops}\) are at 50-80% capacity or availability. Such a state requires that succeeding NSRs should not exceed the remaining capacity of available network resources. 3. **Network State 3 (\(S_{3}\))** signals that all network resources are below 50% capacity or availability. This state indicates either that the network is currently serving many NSRs or the NSRs currently served are utilizing a large amount of resource capacity. Based on these network states, a combination of values for each \(w\) of a RIF is assigned in Table 2.  
The weight assignments in Table 2 show that \(s_{3}\) requires a careful allocation of resources for an NSR to avoid network failure. State \(s_{2}\), on the other hand, prioritizes resource allocation for computing capacity and bandwidth to incoming NSRs. Finally, \(s_{1}\) has a lower priority on resource allocation since the network has high resource availability. Next, the accumulated reward for a specific action \(a\) at time \(t\) is denoted as \(Q_{t}(a)\) and then calculated after exploring all possible actions from the action space as,  
Figure 2:0  
where \(y\) is the number of instances performed by the agent for action \(a\), and \(R_{i}\) is the reward earned for each action. The Upper Confidence Bound (UCB) strategy is utilized to facilitate the exploration and exploitation of actions. Through this, the agent decides whether to continue choosing an action that gives a particular reward value (exploitation) or to check for other possible actions that could yield higher rewards (exploration). The UCB for action \(a_{i}\) is expressed as follows,  
Figure 2:1