# Load-Balanced Dynamic SFC Migration Based on Resource Demand Prediction
## 4 Algorithm Design
### PPO-Based SFC Migration Algorithm
#### 4.2.1 MDP Model  
As part of dynamic optimization, we take into account reducing the variance of network resources following VNF migration under time-varying traffic. Since the state space, action space, state transition probability, and reward function are represented by  
Figure 3: An LSTM model with attention.  
the four-element tuple \((S,A,P,R)\), respectively, the optimization aim is changed into a Discrete Time Markov Decision Process (DTMDP).  
The network's state space at time \(t\) is represented by the state space \(S\), \(s(t)=\{\psi(t),\xi(t),C(t),M(t)\}\in S\), where \(\psi(t)\) denotes the state space of the physical node and \(\xi(t)\) denotes the state space of the physical link, respectively, as follows:  
\[\psi(t)=\{\varphi_{1}(t),\varphi_{2}(t),\ldots,\varphi_{n}(t)\}\ \forall n\in N^{p} \tag{28}\]  
\[\xi(t)=\{\mathcal{S}_{1,1}(t),\mathcal{S}_{1,2}(t),\ldots,\mathcal{S}_{n,m}( t)\}\ \forall n,m\in N^{p},\ \forall l_{nm}\in L^{p} \tag{29}\]  
The state space of the CPU and the memory resource requirements of VNF are represented by \(C(t)\) and \(M(t)\), respectively. These demand data reflect the anticipated outcome of VNF's resource needs. The two resource requirements' state spaces can be shown as follows:  
\[C(t)=\left\{C_{1,1}^{v}(t),C_{1,2}^{v}(t),\ldots,C_{i,j}^{v}(t)\right\}\ \forall i\in F,j\in N_{i}^{v} \tag{30}\]  
\[M(t)=\left\{M_{1,1}^{v}(t),M_{1,2}^{v}(t),\ldots,M_{i,j}^{v}(t)\right\}\ \forall i\in F,j\in N_{i}^{v} \tag{31}\]  
The set of mapping activities that VNF can perform is denoted by action space A. The mapping action space of VNF can now be expressed as follows:  
\[a(t)=\left\{a_{1,1}(t),a_{1,2}(t),\ldots,a_{i,j}(t)\right\}\ \forall i\in F,j\in N_{i}^{v},a(t)\in A \tag{32}\]  
where \(a_{i,j}(t)\) is the mapping action that the \(j\)-th VNF on SFC \(i\) may perform. This includes the selection scheme of the physical node mapping variable \(x_{n}^{i,j}\) and the virtual link mapping variable \(y_{n,m}^{j,k}\).  
The transition probability \(P\), which can be expressed as \(p(s(t+1)s(t),a(t))\), is the probability of moving to the next state \(s(t+1)\) following an action \(a(t)\in a\) in the current state \(s(t)\in s\).  
The network creates a mapping strategy \(\pi\) when it performs action \(a(t)\) under state \(s(t)\). The optimization aims states that an instantaneous reward \(r(t)\) can be obtained if the mapping approach \(\pi\) fulfills system restrictions.  
\[r(t)=-L_{\text{total}}(t) \tag{33}\]  
The network uses the value function \(V^{\pi}\) to assess the quality of the current VNF mapping strategy \(\pi\) at this precise instant. \(V^{\pi}\) can be represented as follows:  
\[\begin{array}{rl}V^{\pi}(s(t),a(t))&=E[R(t)s(t),a(t)]\\ &=E[r(t)+\gamma_{v}r(t+1)+\cdots s(t),a(t)]\\ &=E[r(t)+\gamma_{v}V^{\pi}(s(t+1),a(t+1))s(t),a(t)]\end{array} \tag{34}\]  
where the importance of future rewards in relation to present rewards is represented by the discount factor, \(\gamma_{v}\in[0,1]\). The optimal strategy for VNF migration can be expressed as follows:  
\[\pi^{*}=\operatorname*{argmax}_{a}V^{\pi}(s,a)\ \forall s,a \tag{35}\]