## 4 Dynamic Placement of SFC Based on Meta Reinforcement Learning and Fuzzy Logic
### Inner Model
10: The input \(s_{p}\) of the SPFN is formed by obtaining the ingress and egress nodes \((e,e+1)\), where \(e\) ranges from \(1\) to \(|F_{i}|+1\), along with the updated network state. 11:for\(e=1\) to \(|F_{i}|+1\)do
12: Evaluate links by fuzzy logic and choose the top \(k\) links with better performance. 13:\(y_{p}\leftarrow\) Place \(s_{p}\) into \(DDQN_{2}\)\(\theta_{p}\) for getting the output. 14: Record \(y_{p}\) into \(\Omega_{i}\). 15: The new network status \(s^{\prime}_{p}\) is observed by the agent. 16: Set \(s_{p}=s^{\prime}_{p}\). 17:endfor
18:endwhile
```  
**Algorithm 2** Meta adaption procedure  
After training the neural network, we can use the trained model to realize dynamic SFC placement in IoT-MEC networks, which can be broken down into three stages: (1) Acquire the current network state \(S_{s}\) using the SDN controller and send it to the agent; then, use \(DDQN_{1}\) to select the optimal combination of VNF instances, i.e., determine the cloudlets for IoT-SRs to traverse. Then, dynamically place VNF instances according to the action with the highest reward. (2) Conduct a preliminary evaluation of each link by considering available bandwidth, delay, and packet loss rate using fuzzy logic. (3) Choose the top \(k\) links in terms of performance and use \(DDQN_{2}\) to further choose the routing action with the highest reward based on the output of \(DDQN_{1}\) and the current state (\(S_{p}\)). The pseudocode of the online running algorithm is depicted in Algorithm 3. ```
0: VNF placing network: \(\theta_{s}\); The network for searching SFC path: \(\theta_{p}\); Remaining resource ratios: \(\left\{\omega_{1}^{cpu},\omega_{2}^{cpu},\ldots,\omega_{|C|}^{cpu}\right\}\) and \(\left\{\omega_{1}^{bw},\omega_{2}^{bw},\ldots,\omega_{|L|}^{bw}\right\}\); IoT-SRs set \(R\). 0: The routing path of IoT-SR which is denoted as \(\Omega\). 1:while\(R\
eq\emptyset\)do
2: Choose an IoT-SR\({}_{i}\) from \(R\) which is represented as \(\left(S_{i},D_{i},F_{i},R_{i,p}^{cpu},R_{i}^{bw},R_{i}^{delay}\right)\). 3: Initialize routing path \(\Omega_{i}=\emptyset\). 4: The SDN controller is interacted with by the agent to obtain the network status and information, serving for the input \(s_{s}\) for the VNF placing network. 5: Get the trained \(DDQN_{1}\) model based on Algorithm 2. 6: Get the trained \(DDQN_{1}\) model based on Algorithm 2. 7: The output is obtained by applying \(s_{s}\) to \(DDQN_{1}\)\(\theta_{s}\), and it is assigned to \(y_{s}\). 8: The action \(a_{s}\) which has the highest Q value among \(y_{s}\) is selected. 9: By executing action \(a_{s}\), the VNF instances are dynamically placed according to its specifications. This action yields a reward \(r_{s}\) and leads to the acquisition of the subsequent status \(s^{\prime}_{s}\). The obtained sample \((s_{s},a_{s},r_{s},s^{\prime}_{s})\) is then stored within the experience replay buffer \(E_{s}\). 10: Get the trained \(DDQN_{2}\) model based on Algorithm 2. 11: The input \(s_{p}\) of the SPFN is formed by obtaining the ingress and egress nodes \((e,e+1)\), where \(e\) ranges from \(1\) to \(|F_{i}|+1\), along with the updated network state. 12:for\(e=1\) to \(|F_{i}|+1\)do
13: Evaluate links by fuzzy logic and choose the top \(k\) links with better performance. 14:\(y_{p}\leftarrow\) Place \(s_{p}\) into \(DDQN_{2}\)\(\theta_{p}\) for getting the output. 15: Record \(y_{p}\) into \(\Omega_{i}\). 16: The new network status \(s^{\prime}_{p}\) is observed by the agent. 17: Set \(s_{p}=s^{\prime}_{p}\). 18:endfor
19:endwhile
```  
**Algorithm 3** Running algorithm of MRLF-SFCP