applied sciences  
A Meta Reinforcement Learning Approach for SFC Placement in Dynamic IoT-MEC Networks  
[PERSON] 1,2, [PERSON] 3,* and [PERSON] 4,*  
Footnote 1: [[https://www.mdpi.com/journal/applsci](https://www.mdpi.com/journal/applsci)]([https://www.mdpi.com/journal/applsci](https://www.mdpi.com/journal/applsci))  
Footnote 2: [[https://www.mdpi.com/journals/applsci](https://www.mdpi.com/journals/applsci)]([https://www.mdpi.com/journals/applsci](https://www.mdpi.com/journals/applsci))  
Footnote 3: [[https://www.mdpi.com/journals/applsci](https://www.mdpi.com/journals/applsci)]([https://www.mdpi.com/journals/applsci](https://www.mdpi.com/journals/applsci))  
Footnote 4: [[https://www.mdpi.com/journals/applsci](https://www.mdpi.com/journals/applsci)]([https://www.mdpi.com/journals/applsci](https://www.mdpi.com/journals/applsci))  
Footnote 5: [[https://www.mdpi.com/journals/ap](https://www.mdpi.com/journals/ap)]([https://www.mdpi.com/journals/ap](https://www.mdpi.com/journals/ap))  
Nevertheless, lower latency and increased computing resources are typically demanded by these IoT services. By leveraging multiaccess edge computing (MEC) technology, placing virtual network functions (VNFs) in cloudlets not only minimizes the end-to-end latency of IoT services but also supplies computational resources for IoT equipment [5; 6]. The traffic requests that are produced by IoT equipment also usually need to sequentially pass through various VNF instances [7]. We refer these requests as IoT SFC requests (IoT-SRs). To address the resource limitations in cloudlets, it is crucial for the SDN controller to devise optimal policies for the dynamic placement of VNFs and the orderly routing of traffic associated with IoT-SRs. This problem is generally called the dynamic SFC placement problem. Existing works mainly focus on the placement of VNFs [8; 9] or investigate the traffic routing problem [10; 11] in NFV-enabled SDNs. Few literature reports solve the placement of VNFs and routing paths in a dynamic network environment such as IoT networks considering the following factors: (1) the dynamic nature of network scenarios and the random arrival of traffic; (2) the presence of multiple instances of each type of VNF distributed across diverse cloudlets, allowing for on-demand and dynamic placement of VNF instances; (3) meeting the quality of service (QoS) requirements of IoT services, including factors such as latency, packet loss rate, and load balancing; (4) striking a balance between cost and end-to-end delay. Furthermore, timely and efficient resolution of the dynamic SFC placement problem remains challenging using common optimization approaches (for example, exact [12] or heuristic [13] approaches, among others). In particular, large-scale networks often present computational challenges when utilizing exact methods. As heuristics are prone to becoming trapped in local minima when the networks are characterized by dynamic and rapid change, this method is not suitable for the VNF placement problem in a dynamic IoT network. Given its proficiency in decision-related issues and adaptability to dynamic network environments, deep reinforcement learning (DRL) has emerged as an intelligent method employed to address the VNF placement and traffic routing problems [14]. However, in dynamical IoT-MEC scenarios, these methods exhibit limitations in their ability to adapt and tend to optimize the placement of one VNF at a time, making them susceptible to local optima. When the IoT-MEC network scenario changes, the original network parameters fail, and the neural network needs to be retrained, which results in poor adaption and low learning efficiency. In our work, the network state transitions are captured using a Markov decision process (MDP). Due to the ability of meta reinforcement learning to learn a set of universal control strategies, agents can learn how to quickly adapt to new tasks without the need for retraining [15; 16; 17].