## 4 Deployment of Cloud-Native Core Network in ComEdge
### DQN-Based CNF Deployment Algorithm (DQN-CNFDA)  
We developed a CNF deployment framework based on DQN to address the intricate relationships among service requests, provisioning costs, and control traffic overhead. This framework is designed to reduce costs by learning spatiotemporal patterns within service requests. Once the deployment of each CNF is completed, achieving user registration services is possible through CNFs interacting within or between nodes following the communication protocol process. To adaptively place CNFs in response to fluctuations in service requests, our approach, DQN-CNFDA, learns the optimal policy by gathering rewards through trial-and-error interactions with the environment [28]. To minimize the operation costs of the cloud-native edge core network, we begin by quantifying the communication between nodes and calculating the associated cost within the core network. We simplify the model by solely considering the communication cost between nodes, disregarding the cost within nodes. The meanings of each notation are explained in Table 2.  
Formula (1) indicates the occurrence of communication between CNF \(f\in F_{s}\) and CNF \(f+1\in F_{s}\) across the edge nodes within the time interval \(t\). \(M^{t}(f,s)\) equals zero if CNF \(f\) and \(f+1\) are provisioned the same edge node; otherwise, \(M^{t}(f,s)\) equals one. \[M^{t}(f,s)=\sum_{p}\sum_{p^{\prime}}\sum_{f\in F_{s}}\sum_{f^{\prime}\in F_{ s}}\sum_{e\in E}\sum_{e^{\prime}\in E}\ \ \ min(1,\left|e-e^{\prime}\right|\cdot x_{p,f,e}^{t} \cdot x_{p^{\prime},f^{\prime},e^{\prime}}^{t}). \tag{1}\] \begin{table}
\begin{tabular}{c c} \hline \hline
**Notation** & **Description** \\ \hline \(E\) & Set of edge nodes \\ e & Edge nodes index \\ \(N_{E}\) & Number of edge nodes \\ \(u_{e}^{t}\) & Number of available resource units in edge node \(e\) at time \(t\) \\ \(S\) & Set of services \\ \(s\) & Service index \\ \(F_{s}\) & Set of CNFs in service \(s\) \\ \(f\) & CNF index in set of CNFs \(F_{s}\) \\ \(p\) & CNF type index \\ \(x_{p,f,e}^{t}\) & CNF \(f\) with type \(p\) is provisioned on edge node \(e\) at time \(t\) or not \\ \(n_{s,f,e}^{t}\) & Number of service \(s\) request for CNF \(f\) to edge \(e\) at time \(t\) \\ \hline \hline \end{tabular}
\end{table} 
Table 2: The meanings of each notation. Figure 5: Three Examples of edge core network deployment. (**a**) A single core network is deployed on a single edge node. (**b**) A single core network is deployed on multiple edge nodes. (**c**) Multiple core networks are deployed on multiple edge nodes. \(D_{CM}\) in (2) represents the cost of individual communication. Meanwhile, \(C_{CM}^{t}\) represents the overall cost of communication between nodes necessary to accomplish the service core network. \[C_{CM}^{t}=\sum_{s\in S}\sum_{f\in F_{s}}M^{t}(f,s)\sum_{e\in E}n_{s,f,e}^{t} \cdot D_{CM}. \tag{2}\]  
\(D_{OP}\) in (3) represents the deployment cost of an individual CNF, and \(C_{OP}^{t}\) represents the deployment cost of the entire edge core network across all the nodes. \[C_{OP}^{t}=\sum_{f\in F_{s}}\sum_{e\in E}x_{p,f,e}^{t}\cdot D_{OP}. \tag{3}\]  
The objective of optimization can be described as  
\[\min\quad C_{OP}^{t}+C_{CM}^{t}. \tag{4}\]  
Resources on edge nodes are limited. CNFs deployed should not exceed the number of available resource units on edge nodes. \[\sum_{p}\sum_{f\in F_{s}}x_{p,f,e}^{t}\leq u_{e}^{t},\quad\forall t,e\in E. \tag{5}\]  
We use the available resource units \(u_{e}^{t}\) of each edge node \(e\) within time \(t\) as the state space \(s_{t}\). \(A\) represents the action space and an action \(a_{t}\) at time \(t\) is one of the combinations of edge clouds, where the number of combinations of edge nodes is \(2^{N_{E}}\). \[s_{t}=\{u_{1}^{t},u_{2}^{t},u_{3}^{t},u_{1}^{t},\ldots,u_{N_{E}}^{t}\}. \tag{6}\]  
Reward \(r_{t}\) at time \(t\) is defined as  
\[r_{t}=-C_{OP}^{t}-C_{CM}^{t}. \tag{7}\]  
Algorithm 1 introduces the CNFs deployment algorithm based on DQN. The weight \(\theta\) is initialized in Line 1.