# Joint Pricing, Server Orchestration and Network Slice Deployment in Mobile Edge Computing Networks
The NPL and RAL share the common goal of maximizing the utility of slices, which is correlated with delay and cost. In our scheme, the IPL, NPL and RAL make decisions sequentially, with each layer following the decisions of the previous layer. Their relationship can be modeled as a three-stage Stackelberg game, where players make decisions in a strict order [21]. During the decision-making process, the players who decide first are the leaders, while the players who decide later are the followers. The Stackelberg game provides decision support for both leaders and followers by analyzing their strategies, aiming to find the Stackelberg equilibrium [22]. Backward induction is commonly used to obtain Stackelberg equilibrium solutions [22]. However, the aforementioned three-stage Stackelberg game constitutes a complex combinatorial optimization problem, which is NP-hard, rendering backward induction inapplicable [23]. In contrast, deep reinforcement learning (DRL) works well in solving complex optimization problems. DRL enables self-adaptation and strategy improvement by interacting with the environment. Deep Q-network (DQN) and its variants have been widely applied to DRL problems involving discrete action spaces. Among them, double DQN (DDQN) and dueling DQN (D3 QN) are two significant improvements over the standard DQN. DDQN mitigates the overestimation bias in Q-learning by decoupling action selection and evaluation, while D3 QN introduces separate value and advantage estimations to improve learning stability [24]. Recent advancements in D3 QN-based DRL have demonstrated its effectiveness in wireless communications. For instance, in [25], the authors proposed a D3 QN-based path planning solution for unmanned aerial vehicles, incorporating quantum-inspired experience replay to optimize experience sampling and improve training efficiency. This study highlighted the potential of D3 QN in solving complex wireless network problems. However, D3 QN has limitations in dealing with the problem of exponentially growing discrete action spaces, which exists in our model. In this context, authors in [26] gave a solution with their proposed branching dueling Q-network (BDQ), which is a branching variant of the dueling double deep Q-network (D3 QN). BDQ's neural network builds network branches for each action dimension while retaining a shared module. The shared module encodes the latent representation of the state and provides information for each action branch. The number of the network's outputs in BDQ grows linearly with the increase in the action dimension. Currently, many works have verified the superiority of BDQ in solving large-scale discrete action spaces problems [27; 28; 29]. In BDQ, because each action branch has an independent network, inter-branch coordination is weakened, potentially resulting in joint actions that fail to meet the requirements. When these kind of illegal actions occur frequently, applying a fixed penalty can hinder the agent's exploration and distort the value function estimation. Thus, it is crucial to develop an effective method for avoiding illegal actions to ensure the convergence of BDQ. Based on the above context, the main contributions of this paper are summarized as follows:  
* We design an innovative wireless network model that integrates MEC networks with network slicing, combining MEC servers and base stations with VNF chains of network slices. This design enhances the configurability and scalability of MEC networks. * We divide the system into the InP layer, network planning layer, and resource allocation layer, which are responsible for pricing, server orchestration and slice deployment, and resource allocation, respectively. To the best of our knowledge, this is the first work to comprehensively consider these interconnected challenges. Moreover, we build a three-stage Stackelberg game model according to the relationships among these three layers. Then, we prove the existence of a Stackelberg equilibrium. * We design a three-layer DRL model to obtain the Stackelberg equilibrium.