# Joint Pricing, Server Orchestration and Network Slice Deployment in Mobile Edge Computing Networks
## 2 Related Work
_(3) Pricing and Stackelberg Game:_ For the problem of MEC server orchestration and network slice deployment, many studies attempted to optimize different objectives, such as delay, deployment cost and number of servers. However, only a few studies have considered the benefit of an InP. For the InP, revenue and energy consumption are more of a concern. For example, ref. [31] optimized resource allocation in a multi-tenant scenario to maximize the InP's revenue and provide satisfactory services. However, just like [31], many works that considered the InP's revenue ignored the impact of pricing. Recently, the Stackelberg game has been widely used to seek the equilibrium between pricing strategies and resource allocation strategies. To illustrate, in [22], the interaction between access service provider (ASP) and users is formulated as a multi-leader, multi-follower Stackelberg game. As the leader, the ASP decides the access price based on users' expected budgets and the pricing strategies of other ASPs. As the follower, users decide the access bandwidth to be purchased based on the prices. Inspired by this, in this paper, we design a three-layer Stackelberg game with the InP as the leader and the NPL as the follower and also with the NPL as the leader and the RAL as the follower. _(4) Reinforcement Learning Approach:_ Recently, multi-layer reinforcement learning has made significant progress in solving the Stackelberg game problem, which is NP-hard. In [32], the pricing decisions of MEC servers and offloading decisions of users were modeled as a multi-leader multi-follower Stackelberg game. To enhance the applicability of the pricing decisions, this work proposed using a multi-armed bandit (MAB) in the leader layer. However, the MAB solves the problem on a stateless premise, which does not apply to situations where pricing needs to be dynamically adjusted according to the state of the network. Similar to our scheme, in [33; 34], a three-layer DRL algorithm was employed to solve the three-layer Stackelberg games. These works have proven that the three-layer DRL was able to reach an equilibrium solution. Although multi-layer DRL has proven to be an effective approach for solving multi-layer Stackelberg games, the current research has yet to explore its applicability in handling multi-layer Stackelberg games with large-scale discrete action spaces. The convergence difficulty of multi-layer DRL is inherently higher than that of single-layer DRL, and this challenge is further exacerbated in scenarios with large-scale action spaces, imposing greater demands on model training efficiency. Therefore, exploring methods to reduce the convergence difficulty of multi-layer DRL is critical. Ref. [26] proposed the BDQ algorithm to enhance the applicability of DRL in large-scale discrete action spaces. Then, ref. [27] applied BDQ to the vehicle association problem and validated its effectiveness. However, existing BDQ-related works have a major limitation: they overlook the fact that the branching structure may weaken the correlation between sub-actions, leading to joint actions that fail to meet constraints, thereby compromising the effectiveness of the strategy. Building on the above context, we propose a three-layer DRL algorithm based on D3 QN-DBDQ to solve the three-stage Stackelberg game. At the same time, we innovatively design an illegal action modification algorithm to enhance the convergence of BDQ. To clearly illustrate how our work differs from related works, we summarize the aspects covered in related works in Table 1.