# Joint Pricing, Server Orchestration and Network Slice Deployment in Mobile Edge Computing Networks
## 6 Performance Evaluation
### Convergence Performance  
Due to the stochasticity of the agent and the inherent uncertainty of the state, the reward curve exhibits notable fluctuations. To visually demonstrate the convergence process, we partitioned the training procedure into epochs, each consisting of 500 episodes. The learning rate balances exploration and convergence and is essential for achieving the global optimal solution. A high learning rate may cause excessive parameter updates, potentially skipping over global or local optima. Meanwhile, a low learning rate may trap the model in local optima, preventing it from escaping and finding the global optimal solution. Because a lower learning rate results in slower convergence, it is preferable to select the largest learning rate within the optimal range. In Figure 5, we illustrate the convergence performance of the IPL, NPL and RAL under different learning rates. The results show that when the learning rate is set to \(10^{-4}\), it performs best in the IPL, NPL and RAL. Due to the reasons mentioned above, other learning rates fail to achieve better solutions. With \(M=3\) and \(N=2\), we present the convergence process of D3 QN-DMBDQ and compare it with D3 QN without dueling architecture (i.e., DDQN) and DMBDQ without dueling architecture (DMBDQ-WOD) in Figure 6. Figure 6 illustrates the convergence process of the IPL, which is the leader of the three-layer structure. Due to the relatively small action space of the IPL and the fact that its decision-making process is mainly based on the current environmental state, it converges rapidly and tends to stabilize after approximately 100 epochs. The rapid convergence of the IPL provides a stable price basis for the NPL and the RAL, reducing the uncertainty in the initial learning stage of the lower layers. Figure 6b shows the convergence process of the NPL and the RAL, whose overall convergence speed is relatively slow. The reason lies in the large action spaces, which makes the learning and policy optimization processes more complex. In addition, the policies of the NPL and the RAL rely on the pricing information provided by the IPL, and the RAL also depends on the server activation and slice deployment schemes of the NPL. In the early stage of training, the strategy of the IPL has not been completely stable, which increases the policy optimization difficulty of the lower layers. As the training process progresses, the synergy among the three layers promotes the convergence of the entire system. That is, the IPL provides a stable pricing scheme, the NPL finds the optimal server activation and slice deployment schemes and the RAL determines the optimal reso  
Figure 5: Convergence performance of D3 QN-DMBDQ under different learning rates. (**a**) Convergence process of the IPL under different learning rates (\(\text{tr}_{\text{-}}\text{NPL}_{\text{-}}\text{RAL}=10^{-4}\)); (**b**) convergence process of the NPL and the RAL under different learning rates (stochastic pricing scheme). Additionally, it can be seen that D3 QN and DMDDQ achieve higher rewards than DDQN and DMDDQ-WOD. Thus, D3 QN-DMBDQ is closer to the Stackelberg equilibrium. This is because the dueling architecture can mine valuable states and use them to achieve fast convergence [45]. In addition, the dueling architecture splits the state-action value function into a state value function and an action advantage function, which allows the network to better capture the relationship between states and actions and learn better strategies. To demonstrate the superiority of the proposed DMDDQ in dealing with large-scale action spaces, we train the DMDDQ individually by setting the prices in the IPL to random values. The convergence process of DMDDQ under different \(M\) and \(N\) are shown in Figure 7 and Figure 8, respectively. We compare DMDDQ with (1) DBDQ without modification of the illegal actions, giving a penalty for the illegal actions (DBDQ-WOM); and (2) double-layer D3 QN (DD3 QN).