## 4 Simulation Results  
In this section, we evaluate the performance of the EMPPO algorithm within a simulated low-latency MEC environment, encompassing diverse scenarios such as random task arrivals, varying data packet sizes, and dynamic channel conditions. This assessment aims to show the adaptability of EMPPO in real-time offloading decisions against the baseline strategies. We developed our low-latency MEC network simulation environment in Python 3.9 and refined the algorithms from the widely recognized open-source Stable Baselines3 reinforcement learning library [38] to formulate our EMPPO approach. The simulation experiments were conducted on a setup running Windows 11, comprising an AMD Ryzen 7 5800 CPU paired with 32 GB of system memory and an NVIDIA GeForce RTX 3070 Ti graphics card. In particular, we consider a low-latency MEC network composed of three UEs and one edge server. The setting of specific parameters is based on the 3 GPP protocol as much as possible to be close to the real scenario. The error probability constraint for packet transmission \(\epsilon_{max}=0.001\). Tasks arrive randomly in each frame with a probability \(p=0.4\), and the size of the data packet \(m_{k,i}\) arriving at each UE typically varies between 500 and 2500 bits. Each UE transmits data packets to the edge server through independent wireless channels. The average SNR of three channels may vary and are set as \([\overline{\gamma}-\Delta_{\overline{\gamma}},\overline{\gamma},\overline{ \gamma}+\Delta_{\overline{\gamma}}]\), where \(\overline{\gamma}=10\) dB, \(\Delta_{\overline{\gamma}}=3\) dB. The channel correlation coefficient \(\rho\) is set as 0.8. The length of each frame is 1 ms, comprising 1500 symbols, with 1000 symbols for transmission and 500 symbols for computation, and one symbol's duration is 66.7 \(\upmu\)s.  
In our proposed EMPPO framework, there are two networks: an actor network and a critic network. Both have one input layer, two hidden layers, and one output layer. The hidden layers are made up of 64 neurons. The other parameter settings of our algorithm are shown in Table 1.  
To evaluate the proposed algorithm, we compare our algorithm with the following baselines. 1. Random Offloading (RO): Randomly select some UEs for data packet offloading. 2. Average Offloading (AO): Select UEs for data packet offloading according to a predefined sequence. 3. Proximal Policy Optimization (PPO): The standard PPO algorithm, which has the same parameters as EMPPO. 4. Verified Random Offloading (VRO): Randomly select some UEs to offload data packets and verify that all selected packets within a frame can meet the error probability constraints. 5. Freshness Greedy Offloading (FGO): Always select the UE with the oldest packet for transmission, until the remaining blocklength makes the selected packet unable to meet the error probability constraints. 6. Channel Greedy Offloading (CGO): Always select the UE with best channel state for transmission, until the remaining blocklength makes the selected packet unable to meet the error probability constraints. To minimize the impact of randomness, our proposed method and all baselines will be tested 500 times, and the average result of these tests will be taken as the output. Figure 5 shows the convergence performance of the proposed EMPPO and the standard PPO. EMPPO starts its learning process with an average reward of around 50, higher than PPO's starting point of around 38. This is because EMPPO masks actions that can not meet the error probability constraints. Therefore, even when the action selection tends to be random at the beginning of the iteration, EMPPO can still achieve a better reward. As the iterations progress, both EMPPO and PPO quickly improve their rewards in the first 500 episodes. After 500 episodes, the reward of EMPPO gradually converges to around 87, while the reward of PPO converges to about 45. This shows that EMPPO has a similar iteration efficacy to the standard PPO and can obtain a higher reward.