## 4 Simulation Results
Figure 6 shows the influence of the task arrival probability. It can be seen that EMPPO has the lowest average task AuD across all task arrival probabilities, followed by the greedy methods FGO and CGO, followed by PPO, with RO constantly being the worst. As the task arrival probability increases, the average task AuD for all methods increases. This is due to an increased task arrival probability, potentially causing data packets to wait in the queue longer before being transmitted, and tasks have to wait longer for all the data packets to arrive. In this case, the average task AuD for CGO exceeds FGO's, which means freshness-based packet transmission (i.e., FGO) is more efficient than channel-based transmission (i.e., CGO). This is because CGO tends to send all the packets from channels with good channel quality first, making it harder to transmit packets from channels with poor channel quality. There is a certain gap between the performance of PPO and EMPPO, and the gap gradually increases with the increase in task arrival probability. This indicates that EMPPO can deal with a large number of data packets more effectively than PPO, \begin{table}
\begin{tabular}{l l l} \hline \hline
**Description** & **Symbol** & **Value** \\ \hline Number of iterations & \(N\) & \(1\times 10^{6}\) \\ Number of episodes & \(n_{e}\) & 5000 \\ Learning Rate & \(\alpha\) & 0.0003 \\ Number of Steps & \(n_{steps}\) & 2000 \\ Batch Size & \(B\) & 100 \\ Discount Factor & \(\gamma\) & 0.99 \\ Clip Range & \(\rho\) & 0.2 \\ Exploration Rate & \(\epsilon\) & 1 \\ Exploration Decay Rate & \(\delta\) & 0.995 \\ GAE Factor & \({}_{\lambda}\)\({}^{GAE}\) & 0.95 \\ \hline \hline \end{tabular}
\end{table} 
Table 1: EMPPO algorithm parameters. which may be because EMPPO reduces the exploration of invalid action space through variable masks. At low task arrival probability, EMPPO's performance is comparable to that of FGO and CGO, and the average task AuD initially decreases and then increases as the probability rises. This can be attributed to the less frequent arrival of new tasks and, as a result, a decelerated refresh rate of the task AuD. At high task arrival probability, EMPPO significantly reduces the average task AuD more than other methods. This suggests that EMPPO can successfully balance packet freshness and channel conditions, performing better under high-load scenarios. Figure 7 shows the influence of the channel correlation coefficient. EMPPO consistently achieves the lowest average task AuD through various channel correlation coefficients. The average task AuD time for all methods exhibits a slow increase following a rise in the channel correlation coefficient. Among these, the greedy methods FGO and CGO show a greater sensitivity to changes in the channel correlation coefficient. A significant increase  
Figure 5: The convergence of reward. Figure 6: The average task AuD under different task arrival probability. in average task arrival time is observed under the extreme channel correlation coefficient \(\rho=0.99\). This is because the channel conditions change slowly at this channel correlation coefficient, making it difficult to take advantage of channel variations and transmit data packets with shorter blocklength during optimal channel conditions. Figure 8 shows the influence of the average SNR difference between channels. As the difference increases, the average task AuD for all methods grows, with EMPPO recording the lowest average task AuD, followed by CGO. Also, the gap in average task AuD between EMPPO and CGO gradually increases. This is mainly because CGO selects the UE corresponding to the channel with the highest SNR for data transmission. When the difference is zero, the channels can be regarded as three independent channels with the same average SNR, having equal transmission probabilities for each UE. However, as the difference increases, the channel with the highest average SNR is more likely to transmit data, while the channel with the lowest average SNR is likely to accumulate data packets.