## 4 Simulation Results
Figure 8 also shows that PPO and EMPPO methods based on reinforcement learning are less affected by the increase in channel differences, indicating that the method based on reinforcement learning can adapt to and balance the transmission of channels under different channel states. Figure 8: The average task AuD under different average SNR differences between channels. Figure 7: The average task AuD under different channel correlation coefficients. Figure 9 shows the influence of the channel average SNR. As the SNR increases, the average task AuD of each method decreases. When the average SNR is high, data packets can be transmitted with a shorter blocklength, allowing more packets to be transmitted in a single frame. The results of methods based on greedy algorithms tend to be closer to EMPPO. However, when the average SNR drops, data packets require longer blocklength for transmission, and often, a task's data packets could be transmitted over multiple frames. In this scenario, deciding which data packets are transmitted in which frames significantly impacts the average task AuD. EMPPO performs better than other methods, suggesting that EMPPO is more applicable in environments with a low SNR. In extremely low SNR environments, a frame's total blocklength might not be enough to transmit a single data packet. Thus, the average task AuD differences among different methods become less pronounced. In Figure 9, we can observe that RO and AO show different convexity characteristics from other methods. That is because their performance approaches the maximum average task AuD. The maximum average task AuD is influenced by the number of simulation steps, which is set to 200 in the simulation. This leads to a maximum average task AuD of 100 ms (this corresponds to scenarios where no data packets are transmitted in any frame). An algorithm performing close to this upper bound will show a reduced rate of change, hence showing the distinct convexity. We can also observe this trend in Figures 6 and 10. Figure 10 shows the influence of UE number. When the number of UEs increases, the task AuD of each method increases. This is because the one task's packet number is increasing (equal to the number of UEs). While the available blocklength for transmission is fixed, we need more frames to transmit the packets for one task. When the number of UEs is small, EMPPO's task AuD is similar to that of FGO and CGO. However, when the number of UEs is large, the performance of EMPPO is significantly better than other methods, which is consistent with the performance under different task arrival probabilities and confirms our analysis of Figure 6; that is, EMPPO can achieve better performance than other methods under high-load scenarios, and standard PPO has a lot of invalid action space to explore, which leads to a lower task AuD. Figure 9: The average task AuD under different channel average SNR. Figure 11 shows the influence of data packet size range. We have set up three different ranges for data packet size, each with the same average packet size of \(m=1500\) bits. Figure 11 shows that the average task AuD decreases as the packet size range decreases. Specifically, when the range is set to \([800,2200]\), the average task AuD of EMPPO drops by about 33% and 7% compared to ranges \([300,2800]\) and \([500,2500]\). On the one hand, this is because a smaller packet size range results in a more concentrated distribution of packet size, which leads to a more focused distribution of the required blocklength, allowing the agent to predict the required blocklength for packets more accurately. On the other hand, the probability of encountering a large packet size is reduced, which decreases the probability of packets requiring a blocklength that exceeds frame length constraints. This enables packets to be transmitted successfully under relatively poor channel SNR.