# Service-Driven Dynamic Beam Hopping with Resource Allocation for LEO Satellites
## 4 Simulation Results and Analysis
### Performance of the Applied Algorithm  
The episode rewards of the MARL-VDN algorithm applied in this paper during offline training in a dynamic environment are shown in Figure 5, where the weight coefficient \(\alpha\) equals 0.5. The weight coefficient \(\alpha\) is used to balance delay fairness and user satisfaction. \begin{table}
\begin{tabular}{c c} \hline \hline
**Algorithm Parameters** & **Values** \\ \hline Training episode & 2000 \\ Time slots per episode \(T\) & 100 \\ Learning rate & \(10^{-4}\) \\ Experience pool size & 2000 \\ Target network learning frequency & 100 \\ Minibatch size & 64 \\ Discount factor & 0.9 \\ Initial exploration rate & 1 \\ Final exploration rate & 0.05 \\ \hline \hline \end{tabular}
\end{table} 
Table 4: MARL-VDN parameters. The larger \(\alpha\) is, the more the system emphasizes delay fairness. Conversely, the smaller \(\alpha\) is, the more the system emphasizes user satisfaction. In this paper, when \(\alpha\) equals 0.5, multi-agents can better balance the two optimization objectives of delay fairness and user satisfaction. As can be clearly observed from the figure, during the 2000-episode training process, the reward curve gradually rises and eventually converges, which indicates that agents can cooperate with each other during the training process. In addition, the reward curve in the figure not only has fast convergence but also long-term stability, enabling this algorithm framework to adapt to complex and variable traffic demands with ease. To validate the long-term delay fairness performance of the algorithm, we tested scenarios with different traffic demands in 100 time slots. The traffic demands of each beam position range from 1500 to 2200 Mbps. \(\alpha\) still equals 0.5. Figure 6 shows the convergence of the average delay of data packets waiting as well as transmission during the training process of the algorithm. We can observe that during the 2000-episode training process, the delay fairness first decreases and then converges to a stable state. Eventually, the delay fairness converges to about 5.2 ms. This result proves that after sufficient training, the algorithm can effectively reduce the average delay of data packets at each beam position in scenarios with different traffic demands; it can achieve good delay fairness, adapt to long-term traffic changes, and has the ability to guarantee long-term delay fairness performance in the multi-agent system. Figure 7 shows the episode user satisfaction during the training of the algorithm in a dynamic environment. We can see that during the 2000-episode training process, the user satisfaction gradually increases and stabilizes, and the final satisfaction level remains at 85%. The simulation result indicates that the MARL-VDN algorithm can maintain a  
Figure 5: The episode reward under the optimization weight \(\alpha=0.5\). Figure 6: Episode delay fairness under optimization weight \(\alpha=0.5\). high level of resource utilization efficiency in the face of dynamic traffic changes, which strongly verifies the effectiveness of the multi-agent cooperation framework. The gentle characteristic of the latter part of the curve in the figure reflects that the system has strong anti-interference ability and is very suitable for dynamic satellite communication scenarios. In scenarios where ground users are unevenly distributed and demands are time-varying, the algorithm applied in this paper can dynamically adjust the beam resource allocation to effectively guarantee quality of service to most users. Furthermore, the sustained high user satisfaction observed even after 2000 episodes demonstrates the system's capacity for continuous optimization, effectively adapting to long-term variations in traffic demands. After training, user satisfaction can be stably maintained at a relatively high level, which reflects the good adaptation relationship between the throughput and the total traffic demand of the algorithm applied in this paper during long-term operation.