## 4 Evaluation
### Dynamic Environment Complex Experiment  
Given the dynamic user behavior and heterogeneous network conditions, we know that multiagent solutions can achieve relatively stable results and stay near these results for a long time. We compare the performance of multiagent deep deterministic policy gradient (MADDPG) and AMARL models with added attention mechanisms in more complex situations. MADDPG is an algorithm designed for multiagent, continuous behavior spaces. The predecessor of MADDPG is DDPG. The purpose of the DDPG algorithm is to solve the problem of reinforcement learning in the continuous behavior space. MADDPG is improved on the basis of DDPG to make it suitable for cooperative task learning between multiagents. The results are shown in Figure 9. The agents trained under AMARL can effectively use all the information received, and can better adapt to complex and changeable dynamic environments.  
Further, we also compare the average QoE and average bitrate of different methods. Figure 10 shows the average QoE of users and the average bitrate of video streams for four different methods. As can be seen from the figure, both AMARL and step-by-step reinforcement learning can make the average bitrate of the video stream reach a higher value, but the user QoE of AMARL is 70% higher than it. For both greedy and random strategies, both indicators of AMARL are well above their values.  
We also explored the performance under different numbers of users and edge servers. Figures 11 and 12 show the trend of each parameter depending on the number of users or agents. Figures 11 and 12 demonstrate that the model obtained by AMARL has better performance in average user QoE compared with other methods. For the bitrate allocation shown in Figures 11 and 12, the effect of AMARL is weaker than that of the Step-by-step reinforcement learning method, but in terms of the smoothness shown in Figures 11 and 12, the effect of AMARL is significantly better than that of the step-by-step reinforcement learning method. This also reflects the importance of the adaptive bitrate algorithm. It should be noted that from (1), the smaller the smoothness, the better the user experience. In summary, our proposed model performs well in complex dynamic environments and always maintains a relatively high bitrate. Moreover, the change in bitrate is relatively smooth. Through experiments on Pensieve, it can be seen that although the same parameters and dataset are used, the training results vary greatly, probably because the input of Pensieve contains RTT, and the users selected by the greedy and random strategies are different before each training; i.e., the selection of RTT is also different. Table 4 shows a comparison of the number of servers used by different algorithms under  
Figure 9: Convergence speed of AMARL and MADDPG.  
different numbers of agents. It can be seen that our proposed algorithm uses the fewest number of servers under different numbers of agents.  
Figure 11: Parameter variation trend as the number of agents changes: (**a**) QoE, (**b**) bitrate, and (**c**) smoothness.  
Figure 10: Experimental results of (**A**) AMARL, (**B**) step-by-step reinforcement learning, (**C**) greedy strategy, and (**D**) random strategy.