## 5 Results and Discussion
### Performance of the Proposed Reward Scheme Designs  
In this paper, the setting of the reward mechanism had two main aspects. The first was the design of the reward function and the second was the design of the reward assignment scheme. There were four components in the reward function, including the collision evaluation, headway evaluation, speed evaluation, and rule evaluation. In order to validate the effectiveness of the reward function for decision-making, we set up an intersection crossing scenario. There were three vehicles in this scenario, where one HDV shown in blue was going straight and two CAVs shown in green were turning left, denoted as CAV\({}_{1}\) and CAV\({}_{2}\), respectively. In order to present the spatial relationship of the vehicles more intuitively, the positions of the vehicles were represented by the distance from the stop line along their path, rather than their world coordinates. The positions were negative when the vehicles crossed the stop line. The position and speed curves of all the vehicles during the whole process are shown in Figure 6. Figure 7 shows a sequence of the time slices for demonstrating the inter-vehicle interaction. Apparently, all the vehicles crossed through the conflict zone safely. As shown in Figure 5(a), when there was no interaction between vehicles, they could accelerate to the maximum speed of 10m/s to obtain a higher reward. In accordance with the traffic rules introduced in Section 3.2, the vehicle going straight goes first, and the one on the right has the right of way. Thereby, the passage sequence of the three vehicles was HDV, CAV\({}_{1}\), and CAV\({}_{2}\), which was exactly what is shown in Figure 7. There were two interactions during the crossing. CAV\({}_{1}\) and the HDV first interacted in the same lane, where CAV\({}_{1}\) slowed down and pulled away from the HDV ahead of it. The second interaction took place between the two CAVs. Since CAV\({}_{2}\) should have crossed later than CAV\({}_{1}\), CAV\({}_{2}\) decelerated to make way at 12.4 s. All the above analyses demonstrated that the designed reward function was qualified for the effective decision-making for the CAVs. \begin{table}
\begin{tabular}{c c c c c} \hline \multirow{2}{*}{**Metrics**} & \multicolumn{4}{c}{**Algorithm and Settings**} \\ \cline{2-5}  & **Benchmark +2C3H** & **Benchmark +4C5H** & **Attn-MAPPO +2C3H** & **Attn-MAPPO +4C5H** \\ \hline Collision rate & 0 & 0.13 & 0 & 0.03 \\ \hline Average speed (m/s) & 10.08 & 8.77 & 9.91 & 8.95 \\ \hline \end{tabular}
\end{table} 
Table 1: Performance comparison between the benchmark and the proposed Attn-MAPPO (proposed) algorithm. Figure 6: The speed and position curves of all the vehicles during the whole crossing; (**a**) the speed, (**b**) the position. To validate the performance of the proposed reward assignment scheme, the global reward, local reward, and the proposed weighted reward scheme were used for the training in the simple setting, i.e., the two CAVs and three HDVs. As many people are aware, collisions are more likely to occur in the areas within the intersections than in the straight lanes. When the system was penalized, the vehicle nearest the intersection should have been assigned more penalties to help it execute the correct decision. The same went for receiving a reward. The results are shown in Figure 8. As excepted, the results confirmed that the proposed weighted assignment outperformed the other two assignment schemes. Table 2 shows a clear improvement in the collision rates and the average speed metrics. This was because the global reward was not an accurate representation of the contribution of a CAV. Although the local reward improved on this, the reward could not be assigned based on the reward. Figure 7: The frames of the time slices showing the interactions performed by the simulation platform (blue: HDV; green: CAVs); (**a**) 9 s, (**b**) 12 s, (**c**) 15 s, and (**d**) 18 s.  
on an agent's location.