## 4 Results
### Experimental Results  
The overall results from 200 simulations are shown in Figure 9. The results confirmed that GGA and GGP were qualified as the upper bound of all teams. With respect to the average score in Figure 9A, GGA showed the best performance. It outperformed all other teams' strategies, except for GGP, the most defensive strategy. Unlike the other teams (usually consisting of two defenders and two attackers), GGA was composed of four attackers, as shown in Figure 5B. We therefore assumed that GGA was able to score more than the others (Figure 9A).  
Despite the fact that the GGP ranked second in the average score, GGP showed the highest winning rate among the shortlisted teams. It could thus be seen as the most competitive winning strategy of all the teams. Based on strong physical contact with their opponents, the GGP pursued quick and aggressive behavior to score. This resulted in a consistent performance for all opponents. Since football is a game wherein winning rates  
Figure 8: Screenshots of the simulations. Blue and red triangles refer to the defenders and the attackers, respectively. (**A**) GGA, (**B**) GGP, (**C**) QMIX, (**D**) IQL.  
are more important than scores, it is rational to conclude that the best strategy was GGP (Figure 9C).  
Using these results, we analyzed the performance of the two MARL-based football teams. As expected, they were superior to the baseline, RW, but inferior to the upper bound, GGA and GGP. RW never won against any other teams. First, the two MARL-based football teams outperformed the RW team with regard to the score and winning rate (Figure 9A,C,D). The average scores of the two MARL-based teams were approximately three times higher than that of RW.  
To examine the impact of the reward signal, we measured the effect size between IQL and RW and between QMIX and RW using _Hedge's g_: RW was a baseline implementation for an artificial football team without using a reward signal, whereas the two MARL-based teams formed the treatment group of artificial football teams using a reward signal (Figure 9B). We intended to measure the effect with or without a reward signal to indirectly present the impact of a reward in the design of artificial football teams. The _Hedge's g_ values were 0.91 for the RW and IQL pair and 0.87 for the RW and QMIX pair. This result highlighted the strong effect when using a reward signal to design artificial football agents' behavior.  
Compared to GGA and GGP, the upper bounds, the two MARL-based football teams showed a lower performance with respect to both the average score and the winning rate. However, the MARL-based teams interestingly exhibited the most competitiveness against GGP. Since no teams were able to defeat GGP, we compared the average scores of all the teams. With GGP, we assumed that the lower the score by which the team lost, the more competitive the team was. The two MARL-based teams showed the best performance (\(-1.4\) and \(-2.6\) for QMIX and IQL, respectively) out of all other teams in this regard. In particular, they even outperformed the GGA (\(-3.2\)) team that specialized in offense and presented the best average score (Figure 9E).  
Figure 9: Simulation results: (**A**) Average score. (**B**) Effect size between IQL and RW and between QMIX and RW (Hedge’s g). (**C**) Average winning rate. (**D**) Reliability. (**E**) Simulation results in terms of average scores. The first column refers to the team, and the first row refers to the opponent. A positive number refers to the score with which the team won the game. A negative number refers to the number of points by which the team failed to win the game.