# A Multi-Agent Based Intelligent Training System for Unmanned Surface Vehicles
## 4 Multi-Agent-Based Simulator Design
### Experiment Results and Validation Analysis  
We have evaluated our intelligent training system using four distinct scenarios of the island conquering case study. The parameters used in Scenario 1 for both team are shown in Table 4. Furthermore, in Scenario 1, sinking an opponent and conquering islands were both worth two points each. In Scenario 2, the number of USVs in a team was reduced to three, and sinking an opponent point was reduced to one. The simulation for Scenarios 1 and 2 was run concurrently on two different machines for 2020 from 20 randomly-generated chromosomes as the initial rules for the blue force against the enemy team, which used a human-sampled rules sets. Figure 9 shows the snapshots of the competition as simulation progresses. If the blue force began an episode knowing exactly what to do in the environment and the red force was yet to learn, then the blue force should perform better than the red force in the initial stages of the simulation. However, as the simulation progresses, the performance of the red force should be improving since the red force learns, while the blue force uses the static rule. Furthermore, if the blue blue force is using a different set of rules even though a static set of rules, then there is a tendency for a dynamic rise and fall in the performance measure by the strength of a simulating rule set. \begin{table}
\begin{tabular}{c c} \hline \hline
**Parameter** & **Value** \\ \hline Population size & 20 \\ Number of generations & 101 (initial population included) \\ Crossover probability & 0.7 \\ Mutation probability & 0.1 \\ Mutation decay Schedule & Exponential decay (decay rate = 0.15) \\ Crossover operator & BLXcrossover (alpha = 0.15) \\ Mutation operator & Adaptive non-uniform mutation (b = 5) \\ Selection operator & Tournament selection \\ \hline \hline \end{tabular}
\end{table} 
Table 3: The GA parameters used for training the ally team. _Appl. Sci._**2019**, \(9\), 1089 \begin{table}
\begin{tabular}{c c} \hline \hline
**Parameter** & **Value** \\ \hline Number of boats per team & 6 \\ Number of islands & 5 \\ Simulation steps per episode & 30 \\ Time per step & 3 s \\ Radar radius & 140 \\ Forward firing angle & 20 \\ Boat’s velocity & 4/system iteration \\ Firing range & 100 \\ \hline \hline \end{tabular}
\end{table} 
Table 4: The fixed simulation parameters. Figure 9: Illustration of the best-performing rules as training progresses. At the initial stages, the blue team had total control over the red team. However, during the final stages of the training, the performance of the red surpasses that of the blue. The observed simulation results of both teams during training are presented in Figure 10. As can be seen in Figure 10,c, the blue force was performing better than the red force during the first generation of rules. However, after 10 and 13 generations, respectively, the red force began to outperform the blue force in the environment, and this is what caused the red force performance to rise in Figure 10,c. In Figure 10,d, we compare the highest score attained by both teams in each generation. Furthermore, the same analogy can be drawn from this graph, as in the the initial generations, the blue force seemed to be scoring high as compared to the counterpart red force. Figure 10: _Cont._  
In the third simulation, the capabilities of the red force was constrained below that of its counterpart blue force. With each team consisting of 10 boats, the firing range, detection range, and firing radius of the ally team was set to 80% of the opponent team, respectively. Furthermore, we set the initial population of rules to that of the rule base of Simulation 1. Figure 9(e),f shows the performance graphs of both teams. Finally, a simulation of 15 boats with the objective of destroying more opponent boats while still countering islands was conducted. To this end, we assigned three points for destroying opponent boats and a point for conquering an island.