# A Period Training Method for Heterogeneous UUV Dynamic Task Allocation
## 4 Simulation Experiment
### Simulation Experimental Results  
Figure 6 shows the allocation results of MARLAM-PTM in three cases. The values and times in Table 1 represent the values of the objective function in Equation (1) and the running times of the different methods, respectively. Figure 6a displays the allocation result of case 1. The maximum task load of the UUVs in case 1 can be calculated with the constraint given in (4), i.e., \(14=ceil(40/3)\). As shown in Figure 6a, the numbers of allocation tasks for the four UUVs are 9, 8, 9, and 14, respectively. The scheme allocation of MARL-TPM meets the maximum task load constraint. In case 1, the UUVs conduct the allocation task sets according to the planning order and search them using the spiral search strategy. After completing the tasks, all UUVs return to the base. As shown in Table 1, the proposed MARLAM-PTM exhibits a fast solution speed due to the new solving framework. Figure 6b demonstrates the scheme allocation of MARLAM-PTM in case 2, where the numbers of tasks and training conditions are different. Some task areas have been searched and are connected by broken lines, and the brown dots represent the new search tasks. In Figure 6b, the new search tasks are mainly located in the left corner of the task environment, near the original allocation tasks of UUV 2. In the reallocation of schemes, most of the new tasks are assigned to UUV 2 in real time, ensuring the reduction in the overall traveling distances of the UUVs. As shown in the results of case 2, the MARL-AMPTM adaptively reallocates tasks to each UUV and ensures that all new task areas can be searched. Figure 5: The original task situation with the approximate area model. (**a**) Irregular target areas; (**b**) approximate area models. As shown in Figure 6c, in case 3, the number of UUVs and tasks are both different from the training conditions of the two MARL models. During the process of carrying out tasks, UUV 4 is broken, and the unfinished tasks need to be reallocated to other UUVs. The locations of the unfinished tasks are mainly near the task sets of UUV 3. Figure 6c shows most of them are reallocated to UUV 3. It is noteworthy that the result in case 3 shows that the MARLAM-PTM model obtains a lower value than the MARLAM model. The periodically changing training conditions and the greedy parameter replacement rule in the PTM enable the model to maintain robustness in solving problems with different numbers of UUVs. To test the robustness and scalability of MARLAM-PTM in various situations, we add four cases, and each of these contains three different task allocation problems. Ash shown Table 2 and Figures 7-10, the additional tests have different settings. It is worth noting that we use the same trained model in all four cases, which have different test conditions. Table 2 and Figures 7-10 show the results of task allocation. \begin{table}
\begin{tabular}{c c c c c c c} \hline \hline
**Algorithm** & \multicolumn{2}{c}{**Case 1**} & \multicolumn{2}{c}{**Case 2**} & \multicolumn{2}{c}{**Case 3**} \\ \hline  & **Value** & **Time** & **Value** & **Time** & **Value** & **Time** \\ \hline MTWPS & 209.2 & 12.3 s & 223.1 & 13.1 s & 197.1 & 12.5 s \\ DDPIO & 206.5 & 14.6 s & 264.6 & 17.2 s & 211.1 & 14.9 s \\ MARLAM & 203.4 & 2.0 s & 210.2 & 2.3 s & 197.0 & 1.5 s \\ MARLAM-PTM & 205.8 & 1.8 s & 205.6 & 2.4 s & 183.3 & 1.5 s \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Optimization results of different algorithms. Figure 6: The results of MARL-AMPTM: (**a**) the allocation result of case 1; (**b**) the reallocation result of case 2; (**c**) the reallocation result of case 3.