# Advanced Cooperative Formation Control in Variable-Sweep Wing UAVs via the MADDPG-VSC Algorithm
## 3 Cooperative Control of Variable-Sweep Wing UAV via MADDPG
### Structure and Training Process of the MADDPG-VSC Algorithm
#### 3.4.2 Algorithmic Performance Evaluation  
The UAV trajectories, shown in Figure 8, compare the performance of the MADDPG, soft actor-critic (SAC) and MADDPG-VSC algorithms. MADDPG, known for its effectiveness in multi-agent reinforcement learning tasks, particularly in handling continuous action spaces, has been widely adopted in UAV formation control. SAC offers a robust alternative with its stochastic policy, effectively balancing exploration and exploitation, and is particularly known for its quick convergence and stable performance in continuous action tasks. Both MADDPG and SAC successfully guide the three variable-sweep wing UAVs toward their target, navigating obstacles. However, MADDPG-VSC shows distinct advantages in more dynamic scenarios. The presence of obstacles necessitates longer trajectories and the application of a deformation acceleration strategy. While all UAVs reached their destination using MADDPG and SAC, MADDPG shows a notable trajectory deviation for UAV\({}_{2}\) when encountering an obstacle, bringing it closer to the obstacle than intended. In comparison, SAC exhibits more precise performance than MADDPG in early training phases but struggles to maintain accuracy in more complex, obstacle-rich environments. In contrast, MADDPG-VSC demonstrates only minor deviations, resulting in a more accurate and efficient flight trajectory. This highlights MADDPG-VSC's superior precision and effectiveness in highly dynamic environments, particularly in managing precise navigation and obstacle avoidance, thus outperforming SAC in later stages of training and complex scenarios. The results from Tables 5-7 demonstrate that MADDPG-VSC shows a clear advantage in regards to stability, as reflected in both maximum and total deviations. For instance, MADDPG-VSC's maximum deviation is 18.17 m, compared to MADDPG's 24.06 m and SAC's 19.00 m. Its total deviations of 11.83 m and 10.55 m are also smaller than those of SAC and MADDPG. This indicates that in multi-UAV cooperative tasks and complex environments, MADDPG-VSC achieves better trajectory control and reduced deviations. Its formation control capability and stability, particularly in dynamic scenarios, are clearly more reliable. \begin{table}
\begin{tabular}{p{113.8 pt} p{56.9 pt} p{56.9 pt} p{56.9 pt}} \hline \hline
**Algorithm** &
\begin{tabular}{c} **Maximum Deviation** \\ **(m)** \\ \end{tabular} & **Average Deviation (m)** & **Median Error (m)** & **Total Deviation (m)** \\ \hline MADDPG & 13.88 & 4.07 & 7.78 & 16.43 \\ MADDPG–VSC & 9.75 & 3.12 & 1.69 & 11.83 \\ SAC & 10.00 & 4.00 & 3.50 & 12.50 \\ \hline \hline \end{tabular}
\end{table} 
Table 6: UAV\({}_{2}\) absolute position error. \begin{table}
\begin{tabular}{p{113.8 pt} p{56.9 pt} p{56.9 pt} p{56.9 pt} p{56.9 pt}} \hline \hline
**Algorithm** &
\begin{tabular}{c} **Maximum Deviation** \\ **(m)** \\ \end{tabular} & **Average Deviation (m)** & **Median Error (m)** & **Total Deviation (m)** \\ \hline MADDPG & 12.38 & 3.91 & 2.10 & 14.66 \\ MADDPG-VSC & 8.79 & 2.55 & 1.57 & 10.55 \\ SAC & 9.05 & 3.15 & 2.50 & 12.50 \\ \hline \hline \end{tabular}
\end{table} 
Table 7: UAV\({}_{3}\) absolute position error. \begin{table}
\begin{tabular}{p{113.8 pt} p{56.9 pt} p{56.9 pt} p{56.9 pt} p{56.9 pt}} \hline \hline
**Algorithm** &
\begin{tabular}{c} **Maximum Deviation** \\ **(m)** \\ \end{tabular} & **Average Deviation (m)** & **Median Error (m)** & **Total Deviation (m)** \\ \hline MADDPG & 24.06 & 11.13 & 6.04 & 25.31 \\ MADDPG–VSC & 18.17 & 8.84 & 5.84 & 21.50 \\ SAC & 19.00 & 9.00 & 7.00 & 22.00 \\ \hline \hline \end{tabular}
\end{table} 
Table 5: UAV\({}_{1}\) absolute position error. Trajectory accuracy, reflected through average and median deviations, further emphasizes MADDPG-VSC's consistent performance. In Table 5, MADDPG-VSC's average deviation is 8.84 m, compared to MADDPG's 11.13 m and SAC's 9.00 m. Although SAC performs similarly to MADDPG-VSC in simpler environments, MADDPG-VSC excels in more complex scenarios.