## 4 Proposed Solution
### Main Characteristics of the Multi-Agent Gradient Descent Algorithm (MAGD)
Different colors correspond to different algorithms, and diamond markers indicate outliers. Figure 1:7 Box plot of the minimum values and execution times obtained by various algorithms for the Zettl function. The left plot shows the best cost, while the right plot represents execution time. Different colors correspond to different algorithms, and diamond markers indicate outliers. Mathematical Benchmarks Experiments: Box Plots Analysis

By observing the box plots in the left subplot, which visualize the minimum cost found by different algorithms, it is evident that MAGD, FMA, and SBGD achieve the most optimal values of the cost functions. They show nearly identical statistical distributions in most of the experiments, suggesting no significant difference between the three algorithms in terms of optimal solution finding. However, when we analyze the right subplot, which visualizes the execution time variability for each algorithm, we can clearly observe an additional advantage of MAGD, as it takes the shortest execution time among the three algorithms. Adam and GD significantly underperform compared to the multi-agent or swarm-based algorithms in terms of optimal value finding, as evident from the box plots of best costs. The entire distribution of the optimal costs is significantly higher than that of the superior algorithms. Adam exhibits superior capability in finding optimal solutions compared to GD in most benchmarks. However, the improvement in solution quality is considerably smaller than that achieved by multi-agent algorithms. Despite the shorter execution time of Adam and GD, they do not show a significant difference from MAGD, meaning that

Figure 1:8 Box plot of the minimum values and execution times obtained by various algorithms for the Leon function. The left plot shows the best cost, while the right plot represents execution time. Different colors correspond to different algorithms, and diamond markers indicate outliers. Figure 1:9 Box plot of the minimum values and execution times obtained by various algorithms for the Drop-Wave function. The left plot shows the best cost, while the right plot represents execution time. Different colors correspond to different algorithms, and diamond markers indicate outliers. Figure 2:0 Box plot of the minimum values and execution times obtained by various algorithms for the Langermann function. The left plot shows the best cost, while the right plot represents execution time. Different colors correspond to different algorithms, and diamond markers indicate outliers. their short execution time cannot compensate for their significant underperformance in terms of optimal value finding. Regarding MAGD and SBGD, although they exhibit similar optimality-finding capabilities, SBGD often requires longer execution times. This difference is not very apparent in the visualization due to the scale of the figures and the high bias of the FMA chart. The primary challenge with SBGD lies in tuning its numerous hyperparameters to strike a balance between exploration and time efficiency. In this study, the chosen parameter settings for SBGD resulted in no significant superiority over MAGD while incurring additional time overhead. This further highlights that MAGD provides an efficient and cost-effective solution. #### 5.2.2 Mathematical Benchmarks Experiments Numerical Results

To make the information clearer and easier to digest, in Table 5, we present the median of the absolute difference between the true global minima and the found minima from the 10 seeds for each benchmark. For simplicity, we refer to this difference as the 'error' of the algorithm as illustrated in Equation (21). We prefer the median over the mean because it is not affected by outliers. Figure 2:1

where

* Median_Error\({}_{f,a}\): The median error for the algorithm \(a\) when applied to the function \(f\), calculated over 10 different random seeds. * \(f^{*}\): The global minimum value for function \(f\) based on the true global minima (this is a fixed value for the function across all seeds and algorithms).