## 4 Proposed Solution
### Main Characteristics of the Multi-Agent Gradient Descent Algorithm (MAGD)
* \(tol_{\text{res}}\in[0,1]\): This parameter is equivalent to \(\epsilon_{\text{conv}}\) in MAGD and represents the convergence tolerance. * \(\lambda\) and \(\gamma\in[0,1]\): These parameters control the learning rate updating procedure in SBGD. For further details, refer to [15]. 2.1 Box Plot of Minimum Values of Benchmark Mathematical Functions Found by Different Algorithms Through 10 Different Seeds

Box plots, also known as box-and-whisker plots, are diagrams that visualize the statistical distribution of data [45] (p. 2). These diagrams represent compact and easily understandable statistical metrics of the data, such as the median, Interquartile Range (IQR), distribution fences, and outliers. This allows for comparing different algorithms across numerous experiments by summarizing them in a single diagram. Figure 1 provides a guide to help interpret the box plot diagram. Chart (A) shows the conventional box plot, while chart (B) displays a specialized version known as a violin plot, which includes additional information about the distribution density of the points. However, we have used the conventional box plot in our paper because it is clearer and more elegant. \begin{table}
\begin{tabular}{c c} \hline \hline
**Parameter** & **Value** \\ \hline \(tol_{\text{rms}}\) & \(10^{-3}\) \\ \(tol_{m}\) & \(10^{-4}\) \\ \(\lambda\) & 0.2 \\ \(\gamma\) & 0.9 \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Setting of the exclusive Hyperparameters for SBGD in mathematical benchmark functions experiments. \begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Algorithm** & \(N_{agents}\) & \(\alpha\) & \(\beta_{1}\) & \(\beta_{2}\) & \(\epsilon_{conv}\) & \(\epsilon\) \\ \hline GD & 1 & 0.01 & - & - & \(10^{-4}\) & - \\ Adam & 1 & 0.01 & 0.9 & 0.999 & \(10^{-4}\) & \(10^{-8}\) \\ FMA & 100 & 0.01 & 0.9 & 0.999 & \(10^{-4}\) & \(10^{-8}\) \\ MAGD & 100 & 0.01 & 0.9 & 0.999 & \(10^{-4}\) & \(10^{-8}\) \\ SBGD & 100 & 3.00 & - & - & \(10^{-4}\) & - \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Hyperparameters setting for All Algorithms in mathematical benchmark functions experiments. \begin{table}
\begin{tabular}{c c} \hline \hline
**Parameter** & **Value** \\ \hline \(tol_{\text{rms}}\) & \(10^{-3}\) \\ \(tol_{m}\) & \(10^{-4}\) \\ \(\lambda\) & 0.2 \\ \(\gamma\) & 0.9 \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Setting of the exclusive Hyperparameters for MAGD in mathematical benchmark functions experiments. The results of the experiments on mathematical benchmark functions are presented in Figures 2-21. These figures display the box plot diagrams of the optimal minimum values (left axis) and execution durations (right axis) in seconds for the different algorithms across the 10 random generation seeds. Figure 1: Structure of box plots: (**A**) A box plot depicting the distribution using five statistical measures: minimum, first quartile (Q1, 25 th percentile), median (Q2, 50 th percentile), third quartile (Q3, 75 th percentile), and maximum. The whiskers extend to the smallest and largest data points within the lower and upper fences (\(Q1-1.5\times IQR\) and \(Q3+1.5\times IQR\)). Data points beyond these limits are considered outliers and are marked with asterisks (*). The notch around the median represents the 95% confidence interval for the median; (**B**) A violin plot, a variant of the box plot that provides additional insights into the data distribution. It visualizes the probability density distribution along the y-axis. The central box denotes the interquartile range (IQR), and the white dot marks the median. _Note: Red-colored text highlights key statistical measures (minimum, maximum, quartiles, and median)_, adapted from [45] (p. 3). Figure 2: Box plot of the minimum values and execution times obtained by various algorithms for the Rastrigin function. The left plot shows the best cost, while the right plot represents execution time. Different colors correspond to different algorithms, and diamond markers indicate outliers.