## 4 Proposed Solution
### Main Characteristics of the Multi-Agent Gradient Descent Algorithm (MAGD)
\begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Function Name** & **MAGD** & **FMA** & **SBGD** & **Adam** & **GD** \\ \hline Rastrigin & 0.3666 & 5.9666 & 0.9698 & **0.0614** & 0.5393 \\ Ackley & 0.2912 & 4.7500 & 0.5302 & 0.0529 & **0.0253** \\ Goldstein–Price & 1.2312 & 12.4433 & 1.4645 & **0.3007** & 1.1632 \\ Levy-N13 & 0.8149 & 15.9472 & 1.1628 & 0.1855 & **0.1802** \\ Himmelblau & 0.3184 & 4.0358 & 0.6075 & **0.1071** & 0.1223 \\ Three-Hump Camel & 0.5151 & 5.7206 & 0.3478 & **0.1715** & 0.3120 \\ Griewank & 0.7627 & 17.4497 & 0.3462 & **0.2798** & 0.7080 \\ Cross-in-Tray & 0.1701 & 5.8178 & 0.8469 & 0.2595 & **0.0011** \\ Egg Holder & 1.1273 & 49.7233 & 0.5644 & 0.5661 & **0.5376** \\ Michalewicz & 0.1725 & 2.9696 & 0.2883 & 0.0313 & **0.0021** \\ Alpine & 0.4941 & 5.2489 & 0.6447 & 0.2768 & **0.2611** \\ Schaffer N2 & 0.8606 & 50.4603 & 0.3433 & **0.0251** & 0.5355 \\ Schaffer N4 & 0.8669 & 50.1864 & 0.3567 & **0.0259** & 0.5462 \\ Easom & 0.0405 & 0.0404 & 0.0627 & **0.0008** & 0.0012 \\ Shubert & 0.9665 & 23.1907 & 1.9983 & **0.1549** & 1.2867 \\ Schwefel & 0.4966 & 22.5983 & 0.2993 & 0.2717 & **0.2537** \\ Zettl & 0.5216 & 6.2368 & 0.8068 & 0.4134 & **0.4000** \\ Leon & 0.4899 & 7.5925 & 0.5555 & **0.3356** & 0.3564 \\ Drop-Wave & 0.2668 & 3.2161 & 0.3625 & 0.0337 & **0.0283** \\ Langermann & 1.7498 & 33.9237 & 3.0435 & **0.2352** & 0.2415 \\ \hline \hline \end{tabular}
\end{table} 
Table 8: Median execution time (in seconds) across 10 random seeds for different algorithms on various mathematical benchmark functions. Bold values indicate the best performance. Highest Density Interval (HDI), which represents the Bayesian-estimated interval where 95% of the distribution values lie [47] (p. 15). The HDI interval of execution times offers a more reliable metric for judging which algorithm takes less time. Furthermore, we conducted a \(t\)-test to calculate the \(p\)-values for the execution time differences between MAGD and each of the other benchmarks. The statistical metrics were derived from 200 samples (20 functions and 10 seeds per function) to ensure more robust results. Table 9 presents a comparison between MAGD and SBGD in terms of various statistical attributes, while Table 10 presents the \(p\)-values (probability of equivalent means between two samples) for the execution time differences between MAGD and each other benchmark. Tables 9 and 10 show a significant difference between MAGD and all other algorithms. Compared to GD and Adam, this difference highlights a relative disadvantage due to MAGD's longer duration. However, the significantly shorter execution time of MAGD compared to SBGD demonstrates a clear advantage, especially considering its similar



Figure 23: Convergence patterns of mathematical functions: Group 2. Figure 24: Convergence patterns of mathematical functions: Group 3. Figure 22: Convergence patterns of mathematical functions: Group 1. ### Neural Network Training

In this subsection, we present the results of applying our algorithm to a real-world problem: neural network training. We trained shallow neural networks and evaluated their performance on various benchmark datasets. The results of our algorithm were compared with those of FMA, SBGD, Adam and GD. #### 5.3.1 Experimental Design

The neural network consists of three layers: an input layer, a hidden layer, and an output layer. The input layer has weights shaped as (input_features_number, hidden_neurons_number). The hidden layer has a number of neurons equal to \((2/3)\times\text{input\_features\_number}+1\). The output layer contains one neuron for regression tasks and a variable number of neurons for classification tasks. The activation function is linear for regression and softmax for classification. Table 11 lists the datasets used in the experiments. Figure 26: Convergence patterns of mathematical functions: Group 5. Figure 25: Convergence patterns of mathematical functions: Group 4.