## 4 Proposed Solution
### Main Characteristics of the Multi-Agent Gradient Descent Algorithm (MAGD)
\begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Dataset Name** & **MAGD** & **FMA** & **SBGD** & **Adam** & **GD** \\ \hline Iris & 2.0073 & 7.4819 & **0.1353** & 0.7797 & 0.7647 \\ Wine & 1.8196 & 7.8309 & **0.1648** & 0.8613 & 0.8372 \\ Breast Cancer-Wisconsin & 2.2510 & 10.2646 & **0.4303** & 1.1015 & 1.0794 \\ Digits & 11.5828 & 53.5168 & **3.0308** & 6.4266 & 6.6754 \\ Pima-Indians-Diabetes & 1.9711 & 8.9067 & **0.4297** & 1.0061 & 1.0352 \\ Connectionist Bench & 2.6584 & 16.9505 & **0.4844** & 2.5504 & 2.4633 \\ Glass Identification & 1.3941 & 7.9909 & **0.2585** & 0.9120 & 0.9146 \\ Balance Scale & 1.7598 & 8.3114 & **0.1932** & 0.9340 & 0.8763 \\ Banknote Authentication & 1.8752 & 9.4750 & **0.5279** & 1.0181 & 0.9934 \\ Adult & 160.4189 & 802.5116 & **40.1365** & 89.4274 & 88.4908 \\ Diabetes & 8.0211 & 7.0622 & **0.1925** & 0.8138 & 0.7878 \\ California Housing & 1.9058 & 57.6776 & **0.24739** & 7.1823 & 8.1433 \\ Boston Housing & 1.7830 & 7.3301 & **0.2278** & 0.8431 & 0.8244 \\ Auto MPG & 1.2297 & 6.7451 & **0.1115** & 0.7248 & 0.7024 \\ Concrete Compressive Strength & 1.7650 & 7.5510 & **0.1668** & 0.8032 & 0.7982 \\ Energy Efficiency & 2.2301 & 7.2413 & **0.1361** & 0.7633 & 07519 \\ Yacht Hydrodynamics & 1.088 & 6.6225 & **0.1232** & 0.7054 & 0.6873 \\ Forest Fires & 2.1748 & 8.4036 & **0.2031** & 0.9233 & 0.8603 \\ Airfoil Self-Noise & 2.0718 & 7.6019 & **0.1645** & 0.8126 & 0.8074 \\ Concrete Slump Test & 1.6497 & 6.4361 & **0.1177** & 0.6745 & 0.6580 \\ \hline \hline \end{tabular}
\end{table} 
Table 22: Statistical comparison of iterations required for convergence for different algorithms in neural network training. \begin{table}
\begin{tabular}{c c} \hline \hline
**Algorithm** & **Mean \(\pm\) Std. Dev.** \\ \hline MAGD & \(499\pm 1\) \\ FMA & \(500\pm 0\) \\ SBGD & \(10\pm 4\) \\ Adam & \(500\pm 0\) \\ GD & \(500\pm 0\) \\ \hline \hline \end{tabular}
\end{table} 
Table 23: Statistical comparison between the execution duration of different algorithms for neural network training. Tables 23 and 24 indicate that FMA's mean duration is significantly longer than that of the other algorithms. Adam and GD exhibit approximately equivalent mean durations and standard deviations. SBGD has the shortest execution time, while MAGD takes slightly longer than Adam, with a somewhat broader range within which 90% of the execution times fall. Given that Adam is the most competitive algorithm to MAGD, determining the differences in execution time relative to Adam is of primary interest. Table 23 initially suggests no statistically significant difference between the execution times of MAGD and the other algorithms. However, these values appear inconsistent with observations, as the differences in execution times are visually apparent. The unreliability of the initial \(t\)-test results stems from the presence of outliers, which can distort the findings. To address this, the outlier case (Adult dataset) was removed, and the \(t\)-test was repeated. The revised \(p\)-values, shown in Table 25, yield more interpretable results. The updated results demonstrate that the difference in execution time between MAGD and Adam is not statistically significant. This aligns with our earlier conclusion in the time complexity Section 4.2.1, which asserts that as \(d\gg N_{0}\) (a scenario that occurs frequently in our case), or \(t_{max}\gg N_{0}\) (in our case, \(500\gg 10\)), MAGD's time cost approaches Adam's execution time. Adam with the Same Time Budget vs. MAGD

The Adam algorithm, one of the most widely used optimization methods in machine learning, warrants a comparative analysis to assess whether it can achieve similar performance to MAGD when allocated an equivalent execution time. Such an investigation is crucial for evaluating whether Adam, with extended iterations, can match MAGD's performance, potentially making MAGD unnecessary in time-sensitive applications. Initially, all algorithms were executed for a maximum of 500 iterations.