## 4 Proposed Solution
### Main Characteristics of the Multi-Agent Gradient Descent Algorithm (MAGD)
**Classification Test Results Discussion:** The tables reveal the underperformance of SBGD and GD. Since FMA is excluded from the comparison, the focus is on comparing MAGD and Adam. To formalize the comparison, the following symbol is introduced:

* \(A_{1}\succ A_{2}\): indicates that algorithm \(A_{1}\) absolutely outperforms \(A_{2}\). Using this notation, the superiority ratio of algorithm \(A_{1}\), denoted \(P_{super}\in[-100\%,\)\(100\%]\), is defined in Equation (29). This metric quantifies the proportion of datasets where \(A_{1}\) outperforms \(A_{2}\), adjusted for cases of underperformance. A negative value indicates underperformance, while \(0\%\) denotes equivalence. \[P_{super}(A_{1}\succ A_{2})=100\cdot\frac{\sum\bigl{(}score_{A_{1},i}>score_ {A_{2},i}\bigr{)}-\sum\bigl{(}score_{A_{1},i}<score_{A_{2},i}\bigr{)}}{n} \tag{29}\]

Here,

* \(A_{1}\) and \(A_{2}\): the algorithms being compared. * \(n\): total number of datasets. * \(score_{A_{i},i}\): performance score of algorithm \(A\) on the \(i\)-th dataset. \begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Dataset Name** & **MAGD** & **FMA** & **SBGD** & **Adam** & **GD** \\ \hline Diabetes & 0.41 & 0.32 & 0.33 & 0.38 & **0.50 \({}^{*}\)** \\ California Housing & **0.71 \({}^{*}\)** & **0.71** & 0.45 & **0.71 \({}^{*}\)** & 0.58 \\ Boston Housing & **0.89 \({}^{*}\)** & **0.89** & 0.56 & 0.88 & 0.72 \\ Auto MPG & 0.89 \({}^{*}\) & **0.90** & 0.79 & 0.89 \({}^{*}\) & 0.79 \\ Concrete Strength & **0.85 \({}^{*}\)** & **0.85** & 0.56 & **0.85 \({}^{*}\)** & 0.63 \\ Energy Efficiency & 0.96 \({}^{*}\) & **0.97** & 0.84 & 0.95 & 0.89 \\ Yacht Hydrodynamics & **0.98 \({}^{*}\)** & **0.98** & 0.58 & **0.98 \({}^{*}\)** & 0.62 \\ Forest Fires & \(-23.66\) & \(-19.46\) & \(-1.58\) & \(-17.24\) & \(-\)**0.98 \({}^{*}\)** \\ Airfoil Self-Noise & **0.79 \({}^{*}\)** & **0.79** & 0.37 & 0.72 & 0.55 \\ Concrete Slump Test & 0.95 & **0.96** & 0.74 & **0.96 \({}^{*}\)** & 0.82 \\ \hline \hline \end{tabular}
\end{table} 
Table 18: \(R^{2}\) scores of different algorithms across various regression datasets. Bold values highlight the highest score achieved on each dataset. (\({}^{*}\)) indicates the maximum score among all algorithms except FMA. \begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Dataset Name** & **MAGD** & **FMA** & **SBGD** & **Adam** & **GD** \\ \hline Iris & **100\% \({}^{*}\)** & **100\%** & 80\% & **100\% \({}^{*}\)** & 78\% \\ Wine & 98\% \({}^{*}\) & 98\% & 94\% & **98\% \({}^{*}\)** & **98\% \({}^{*}\)** \\ Breast-Cancer-Wisconsin & 96\% \({}^{*}\) & 96\% & 94\% & **96\% \({}^{*}\)** & 96\% \({}^{*}\) \\ Digits & 97\% \({}^{*}\) & 97\% & 82\% & **97\% \({}^{*}\)** & 77\% \\ Pima-Indians-Diabetes & **79\% \({}^{*}\)** & **79\%** & 77\% & 76\% & 77\% \\ Connectionist Bench & 78\% \({}^{*}\) & 76\% & 75\% & 76\% & 76\% \\ Glass Identification & 62\% & 57\% & 48\% & 55\% & 45\% \\ Balance Scale & **90\% \({}^{*}\)** & **90\%** & **90\% \({}^{*}\)** & 87\% & 80\% \\ Banknote Authentication & **99\% \({}^{*}\)** & 98\% & 90\% & 98\% & 85\% \\ Adult & **85\% \({}^{*}\)** & **85\%** & 79\% & **85\% \({}^{*}\)** & 80\% \\ \hline \hline \end{tabular}
\end{table} 
Table 17: Accuracy scores of different algorithms across various classification datasets. Bold values highlight the highest accuracy achieved on each dataset. (\({}^{*}\)) indicates the maximum accuracy among all algorithms except FMA. Table 19 summarizes the superiority ratio of MAGD over Adam in classification tasks. While MAGD achieves the highest accuracy across all datasets, its accuracy is comparable to Adam's in 50% of the experiments. Thus, MAGD exhibits a 50% improvement ratio over Adam. **Regression Results Discussion:** In regression tasks, MAGD achieves the maximum \(R^{2}\) score in 50% of the datasets, with Adam also achieving the maximum in 50%. GD attains the highest \(R^{2}\) score in two cases, whereas SBGD achieves none. To clarify MAGD's absolute superiority, Table 20 reports the superiority ratio of MAGD compared to Adam and GD in regression tasks.