## 4 Proposed Solution
### Main Characteristics of the Multi-Agent Gradient Descent Algorithm (MAGD)
The hyperparameters used for each algorithm during neural network training are presented in Table 12. Algorithm-specific parameters for MAGD and SBGD are detailed in Tables 13 and 14, respectively. All algorithms were initialized with weights randomly generated within the range \([-1,1]\). For SBGD, FMA, and MAGD, the number of agents was set to 10, a reduced value due to resource constraints. Datasets were shuffled and split into 70% training and 30% testing sets. To ensure consistency, the random seed for weight initialization was set to 42 across all algorithms. \begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**No.** & **Dataset Name** & **Features** & **Samples** & **Task** \\ \hline
1 & Iris & 4 & 150 & Classification \\
2 & Wine & 13 & 178 & Classification \\
3 & Breast-Cancer-Wisconsin & 30 & 569 & Classification \\
4 & Digits & 64 & 1797 & Classification \\
5 & Pima-Indians-Diabetes & 8 & 768 & Classification \\
6 & Connectionist Bench & 8 & 208 & Classification \\
7 & Glass Identification & 9 & 214 & Classification \\
8 & Balance Scale & 4 & 625 & Classification \\
9 & Banknote Authentication & 4 & 1372 & Classification \\
10 & Adult & 14 & 48,842 & Classification \\
11 & Diabetes & 10 & 442 & Regression \\
12 & California Housing & 8 & 20,640 & Regression \\
13 & Boston Housing & 13 & 506 & Regression \\
14 & Auto MPG & 7 & 398 & Regression \\
15 & Concrete Compressive Strength & 8 & 1030 & Regression \\
16 & Energy Efficiency & 8 & 768 & Regression \\
17 & Yacht Hydrodynamics & 7 & 308 & Regression \\
18 & Forest Fires & 12 & 517 & Regression \\
19 & Airfoil Self-Noise & 5 & 1503 & Regression \\
20 & Concrete Slump Test & 7 & 103 & Regression \\ \hline \hline \end{tabular}
\end{table} 
Table 11: Experimental datasets. \begin{table}
\begin{tabular}{c c c c c c c} \hline \hline
**Algorithm** & \(\mathbf{N_{agents}}\) & \(\mathbf{\alpha}\) & \(\mathbf{t_{\max}}\) & \(\mathbf{c_{conv}}\) & \(\mathbf{\beta_{1}}\) & \(\mathbf{\beta_{2}}\) & \(\mathbf{c}\) \\ \hline MAGD & 10 & 0.03 & 500 & \(1\times 10^{-4}\) & 0.9 & 0.999 & \(1\times 10^{-8}\) \\ FMA & 10 & 0.03 & 500 & \(1\times 10^{-4}\) & 0.9 & 0.999 & \(1\times 10^{-8}\) \\ SBGD & 10 & 3 & 500 & \(1\times 10^{-4}\) & - & - & - \\ Adam & 1 & 0.03 & 500 & \(1\times 10^{-4}\) & 0.9 & 0.999 & \(1\times 10^{-8}\) \\ GD & 1 & 0.03 & 500 & \(1\times 10^{-4}\) & - & - & - \\ \hline \hline \end{tabular}
\end{table} 
Table 12: Hyperparameters settings for neural network training across different algorithms. The symbol “-” indicates that the parameter is absent in the respective algorithm. \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**No.** & **Dataset Name** & **Features** & **Samples** & **Task** \\ \hline
1 & Iris & 4 & 150 & Classification \\
2 & Wine & 13 & 178 & Classification \\
3 & Breast-Cancer-Wisconsin & 30 & 569 & Classification \\
4 & Digits & 64 & 1797 & Classification \\
5 & Pima-Indians-Diabetes & 8 & 768 & Classification \\
6 & Connectionist Bench & 8 & 208 & Classification \\
7 & Glass Identification & 9 & 214 & Classification \\
8 & Balance Scale & 4 & 625 & Classification \\
9 & Banknote Authentication & 4 & 1372 & Classification \\
10 & Adult & 14 & 48,842 & Classification \\
11 & Diabetes & 10 & 442 & Regression \\
12 & California Housing & 8 & 20,640 & Regression \\
13 & Boston Housing & 13 & 506 & Regression \\
14 & Auto MPG & 7 & 398 & Regression \\
15 & Concrete Compressive Strength & 8 & 1030 & Regression \\
16 & Energy Efficiency & 8 & 768 & Regression \\
17 & Yacht Hydrodynamics & 7 & 308 & Regression \\
18 & Forest Fires & 12 & 517 & Regression \\
19 & Airfoil Self-Noise & 5 & 1503 & Regression \\
20 & Concrete Slump Test & 7 & 103 & Regression \\ \hline \hline \end{tabular}
\end{table} 
Table 13: MAGD exclusive hyperparameters settings for neural network training. #### 5.3.2 Results of Neural Network Training

Training Performance and Accuracy

To compare the algorithms, we define the evaluation metrics used for this analysis, which are presented in Equations (24)-(26).