## 4 Proposed Solution
### Main Characteristics of the Multi-Agent Gradient Descent Algorithm (MAGD)
The inputs include the following:  
```
Input:\(T_{\text{conv}}\), \(\epsilon_{\text{conv}}\), \(T_{\text{red}}\), \(N_{\text{agents}}\), \(f(x)\), \(t_{\text{max}}\), \(\alpha\), \(\beta_{1}\), \(\beta_{2}\), \(\epsilon\) Output:\(x_{best}\)  Initialize agents \(x_{i}^{(0)}\sim\mathcal{U}(X)\), for \(i=1,\ldots,N_{\text{agents}}\);  Initialize \(m_{i}^{(0)}\gets 0\), \(v_{i}^{(0)}\gets 0\), for \(i=1,\ldots,N_{\text{agents}}\); for\(t\gets 1\)to\(t_{\text{max}}\)do foreachagent\(x_{i}\)do \(\
abla f_{i}^{(t)}=\
abla f(x_{i}^{(t)})\);// Compute gradient \(m_{i}^{(t+1)}\leftarrow\beta_{1}m_{i}^{(t)}+(1-\beta_{1})\
abla f_{i}^{(t)}\);// Equation (1) \(v_{i}^{(t+1)}\leftarrow\beta_{2}v_{i}^{(t)}+(1-\beta_{2})(\
abla f_{i}^{(t)}) ^{2}\);// Equation (2) \(\hat{m}_{i}^{(t+1)}\leftarrow\frac{m^{(t+1)}}{1-\beta_{1}^{(t)}}\), \(\quad\hat{v}_{i}^{(t+1)}\leftarrow\frac{v_{i}^{(t+1)}}{1-\beta_{2}^{(t)}}\); \(x_{i}^{(t+1)}\leftarrow x_{i}^{(t)}-\alpha\frac{\hat{m}_{i}^{(t+1)}}{\hat{v}_{i }^{(t+1)}+\epsilon}\); \(f_{i}^{(t+1)}\gets f(x_{i}^{(t+1)})\);// Update cost \(x_{best}\gets argmin_{x}(f^{(t+1)})\);// Update best agent // Removing underperformed agents if\(t>T_{\text{conv}}\)then foreachagent\(x_{i}\)do  Converged\({}_{i}\leftarrow(\|f_{i}^{(t)}-f_{i}^{(t-T_{\text{conv}})}\|_{1}<\epsilon_{\text{conv}})\); ifConverged\({}_{i}\); & (\(f_{i}^{(t+1)}>f_{best}^{(t+1)}\))then  Remove agent\(x_{i}\); \(N_{\text{agents}}\gets N_{\text{agents}}-1\); // Periodic reduction procedure if\(t\bmod T_{\text{red}}=0\)then \(f^{(t+1)}\leftarrow\frac{1}{N_{\text{agents}}}\sum_{i=1}^{N_{\text{agents}}}f_{i }^{(t+1)}\);// Compute the mean cost foreachagent\(x_{i}\)do if\(f_{i}^{(t+1)}>f^{(t+1)}\)then  Remove agent\(x_{i^{\prime}}\); \(N_{\text{agents}}\gets N_{\text{agents}}-1\); // Terminate the algorithm if it converged if\(\|f_{best}^{(t+1)}-f_{best}^{(t)}\|_{1}<\epsilon_{\text{conv}}\)then Terminate the algorithm and output \(x_{best}\); else Update \(t\gets t+1\) and continue the next iteration return\(x_{best}\);
```  
**Algorithm 2**MAGD Algorithm  
The pseudocode for MAGD is detailed in Algorithm 2. The inputs include the following:  
``` Input:\(T_{\text{conv}}\), \(\epsilon_{\text{conv}}\), \(T_{\text{red}}\), \(N_{\text{agents}}\), \(f(x)\), \(t_{\text{max}}\), \(\alpha\), \(\beta_{1}\), \(\beta_{2}\), \(\epsilon\) Output:\(x_{best}\)  Initialize agents \(x_{i}^{(0)}\sim\mathcal{U}(X)\), for \(i=1,\ldots,N_{\text{agents}}\);  Initialize \\(x\): Search space. * \(f(x)\): Cost function to minimize. * \(\alpha\in(0,1]\): Learning rate (typically less than 0.3). * \(\beta_{1},\beta_{2}\in(0,1]\): Exponential decay rates for Adam (default values: 0.9, 0.999). * \(\epsilon\): Small constant to prevent division by zero in Adam. * \(t_{\text{max}}\): Maximum number of iterations. * \(N_{\text{agents}}\): Initial number of agents. * \(T_{\text{conv}}\in[1,t_{\text{max}}]\): Iterations after which convergence is checked. A smaller value reduces computational cost but may compromise solution quality (default: between 1 and 10). * \(\epsilon_{\text{conv}}\): Threshold for agent convergence (default: 0.0001). * \(T_{\text{red}}\in[1,t_{\text{max}}]\): Interval for periodic agent reduction. This value should be significantly smaller than \(t_{\text{max}}\) to facilitate faster pruning. ### Time and Space Complexity Analysis of MAGD

#### 4.2.1 Time Complexity

Gradient Computation and Update

The time complexity in each iteration arises from gradient computation and position updates. For each agent, computing the gradient and updating its position requires \(O(d)\), where \(d\) is the dimensionality of the agent's position.