## 4 Proposed Solution
### Main Characteristics of the Multi-Agent Gradient Descent Algorithm (MAGD)  
The Multi-Agent Gradient Descent (MAGD) algorithm incorporates the following key features:  
1. Multi-Agent Initialization: The algorithm begins by generating a set of uniformly distributed initial solutions in the search space, each termed an \"agent\". The initialization is performed according to Equation (3): \[x_{i}\sim\mathcal{U}(x),\quad\text{for }i=1,\ldots,N_{\text{agents}},\] (3) where \(\mathcal{U}(x)\) denotes the uniform distribution, ensuring equal sampling probability across the search space. Since there is typically no prior information about regions more likely to contain the optima, a uniform distribution is preferred. However, if domain-specific knowledge is available, other distributions tailored to the problem may be used. Here, \(x\) represents the search space, and \(N_{\text{agents}}\) is the number of initial agents. 2. Adam Updating Rule Integration: Building on the substantial improvements demonstrated by the Adam algorithm [25], MAGD incorporates its position-update rule (Equations (1) and (2)) for each agent. This integration enhances the exploratory power of individual agents, contributing to the overall efficiency of the algorithm. 3. Adaptive Agent Pruning: Adaptive Agent Pruning is the most indispensable feature of MAGD for maintaining computational efficiency. This mechanism eliminates non-promising agents, significantly reducing execution time while preserving the exploration benefits of diverse initial positions. It employs two distinct strategies: (a) Convergence-Based Agent Reduction: As iterations progress, MAGD evaluates each agent's convergence status. Converged agents with inferior performance are removed. Convergence is determined by a negligible change in cost over a specified period \(T_{\text{conv}}\), expressed as: \[\text{Converged}=\|f^{(t+T_{\text{conv}})}-f^{(t)}\|<\epsilon_{\text{conv}},\] (4) where \(f^{(t)}\) and \(f^{(t+T_{\text{conv}})}\) represent the agent's cost at iterations \(t\) and \(t+T_{\text{conv}}\), respectively. \(\epsilon_{\text{conv}}\) is a small threshold (e.g., 0.001). It is important to recall that the removal of agents is performed with caution to ensure that the best agent is not lost. This principle aligns with the overarching goal of maintaining the quality of solutions during the pruning process. (b) Periodic Performance-Based Reduction: At fixed intervals (\(T_{\text{red}}\)), agents with costs above the mean cost are pruned. This interval is a hyperparameter, typically set to a small fraction of the maximum iterations, such as \(1\%\). Periodic reductionaccelerates computational efficiency. The pruning process is disabled once only one agent remains, thereby eliminating the additional computational complexity associated with this process, while the remaining agent continues to search for the global optimum, albeit without a guarantee of success. Stopping Criteria: The stopping criteria can be tailored to the specific problem and may include reaching the maximum number of iterations, detecting convergence, or satisfying another user-defined condition. In our implementation, the algorithm terminates upon convergence of the best solution or reaching the maximum iteration count. The pseudocode for MAGD is detailed in Algorithm 2.