## 4 Proposed Solution
### Main Characteristics of the Multi-Agent Gradient Descent Algorithm (MAGD)
However, MAGD incorporates a mechanism that exponentially reduces storage requirements over time, as described by Equation (8). This ensures that storage remains manageable throughout the optimization process. To demonstrate the effectiveness of the adaptive agent reduction process, we implemented the standard Adam algorithm with the same number of agents as MAGD but without the reduction process. This implementation, referred to as Fixed Multi-Agent Adam (FMA), serves as a straightforward application of Adam to multiple agents and is used solely for comparison. ## 5 Experiments and Results

### Experimental Setup and Implementation Details

In this section, we present the experiments conducted and discuss the results obtained from the implementation of our algorithm. The experiments were performed on a Kaggle online server equipped with an Intel(R) Xeon(R) CPU @ 2.2 GHz (manufactured by Intel Corporation, Santa Clara, CA, USA), featuring 4 physical cores and 2 threads per core, providing a total of 8 logical cores. The server has 32 GB of RAM and does not utilize GPUs. All algorithms were implemented in Python. The algorithm has been benchmarked against GD, Adam, and SBGD. These choices are deliberate and well-founded: GD represents the foundational approach for gradient-based optimization and is a natural baseline. Adam, in contrast, is currently the dominant algorithm in machine learning optimization. SBGD was included due to its shared characteristics with MAGD, as both are multi-agent-based and incorporate strategies for agent reduction. However, important differences exist between the two. SBGD is a swarm-based algorithm that relies on inter-agent communication, which introduces additional computational overhead and increases the number of hyperparameters requiring fine-tuning. These hyperparameters are often interdependent, complicating their adjustment and making the overall implementation of SBGD more challenging. By contrast, MAGD avoids inter-agent communication entirely, significantly reducing computational complexity, minimizing hyperparameter requirements, and enhancing implementation simplicity. For MAGD, FMA, and SBGD, the agent updates in each iteration were executed sequentially (without explicit parallelization). However, simultaneous updates of agents were achieved by leveraging the vectorization capabilities of the NumPy library. This approach utilized SIMD (Single Instruction, Multiple Data) operations to optimize memory access and execution time per core, effectively avoiding the overhead associated with Python for loops. The execution time for each algorithm was measured by recording the timestamp at the start and end of the execution using the Python time library. The complete implementation code is provided in the Supplementary Material. ### Mathematical Functions Experiments and Results

The proposed MAGD algorithm was tested on 20 benchmark mathematical functions for optimization. These functions are summarized in Table 1. All functions were evaluated in two dimensions and are non-convex, meaning they have multiple local minima. MAGD was implemented to find the minimum values of these functions and was compared to GD, Adam, and Swarm-Based Gradient Descent (SBGD) [15].