# Efficient Task Allocation in Multi-Agent Systems Using Reinforcement Learning and Genetic Algorithm
## 4 A Learning Algorithm
### Genetic Algorithm-Enhanced Proximal Policy Optimization
As complexity increases in the second scenario (Figure 3), GAPPO achieves optimal performance with fewer iterations, while competing algorithms exhibit erratic learning curves. In the third scenario (Figure 4), GAPPO maintains robust performance with rapid convergence and minimal fluctuations, while other algorithms struggle with instability. In the fourth scenario (Figure 5), GAPPO continues to excel, reaching optimal performance quickly despite increased task volume, while alternative algorithms progress slowly and experience frequent regressions. Across all scenarios, GAPPO consistently reduces maximum completion time, demonstrating strong adaptability and efficiency in handling large-scale multi-agent task allocation problems. Its rapid convergence and stability establish it as a highly effective solution for dynamic reinforcement learning tasks in complex environments. In conclusion, GAPPO outperforms other algorithms in scalability and efficiency, making it well suited for real-world applications that require fast, reliable task allocation in complex, dynamic settings. GAPPO demonstrates a significant advantage in energy efficiency for multi-agent task allocation. As shown in Figure 6, it consistently outperforms other algorithms in battery retention. In a setup with 100 agents and 20 tasks, GAPPO maintained 91.9% battery by the end of the process, while A2C, PPO, DQN, and DDPG retained only 52.2-55.5%. This notable gap, observed under identical conditions, underscores GAPPO's superior energy management without compromising task allocation performance. Figure 3: Convergence performance of five algorithms in the environment with 300 agents and 30 tasks. The pink dotted line indicate the value of the optimal solution, which is 1 h.

Figure 2: Convergence performance of five algorithms in the environment with 100 agents and 20 tasks. The pink dotted line indicates the value of the optimal solution, which is 1 h.

Each agent's battery level is initialized at 20, and the reduction in battery power is proportional to the distance traveled. Therefore, the battery consumption is represented as

\[E_{i}^{k+1}=E_{i}^{k}-k\cdot d_{ij}, \tag{19}\]

where \(k\) is a proportionality constant representing the battery consumption rate per unit distance. Figure 4: Convergence performance of five algorithms in the environment with 500 agents and 30 tasks. The pink dotted line indicates the value of the optimal solution, which is 1 h.

Figure 5: Convergence performance of five algorithms in the environment with 500 agents and 50 tasks. The pink dotted line indicates the value of the optimal solution, which is 1 h.

In larger-scale setups with 300 and 500 agents, the initial battery capacity was increased from 20 to 30 units to match the increased agent count. Despite this, GAPPO maintained a clear edge in energy management. Its battery percentage remained consistently high, between 91.7% and 91.8%, while A2C, PPO, DQN, and DDPG ranged from 74.2% to 81.6%. This sustained advantage, even with a higher initial battery capacity and increased task load, highlights GAPPO's scalability and robustness in energy optimization. While the primary focus of GAPPO is to optimize energy efficiency in multi-agent systems, its broader implications must also be considered. One key aspect is the fairness of resource allocation, as reinforcement learning-based policies may inadvertently favor specific agents or regions if not properly designed. Ensuring that GAPPO maintains balanced allocations across all agents is crucial to prevent systemic biases in real-world applications. From a sustainability perspective, our algorithm reduces energy consumption, which is particularly beneficial for large-scale deployments, such as smart grids or autonomous fleets. However, it is important to consider the trade-offs between computational cost and energy savings. Training reinforcement learning models requires significant computational resources, and optimizing this aspect remains an open challenge for sustainable AI development.