# Efficient Task Allocation in Multi-Agent Systems Using Reinforcement Learning and Genetic Algorithm
## 4 A Learning Algorithm
### Genetic Algorithm-Enhanced Proximal Policy Optimization
The results demonstrate that GAPPO consistently outperformed other representative algorithms, achieving efficient task allocations and maintaining robust performance across diverse conditions. To provide additional insights into the algorithm's effectiveness, a bar chart was employed to compare energy consumption across the scenarios. The findings highlight GAPPO's capability to optimize task allocation while ensuring effective resource utilization. Overall, the evaluation underscores the practicality of GAPPO in addressing complex task allocation problems in multi-agent systems, further validating its potential to meet diverse system requirements. ### Task Allocation

This subsection focuses on evaluating the effectiveness of our proposed algorithm in environment. The GAPPO algorithm is compared with the following representative algorithms. 1. Advantage Actor-Critic (A2C) [41]: The A2C algorithm optimizes decision-making by integrating policy and value functions. The advantage function \(A(s_{t},a_{t})\) is expressed as \[A(s_{t},a_{t})=Q(s_{t},a_{t})-V(s_{t}),\] (16) where \(Q(s_{t},a_{t})\) denotes the expected return from action \(a_{t}\) in state \(s_{t}\), and \(V(s_{t})\) provides the baseline value of \(s_{t}\). This structure enhances stability in learning by reducing gradient variance. 2. Proximal Policy Optimization (PPO) [42]: PPO is an on-policy algorithm designed to balance exploration and exploitation by constraining policy updates, which limits excessive updates and enhances stability and sample efficiency in learning. 3. Deep Q-Network (DQN) [43]: DQN is a value-based reinforcement learning algorithm that approximates the optimal action-value function \(Q(s,a)\) using deep neural networks. Leveraging experience replay and a target network, DQN stabilizes learning and updates the Q-values as \[\delta=r_{t}+\gamma\max_{a^{\prime}}Q(s_{t+1},a^{\prime})-Q(s_{t},a_{t}),\] (17) \[Q(s_{t},a_{t})\gets Q(s_{t},a_{t})+\alpha\,\delta,\] where \(\alpha\) is the learning rate, \(r_{t}\) is the reward, and \(\gamma\) is the discount factor. 4. Deep Deterministic Policy Gradient (DDPG) [44]: DDPG is an off-policy algorithm that addresses continuous action spaces by combining an Actor-Critic framework with deterministic policy gradients. The actor network directly optimizes the policy, while the critic network evaluates the action-value function, with the policy gradient update given by \[\
abla_{\theta}J\approx\mathbb{E}_{s_{t}\sim\rho^{n}}\left[\
abla_{a}Q(s_{t}, a|\theta^{Q})\
abla_{\theta}\mu(s_{t}|\theta^{\mu})\right],\] (18) where \(Q(s,a|\theta^{Q})\) is the action-value function, \(\mu(s_{t}|\theta^{\mu})\) is the policy, and \(\theta^{\mu}\) and \(\theta^{Q}\) are the parameters of the actor and critic networks, respectively. Four scenarios are examined: (i) 100 agents are allocated to 20 tasks; (ii) 300 agents are allocated to 30 tasks; (iii) 500 agents are allocated to 30 tasks; (iv) 500 agents are allocated to 50 tasks. Monte Carlo simulations are conducted for each scenario, with results generated from over 100 runs per scenario. In each simulation trial, agents are randomly allocated, with an initial random allocation conducted for each agent at the starting moment, and the maximum number of iterations is set to 200 for the first scenario, while it is set to 300 for the subsequent three scenarios. The time unit \(\Delta t\) is defined as 1 hour (h) across all scenarios. To evaluate reinforcement learning algorithms, we compare GAPPO, A2C, PPO, DQN, and DDPG across four increasingly complex scenarios, where agent capacity and task load are balanced, and the optimal completion time remains 1 unit time. The results demonstrate GAPPO's superior performance, rapid convergence, and stability in large-scale task allocation. In the first scenario (Figure 2), GAPPO quickly minimizes completion time and stabilizes at a low value, whereas other algorithms show slower learning and fluctuations.