# Multi-Agent Reinforcement Learning: A Review of Challenges and Applications
## 5 Applications
The algorithm was tested in scenarios with two UAVs and six UAVs and in both cases managed to achieve the optimal configuration. In [53], a multi-agent approach based on a CommNet architecture [42] was proposed to coordinate the operation of the wireless recharging towers of a group of UAVs. The aim of the algorithm was to schedule the UAV to serve, determine how much energy should be delivered, and share the energy between charging towers. Each tower had access to a local photo-voltaic power generator, and the energy was shared to minimizethe purchase of electricity from the energy market, resulting in lower operating costs. The proposed algorithm was tested in an urban simulation consisting of 30 UAVs and four charging towers. It performance improvements over the baseline policies of random sharing and scheduling. Applications of MARL do not consider only the control of UAV fleets. In [54], the problem of a joint top-down active search of multiple objects was addressed. Each detector was an agent, and deep reinforcement learning was used to learn the optimal policy for object localization. Coordination between agents was obtained by the use of a differential message for each agent, which was a function of the action selected and the state of the environment with parameters \(\theta_{m}\), \(m(a,s;\theta_{m})\). The agent-wise Q-function was then defined as  
\[Q:=Q^{(i)}(a^{(i)},m^{(i)},s^{(i)},m^{(-i)};\theta_{a}^{(i)},\theta_{m}^{(i)}) \tag{28}\]  
The agent could control the effect of messages from other agents in its decision making process by the use of a learning gating mechanism. The algorithm was tested with a two-agent implementation on a series of datasets for joint image detection of man-bike, ball-racket, or person-handbag; the joint model achieved good performance using fewer iterations than the single-agent counterpart. Agents managed to \"help\" each other by sending clues about the correlations between object locations in the messages. In [55], a deep multi-agent reinforcement algorithm was developed to regulate the energy exchange between a community of buildings with heterogeneous energy production and storage capabilities. The objective was to reach nearly zero energy community (nZEC) status, which is defined as \"A micro-grid that has distributed generation, storage, delivery, consumption and zero net annual energy balance.\" The buildings were modeled as DRL agents, and the authors proposed the presence of a community monitoring service (CMS) to aggregate data from all the agents and enable cooperation. Each agent used its local energy generation, consumption, and storing states as its states, and the energy balance of the entire community. This information was used to select an optimal action, maximizing the global reward signal given by the CMS, which was the negative of the community energy status  
\[r=-\left(\sum_{i=1}^{n}c(h_{i})-g(h_{i})\right) \tag{29}\]  
where \(c(h_{i})\) is the energy consumed by the i-th house and \(g(h_{i})\) is the energy generated by the \(i\)-th house. This approach was tested in a simulation of a summer and winter setting with up to 10 agents and confronted with some behavioral baselines, including never-share, always share and a random selection of actions. Baselines were outperformed especially in the summer setting. A limitation of this approach is that the learning was conducted using an episodic base approach and thus there is no guarantee that using an online learning approach would lead to the same convergence to an optimum. In [56], a novel multi-agent reinforcement learning algorithm called Equilibrium selection MARL (ES-MARL) to control the energy scheduling of residential microgrid was presented. The microgrid consisted of renewable energy generators (RG) (wind turbines and photo-voltaic), households that demanded energy from the grid and a number of electric vehicles (EV) that could offer or request energy when connected to a recharging station with a Vehicle to Grid (V2G) connectivity.