# Networked Microgrids: A Review on Configuration, Operation, and Control Strategies
## 2 Networked Microgrids' Configuration
### Formation
#### 2.1.6 Deep Reinforcement Learning-Based Approaches  
Deep reinforcement learning (DRL) has emerged as a potent method across various domains, demonstrating its efficacy in reconfiguring distribution networks into NMGs [53; 54; 55; 56; 57; 58; 59; 60]. DRL utilizes advanced machine learning techniques to dynamically configure and optimize microgrid compositions within larger networks. This involves modeling the system as a Markov Decision Process (MDP), where the formation of microgrids is determined through a sequence of decision-making steps. An agent, representing the microgrid's controller or distribution system operator, interacts with the environment and learns an optimal policy through trial and error, taking actions like selecting energy sources, adjusting network connections, and optimizing parameters to maximize cumulative rewards. These rewards align with objectives such as load pickup [55; 58; 59] and topology feasibility [53; 54]. DRL empowers microgrids to autonomously adapt, integrate renewable energy, and optimize configurations in real time, making them more adaptive, resilient, and efficient contributors to the broader power infrastructure [64]. The crucial aspect of DRL methods lies in their model-free nature, allowing for their application in an online mode. This capability facilitates swift decision making through straightforward numerical calculations, eliminating the necessity for actual power system modeling or the formulation of complex power flow equations.  
DRL-based methods commonly employed in the formation of NMGs encompass Deep Q-Networks (DQN) and multi-agent DQN. DQN employs an artificial neural network to functionally approximate state-action pairs, with Q values trained to derive optimal decisions. In works such as [53; 54; 55; 56; 57], DQN is applied to dynamically configure NMGs, aiming to enhance system resilience by accommodating all loads while considering operational constraints, power flow constraints, and radial topology assumptions. Achieving Dynamic NMGs through DQN hinges on enabling the Q-network to learn appropriate responses within the MDP. Numerous techniques have been explored to enhance the efficiency of DQN learning. Notably, experience replay is employed to avoid overwriting experiences, Epsilon-greedy-based exploration facilitates early-stage convergence, and the utilization of a Double Q-Network proves to be highly effective in mitigating overestimation [53; 54].  
DQNs are executed in a centralized fashion, involving a single centralized agent interacting with the environment at each time step. Consequently, training the centralized DRL on extensive power systems with thousands of nodes would demand an impractically lengthy duration to train a vast number of neural network parameters. To tackle this issue in the context of NMGs, a multi-agent DQN method is introduced in [58; 59; 60], where agents collaborate to expedite prey capture compared to a single agent through the sharing of acquired knowledge.  
During extreme events such as natural disasters or cyber-attacks, the data from the power system encounters challenges related to scarcity and accuracy. These issues can have detrimental effects on DRL training processes, potentially leading to incorrect decisions during the configuration of NMGs. To overcome this challenge, ref. [54] employs Generative Adversarial Networks, GANs, an unsupervised and model-free method capable of automatically extracting data features without the need for labeling. This approach enhances the robustness of the training process by addressing data scarcity and accuracy concerns in challenging scenarios.