## 1 Introduction
The multi-agent deep deterministic policy gradient (MADDPG) algorithm [18] is used to solve Markov games in problems such as swarm intelligence perception [19] and active/reactive power coordinated scheduling [20]. This paper proposes a real-time energy management strategy for MMGs based on multi-agent deep reinforcement learning (MADRL), in which the objectives of the distribution system operator (DSO) are to improve the profit from selling power and improve the smoothness of the power exchange at the PCC, and the objective of each MG is to reduce its own operating cost. During operation, each agent can only obtain local observation information. The DSO agent adjusts the retail electricity price according to the operation state of the MMG, and each MG agent schedules according to the observation information. The decision-making problem of the MMG is modeled as a Markov game and solved by the MADDPG. The cooperation and competition between agents can be fully considered. Multiple agents use centralized training to reach equilibrium as soon as possible, and then they are deployed in a distributed manner to make decisions in real time. In addition, in order to improve training efficiency, this paper proposes the use of the prioritized-experience-replay method [21], in which the experience and priority are stored in a binary tree structure. The major contributions of this paper are summarized as follows:  
1. A real-time energy management strategy is developed for the MMG to optimize the operation of the DSO and individual MGs. The decision-making problem of the MMG is modeled as a Markov game. Unlike a centralized optimization algorithm, the proposed model can take into consideration the interests of each agent and simulate the game process of agents. 2. A prioritized multi-agent deep deterministic policy gradient (PMADDPG) is proposed to solve the Markov game, where the centralized training mechanism is applied to learn a coordination strategy. The trained agents can be deployed in a distributed manner and make decisions in real time. In addition, a prioritized experience replay (PER) strategy is adopted to improve learning efficiency considering that the decision-making process of multi-agent with partial observable information is non-stationary. According to the proposed method, all agents can make decisions based on local observation, which reduces the requirements for the communication network. In addition, the inferences of the neural network can allow making decisions quickly and greatly alleviate the communication burden. It avoids solving complex optimization problems through the time-consuming heuristic algorithms and realizes real-time decision making. 4. Privacy protection. In the execution stage, all agents make decisions based on local information. There is no information exchange between MMGs, and the DSO will not obtain the detailed information inside the MGs but only observe them as a whole. Thus, the privacy of MGs is protected. The remainder of the paper is organized as follows. Section 2 formulates the multi-microgrid energy management problem, including the DSO model and MMGs model. In Section 3, the multi-microgrid energy management problem is converted into a Markov game model with multiple agents. Section 4 proposes the PMADDPG to solve the Markov game model with multiple agents. In Section 5, the numerical simulation results are shown and analyzed. Finally, Section 6 concludes the paper.