# A Deep Reinforcement Learning Approach for Microgrid Energy Transmission Dispatching
The traditional research usually ignores the influence of line loss, which has some deviation in Hierarchical application. Therefore, in order to solve this problem, a comprehensive optimization dispatching model of microgrid energy considering line loss is proposed in this paper. According to the characteristics of microgrids, a multi-time Markov decision model was constructed, and a Hierarchical deep Q-network (HDQN) method was designed. The optimal decision estimation of discrete variables and continuous variables was realized through a neural network, so as to obtain the approximate optimal dispatching strategy, which involves complicated transmission energy allocation and battery charging/discharging management. In essence, the microgrid energy transmission dispatching problem studied in this paper is a constrained non-linear programming problem. However, the current research mainly focuses on the overall dispatching of energy and lacks in-depth analysis and research on the energy distribution of each power consumption unit; that is to say, it is difficult to allocate the power of each line. Considering the relationship between power and voltage/current, the microgrid optimal transmission dispatching problem studied in this paper is a special nonlinear programming problem, that is, a quadratic programming problem with quadratic constraints. Therefore, in this paper, power flow control is included in the behavioral reward as the goal, to form a microgrid power controller based on deep reinforcement learning, in which the adjustable resources of load, diesel generator and energy storage device are time-varying. The paper adopts a two-stage control mode. Firstly, the output power of the system should meet the load demand of the microgrid. Secondly, the adjustability and continuity of the power output or input are ensured when charging/discharging the energy storage. Finally, in the case of considering line loss, the power on all lines and the actions of battery charging/discharging are controlled to ensure the dynamic balance among renewable energy, energy storage, diesel generators and loads, so as to ensure the optimal steady state operation of the microgrid. Considering that the energy supply of the microgrid is provided by non-dispatchable energy sources (e.g., photovoltaic and wind power) and dispatchable energy sources (e.g., diesel generation and battery), its energy dispatching optimization is to optimize power generation output and distribution line flow by regulating the supply of schedulable energy and reducing the operation cost of the microgrid. Different from the high-voltage transmission system, the distinct low-voltage power supply characteristics of a microgrid result in significantly higher power losses as the line lengthens, due to the increased current levels required for transmitting the same amount of power. Therefore, the optimal dispatching of energy in the microgrid should consider the power loss on the distribution line. However, most of the existing research does not consider the power loss of the transmission line, making the operation cost predicted by various models higher. Therefore, taking into account the power loss of distribution line, this paper reduces the energy loss and operation cost of the microgrid by adjusting the transmission power of each line. This paper is organized into six sections. Section 1 is a general introduction. Section 2 introduces the work related to the optimal dispatching of microgrids. Section 3 presents the energy transmission dispatching problem in microgrid systems. Section 4 proposes the deep reinforcement learning approach model and framework. Section 5 introduces the experimental results and discusses them through case studies. The conclusion is drawn in Section 6.