## 1 Introduction  
The effort in decarbonizing a power system is undergoing major changes including a large amount of distributed generation (DG) replacing conventional generators and transportation undergoing electrification. Most DG, such as photovoltaic (PV) generation and electric vehicle (EV) charging stations, are connected to a distribution network in a single phase. The high penetration of DG and EV charging stations has resulted in new challenges for maintaining the proper voltage quality in a distribution network. The fluctuating characteristic output patterns of single-phase DG and the irregularity of EV charging patterns [1, 2] aggravated the voltage unbalance. Unbalanced voltages cause heat, vibration, and inefficiency by shortening the lifespan of three-phase transformers and power-electronics-based equipment [3, 4]. Especially at the point of common coupling (PCC) in the distribution network (DN), it is necessary to reduce the voltage unbalance because it is directly connected to the high-voltage stage with a three-phase transformer. Various methods have been proposed to mitigate the voltage unbalance at high PV penetration rates in distribution networks. Power-electronics-based equipment, such as static synchronous compensators or passive devices, such as shunt capacitors, can mitigate the unbalanced voltage by compensating for the reactive power. However, these methods have low economic feasibility because they require additional equipment and maintenance costs. Most strategies proposed for using PV inverters to improve phase unbalance rely on centralized cooperative control to solve the economic problem. For example, [5] presents a bi-level VVO framework for CVR by coordinating the operation of distribution system's legacy voltage control devices and smart inverters to efficiently handle the discrete and continuous control variables. Ref. [6] analyses and compares four of the most relevant smart EV charging controls as follows: (1) active power droop control; (2) reactive power droop control for single-phase EV chargers; (3) Load Balancing control; and (4) Sequence Compensation control (SC), with the aim of reducing the VUF and avoiding under-voltage conditions. In a previous article [7], a method for mitigating voltage unbalance by adjusting reactive power injections from PV systems was presented. By using the Karush-Kuhn-Tucker optimization method, the negative and zero phase components of the voltage can be selectively reduced. In another study [8], the voltage unbalance in the DN was solved using the reactive power compensation capability of the inverter through centralized and decentralized methods. The centralized method solves the voltage unbalance by presenting a mathematically calculated value using a Steinmetz design. However, the process of centralized control requires three steps: (1) receiving measured data through communication; (2) calculating set points based on their optimization problem, which is formulated to obtain some optimal parameters with the predefined objective functions; and (3) sending the set points to each control unit. These procedures are repeated every control time. In the process, the load on the system or the output of renewable energy has already changed. And computation time is accompanied by a correlation between the scale of the power system and the computational load. Thus, centralized controls are difficult to dispatch accurately in real time. On the other hand, the Deep Reinforcement Learning (DRL) method calculates the optimal parameters based on offline analysis results and uses them for the next day's online operation. Moreover, the optimal policies which are composed by neural network are determined by performing power flow calculations on the previous day. The Neural networks (NNs) take little time (less than 1 ms) to derive the output, so NN has an advantage in using real time [9]. This feature can be usefully applied when the fast response in the parameter updates of the control unit is required for online operation.