## 3 Latest Applications in Distribution Systems
### Volt/VAR Control  
Volt/VAR control (VVC), as another core module in DMSs, aims to minimize voltage violation and reduce power losses by optimizing voltage profiles in distribution systems [69]. It determines the best available combination of settings from controllable devices, including on-load tap changers (OLTCs), capacitor banks (CBs), inverter-connected photovoltaics (PVs), voltage regulators, and dispatchable electric vehicles (EVs). Conventionally, model-based optimization methods such as conic relaxation and mixed integer linear programming are adopted. However, not only does DER penetration cause variations in the network topology, but it also introduces intermittency and uncertainty in DER generation [46]. These changes might cause sharp fluctuations in voltage, which lead to serious overvoltage/undervoltage issues.  
Since 2019, the development of VVC has centered around data-driven approaches, with a particular emphasis on DRL-based adaptive decision-making. Instead of depending on accurate power flow models to formulate an optimization model for VVC, decision making agent(s) are represented as NNs and trained to determine the control variables in a data-driven adaptive manner. This framework allows for sample-efficient training and effective online exploration of the state-action space, which enhances the adaptability of the strategy to capture the stochastic behaviors of DER generation. The DRL algorithms for VVC methods evolve from deep Q-network (DQN) [46], deep deterministic policy gradient (DDPG) [47], twin delayed deep deterministic (MATD3) policy gradient [48], PPO [49], and SAC [50]. This topic gradually migrates to two branches: safe DRL [51; 52] for satisfying inequality constraints and robust DRL against modeling errors of distribution networks [53]. Safe DRL seeks to feasible control actions within required constraints towards actions or states, since certain control actions violating these constraints might cause equipment damage and undermine the operating reliability. In [53], an adversarial MDP (AMDP), which utilizes an adversarial agent representing modeling errors, is formulated. Furthermore, a joint adversarial SAC algorithm is proposed, which enhances the robustness through offline training and guarantees an efficient convergence of the training process. Another recent development in DRL is the use of meta-learning, which trains a model to learn how to learn. This approach allows the model to quickly adapt to new operating environments; see a paradigm in [70] for power system emergency control.  
Beyond the centralized VVC, multi-agent DRL, featuring \"centralized/decentralized training, decentralized implementation\", is developed for cooperative control by multiple agents to learn optimal policies that maximize their individual and collective rewards, meanwhile preserving privacy. Among them, the attention mechanism is embedded to weigh the impact of other neighboring agents during offline training [54]. Reference [49] adopts a GCN which captures the known topology-based relationships between neighboring nodes in a PPO-based DRL framework. Furthermore, the safety of battery energy storage systems (BESSs) is often overlooked in MARL-based VVC methods, and an agent's actions that cause excessive charging or discharging of the BESS shorten the lifespan. Thus, ref. [48] proposes a physics-based shielding mechanism in PV/battery energy storage systems.