## 1 Introduction
The first category of methods is based on DMP, such as second-order cone programming (SOCP) [11] and mixed-integer nonlinear programming (MINLP) [12], which handle non-convex constraints through convex relaxation. However, these methods' computational efficiency drops significantly when handling multi-timestep coupled decisions. For instance, in a distribution network with 100 nodes, the solution time of MINLP may exceed 10 min [13], which makes it difficult to meet second-scale reactive power regulation requirements. Deterministic mathematical programming methods rely on accurate grid parameters, which may cause optimization results to deviate from the practical feasible region. Additionally, in the presence of uncertainties, mechanisms such as scenario reduction and chance constraints must be introduced through stochastic programming or robust optimization. For example, the variable scale of two-stage robust optimization increases by a factor of 3-5 [14], significantly increasing the computational burden. The second category employs heuristic intelligent algorithms, such as particle swarm optimization (PSO) [15] and genetic algorithms (GA) [16]. These algorithms do not rely on accurate power system model parameters or assume differentiability or continuity of objective functions, thus enabling them to handle highly non-convex and discrete problems [17]. However, heuristic algorithms struggle to strictly satisfy voltage security constraints and often use penalty functions to address violations. Experiments show that improper setting of penalty coefficients can lead to an increase in the violation rate [18]. The third-generation methods adopt deep reinforcement learning (DRL), such as the Deep Q-Network (DQN) [19] and Proximal Policy Optimization (PPO) [20], to enable end-to-end control policy generation. However, they commonly suffer from an insufficient state-space representation capacity [21]. Existing DRL approaches have also attempted to integrate graph neural networks (GNNs). For example, the Graph-QN architecture proposed in [22] to address grid topology dynamics handles power system topological features but it fails to establish explicit security constraint mechanisms. Early Deep Deterministic Policy Gradient (DDPG) algorithms approximated both the value function and policy using deep neural networks (DNNs), enabling the direct handling of continuous state and action spaces. By integrating the strengths of Q-learning and deterministic policy gradients, DDPG learns optimal policies through interaction with the environment. In robotics, DDPG has achieved notable success in complex control tasks such as bipedal locomotion and robotic arm manipulation [23]. In the power systems domain, DDPG has also been explored to address various control challenges [24]. For example, it has been applied to regulate hybrid distribution transformers [25] and photovoltaic (PV) smart inverters [26] to maintain voltage levels within permissible limits. Furthermore, the Twin Delayed DDPG (TD3) algorithm [27] improves upon DDPG by incorporating a double Q-network mechanism to reduce overestimation bias and enhance value estimation accuracy. While DDPG and TD3 reduce manual tuning and improve training efficiency, their performance remains limited in reactive power and voltage optimization tasks for distribution networks with high penetration of distributed energy resources, particularly in handling state representation and enforcing safety constraints. To address the voltage instability issues caused by the integration of distributed power sources, severe load fluctuations, and the widespread use of power electronic devices, research is being conducted on the optimization of reactive power distribution and voltage regulation strategies to improve the voltage stability and quality of power grids and prevent voltage collapse. Notably, recent studies have made breakthroughs in addressing security constraints. [PERSON] et al.