## 5 Conclusions and Future Work  
To tackle the growing uncertainties of today's power systems with increased penetration of renewable generation and new-type load behavior, this paper presents an innovative control framework using SAC RL agents to provide fast voltage control, line flow control, and transmission loss reduction. Compared to the traditional OPF-based approaches that require full-system information and accurate parameters to conduct mass sensitivity analysis for effective controls, the developed RL-based control framework demonstrates the merits of fast learning from past experiences and the capability to adapt to unforeseen conditions in real time. From comprehensive case studies conducted in the control center, both voltage control and line flow control agents provide effective actions for mitigating security issues, verified in the live operational environment of the East China power system. It is worth mentioning that the SAC agent-based control actions were tested on top of the closed-loop actions from the existing automatic voltage control (AVC) and automatic generation control (AGC) systems, further demonstrating the advantages of the proposed method in dealing with the complex operational scenarios of bulk power systems. The developed method and system offer an effective solution by providing prompt controls to deal with voltage violation and line flow violation issues in a regional power system, caused by the growing uncertainties from generation and load, especially when traditional control systems fail to provide effective controls or they are under maintenance. When designing RL-based agents for specific control tasks in power system operation, several important factors need to be considered:  
1. It is important to have high-fidelity power system simulators to accurately capture the system behavior before and after each control action, providing a reliable environment for training the agents. 2. It is also important to include large-scale representative operating conditions in the power system in the form of samples that cover the feature space more evenly so that RL agents can learn directly from interacting with these samples, especially those with operational risks. 3. Design of reward functions plays an important role in the effectiveness and efficiency of RL agent training in performing specific control tasks. Hyperparameter tuning is always a good practice to ensure better control performance of the agents. The proposed framework will be further enhanced and tested in future work by combining both voltage and line flow control objectives. The goal is to provide autonomous, fast, and effective solutions in real time for mitigating various operating risks simultaneously, including line flow control, transmission loss reduction, and others. A multi-agent approach will also be investigated for coordinated control and decision-making among multiple agents, each focusing on its regions, zones, or objectives to achieve improved overall control performance. **Author Contributions**: Conceptualization, J.L. and J.Z. ; methodology, R.D. and Q.G. ; software, J.L. and R.D. ; validation, Q.G., J.Z. and G.X. ; formal analysis, J.Z. ; investigation, J.L. ; resources, J.L. and J.Z. ; data curation, R.D. ; writing--original draft preparation, J.L. and R.D. ; writing--review and editing, J.Z. and Q.G. ; visualization, G.X. ; supervision, R.D. ; project administration, J.Z. ; funding acquisition, R.D. All authors have read and agreed to the published version of the manuscript. **Funding**: This work is supported by the National Natural Science Foundation of China (U22B6007) and the Fundamental Research Funds for the Central Universities (226-2024-00244). **Institutional Review Board Statement**: Not applicable. **Informed Consent Statement**: Not applicable. **Data Availability Statement**: The data presented in this study are available on request from the corresponding author. The data are not publicly available due to that operational data are not allowed to be shared in the public domain.