  Deriving fast and effectively coordinated control actions remains a grand
challenge affecting the secure and economic operation of today's large-scale
power grid. This paper presents a novel artificial intelligence (AI) based
methodology to achieve multi-objective real-time power grid control for
real-world implementation. State-of-the-art off-policy reinforcement learning
(RL) algorithm, soft actor-critic (SAC) is adopted to train AI agents with
multi-thread offline training and periodic online training for regulating
voltages and transmission losses without violating thermal constraints of
lines. A software prototype was developed and deployed in the control center of
SGCC Jiangsu Electric Power Company that interacts with their Energy Management
System (EMS) every 5 minutes. Massive numerical studies using actual power grid
snapshots in the real-time environment verify the effectiveness of the proposed
approach. Well-trained SAC agents can learn to provide effective and subsecond
control actions in regulating voltage profiles and reducing transmission
losses.
