  Planning future operational scenarios of bulk power systems that meet
security and economic constraints typically requires intensive labor efforts in
performing massive simulations. To automate this process and relieve engineers'
burden, a novel multi-stage control approach is presented in this paper to
train centralized and decentralized reinforcement learning agents that can
automatically adjust grid controllers for regulating transmission line flows at
normal condition and under contingencies. The power grid flow control problem
is formulated as Markov Decision Process (MDP). At stage one, centralized soft
actor-critic (SAC) agent is trained to control generator active power outputs
in a wide area to control transmission line flows against specified security
limits. If line overloading issues remain unresolved, stage two is used to
train decentralized SAC agent via load throw-over at local substations. The
effectiveness of the proposed approach is verified on a series of actual
planning cases used for operating the power grid of SGCC Zhejiang Electric
Power Company.
