## 3 Preventive Control Policy Construction
### Construction Method  
The PCMD is divided into three steps, which are data collection, failure diagnosis, and preventive control policy fitting. In the first step of PCMD, it needs to collect data including the voltage of nodes, frequency, the current of nodes, and the robustness for CCF, and it needs also to consider the control cost reduction as a feedback variable. In the second step of PCMD, it needs to detect failure symptoms and predict a CCF propagation sequence from the occurrence of an initial failure. In the third step of PCMD, it uses the deep neural network to maintain a parameterized PCP \(l\)(\(s\),\(w_{l}\)) pursuing the maximum of the infinite horizon discounted expected cumulative reward. Based on the model defined in definition 1, the infinite horizon discounted expected cumulative reward from an initial state \(g_{0}\) is \(E\left[\sum\limits_{i=0}^{\infty}\gamma^{i}r(F_{i})\right]\), \(E\) is the expectation about different trajectories, \(F_{i}\) is a steady state transition function in a trajectory, and \(\gamma\) is the discount factor. In addition, the PCMD needs also  
Figure 3: A finite automaton preventive control model of an ADN. deep neural networks to maintain a parameterized target PCP \(l^{\prime}(s,w_{l^{\prime}})\) and parameterized action-value functions (\(Q(s,l(s,w_{l}),w_{Q})\) and \(Q^{\prime}(s,l^{\prime}(s,w_{l^{\prime}}),w_{Q^{\prime}})\)) for constructing \(l(s,w_{l})\) steadily. Where \(s\)\(\in\)\(S\) is a CCF propagation sequence, and \(s_{i}\) is a CCF propagation sequence in a steady state \(g_{i}\). \(w_{l}\), \(w_{l^{\prime}}\), \(w_{Q}\), and \(w_{Q^{\prime}}\) are the parameters of parameterized functions. According to Definition 1, many possible expected steady states could be reached from the current steady state when a PCP is attempted to block a CCF propagation sequence. The goal of the constructed PCP is to be selected to reach an optimal steady state from the current steady state with minimum cost. Those requirements about the expected steady state are described by an objective function, as shown in Formula (1). \[\min\left(\left(\sum_{i=1}^{|V_{P1}\cup V_{P5}|}W_{Gi}\Delta P_{Gi}\right)+W_{ RP}(1-R_{PF})\right) \tag{1}\]  
where \(V_{P1}\) denotes the set of power generation nodes (distributed generators), and \(V_{P5}\) denotes the set of external nodes (the energy from power transmission lines). \(W_{Gi}\) denotes the weight of the control cost objective, and \(W_{RP}\) denotes the weight of the robustness (against CCF) objective. \(\Delta P_{Gi}\) is defined in Definition 1. \(R_{PF}\) (a discrete quantity) denotes the proportion of failure nodes during the propagation of CCF in an ADN. Formula (1) describes the control cost objective and the robustness (against CCF) objective. Instead of using the Pareto idea, the commonly used weighting method [19] is used to integrate multiple objectives into a single objective function. During the process of blocking a CCF propagation sequence, the safety requirements and the basic physical constraints should be obeyed. Formula (2) describes inequality constraints (including continuous quantities and discrete quantities) corresponding to safety requirements. Formula (3) describes the equality constraint of power flow equations in the PN. \[\left\{ \begin{array}{c}P_{Gi\_min}\leq P_{Gi}\leq P_{Gi\_max},Gi\in V_{P1}\\ Q_{Gi\_min}\leq Q_{Gi\_max},Gi\in V_{P1}\\ I_{Pi}\leq I_{Pi\_max},Pi\in V_{P}\\ v_{i\_min}\leq v_{i}\leq v_{i\_max},i\in V_{P}\\ \Delta P_{Gi\_min}\leq\Delta P_{Gi\_max},Gi\in V_{P1}\\ 0\leq R_{PF}\leq 1\\ fref_{\min}\leq fref\leq fref_{\max}\end{array} \right. \tag{2}\]  
\[\left\{ \begin{array}{c}P_{Gi}-P_{Li}-\sum_{j=1}^{Nuring}|v_{i}||v_{j}||Y_{ ij}|cos\big{(}\alpha_{ij}-\zeta_{i}+\zeta_{j}\big{)}=0\qquad i=1,\ldots, NON_{P}\\ Q_{Gi}-Q_{Li}+\sum_{j=1}^{Nuring}|v_{i}||v_{j}||Y_{ij}|sin\big{(}\alpha_{ij}- \zeta_{i}+\zeta_{j}\big{)}=0\qquad i=1,\ldots,NON_{P}\end{array} \right.