## 3 Preventive Control Policy Construction
### Convergence  
In the PCMD, it is necessary to ensure that the construction process converges and the PCP against CCF generated from the construction process exists. These requirements are guaranteed by the two step parameter's adjustments. In the first step, the convergence of the construction process could be guaranteed by adjusting the parameters in fitted functions (\(l(s,w_{l})\), \(l^{\prime}(s,w_{l^{\prime}})\), \(Q(s,l(s,w_{l}),w_{Q})\), and \(Q^{\prime}(s,l^{\prime}(s,w_{l^{\prime}}),w_{Q^{\prime}})\)). In the second step, the existence of the PCP against CCF could be guaranteed by adjusting additionally the parameters in the reward \(r\). The parameters fitted functions need to be adjusted, including the learning rate \(\delta\), the initial weights of fitted functions (\(l(s,w_{l})\), \(l^{\prime}(s,w_{l^{\prime}})\), \(Q(s,l(s,w_{l}),w_{Q})\), and \(Q^{\prime}(s,l^{\prime}(s,w_{l^{\prime}}),w_{Q^{\prime}})\)), the parameter \(\alpha\), and other parameters of fitted functions based on deep neural networks. **Theorem 1**. : _Given the equation \(w=\varphi(w)\), if its derivative \(\varphi^{\prime}(w)\) is continuous on a closed interval [\(w_{1}\), \(w_{2}\)], when \(w\!\in\!\! [w_{1},w_{2}]\), \(\varphi(w)\!\in\!\! [w_{1},w_{2}]\), there exists a constant \(0<z<1\), such that when \(w\!\in\!\! [w_{1},w_{2}]\), \(\|\varphi^{\prime}(w)\|\leq z<1\), the following results hold._  
1. \(\varphi(w)\) has a unique fixed point \(w_{\perp}\) on the closed interval \([w_{1},w_{2}]\). 2. For any initial value \(w_{0}\!\in\!\! [w_{1},w_{2}]\), the iterative scheme \(w_{k+1}=\phi(w_{k})\) (\(k=0,\,1,\,2,\,\ldots\) ) converges, and \(\lim\limits_{k\to\infty}w_{k}=w_{\perp}\). 3. For the sequence \(\{w_{k}\}_{k=0}^{\infty}\), there exists an asymptotic error estimation shown in Formula (12). \[\lim\limits_{k\to\infty}\frac{w_{k}-w_{\perp}}{w_{k-1}-w_{\perp}}=\phi^{\prime }(w_{\perp})\] (12) The process of fitting \(l_{\perp}(s)\) is actually a fixed-point iterative scheme, and the weight \(w_{l}\) of the fitted PCP \(l(s,w_{l})\) is obtained by solving the equation \(w_{l}=\varphi(w_{l})\). The iterative function \(\varphi\) is defined as shown in Equation (13). \[\phi(w_{l})=w_{l}+\delta\left(\frac{1}{N_{5}}\!\sum_{i=1}^{N_{5}}\left(\
abla _{a}Q(s_{i},l(s_{i},w_{l}),w_{Q})\right)\left(\
abla_{w_{l}}l(s_{i},w_{l}) \right)\right)\] (13) where \(N_{5}\) is the size of a trajectory data sample from a data pool, \(s_{i}\) is a CCF sequence in a steady state \(g_{i}\). If the adjustments of the learning rate \(\delta\) and the initial weight of the fitted function \(l(s,w_{l})\) meet the preconditions of Theorem 1, then the weight sequence of the \(l(s,w_{l})\) converges to the solution \(w_{l}^{(\perp)}\) of the equation \(w_{l}=\varphi(w_{l})\). The following analysis then explains how the initial weights of the fitted functions (\(l^{\prime}(s,w_{l^{\prime}})\), \(Q(s,l(s,w_{l}),w_{Q})\) and \(Q^{\prime}(s,l^{\prime}(s,w_{l^{\prime}}),w_{Q^{\prime}})\)) are adjusted. For example, due to the weights update \(w_{Q^{\prime}}^{(k+1)}=\alpha w_{Q}^{(u)}+(1-\alpha)w_{Q^{\prime}}^{(k)}\) at step \(k\), there is an error equation shown in Equation (14). \[w_{Q^{\prime}}^{(k+1)}-w_{Q}^{(\perp)}=\alpha\Big{(}w_{Q}^{(u)}-w_{Q}^{(\perp) }\Big{)}+(1-\alpha)\Big{(}w_{Q^{\prime}}^{(k)}-w_{Q}^{(\perp)}\Big{)} \tag{14}\]  
where \(w_{Q^{\prime}}^{(k)}\) denotes the weight of the \(Q^{\prime}\)(s,\(J\)(s,\(w_{T}\)),\(w_{Q^{\prime}}\)) at step \(k\). \(w_{Q}^{(u)}\) denotes the weight of the \(Q\)(s,\(I\)(s,\(w_{I}\)),\(w_{Q}\)) at step \(u\), and \(u\geq k\). \(w_{Q}^{(\perp)}\) denotes the true value of the weights \(w_{Q}\) and \(w_{Q^{\prime}}\). An inequation is obtained by taking the norm of the elements in Equation (14), as shown in Formula (15). \[\left\|w_{Q^{\prime}}^{(k+1)}-w_{Q}^{(\perp)}\right\|\leq\alpha\Big{\|}w_{Q}^ {(u)}-w_{Q^{\prime}}^{(k)}\Big{\|}+\left\|w_{Q^{\prime}}^{(k)}-w_{Q}^{(\perp)}\right\| \tag{15}\]  
where \(0<\alpha<1\).