# Faulty Links' Fast Recovery Method Based on Deep Reinforcement Learning
## 3 Faulty-Path Recovery Scheme Based on DDPG-LBBP
### DDPG-LBBP Agent Interacts with the Environment
For the purposes of this article, parameter \(\xi\) was assigned a value of 0.3, \(\eta\) was set to 0.4, and \(\tau\) was designated as 0.3, reflecting the relative weights given to these network performance metrics in determining the overall reward value. In order to ensure the rapid recovery of faulty links in the communication network of the wide-area measurement system, this paper calculated the link weight value based on the DDPG algorithm improved by the GRU neural network. The specific process is shown in Algorithm 1. ``` Random initialization: parameters \(\theta^{\mu}\), \(\theta^{\mu\prime}\), \(\theta^{Q}\), \(\theta^{Q\prime}\), and \(D\) Input: Link status information in network topology \(s_{t}\)= \(\{delay_{t}(e_{i_{j}}),bandwidth_{e_{i_{j}}}(t),Loss_{e_{i_{j}}}(t)\}\) Output: Link \(e_{i,j}\) weight \(weight_{e_{i,j}}(1)\) (1) For episode = 1, \(M\)do: (2) Initialize \(s_{1}\leftarrow\) Initialize noise strategy \(\aleph\) (3) For\(t=1\), \(T\)do: (4) Select action \(a_{t}=\mu(s_{t}|\theta^{\mu})\) according to the current policy (5) Execute \(a_{t}=\mu(s_{t}|\theta^{\mu})\) to make the SDN controller build phasor data transmission paths (6) Obtain \(r_{t}\) and \(s_{t+1}\) (7) Store transition \((s_{t},a_{t},r_{t},s_{t+1})\) in D (8) Sample a random minibatch of \(N*(s_{t},a_{t},r_{t},s_{t+1})\) from D (9) Calculate target return value \(y_{t}\), \(y_{t}=r_{t}+\gamma Q^{\prime}\Big{(}s_{t+1},\mu^{\prime}\Big{(}s_{t+1}\Big{|} \theta^{\mu^{\prime}}\Big{)}\Big{|}\theta^{Q^{\prime}}\Big{)}\) (10) Update the critic GRU network and update \(s_{t+1}\), the network parameters, by minimizing the error through \(L=\frac{1}{N}\!\sum_{i}\big{(}y_{t}-Q\big{(}s_{t},a_{t}|\theta^{Q}\big{)} \big{)}^{2}\) (11) Update the parameters of the actor GRU network using the critic GRU network \(\
abla_{\theta^{\mu}}G(\mu_{\theta^{\mu}})\approx\frac{1}{N}\!\sum_{i} \
abla_{a}Q\left(s,a|Q^{a}\right)\big{|}_{s=s_{t},a=\mu(s_{t})}\
abla_{\theta ^{\mu}}\mu(s|\theta^{\mu})|s_{t}\) (12) Update the target network using \(\theta^{Q^{\prime}}\leftarrow\tau\theta^{Q}+(1-\tau)\theta^{Q^{\prime}}\) and \(\theta^{\mu\prime}\leftarrow\tau\theta^{\mu}+(1 data for future training iterations. Lines 9 through 12 randomly select training samples from the experience replay buffer D to facilitate the training of both critic GRU network and actor GRU network. This approach ensures that the networks are updated based on a diverse set of experiences, enhancing their generalization capabilities. After several rounds of iteration, the improved DDPG algorithm based on the GRU neural network converges, and finally, the link weight value \(weight_{e_{i,j}}\) in the wide-area measurement system communication network is output. ### Method for Implementing Maximum Disjoint Backup Paths

Due to the physical layout limitations of nodes and links in the communication network topology of the wide-area measurement system, there are overlapping nodes and links in the main path and backup path, causing link congestion on the backup path for fault recovery and prolonging the faulty link. The recovery time may even cause the secondary link to fail. In order to solve the problem of secondary network link failure caused by overlapping links and nodes in the main path and backup path, this paper designed a maximum disjoint backup path that avoids the overlapping of nodes and links in the main path and backup path. According to Section 3.2, the new link weight in the network topology is calculated in Section 1, which is used to restore the faulty link in the network topology, improve the reliability of the backup path, and ensure that the vector data can be quickly restored and transmitted according to the backup path. In the communication network of the wide-area measurement system, based on the characteristics of all PMUs transmitting phasor data to the PDC, the Dijkstra algorithm is used to construct the shortest path tree \(T\) with the switch connected to the PDC as the destination node \(d\). The source node \(s\) follows the shortest path to the root node \(d\).