# Faulty Links' Fast Recovery Method Based on Deep Reinforcement Learning
## 3 Faulty-Path Recovery Scheme Based on DDPG-LBBP
### DDPG-LBBP Agent Interacts with the Environment
To transmit data, the path \(p_{s,d}\) from the source node \(s\) to the root node \(d\) in the network topology is expressed as \(p_{s,d}=e_{s,i}:e_{i,i+1}:\cdots:e_{j,d}\). The weight of path \(p_{s,d}\) is calculated based on the action value \(a_{w_{i,j}}\) issued by the DDPG-LBBP agent. The specific calculation process of the weight of path \(p_{s,d}\) is shown in Equation (9):

\[d_{p_{s,d}}=\sum_{i=1}^{n}\sum_{j=1}^{n}a_{w_{i,j}}\;i,j,n\in N^{*} \tag{9}\]

Based on the PMU in the WAMS periodically sending the collected phasor information to the PDC, the DDPG-LBBP algorithm proposed in this paper predicts the traffic changes in the network topology and computes the maximum disjoint backup path \(bp_{s,d}\) for each path in the shortest path tree \(T\). The maximum disjoint backup path B is computed by the DDPG-LBBP algorithm in this paper. The DDPG-LBBP algorithm incorporates data transmission delay, link utilization, and packet loss rate into the path calculation process, reducing the likelihood of network congestion caused by the backup path. The steps of the maximum disjoint backup path algorithm are: (1) Traverse the shortest path tree \(T\) in the WAMS communication network topology and initialize the link weights in tree \(T\) to infinity. (2) Traverse all neighbor nodes of the starting node \(s\) in the network topology, select a node \(v\) with the smallest link weight from the unvisited neighbor nodes, modify this node as the current node, and add it to the node set. (3) Repeat step (2) until all nodes have been visited and the shortest path \(bp_{s,d}\) from the starting node \(s\) to the destination node \(d\) has been found. (4) After repeating the above steps, a new shortest path tree is computed which represents the maximum disjoint backup from all switches connected to the PMU to the destination node \(d\) path. The specific implementation process is shown in Algorithm 2. The maximum disjoint backup path network topology is shown in Figure 4.

As shown in Figure 4, the data forwarding path in the communication network topology is represented by lines with arrows, the data forwarding path in the network topology is calculated based on the action value generated by the DDPG-LBBP agent. The black line represents the data transmission path from node \(s\) to destination node \(d\), the red line represents the maximum disjoint backup path calculated for the main path between nodes, and the red node number 7 represents the destination node \(d\) of all nodes. The data transmission process in the network topology is as follows: (1) node 1 to node 7 transmit data through the main path \(p_{1,7}=e_{1,5}::e_{5,6}::e_{6,7}\); (2) when link \(e_{1,5}\) in the path fails, the main path must be quickly switched to the maximum disjoint backup path \(p_{1,7}=e_{1,8}::e_{8,9}::e_{9,6}::e_{6,7}\), which ensures data transmission between node 1 to node 7. ### Backup Path Installation Method Implementation

The DDPG-LBBP algorithm enables rapid recovery of failed links by calculating the maximum disjoint backup paths for the WAMS communication networks. To prevent conflicts between packet forwarding rules based on the backup and main paths, and to meet the high real-time requirements of the WAMS communication networks, this paper designed a backup tag and incorporated it into the backup path installation process. Based on the pre-calculated maximum disjoint backup paths in the WAMS communication network topology, a backup tag was added to the header of the phase packet in the faulty link, guiding the phase packet to complete data transmission along the backup path. For a faulty link in the main path, the phase data from the faulty link can be redirected to the PDC based on the obtained maximum disjoint path, thus resuming the transmission of phase data. To reduce the backup path storage cost at the switch, when a link fails, the switch upstream of the faulty link adds a backup tag label to the header of the phase packet.