# Faulty Links' Fast Recovery Method Based on Deep Reinforcement Learning
## 3 Faulty-Path Recovery Scheme Based on DDPG-LBBP
### DDPG-LBBP Agent Interacts with the Environment
This paper referred to the PDC and PMU layout algorithms in [20; 21] to determine the positions of the PMU and PDC in the IEEE 30 and IEEE 57 benchmark power systems, as shown in Figure 6. In order to simulate realistic scenarios more vividly and conduct a comprehensive evaluation of the DDPG-LBBP algorithm's performance, based on the results of preliminary pre-experiments, we regulated the transmission rate of phasor data in the WAMS network to set different levels of data traffic intensity. Specifically, the traffic intensity was set within the range of 20-80% of the link's maximum capacity. This range encompasses the typical operating conditions of the actual power system communication network, from low-load to relatively high-load scenarios. The bandwidth of each link was set to 1 Gbps, with a link delay defined as 1 ms/200 km. The maximum number of flow rules in the OpenFlow switch was capped at 1000. The Iperf tool was employed to simulate and generate phasor data flow within the WAMS network topology [22] and to compute the recovery delay of the failed components. In Figure 6, the red node represents the switch connected to the PDC, and the green node represents the node connected to the PMU. Among them, Figure 5(a) shows the IEEE 30 benchmark power system communication network topology, where a PDC was deployed at node 17 and 10 PMUs were installed at nodes [1; 2; 6; 9; 10; 12; 15; 19; 25; 27]. In Figure 5(b), the IEEE 57 benchmark power system communication network topology is shown, with a PDC deployed at node 22 and 10 PMUs installed at nodes {1, 4, 8, 10, 20, 21, 24, 28, 31, 32, 36, 41, 44, 46, 49, 52, 55, 57}. \begin{table}
\begin{tabular}{c c c} \hline \hline
**Software Name** & **Function in the Experiment** & **Interaction with Other Components** \\ \hline Ubuntu 18.04 & Offered stable runtime, managed resources, eased software installation & Supported Python, enabled SDN component comms, resource-supply Mininet \\ Python 3.6.9 & Allowed development of algorithms and processed data & Invoked other libraries and interacted with Mininet and Ryu \\ TensorFlow 1.8.0 & Supported the training of deep learning models & Built and optimized models within Python \\ Gym 0.26.2 & Provided a simulation environment for reinforcement learning & Facilitated interaction during algorithm training \\ OpenFlow protocol V1.3 & Enabled communication between the controller and switches and configuration & Managed flow tables \\ Mininet & Simulated the network environment & Served as a test scenario for the algorithm \\ simulation & Collected network information and & Communicated with switches and provided \\  & guided data forwarding & data for the algorithm \\ \hline \hline \end{tabular}
\end{table} 
Table 1: The table of software functions and interaction relationships. ### Comparative Evaluation of DDPG-LBBP

In order to evaluate the performance of the proposed faulty-link fast recovery method, the DDPG-LBBP algorithm, the experiment compared DDPG-LBBP with other current representative faulty-link recovery methods. The algorithms compared included the (1+2\(\epsilon\))-BPCA backup path recovery algorithm ((1+2\(\epsilon\))-Approximation Backup Path Construction Algorithm) [18], FFRLI fault recovery method (Fast Fault Recovery Scheme Based on Link Importance) [17] and LIR faulty-link recovery method (Low Interruption Ratio Link Fault Recovery Scheme) [16]. The main comparison contents included the faulty-link recovery delay, packet loss rate after the faulty link was restored, and faulty-link recovery success rate. ### Faulty-Link Recovery Delay

The fault recovery delay of the DDPG-LBBP algorithm in the power system communication network was tested on the IEEE 30 and IEEE 57 benchmarks. The faulty-link recovery delay was taken as the key indicator for experimental comparison. The faulty-link recovery delay was determined by the switch port's switching time and the transmission delay of the phasor data packet to the destination node \(d\) along the backup path.