# Faulty Links' Fast Recovery Method Based on Deep Reinforcement Learning
## 3 Faulty-Path Recovery Scheme Based on DDPG-LBBP
### DDPG-LBBP Agent Interacts with the Environment
Based on this backup tag label, the packet redirects traffic from the failed link to the backup

Figure 4: Maximum disjoint backup path network topology. path and directs the phase data packet to the PMU (or PDC). At the PMU (or PDC), the backup tag label is removed from the phase packet, which then continues along the primary path to the PDC. The backup tag label installation process is shown in Figure 5.

The phase data in the WAMS communication network are transmitted along the main path. The OpenFlow switch performs a matching operation on the phase data packet based on the flow table entry. The matching field in the flow table entry matches the source and destination IP addresses of the packet. After a successful match, the packet is processed according to the instructions in the flow table entry. The red node 8 in Figure 5 represents the switch connected to the PDC, and the blue node 4 represents the node connected to the PMU. When a failure occurs in the primary path, the switch upstream of the failed link detects that the current forwarding port is unavailable. This switch then adds a backup tag to the phase packet's header, directing the packet to be transmitted along the backup path. When the phase packet reaches the switch connected to the PMU, the switch strips the backup tag from the packet. ## 4 Experiment and Result Analysis

### Experimental Environment and Parameter Configuration

The experimental hardware environment used high-performance hardware equipment, including an Intel Core i7-13700H 5.0 GHz processor (from Intel Corporation, based in Santa Clara, CA, USA), NVIDIA RTX 4060 graphics card, 16 GB of memory, and a 1 TB solid-state hard disk drive, to ensure high-performance and stable operation of the experiment. The experimental software environment included Ubuntu 18.04 version, Python 3.6.9 version, TensorFlow 1.8.0 version, Gym 0.26.2 version, simulation tools, and controllers, including OpenFlow protocol V1.3, Mininet simulation tool V2.3.0, Ryu controller V4.34, and configured with an SDN-integrated wide-area measurement communication network topology. Among them, the OpenFlow protocol played a pivotal role in ensuring secure communication between the Ryu controller and the OpenFlow switch. By utilizing this protocol, the Ryu controller was able to monitor and gather real-time status information from the network topology, thereby facilitating network management and optimization. The Mininet simulation tool simulated Open Flow switches and network links to create a virtual network environment with real wide-area measurement communication network characteristics. The Ryu controller collected real-time topology operating status information

Figure 5: Backup trail flow table configuration information. in the wide-area measurement communication network, transmitted the status information to the DDPG agent for iterative training of the agent, and provided the data transmission path between nodes in the topology to each OpenFlow switch. The table of software functions and interaction relationships in the experiment is shown in Table 1.

In order to verify the effectiveness of the DDPG-LBBP algorithm proposed in this paper, the experimental process was based on the IEEE 30 and IEEE 57 benchmark power system communication network topologies, which are commonly used in the power system field. The performance of the DDPG-LBBP algorithm was tested, following the simplified methods commonly applied in IEEE benchmark power system communication network research [19]. The communication network topology was defined to match the node locations and interconnections of the IEEE benchmark power system topology.