# BPDM-GCN: Backup Path Design Method Based on Graph Convolutional Neural Network
## 4 Backup Path Design Method Based on BPDM-GCN
### Data Transmission Path Optimization Algorithm Based on BPDM-GCN
The update process of the online policy GCN network is shown in Equation (5). \[ \begin{array}{l}\
abla_{\theta^{\mu}}G(\mu_{\theta^{\mu}})=grad[Q]*grad[\mu] \\ \approx\frac{1}{N}\sum_{i}\
abla_{a}Q(s,a|Q^{a})|_{s=s_{t},a=(s_{t})}\
abla_{ \theta^{\mu}}\mu(s|\theta^{\mu})|s_{t}\end{array}  \tag{5}\]  
In Equation (5), \(\
abla_{\theta^{\mu}}G(\mu_{\theta^{\mu}})\) represents the policy function \(G\) with respect to parameter \(\theta^{\mu}\). \(grad[Q]\) is the gradient information transmitted in \(Q\) GCN. \(grad[\mu]\) is provided by the online strategy GCN network, which ensures that the online strategy network selects high-yield actions. \(N\) represents the number of randomly sampled samples. Figure 2: DDPG algorithm framework improved based on GCN. \(\
abla_{a}Q\left(s,a|Q^{a}\right)|_{s=s_{t},a=\mu(s_{t})}\) represents the gradient of the \(Q\) function \(Q(s,a|Q^{a})\) with respect to action \(a\) when action \(a\) takes the current \(\mu(s_{t})\) output of the policy network in state \(s_{t}\). \(\
abla_{\theta\mu}\mu(s|\theta^{\mu})|s_{t}\) represents the gradient of the policy network \(\mu(s|\theta^{\mu})\) with respect to parameter \(\theta^{\mu}\) in state \(s_{t}\). (2) Target network update: The target network includes the target policy GCN network and the target \(Q\) GCN network. The target strategy GCN network selects the network topology state \(s_{t+1}\) from the experience replay pool \(D\) and generates \(a_{t+1}=\mu^{\prime}\left(s_{t+1}|\theta^{\mu^{\prime}}\right)\) after training. The target \(Q^{\prime}\) GCN network trains with inputs \(s_{t+1}\) and \(a_{t+1}\) to derive \(Q^{\prime}\left(s_{t+1},\mu^{\prime}\left(s_{t+1}\middle|\theta^{\mu^{\prime}} \right)\middle|\theta^{Q^{\prime}}\right)\), computes the target reward value \(y_{t}\), and subsequently transmits it to the online \(Q\) GCN network. The computation process for \(y_{t}\) is illustrated in Equation (6). \[y_{t}=r_{t}+\gamma Q^{\prime}\left(s_{t+1},\mu^{\prime}\left(s_{t+1}\middle| \theta^{\mu^{\prime}}\right)\middle|\theta^{Q^{\prime}}\right) \tag{6}\]  
In Equation (6), \(r_{t}\) represents the reward value obtained by taking action \(a_{t}\) in state \(s_{t}\). \(\gamma\in[0,1]\) represents the discount factor. \(Q^{\prime}\left(s_{t+1},\mu^{\prime}\left(s_{t+1}\middle|\theta^{\mu^{\prime} }\right)\middle|\theta^{Q^{\prime}}\right)\) represents the value evaluation of action \(\mu^{\prime}\left(s_{t+1}\middle|\theta^{\mu^{\prime}}\right)\) by the target \(Q\) GCN network in state \(s_{t+1}\). Among them, \(\mu^{\prime}\left(s_{t+1}\middle|\theta^{\mu^{\prime}}\right)\) is the action generated by the target policy GCN network based on state \(s_{t+1}\) and parameter \(\theta^{\mu^{\prime}}\). \(\theta^{Q^{\prime}}\) is obtained by regularly copying online \(Q\) GCN network parameter \(\theta^{Q}\). The GCN-based DDPG algorithm and the wide-area measurement system communication network topology facilitate agent training through multiple interactions. The agent dynamically adjusts the output action according to the phasor data transmission delay, bandwidth occupancy, packet loss rate, and reward value in the network topology until the agent training converges. The processes of state mapping, action mapping, and reward value mapping are as follows. (1) State mapping: The link state information and network topology in the wide-area measurement system communication network are input in matrix form. The link state information is represented by a vector matrix \(E\in R^{I\times C}\). \(J\) represents the number of rows in the matrix. Each row represents the feature vector \(s_{e_{ij}}=\left[delay\left(e_{i_{j}}\right),delay\left(v_{i}\right),Occupancy \left(e_{i_{j}}\right),\right.\)\(loss\left(e_{i_{j}}\right)]\) of link \(e_{i_{j}}\) in the wide-area measurement system communication network. \(C\) represents the dimension of the node feature vector. The vector matrix is \(E=\left[s_{e_{1},e_{3},\cdots}s_{e_{ij}}\right]^{T}\). The network topology is represented by the adjacency matrix \(A\in R^{I\times I}\).