# Improving Graph Neural Network Models in Link Prediction Task via A Policy-Based Training Method
## 3 Methodology
### The Training Procedure of the GNN Model with PbTRM in the Link Prediction Task  
Compared to the training procedure of GNN models in the previous studies, the differences in this work are described as follows:  
The randomly sampled unobserved edges are sequentially input into the negative sample selector. The negative sample selector obtains state vectors of the randomly sampled unobserved edges and feeds the state vectors into the policy network. According to the actions output from the policy network, high-quality negative samples are selected from the randomly sampled unobserved edges to compose the negative sample set \(E_{train}^{ng}\). The state vectors and actions of the randomly sampled unobserved edges are recorded to update the parameters \(\theta_{p}\) of the policy network.  
In this work, we set a sampling period in the proposed PbTRM, where each sampling period contains \(T\) training epochs. To calculate the reward \(R\) for the policy network, we need a base value of the prediction performance of the GNN model before the first update of the policy network. Therefore, we construct the negative sample set \(E_{train}^{ngg}\) by randomly sampling from unobserved edges in the first sampling period. The prediction performance of the GNN model, which is evaluated in the first sampling period, is set as the base value. Starting from the second sampling period, the negative sample set \(E_{train}^{ngg}\) is updated by the negative sample selector following Algorithm 1, and the policy network is updated according to the reward, state vectors, and actions in this sampling period.  
The detailed training procedure of the GNN model with PbTRM in the link prediction task is described in Algorithm 2.  
```
1:Input: Network \(G\) and \(T\) value of a sampling period
2:GDPUP: Edge probabilities of edges in \(U-E_{train}^{pos}\)
3:\(1\):afor epoch in \([1:epoch\_num]\):
4:if\(epoch\%T=1\) or \(T=1\): #In the first training epoch of a sampling period
5:if epoch = 1: #In the first training epoch of the first sampling period
6: Obtain the negative sample set \(E_{train}^{ngg}\) by randomly sampling from \(U-E_{train}^{pos}\)
7:else
8: Obtain the negative sample set \(E_{train}^{ngg}\) following Algorithm 1
9: Calculate the edge probabilities of the edges in \(E_{train}^{pos}\) and \(E_{train}^{ngg}\)
10: Calculate the loss function value for the GNN model
11: Update the parameters of the GNN model according to the loss function value
12:if\(epoch\%T=0\) or \(T=1\): #In the last training epoch of a sampling period
13: Evaluate the prediction performance \(\mathbb{P}\) of the GNN model following Equation (5)
14:if epoch = \(T\): #In the last training epoch of the first sampling period
15:\(\mathbb{P}^{pre}=\mathbb{P}\) #Base value for the first update of the policy network
16:else
17:\(\mathbb{P}^{cur}=\mathbb{P}\)
18: Calculate the reward \(R\) for the policy network following Equation (4)
19: Update the parameters \(\theta_{p}\) of the policy network following Equation (8)
20:endif
21:endfor
22: Calculate the edge probabilities of edges in \(U-E_{train}^{pos}\)
```  
**Algorithm 2** The policy-based training method (PbTRM)