# Improving Graph Neural Network Models in Link Prediction Task via A Policy-Based Training Method
###### Abstract  
Graph neural network (GNN), as a widely used deep learning model in processing graph-structured data, has attracted numerous studies to apply it in the link prediction task. In these studies, observed edges in a network are utilized as positive samples, and unobserved edges are randomly sampled as negative ones. However, there are problems in randomly sampling unobserved edges as negative samples. First, some unobserved edges are missing edges that are existing edges in the network. Second, some unobserved edges can be easily distinguished from the observed edges, which cannot contribute sufficiently to the prediction task. Therefore, using the randomly sampled unobserved edges directly as negative samples is difficult to make GNN models achieve satisfactory prediction performance in the link prediction task. To address this issue, we propose a policy-based training method (PbTRM) to improve the quality of negative samples. In the proposed PbTRM, a negative sample selector generates the state vectors of the randomly sampled unobserved edges and determines whether to select them as negative samples. We perform experiments with three GNN models on two standard datasets. The results show that the proposed PbTRM can enhance the prediction performance of GNN models in the link prediction task. graph neural network; link prediction; deep learning; reinforcement learning +
Footnote †: journal: Medical  
negative samples, which harm the prediction performance of GNN models. In the second case, some of the randomly sampled unobserved edges can be easily distinguished from the observed edges by GNN models. Choosing such easily distinguishable unobserved edges as negative samples results in easy negative samples, which are insufficient for the training of GNN models. Because of the aforementioned two cases, it is requisite to select high-quality negative samples from unobserved edges for the training of GNN models to enhance their prediction performance in the link prediction task. In this work, a policy-based training method (PbTRM) is proposed to improve the negative sampling procedure of GNN models in the link prediction task. The proposed PbTRM still utilizes observed edges as positive samples. The difference from the previous studies is that we design a negative sample selector to select high-quality negative samples from randomly sampled unobserved edges. In more detail, a policy network is employed as the core module of the negative sample selector. For each randomly sampled unobserved edge, the negative sample selector obtains its state vector, which is the input to the policy network. Subsequently, the policy network outputs an action for each randomly sampled unobserved edge, which determines whether to select it as a negative sample. Extending the Policy Gradient [14], we use the change in the prediction performance of the GNN model as the reward for the policy network, where the reward of the improved prediction performance is positive. The parameters of the policy network are updated after finishing a sampling period. Finally, we illustrate the effectiveness of the proposed PbTRM with three GNN models, including GCN [1], GraphSAGE [15], and GAT [16]. The experimental results demonstrate that the proposed PbTRM enhances the prediction performance of these GNN models on two standard datasets, i.e., Cora [17] and CiteSeer [17], in the link prediction task. In summary, the contributions of this work are as follows:  
* We introduce a negative sample selector, which employs a policy network as its core module, to reduce the impact of existing false and easy negative samples in the training set for the GNN models in the link prediction task. * Armed with the negative sample selector, we design a policy-based training method (PbTRM), which generates the negative samples through the negative sample selector in each sampling period and updates the parameters of the policy network after finishing a sampling period.