# KGGCN: Knowledge-Guided Graph Convolutional Networks for Distantly Supervised Relation Extraction
## 5 Experimental Setup
### Comparison with Baselines  
To verify the effectiveness of our model, we compare it against the baselines on the NYT-FB and GDS datasets. Since GDS is a recently proposed dataset, we only compare our model with neural-based methods on this dataset. Figure 4 summarizes the comparison results in terms of PR curves on the datasets. From the comparison results illustrated in Figure 4, we can observe that:  
(1) The neural-based methods significantly outperform the feature based methods Mintz and MultiR on the NYT-FB dataset. The results demonstrate that the human-designed features are limited in relation extraction, and the use of NLP tools to generate features often leads to the propagation and accumulation of errors. It also demonstrates the robustness and effectiveness of the neural models for relation extraction;  
(2) The BGWA and PCNN+ATT outperform the PCNN model over the entire range of recall on both datasets, which indicates the attention mechanism is helpful for distantly supervised relation extraction. The higher performance of the RESIDE model over BGWA and PCNN+ATT demonstrates that the additional side information (relation alias and entity types) from knowledge graphs helps in improving the performance of the model;  
(3) Our proposed model KGGCN achieves the best performance compared with all the baselines on both datasets. Especially in contrast to feature-based methods, our model increases by more than 40% when the recall is larger than 0.25. Compared with other neural-based methods, our model also has a significant improvement. All these demonstrate that the prior knowledge from FrameNet and knowledge graphs can effectively guided the encoding of the sentence-level and bag-level features;  
(4) The overall performance of the neural-based models on GDS dataset is better than that on NYT-FB dataset. This is because that the NYT-FB dataset contains many negative instance bags. All sentences in these bags are wrongly labeled, which is very noisy for \begin{table}
\begin{tabular}{l l l} \hline \hline
**Para** & **Description** & **Value** \\ \hline \(d_{w}\) & Word\(\backslash\)entity\(\backslash\)relation Embedding & 50 \\ \(d_{p}\) & Position Embedding & 5 \\ \(\beta\) & Regularization Factor & 0.001 \\ \(\lambda\) & Learning Rate & 0.5 \\ \(p\) & Dropout probability & 0.5 \\ \(L\) & GCN Layers & 2 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Hyper-parameters used in our experiments. Figure 4: Performance comparison for proposed model and previous baselines in terms of precision-recall curves. (**a**) Comparison of precision-recall curves on NYT-FB dataset. (**b**) Comparison of precision-recall curves on GDS dataset. relation extraction. However, the GDS dataset is constructed based on the at-least-one restriction, i.e., each bag contains at least one sentence that exactly expresses the relation of the corresponding entity pair. Moreover, the GDS dataset only has 5 relation types, which is easier for classification. For the proposed model and the neural-based methods, we also evaluate the performances of them in terms of P@N on the NYT-FB dataset. Following the experiment settings in the previous works [26], we randomly select one, two, and all sentences for each entity pair to form the instance bags. We evaluate the P@N of the models on each circumstance. From Table 3, we can observe that:  
(1) The models with attention mechanism (PCNN+ATT, BGWA, RESIDE, KGGCN) achieve better performance than the PCNN (without attention mechanism) on each case. The training sentences generated by using the distant supervision may be wrongly labeled, because not all sentences contain the entity pair exactly express the corresponding textual relations, and not each sentence contributes equally for the encoding of a bag.