# KGGCN: Knowledge-Guided Graph Convolutional Networks for Distantly Supervised Relation Extraction
## 5 Experimental Setup
### Ablation Study  
In order to analyze the effect of various components of the proposed KGGCN on its performance, we conduct an ablation study on the NYT-FB validation set. We define four variant models with cumulatively removed components, including _KGGCN w/o KG_, _KGGCN w/o LLI_, _KGGCN w/o ALL_ and _KGGCN w TransE_.  
\begin{table}
\begin{tabular}{l c c c c c c c c c c c} \hline \hline  & \multicolumn{4}{c}{**One**} & \multicolumn{4}{c}{**Two**} & \multicolumn{4}{c}{**All**} \\ \cline{2-13}
**Models** & **P@100** & **P@200** & **P@300** & **Mean** & **P@100** & **P@200** & **P@300** & **Mean** & **P@100** & **P@200** & **P@300** & **Mean** \\ \hline PCNN & 73.3 & 64.8 & 56.8 & 65.0 & 70.3 & 67.2 & 63.1 & 66.9 & 72.3 & 69.7 & 64.1 & 68.7 \\ PCNN+ATT & 73.3 & 69.2 & 60.8 & 67.8 & 77.2 & 71.6 & 66.1 & 71.6 & 76.2 & 73.1 & 67.4 & 72.2 \\ BGWA & 78.0 & 71.0 & 63.3 & 70.8 & 81.0 & 73.0 & 64.0 & 72.7 & 82.0 & 75.0 & 72.0 & 76.3 \\ RESIDE & 80.0 & 75.5 & 69.3 & 74.9 & 83.0 & 73.5 & 70.6 & 75.7 & 84.0 & 78.5 & 75.6 & 79.4 \\ \hline
**KGGCN (ours)** & **87.6** & **78.3** & **72.8** & **79.6** & **85.1** & **76.2** & **73.8** & **78.4** & **86.7** & **81.4** & **76.3** & **81.5** \\ \hline \hline \end{tabular}
\end{table}
Table 3: Performances comparison of KGGCN with neural-based baseline models in terms of P@N using different number of sentences in bags on NYT-FB dataset. One, two, All mean the number of sentences randomly selected from a instance bag.  
Specifically, _KGGCN w/o KG_ denotes removing the knowledge attention from the knowledge graph and generating the bag representation by calculating the mean of all sentences in the bag. _KGGCN w/o LU_ denotes removing the knowledge attention from lexical units, and _KGGCN w/o ALL_ denotes removing both the knowledge attention from the knowledge graph and lexical units. _KGGCN w TransE_ denotes replacing the knowledge graph embedding module with the TransE. The experimental results in terms of P@N are shown in Table 4. According to the results, we can observe that when removing different components from KGGCN, the performance of the variant models drops drastically. Particularly, by removing the word-level knowledge attention (i.e., KGGCN w/o LU) and sentence-level attention (i.e., KGGCN w/o KG), the performance decreases 2.5 and 7.2, respectively, in terms of P@N mean for all sentences. When removing both modules above (i.e., KGGCN w/o ALL), the performance of the variant model drops 13% in terms of P@N mean for all sentences. These demonstrate the effectiveness of the prior knowledge from knowledge graph and lexical units. In addition, in order to make an in-depth evaluation of the knowledge graph embedding module of our proposed method, we replace it with TransE to generate the representation of the entities and relations in the knowledge graph (i.e., KGGCN w TransE). The results show 1.4 drops in terms of P@N mean for all sentences. It demonstrates that our knowledge graph embedding module can effectively extract the semantic and structure information, as well as the interaction between entities and relations in the knowledge graph.  
Outperforming these variant models highlights our model's ability to capture sentence-level and bag-level features. All these experimental results demonstrate that the external information from knowledge graph and FrameNet can be the prior knowledge to guide the extraction of textual features, which helps to improve the distantly supervised relation extraction.