# KGGCN: Knowledge-Guided Graph Convolutional Networks for Distantly Supervised Relation Extraction
## 5 Experimental Setup  
### Dataset and Evaluation Metrics  
The datasets of our experiment contain two parts, knowledge graphs and text corpus. We use FB60K as KG to learn the representation of entities and KG relations. The FB60K is extracted from Freebase (FB) and extended from the dataset developed by [PERSON] et al. [24]. There are 1324 relations, 69,512 entities and 335,350 facts in this dataset. We adopt two widely used datasets of distantly supervised relation extraction as text corpus to demonstrate the effectiveness of our method and baselines. They are NYT-FB [24] and GDS [40] datasets, where the statistical comparison of them are illustrated in Table 1.  
NYT-FB: The NYT-FB dataset is developed by [PERSON] et al. [24], which is constructed by aligning the New York Times (NYT) corpus with Freebase facts. The association between the NYT and FB is built by performing a string match between entity mentions in NYT and canonical names of entities in FB. The entity mentions are fired by using the Stanford named entity recognizer. NYT-FB is a standard benchmark for distantly supervised relation extraction in most of the previous works [26,40], which contains 52 predefined relation types and a null class \(NA\) relation (no relation between two entities). The most common relations in this dataset are _location_, _nationality_, _capital_, _place_lived_, and _neighborhood_of_. The training instances are obtained by aligning the sentences from the NYT corpus of years 2005-2006. The test instances are obtained by aligning sentences from 2007. There are 570,088 sentences, 291,699 entity pairs in the training set, and 172,488 sentences, 96,678 entity pairs in the testing set. Since this dataset does not have a validation set, we split the training set into 80% for training, and 20% for validation. This dataset is available at: [[https://drive.google.com/file/d/1](https://drive.google.com/file/d/1) UD86c_6O_NSBn2 DDYirk6 ygaHy_fTL-hN/view?usp=sharing]([https://drive.google.com/file/d/1](https://drive.google.com/file/d/1) UD86c_6O_NSBn2 DDYirk6 ygaHy_fTL-hN/view?usp=sharing), accessed on 20 May 2021.  
GDS: The Google distant supervision (GDS) dataset is developed by [PERSON] et al. [40], which is extended from the Google relation extraction corpus. There are 5 relation types, including _perGraduatedInstitution_, _perHasDegree_, _perPlaceOfBirth_, _perPlaceOfDeath_, and a \(NA\) relation. Each instance bag in this dataset is guaranteed to contain at least one sentence which expresses the relation type assigned to that instance bag, which alleviates the noise in distant supervision setting. This makes automatic evaluation more reliable. The dataset is divided into three parts, 60% for training, 10% for validation, and 30% for testing. There are 11,297 sentences and 6498 entity pairs in the training set, 1864 sentences and 1082 entity pairs in the validation set, and 5663 sentences and 3247 entity pairs in the testing set. This dataset is available at: [[https://drive.google.com/file/d/1](https://drive.google.com/file/d/1) UMS4 EmWv5 SWXfaSl_ZC4 DC73 dk3 JyHeq/view?usp=sharing]([https://drive.google.com/file/d/1](https://drive.google.com/file/d/1) UMS4 EmWv5 SWXfaSl_ZC4 DC73 dk3 JyHeq/view?usp=sharing), accessed on 20 May 2021.  
Following previous works [26], we evaluate our model on the held-out test set from the datasets. It evaluates our model by comparing the relation facts recognized from the test sentences with those in Freebase. To show the performance of our model, we use precision-recall (PR) curves and top-N precision (P@N) as metrics in our experiments. The PR curves are constructed using the model predictions on all entity pairs in the test set for all relation types sorted by the confidence scores from the highest to lowest.