# Enriching Knowledge Base by Parse Tree Pattern and Semantic Filter
## 6 Experiments
### Evaluation of Newly Extracted Knowledge  
In order to investigate whether parse tree patterns and semantic filters produce accurate new triples, the triples extracted by the \"parse tree + semantic filter\" patterns are compared with those extracted by \"lexical + frequency filter,\" \"lexical + semantic filter,\" and \"parse tree + frequency filter\" patterns. Since the Wikipedia corpus is excessively large, 15 million sentences are randomly sampled from the corpus and new triples are extracted from the sentences. Table 2 shows the detailed statistics of the number of matched patterns and triples extracted with the patterns. According to this table, the number of matched lexical sequence patterns is 255 and that of parse tree patterns is 713. As a result, the numbers of new triples extracted by lexical sequence patterns and parse tree patterns are 32,113 and 104,311 respectively. Though lexical sequence patterns and parse tree patterns are generated from and applied to an identical dataset, the parse tree patterns extract 72,198 more triples than the lexical sequence patterns, which implies that the coverage of parse tree patterns is much wider than that of lexical sequence patterns. In the evaluation of new triples, top-100 triples are selected for each relation according to the ranks, and the correctness of the 4000 (=100 triples \(\cdot\) 10 relations \(\cdot\) 4 pattern types) triples is checked manually by two assessors. As done in previous experiments, only the triples marked as true by both assessors are considered correct. Table 3 summarizes the accuracy of the triples extracted by \"parse tree + semantic filter\" patterns and those by \"lexical + frequency filter,\" \"lexical + semantic filter,\" and \"parse tree + frequency filter\" patterns. The triples extracted by \"parse tree + semantic filter\" patterns achieve 60.1% accuracy, while those by \"parse tree + frequency filter,\" \"lexical + semantic filter,\" and \"lexical + frequency filter\" achieve 53.9%, 38.2%, and 32.4% accuracy, respectively. The triples extracted by \"parse tree + semantic filter\" patterns outperform those by \"lexical + frequency filter\" patterns by 27.7%. They also outperform triples extracted by the \"lexical + semantic filter\" and \"parse tree + frequency filter\" patterns by 21.9% and 6.2%, respectively. These results prove that knowledge enrichment is much improved by using parse tree patterns and the proposed semantic filter. Most incorrect triples by parse tree patterns come from three relations of _deathPlace_, _field_, and _religion_. The accuracy of new triples goes up to 74.0% without the relations. The reason why _deathPlace_ produces many incorrect triples is explained above. For relations of _field_ and _religion_, it is found out that a few incorrect patterns which are highly ranked by the semantic filter produce most new triples. Therefore, it is our future work to solve the problems. After generating all possible pattern tree candidates, the irrelevant candidates are removed using Equation (1). \(\theta_{r}\)'s of each relation \(r\) used for filtering irrelevant candidates are given in Table 4. On average, 71 patterns of each relation are matched with Wikipedia sentences, but only 37 patterns remain after semantic filtering. And then, among 104,311 triples, triples which are extracted from eliminated patterns are excluded from results. As a result, 12,522 new triples are extracted and added into the seed knowledge.