# Enriching Knowledge Base by Parse Tree Pattern and Semantic Filter
## 6 Experiments
### Evaluation of Parse Tree Patterns  
We show the superiority of a parse tree representation of patterns by comparing it with a lexical representation. For the evaluation of patterns, ten most-frequent relations are selected among 30 relations. The ten relations used are _artist_, _board_, _crosses_, _deathPlace_, _field_, _location_, _publisher_, _religion_, _spouse_, and _team_. Although only one third of DBpedia relations are used, the ten relations can cover most of the pattern candidates. That is, 63,704 unique pattern candidates are generated from 30 relations, but 75% of them are covered by the ten relations.  
All the triples of the DBpedia ontology that correspond to the ten predicates are employed as seed triples. In order to generate both kinds of patterns, 100 sentences are randomly sampled from the Wikipedia corpus for each relation. Since one pattern is generated from a sentence, every relation has 100 patterns for parse tree representation and lexical representation, respectively. In order to obtain lexical sequence patterns which are used for previous work such as BOA or OLLIE, we follow only the pattern search stage of BOA. The correctness of each pattern is measured by two human assessors. For each pattern of a relation, the assessors determine if the words in the pattern deliver the meaning of the relation accurately. Finally, only the patterns which both assessors agree with as true are considered to be correct.  
Figure 4 shows the comparison result of parse tree and lexical sequence patterns. The X-axis of this figure represents relations, and the Y-axis is the accuracy of patterns. The proposed parse tree patterns show consistently higher accuracy than the lexical sequence patterns for all relations. The average accuracy of parse tree patterns is 68%, while that of lexical sequence patterns is just 52%. The maximum accuracy difference between the two pattern representations is 35% for the relation _publisher_. Since parse trees represent dependency relations between words and thus it can reveal long distance dependencies of non-intervening words, higher accurate patterns are generated by parse trees.  
After investigating all 1000 (=100 patterns 10 relations) parse tree patterns, it is found that around 34% of words appearing at the patterns are non-intervening words and about 45% patterns contain at least one non-intervening word. The fact that many patterns contain non-intervening words implies that the proposed parse tree pattern represents long distance dependencies between words effectively. For example, consider the following sentence and a triple \(\langle Floating\)_into the \(Night,artist,\)_[PERSON] \([PERSON]\rangle\).  
_Most notably he produced and wrote lyrics for [PERSON]'s first two albums, Floating into the Night (1989) and The Voice of Love (1993)._  
Figure 4: The accuracy comparison of parse tree patterns and lexical sequence patterns.  
From this sentence, the lexical pattern extracts \(\langle first\ two\ album(s)\rangle\) as a pattern, and the pattern contains meaningless words such as _first_ and _two_. However, the following parse tree pattern excludes such non-intervening words.  
\[[\text{{Subject}}]\leftarrow\text{appos}\leftarrow[\text{album}]\rightarrow\text{poss} \rightarrow[\text{{Object}}].\]