We explore a hybrid approach for Chinese definitional question answering by combining deep linguistic analysis with surface pattern learning . We answer four questions in this study : 1 ) How helpful are linguistic analysis and pattern learning ? 2 ) What kind of questions can be answered by pattern matching ? 3 ) How much annotation is required for a pattern-based system to achieve good performance ? 4 ) What linguistic features are most useful ? Extensive experiments are conducted on biographical questions and other definitional questions . Major findings include : 1 ) linguistic analysis and pattern learning are complementary ; both are required to make a good definitional QA system ; 2 ) pattern matching is very effective in answering biographical questions while less effective for other definitional questions ; 3 ) only a small amount of annotation is required for a pattern learning system to achieve good performance on biographical questions ; 4 ) the most useful linguistic features are copulas and appositives ; relations also play an important role ; only some propositions convey vital facts . Due to the ever increasing large amounts of online textual data , learning from textual data is becoming more and more important . Traditional document retrieval systems return a set of relevant documents and leave the users to locate the specific information they are interested in . Question answering , which combines traditional document retrieval and information extraction , solves this problem directly by returning users the specific answers . Research in textual question answering has made substantial advances in the past few years ( Voorhees , 2004 ) . Most question answering research has been focusing on factoid questions where the goal is to return a list of facts about a concept . Definitional questions , however , remain largely unexplored . Definitional questions differ from factoid questions in that the goal is to return the relevant " answer nuggets " of information about a query . Identifying such answer nuggets requires more advanced language processing techniques . Definitional QA systems are not only interesting as a research challenge . They also have the potential to be a valuable complement to static knowledge sources like encyclopedias . This is because they create definitions dynamically , and thus answer definitional questions about terms which are new or emerging ( Blair-Goldensoha et al . , 2004 ) . One success in factoid question answering is pattern based systems , either manually constructed ( Soubbotin and Soubbotin , 2002 ) or machine learned ( Cui et al . , 2004 ) . However , it is unknown whether such pure pattern based systems work well on definitional questions where answers are more diverse . Deep linguistic analysis has been found useful in factoid question answering ( Moldovan et al . , 2002 ) and has been used for definitional questions ( Xu et al . , 2004 ; Harabagiu et al . , 2003 ) . Linguistic analy-sis is useful because full parsing captures long distance dependencies between the answers and the query terms , and provides more information for inference . However , merely linguistic analysis may not be enough . First , current state of the art linguistic analysis such as parsing , co-reference , and relation extraction is still far below human performance . Errors made in this stage will propagate and lower system accuracy . Second , answers to some types of definitional questions may have strong local dependencies that can be better captured by surface patterns . Thus we believe that combining linguistic analysis and pattern learning would be complementary and be beneficial to the whole system . Work in combining linguistic analysis with patterns include Weischedel et al . ( 2004 ) and Jijkoun et al . ( 2004 ) where manually constructed patterns are used to augment linguistic features . However , manual pattern construction critically depends on the domain knowledge of the pattern designer and often has low coverage ( Jijkoun et al . , 2004 ) . Automatic pattern derivation is more appealing ( Ravichandran and Hovy , 2002 ) . In this work , we explore a hybrid approach to combining deep linguistic analysis with automatic pattern learning . We are interested in answering the following four questions for Chinese definitional question answering : How helpful are linguistic analysis and pattern learning in definitional question answering ? If pattern learning is useful , what kind of question can pattern matching answer ? How much human annotation is required for a pattern based system to achieve reasonable performance ? We have explored a hybrid approach for definitional question answering by combining deep linguistic analysis and surface pattern learning . For the first time , we have answered four questions regarding Chinese definitional QA : deep linguistic analysis and automatic pattern learning are complementary and may be combined ; patterns are powerful in answering biographical questions ; only a small amount of annotation ( 2 days ) is required to obtain good performance in a biographical QA system ; copulas and appositions are the most useful linguistic features ; relation extraction also helps . Answering " What-is " questions is more challenging than answering " Who-is " questions . To improve the performance on " What-is " questions , we could divide " What-is " questions into finer classes such as organization , location , disease , and general substance , and process them specifically . Our current pattern matching is based on simple POS tagging which captures only limited syntactic information . We generalize words to their corresponding POS tags . Another possible improvement is to generalize using automatically derived word clusters , which provide semantic information .

Challenge: Although there are some open questions such as how helpful are linguistic analysis and pattern learning, the definitional question is under explored.
Approach: They evaluate a hybrid approach for Chinese definitional question answering by combining deep linguistic analysis with surface pattern learning.
Outcome: They report findings such as linguistic analysis can complement pattern learning, and that a small amount of annotation is sufficient for a biographical question answering.