mining redundancy in candidate-bearing snippets to improve web question answering
Conventional question answering (QA) techniques independently process candidate-bearing snippets to select an exact answer to a question from candidate answers. This paper presents two novel ways of utilizing redundancy in candidate-bearing snippets to help select an exact answer to a question in our Web QA system, i.e., cluster-based language model (CLM-M) and unsupervised SVM classifier (U-SVM) techniques. The comparative experiments demonstrate that the proposed methods significantly outperform the language model-based (LM-M) and supervised SVM-based (S-SVM) techniques that do not utilize this redundancy in the candidate-bearing snippets. Using the CLM-M, the top_1 score is increased from 36.03\% (LM-M) to 46.96\%; and the top_1 improvement in the U-SVM over the S-SVM is about 23\%. Moreover, a cross-model comparison shows that the performance ranking of these models is: U-SVM > CLM-LM > LM-M > S-SVM > R-M (the retrieval-based model).
question answering language model data redundancy support vector machine