Exploiting W ord N et Synset and Hypernym Representations for Answer Selection (2020)

Abstract: Answer selection (AS) is an important subtask of document-based question answering (DQA). In this task, the candidate answers come from the same document, and each answer sentence is semantically related to the given question, which makes it more challenging to select the true answer. WordNet provides powerful knowledge about concepts and their semantic relations so we employ WordNet to enrich the abilities of paraphrasing and reasoning of the network-based question answering model. Specifically, we exploit the synset and hypernym concepts to enrich the word representation and incorporate the similarity scores of two concepts that share the synset or hypernym relations into the attention mechanism. The proposed WordNet-enhanced hierarchical model (WEHM) consists of four modules, including WordNet-enhanced word representation, sentence encoding, WordNet-enhanced attention mechanism, and hierarchical document encoding. Extensive experiments on the public WikiQA and SelQA datasets demonstrate that our proposed model significantly improves the baseline system and outperforms all existing state-of-the-art methods by a large margin.

## introduction answer selection (as) is a challenging subtask of document-based question answering (dqa) in natural language processing (nlp). the as task is to select a whole answer sentence from the document and can be regarded as a ranking problem, which is different from the machine reading comprehension (mrc) task on the squad and ms-marco datasets. compared with a single word or phrase, returning the full sentence often adds more value as the user can easily verify the correctness without reading a lengthy document @xcite . in * corresponding author. lots of fruits on answer selection have been achieved via deep learning models, including convolutional neural network (cnn) @xcite , recurrent neural network (rnn) @xcite , attention-way @xcite and generative adversarial networks (gan) @xcite . recently proposed models often consist of an embedding layer, an encoding layer, an interaction layer, and an answer layer @xcite @xcite . different from other question answering like community-based question answering, the candidate answers of dqa come from the same document, and each candidate answer is semantically related to the question. from the example in to tackle this challenge, we propose to leverage wordnet knowledge base into the neural network model. our hypothesis is that the ability of paraphrase and reasoning is essential to the questionanswering task. wordnet is a semantic network @xcite , where the words that are related in meanings are interlinked by means of pointers, which stand for different semantic relations. it organizes concepts mainly with the is-a relation, where a concept is a set of word senses (synset). on the one hand, we apply the synset information to enrich the sentence's paraphrase representation, which could distinguish the candidate answers in the latent semantic space to some degree. on the other hand, we apply the hypernym information to capture reasoning knowledge. the real case from the wikiqa dataset in the overall framework of our proposed model is shown in figure 1 , which mainly consists of four modules. first, we apply the synset and hypernym information to enrich the word representation. second, we use an rnn module to encode the wordnet-enhanced word representation. third, we propose to use the synset's and hypernym's relation score based on two senses' path in the wordnet to enrich the attention mechanism. specifically, the attention similarity matrix is not only measured by a similarity score over hidden vectors produced by cnn or rnn networks but also measured based on the synset and hypernym relation scores of two concepts in wordnet. and then following the compareaggregate framework @xcite , we combine the original representation with the attention representation. finally, considering the strong relations among context sentences, we employ a hierarchical neural network for answer sentence selection. we conduct extensive experiments on the public wikiqa and selqa datasets. the results show that our proposed wordnet-enhanced hierarchical model outperforms the baseline models by a large margin and achieves state-of-the-art performance on both datasets. on the wikiqa data, it obtains a map of 77.02, which beats the existing best result by 1.62 points; on the selqa data, it achieves a map of 91.71, which outperforms the previous best result by 2.57 points. ## model description given a question q and the sentences a i , @xmath0, 2, ..., s in a document d, our model aims to select the best sentence which could answer the question. ## related work in the nlp field, many problems involve matching two or more sequences to make a decision. for the dqa task, most of the studies also consider this problem as text matching, and they compute the semantic similarity between the question and candidate answers to decide whether a sentence in the document could answer the question. there have been various deep neural network models proposed to tackle sentence pairs matching. two kinds of matching strategies have been considered: the first is to convert the whole source and target sentences into embedding vectors in the latent spaces respectively, and then calculate the similarity score between them; the second is to calculate the similarities among all possible local positions of the source and target sentences and then summarize the local scores into the final similarity value. as for works using the first strategy, @xcite apply a tensor transformation layer on cnn-based embeddings to capture the interactions between the question and answer. @xcite employ the long short-term memory (lstm) network to address this problem. in the second strategy, @xcite build hierarchical convolution layers on the word similarity matrix between sentences, and @xcite propose multigrancnn to integrate multiple granularity levels of matching models. for the dqa task, the notable work is the compare-aggregate structure, which is first proposed by @xcite . following this structure, @xcite propose the dynamicclip way to compute the attention score. our basic model also adopts this structure, but with a different implementation. what's more, we employ a hierarchical module to capture inter-sentence relations. exploiting the background knowledge and common sense to improve nlp tasks' performance has long been a heated research topic. to facilitate nlp tasks, various semantic knowledge bases (kbs) have been constructed, ranging from manually annotated semantic networks like wordnet @xcite to semi-automatically or automatically constructed knowledge graphs like freebase @xcite . recently, several approaches have been proposed to leverage the prior knowledge in neural networks on different tasks @xcite @xcite . @xcite fuse the prior knowledge into word representations with a knowledge gate by using question categories for the qa task and topics for the conversation task. @xcite propose a kblstm network architecture, which incorporates the background knowledge into lstm to improve machine reading. unlike the two approaches, our model directly employs the synset and hypernym concepts information to enrich the word representation. @xcite use wordnet to measure the semantic relatedness of word pairs for the natural language inference task, including synonym, antonym, hypernym, and same hypernym. each of these features is denoted as a real number and is incorporated into the neural networks. compared to the feature vectors derived from the wordnet, our model directly employ the synset and hypernym relation scores to enrich the attention mechanism. @xcite present an entailment model for solving the natural language inference (nli) problem that utilizes conceptnet as an external knowledge source, while our method mainly focus on the wordnet. ## conclusion in this paper, we exploit a wordnet-enhanced hierarchical model to address the answer selection problem. based on wordnet's prior knowledge, the proposed model applies the synset and hypernym concepts to enrich word representations and uses synset and hypernym relation scores between two concepts to enhance the traditional attention score. extensive experiments conducted on two benchmark datasets demonstrate that our method significantly improves the baseline model and outperforms state-of-the-art results by a large margin. our approach obtains 1.62% improvement and 2.57% improvement in map on the wikiqa and selqa datasets, respectively, compared to the stateof-the-art results. in the future, we would like to