A Simple Approach to Jointly Rank Passages and Select Relevant Sentences in the OBQA Context (2022)

Abstract: In the open book question answering (OBQA) task, selecting the relevant passages and sentences from distracting information is crucial to reason the answer to a question. HotpotQA dataset is designed to teach and evaluate systems to do both passage ranking and sentence selection. Many existing frameworks use separate models to select relevant passages and sentences respectively. Such systems not only have high complexity in terms of the parameters of models but also fail to take the advantage of training these two tasks together since one task can be beneficial for the other one. In this work, we present a simple yet effective framework to address these limitations by jointly ranking passages and selecting sentences. Furthermore, we propose consistency and similarity constraints to promote the correlation and interaction between passage ranking and sentence selection. The experiments demonstrate that our framework can achieve competitive results with previous systems and outperform the baseline by 28% in terms of exact matching of relevant sentences on the HotpotQA dataset.

## introduction open book question answering (obqa) requires a system to find the relevant documents to reason the answer to a question. it has wide and practical natural language processing (nlp) applications such as search engines @xcite and dialogue systems @xcite . among several obqa datasets @xcite @xcite , hotpotqa @xcite is more challenging because it requires a system not only to find the relevant passages from large corpus but also find the relevant sentences in the passage which eventually reach to the answer. such a task also increases the interpretability of the systems. to address this challenge, most of the previous work @xcite ; tu figure 1 : an example from the hotpotqa dataset, where the question should be answered by combining supporting facts(sp) from two passages. in the sp, the first string refers to the title of passage, and the second integer means the index of the sentence. and selecting relevant sentences are not well utilized by these systems. intuitively, if a passage is ranked high, then some sentences in the passage should be selected as relevant. on the other hand, if a passage is ranked low, then all sentences in the passage should be classified as irrelevant. to build a green ai system and take advantage of multi-task learning, we introduce a two-in-one model, a simple model trained on passage ranking and sentence selection jointly. more specifically, our model generates passage representations and sentence representations simultaneously, which are then fed to a passage ranker and sentence classifier respectively. then we promote the interaction between passage ranking and sentence classification using consistency and similarity constraints. the consistency constraint is to enforce that the relevant passage includes relevant sentences, while the similarity constraint ensures the model to generate the representation of relevant passages more closer to the representations for relevant sentences than irrelevant ones. the experiments conducted on the hotpotqa datasets demonstrate that our simple model achieves competitive results with previous systems and outperforms the baselines by 28%. ## related work hotpotqa systems a straightforward way to solve the hotpotqa challenge is to build a hierarchical system @xcite , meaning a system first ranks relevant passages and then identifies relevant sentences from the selected passages. such a hierarchical system involves multiple models thus requires long inference time. more importantly, such a system only leverages the impact of passage ranking on sentence selection but ignores the influence of the sentence selection on the passage ranking. our framework achieves these two tasks by one model and facilitates the interaction by two constraints. @xcite proposes a pipeline based on three bert models @xcite to solve the hotpotqa challenge. the system first selects relevant sentences and then detects the answer span, finally, identifies the relevant sentences according to the answer span. though the pipeline is strong, the way it solves the problem is opposite to human beings. we, humans, identify the relevant sentences, and then give the answer span. many existing works demonstrate the effectiveness of graph neural networks(gnn) on hotpotqa challenge @xcite . since gnn is out of the scope of this work, 107 we do not compare it with these frameworks. in this work, we mainly focus on the latter setting. ## conclusion in this work, we present a simple model, two-in-one, to rank passage and classify sentence together. by jointly training with passage ranking and sentence selection, the model is capable of capturing 5 185 the correlation between passages and sentences.