Multi-step Retriever-Reader Interaction for Scalable Open-domain Question Answering

1 INTRODUCTION . Open-domain question answering ( QA ) ( Voorhees et al. , 1999 ) involves a retriever for selecting relevant context from a large corpora of text ( e.g . Wikipedia ) and a machine reading comprehension ( MRC ) model for ‘ reasoning ’ on the retrieved context . A lot of effort has been put into designing sophisticated neural MRC architectures for reading short context ( e.g . a single paragraph ) , with much success ( Wang & Jiang , 2017 ; Seo et al. , 2017 ; Xiong et al. , 2017 ; Wang et al. , 2018c ; Yu et al. , 2018 , inter alia ) . However , the performance of such systems degrades significantly when combined with a retriever in open domain settings . For example , the exact match accuracy of DrQA ( Chen et al. , 2017 ) , on the SQUAD dataset ( Rajpurkar et al. , 2016 ) degrades from 69.5 % to 28.4 % in open-domain settings . The primary reason for this degradation in performance is due to the retriever ’ s failure to find the relevant paragraphs for the machine reading model ( Htut et al. , 2018 ) . We propose the following two desiderata for a general purpose open-domain QA system - ( a ) The retriever model should be fast , since it has to find the relevant context from a very large text corpora and give it to the more sophisticated and computationally expensive MRC model ( b ) Secondly , the retriever and reader models should be interactive , i.e . if the reader model is unable to find the answer from the initial retrieved context , the retriever should be able to learn to provide more relevant context to the reader . Open-domain QA systems such as R3 ( Wang et al. , 2018a ) and DS-QA ( Lin et al. , 2018 ) have sophisticated retriever models where the reader and retriever are jointly trained . However , their retriever computes question-dependent paragraph representation which is then encoded by running an expensive recurrent neural network over the tokens in the paragraph . Since the retriever has to rank a lot of paragraphs , this design does not scale to large corporas . One the other hand , the retriever model of QA systems such as DrQA ( Chen et al. , 2017 ) is based on a tf-idf retriever , but they lack trainable parameters and are consequently unable to recover from mistakes . This paper introduces an open domain architecture in which the retriever and reader iteratively interact with each other . Our model first pre-computes and caches representation of context ( paragraph ) . These representations are independent of the query unlike recent architectures ( Wang et al. , 2018a ; 1Code and pretrained models are available at https : //github.com/rajarshd/Multi-Step-Reasoning Lin et al. , 2018 ) and hence can be computed and stored offline . Given an input question , the retriever performs fast inner product search to find the most relevant contexts . The highest ranked contexts are then passed to the neural MRC model . Our architecture is agnostic to the choice of the reader architecture and we show that multi-step-reasoning increases performance of two state-of-the-art MRC architectures - DrQA ( Chen et al. , 2017 ) and BiDAF ( Seo et al. , 2017 ) . It is possible that the answer might not exist in the initial retrieved paragraphs or that the model would need to combine information across multiple paragraphs ( Wang et al. , 2018b ) . We equip the reader with an additional gated recurrent unit ( Cho et al. , 2014 ) which takes in the state of the reader and the current query vector and generates a new query vector . This new query vector is then used by the retriever model to re-rank the context . This allows the model to read new paragraphs and combine evidence across multiple paragraphs . Since the retriever makes a ‘ hard selection ’ of paragraphs to send to the reader , we train the retriever and the reader jointly using reinforcement learning . Our architecture draws inspiration from how students are instructed to take reading comprehension tests ( Cunningham & Shablak , 1975 ; Bishop et al. , 2006 ; Duggan & Payne , 2009 ) . Given a document containing multiple paragraphs and a set of questions which can be answered from the document , ( a ) the student quickly skims the paragraphs , ( b ) then for each question , she finds the most relevant paragraphs that she thinks will answer the question . ( c ) She then carefully reads the chosen paragraph to answer the question ( d ) However , if the chosen paragraph does not answer the question , then given the question and the knowledge of what she has read till now , she decides which paragraph to read next . Step ( a ) is akin to our model encoding and storing the question independent paragraph representations and step ( b ) corresponds to the inner product search to find the relevant context . The reading of the context by the sophisticated neural machine reader corresponds to step ( c ) and the last step corresponds to the iterative ( multi-step ) interaction between the retriever and the reader . To summarize , this paper makes the following contributions : ( a ) We introduce a new framework for open-domain QA in which the retriever and reader iteratively interact with each other via a novel multistep-reasoning component allowing it to retrieve and combine information from multiple paragraphs . ( b ) Our paragraph representations are independent of the query which makes our architecture highly scalable and we empirically demonstrate it by running large scale experiments over millions of paragraphs . ( c ) Lastly , our framework is agnostic to the architecture of the reader and we show improvements on two widely used neural reading comprehension models . 2 BASIC COMPONENTS OF OUR MODEL . The architecture of our model consists of three main components - ( a ) paragraph retriever - that computes a relevance score for each paragraph w.r.t a given query and ranks them according to the computed score . ( b ) reader - a more sophisticated neural machine reading model that receives few top-ranked paragraphs from the retriever and outputs a span of text as a possible answer to the query and ( c ) multi-step-reasoner - a gated recurrent unit that facilitates iterative interaction between the retriever and the reader . Formally , the input to our model is a natural language question Q = q1 , q2 , . . . , qn consisting of n tokens and a set of paragraphs P = { p1 , p2 , . . . pK } . Our model extracts a span of text a as the answer to the question from the paragraphs in P. Note that the set of paragraphs in P can be the paragraphs in a set of documents retrieved by a search engine or it could be all the paragraphs in a large text corpus such as Wikipedia . Next we describe each individual component of our model . 2.1 PARAGRAPH RETRIEVER . The paragraph retriever computes a score for how likely a paragraph is to contain an answer to a given question . The paragraph representations are computed independent of the query and once computed , they are not updated . This allows us to cache the representations and store them offline . The relevance score of a paragraph is computed as a inner product between the paragraph and the query vectors . The paragraph and query representations are computed as follows . Given a paragraph p = { p1 , p2 , . . . , pm } consisting of m tokens , a multi-layer recurrent neural network encodes each tokens in the paragraph — { p1 , p2 , . . . , pm } = RNN ( { p1 , p2 , . . . , pm } ) , where p j ∈R2d encodes useful contextual information around the j-th token . Specifically we choose to use a multilayer bidirectional long-short term memory network ( LSTM ) ( Hochreiter & Schmidhuber , 1997 ) and take p j as the hidden units in the last layer of the RNN . We concatenate the representation computed by the forward and the backward LSTM . To compute a single paragraph vector p ∈ R2d from all the token representations , we combine them using weights b j ∈ R b j = exp ( w ·pj ) ∑ j′ exp ( w ·p j′ ) p =Ws ∑ j′ b j′ ·pj′ Here b j encodes the importance of each token and w ∈ R2d , Ws ∈ R2d×2d are learned weights . The query q = { q1 , q2 , . . . , qn } is encoded by another network with the same architecture to obtain a query vector q ∈ R2d . Next the relevance score of a paragraph w.r.t the query ( score ( p , q ) ∈ R ) is computed by a simple inner product — score ( p , q ) = 〈p , q〉 . The paragraph retriever then returns the top scoring k paragraphs to the reader . Fast inner product search . Inner product can be efficiently computed on a GPU , however in our experiments with large corpus with over million paragraphs , ( a ) the paragraph vectors require more memory than available in a single commodity GPU ( b ) it is computationally wasteful to compute inner products with all paragraph vectors and we can do better by leveraging fast nearest neighbor ( NN ) search algorithms . Since the retriever has to find the k paragraphs with the highest inner products w.r.t query , our problem essentially reduces to maximum inner product search ( MIPS ) . There exists data structures for fast NN search in metric spaces such as Cover Trees ( Beygelzimer et al. , 2006 ) . But we can not use them directly , since triangle equality does not hold in inner product space . Borrowing ideas from Bachrach et al . ( 2014 ) ; Zaheer et al . ( 2019 ) , we propose to use nearest neighbor ( NN ) search algorithms to perform the MIPS in sublinear time as follows . Let u be an upper bound for the L2 norm for all paragraphs , i.e . u≥ ‖p‖ , ∀p . Next , we modify the original paragraph and query vectors such that searching for the k-nearest neighbor w.r.t L2 distance with the modified vectors is equivalent to finding the k nearest ( original ) paragraph vectors in the inner-product space . Define the augmented paragraph vectors as p̃i = [ pi ; √ u2−‖pi‖2 ] and augmented query vector as q̃ = [ q ; 0 ] , where [ ; ] denotes concatenation . Note , with this transformation , 〈p̃i , q̃〉= 〈pi , q〉 . Now , NN ( p̃i , q̃ ) = argmin i ‖p̃i− q̃‖2 = argmin i ‖p̃i‖2 +‖q̃‖2−2〈p̃i , q̃〉 = argmin i ( u2−2〈p̃i , q̃〉 ) = argmax i 〈p̃i , q̃〉= argmax i 〈pi , q〉= MIPS ( pi , q ) Many exact NN search algorithms can find k-NN in time ( sublinear ) logarithmic in number of paragraphs ( Beygelzimer et al. , 2006 ; Zaheer et al. , 2019 ) , after an one-time preprocessing . In our architecture , the preprocessing can be done because the set of all paragraphs is fixed for all the queries . We chose to use SGTree ( Zaheer et al. , 2019 ) to perform the NN/MIPS due to its fast construction time and competitive performance reducing the search to logarithmic time per query . Note that our paragraph representations are independent of the given query and are never updated once they are trained . After training is completed , we cache the p ∈ R2d vectors . This is unlike many recent work in open-domain QA . For example , in R3 ( Wang et al. , 2018a ) , the retriever uses Match-LSTM model ( Wang & Jiang , 2017 ) to compute question-matching representations for each token in the passage . The paragraph representations are then obtained by running a bi-directional RNN over these matched representation . Although powerful , this architecture design will not scale in open-domain settings where the retriever has to re-compute new representations for possibly millions of paragraphs for every new question . In contrast , once training is over , we cache the paragraph representation and use it for every query at test time . Training - Following previous work ( Htut et al. , 2018 ; Lin et al. , 2018 ; Wang et al. , 2018a ) , we gather labels for paragraphs during training using distant supervision ( Mintz et al. , 2009 ) . A paragraph that contains the exact ground truth answer string is labeled as an positive example . For a positive ( negative ) labeled paragraph , we maximize ( minimize ) the log ( σ ( score ( p , q ) ) ) . The number of layers of the bi-directional LSTM encoder is set to three and we use Adam ( Kingma & Ba , 2014 ) for optimization . Once training is done , we pre-compute and cache the paragraph representation of each dataset in our experiments .

Summary: The paper proposes a multi-document extractive machine reading model and algorithm. The model is composed of 3 distinct parts. First, the document retriever and the document reader that are states of the art modules. Then, the paper proposes to use a "multi-step-reasoner" which learns to reformulate the question into its latent space wrt its current value and the "state" of the machine reader.