Latent Question Reformulation and Information Accumulation for Multi-Hop Machine Reading

1 INTRODUCTION . The ability to automatically extract relevant information from large text corpora remains a major challenge . Recently , the task of question-answering has been largely used as a proxy to evaluate the reading capabilities of neural architectures . Most of the current datasets for question-answering focus on the ability to read and extract information from a single piece of text , often composed of few sentences ( Rajpurkar et al. , 2016 ; Nguyen et al. , 2016 ) . This has strengthened the emergence of easy questions in the sense of Sugawara et al . ( 2018 ) and influenced the recent state-of-the-art models to be good at detecting patterns and named entities ( Devlin et al. , 2018 ; Yu et al. , 2018 ; Wang et al. , 2017 ) . However they still lack actual reasoning capabilities . The problem of reasoning requires machine comprehension models to gather and compose over different pieces of evidence spread across multiple paragraphs . In this work , we propose an original neural architecture that repeatedly reads from a set of paragraphs to aggregate and reformulate information . In addition to the sequential reading , our model is designed to collect pieces of information in parallel and to aggregate them in its last layer . Throughout the model , the important pieces of the document are highlighted by what we call a reading module and integrated into a representation of the question via our reformulation module . Our contributions can be summarised as follows : • We propose a machine reading architecture , composed of multiple token-level attention modules , that collect information sequentially and in parallel across a document to answer a question , • We propose to use an input-length invariant question representation updated via a dynamic max-pooling layer that compacts information form a variable-length text sequence into a fixed size matrix , • We introduce an extractive reading-based attention mechanism that computes the attention vector from the output layer of a generic extractive machine reading model , • We illustrate the advantages of our model on the HOTPOTQA dataset . The remainder of the paper is organized as follows : Section 2 presents the multi-hop machine reading task , and analyses the required reasoning competencies . In Section 3 , we detail our novel reading architecture and present its different building blocks . Section 4 presents the conducted experiments , several ablation studies , and qualitative analysis of the results . Finally , Section 5 discusses related work . Our code to reproduce the results is publicly available at ( removed for review ) . 2 TEXT-BASED QUESTION-ANSWERING AND MACHINE REASONING . The task of extractive machine reading can be summarized as follows : given a document D and a questionQ , the goal is to extract the span of the document that answers the question . In this work , we consider the explainable multi-hop reasoning task described in Yang et al . ( 2018 ) and its associated dataset : HOTPOTQA . We focus our experiments on the ” distractor ” configuration of the dataset . In this task , the input document D is not a single paragraph but a set of ten paragraphs coming from different English Wikipedia articles . Answering each question requires gathering and integrating information from exactly two paragraphs ; the eight others are distractors selected among the results of a tf-idf retriever ( Chen et al. , 2017 ) . These required paragraphs are called the gold paragraphs . There are two types of questions proposed in this dataset : extractive ones where the answer is a span of text extracted from the document and binary yes/no questions . In addition to the answer , it is required to predict the sentences , also called supporting facts , that are necessary to produce the correct answer . This task can be decomposed in three subtasks : ( 1 ) categorize the answer among the three following classes : yes , no , text span , ( 2 ) if it is a span , predict the start and end positions of this span in the document , and ( 3 ) predict the supporting sentences required to answer the question . In addition to the ” distractor ” experiments , we show how our proposed approach can be used for opendomain question answering and evaluate the entire reading pipeline on the ” fullwiki ” configuration of the HotpotQA dataset . In this configuration , no supporting documents are provided , and it is required to answer the question from the entire Wikipedia corpus . Among the competencies that multi-hop machine reading requires , we identify two major reasoning capabilities that human readers naturally exploit to answer these questions : sequential reasoning and parallel reasoning . Sequential reasoning requires reading a document , seeking a piece of information , then reformulating the question and finally extracting the correct answer . This is called multi-hop question-answering and refers to the bridge questions in HOTPOTQA . Another reasoning pattern is parallel reasoning , required to collect pieces of evidence for comparisons or question that required checking multiple properties in the documents . Figure 1 presents two examples from HOTPOTQA that illustrate such required competencies . We hypothesize that these two major reasoning patterns should condition the design of the proposed neural architectures to avoid restricting the model to one or the other reasoning skill . 3 THE MODEL . In this section , we describe the Latent Question Reformulation Network ( LQR-net ) , shown in Figure 2 . This multi-hop model is designed as an association of four modules : ( 1 ) an encoding module , ( 2 ) a reading module , ( 3 ) a question reformulation module , and ( 4 ) an answering module . ( 1 ) and ( 4 ) are input and output modules , whereas ( 2 ) and ( 3 ) constitute a hop , and are repeated respectively T and T − 1 times : the answering module does not require a last reformulation step . Given a document and a question , the reading module is in charge of computing a question-aware representation of the document . Then , the reformulation module extracts essential elements from this document representation and uses them to update a representation of the question in a latent space . This reformulated question is then passed to the following hop . The model can have multiple heads , as in the Transformer architecture ( Vaswani et al. , 2017 ) . In this case , the iterative mechanism is performed several times in parallel in order to compute a set of independent reformulations . The final representations of the document produced by the different heads are eventually aggregated before being fed to the answering module . This module predicts the answer and the supporting facts from the document . The following parts of this section describe each module that composes this model . Note : The model is composed of K independent reading heads that process the document and question in parallel . To not overload the notations of the next parts , we do not subscript all the matrices by the index of the head and focus on the description of one . The aggregation process of the multi-head outputs is explained in Section 3.5 . 3.1 ENCODING MODULE . We adopt a standard representation of each token by using the pre-trained parametric language model BERT ( Devlin et al. , 2018 ) . Let a document D = { p1 , p2 , . . . , p10 } be the set of input paragraphs , of respective lengths { n1 , . . . , n10 } , associated to a question Q of length L. These paragraphs are independently encoded through the pre-trained BERT model . Each token is represented by its associated BERT hidden state from the last layer of the model . The tokens representations are then concatenated to produce a global representation of the set of 10 paragraphs of total length N = ∑10 i=1 ni . The representations are further passed through a Bidirectional Gated Recurrent Unit ( BiGRU ) ( Cho et al. , 2014 ) to produce the final representation of the document ED ∈ RN×2h and question EQ ∈ RL×2h , where h is the hidden state dimension of the BiGRUs . EQ = BiGRU ( BERT ( Q ) ) , ED = BiGRU ( [ BERT ( p1 ) ; . . . ; BERT ( p10 ) ] ) , ( 1 ) where [ ; ] is the concatenation operation . To compute the first representation of the question U ( 0 ) , we use an interpolation layer to map EQ ∈ RL×2h to U ( 0 ) ∈ RM×2h where M is an hyperparameter of the model . Intuitively , RM×2h corresponds to the space allocated to store the representation of the question and its further reformulations . It does not depend on the length of the original question L . 3.2 READING MODULE . Our model is composed of T hops of reading that sequentially extract relevant information from a document regarding the current reformulation of the question . At step t , given a representation of the reformulated question U ( t ) ∈ RM×2h and a representation of the document ED ∈ RN×2h , this module computes a question-aware representation of the document . This module is a combination of two layers : a document-question attention followed by a document self-attention . Document-Question Attention : We first construct the interaction matrix between the document and the current reformulation of the question S ∈ RN×M as : Si , j = w1E D i , : +w2U ( t ) j , : +w3 ( E D i , : U ( t ) j , : ) , ( 2 ) where w1 , w2 , w3 are trainable vectors of R2h and the element-wise multiplication . Then , we compute the document-to-question attention Cq ∈ RN×2h : Pi , j = exp ( Si , j ) ∑M k=1 exp ( Si , k ) , Cqi , : = M∑ j=1 Pi , jU ( t ) j , : . ( 3 ) And the question-to-document attention qc ∈ R2h : mi = max j∈ { 1 , ... , M } Si , j , p = softmax ( m ) , qc = N∑ j=1 pjE D j , : . ( 4 ) Finally , we compute the question-aware representation of the document X ( t ) ∈ RN×8h : X ( t ) i , : = [ E D i , : ; C q i , : ; E D i , : C q i , : ; q c Cqi , : ] , ( 5 ) where [ ; ] concatenation operation . Finally , we use a last BiGRU that reduces the dimension of X ( t ) to N × 2h . This specific attention mechanism was first introduced in the Bidirectional Attention Flow model of Seo et al . ( 2017 ) . We hypothesize that such token-level attention will produce a finer-grained representation of the document compared to sentence-level attention used in state-of-the-art Memory Network architectures . Document Self-Attention : So far , the contextualization between the ten paragraphs has only be done by the BiGRUs of equation 1 . One limitation of the current representation of the document is that each token has very limited knowledge of the other elements of the context . To deal with long-range dependencies , we apply this same attention mechanism between the question-aware representation of the document , X ( t ) , and itself to produce the reading module output V ∈ RN×2h . This self-contextualization of the document has been found useful in our experiments as presented in the ablation analysis of Section 4.3 .

Summary: This paper proposes a model for multi-hop question answering, specifically in a closed domain setting where the relevant paragraphs are present within a few distractor paragraphs. Their model has the following components — (a) a reader module that reads and collect information from paragraphs, (b) a query reformulation module, that reformulates the query to retrieve the next relevant document required to answer the question. Following the effectiveness of multi-head attention, the reader and reformulation module has K-heads that forms independent reformulations. The k-heads are aggregated via a simple summing and the reformulated query representation is used in the next step of the pre-defined hop.