Latent Question Reformulation and Information Accumulation for Multi-Hop Machine Reading
Authors: Quentin Grail
Julien Perez
Eric Gaussier
Venue:  (2020)
Keywords: question-answering
machine comprehension
deep learning

Abstract: Multi-hop text-based question-answering is a current challenge in machine comprehension. 
This task requires to sequentially integrate facts from multiple passages to answer complex natural language questions.
In this paper, we propose a novel architecture, called the Latent Question Reformulation Network (LQR-net), a multi-hop and parallel attentive network designed for question-answering tasks that require reasoning capabilities.
LQR-net is composed of an association of \textbf{reading modules} and \textbf{reformulation modules}.
The purpose of the reading module is to produce a question-aware representation of the document.
From this document representation, the reformulation module extracts essential elements to calculate an updated representation of the question.
This updated question is then passed to the following hop.
We evaluate our architecture on the \hotpotqa question-answering dataset designed to assess multi-hop reasoning capabilities.
Our model achieves competitive results on the public leaderboard and outperforms the best current \textit{published} models in terms of Exact Match (EM) and $F_1$ score.
Finally, we show that an analysis of the sequential reformulations can provide interpretable reasoning paths.

Under review as a conference paper at ICLR 2020
LATENT QUESTION REFORMULATION AND INFOR- MATION ACCUMULATION FOR MULTI-HOP MACHINE READING
Anonymous authors Paper under double-blind review
ABSTRACT
Multi-hop text-based question-answering is a current challenge in machine com- prehension. This task requires to sequentially integrate facts from multiple pas- sages to answer complex natural language questions. In this paper, we propose a novel architecture, called the Latent Question Reformulation Network (LQR- net), multi-hop and parallel attentive network designed for question-answeriine tasks that require reasoning capabilities. I-R-net is composed of an association reading modules and reformulation modules. The purpose the reading module is to produce question-aware representation of the document. From this document representation, the reformulation module extracts essential elements to caleulate updated representation of the question. This updated question is then passed to the following hop. We evaluate our architecture on the HOTPOTQA gunsernn-answering dataset designed assess multi-hop reasoning capabilities. Our model achieves competitive results on the public leaderboard and outper- forms the best current published models in terms of Exact Match (EM) and Fi score. Finally, we show that an analysis of the sequential reformulations can pro- vide interpretable reasoning paths.
INTRODUCTION
The ability to automatically extract relevant information from large text corpora remains major challenge. Recently, the task of question-answeree has been largely used as a proxy to evaluate the reading capabilities of neural architectures. Most of the current datasets for question-answering focus on the ability to read and extract information from a single piece of text, often composed of few sentences (Rajpurkar al. 2016: Nguyen al., (016). This has strengthened the emergence of easy questions in the sense of SSugwwara et al. (2018) and influenced the recent statc-of-ther
models to be good detecting patterns and named entities (Devlin et al. 2018 Yu al., 2018
Wang et al. 2017). However they still lack actual reasoning capabilities.
The problem of reasoning requires machine comprehension models to gather and compose over dif- ferent pieces of evidence spread across multiple paragraphs. In this work, we propose an original neural architecture that repeatedly reads from a set of paragraphs to aggregate and reformulate infor- mation. In addition to the sequential reading, our model is designed to collect pieces of information in parallel and to aggregate them its last layer. Throughout the model, the important pieces of the document are highlighted by what we call reading module and integrated into representation of the question via our reformulation module. Our contributions can be summarised as follows:
We propose a machine reading architecture, composed of multiple token-level attention modules, that collect information sequentially and in parallel across document answer a question,
We propose to use an input-length invariant question representation updated via a dynamic max-pooling layer that compacts information form variable-length text sequence into fixed size matrix,
We introduce an extractive reading- based attention mechanism that computes the attention vector from the output layer of a generic extractive machine reading model,

---

Under review as a conference paper at ICLR 2020
We illustrate the advantages of model on the HOTPOTQA dataset.
The remainder of the paper is organized as follows: Section
2 presents the multi-hop machine read- ing task, and analyses the required reasoning competencies. In Section 3
we detail our novel reading
architecture and present its different building blocks. Section presents the conducted experiments, several ablation studies, and qualitative analysis of the results. Finally, Section discusses related work.
Our code to reproduce the results is publicly available at (removed for review).
TEXT-BASED QUESTION-ANSWERNN AND MACHINE REASONING
(band)
American formed in Fulleton, Califoniaa
Apretim Najumdar 1978), classical musician India. He plays
Apratim Majumdar
Adorable
(band)
rock formed Coventry
Sarod
stringgs instrument of India, used mainty Indian
Which formed Ltt Adorabic? Aderable
Q: What class instrument does Apatim Majumder play" stringed instrument
Examples of reasoning paths to answer two questions of the HOTPOTQA dataset. this picture, we do not display the full paragraphs, but only the supporting facts.
Figure I:
The task of extractive machine reading can be summarized follows: given document D and question Q, the goal is to extract the span the document that answers the question. this work, we consider the explainable multi-hop reasoning task described in Yang al. (2018) and its associated dataset: HOTPOTQA We focus our experiments on the "distractor" configuration experiinee
the dataset. In this task, the input document D is not single paragraph but set of ten paragraphs coming from different English Wikipedia articles. Answering each question requires gathering and integrating information from exactly two paragraphs; the eight others are distractors selected among the results of a tf-idf retriever (Chen et al. 2017). These required paragraphs are called the gold paragraphs. There are two types of questions proposed in this dataset: extractive ones where the answer is span of text extracted from the document and binary yes/no questions. In addition to the answer, it required predict the sentences, also called supporting facts, that are necessary to produce the correct answer. This task can be decomposed in three subtasks: (1) categorize the answer among the three following classes: yes, no, text span, (2) if a span, predict the start and end positions of this span in the document, and (3) predict the supporting sentences required to answer the question. In addition to the "distractor" experiments, we show how our proposed approach can be used for open- domain question answering and evaluate the entire reading pipeline the "fullwiki"" configuration of the HotpotQA dataset. In this configuration, no supporting documents are provided, and it is required to answer the question from the entire Wikipedia corpus.
Among the competencies that multi-hop machine reading requires, we identify two major reasoning capabilities that human readers naturally exploit answer these questions: sequential reasoning and parallel reasoning. Sequential reasoning requires reading a document, seeking a piece in- formation, then reformulating the question and finally extracting the correct answer. This is called multi-hop question-answett ing pattern is parallel reasoning, required to collect pieces of evidence for comparisons or question that required checking multiple properties in the documents. Figure [ presents two examples from HOTPOTQA that illustrate such required competencies. We hypothesize that these two major rea- soning patterns should condition the design of the proposed neural architectures to avoid restricting the model to one or the other reasoning skill.
and refers to the bridge questions in HOTPOTQA Another reason-
3 THE MODEL
In this section, we describe the Latent Question Reformulation Network (LQR-net), shown in Figure
2 This multi-hop model is designed as an association four modules: (1) an encoding module, (2) reading module, (3) question reformulation module, and (4) an answering module. (1) and (4) are input and output modules, whereas (2) and (3) constitute a hop, and are repeated respectively T and T I times: the answering module does require a last reformulation step.
2

---

Under review as a conference paper at ICLR 2020
Puty
Encoding
sequential hape
Connected
E
paralial habide
Reading
Reading
Module
Module
V
U
(T--1)
Encoding
Interpolation
Answering Module
Reformulation
Module
U
UQ)
U1
BIGRA
X:0
FC
Ques- Doc
Self Attention
V0
E
Attention
E
Dynamic Pool
Reading Module
Reformulation Module
Answering Madule
Figure 2: Overview of LQR-net with K parallel heads and T sequential reading modules. In this architecture, latent representation of the question is sequentially updated to perform multi-hop reasoning. K independent reading heads collect pieces of information before feeding them to answering module. Sections present the different building blocks of this end-to-end trainable model.
Given a document and question, the reading module is in charge computing a question-aware representation of the document. Then, the reformulation module extracts essential elements from this document representation and uses them to update a representation of the question in a latent space. This reformulated question is then passed to the following hop.
The model can have multiple heads, as in the Transformer architecture (Vaswani et al. 2017). In this case, the iterative mechanism is performed several times in parallel in order to compute set of independent reformulations. The final representations of the document produced by the different heads are eventually aggregated before being fed to the answering module. This module predicts the answer and the supporting facts from the document. The following parts of this section describe each module that composes this model.
The model is composed of K independent reading heads that process the document and question in parallel. To not overload the notations of the next parts, we do not subscript all the matrices by the index of the head and focus on the description of one. The aggregation process of the multi-head outputs is explained in Section
Note:
3.1 ENCODING MODULE
We adopt a standard representation of each token by using the pre-trained parametric language model BERT (Devlin et al. 2018). Let document D {p1 . PPo)} be the set of input paragraphs, of respective lengths {7 - ). associated to a question Q of length L. These paragraphs are independently encoded through the pre-trained BERT model. Each token is represented by its associated BERT hidden state from the last layer of the model. The tokens representations then concatenated to produce a global representation of the set of 10 paragraphs of total length ni. The representations are further passed through a Bidirectional Gated Recurrent Unit
N Σ (BiGRU) (Coo et al. 2014) to produce the final representation of the document ED E N22 and question E4 R 22
where is the hidden state dimension of the BiGRUs.
E BiGRU(BERT(Q))
E BiGRU BERT (p)): ...; (PRT()
(1)
where [:] is the concatenation operation.

---

Under review as a conference paper at ICLR 2020
To compute the first representation of the question U E LR to U E M22h where M is an hyperraaaaeeter of the model. Intuitively, RMx2 corresponds to the space allocated to store the representation of the question and its further reformu- lations. It does not depend on the length of the original question L.
interpolation layer map
3.2 READING MODULE
Our model composed of T hops of reading that sequentially extract relevant information from a document regarding the current reformulation of the question. At step t, given representation of the reformulated question U RM Mx2h and representation of the document E E .R. Nx2h, this module computes a question-aware representation of the document. This module is a combination of two layers: document-question attention followed by document self-attention.
x2h
Document- Document-Questionn Attention: We first construct the interaction matrix between the document and the current reformulation of the question S R RNN as:
S = W. + U2 J.: + W !
(2)
where W1, W2 Wy trainable vectors of R and @ the deement-wiss multiplication. Then, compute the doccument-o-qquesiion attention C E RNx2h;
exp(S..)
M
Prj
(4) C
Σ j=1
(3)
M Ek= exp(
And the question-to-doeumenn attention of E 2R
m;
Sej. jE (E(....,M)
p softmax(m),
de pj Ep.
(4)
j=1
Finally, we compute the question-aware representation of the document ( E RN88h
P C E. O C%,: O C.,l
(5)
where [:] concatenation operation. Finally, we use last BiGRU that reduces the dimension of (1) to N x 2h. This specific attention mechanism was first introduced the Bidirectional Attention Flow model of Seo et al. (2017). We hypothesize that such token-level attention will produce a finer-graince representation the document compared to sentence- attention used in state-of-the-art Memory Network architectures.
Document Self-Attention: So far, the contextualization between the ten paragraphs has only be done by the BiGRUs of equation [ One limitation of the current representation of the document that each token has very limited knowledge of the other elements of the context. To deal with longerange dependencies, we apply this same attention mechanism between the question-as question-aware aware rep- resentation of the document, X(O), and itself to produce the reading module output V e N x2h This eeffccontextualizaatinn of the document has been found useful in experiments presented in the ablation analysis of Section 4.3
3.3 QUESTION REFORMULATION MODULE
reformulation module takes as input the output of the previous attention module (((,, the pre- vious representation of the reformulated question U and an encoding of the document D It produces updated reformulation of the question (t+1))
srading-based Attention: Given (t) we compute (()) e V and p(te e EN using two BiGRUs followed by linear layer and softmax operator. They are computed from:
(t)= (1))
(0e B
(6)
p (f) softmax( (erimaxxxw,YYee (()))
p(0e softmax(w, (00)),
4

---

Under review as a conference paper at ICLR 2020
where wgand w, are trainable vectors of R The two probability vectors ( and pite not used to predict an answer but to compute reading-based attention vector a!)) over the document. Intuitively, these probabilities represent the belief of the model at step f of the probability for each word to be the beginning and the end of the answer span. We define the reading-based attention of a token as the probability that the predicted span has started before this token and will end after. can be computed follows:
N
10)
pk
()))
k=
k=1
Finally, we use these attention values to re-weight each token of the document representation. We compute EOOD E Rx2x with:
E aa E.
(8)
Dynamic Max-Pooling: This layer aims at collecting the relevant elements of ((t) to add to the current representation of dimension M 2h. We partition the of the initial sequence into M approximately equal parts. It produces grid of M x 2h in which we apply a max-pooling operator in each individual window. As result, matrix of fixed dimension adequately represents the input, preserving the global structure of the document, and focusing on the important elements of each region. This can be seen as an adaptation the dynamic pooling layer proposed by Socher et al. 2011.
Formally, let B(1) be the input matrix representation, we dynamically compute the kernel size, w, of the max-pooling according to the length of the input sequence and the required output shape: N 1. [-] being the ceiling function. Then the output representation of this pooling layer will be O E Mx24 where
4.7
max Be (ftr)....(++1))a)
(SaJ).
(9)
Finally, to compute the updated representation of the question ((++1) Mx2h we sum U1)) and O(0.
3.4 ANSWERING MODULE
The answering module is a sequence of four BiGRUs, each of them followed by a fully connected layer. Their respective goal is to supervise (1) the supporting facts yat (2) the answer starting and (3) ending probabilities, p", p", of each word of the document. (4) The last layer is used as a three-way way classifier to predict p" the probability of the answer be classified as yes, no span of text.
Y BY
Y BY"
Y BiGRU(YY))
yd (V
(10)
p" = softmax(w,, (
p softmax (w, YO)
D soltmax (w, Y")
where w, E RR, w. E R W. E hxx3 are trainable parameters.
To predict the supporting facts, we construct a sentence based representation of the document. Each sentence is represented by the concatenation of its starting and ending supporting fact tokens from ysf We compute y.t the probability sentence of example of being a supporting fact with linear layer followed by sigmoid function.
3.5 MULTI-HEAD VERSION
We define a multi-head version of the model. In this configuration, we use a set of independent parallel heads. All heads are composed the same number of reading and reformulation modules. Each head produces a representation (T) of the document. We finally sum these matrices to compute the input of the answering block.
5

---

Under review as a conference paper at ICLR 2020
3.6 TRAINING
We jointly optimize the model on the three subtasks (supporting facts, span position, classifier yeshno/span) by minimising linear combination of the supporting facts loss Lat the span loss Synan and the class loss Celass Let Ng be the number of examples in the training dataset. (0 is defined by:
nbs;
1
(o)
(11)
Na
nbs;
where nbs; corresponds to the number of sentences in the document i. Vij being 1 if the sentence of the document is a supporting fact otherwise 0.
Selecting the answer in multi-hop reading datasets is a weakly supervised task. Indeed, similarly to the observations of Min al. (2019a) for open-domain question-answent and discrete reasoning tasks, it is frequent for given answer of HOTPOTQA to appear multiple times in its associated document. In our case, we assume that all mentions of the answer the supporting facts are related to the question. We tag as valid solution, the start and end positions of all occurrences of the answer the given supporting facts.
Sisan (0) is defined by:
Na Σ DKL (pily) (2)) + DKL (pf(lllu
C Span (0)
(12)
Na
where y/ "/ E ERN Dj E R are vectors containing the value 1/n; at the start, end positions of all the occurrences of the answer, 0 otherwise; n; being the number of occurrences of the answer in the context.
Lelass()) is defined by:
Na
Celass (0)
log(p (4)): 4,31
(13)
where yi corresponds to the index of the label of the question type {yes, no, span]. We finally define the training loss follows:
((0) Lelass (0) + (0)+ BCp
(14)
where and are hyperparameters tuned by cross-validation.
EXPERIMENTS
4.1 DATA AUGMENTATION
In the original HOTPOTQA dataset, the two gold paragraphs required answer given question come with eight distractor paragraphs. These eight distractor paragraphs, collected from Wikipedia, selected among the results of bigram tf-idf retriever (Chen et al. (017)) using the question as the query. As an augmentation strategy, we created additional "easier" examples by combining the two gold paragraphs with eight other paragraphs randomly selected in the dataset. For each example of the original training set, we generate an additional "easier" example. These examples are shuffled in the dataset.
4.2 IMPLEMENTATION DETAILS
Our model is composed of 3 parallel heads (K 3) each of them composed of two reading modules and one reformulation module (T 2). We set the hidden dimension of all the GRUs to d = 80. We M 100 to allocate space of R 100x160 store the question and its reformulations. We use

---

Under review as a conference paper at ICLR 2020
Answer
Sup Fact
Joint
Model
EM
F1
EM
F1
EM
F1
L-RR-eet (our)
60.20
73.78
56.21
84.09
36.56
63.68
DFGN (Qiu al. 2019
56.31
69.69
51.50
81.62
33.62
59.82
QFE (Nishida et al. 2019)
53.86
68.06
57.75
84.49
34.63
59.61
Baseline Model Yang et al. 2018
45.60
59.02
20.32
64.49
10.83
40.16
DecompRC (Min et al. 2019b
55.20
69.63
N/A
N/A
N/A
N/A
Self Self-Assemlling NMN (Jang & Bansal 2019
49.58
62.71
N/A
N/A
N/A
N/A
Performance comparison on the private test set of HOTPOTQA in the distractor setting. We compare our model, in term of Exact Match and Fj scores, against the published models at the time of submission (September 25th). Our submission is tagged as LQR-net + BERT- Base (single model) on the official leaderboard hhttps://hotpooe github. io/)
Table 1:
pre-trained BERT-base-casse model (Devlin et al. 2018) and adapt the implementation of Hugging Face to compute embedding representations documents and questions. We optimize the network using the Adam optimizer (Kingma & Ba, 2015) with an initial learning rate of le We set to and B to 10. All these parameters have been defined through cross-validation.
4.3 RESULTS ABLATION ANALYSIS
Table I presents the performance of our LQR-net on the distractor setting of the HOTPOTQA dataset. We compare model against the published approaches evaluated the HOTPOTQA dataset. We can see from this table that our model achieves strong performance on the answer prediction task. outperforms the current best model by 3.9 points of EM and 4.1 points of Fi score. Our model also achieves competitive performance for the evidence extraction task. The LQR-net achieves state-of- the-art performance the joint task improving the best published approaches by 2.9 points on EM and 3.9 points of Fi.
Answer
Sup Fact
Joint
Model
EM
Fi
EM
Fi
EM
Fi
LQR
60.0
74.1
55,8
83.9
36.5
64.0
Data aug
59.3
73.4
52.8
84.2
34.4
63.6
CE Loss
59.6
73.6
52.7
83.5
34.4
63.2
59.2
73.2
48.9
83.8
31.6
63.0
Self-Att
53.4
66.8
48.9
79.2
30.1
55.7
=
53.4
67.2
48.3
78.2
28.8
55.1
= 1
51.8
65.2
42.1
72.1
25.8
50.7
Table 2: Comparison of different architectures and model choices against the best configuration on the development of HotpotQA.
To evaluate the impact of the different components of our model, we perform ablation analysis. Table presents the results of this analysis.
Impact of sequential and parallel reading: We study the contributions of the sequentiality in the model and of the multiple parallel heads. We compare our model to a similar architecture without the sequential reformulation (I 1). We find that this sequential association of reading modules and reformulation modules is a critical component. Fi score decreases by 6.9 points for the answer prediction task and 5.7 points for the evidence extraction task when the model does not have the capability to reformulate the question.

---

Under review as a conference paper at ICLR 2020
The impact of the parallel heads is more limited than the sequentiality but still remains significant. Indeed, the configuration that uses only a single head (K' = 1) stands F1 points below the best model on the joint metric.
Weak supervision of the answer: In this work, we propose to label as positive all occurrences of the answer in the supporting facts. We compare this configuration to the standard approach, where only the first occurrence of the answer is labeled as positive and the others as negative. In this last configuration, the span loss corresponds and end probabilities and the target positions. This decreases the joint F1 score by 0.8 points.
cross-entropy loss (CE loss) between the predicted start
Impact of the self-attention layer: We study the impact of the self-attention layer in the reading module. We found that this self-attention layer is an essential component in the reading process. Indeed, when we omit this layer, the Fi score decreases by 8.3 points on the joint metric. This out- lines the necessity to be able to propagate long-range information between the different paragraphs and not only in the local neighborhood of a token. Compared to previously proposed approaches, this layer does not rely on any handcrafted relationship across words.
Question as a single vector: Finally, we study the case where the question representation reduced to vector of 2R (M = 1). This configuration achieves the worst results of our analysis, dropping the joint F1 score by 13.3 points and highlights the importance of preserving representation of the question matrix to maintain its meaning.
4.4 OPEN-DOMAN EXPERIMENTS
In this part, we describe how we integrated our model into an entire reading pipeline for open- domain question answering. In this setting, no supporting documents are associated to each ques- tion, and it is required to retrieve relevant context from large text corpora such as Wikipedia. We adopt a two-stage process, similar Chen et al. (2017); Clark & Gardner (2018), to answer multi- hop complex questions based on the 5 million documents of Wikipedia. First, we use a paragraph retriever to select a limited amount of relevant paragraphs from a Wikipedia dump, regarding a nat- ural language question. Second, fed our LQR model with the retrieved paragraphs to extract the predicted answer. We evaluate this approach on the open-domain configuration of the HotpotQA dataset called fullwiki.
We use a standard TF-IDF based paragraph retriever to retrieve the paragraphs the most related to the question. In addition to these paragraphs, we consider as relevant their neighbors in the Wikipedia graph, i.c. the documents linked to them by hyperlinks. In our experiments, we considered as relevant, the top 10 paragraphs and their associated neighbors.
Table 3 shows the results of our approach compared to other published models. Although we are using a very simple retriever, only based on TF-ID,, we report strong results on the open-domain question answering task of HotpotQA. The only published approach (Nie et al. 2019) that outper- forms us being a combination of sentence/paragraph retrieval based on BERT encodings.
4.5 QUALITATIVE ANALYSIS
Question Reformulation and Reasoning Chains: Because our model reformulates the question in a latent space, we cannot directly visualize the text of the reformulated question. However, one way assess the effectiveness of this reformulation is to analyze the evolution of pz and pe across the two hops of the model. We present in Figure S an analysis of the evolution of these probabilities on two bridge samples of the development dataset. We display the reading-based attention, that corresponds to the probabilities for each word to be part of the predicted span, computed from pº and pf in Equation 7 These examples show this attention before the first reformulation of the question and in the answering module.
From these observations, we can see that the model tends to follow a natural reasoning path to answer bridge questions. Indeed, before the first reformulation module, the attentions tend to focus on the first step of reasoning. For the question "What award did the writer of Never Let Me Go novel win in 1989?". the model tends to focus the name of the writer at the first step. before jumping the award description in the second step. Similarly, for the question "What is the population according to the 2007 population census of the city in which the National Archives and l Library of Ethiopia
8

---

Under review as a conference paper at ICLR 2020
Answer
Sup Fact
Joint
Model
EM
F1
EM
F1
EM
F1
SemanticRetrieallRRS (Nie et al.
2019)
46.50
58.80
39.90
71.5
26.6
49.2
-QR-net (our)
43.00
54.0
30.10
58.90
18.90
39.20
GOLDEN Retriever' (Qi et 2019
37.92
48.58
30.69
64.4
18.04
39.13
CogQA DDng et al. 2019)
37.60
49.40
23.10
58.5
12.2
35.3
MUPPET Feldman
Yaniy
2019)
31.07
40.42
17.00
47.71
11.76
27.62
QFE (Nishida et al. 2019
28.70
38.10
14.20
44,40
8.69
23.1
Baseline Model ( Yang et al.
2018
24.68
34.36
5.28
40.98
2.54
17.73
DecompRC (Min et al.
19b)
N/A
43.26
N/A
N/A
N/A
N/A
Table 3: Performance comparison on the development set of HOTPOTQA in the fullwiki setting.
We compare our model in terms of Exact Match and Fj scores against the published models at the time of submission (November 15th).. f indicates that the paper does not report the results the development set of the dataset; we display their results on the test set.
Queedlon:: Whal aword settee of Newer Go conal wis
Basibrr proze
Prodicted avooure aine woutoo vitoo prrtoor
Aectave and Librrry Eharlla beated? 2007 pay de ensso de city Netional
beliee Badiem en
Peodl ARsWer X.50L
Nexer Me crosel))
Novee Go si Rissspunt scieace bettos spvel jepsacacbom Bisteis Sauae Blagens liaasan premocade the UNSii for the 2002 Auther ti Chutto Awoce and bet the 2000 hotiocad Book Cittaes Strote beand.
shertiooed
11 XIUUUOO SUBA, SORCRIES 2007 peaatii omnos with exond prowts 3.2%%
The Renratos of the Day work unudtd lie Nis Booker thice ttxtirn 1550.
Nutionad Acchibee and Libeary Elhoogie
eamaine tiosary avoticns
Nevar Let New Ga (novel)
Addio Abube
trenee MeRT was the 2007 l with anead prowts
we chanttooell for the pati Banier mene swnni revey and promevely wn troms dy The Nerra the Dig
Aetter Cherko Jonudd
Nutional Book cittics Curcs Joaand
Nutionad Ancibee and Library Elhaple ol Uhe ccartyy
The Rematiss of Day
The werk vea eveeded the Bil BOORER BSER BE Dicbis in 1903.
Figure 3: Distribution of the probabilities for each word to be part of the predicted span, before the first reformulation module and in the answering module. We display the reading- attention computed in Equation and the reading-based attention computed from p and pd from Equation In these examples, we show only the supporting facts.
10
located?" "
the model focusing on Addis Ababa the first step, i.e the name of the city where the National Archives and Library of Ethiopia are located and then jumping to the population of this city in the next hop. We display more visualizations of the sequential evolution of the answer probabilities in Appendix A
Limitations: We manually examine one hundred errors produced by our multi-step reading archi- tecture on the development set of HOTPOTQA We identify three recurrent cases of model failure: (1) the model stops at the first hop of required reasoning. erties, and (3) the answer does not match all the requirements of the question. We illustrate these three recurrent types of error with examples from the dataset Appendix B
(2) the model fails comparing two prop-
During this analysis of errors, we found that in only 3% of the cases, the answer is selected among one of the distractor paragraphs instead of gold one. Our architecture successfully detects the rele- vant paragraphs regarding question even among similar documents coming from a tf-idf retriever. Moreover, there are no errors where the model produces a binary yes/no answer instead of extracting text span and vice versa. Identifying the type of question is not challenging for the model. This might be explained by the question's "patterns" that are generally different between binary yes/no and extractive questions.
RELATED WORK
Multi-hop Machine Comprehension: The cresttio-anssering task has recently increased its pop- ularity way to assess machine reading comprehension capabilities. The emergence of large

---

Under review as a conference paper at ICLR 2020
scale datasets such as CNN/Daily Mail, (Hermann al. 2013). SQuAD (Raiuurkrr al. 2016)
MSMARCO (gguyen et al. 2016) have encouraged the development of multiple machine reading models (Devlin et al. 2018 Wang et al. 2018 Tan al. 2017). These models mainly com- posed of multiple attention layers that update the representation of the document conditioned by a representation of the question.
However, most of this work focuses on the ability to answer questions from a single paragraph, often limited to a few sentences. [Weston et al. (2015a); Joshi et al. (2017) were the first attempts to introduce the task of multi-documents question-answering QAngaroo (Welbl et al. (2018) is another dataset designed evaluate multi-hop reading architectures. However, state-of ofate-of-the- the-hhtetttt art architectures on this task (Zhong et al. [2019] [Cao et al. (2019) tend to exploit the structure of the dataset by using the proposed candidate spans as an input of the model.
Recently, different approaches have been developed for HoTPOTQA focusing on the multiple chal- lenges of the dataset. [Nshida et al. (2019) focuses on the evidence extraction task and highlight its similarity with the extractive summarization task. Related works also focus on the interpretation of the reasoning chain with an explicit decomposition of the question (Min et al. 2019b) or a de- composition of the reasoning steps (Jiang & Bansal 2019). Other models like Qiu et al. (2019) aim at integrating a graph reasoning type of attention where the nodes are recognized by a BERT NER model over the document. Morcover, this model leverages on handcrafted relationships between tokens.
Related to approach, different papers have investigated the idea of question reformulation to build multi-hop open-domain question answering models. Das et al. (2019) proposes a framework composed of iterative interaction between document retriever and reading model. The question reformulation is performed by a multi-step-reasooer module trained via reinforcement learning- Similarly. Feldman EI- Yaniv (2019) introduces multi-hop paragraph retriever. They propose a reformulation component integrated into a retrieving pipeline to iteratively retrieve relevant docu- ments. These works are complementary to ours by focusing mostly on the document retrieving part of the problem while we focus on the answer extraction task, and could be combined together.
Memory Networks: Memory networks are a generic type of architecture Weston et al. (2015b); Sukhbaatar et al. (2015): Miller et al. (2016) designed to iteratively collect information from memory cells using attention mechanism. They have been used to read from sentences, paragraphs, and knowledge bases. In these models, the answer layer uses the last value of the controller to predict the answer. Two main differences with our architecture are the representation of the controller and the associated attention mechanism. Indeed, in these models, the controller is reduced to a single vector, and the attention mechanism is based on a simple dot-product between each token of the document and the representation of the controller. We utilize a token-level attention mechanism compared to the sentence-level one, classically used Memory Networks.
The transformer architecture has been introduced by Vaswani et al. (2017) in the context of machine translation. It is mainly composed of attention layers in both the encoder and the decoder module. The transformer networks introduced the so-called multi-head attention, consisting of several attention layers rumming in parallel. This multi-head attention allows the model to concurrently access information from different representations of the input vector. Inspired by this work, we designed our multi-head module to read in parallel into different representations of the document while solely accumulate information into the representation of the question.
Transformer Networks:
6 CONCLUSION
In this paper, we propose novel multi-hop reading model designed for question question-answering answering tasks that explicitly require reasoning capabilities. We have designed our model to gather information sequentially and in parallel from a given set of paragraphs to answer natural language question. Our neural architecture, uses a sequence of token-level attention mechanisms to extract relevant information from the paragraphs and update a latent representation of the question. Our proposed model achieves competitive results on the HOTPOTQA reasoning task and performs better than the current best published approach in terms of both Exact Match and Fi score. In addition, we show that an analysis of the sequential attentions can possibly provide human- inteepraateeee reasoning chains.
10

---

Under review as a conference paper at ICLR 2020
REFERENCES
Lisa Bauer, Yicheng Wang, and Mohit Bansal. Commonsense for generative multi-hop question answering tasks. Ellen Riloff, David Chiang, Julia Hockenmaier, and aun'ichi Tsujii (eds.), Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Brussels, Belgium, October 31- November 4, 2018, pp. 4220-4230. Association for Compu- tational Linguistics, 2018. URL
d18-1454
Nicola De Cao, Wilker Aziz, and Ivan Titov. Question answering by reasoning across documents with graph convolutional networks. In Jill Burstein, Christy Doran, and Thamar Solorio (eds.), Proceedings of the 2019 Conference of the North American Chapter of the Association for Com- putational Linguisties: Human Language Technologies, NAACL-HI.T 2019, Minneapolis, MN, USA, June 2-7, 2019, Volume (Long and Short Papers), pp. 2306-231777771 Assooiaat tational Linguistics, 2019, URLhttps: www Sttt::/://ww....l...bbboor/////1 1240/
Association for Compu-
Danqi Chen, Adam Fisch, Jason Weston, and Antoine Bordes. Reading wikipedia to answer open- domain questions. In Proceedings of the 55th Annual Meeting of the Association for Computa- tional Linguistics, ACL 2017, Vancouver, Canada, July 30 August 4, Volume Long Papers, doi: 10.18653/1//11-11777 URL sttps://dolt mropiiorgg100 10tp///311111//// 18653/
pp. 1879-1879, 2017.
01/P17-1177
Kyunghyun Cho, Bart van Merrienboer, Çaglar Gülçehre, Dzmitry Bahdanau, Fethi Bougares, Hol- ger Schwenk, and Yoshua Bengio. Learning phrase representations using RNN encoder-decoder for statistical machine translation. In Alessandro Moschitti, Bo Pang, and Walter Daelemans (eds.), Proceedings of the 2014 Conference on Empirical Methods in Natural Language Pro- cessing, EMNLP 2014, October 25-29, 2014, Doha, Qutar, A meeting of SIGDAT, Special Interest Group of the ACI. pp. 1724-1734. ACI,, 2014. URL https: /tte:///wwww /www ac web. org/
anthology/D14- 1179/
Christopher Clark and Matt Gardner. Simple and effective multi-paragraphe reading comprehension. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, ACL 2018, Melbourne, Australia, July 15-20, 2018, Volume 1: Long Papers, pp. 145-855, 2018. URL httpss///ccaaahhloogy.. http:oloww..nnnoappsss/eeeeee..e
1178
Rajarshi Das, Shehzaad Dhuliawala, Manzil Zaheer, and Andrew McCallum. Multi-step retriever- reader interaction for scalable open-domain question answering. In International Confer- ence Learning Representations, 2019. URL htttp::://ppnrevieew net wttprrrvwwwe.eeeeeooummaaaaa
HkfPSh05K7
Jacob Devlin, Ming- Wei Chang, Kenton Lee, and Kristina Toutanova. BERT: pre-training of deep bidirectional transformers for language understanding. CoRR, abs/1810.04805, 2018. URL
httt:://arxivv org wtwpg//ab///1...ooo 04805
Ming Ding, Chang Zhou, Qibin Chen, Hongxia Yang, and Jie Tang. Cognitive graph for multi- hop reading comprehension at scale. In Proceedings of the 57th Annual Meeting of the Associ- ation for Computational Linguistics, pp. 2694-2703, Florence, Italy, July 2019. Association for Computational Linguistics. doi: 10.18653/v1/P19- 1259. URL www. aclweb. org/
anthology/PB 19 1259
Yair Feldman and Ran El-Yami Yaniv. Multi-hop paragraph retrieval for open-domain question answering. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp- 2296-2309, Florence, Italy, July 2019. Association for Computational Linguistics. doi: 10.18653/
v1/P19-1222. URL https: www wwwwwwlwbbe....
wrg. wwtpbb..rgggggttatoaeeoe wtt/anttoloog///ii... 1222
Karl Moritz Hermann, Tomás Kocisky, Edward Grefenstette, Lasse Espeholt, Will Kay, Mustafa Suleyman, and Phil Blunsom. Teaching machines to read and compre- hend. In Advances in Neural Information Processing Systems 28: Annual Confer- on Neural Information Processing Systems 2015, December 7-12, 2015. Montreal, Quebec. Canada, pp. 1693-1701. 2015. URL 1945-teaching-mee nes to read compeehennd
nips cc/paper/

---

Under review as a conference paper at ICLR 2020
Yichen Jiang and Mohit Bansal. Self-assembling modular networks for interpretable multi-hop reasoning. CoRR, 0tts/1909.0550 2019. URL htttt:://aaxi..or//aaaaa. www.ab/s19990000 05803
Mandar Joshi, Eunsol Choi, Daniel S. Weld, and Luke Zettlemoyer. Triviaqa: A large scale distantly supervised challenge dataset for reading comprehension. In Regina Barzilay and Min- Yen Kan (eds.), Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, ACL 2017, Vancouver. Canada, July 30 August 4, Volume 1: Long Papers, pp. 1601-1611. Association for Computational Linguistics, 2017. doi: 10.18655//1/117-114477 URL https: 7/doi.org// 18t...11/333/11111111 1147
Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In Yoshua Bengio and Yann LeCun (eds.), 3rd International Conference on Learning Representations, ICLR 2015, San Diego CA, USA, May 7-9, 2015, Conference Track Proceedings, 2015. http:
//arxiv.org/abitt /142. 6980
Alexander H. Miller, Adam Fisch, Jesse Dodge, Amir-Hossein Karimi, Antoine Bordes, and Ja- son Weston. Key-value memory networks for directly reading documents. In Jian Su, Xavier Carreras, and Kevin Duh (eds.). Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, EMNLP 2016, Austin, Texas, USA, November 1-4, 2016, pp- 1409. The Association for Computational Linguistics, 2016. URL http://aclweb)
pdf
Sewon Min, Danqi Chen, Hannaneh Hajishirzi, and Luke Zettlemoyer. A discrete hard EM approach for weakly supervised question answering. CoRR, abs/1909 1909,04849, 2019a. URL
http://arxiv 19tp://0/bb999999499999
Sewon Min, Victor Zhong, Luke Zettlemoyer, and Hannaneh Hajishirzi. Multi-hop reading compre- hension through question decomposition and rescoring. In Anna Korhonen, David R. Traum, and Lluís Mårquez (eds.), Proceedings of the 57th Conference of the Association for Com- putational Linguistics, ACI. 2019, Florence, Italy, July 28- August 2, 2019, Volume 1: Long Papers, pp. 6097-6109. Association for Computational Linguistics, 2019b. URL https: 7/ 7/www. wwwwww.wwbb...... wttp:///oonooooooo/eee-------- 1613/
Tri Nguyen, Mir Rosenberg, Xia Song, Jianfeng Gao, Saurabh Tiwary, Rangan Majumder, and Li Deng. MS MARCO: A human generated machine reading comprehension dataset. In Proceed- ings of the Workshop Cognitive Computation: Integrating neural and symbolic approaches 2016 co-located with the 30th Annual Conference on Neural Information Processing Systems (NIPS 2016), Barcelona, Spain, December 2016., 2016. URL http:://///.. /ceur-e---e// ws orgg/ Vo1-1773/CoCoNT
2016 wwwppa.pppp...... p
Yixin Nic, Songhe Wang, and Mohit Bansal. Revealing the importance of semantic retrieval for machine reading at scale. In Proceedings of the 2019 Conference on Empirical Methods in Nat- ural Language Processing and the 9th International Joint Conference on Natural Language Pro- cessing (EMNLP-IJCH) -JCCLLP, pp. 2553-2566, Hong Kong, China, November 2019. Association for Computational Linguistics. doi: 10. 10.18653/v1/D19-1222 URL (tLttt::: htttpp::/www...... www.aclweb.
att/anntheoogy/D115
Kosuke Nishida, Kyosuke Nishida, Masaaki Nagata, Atsushi Otsuka, Itsumi Saito, Hisako Asano, and Junji Tomita. Answering while summarizing: Multi-task learning for multi-hop QA, with ev- idence extraction. In Anna Korhonen, David R. Traum, and Lluís Marquez (eds.), Proceedings of the 57th Conference of the Association for Computational Linguistics, ACI. 2019, Florence, Italy, July 28- August 2, 2019, Volume 1: Long Papers, pp. -335-2345 Association for Computational Linguistics, 2019. URL htt.stt:s:/ www. ac wew. .og.eb.rr/// http.o/////aoooooooo11111122
Peng Qi. Xiaowen Lin, Leo Mehr, Zijian Wang, and Christopher D. Manning. Answering complex open-domain questions through iterative query generation. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Confer- ence Natural Language Processing (EMNLP-IJCNLP,, pp. 2590-2600 2602, Hong Kong, China, November 2019. Association for Computational Linguistics. doi: 10.18653/v1/DI 1261. URL https:/ // www. c web. www.orgg/atthol.oeooooo 9-
12

---

Under review as a conference paper at ICLR 2020
Lin Qiu, Yunxuan Xiao, Yanru Qu, Hao Zhou, Lei Li, Weinan Zhang, and Yong Yu. Dynami- cally fused graph network for molti-hop reasoning. In Anna Korhonen, David R. Traum, and Lluis Märquez (eds.), Proceedings of the 57th Conference of the Association for Computational Linguistics, ACI. 2019, Florence, Italy, July 28- August 2, 2019, Volume 1: Long Papers, pp- 6140-6150. Association for Computational Linguistics, 2019. htLtttss:: www.www..... ac web. http//aahhooooo11laa1o
Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. Squad: 100, 000+ questions for
machine comprehension of text. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, EMNLP 2016, Austin, Texas, USA, November 1-4, 2016, pp- 2383-2392, 2016. URLhttp:: aacl web. org/ hatp//anth..... http///g////////eeeeeegp 1264 pdf
Minjoon Seo, Aniruddha Kembhavi, Ali Farhadi, and Hananneh Hajishirzi. Bidirectional atten- tion flow for machine comprehension. In Interaational Conference on Learning Representations, 2017.
Richard Socher, Eric H. Huang, Jeffrey Pennington, Andrew Y. Ng, and Christopher D. Manning. Dynamic pooling and unfolding recursive autoencoders for paraphrase detection. In Proceedings of the 24th International Conference on Neural Information Processing Systems, NIPS' 11, pp. 801-809, USA, 2011. Curran Associates Inc, ISBN 978-1-61839-599-3. URL http://dl.
acm. o.m.org//e http://iiaiiiiooo www.:dmmmmmmm.....m.m 2986549
Saku Sugawara, Kentaro Inui, Satoshi Sekine, and Akiko Aizawa. What makes reading comprehen-
sion questions easier? In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Brussels, Belgium, October 31 November 4, 2018, pp. 4208 4219, 2018. URL pestps:/aaclanthhn wtt..ino.ppppprr////...//o////.a 8- 453/d18 1453
Sainbayar Sukhbaatar, Arthur Szlam, Jason Weston, and Rob Fergus. End-to-end mem- ory networks. In Advances in Neural Information Processing Systems 28: Annual Con- ference on Neural Information Processing Systems 2015, December 7-12, 2015, Montreal, Quebec, Canada, pp. 2440-2448, 2015. URL http://papes:: nips cc cc/paper/
Chuanqi Tan, Furu Wei, Nan Yang, Weifeng Lv, and Ming Zhou. S-net: From answer extraction to answer generation for machine reading comprehension. CoRR, abs/1706.04815. 2017. URL
http://arxiv.. www.wawwww........aaaaaaaaaaaaaa
04815
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz. Kaiser, and Illia Polosukhin. Attention is all you need. In Isabelle Guyon, Ul- rike von Luxburg, Samy Bengio, Hanna M. Wallach, Rob Fergus, S. V. N. Vishwanathan, and Roman Gameet (eds.), Advances in Neural Information Processing Systems 30: An- nual Conference on Neural Information Processing Systems 2017, 4-9 December 2017, Long Beach, CA, USA, pp. 6000 6000-6010, 2017.
URL http://papers: nips. cc cc/paper/
71Bl-attemntoon-s--all--
Wei Wang, Ming Yan, and Chen Wu. Multi goltiigrrnuularity hierarchical attention fusion networks for reading comprehension and question answering. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Lonng Papers), volume 1, pp. 1705-1714, 2018.
Wenhui Wang. Nan Yang, Furu Wei, Baobao Chang, and Ming Zhou. Gated self matching networks for reading comprehension and question answering. In Proceedings of 55th Annual Meeting of the Association for Computational Linguistics, ACL 2017 Vancouver, Canada, July 30 - August
4, Volume 1: Long Papers, pp. 189-198, 2017. doi: 11.18653/1//17--1018 URL https:
//doi. .rg ptgg/o..165..0 wtt..18653vv////
1018
Johannes Welbl, Pontus Stenetorp, and Sebastian Riedel. Constructing datasets for multi-hop read- ing comprehension across documents. Transactions of Association for Computational Linguis- tics (TACL), 6:287 2018. htt.ittts///tassccl..ooo /itp///x///////xx/./// https//ooosonnneee.pp//aaaaaaal php/tac1/
art articl wtwpll/vvwwwwei
13

---

Under review as a conference paper at ICLR 2020
Jason Weston, Antoine Bordes, Sumit Chopra, and Tomas Mikolov. Towards ai-complete question answering: A set prerequisite toy tasks. CoRR, abs/1502.05698, 2015a.
URL http://
arxiv.org/ale abs /1502. 05698
Jason Weston, Sumit Chopra, and Antoine Bordes. Memory networks. CoRR, abs/1410.3916. 2015b.
Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Bengio, William W. Cohen, Ruslan Salakhutdinov, and Christopher D. Manning. HotpotQA: dataset for diverse, explainable multi-hop question answering. In Proceedings of the Conference on Empirical Methods in Natural Language Pro- cessing (EMNLP), 2018.
Adams Wei Yu, David Dohan, Quoc Le, Thang Luong, Rui Zhao, and Kai Chen. Fast and accurate reading comprehension by combining self-attention and convolution. In International Confer- ence on Learning Representations, 2018. URL
net/ee/wwwwwweeeeeeeeee.eeeeeeee formm??rrmmm ?id=
B14TIG- RW
Victor Zhong, Caiming Xiong, Nitish Keskar, and Richard Socher. Coarse-grain fine-grain coatten- tion network for multi-evidence question answering. In International Conference on Learning Representations, 2019.
14

---

Under review as a conference paper at ICLR 2020
A SEQUENTIAL EVOLUTION OF THE ANSWER SPAN PROBABILITIES
This section includes examples from the HOTPOTQA A development set that illustrate the evolution of the probabilities for each word be part of the predicted span, before the first reformulation module and in the answering module presented in Section 4.5 For each example, we show only the text of the two gold paragraphs. identifies the supporting facts these visualizations.
Question: What is the population according to the 2007 population census of the city in which the National Archives and Library of Ethiopia is located?
Answer: 3,384,569 (2.00)
Predicted answer: 3,384,569
****.** Before Reformulation ****.*s
Addis Ababa
Addis Ababa (Amharic: M& M0 "Addis Abäba "1, "new flower"; Oromo: "Finfinne", 'Naatrrl Spring(s)") Addis Abeba (the spelling used by the official Ethiopian Mapping Authority), is the capital and largest city of Ethiopia. It has population of 3,384,569 according to the 2007 population census, with annual growth rate of 3.8%. This number has been increased from the originally published 2,738,248 figure and appears to be still largely
underestimated.
National Archives and Library of Ethiopia
The National Archives and Library Ethiopia, located Addis Ababa is the national library and archives the country,
The library was inaugurated 1944 by Emperor Haile Selassie and began service with books donated by the emperor.
......* After Reformulation
Addis Ababa
Addis Ababa (Amharic: Mih Man "Addis Abäba "], "new flower"; Oromo: "Finfinne", "Natural Spring(s)") or Addis
Abeba (the spelling used the official Ethiopian Mapping Authority), is the capital and largest city of Ethiopia. It has population of 3,384,569 according to the 2007 population census, with annual growth rate of 3.8%. This number has been increased from the originally published 2,738,248 figure and appears to be still largely underestimated.
National Archives and Library of Ethiopia
The National Archives and Library of Ethlopla, located in Addis Ababa, is the national library and archives of the country.
The library inaugurated in 1944 by Emperor Haile Selassie and began service with books donated by the emperor.
Question: Vince Phillips held Junior welterweight title by organization recognized by what larger Hall of Fame?
Answer: International Boxing Hall Fame
Predicted answer: international boxing hall of fame
******* Before Reformulation ***..**
International Boxing Federation
The International Boxing Federation (IBF) is one of four major organizations recognized by the International Boxing Hall of Fame (IBHOF) which sanction world championship boxing bouts, alongside the World Boxing Association (WBA), World Boxing Council (WBC) and World Boxing Organization (WBO).
Vince Phillips
Vincent Edwards "Vince" Phillips (born July 23, 1963) is an American former professional boxer who competed from 1989 2007.
He held the IBF junior welterweight title from 1997 to 1999 notably scoring an upset victory against then
undefeated Kostya Iszyu become champion.
**...** After Reformulation
International Boxing Federation
International Boxing Federation (IBF) one four major organizations recognized by International Boxing Hall of Fame [IHHOF] which sanction world championship boxing bouts, alongside the World Boxing Association (WBA), World Boxing Council (WBC) and World Boxing Organization (WBO).
Vince Phillips
Vincent Edwards "Vince" Phillips (born July 1963) American former professional boxer who competed from
1989 to 2007.
He held the IBF junior welterweight title from 1997 1999, notably scoring upset victory against then- undefeated Kostya Tszyu to become champion.
15

---

Under review as a conference paper at ICLR 2020
Question: What award did the writer of Never Let Me Go novel win in 1989?
Answer: Booker Prize for Fiction
Predicted answer: man booker prize for fiction
......* Before Reformulation **....s
Never Let Me (novel)
Never Let Me Go is 2005 dystopian science fiction novel by Japanese-born British author Kazuo Ishiguro.
It was shortlisted for the 2005 Booker Prize (an award Ishiguro had previously won 1989 for "The Remains the
Day"), for the 2006 Arthur C. Clarke Award and for the 2005 National Book Critics Circle Award.
Time" magazine named the best novel 2005 and included the novel in its 'TIME 100 Best English-language
Novels from 1923 to 2005".
It also received an ALA Alex Award in 2006.
film adaptation directed by Mark Romanek released in 2010; a Japanese television drama aired 2016.
The Remains
Day
* The Remains of the Day is 1989 novel by British writer Kazuo Ishiguro
The work was awarded Man Booker Prize Fiction in 1989.
A film adaptation of the novel, made in 1993 and starring Anthony Hopkins and Emma Thompson, nominated for eight Academy Awards.
After Reformulation
Never Let Me Go (novel)
Never Let Me Go is a 2005 dystopian science fiction novel by Japanese-born British author Kazuo Ishiguro, 1989 for "The Remains of the Day"), for the 2006 Arthur C. Clarke Award and for the 2005 National Book Critics Circle Award.
It shortlisted for the 2005 Booker Prize award Ishiguro had previously
Time" magazine named the best novel 2005 and included the novel in its "TIME 100 Best English-language
Novels from 1923 to 2005".
It received ALA Alex Award 2006.
A film adaptation directed by Mark Romanek was released in 2010; a Japanese televiston drama aired in 2016.
The Remains of the Day
The Remains of the Day is a 1989 novel by British writer Kazuo Ishiguro. The work was awarded the Man Booker Prize for Fiction in 1989. A film adaptation the novel, made in 1993 and starring Anthony Hopkins and Emma Thompson, was nominated for eight Academy Awards.
16

---

Under review as a conference paper at ICLR 2020
Question:Tysons Galleria is located in what county?
Answer: Fairfax County
Predicted answer: fairfax county
**...* Before Reformulation **sss*x
McLean, Virginia
McLean () is census-designated place (CDP) Fairfax County in Northern Virginia.
McLean home many diplomats, bustnessmen, members Congress, and high-ranking government officials proximity Washington, D.C. and the Central Intelligence Agency.
partially due
It location of Hickory Hill, the former home Ethel Kennedy, the widow Robert F. Kennedy,
It also the location of Salona, the former home of Light-Horse Harry Lee, the Revolutionary War hero. The community had an estimated total population of 53,673 in 2015, according to estimates prepared by the United States Census Bureau.
It is located between the Potomac River and the town of Vienna. McLean known
luxury homes and its high-end shopping destinations: the Tysons Corner Center and the
Tysons Galleria.
The two McLean zip codes 22101 and 22102 are among most expensive States.
Virginia United
Tysons Galleria
Tysons Galleria is three-level super-regional mall owned by General Growth Properties located at 2001 International Drive, McLean, Virginia, in Tysons Corner: It is the second- larond- mall in McLean/Tysons Corner, and one of the largest in the Washington metropolitan area.
After Reformulation ****..*
McLean, Virginia
McLean () is bensus-designed place (CDP) Fairfax County in Northern Virginia.
McLean home many diplomats, businessmen, members Congress, and high-ranking government officials partially due its proximity to Washington, D.C. and the Central Intelligence Agency.
It location Hickory Hill, the former home Ethel Kennedy, the widow Robert Kennedy. It is also the location of Salona, the former home of Light-Horse Harry Lee, the Revolutionary War hero. The community had an estimated total population of 53,673 in 2015, according to estimates prepared by the United States Census Bureau.
It is located between the Potomac River and the town of Vienna. McLean known for its luxury homes and its high-end shopping destinations: the Tysons Corner Center and the Tysons Galleria.
The two McLean zip codes 22101 and 22102 are among States.
expensive Codes Virginia and the United
Tysons Galleria
* Tysons Galleria is three-level super-regional mall owned by General Growth Properties located at 2001 International Drive, McLean, Virginia, in Tysons Corner. It is the second-largest mall in McLean/Tysons Corner, and one of the largest in the Washington metropolitan area.
17

---

Under review as a conference paper at ICLR 2020
Question: John MacGregor, Baron MacGregor of Pulham Market was educated at the University of St Andrews and another university established what monach?
Answer: King George IV
Predicted answer: king george iv
Before Reformulation
King's College London
King's College London (Informally King's or KCL) is a public research university located in London, United Kingdom, and founding constituent college of the federal University of London.
King's was established in 1829 by King George IV and the Duke of Wellington, when it received its first royal charter (as college), and claims to be the fourth oldest universily in England.
In 1836, King's became one of the two founding colleges of the University of London.
the late 20th century, King's grew through series of mergers, including with Queen Elizabeth College and Chelsea College of Science and Technology (in 1985), the Institute of Psychiatry (in 1997), the United Medical and Dental Schools of Guy's (In 1998).
Thomas' Hospitals and the Florence Nightingale School of Nursing and Midwifery
John MacGregor Baron MacGregor of Pulham Market
John Roddick Russell MacGregor, Baron MacGregor of Pulham Market, OBE PC FKC (born 14 February 1937), politician in the United Kingdom.
He was educated at Merchiston Castle School, then at the University of St Andrews (MA economics and history. 1959) and at King's College London (LLB, 1962).
Prior to the 1979 general election he worked for Hill Samuel, merchant bank.
After Reformulation
King's College London
King's College London (informally King's or KCL) is a public research university located in London, United Kingdom, and founding constituent college of the federal University of London.
King's was established in 1829 by King George IV and the Duke of Wellington, when it received its first royal charter (as college), and claims
the fourth oldest university
England.
In 1836, King's became one of the two founding colleges of the University of London.
the late 20th century, King's grew through series of mergers, including with Queen Elizabeth College and Chelsea College of Science and Technology (in 1985), the Institute of Psychiatry (in 1997), the United Medical and Dental Schools of Guy's and St Thomas' Hospitals and the Florence Nightingale School of Nursing and Midwifery (in 1998).
John MacGregor, Baron MacGregor of Pulham Market John Roddick Russell M Baron MacGregor of Pulham Market, OBE PC FKC (born 14 February 1937), is politician in the United He was educated at Merchiston Castle School, then at the University of St Andrews (MA economics and history, 1959) and at King's College London (LLB, 1962).
Prior to the 1979 general election he worked for Hill Samuel, merchant bank.
B ERROR ANALYSIS
This section includes examples from the HOTPOTQA development set that illustrate the categories of errors presented in Section 4.5 For each example, we show only the text of the two gold paragraphs. * identifies the supporting facts in these visualizations.
The model stops at the first hop of required reasoning:
Question: What is one of the stars of The Newcomers known for? Answer: superhero roles as Marvel Comics
Predicted answer: chris evans
Chris Evans (actor)
* Christopher Robert Evans (born June 13, 1981) is an American actor and filmmaker.
Evans is known for his superhero roles as Marvel Comics characters Steve Rogers / Captain America in Marvel Cinematic Universe and Johnny Storm/ Human Torch in "Fantastic Four" and.
The Newcomers (film)
The Newcomers is a 2000 American family drama film directed by James Allen Bradley and starring Christopher McCoy, Kate Bosworth, Paul Dano and Chris Evans. Christopher McCoy plays Sam Docherty, a boy who moves to Vermont with his family, hoping to make a fresh start away from the city. It was filmed in Vermont, and released by Artist View Entertainment and MTI Home Video.
The model fails at comparing two properties:
18

---

Under review as a conference paper at ICLR 2020
Question: Who younger, Wayne Coyne or Toshiko Koshijima?
Answer:
Toshiko Koshijima
Predicted answer: wayne michael coyne)
Wayne Coyne
Wayne Michael Coyne(born January 13, 1961) is an American musician. He the lead singer, occasional backing vocalist, guitarist, keyboardist, theremin player and songwriter for the band the Flaming Lips.
Toshiko Koshijima
Toshiko Koshijima ELUE ELC, Koshijima Toshiko ,born March 3, 1980 in Kanazawa, Ishikawa) a Japanese singer:
Along with composer, record producer and DJ Yasutaka Nakata, she is lead vocalist of the electronica band Capsule, which they formed 1997 when both were 17.
Their formal debut came in 2001 with the release of the single "Sakura".
Two more singles and their debut album, "High Collar Girl", followed the same year.
The answer does not match all the requirements of the question:
Question: Approximately how many locations BJ's Wholesale Club operating in, as early 2008? Answer: 650 locations Predicted 500
US Vision
U.S. Vision, wholly owned subsidiary of Refac Optical Group, is international optometric dispensary chain.
The vast majority of these locations are leased spaces in large department stores, such J.C. Penney, Boscov's, and The Bay.
As of May 8, 2007, 500 locations in 47 states and Canada are in operation, consisting of licensed departments and freestanding stores.
in early 2008, due to an acquisition of BJ's Optical Centers located in many BJ's Wholesale Clubs, that number has grown to approximately 650 locations.
U.S. Vision deals mainly in prescription eyewear, contact lenses, and optometry offices.
BJ's Wholesale Club
BJ's Wholesale Club Inc., commonly referred simply BJ's, is an American membership-only warehouse club chain operating on United States East Coast, as well as in the state Ohio.
19


Reviews:
[{"review_id": "HJlGvnVTKS", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper proposes to iteratively reformulate questions in the latent space for multi-hop question answering. The reformulation of the question depends on the question-aware representation of the documents. \n\nThe authors experiment their model on the HotpotQA dataset and achieve the state of the art performance. But, it's important to experiment with some other datasets. One option could be the Complex WebQuestions [1]. \n\nSince T is set to 2 in your experiment, is it possible that your model simply predict the intermediate answers and use it to reformulate the question representation? In Talmor et. al paper [1], the SplitQA splits the questions into two subquestions, and appends the answer of the first subquestion to the second subquestion.\n\nAnother related work is PullNet by Sun et. al [2].\n\n[1] Talmor, Alon, and Jonathan Berant. \"The web as a knowledge-base for answering complex questions.\" arXiv preprint arXiv:1803.06643 (2018).\n[2] Sun, Haitian, Tania Bedrax-Weiss, and William W. Cohen. \"PullNet: Open Domain Question Answering with Iterative Retrieval on Knowledge Bases and Text.\" arXiv preprint arXiv:1904.09537 (2019).", "extra_scores": {"experience_assessment": "I have published one or two papers in this area.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "N/A"}, "extra_text": {}}, {"review_id": "Bkg7wynJYB", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The authors propose a multi-hop latent question reformulation system that performs well in the question-answering setup. The system achieves best current published result on the HotpotQA dataset.\n\nThe system achieves that using question-aware representation of the document.\n\nQuestion: would it be possible to re-generate, at least in an approximate form, one of the reformulations of the question, using decoding? It seems that the visualization of these intermediate forms would allow to understand the model better.", "extra_scores": {"experience_assessment": "I have published one or two papers in this area.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I did not assess the derivations or theory.", "review_assessment:_checking_correctness_of_experiments": "I did not assess the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I made a quick assessment of this paper."}, "extra_text": {}}, {"review_id": "B1gKAOzGcS", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper proposes a model for multi-hop question answering, specifically in a closed domain setting where the relevant paragraphs are present within a few distractor paragraphs. Their model has the following components — (a) a reader module that reads and collect information from paragraphs, (b) a query reformulation module, that reformulates the query to retrieve the next relevant document required to answer the question. Following the effectiveness of multi-head attention, the reader and reformulation module has K-heads that forms independent reformulations. The k-heads are aggregated via a simple summing and the reformulated query representation is used in the next step of the pre-defined hop. \n\nSpecifically, they start by encoding the question and paragraphs by BERT embeddings. Next they concatenate the fixed set of paragraphs (10 in their experiments) and encode it with a recurrent NN (bi-GRU) to produce token-level recurrent representation. This is followed by a reading module which dies document question attention in BiDAF (Seo et al., 2017) style followed by a self attention module. Next, in the query reformulation phase a convolution style filter is passed across the token embeddings of the document to gather / pool information required for querying. The current query representation is added to the new pooled representation to obtain the updated query representation. This is followed by an answering module that runs 4 layers of Bi-GRU and has an outlet for computing loss for each of the following — supporting facts , start , end of the answer span representation and a classifier to determine if its a yes-no question or a span answer. The network is trained via supervision for all the aforementioned 4 outlets. \n\nEmpricially, this paper shows gain in the distractor setting for HotpotQA dataset. \n\n\nStrengths: \nQuery reformulation is an important strategy for IR, and multi-hop QA and it is nice to see that this model adapts it. Each module of the network seems to be carefully designed and the ablation results and analysis are helpful.\n\nWeaknesses:\n1. Related work: One of the major weakness of the paper is the missing related work. There are two well-established paper Das et al (ICLR 2019) — Multi-step Retriever-Reader Interaction for Scalable Open-domain Question Answering and Feldman and Yaniv (ACL 2019) — Multi-Hop Paragraph Retrieval for Open-Domain Question Answering that show query reformulation is effective for multi-hop question answering. Das et al 2019 proposes that the query reformulation module is independent of the reader module as long as it has access to the hidden representation of the reader module. This paper builds off from the original papers by carefully designing the reader and the reformulation modules (which is great!), but never mentions any of the above papers. I think that should be fixed.s\n2. The setting considered in this paper is closed-domain where the number of paragraphs to be read are fixed and pre-determined. That is a very unrealisitic setting for a general purpose QA system. The paper should consider testing on the open domain setting of the HotpotQA dataset. The other two papers mentioned above (Das et al 2019, Feldman & Yaniv 2019) both test on open-domain setting. Moreover, it has been also shown recently that the distractor setting can easily be fooled and is not a great benchmark for testing the reasoning capabilities.\n3. For a real QA system to be deployed in production setting, the system needs to be fast. I am afraid the model proposed by this paper is very computationally expensive. Apart from encoding the question and paragraphs by BERT, they have multiple Bi-GRU encodings (8 additional). That is going to be computationally intensive and the authors should strongly consider other approaches such as replacing bi-grus with transformers that can encode sequences in parallel, and parameter sharing. \n\nFor the above strong weaknesses, I am forced to give a low score to the current paper. ", "extra_scores": {"experience_assessment": "I have published one or two papers in this area.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper thoroughly.", "review_assessment:_checking_correctness_of_experiments": "I carefully checked the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "N/A"}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper proposes a novel approach, Latent Question Reformulation Network (LQR-net), a multi-hop and parallel attentive network designed for question-answering tasks that require multi-hop reasoning capabilities. Experimental results on the HotPotQA dataset achieve competitive results and outperform the top system in terms of exact match and F1 scores. However, reviewers note the limited setting of the experiments on the unrealistic, closed-domain setting of this dataset and suggested experimenting with other data (such as complex WebQuesitons). Reviewers were also concerned about the scalability of the system due to the significant amount of computations. They also noted several previous studies were not included in the paper. Authors acknowledged and made changes according to these suggestions. They also included experiments only on the open-domain subset of the HotPotQA in their rebuttal, unfortunately the results are not as good as before. Hence, I suggest rejecting this paper.
Decision: Reject