Learning to Attend On Essential Terms: An Enhanced Retriever-Reader
Model for Open-domain Question Answering
Jianmo Ni1∗, Chenguang Zhu2 , Weizhu Chen3 , Julian McAuley1
1
University of California, San Diego2
Microsoft Speech and Dialogue Research Group3
Microsoft Dynamics 365 AI{jin018,jmcauley}@ucsd.edu, {chezhu,wzchen}@microsoft.com

Abstract
Open-domain question answering remains achallenging task as it requires models that arecapable of understanding questions and answers, collecting useful information, and reasoning over evidence. Previous work typically formulates this task as a reading comprehension or entailment problem given evidence retrieved from search engines. However, existing techniques struggle to retrieve
indirectly related evidence when no directly
related evidence is provided, especially forcomplex questions where it is hard to parse
precisely what the question asks. In this paper we propose a retriever-reader model thatlearns to attend on essential terms during thequestion answering process. We build (1) an
essential term selector which first identifies themost important words in a question, then reformulates the query and searches for related evidence; and (2) an enhanced reader that distinguishes between essential terms and distracting words to predict the answer. We evaluate
our model on multiple open-domain multiplechoice QA datasets, notably performing at thelevel of the state-of-the-art on the AI2 Reasoning Challenge (ARC) dataset.

1

Introduction

Open-domain question answering (QA) has been
extensively studied in recent years. Many existing works have followed the ‘search-and-answer’
strategy and achieved strong performance (Chenet al., 2017; Kwon et al., 2018; Wang et al., 2018b)
spanning multiple QA datasets such as TriviaQA(Joshi et al., 2017), SQuAD (Rajpurkar et al.,2016), MS-Macro (Nguyen et al., 2016), ARC(Clark et al., 2018) among others.
However, open-domain QA tasks become inherently more difficult when (1) dealing with questions with little available evidence; (2) solving∗
Most of the work was done during internship at Microsoft, Redmond.

questions where the answer type is free-form text
(e.g. multiple-choice) rather than a span among
existing passages (i.e., ‘answer span’); or when
(3) the need arises to understand long and complex questions and reason over multiple passages,
rather than simple text matching. As a result, it isessential to incorporate commonsense knowledge
or to improve retrieval capability to better capture
partially related evidence (Chen et al., 2017).
As shown in Table 1, the TriviaQA, SQuAD,
and MS-Macro datasets all provide passages
within which the correct answer is guaranteed toexist. However, this assumption ignores the difficulty of retrieving question-related evidence froma large volume of open-domain resources, especially when considering complex questions which
require reasoning or commonsense knowledge.
On the other hand, ARC does not provide passages
known to contain the correct answer. Instead, thetask of identifying relevant passages is left to thesolver. However, questions in ARC have multiple answer choices that provide indirect information that can help solve the question. As suchan effective model needs to account for relations
among passages, questions, and answer choices.
Real-world datasets such as Amazon-QA (a corpus of user queries from Amazon) (McAuley andYang, 2016) also exhibit the same challenge, i.e.,
the need to surface related evidence from which to
extract or summarize an answer.
Figure 1 shows an example of a question in the
ARC dataset and demonstrates the difficulties inretrieval and reading comprehension. As shown
for Choice 1 (C1), a simple concatenation of the1

For SQuAD and TriviaQA, since the questions are pairedwith span-type answers, it is convenient to obtain ranking supervision where retrieved passages are relevant via distant supervision; however free-form questions in ARC and AmazonQA result in a lack of supervision which makes the problem
more difficult. For MS-Macro, the dataset is designed to annotate relevant passages though it has free-form answers.

# of questions

Opendomain

MultiplechoicePassageretrieval

No ranking
supervision1

ARC (Clark et al., 2018)
Amazon-QA (McAuley and Yang, 2016)≈ 7K≈ 1.48M

3
3

3

3

3
3

SQuAD (Rajpurkar et al., 2016)
TriviaQA (Joshi et al., 2017)
MS-Macro (Nguyen et al., 2016)≈ 100K≈ 650K≈ 1M

3
3
3

DatasetTable 1: Differences among popular QA datasets.
Q: Mercury , the planet nearest to the Sun , has extreme surface temperatures , ranging from 465
C in sunlight to -180 C in darkness . Why is there such a large range of temperatures on Mercury?
C1: The planet is too small to hold heat.

…

C4: The planet lacks an atmosphere to hold heat .

Query1 = Q+C1

Query4 = Essential-term(Q)+C4

Query1: Mercury , the planet nearest to the Sun , has extreme surface
temperatures , ranging from 465 C in sunlight to -180 C in darkness .
Why is there such a large range of temperatures on Mercury? The
planet is too small to hold heat.

Retrieving evidence

Retrieving evidence
S1: Other planets such as Mercury has extreme
hot and cold temperatures .
S2: The planet Mercury is too small and has too
little gravity to hold onto an atmosphere.
…

Query4: Mercury extreme surface temperatures.
The planet lacks an atmosphere to hold heat .

…

S1: The lack of atmosphere also contributes to theplanet 's wild temperature extremes .
S2: Mercury is the closest planet to the sun and has athin atmosphere, no air pressure and an extremely
high temperature.
…

Sending evidence to reader
MRC

Figure 1: Example of the retrieve-and-read process to solve open-domain questions. Words related with the question are in bold; and words related with C1 and C4 are in italics.

question and the answer choice is not a reliable
query and is of little help when trying to find supporting evidence to answer the question (e.g. wemight retrieve sentences similar to the question or
the answer choice, but would struggle to find evidence explaining why the answer choice is correct). On the other hand, a reformulated query
consisting of essential terms in the question and
Choice 4 can help retrieve evidence explaining
why Choice 4 is a correct answer. To achieve this,the model needs to (1) ensure that the retrieved
evidence supports the fact mentioned in both thequestion and the answer choices and (2) capture
this information and predict the correct answer.
To address these difficulties, we propose an
essential-term-aware Retriever-Reader (ET-RR)
model that learns to attend on essential terms during retrieval and reading. Specifically, we develop
a two-stage method with an essential term selector
followed by an attention-enhanced reader.
Essential term selector. ET-Net is a recurrent
neural network that seeks to understand the question and select essential terms, i.e., key words,
from the question. We frame this problem as aclassification task for each word in the question.

These essential terms are then concatenated witheach answer choice and fed into a retrieval engine
to obtain related evidence.
Attention-Enhanced Reader.
Our neuralreader takes the triples (question, answer choice,retrieved passage) as input. The reader consistsof a sequence of language understanding layers:
an input layer, attention layer, sequence modeling layer, fusion layer, and an output layer. The
attention and fusion layers help the model to obtain a refined representation of one text sequencebased on the understanding of another, e.g. a passage representation based on an understanding of
the question. We further add a choice-interaction
module to handle the semantic relations and differences between answer choices. Experiments showthat this can further improve the model’s accuracy.
We evaluate our model on the ARC Challenge dataset, where our model achieves an accuracy of 36.61% on the test set, and outperformed all leaderboard solutions at the time ofwriting (Sep. 2018). To compare with otherbenchmark datasets, we adapt RACE (Lai et al.,2017) and MCScript (Ostermann et al., 2018) tothe open domain setting by removing their super-vision in the form of relevant passages. We also
consider a large-scale real-world open-domain
dataset, Amazon-QA, to evaluate our model’sscalability and to compare against standard benchmarks designed for the open-domain setting. Experiments on these three datasets show that ETRR outperforms baseline models by a large margin. We conduct multiple ablation studies to show
the effectiveness of each component of our model.
Finally, we perform in-depth error analysis to explore the model’s limitations.

2

Related Work

There has recently been growing interest in building better retrievers for open-domain QA. Wanget al. (2018b) proposed a Reinforced RankerReader model that ranks retrieved evidence andassigns different weights to evidence prior to processing by the reader. Min et al. (2018) demonstrated that for several popular MRC datasets(e.g. SQuAD, TriviaQA) most questions can be
answered using only a few sentences rather thanthe entire document. Motivated by this observation, they built a sentence selector to gather thispotential evidence for use by the reader model.
Nishida et al. (2018) developed a multi-task learning (MTL) method for a retriever and reader in order to obtain a strong retriever that considers certain passages including the answer text as positive samples during training. The proposed MTL
framework is still limited to scenarios where itis feasible to discover whether the passages contain the answer span. Although these works haveachieved progress on open-domain QA by improving the ranking or selection of given evidence,few have focused on the scenario where the modelneeds to start by searching for the evidence itself.
Scientific Question Answering (SQA) is a representative open-domain task that requires capability in both retrieval and reading comprehension. In this paper, we study question answeringon the AI2 Reasoning Challenge (ARC) scientific
QA dataset (Clark et al., 2018). This dataset contains multiple-choice scientific questions from 3rd
to 9th grade standardized tests and a large corpus of relevant information gathered from searchengines. The dataset is partitioned into “Challenge” and “Easy” sets. The challenge set consistsof questions that cannot be answered correctly byeither of the solvers based on Pointwise Mutual
Information (PMI) or Information Retrieval (IR).

Existing models tend to achieve only slightly better and sometimes even worse performance thanrandom guessing, which shows that existing models are not well suited to this kind of QA task.
Jansen et al. (2017) first developed a rule-based
focus word extractor to identify essential terms in
the question and answer candidates. The extracted
terms are used to aggregate a list of potential answer justifications for each answer candidate. Experiments shown that focus words are beneficial
for SQA on a subset of the ARC dataset. Khashabi
et al. (2017) also worked on the problem of finding essential terms in a question for solving SQA
problems. They published a dataset containing
over 2,200 science questions annotated with essential terms and train multiple classifiers on it.
Similarly, we leverage this dataset to build an essential term selector using a neural network-basedalgorithm. More recently, Boratko et al. (2018)
developed a labeling interface to obtain high quality labels for the ARC dataset. One finding is thathuman annotators tend to retrieve better evidence
after they reformulate the search queries which are
originally constructed by a simple concatenationof question and answer choice. By feeding theevidence obtained by human-reformulated queries
into a pre-trained MRC model (i.e., DrQA (Chenet al., 2017)) they achieved an accuracy increaseof 42% on a subset of 47 questions. This shows
the potential for a “human-like” retriever to boost
performance on this task.
Query reformulation has been shown to beeffective in information retrieval (Lavrenko and
Croft, 2001). Nogueira and Cho (2017) modeled
the query reformulation task as a binary term selection problem (i.e., whether to choose the termin the original query and the documents retrieved
using the original query). The selected terms arethen concatenated to form the new query. Insteadof choosing relevant words, Buck et al. (2018) proposed a sequence-to-sequence model to generate
new queries. Das et al. (2019) proposed Multistep Retriever-Reader which explores an iterative
retrieve-and-read strategy for open-domain question answering. It formulates the query reformulation problem in the embedding space where thevector representation of the question is changed to
improve the performance. Since there is no supervision for training the query reformulator, all thesemethods using reinforcement learning to maximize the task-specific metrics (e.g. Recall for para-graph ranking, F1 and Exact Matching for spanbased MRC). Different from these works, we train
the query reformulator using an annotated dataset
as supervision and then apply the output to a separate reader model. We leave the exploration oftraining our model end-to-end using reinforcement
learning as future work.

3

Approach

In this section, we introduce the essential-termaware retriever-reader model (ET-RR). As shownin Figure 2, we build a term selector to discoverwhich terms are essential in a question. The selected terms are then used to formulate a more efficient query enabling the retriever to obtain related
evidence. The retrieved evidence is then fed to thereader to predict the final answer.
For a question with q words Q = {wtQ }qt=1
along with its N answer choices C = {Cn }N
n=1
where Cn = {wtC }ct=1 , the essential-term selector
chooses a subset of essential terms E ⊂ Q, which
are then concatenated with each Cn to formulate aquery. The query for each answer choice, E + Cn ,is sent to the retriever (e.g. Elastic Search2 ), andthe top K retrieved sentences based on the scores
returned by the retriever are then concatenated into
the evidence passage Pn = {wtP }K
t=1 .
Next, given these text sequences Q, C, andP = {Pn }N
n=1 , the reader will determine a matching score for each triple {Q, Cn , Pn }. The answer
choice Cn∗ with the highest score is selected.
We first introduce the reader model in Section 3.1 and then the essential term selector in Section 3.2.
3.1

Reader Model

3.1.1 Input Layer
To simplify notation, we ignore the subscript ndenoting the answer choice until the final output
layer. In the input layer, all text inputs—the question, answer choices, and passages, i.e., retrieved
evidence—are converted into embedded representations. Similar to Wang (2018), we consider thefollowing components for each word:
Word Embedding. Pre-trained GloVe word embedding with dimensionality dw = 300.
Part-of-Speech Embedding and Named-Entity
Embedding. The part-of-speech tags and namedentities for each word are mapped to embeddings
with dimension 16.
2

https://www.elastic.co/products/elasticsearch

Relation Embedding. A relation between eachword in P and any word in Q or C is mapped toan embedding with dimension 10. In the case thatmultiple relations exist, we select one uniformly
at random. The relation is obtained by querying
ConceptNet (Speer et al., 2017).
Feature Embeddings. Three handcrafted features
are used to enhance the word representations: (1)Word Match; if a word or its lemma of P exists in
Q or C, then this feature is 1 (0 otherwise). (2)Word Frequency; a logarithmic term frequency is
calculated for each word. (3) Essential Term; forthe i-th word in Q, this feature, denoted as wei ,is 1 if the word is an essential term (0 otherwise).
Let we = [we1 , we2 , . . . , weq ] denote the essential
term vector.
For Q, C, P, all of these components are concatenated to obtain the final word representations
WQ ∈ Rq×dQ , WC ∈ Rc×dC , WP ∈ Rp×dP ,where dQ , dC , dP are the final word dimensionsof Q, C, and P.
3.1.2 Attention Layer
As shown in Figure 2, after obtaining word-level
embeddings, attention is added to enhance word
representations. Given two word embedding sequences WU , WV , word-level attention is calculated as:0

MU V = WU U · (WV V)>
0

MU V = softmax(MU V )

(1)

V
WU
= MU V · (WV V),

where U ∈ RdU ×dw and V ∈ RdV ×dw are two
matrices that convert word embedding sequences0
to dimension dw , and MU V contains dot products
between each word in WU and WV , and soft0
max is applied on MU V row-wise. Three types
of attention are calculated using Equation (1):(1) question-aware passage representation WPQ ∈Rp×dw ; (2) question-aware choice representation
QWC∈ Rc×dw ; and (3) passage-aware choice repP ∈ Rc×dw .
resentation WC
3.1.3 Sequence Modeling Layer
To model the contextual dependency of each text
sequence, we use BiLSTMs to process the word
representations obtained from the input layer andattention layer:
Hq = BiLSTM[WQ ]Q
P
Hc = BiLSTM[WC ; WC
; WC
]
p

H

= BiLSTM[WP ; WPQ ],(2)

+ Choice 1 =Essential
terms

Question

Passage 1

Query 1
Query N

Attention layer

Passage N

Sequence Modeling layer+

Attention enhanced reader
Question

Choice N

Input layer…

+ Choice N =Choice 1 + … +Question

IR retriever

Essential term
selector

Choice 1

Passage 1…

Question

Input layer
Attention layer
Sequence Modeling layer

Score 1

Passage N

Choice N(b) Essential term
selector

Prediction

Input layer
Attention layer…

Sequence Modeling layer

Fusion layer
Output layer

Projection layer

Hp

Hq

Hc

Question-aware
matching

Self fusion

Self fusion

𝑝𝑝

𝑞𝑞

𝑐𝑐

Fusion layer…
Softmax

Output layer
Score N(a) Retriever-reader
model

Prediction

Choiceinteraction(c) Fusion layer

Figure 2: Model structure for our essential-term-aware retriever-reader model.

where Hq ∈ Rq×l , Hc ∈ Rc×l , and Hp ∈Rp×l are the hidden states of the BiLSTMs, ‘;’ isfeature-wise concatenation, and l is the size of thehidden states.
3.1.4 Fusion Layer
We further convert each question and answer
choice into a single vector: q ∈ Rl and c ∈ Rl :>αq = softmax([Hq ; we ] · wsq), q = Hq> αq>αc = softmax(Hc · wsc), c = Hc> αc ,(3)where the essential-term feature we from Section 3.1.1 is concatenated with Hq , and wsq andwsc are learned parameters.
Finally, a bilinear sequence matching is calculated between Hp and q to obtain a question-aware
passage representation, which is used as the final
passage representation:
αp = softmax(Hp · q);

p = Hp> αp .

(4)

3.1.5 Choice Interaction
When a QA task provides multiple choices for selection, the relationship between the choices canprovide useful information to answer the question.
Therefore, we integrate a choice interaction layer
to handle the semantic correlation between multiple answer choices. Given the hidden state Hcn ofchoice cn and Hci of other choices ci , ∀i 6= n, we
calculate the differences between the hidden states
and apply max-pooling over the differences:
1 X cicinter = Maxpool(Hcn −H ), (5)N −1i6=nwhere N is the total number of answer choices.
Here, cinter characterizes the differences between

an answer choice cn and other answer choices.
The final representation of an answer choice is updated by concatenating the self-attentive answer
choice vector and inter-choice representation as
cfinal = [c; cinter ].
3.1.6 Output Layer
For each tuple {q, pn , cn }N
n=1 , two scores are calculated by matching (1) the passage and answer
choice and (2) question and answer choice. We
use a bilinear form for both matchings. Finally, asoftmax function is applied over N answer choices
to determine the best answer choice:pc finalspc;
n = pn W cn

qc finalsqc
n = qW cn

s = softmax(spc ) + softmax(sqc ),(6)

qc
where spc
n , sn are the scores for answer choice
1 ≤ n ≤ N ; spc , sqc are score vectors for all N
choices; and s contains the final scores for eachanswer choice. During training, we use a crossentropy loss.

3.2

Essential Term Selector

Essential terms are key words in a question thatare crucial in helping a retriever obtain related evidence. Given a question Q and N answer choices
C1 , . . . , CN , the goal is to predict a binary variable yi for each word Qi in the question Q, whereyi = 1 if Qi is an essential term and 0 otherwise.
To address this problem, we build a neural model,
ET-Net, which has the same design as the reader
model for the input layer, attention layer, and sequence modeling layer to obtain the hidden state
Hq for question Q.
In detail, we take the question Q and the concatenation C of all N answer choices as input toQuestion

Model

If an object is attracted to a magnet, theobject is most likely made of (A) wood (B)plastic (C) cardboard (D) metal

MaxPMISumPMI
PropSurf
PropLemET Classifier
ET-Net# annotators 5
Annotation If,0; an,0; object,3; is,0; attracted,5;
to,0; a,0; magnet,,5; the,0; object,1; is,0;
most,0; likely,0; made,2; of,0Table 2: Example of essential term data.

(7)

As shown in Figure 2, the hidden states obtained
from the attention layer are then concatenated withthe embedded representations of Q and fed into
a projection layer to obtain the prediction vectory ∈ Rq for all words in the question:f] · ws ,y = [Hq ; WQ

(8)

where ws contains the learned parameters, andf
WQ
is the concatenation of the POS embedding,
NER embedding, relation embedding, and featureembedding from Section 3.1.1.
After obtaining the prediction for each word, we
use a binary cross-entropy loss to train the model.
During evaluation, we take words with yi greaterthan 0.5 as essential terms.

4

Recall

F1

0.88
0.88
0.68
0.76
0.91
0.74

0.65
0.65
0.64
0.64
0.71
0.90

0.75
0.75
0.66
0.69
0.80
0.81

Table 3: Performance of different selectors.

ET-Net. Q and C first go through an input layer
to convert to the embedded word representation,and then word-level attention is calculated to obC
tain a choice-aware question representation WQ
as in Equation (1). We concatenate the word representation and word-level attention representationof the question and feed it into the sequence modeling layer:C
Hq = BiLSTM[WQ ; WQ].

PrecisionExperiments

In this section, we first discuss the performanceof the essential term selector, ET-Net, on a public
dataset. We then discuss the performance of thewhole retriever-reader pipeline, ET-RR, on multiple open-domain datasets.For both the ET-Net
and ET-RR models, we use 96-dimensional hidden states and 1-layer BiLSTMs in the sequencemodeling layer. A dropout rate of 0.4 is applied
for the embedding layer and the BiLSTMs’ output layer. We use adamax (Kingma and Ba, 2014)
with a learning rate of 0.02 and batch size of
32. The model is trained for 100 epochs. Our
code is released at https://github.com/
nijianmo/arc-etrr-code.

4.1

Performance on Essential Term Selection

We use the public dataset from Khashabi et al.
(2017) which contains 2,223 annotated questions,each accompanied by four answer choices. Table 2 gives an example of an annotated question.
As shown, the dataset is annotated for binary classification. For each word in the question, thedata measures whether the word is an “essential”
term according to 5 annotators. We then split thedataset into training, development, and test sets using an 8:1:1 ratio and select the model that performs best on the development set.
Table 3 shows the performance of our essential
term selector and baseline models from Khashabi
et al. (2017). The second best model (ET Classifier) is an SVM-based model from Khashabi et al.
(2017) requiring over 100 handcrafted features.
As shown, our ET-Net achieves a comparable result with ET Classifier in terms of the F1 Score.
Table 4 shows example predictions made by ETNet. As shown, ET-Net is capable of selecting
most ground-truth essential terms. It rejects certain words such as “organisms” which have a high
TF-IDF in the corpus but are not relevant to answering a particular question. This shows its ability to discover essential terms according to thecontext of the question.
4.2

Performance on Open-domain
Multiple-choice QA

We train and evaluate our proposed pipeline
method ET-RR on four open-domain multiplechoice QA datasets. All datasets are associatedwith a sentence-level corpus. Detailed statistics
are shown in Table 5.
• ARC (Clark et al., 2018): We consider the‘Challenge’ set in the ARC dataset and use
the provided corpus during retrieval.
• RACE-Open: We adapted the RACE dataset(Lai et al., 2017) to the open-domain setting.
Originally, each question in RACE comesExample questions

Dataset

Train

Dev

Test

CorpusWhich unit of measurement can be used to describe thelength of a desk ?
One way animal usually respond to a sudden drop in
temperature is by
Organisms require energy to survive. Which of the following processes provides energy to the body ?

ARC
RACE-OpenMCScript-Open
Amazon-Patio
Amazon-Auto
Amazon-Cell

1,119
9,531
1,036
36,58749,64340,842

299
473
156
4,531
6,205
5,105

1,172
528
319
4,515
6,206
5,106

1.46M
0.52M
24.2K
2.55M
7.32M
1.86M

Table 4: Examples of essential term prediction (in questions) by ET-Net. True positives are marked bold andunderlined while false positives are only underlined.

Table 5:
Statistics on ARC, RACE-Open,MCScript-Open and Amazon-QA. Corpus size isthe number of sentences.

DatasetExample questions

ARC

The best way to separate salt from water is with the use of
Which geologic process most likely caused the formation of the Mount St. Helens Volcano?

RACE-Open

According to the article, what does the band Four Square hope to do in the future?
According to the article we know it is to prevent the forests from slowly disappearing.

Amazon-QA

For anyone with small ears, do these fit comfortably or do they feel like they are always going to fall out,
not in correctly, etc.
Does it remove easily and does it leave any sticky residue behind? thanks in advance.

Table 6: Example of predictions on ARC, RACE-Open and Amazon-QA. Predicted terms are underlined.

with a specific passage. To enable passage
retrieval, we concatenate all passages into a
corpus with sentence deduplication.3• MCScript-Open: The MCScript (Ostermann
et al., 2018) dataset is also adapted to the
open-domain setting. Again we concatenate
all passages to build the corpus.4• Amazon-QA: The Amazon-QA dataset(McAuley and Yang, 2016) is an opendomain QA dataset covering over one
million questions across multiple productcategories.
Each question is associatedwith a free-form answer. We adapt it into a
2-way multiple-choice setting by randomly
sampling an answer from other questions as
an answer distractor. We split all productreviews at the sentence-level to build the
corpus. We consider three categories fromthe complete dataset in our experiments.
In the experiments, ET-RR uses ET-Net tochoose essential terms in the question. Table 6 shows example predictions on these target
datasets. Then it generates a query for each of
the N answer choices by concatenating essential
3

As short questions might not contain any words which
can relate the question to any specific passage or sentence,
we only keep questions with more than 15 words.
4
We keep questions with more than 10 words rather than
15 words to ensure that there is sufficient data.

terms and the answer choice. For each query, ETRR obtains the top K sentences returned by the retriever and considers these sentences as a passage
for the reader. We set K = 10 for all experiments.
We compare ET-RR with existing retrieve-andread methods on both datasets. As shown in Table 7, on the ARC dataset, ET-RR outperforms all
previous models without using pre-trained models and achieves a relative 8.1% improvement overthe second best BiLSTM Max-out method (Mihaylov et al., 2018). Recently, finetuning on pretrained models has shown great improvement overa wide range of NLP tasks. Sun et al. (2019) proposed a ‘Reading Strategies’ method to finetune
the pre-trained model OpenAI GPT, a language
model trained on the BookCorpus dataset (Radford, 2018). They trained Reading Strategies on
the RACE dataset to obtain more auxiliary knowledge and then finetune that model on the ARC corpus. Table 8 demonstrates the performance comparison of ET-RR and Reading Strategies on ARC.
As shown, though Reading Strategies trained on
both ARC and RACE dataset outperforms ET-RR,
ET-RR outperforms Reading Strategies using only
the ARC dataset at training time.
On the RACE-Open and MCScript-Opendatasets, ET-RR achieves a relative improvementof 24.6% and 10.5% on the test set comparedwith the second best method IR solver respectively. We also evaluate on multiple categories ofModelARC
Test

RACE-OpenTest

MCScript-Open
Test

IR solver
RandomBiDAF
BiLSTM Max-out
ET-RR (Concat)
ET-RR

20.26
25.02
26.54
33.87
35.33
36.61

30.70
25.01
26.89
/
36.87
38.61

60.46
50.02
50.81
/
66.46
67.71

Training Corpus

model

ARC

ARC

Reading Strategies
ET-RR

35.0
36.6

ARC+RACE

Reading Strategies

40.7

Table 8: Comparisons of ET-RR and Reading
Strategies on ARC.

Table 7: Accuracy on multiple-choice selection on ARC,
RACE-Open and MCScript-Open.
Model

Amazon-PatioAmazon-Auto

Aamzon
-Cell

Model

Test

IR solver
Moqa
ET-RR (Concat)
ET-RR

72.80
84.80
96.19
96.61

73.60
86.30
95.21
95.96

70.50
88.60
93.26
93.81

ET-RR
– inter-choice– passage-choice– question-choice– passage-question

36.61
36.36
35.41
34.47
34.05

Table 9: Accuracy on multiple-choice selection on
three product categoris of Amazon-QA.
Pre-trained
3
3

model

RACE

Reading Strategies
OpenAI GPTET-RR (reader)
Bi-attn (MRU)
Hier. Co-Matching

63.8
59.0
52.3
50.4
50.4

Table 10: Experimental results for reader on RACE.

the Amazon-QA dataset. As shown in Table 9,
ET-RR increases the accuracy by 10.33% on average compared to the state-of-the-art model Moqa
(McAuley and Yang, 2016). We also compare ETRR with ET-RR (Concat), which is a variant ofour proposed model that concatenates the questionand choice as a query for each choice. Among
all datasets, ET-RR outperforms ET-RR (concat)
consistently which shows the effectiveness of ouressential-term-aware retriever.
4.3

Ablation study

We investigate how each component contributes tomodel performance.
Performance of reader. Our reader alone canbe applied on MRC tasks using the given passages. Here, we evaluate our reader on the original RACE dataset to compare with other MRC
models as shown in Table 10. As shown, the recently proposed Reading Strategies and OpenAI
GPT models, that finetune generative pre-trainedmodels achieve the highest scores. Among nonpre-trained models, our reader outperforms other

Table 11: Ablation test on attention components of ETRR on ARC. ‘–’ denotes the ablated feature.

baselines: Bi-attn (MRU) (Tay et al., 2018) and
Hierarchical Co-Matching (Wang et al., 2018a) by
a relative improvement of 3.8%.
Attention components. Table 11 demonstrates
how the attention components contribute to theperformance of ET-RR. As shown, ET-RR withall attention components performs the best on the
ARC test set. The performance of ET-RR without
passage-question attention drops the most significantly out of all the components. It is worth noting that the choice interaction layer gives a further
0.24% boost on test accuracy.
Essential term selection. To understand thecontribution of our essential-term selector, we
compare ET-RR with two variants: (1) ET-RR(Concat) and (2) ET-RR (TF-IDF). For ET-RR(TF-IDF), we calculate the TF-IDF scores andtake words with the top 30% of TF-IDF scoresin the question to concatenate with each answer
choice as a query.5
Table 12 shows an ablation study comparing
different query formulation methods and amountsof retrieved evidence K. As shown, with the essential term selector ET-Net, the model consistently outperforms other baselines, given different
numbers of retrievals K. Performance for all models is best when K = 10. Furthermore, only using
TF-IDF to select essential terms in a question is
not effective. When K = 10, the ET-RR (TF-IDF)
5
According to the annotated dataset, around 30% of theterms in each question are labelled as essential.

ModelET-RR(Concat)

ET-RR(TF-IDF)

ET-RR

Top K

Dev

Test

Dev

Test

Dev

Test

5
10
20

39.26
38.93
41.28

33.36
35.33
34.56

39.93
39.43
38.59

34.73
35.24
33.88

39.93
43.96
42.28

35.59
36.61
35.67

Table 12: Comparison of query formulation methods
and amounts of retrieved evidence (i.e., top K) on the
ARC dataset, in terms of percentage accuracy.

method performs even worse than ET-RR (Concat). This illustrates the challenges in understanding what is essential in a question.
Though ET-RR consistently outperforms ETRR (TF-IDF), the improvement is relatively modest on the Test set (around 1.4%). A similar outcome has been reported in Jansen et al. (2017);
Khashabi et al. (2017) where essential term extraction methods have shown around 2%-4% gain
compared with TF-IDF models and struggle to obtain further improvement on SQA tasks. This consensus might show the discrepancy of essential
terms between human and machine (i.e., the essential terms obtained using a human annotateddataset might not be helpful in a machine inference model). Another reason might be the current
retrieval method does not effectively use these essential terms and the performance highly depends
on the dataset. Note that the ET-RR outperforms
ET-RR (TF-IDF) by around 4% on the Dev set.
Therefore, how to develop well-formed single oreven multi-hop queries using these terms are worth
studying in the future.
4.4

Error Analysis

Table 13 shows two major types of error, where thecorrect answer choice is in bold and the predicted
answer choice is in italics.
Retrieved supporting evidence but failed to reason over it. For the first question, there exists evidence that can justify the answer candidate (C).
However, the model chooses (D) which has more
words overlapping with its evidence. This showsthat the model still lacks the reasoning capability
to solve complex questions.
Failed to retrieve supporting evidence. For thesecond question, the retrieved evidence of boththe correct answer (D) and the prediction (B) is
not helpful to solve the question. Queries such as‘what determines the year of a planet’ are neededto acquire the knowledge for solving this question.

The elements carbon, hydrogen, and oxygen are parts ofmany different compounds. Which explains why thesethree elements can make so many different compounds?
(A) They can be solids, liquids, or gases.
(B) They come in different sizes and shapes.
(C) They combine in different numbers and ratios.
* There are many different types of compounds because
atoms of elements combine in many different ways (andin different whole number ratios) to form different compounds.
(D) They can be a proton, a neutron, or an electron.
* Atoms of different elements have a different number ofprotons, neutrons, and electrons.
Which planet in the solar system has the longest year?
(A) The planet closest to the Sun.
(B) The planet with the longest day.
* The planet with the longest day is Venus; a day on Venus
takes 243 Earth days.
(C) The planet with the most moons.
(D) The planet farthest from the Sun.
* The last planet discovered in our solar system is farthest
away from the sun.

Table 13: Examples where ET-RR fails on ARC. The
retrieved evidence for each answer candidate is marked
by *.

This poses further challenges to design a retrieverthat can rewrite such queries.

5

Conclusion

We present a new retriever-reader model (ET-RR)
for open-domain QA. Our pipeline has the following contributions: (1) we built an essential term selector (ET-Net) which helps the model understandwhich words are essential in a question leading
to more effective search queries when retrieving
related evidence; (2) we developed an attentionenhanced reader with attention and fusion among
passages, questions, and candidate answers. Experimental results show that ET-RR outperforms
existing QA models on open-domain multiplechoice datasets as ARC Challenge, RACE-Open,MCScript-Open and Amazon-QA. We also perform in-depth error analysis to show the limitations of current work. For future work, we plan
to explore the directions of (1) constructing multihop query and (2) developing end-to-end retrieverreader model via reinforcement learning.
Acknowledgements. We thank Jade Huang for
proofreading the paper, Liang Wang and Daniel
Khashabi for sharing code and the annotateddataset with us. This work is partly supported by
NSF #1750063. We thank all the reviewers fortheir constructive suggestions.

References
Michael Boratko, Harshit Padigela, Divyendra Mikkilineni, Pritish Yuvraj, Rajarshi Das, Andrew McCallum, Maria Chang, Achille Fokoue, Pavan Kapanipathi, Nicholas Mattei, Ryan Musa, Kartik Talamadupula, and Michael Witbrock. 2018. A systematic classification of knowledge, reasoning, andcontext within the arc dataset. In 1st Workshop on
Machine Reading for Question Answering.
Christian Buck, Jannis Bulian, Massimiliano Ciaramita, Andrea Gesmundo, Neil Houlsby, Wojciech
Gajewski, and Wei Wang. 2018. Ask the right questions: Active question reformulation with reinforcement learning. In ICLR.
Danqi Chen, Adam Fisch, Jason Weston, and Antoine
Bordes. 2017. Reading wikipedia to answer opendomain questions. In ACL.
Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot,
Ashish Sabharwal, Carissa Schoenick, and Oyvind
Tafjord. 2018. Think you have solved question
answering? try arc, the ai2 reasoning challenge.
CoRR, abs/1803.05457.
Rajarshi Das, Shehzaad Dhuliawala, Manzil Zaheer,and Andrew McCallum. 2019. Multi-step retrieverreader interaction for scalable open-domain question
answering. In ICLR.
Peter Jansen, Rebecca Sharp, Mihai Surdeanu, and Peter Clark. 2017. Framing qa as building and ranking
intersentence answer justifications. Computational
Linguistics, 43:407–449.
Mandar Joshi, Eunsol Choi, Daniel S. Weld, andLuke S. Zettlemoyer. 2017. Triviaqa: A large scaledistantly supervised challenge dataset for reading
comprehension. In ACL.
Daniel Khashabi, Tushar Khot, Ashish Sabharwal, andDan Roth. 2017. Learning what is essential in questions. In CoNLL.
Diederik P. Kingma and Jimmy Ba. 2014.
A method for stochastic optimization.
abs/1412.6980.

Adam:
CoRR,

Heeyoung Kwon, Harsh Trivedi, Peter Jansen, Mihai Surdeanu, and Niranjan Balasubramanian. 2018.
Controlling information aggregation for complex
question answering. In ECIR.
Guokun Lai, Qizhe Xie, Hanxiao Liu, Yiming Yang,and Eduard H. Hovy. 2017. Race: Large-scale reading comprehension dataset from examinations. In
EMNLP.
Victor Lavrenko and W. Bruce Croft. 2001. Relevancebased language models. SIGIR Forum, 51:260–267.
Julian McAuley and Alex Yang. 2016. Addressing
complex and subjective product-related queries withcustomer reviews. In WWW.

Todor Mihaylov, Peter Clark, Tushar Khot, and Ashish
Sabharwal. 2018. Can a suit of armor conduct electricity? a new dataset for open book question answering. In EMNLP.
Sewon Min, Victor Zhong, Richard Socher, and Caiming Xiong. 2018. Efficient and robust question answering from minimal context over documents. In
ACL.
Tri Nguyen, Mir Rosenberg, Xia Song, Jianfeng Gao,
Saurabh Tiwary, Rangan Majumder, and Li Deng.
2016. Ms marco: A human generated machine reading comprehension dataset. CoRR, abs/1611.09268.
Kyosuke Nishida, Itsumi Saito, Atsushi Otsuka, Hisako
Asano, and Junji Tomita. 2018. Retrieve-andread: Multi-task learning of information retrieval
and reading comprehension. In CIKM.
Rodrigo Nogueira and Kyunghyun Cho. 2017. Taskoriented query reformulation with reinforcement
learning. In EMNLP.
Simon Ostermann, Michael Roth, Ashutosh Modi, Stefan Thater, and Manfred Pinkal. 2018. Semeval2018 task 11: Machine comprehension using commonsense knowledge. In SemEval@NAACL-HLT.
Alec Radford. 2018. Improving language understanding by generative pre-training. In Technical report.
Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, andPercy Liang. 2016. Squad: 100, 000+ questions formachine comprehension of text. In EMNLP.
Robert Speer, Joshua Chin, and Catherine Havasi.
2017. Conceptnet 5.5: An open multilingual graphof general knowledge. In AAAI.
Kai Sun, Dian Yu, Dong Yu, and Claire Cardie. 2019.
Improving machine reading comprehension withgeneral reading strategies. In NAACL.
Yi Tay, Luu Anh Tuan, and Siu Cheung Hui. 2018.
Multi-range reasoning for machine comprehension.
CoRR, abs/1803.09074.
Liang Wang. 2018. Yuanfudao at semeval-2018 task
11: Three-way attention and relational knowledge
for commonsense machine comprehension.
In
SemEval@NAACL-HLT.
Shuohang Wang, Mo Yu, Shiyu Chang, and Jing Jiang.
2018a. A co-matching model for multi-choice reading comprehension. In ACL.
Shuohang Wang, Mo Yu, Xiaoxiao Guo, Zhiguo Wang,
Tim Klinger, Wei Zhang, Shiyu Chang, Gerald
Tesauro, Bowen Zhou, and Jing Jiang. 2018b. R3:
Reinforced ranker-reader for open-domain question
answering. In AAAI.