Venue: ACL 2017

# Gated-Attention Readers for Text Comprehension

Bhuwan Dhingra\({}^{*}\)   Hanxiao Liu\({}^{*}\)   Zhilin Yang

William W. Cohen    Ruslan Salakhutdinov

School of Computer Science

Carnegie Mellon University

{bdhingra,hanxiaol,zhiling},wochen,rsalakhu}@cs.cmu.edu

BD and HL contributed equally to this work.

###### Abstract

In this paper we study the problem of answering cloze-style questions over documents. Our model, the Gated-Attention (GA) Reader1, integrates a multi-hop architecture with a novel attention mechanism, which is based on multiplicative interactions between the query embedding and the intermediate states of a recurrent neural network document reader. This enables the reader to build query-specific representations of tokens in the document for accurate answer selection. The GA Reader obtains state-of-the-art results on three benchmarks for this task-the CNN & Daily Mail news stories and the Who Did What dataset. The effectiveness of multiplicative interaction is demonstrated by an ablation study, and by comparing to alternative compositional operators for implementing the gated-attention.

Footnote 1: Source code is available on github: [https://github.com/bdhingra/ga-reader](https://github.com/bdhingra/ga-reader)

+
Footnote †: _Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics_, pages 1832–1846 Vancouver, Canada, July 30 - August 4, 2017. ©2017 Association for Computational Linguistics

## 1 Introduction

A recent trend to measure progress towards machine reading is to test a system's ability to answer questions about a document it has to comprehend. Towards this end, several large-scale datasets of cloze-style questions over a context document have been introduced recently, which allow the training of supervised machine learning systems Hermann et al. (2015); Hill et al. (2016); Onishi et al. (2016). Such datasets can be easily constructed automatically and the unambiguous nature of their queries provides an objective benchmark to measure a system's performance at text comprehension.

Deep learning models have been shown to outperform traditional shallow approaches on text comprehension tasks Hermann et al. (2015). The success of many recent models can be attributed primarily to two factors: (1) _Multi-hop architectures_Weston et al. (2015); Sordoni et al. (2016); Shen et al. (2016), allow a model to scan the document and the question iteratively for multiple passes. (2) _Attention mechanisms_Chen et al. (2016); Hermann et al. (2015) borrowed from the machine translation literature Bahdanau et al. (2014), allow the model to focus on appropriate subparts of the context document. Intuitively, the multi-hop architecture allows the reader to incrementally refine token representations, and the attention mechanism re-weights different parts in the document according to their relevance to the query.

The effectiveness of multi-hop reasoning and attentions have been explored orthogonally so far in the literature. In this paper, we focus on combining both in a complementary manner, by designing a novel attention mechanism which gates the evolving token representations across hops. More specifically, unlike existing models where the query attention is applied either token-wise Hermann et al. (2015); Kadlec et al. (2016); Chen et al. (2016); Hill et al. (2016) or sentence-wise Weston et al. (2015); Sukhbaatar et al. (2015) to allow weighted aggregation, the Gated-Attention (GA) module proposed in this work allows the query to directly interact with each dimension of the token embeddings at the semantic-level, and is applied layer-wise as information filters during the multi-hop representation learning process. Such a fine-grained attention enables our model to learn conditional token representations w.r.t. the given question, leading to accurate answer selections.

We show in our experiments that the proposed GA reader, despite its relative simplicity, consis

tently improves over a variety of strong baselines on three benchmark datasets . Our key contribution, the GA module, provides a significant improvement for large datasets. Qualitatively, visualization of the attentions at intermediate layers of the GA reader shows that in each layer the GA reader attends to distinct salient aspects of the query which help in determining the answer.

## 2 Related Work

The cloze-style QA task involves tuples of the form \((d,q,a,{\cal C})\), where \(d\) is a document (context), \(q\) is a query over the contents of \(d\), in which a phrase is replaced with a placeholder, and \(a\) is the answer to \(q\), which comes from a set of candidates \({\cal C}\). In this work we consider datasets where each candidate \(c\in{\cal C}\) has at least one token which also appears in the document. The task can then be described as: given a document-query pair \((d,q)\), find \(a\in{\cal C}\) which answers \(q\). Below we provide an overview of representative neural network architectures which have been applied to this problem.

_LSTMs with Attention:_ Several architectures introduced in Hermann et al. (2015) employ LSTM units to compute a combined document-query representation \(g(d,q)\), which is used to rank the candidate answers. These include the **DeepLSTM Reader** which performs a single forward pass through the concatenated _(document, query)_ pair to obtain \(g(d,q)\); the **Attentive Reader** which first computes a document vector \(d(q)\) by a weighted aggregation of words according to attentions based on \(q\), and then combines \(d(q)\) and \(q\) to obtain their joint representation \(g(d(q),q)\); and the **Impatient Reader** where the document representation is built incrementally. The architecture of the Attentive Reader has been simplified recently in **Stanford Attentive Reader**, where shallower recurrent units were used with a bilinear form for the query-document attention Chen et al. (2016).

_Attention Sum:_ The **Attention-Sum (AS) Reader**Kadlec et al. (2016) uses two bi-directional GRU networks Cho et al. (2015) to encode both \(d\) and \(q\) into vectors. A probability distribution over the entities in \(d\) is obtained by computing dot products between \(q\) and the entity embeddings and taking a softmax. Then, an aggregation scheme named _pointer-sum attention_ is further applied to sum the probabilities of the same entity, so that frequent entities the document will be favored compared to rare ones. Building on the AS Reader, the **Attention-over-Attention (AoA) Reader**Cui et al. (2017) introduces a two-way attention mechanism where the query and the document are mutually attentive to each other.

_Multi-hop Architectures:_ **Memory Networks****(MemNets)** were proposed in Weston et al. (2015), where each sentence in the document is encoded to a memory by aggregating nearby words. Attention over the memory slots given the query is used to compute an overall memory and to renew the query representation over multiple iterations, allowing certain types of reasoning over the salient facts in the memory and the query.

**Neural Semantic Encoders (NSE)**Munkhdalai and Yu (2017) extended MemNets by introducing a _write_ operation which can evolve the memory over time during the course of reading. Iterative reasoning has been found effective in several more recent models, including the **Iterative Attentive Reader**Sordoni et al. (2016) and **ReasoNet**Shen et al. (2016). The latter allows dynamic reasoning steps and is trained with reinforcement learning.

Other related works include **Dynamic Entity Representation network (DER)**Kobayashi et al. (2016), which builds dynamic representations of the candidate answers while reading the document, and accumulates the information about an entity by max-pooling; **EpiReader**Trischler et al. (2016) consists of two networks, where one proposes a small set of candidate answers, and the other reranks the proposed candidates conditioned on the query and the context; **Bi-Directional Attention Flow network (BiDAF)**Seo et al. (2017) adopts a multi-stage hierarchical architecture along with a flow-based attention mechanism; Bajgar et al. (2016) showed a 10% improvement on the CBT corpus Hill et al. (2016) by training the AS Reader on an augmented training set of about 14 million examples, making a case for the community to exploit data abundance. The focus of this paper, however, is on designing models which exploit the available data efficiently.

## 3 Gated-Attention Reader

Our proposed GA readers perform multiple hops over the document (context), similar to the Memory Networks architecture Sukhbaatar et al. (2015). Multi-hop architectures mimic the multi-step comprehension process of human readers, and have shown promising results in several recent models for text comprehension Sordoni et al. ( 

2016; Kumar et al., 2016; Shen et al., 2016). The contextual representations in GA readers, namely the embeddings of words in the document, are iteratively refined across hops until reaching a final attention-sum module Kadlec et al. (2016) which maps the contextual representations in the last hop to a probability distribution over candidate answers.

The attention mechanism has been introduced recently to model human focus, leading to significant improvement in machine translation and image captioning Bahdanau et al. (2014); Mnih et al. (2014). In reading comprehension tasks, ideally, the semantic meanings carried by the contextual embeddings should be aware of the query across hops. As an example, human readers are able to keep the question in mind during multiple passes of reading, to successively mask away information irrelevant to the query. However, existing neural network readers are restricted to either attend to tokens Hermann et al. (2015); Chen et al. (2016) or entire sentences Weston et al. (2015), with the assumption that certain sub-parts of the document are more important than others. In contrast, we propose a finer-grained model which attends to components of the semantic representation being built up by the GRU. The new attention mechanism, called _gated-attention_, is implemented via _multiplicative_ interactions between the query and the contextual embeddings, and is applied per hop to act as fine-grained information filters during the multi-step reasoning. The filters weigh individual components of the vector representation of _each_ token in the document separately.

The design of gated-attention layers is motivated by the effectiveness of multiplicative interaction among vector-space representations, e.g., in various types of recurrent units Hochreiter and Schmidhuber (1997); Wu et al. (2016) and in relational learning Yang et al. (2014); Kiros et al. (2014). While other types of compositional operators are possible, such as concatenation or addition Mitchell Lapata (2008), we find that multiplication has strong empirical performance (section 4.3), where query representations naturally serve as information filters across hops.

### Model Details

Several components of the model use a Gated Recurrent Unit (GRU) Cho et al. (2015) which maps an input sequence \(X=[x_{1},x_{2},\ldots,x_{T}]\) to an output sequence \(H=[h_{1},h_{2},\ldots,h_{T}]\) as follows:

\[r_{t} =\sigma(W_{r}x_{t}+U_{r}h_{t-1}+b_{r}),\] \[z_{t} =\sigma(W_{z}x_{t}+U_{z}h_{t-1}+b_{z}),\] \[\tilde{h}_{t} =\tanh(W_{h}x_{t}+U_{h}(r_{t}\odot h_{t-1})+b_{h}),\] \[h_{t} =(1-z_{t})\odot h_{t-1}+z_{t}\odot\tilde{h}_{t}.\]

where \(\odot\) denotes the Hadamard product or the element-wise multiplication. \(r_{t}\) and \(z_{t}\) are called the _reset_ and _update_ gates respectively, and \(\hat{h}_{t}\) the _candidate output_. A Bi-directional GRU (BiGRU) processes the sequence in both forward and backward directions to produce two sequences \([h^{f}_{1},h^{f}_{2},\dots,h^{f}_{T}]\) and \([h^{b}_{1},h^{b}_{2},\dots,h^{b}_{T}]\), which are concatenated at the output

\[\mathop{\mathrm{GRU}}\limits^{\longleftrightarrow}(X)=[h^{f}_{1}\|h^{b}_{ T},\dots,h^{f}_{T}\|h^{b}_{1}] \tag{1}\]

where \(\mathop{\mathrm{GRU}}\nolimits(X)\) denotes the _full output_ of the Bi-GRU obtained by concatenating each forward state \(h^{f}_{i}\) and backward state \(h^{b}_{T-i+1}\) at step \(i\) given the input \(X\). Note \(\mathop{\mathrm{GRU}}\limits^{\longleftrightarrow}(X)\) is a matrix in \(\mathbb{R}^{2n_{h}\times T}\) where \(n_{h}\) is the number of hidden units in GRU.

Let \(X^{(0)}=[x^{(0)}_{1},x^{(0)}_{2},\dots x^{(0)}_{|D|}]\) denote the token embeddings of the document, which are also inputs at layer 1 for the document reader below, and \(Y=[y_{1},y_{2},\dots y_{|Q|}]\) denote the token embeddings of the query. Here \(|D|\) and \(|Q|\) denote the document and query lengths respectively.

#### 3.1.1 Multi-Hop Architecture

Fig. 1 illustrates the Gated-Attention (GA) reader. The model reads the document and the query over \(K\) horizontal layers, where layer \(k\) receives the contextual embeddings \(X^{(k-1)}\) of the document from the previous layer. The document embeddings are transformed by taking the full output of a document Bi-GRU (indicated in blue in Fig. 1):

\[D^{(k)}=\mathop{\mathrm{GRU}}\limits^{(k)}_{D}(X^{(k-1)}) \tag{2}\]

At the same time, a layer-specific query representation is computed as the full output of a separate query Bi-GRU (indicated in green in Figure 1):

\[Q^{(k)}=\mathop{\mathrm{GRU}}\limits^{(k)}_{Q}(Y) \tag{3}\]

Next, _Gated-Attention_ is applied to \(D^{(k)}\) and \(Q^{(k)}\) to compute inputs for the next layer \(X^{(k)}\).

\[X^{(k)}=\operatorname{GA}(D^{(k)},Q^{(k)}) \tag{4}\]

where GA is defined in the following subsection 



#### 3.1.2 Gated-Attention Module

For brevity, let us drop the superscript \(k\) in this subsection as we are focusing on a particular layer. For each token \(d_{i}\) in \(D\), the GA module forms a token-specific representation of the query \(\tilde{q}_{i}\) using soft attention, and then multiplies the query representation element-wise with the document token representation. Specifically, for \(i=1,\ldots,|D|\):

\[\alpha_{i} =\text{softmax}(Q^{\top}d_{i}) \tag{5}\] \[\tilde{q}_{i} =Q\alpha_{i}\] \[x_{i} =d_{i}\odot\tilde{q}_{i} \tag{6}\]

In equation (6) we use the multiplication operator to model the interactions between \(d_{i}\) and \(\tilde{q}_{i}\). In the experiments section, we also report results for other choices of gating functions, including addition \(x_{i}=d_{i}+\tilde{q}_{i}\) and concatenation \(x_{i}=d_{i}\lVert\tilde{q}_{i}\).

#### 3.1.3 Answer Prediction

Let \(q_{\ell}^{(K)}=q_{\ell}^{f}\rVert q_{T-\ell+1}^{b}\) be an intermediate output of the final layer query Bi-GRU at the location \(\ell\) of the cloze token in the query, and \(D^{(K)}=\overset{\longleftrightarrow}{\operatorname*{GRU}}_{D}^{(K)}(X^{( K-1)})\) be the full output of final layer document Bi-GRU. To obtain the probability that a particular token in the document answers the query, we take an inner-product between these two, and pass through a softmax layer:

\[s=\text{softmax}((q_{\ell}^{(K)})^{T}D^{(K)}) \tag{7}\]

where vector \(s\) defines a probability distribution over the \(|D|\) tokens in the document. The probability of a particular candidate \(c\in\mathcal{C}\) as being the answer is then computed by aggregating the probabilities of all document tokens which appear in \(c\) and renormalizing over the candidates:

\[\Pr(c|d,q)\propto\sum_{i\in\mathbb{I}(c,d)}s_{i} \tag{8}\]

where \(\mathbb{I}(c,d)\) is the set of positions where a token in \(c\) appears in the document \(d\). This aggregation operation is the same as the _pointer sum attention_ applied in the AS Reader Kadlec et al. (2016).

Finally, the candidate with maximum probability is selected as the predicted answer:

\[a^{*}=\mathrm{argmax}_{c\in\mathcal{C}}~{}~{}\Pr(c|d,q). \tag{9}\]

During the training phase, model parameters of GA are updated w.r.t. a cross-entropy loss between the predicted probabilities and the true answers.

#### 3.1.4 Further Enhancements

Character-level EmbeddingsGiven a token \(w\) from the document or query, its vector space representation is computed as \(x=L(w)||C(w)\). \(L(w)\) retrieves the word-embedding for \(w\) from a lookup table \(L\in\mathbb{R}^{|V|\times n_{l}}\), whose rows hold a vector for each unique token in the vocabulary. We also utilize a character composition model \(C(w)\) which generates an orthographic embedding of the token. Such embeddings have been previously shown to be helpful for tasks like Named Entity Recognition Yang et al. (2016) and dealing with OOV tokens at test time Dhingra et al. (2016). The embedding \(C(w)\) is generated by taking the final outputs \(z_{n_{c}}^{f}\) and \(z_{n_{c}}^{b}\) of a Bi-GRU applied to embeddings from

Figure 1: Gated-Attention Reader. Dashed lines represent dropout connections.



a lookup table of characters in the token, and applying a linear transformation:

\[z =z_{n_{c}}^{f}||z_{n_{c}}^{b}\] \[C(w) =Wz+b\]

_Question Evidence Common Word Feature (qecomm)_Li et al. (2016) recently proposed a simple token level indicator feature which significantly boosts reading comprehension performance in some cases. For each token in the document we construct a one-hot vector \(f_{i}\in\{0,1\}^{2}\) indicating its presence in the query. It can be incorporated into the GA reader by assigning a feature lookup table \(F\in\mathbb{R}^{n_{F}\times 2}\) (we use \(n_{F}=2\)), taking the feature embedding \(e_{i}=f_{i}^{T}F\) and appending it to the inputs of the last layer document BiGRU as, \(x_{i}^{(K)}\|f_{i}\) for all \(i\). We conducted several experiments both with and without this feature and observed some interesting trends, which are discussed below. Henceforth, we refer to this feature as the _qe-comm feature_ or just _feature_.

## 4 Experiments and Results

### Datasets

We evaluate the GA reader on five large-scale datasets recently proposed in the literature. The first two, CNN and Daily Mail news stories2 consist of articles from the popular CNN and Daily Mail websites Hermann et al. (2015). A query over each article is formed by removing an entity from the short summary which follows the article. Further, entities within each article were anonymized to make the task purely a comprehension one. N-gram statistics, for instance, computed over the entire corpus are no longer useful in such an anonymized corpus.

Footnote 2: [https://github.com/deepmind/rc-data](https://github.com/deepmind/rc-data)

The next two datasets are formed from two different subsets of the Children's Book Test (CBT)3Hill et al. (2016). Documents consist of 20 contiguous sentences from the body of a popular children's book, and queries are formed by deleting a token from the 21st sentence. We only focus on subsets where the deleted token is either a common noun (CN) or named entity (NE) since simple language models already give human-level performance on the other types (cf. Hill et al. (2016)).

Footnote 3: [http://www.thesspermwhale.com/jaseeweston/babi/CBTest.tgz](http://www.thesspermwhale.com/jaseeweston/babi/CBTest.tgz)

The final dataset is Who Did What4 (WDW) Onishi et al. (2016), constructed from the LDC English Gigaword newswire corpus. First, article pairs which appeared around the same time and with overlapping entities are chosen, and then one article forms the document and a cloze query is constructed from the other. Missing tokens are always person named entities. Questions which are easily answered by simple baselines are filtered out, to make the task more challenging. There are two versions of the training set--a small but focused "Strict" version and a large but noisy "Re-laxed" version. We report results on both settings which share the same validation and test sets. Statistics of all the datasets used in our experiments are summarized in the Appendix (Table 5).

Footnote 4: [https://titcnlp.github.io/who_did_what/](https://titcnlp.github.io/who_did_what/)

### Performance Comparison

Tables 1 and 3 show a comparison of the performance of GA Reader with previously published results on WDW and CNN, Daily Mail, CBT datasets respectively. The numbers reported for GA Reader are for single best models, though we compare to both ensembles and single models from prior work. GA Reader- refers to an earlier version of the model, unpublished but described in a preprint, with the following differences--(1) it does not utilize token-specific attentions within the GA module, as described in equation (5), (2) it does not use a character composition model, (3) it is initialized with word embeddings pretrained on the corpus itself rather than GloVe. A detailed analysis of these differences is studied in the next section. Here we present 4 variants of the latest GA Reader, using combinations of whether the qe-comm feature is used (+feature) or not, and whether the word lookup table \(L(w)\) is updated during training or fixed to its initial value. Other hyperparameters are listed in Appendix A.

Interestingly, we observe that feature engineering leads to significant improvements for WDW and CBT datasets, but not for CNN and Daily Mail datasets. We note that anonymization of the latter datasets means that there is already some feature engineering (it adds hints about whether a token is an entity), and these are much larger than the other four. In machine learning it is common to see the effect of feature engineering diminish with increasing data size. Similarly, fixing the word embeddings provides an improvement for the WDW 

and CBT, but not for CNN and Daily Mail. This is not surprising given that the latter datasets are larger and less prone to overfitting.

Comparing with prior work, on the WDW dataset the basic version of the GA Reader outperforms all previously published models when trained on the Strict setting. By adding the qe-comm feature the performance increases by 3.2% and 3.5% on the Strict and Relaxed settings respectively to set a new state of the art on this dataset. On the CNN and Daily Mail datasets the GA Reader leads to an improvement of 3.2% and 4.3% respectively over the best previous single models. They also outperform previous ensemble models, setting a new state of that art for both datasets. For CBT-NE, GA Reader with the qe-comm feature outperforms all previous single and ensemble models except the AS Reader trained on the much larger BookTest Corpus Bajgar et al. (2016). Lastly, on CBT-CN the GA Reader with the qe-comm feature outperforms all previously published single models except the NSE, and AS Reader trained on a larger corpus. For each of the 4 datasets on which GA achieves the top performance, we conducted one-sample proportion tests to test whether GA is significantly better than the second-best baseline. The p-values are \(0.319\) for CNN, \(<\)\(0.00001\) for DailyMail, \(0.028\) for CBT-NE, and \(<\)\(0.00001\) for WDW. In other words, GA statistically significantly outperforms all other baselines on 3 out of those 4 datasets at the \(5\%\) significance level. The results could be even more significant under paired tests, however we did not have access to the predictions from the baselines.

### GA Reader Analysis

In this section we do an ablation study to see the effect of Gated Attention. We compare the GA Reader as described here to a model which is exactly the same in all aspects, except that it passes document embeddings \(D^{(k)}\) in each layer directly to the inputs of the next layer without using the GA module. In other words \(X^{(k)}=D^{(k)}\) for all \(k>0\). This model ends up using only one query GRU at the output layer for selecting the answer from the document. We compare these two variants both with and without the qe-comm feature on CNN and WDW datasets for three subsets of the training data - 50%, 75% and 100%. Test set accuracies for these settings are shown in Figure 2. On CNN when tested without feature engineering, we observe that GA provides a significant boost in performance compared to without GA. When tested with the feature it still gives an improvement, but the improvement is significant only with 100% training data. On WDW-Strict, which is a third of the size of CNN, without the feature we see an improvement when using GA versus without using GA, which becomes significant as the training set size increases. When tested with the feature on WDW, for a small data size without GA does better than with GA, but as the dataset size increases they become equivalent. We conclude that GA provides a boost in the absence of feature engineering, or as the training set size increases.

Next we look at the question of how to gate intermediate document reader states from the query, i.e. what operation to use in equation 6. Table

\begin{table}
\begin{tabular}{l|c c|c c} \hline \multirow{2}{*}{**Model**} & \multicolumn{3}{c|}{**Strict**} & \multicolumn{2}{c}{**Relaxed**} \\ \cline{2-5}  & Val & Test & Val & Test \\ \hline Human † & – & 84 & – & – \\ \hline Attentive Reader † & – & 53 & – & 55 \\ AS Reader † & – & 57 & – & 59 \\ Stanford AR ‡ & – & 64 & – & 65 \\ NSE † & 66.5 & 66.2 & 67.0 & 66.7 \\ \hline GA– † & – & 57 & – & 60.0 \\ GA (update \(L(w)\)) & 67.8 & 67.0 & 67.0 & 66.6 \\ GA (fix \(L(w)\)) & 68.3 & 68.0 & 69.6 & 69.1 \\ GA (+feature, update \(L(w)\)) & 70.1 & 69.5 & 70.9 & 71.0 \\ GA (+feature, fix \(L(w)\)) & **71.6** & **71.2** & **72.6** & **72.6** \\ \hline \end{tabular}
\end{table}
Table 1: Validation/Test accuracy (%) on WDW dataset for both “Strict” and “Relaxed” settings. Results with “†” are cf previously published works.

\begin{table}
\begin{tabular}{l|c|c} \hline \multirow{2}{*}{**Gating Function**} & \multicolumn{2}{c}{**Accuracy**} \\ \cline{2-3}  & Val & Test \\ \hline Sum & 64.9 & 64.5 \\ Concatenate & 64.4 & 63.7 \\ Multiply & **68.3** & **68.0** \\ \hline
**K** & & \\ \hline
1 (AS) † & – & 57 \\
2 & 65.6 & 65.6 \\
3 & **68.3** & 68.0 \\
4 & **68.3** & **68.2** \\ \hline \end{tabular}
\end{table}
Table 2: **Top: Performance of different gating functions. Bottom: Effect of varying the number of hops \(K\). Results on WDW without using the qe-comm feature and with fixed \(L(w)\).

 

2 (top) shows the performance on WDW dataset for three common choices - sum (\(x=d+q\)), concatenate (\(x=d\|q\)) and multiply (\(x=d\odot q\)). Empirically we find element-wise multiplication does significantly better than the other two, which justifies our motivation to "filter" out document features which are irrelevant to the query.

At the bottom of Table 2 we show the effect of varying the number of hops \(K\) of the GA Reader on the final performance. We note that for \(K=1\), our model is equivalent to the AS Reader without any GA modules. We see a steep and steady rise in accuracy as the number of hops is increased from \(K=1\) to \(3\), which remains constant beyond that. This is a common trend in machine learning as model complexity is increased, however we note that a multi-hop architecture is important to achieve a high performance for this task, and provide further evidence for this in the next section.

### Ablation Study for Model Components

Table 4 shows accuracy on WDW by removing one component at a time. The steepest reduction is observed when we replace pretrained GloVe vectors with those pretrained on the corpus itself. GloVe vectors were trained on a large corpus of about 6 billion tokens Pennington et al. (2014), and provide an important source of prior knowl

\begin{table}
\begin{tabular}{l|c c|c c|c c|c c} \hline \multirow{2}{*}{**Model**} & \multicolumn{2}{c|}{**CNN**} & \multicolumn{2}{c|}{**Daily Mail**} & \multicolumn{2}{c|}{**CBT-NE**} & \multicolumn{2}{c}{**CBT-CN**} \\ \cline{2-9}  & Val & Test & Val & Test & Val & Test & Val & Test \\ \hline Humans (query) † & – & – & – & – & – & 52.0 & – & 64.4 \\ Humans (context + query) ‡ & – & – & – & – & – & 81.6 & – & 81.6 \\ \hline LSTMs (context + query) † & – & – & – & – & 51.2 & 41.8 & 62.6 & 56.0 \\ Deep LSTM Reader † & 55.0 & 57.0 & 63.3 & 62.2 & – & – & – & – \\ Attentive Reader † & 61.6 & 63.0 & 70.5 & 69.0 & – & – & – & – \\ Impatient Reader † & 61.8 & 63.8 & 69.0 & 68.0 & – & – & – & – \\ MemNets † & 63.4 & 66.8 & – & – & 70.4 & 66.6 & 64.2 & 63.0 \\ AS Reader † & 68.6 & 69.5 & 75.0 & 73.9 & 73.8 & 68.6 & 68.8 & 63.4 \\ DER Network † & 71.3 & 72.9 & – & – & – & – & – & – \\ Stanford AR (relabeling) † & 73.8 & 73.6 & 77.6 & 76.6 & – & – & – & – \\ Iterative Attentive Reader † & 72.6 & 73.3 & – & – & 75.2 & 68.6 & 72.1 & 69.2 \\ EpiReader † & 73.4 & 74.0 & – & – & 75.3 & 69.7 & 71.5 & 67.4 \\ AoA Reader † & 73.1 & 74.4 & – & – & 77.8 & 72.0 & 72.2 & 69.4 \\ ReasoNet † & 72.9 & 74.7 & 77.6 & 76.6 & – & – & – & – \\ NSE † & – & – & – & – & 78.2 & 73.2 & 74.3 & **71.9** \\ BiDAF † & 76.3 & 76.9 & 80.3 & 79.6 & – & – & – & – \\ \hline MemNets (ensemble) † & 66.2 & 69.4 & – & – & – & – & – & – \\ AS Reader (ensemble) † & 73.9 & 75.4 & 78.7 & 77.7 & 76.2 & 71.0 & 71.1 & 68.9 \\ Stanford AR (relabeling,ensemble) † & 77.2 & 77.6 & 80.2 & 79.2 & – & – & – & – \\ Iterative Attentive Reader (ensemble) † & 75.2 & 76.1 & – & – & 76.9 & 72.0 & 74.1 & 71.0 \\ EpiReader (ensemble) † & – & – & – & – & 76.6 & 71.8 & 73.6 & 70.6 \\ \hline AS Reader (+BookTest) † � 

edge for the model. Note that the strongest baseline on WDW, NSE (Munkhdalai and Yu, 2017), also uses pretrained GloVe vectors, hence the comparison is fair in that respect. Next, we observe a substantial drop when removing token-specific attentions over the query in the GA module, which allow gating individual tokens in the document only by parts of the query relevant to that token rather than the overall query representation. Finally, removing the character embeddings, which were only used for WDW and CBT, leads to a reduction of about 1% in the performance.

### Attention Visualization

To gain an insight into the reading process employed by the model we analyzed the attention distributions at intermediate layers of the reader. Figure 3 shows an example from the validation set of WDW dataset (several more are in the Appendix). In each figure, the left and middle plots visualize attention over the query (equation 5) for candidates in the document after layers 1 & 2 respectively. The right plot shows attention over candidates in the document of cloze placeholder (\({\tt XXX}\)) in the query at the final layer. The full document, query and correct answer are shown at the bottom.

A generic pattern observed in these examples is that in intermediate layers, candidates in the document (shown along rows) tend to pick out salient tokens in the query which provide clues about the cloze, and in the final layer the candidate with the highest match with these tokens is selected as the answer. In Figure 3 there is a high attention of the correct answer on financial regulatory standards in the first layer, and on us president in the second layer. The incorrect answer, in contrast, only attends to one of these aspects, and hence receives a lower score in the final layer despite the n-gram overlap it has with the cloze token in the query. Importantly, different layers tend to focus on different tokens in the query, supporting the hypothesis that the multi-hop architecture of GA Reader is able to combine distinct pieces of information to answer the query.

## 5 Conclusion

We presented the Gated-Attention reader for answering cloze-style questions over documents. The GA reader features a novel multiplicative gating mechanism, combined with a multi-hop architecture. Our model achieves the state-of-the-art performance on several large-scale benchmark datasets with more than 4% improvements over competitive baselines. Our model design is backed up by an ablation study showing statistically significant improvements of using Gated Attention as information filters. We also showed empirically that multiplicative gating is superior to addi

Figure 2: Performance in accuracy with and without the Gated-Attention module over different training sizes. \(p\)-values for an exact one-sided McNemar’s test are given inside the parentheses for each setting.

 

tion and concatenation operations for implementing gated-attentions, though a theoretical justification remains part of future research goals. Analysis of document and query attentions in intermediate layers of the reader further reveals that the model iteratively attends to different aspects of the query to arrive at the final answer. In this paper we have focused on text comprehension, but we believe that the Gated-Attention mechanism may benefit other tasks as well where multiple sources of information interact.

## Acknowledgments

This work was funded by NSF under CCF1414030 and Google Research.

## References

* Bahdanau et al. (2014) Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. Neural machine translation by jointly learning to align and translate. _arXiv preprint arXiv:1409.0473_, 2014.
* Bajgar et al. (2016) Ondrej Bajgar, Rudolf Kadlec, and Jan Kleindienst. Embracing data abundance: Booktest dataset for reading comprehension. _arXiv preprint arXiv:1610.00956_, 2016.
* Chen et al. (2016) Danqi Chen, Jason Bolton, and Christopher D Manning. A thorough examination of the cnn/daily mail reading comprehension task. _ACL_, 2016.
* Cho et al. (2015) Kyunghyun Cho, Bart Van Merrienboer, Caglar Gulcehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. Learning phrase representations using rnn encoder-decoder for statistical machine translation. _ACL_, 2015.
* Cui et al. (2017) Yiming Cui, Zhipeng Chen, Si Wei, Shijin Wang, Ting Liu, and Guoping Hu. Attention-over-attention neural networks for reading comprehension. _ACL_, 2017.
* Dhingra et al. (2016) Bhuwan Dhingra, Zhong Zhou, Dylan Fitzpatrick, Michael Muehl, and William W Cohen. Tweet2vec: Character-based distributed representations for social media. _ACL_, 2016.
* Hermann et al. (2015) Karl Moritz Hermann, Tomas Kocisky, Edward Grefenstette, Lasse Espeholt, Will Kay, Mustafa Suleyman, and Phil Blunsom. Teaching machines to read and comprehend. In _Advances in Neural Information Processing Systems_, pp. 1684-1692, 2015.
* Hill et al. (2016) Felix Hill, Antoine Bordes, Sumit Chopra, and Jason Weston. The goldilocks principle: Reading children's books with explicit memory representations. _ICLR_, 2016.
* Hochreiter & Schmidhuber (1997) Sepp Hochreiter and Jurgen Schmidhuber. Long short-term memory. _Neural computation_, 9(8):1735-1780, 1997.
* Kadlec et al. (2016) Rudolf Kadlec, Martin Schmid, Ondrej Bajgar, and Jan Kleindienst. Text understanding with the attention sum reader network. _ACL_, 2016.
* Kingma & Ba (2015) Diederik Kingma and Jimmy Ba. Adam: A method for stochastic optimization. _ICLR_, 2015.
* Kiros et al. (2014) Ryan Kiros, Richard Zemel, and Ruslan R Salakhutdinov. A multiplicative model for learning distributed text-based attribute representations. In _Advances in Neural Information Processing Systems_, pp. 2348-2356, 2014.

Figure 3: Layer-wise attention visualization of GA Reader trained on WDW-Strict. See text for details.

 

Sosuke Kobayashi, Ran Tian, Naoaki Okazaki, and Kentaro Inui. Dynamic entity representations with max-pooling improves machine reading. In _NAACL-HLT_, 2016.
* Kumar et al. (2016) Ankit Kumar, Ozan Irsoy, Jonathan Su, James Bradbury, Robert English, Brian Pierce, Peter Ondruska, Ishaan Gulrajani, and Richard Socher. Ask me anything: Dynamic memory networks for natural language processing. _ICML_, 2016.
* Li et al. (2016) Peng Li, Wei Li, Zhengyan He, Xuguang Wang, Ying Cao, Jie Zhou, and Wei Xu. Dataset and neural recurrent sequence labeling model for open-domain factoid question answering. _arXiv preprint arXiv:1607.06275_, 2016.
* Mitchell & Lapata (2008) Jeff Mitchell and Mirella Lapata. Vector-based models of semantic composition. In _ACL_, pp. 236-244, 2008.
* Mnih et al. (2014) Volodymyr Mnih, Nicolas Heess, Alex Graves, et al. Recurrent models of visual attention. In _Advances in Neural Information Processing Systems_, pp. 2204-2212, 2014.
* Munkhhdalai & Yu (2017a) Tsendsuren Munkhhdalai and Hong Yu. Neural semantic encoders. _EACL_, 2017a.
* Munkhdalai & Yu (2017b) Tsendsuren Munkhdalai and Hong Yu. Reasoning with memory augmented neural networks for language comprehension. _ICLR_, 2017b.
* Onishi et al. (2016) Takeshi Onishi, Hai Wang, Mohit Bansal, Kevin Gimpel, and David McAllester. Who did what: A large-scale person-centered cloze dataset. _EMNLP_, 2016.
* Pascanu et al. (2013) Razvan Pascanu, Tomas Mikolov, and Yoshua Bengio. On the difficulty of training recurrent neural networks. _ICML (3)_, 28:1310-1318, 2013.
* Pennington et al. (2014) Jeffrey Pennington, Richard Socher, and Christopher D. Manning. Glove: Global vectors for word representation. In _Empirical Methods in Natural Language Processing (EMNLP)_, pp. 1532-1543, 2014. URL [http://www.aclweb.org/anthology/D14-1162](http://www.aclweb.org/anthology/D14-1162).
* Seo et al. (2017) Minjoon Seo, Aniruddha Kembhavi, Ali Farhadi, and Hannaneh Hajishirzi. Bidirectional attention flow for machine comprehension. _ICLR_, 2017.
* Shen et al. (2016) Yelong Shen, Po-Sen Huang, Jianfeng Gao, and Weizhu Chen. Reasonet: Learning to stop reading in machine comprehension. _arXiv preprint arXiv:1609.05284_, 2016.
* Sordoni et al. (2016) Alessandro Sordoni, Phillip Bachman, and Yoshua Bengio. Iterative alternating neural attention for machine reading. _arXiv preprint arXiv:1606.02245_, 2016.
* Sukhbaatar et al. (2015) Sainbayar Sukhbaatar, Jason Weston, Rob Fergus, et al. End-to-end memory networks. In _Advances in Neural Information Processing Systems_, pp. 2431-2439, 2015.
* Team (2016) Theano Development Team. Theano: A Python framework for fast computation of mathematical expressions. _arXiv e-prints_, abs/1605.02688, May 2016. URL [http://arxiv.org/abs/1605.02688](http://arxiv.org/abs/1605.02688).
* Trischler et al. (2016) Adam Trischler, Zheng Ye, Xingdi Yuan, and Kaheer Suleman. Natural language comprehension with the epireader. _EMNLP_, 2016.
* Weston et al. (2015) Jason Weston, Sumit Chopra, and Antoine Bordes. Memory networks. _ICLR_, 2015.
* Wu et al. (2016) Yuhuai Wu, Saizheng Zhang, Ying Zhang, Yoshua Bengio, and Ruslan Salakhutdinov. On multiplicative integration with recurrent neural networks. _Advances in Neural Information Processing Systems_, 2016.
* Yang et al. (2014) Bishan Yang, Wen-tau Yih, Xiaodong He, Jianfeng Gao, and Li Deng. Learning multi-relational semantics using neural-embedding models. _NIPS Workshop on Learning Semantics_, 2014.
* Yang et al. (2016) Zhilin Yang, Ruslan Salakhutdinov, and William Cohen. Multi-task cross-lingual sequence tagging from scratch. _arXiv preprint arXiv:1603.06270_, 2016.

## Appendix A Implementation Details

Our model was implemented using the Theano (Theano Development Team, 2016) and Lasagne5Python libraries. We used stochastic gradient descent with ADAM updates for optimization, which combines classical momentum and adaptive gradients (Kingma & Ba, 2015). The batch size was 32 and the initial learning rate was \(5\times 10^{-4}\) which was halved every epoch after the second epoch. The same setting is applied to all models and datasets. We also used gradient clipping with a threshold of 10 to stabilize GRU training (Pascanu et al., 2013). We set the number of layers \(K\) to be \(3\) for all experiments. The number of hidden units for the character GRU was set to 50. The remaining two hyperparameters--size of document and query GRUs, and dropout rate--were tuned on the validation set, and their optimal values are shown in Table 6. In general, the optimal GRU size increases and the dropout rate decreases as the corpus size increases.

Footnote 5: [https://lasagne.readthedocs.io/en/latest/](https://lasagne.readthedocs.io/en/latest/)

The word lookup table was initialized with \(100d\) GloVe vectors6Pennington et al. (2014) and OOV tokens at test time were assigned unique random vectors. We empirically observed that initializing with pre-trained embeddings gives higher performance compared to random initialization for all 

datasets. Furthermore, for smaller datasets (WDW and CBT) we found that fixing these embeddings to their pretrained values led to higher test performance, possibly since it avoids overfitting. We do not use the character composition model for CNN and Daily Mail, since their entities (and hence candidate answers) are anonymized to generic tokens. For other datasets the character lookup table was randomly initialized with \(25d\) vectors. All other parameters were initialized to their default values as specified in the Lasagne library.
 


* [184] DOCK. _et al._, "The 2007-day (2009): A new method for the use of a new method to generate the data from the same set of data sets," in _Proceedings of the 2006 ACM SIGGRAPH international conference on Knowledge discovery and data mining_, pages 35-40, Springer, New York, NY, USA, 2009.
* [185] J. M. L. Miller, "A new method for the use of a new method for the 


* [150] D. C. M. A. K. S. R. Ellis, and J. E. G. Learned, "The New Generation of the Universe: The Case for Dark Matter," _Journal of Cosmology**63**, no. 2, p. 479-483, (2000).
* [151] P. F. Harrison, "A new generation of dark matter from a universe," _Physics Letters B_**363**, no. 1, pp. 1-10, (2000).
* [152] H. N. Mohapatra, "On the nature of dark matter in the universe," _Nature_**406**, no. 1, pp. 1-10, (2000).
* [153] T. W. Press, "The Early Universe: Cosmological Implications," _Astrophys. J. Suppl._**40**, no. 1, pp. 1-10, (2000).
* [154] L. Randall, "Dark Matter," _Annals Phys._**42**, no. 1, pp. 1-10, (1998).
* [155] V. Sahni et al., "Inflationary Cosmology with Large Scale Structure," _Physics Reports_**42**, no. 1, pp. 1-10, (2000).

[MISSING_PAGE_POST]

 


Figure 6: Layer-wise attention visualization of GA Reader trained on WDW-Strict. See text for details.

 


Figure 7: Layer-wise attention visualization of GA Reader trained on WDW-Strict. See text for details.