Title: A CONTEXT-AWARE ATTENTION NETWORK FOR INTERACTIVE QUESTION ANSWERING

Abstract: We develop a new model for Interactive Question Answering (IQA), using Gated-Recurrent-Unit recurrent networks (GRUs) as encoders for statements and questions, and another GRU as a decoder for outputs. Distinct from previous work, our approach employs context-dependent word-level attention for more accurate statement representations and question-guided sentence-level attention for better context modeling. Employing these mechanisms, our model accurately understands when it can output an answer or when it requires generating a supplementary question for additional input. When available, user's feedback is encoded and directly applied to update sentence-level attention to infer the answer. Extensive experiments on QA and IQA datasets demonstrate quantitatively the effectiveness of our model with significant improvement over conventional QA models.

1. INTRODUCTION:

The ultimate goal of Question Answering (QA) research is to build intelligent systems capable of naturally communicating with humans, which poses a major challenge for natural language processing and machine learning. Inspired by recent success of sequence-to-sequence models with an encoder-decoder framework (Sutskever et al., 2014; Cho et al., 2014) , researchers have attempted to apply variants of such models with explicit memory and attention to QA tasks, aiming to move a step further from machine learning to machine reasoning (Sainbayar et al., 2015; Kumar et al., 2016; Xiong et al., 2016) . Similarly, all these models employ encoders to map statements and questions to fixed-length feature vectors, and a decoder to generate outputs. Empowered by the adoption of memory and attention, they have achieved remarkable success on several challenging datasets, including the recently acclaimed Facebook bAbI dataset.

However, previous models suffer from the following important limitations. First, they fail to model context-dependent meaning of words. Different words may have different meanings in different contexts, which increases the difficulty of extracting the essential semantic logic flow of each sentence in different paragraphs. Second, many existing models only work in ideal QA settings and fail to address the uncertain situations under which models require additional user input to gather complete information to answer a given question. As shown in Table 1 , the example on the left is an ideal QA problem. We can clearly understand what the question is and then locate the relevant sentences to generate the answer. However, it is hard to answer the question in the right example, because there are two types of bedrooms mentioned in the story and we do not know which bedroom the user refers to. These scenarios with incomplete information naturally appear in human conversations, and thus, effectively handling them is a key capability of intelligent QA models.

To address the challenges presented above, we propose a Context-aware Attention Network (CAN) to learn fine-grained representations for input sentences, and develop a mechanism to interact with the user for comprehensively understanding a given question. Specifically, we employ two-level attention applied at word level and sentence level to compute representations of all input sentences.

The office is north of the kitchen.

The master bedroom is east of the garden. The garden is south of the kitchen.

The guest bedroom is east of the office. Q: What is north of the kitchen? Q: What is the bedroom east of? A: Office A: Unknown Table 1 : Two examples of QA problems. Left is an ideal QA example, where the question is very clear. Right is an example with incomplete information, where the question is ambiguous and it is difficult to provide an answer only using the input statements.

The context information extracted from the input story is allowed to influence the attention over each word, and governs the word semantic meaning contributing to a sentence representation. In addition, an interactive mechanism is activated to generate a supplementary question for the user when the model feels that it does not have complete information to answer a given question. User's feedback is then encoded and exploited to attend over all input sentences to infer the answer. Our proposed model CAN can be viewed as an encoder-decoder approach augmented with two-level attention and an interactive mechanism, rendering our model self-adaptive, as illustrated in Figure 1 . The master bedroom is east of the garden.

The guest bedroom is east of the office.

The guest bedroom is west of the hallway.

The bathroom is east of the master bedroom.

Input Module:

Sentence Attention Mechnism

Word Attention Mechanism:



Question Module:

What is the bedroom east of?

Decoder:



Answer:



Supplemetary Question:

Which bedroom, master one or guest one?

Garden:



Interactive Mechanism:

Master bedroom 

Encoder:



Context Representation:



Question:



Representation:



Answer Module:



2. RELATED WORK:

Recent work on QA has been heavily influenced by research on various models with attention and/or memory. Most of these models employ an encoder-decoder framework, and have been successfully applied to image classification (Seo et al., 2016) , image captioning (Xu et al., 2015; Mnih et al., 2014) , machine translation (Cho et al., 2014; Bahdanau et al., 2015; Luong et al., 2015) , document classification (Yang et al., 2016) , and textual/visual QA (Sainbayar et al., 2015; Yang et al., 2015; Lu et al., 2016; Kumar et al., 2016; Xiong et al., 2016) . For textual QA in the form of statementsquestion-answer triplets, Sainbayar et al. (2015) utilizes an external memory module. It maps each input sentence to an input representation space regarded as a memory component. The output representation is calculated by summarizing over input representations with different attention weights. This single-layer memory can be extended to multi-layer memory by reasoning the content and the question multiple times. Instead of simply stacking the memory layers, Kumar et al. (2016) have introduced a dynamic memory network (DMN) to update the memory vectors through a modified GRU, in which the gate weight is trained in a supervised fashion. To improve DMN by train-ing without supervision, Xiong et al. (2016) encode input sentences with a bidirectional GRU and then utilize an attention-based GRU to summarize these input sentences. Neural Turing Machine (NTM) (Graves et al., 2014) , a model with content and location-based memory addressing mechanisms, has also been used for QA tasks recently. There is other recent work about QA using external resources (Wu et al., 2015; Fader et al., 2014; Savenkov & Emory, 2016; Hermann et al., 2015; Golub & He, 2016) , and exploring dialog tasks (Weston, 2016; Bordes & Weston, 2016) .

Our model in this paper also addresses textual QA in the form of statements-question-answer triplets, but it differs from prior work in two aspects. First, in our attention network, the word attention are context-dependent for generating accurate sentence representations and the sentence attention are question-guided for generating context representation. Second, this new attention mechanism helps our model understand when it can readily output an answer and when it can generate a supplementary question for activating the user interaction. Incorporating user's feedback does not require additional model training and this property makes our model highly self-adaptive.

3. GATED RECURRENT UNIT NETWORKS:

Gated Recurrent Unit (GRU) (Cho et al., 2014) is the basic building block of our model for IQA. GRU has been widely adopted for many NLP tasks, such as machine translation (Bahdanau et al., 2015) and language modeling (Zaremba et al., 2014) . GRU improves Long Short-term Memory (LSTM) (Hochreiter & Schmidhuber, 1997 ) by removing the cell component and making each hidden state adaptively capture the dependencies over different time scales using reset and update gates. For each time step t with input x t and previous hidden state h t-1 , we compute the updated hidden state h t = GRU (h t-1 , x t ) by,

r t = σ(U r x t + W r h t-1 + b r ), z t = σ(U z x t + W z h t-1 + b z ), h t = tanh(U h x t + W h (r t h t-1 ) + b h ), h t = z t h t-1 + (1 -z t ) h t ,

where σ is the sigmoid activation function, is an element-wise product, U r , U z ,

U h ∈ R K×D , W r , W z , W h ∈ R K×K , b r , b z , b h ∈ R K×1

, K is the hidden size and D is the input size.

4. CONTEXT-AWARE ATTENTION NETWORK:

In this section, we first illustrate the proposed model CAN ( § 4.1), including the question module ( § 4.2), the input module ( § 4.3 ) and the answer module ( § 4.4). We then describe each of these modules in detail. Finally, we elaborate the training procedure of CAN ( § 4.5).

4.1. FRAMEWORK:

Given a story represented by N input sentences (or statements), i.e., (l 1 , • • • , l N ), and a question q, our goal is to generate an answer a. Each sentence l t includes a sequence of N t words, denoted as

(w t 1 , • • • , w t Nt )

, and a question with N q words is represented as (w q 1 , • • • , w q Nq ). Let V denote the size of dictionary, including the words from each l t , q and a, and end-of-sentence (EOS) symbols.

The whole framework of our model is shown in Figure 2 , consisting of the following three key parts:

• Question Module: The question module encodes the target question into a vector representation.

• Input Module: The input module encodes a set of input sentences into a vector representation.

• Answer Module: The answer module generates an answer based on the outputs of question and input modules. Unlike traditional QA models, it has two choices, either to output an answer immediately or to interact with the user for further information. Hence, if the model lacks sufficient evidence for answer prediction based on the existing knowledge at current timestamp, an interactive mechanism is enabled. Specifically, the model generates a supplementary question, and the user needs to provide a feedback, which is utilized to estimate an answer. The input module generates a context representation based on input sentences. The answer module has a binary choice, either to generate an answer immediately or to take an interactive mechanism.

4.2. QUESTION MODULE:

Suppose a question is a sequence of N q words, we encode each word w j into a K w -dimensional vector space x q j using an embedding matrix W w ∈ R Kw×V , i.e., x q j = W w [w j ], where [w j ] is a one-hot vector associated with word w j . The sequence order within a sentence significantly affects each word's semantic meaning due to its dependence on the previous words. Thus, a GRU is employed by taking each word vector x q j as input and updating the hidden state g q j ∈ R K h ×1 as:

EQUATION

where the subscript of GRU is used to distinguish other GRUs used in the following sections. The hidden state g q j can be regarded as the annotation vector of word w j by incorporating the word order information. We also explore a variety of encoding schema, such as LSTM and RNN. However, LSTM is prone to overfitting due to large number of parameters, and RNN has a poor performance because of exploding and vanishing gradients (Bengio et al., 1994) .

In addition, each word contributes differently to the representation of a question. For example, in a question 'Where is the football?', 'where' and 'football' play a critical role in summarizing this sentence. Therefore, an attention mechanism is introduced to generate a question representation by focusing on the important words for their semantic meaning. A positive weight γ j is placed on each word to indicate the relative importance of contribution to the representation of the question. Specifically, this weight is measured as the similarity of corresponding word annotation vector g j and a word level latent vector v ∈ R K h ×1 for question which is jointly learned during the training process. The question representation u ∈ R Kc×1 is then generated by a weighted summation of the word annotation vectors and corresponding important weights, where we also use one-layer MLP to transfer it from sentence-level space into context-level space,

EQUATION

where sof tmax is defined as sof tmax

(x i ) = exp(xi) j exp(x j ) , W ch ∈ R Kc×K h , and b (q) c ∈ R Kc×1 .

4.3. INPUT MODULE:

Input module aims at generating a representation for input sentences, including a sentence encoder and a context encoder. Sentence encoder computes a sentence representation, and context encoder calculates a representation of input sentences, both of which are introduced in the following sections.

4.3.1. SENTENCE ENCODER:

For each input sentence l t , containing a sequence of N t words (w 1 , • • • , w Nt ), similar to the question module, each word w i is embedded into word space x t i ∈ R Kw×1 with the embedding matrix W w , and a recurrent neural network is used to capture the context information from the words which have already been generated in the same sentence. Let h t i ∈ R K h ×1 denote the hidden state which can be interpreted as the word annotation in the input space. A GRU retrieves each word annotation by taking word vector as input and relying on previous hidden state,

EQUATION

In Eq.( 4), each word annotation vector takes the sequence order into consideration to learn its semantic meaning based on previous information within a sentence through a recurrent neural network. A question answering system is usually given multiple input sentences which often form a story together. A single word has different meaning in the different stories. Learning a single sentence context at which a word is located is insufficient to understand the meaning of this word, in particular when the sentence is placed in a story context. In other words, only modeling a sequence of words prior to a word within a sentence may lose some important information which results in the failure of the generation of sentence representation. Hence, we take the whole context into account as well to appropriately characterize each word and well understand this sentence's meaning. Suppose s t-1 ∈ R Kc×1 is the annotation vector of previous sentence l t-1 , which will be introduced in the next section. To incorporate context information generated by previous sentences, we feed word annotation h t i and previous sentence annotation s t-1 through a two-layer MLP, where a context-aware word vector e t i ∈ R Kc×1 is obtained as follows:

EQUATION

where W ee , W es ∈ R Kc×Kc and W eh ∈ R Kc×K h are weight matrices, and b

(1)

EQUATION

∈ R Kc×1 are the bias terms. It is worth noting that s t-1 is dependent on its previous sentence. Recursively, this sentence relies on its previous one as well. Hence, our model is able to encode the previous context. In addition, the sentence representation will focus on those words which are able to address the question. Inspired by this intuition, another word level attention mechanism is introduced to attend informative words about the question for generating a sentence's representation. As the question representation is utilized to guide the word attention, a positive weight α t i associated with each word is computed as the similarity of the question vector u and the corresponding context-aware word vector e t i . Then the sentence representation y t ∈ R K h ×1 is generated by aggregating the word annotation vectors with different weights,

EQUATION

4.3.2. CONTEXT ENCODER:

Suppose a story is comprised of a sequence of sentences, i.e., (l 1 , • • • , l N ), each of which is encoded as a K h -dimensional vector y t through a sentence encoder. As input sentences have a sequence order, simply using their sentence vectors for context generation cannot effectively capture the entire context of the sequence of sentences. To address this issue, a sentence annotation vector is introduced to capture the previous context and this sentence's own meaning using a GRU. Given the sentence vector y t and the state s t-1 of previous sentence, we get annotation vector s t ∈ R Kc×1 as:

EQUATION

A GRU can learn a sentence's meaning based on previous context information. However, just relying on GRU at sentence level using simple word embedding vectors makes it difficult to learn the precise semantic meaning for each word in the story. Hence, we introduce a context-aware attention mechanism shown in Eq.( 5) to properly encode each word for the generation of sentence representation, which guarantees that each word is reasoned under the specific context.

Once the sentence annotation vectors (s 1 , • • • , s N ) are obtained as described above, a sentence level attention mechanism is enabled to emphasize those sentences that are highly relevant to the question. We can estimate the attention weight β t with the similarity of the question and the corresponding sentence. Hence, the context representation m is retrieved by summing over all sentence representations associated with corresponding attention weights, and given by:

EQUATION

Similar to bidirectional RNN, our model can be extended to use another sentence-level GRU that moves backward through time beginning from the end of the sequence.

4.4. ANSWER MODULE:

The answer module utilizes a decoder to generate an answer, where it has two output cases according to the understanding ability of both the question and the context. One is to generate the answer immediately after receiving the context and question information. Another one is to generate a supplementary question and then use the user's feedback to predict the answer. This process is taken by an interactive mechanism.

4.4.1. ANSWER GENERATION:

Given the question representation u and the context representation m, another GRU is used as the decoder to generate a sentence as the answer. To fuse u and m together, we sum these vectors rather than concatenating them to reduce the total number of parameters. Suppose xk-1 ∈ R Kw×1 is the predicted word vector in last step, GRU updates the hidden state z k ∈ R Ko×1 as follows,

EQUATION

where Ww = denotes the predicted word vector through the embedding matrix W w . Note that we require that each sentence ends with a special EOS symbol, including question mask and period symbol, which enables the model to define a distribution over sentences of all possible lengths.

Output Choices. In practice, the system is not aways able to answer question immediately based on its current knowledge due to the lack of some crucial information bridging the gap between question and context knowledge, i.e., incomplete issue. Therefore, we allow the decoder to make a binary choice, either to generate an answer immediately, or to enable an interactive mechanism. Specifically, if the model has sufficiently strong evidence for a successful answer prediction based on the well-learned context representation and question representation, the decoder will directly output the answer. Otherwise, the system generates a supplementary question for user, where an example is shown in Table 2 . At this time, this user needs to offer a feedback which is then encoded to update the sentence-level attentions for answer generation. This procedure is our interactive mechanism.

Problem:

The master bedroom is east of the garden. The guest bedroom is east of the office. Target Question: What is the bedroom east of?

System: Which bedroom, master one or guest one? (SQ) User: Master bedroom (User's Feedback) System: Garden (Predicted Answer) Table 2 : An example of interactive mechanism. "SQ" denotes supplementary question.

The sentence generated by the decoder ends with a special symbol, either a question mask or a period symbol. Hence, this special symbol is utilized to make a decision. In other words, if EOS symbol is a question mask, the generated sentence is regarded as a supplementary question and an interactive mechanism is enabled; otherwise the generated sentence is the estimated answer and the prediction task is done. In the next section, we will introduce the details of interactive mechanism.

4.4.2. INTERACTIVE MECHANISM:

The interactive process is summarized as follows: 1) The decoder generates a supplementary question; 2) The user provides a feedback; 3) The feedback is used for answer prediction for the target question. Suppose the feedback contains a sequence of words, denoted as

(w f 1 , • • • , w f N f ).

Similar to the input module, each word w f d is embedded to a vector x f d through an embedding matrix W w . Then the corresponding annotation vector g f d ∈ R K h ×1 is retrieved via a GRU by taking the embedding vector as input, and shown as follows:

EQUATION

)

Based on the annotation vectors, a representation f ∈ R K h ×1 can be obtained by a simple attention mechanism where each word is considered to contribute equally, and given by:

EQUATION

)

Our goal is to utilize the feedback representation f to generate an answer for the target question.

The provided feedback improves the ability to answer the question by distinguishing the relevance of each input sentence to the question. In other words, the similarity of specific input sentences in the provided feedback make these sentences more likely to address the question. Hence, we refine the attention weight of each sentence shown in Eq.( 9) after receiving the user's feedback, given by,

EQUATION

)

EQUATION

where

W rf ∈ R Kc×K h and b (f ) r

∈ R Kc×1 are the weight matrix and bias vector, respectively. Eq.( 13) is an one-layer neural network to transfer feedback representation to context space. After obtaining the newly learned attention weights, we update the context representation using the softattention operation shown in Eq.( 9). This updated context representation and question representation will be used as the input for decoder to generate an answer. Note that for simplifying the problem, we allow the decoder to only generate at most one supplementary question. In addition, one advantage of using the user's feedback to update the attention weights of input sentences is that we do not need to re-train the encoder again once a feedback is entering the system.

4.5. TRAINING PROCEDURE:

During training, all modules share an embedding matrix. There are three different GRUs employed for sentence encoding, context encoding and answer/supplementary question decoding. In other words, the same GRU is used to encode the question, input sentences and the user's feedback. The second one is applied to generate context representation and the third one is used as decoder. Training can be treated as a supervised classification problem to minimize the cross-entropy error of the answer sequence and the supplementary question sequence.

5. EXPERIMENTS:

In this section, we evaluate our approach with many baseline methods based on various datasets.

5.1. EXPERIMENTAL SETUP:

Datasets. In this paper, we use two types of datasets to evaluate the performance of our approach. One is traditional QA dataset, where we use Facebook bAbI English 10k dataset (Weston et al., 2015) . It contains 20 different types of tasks with emphasis on different forms of reasoning and induction. The second is the newly designed IQA dataset, where we extend bAbI to add interactive QA and denote it as ibAbI. Overall, we generate three ibAbI datasets based on task 1 (single supporting fact), task 4 (two argument relations), and task 7 (counting). Specifically, the former two datasets focus on solving ambiguous actors/objects problem, and the latter one is to ask further information that assists answer prediction. Table 3 shows three examples for our three ibAbI tasks.

In addition, we also mix IQA data and corresponding QA data together with different IQA ratios, where the IQA ratio is ranging from 0.3 to 1 (with step as 0.1) and denoted as R IQA . For example, in task 1, we randomly pick R IQA × 100 percent data from ibAbI task 1, and then randomly select the remaining data from bAbI task 1. R IQA = 1 indicates that the whole dataset only consists of IQA problems; otherwise (i.e., ranging from 0.3 to 0.9) it consists of both types of QA problems. Overall, we have three tasks for ibAbI dataset, and eight sub-datasets for each task. In the experiments, 10k examples are used as training and another 1k examples are used as testing.

IQA task 1:

IQA task 4: IQA task 7: John journeyed to the garden.

The master bedroom is east of the garden. John grabbed the bread. Daniel moved to the kitchen.

The guest bedroom is east of the office.

John grabbed the milk. The guest bedroom is west of the hallway.

John grabbed the apple. The bathroom is east of the master bedroom.

Sandra went to the bedroom. Q: Where is he? Q: What is the bedroom east of? Q: How many special objects is John holding? SQ: Who is he? SQ: Which bedroom, master one or guest one? SQ: What objects are you referring to? FB: Daniel FB: Master bedroom FB: Milk, bread A: Kitchen A: Garden A: Two Table 3 : Examples of three different tasks on the generated ibAbI datasets. "Q" indicates the target question. "SQ" is the supplementary question. "FB" refers to user's feedback. "A" is the answer.

Experiment Settings. We train our models using the Adam optimizer (Kingma & Ba, 2014). Xavier initialization is used for all parameters except for word embeddings, which utilize random uniform initialization ranging from -√ 3 to √ 3. The learning rate is set as 0.001. The grid search method is utilized to find optimal parameters, such as batch size and hidden size.

5.2. BASELINE METHODS:

To demonstrate the effectiveness of our approach CAN, we compare it with the following models:

• DMN+: Xiong et al. (2016) improve Dynamic Memory Networks (Kumar et al., 2016) by using stronger input and memory modules, where a bidirectional GRU is adopted to generate representations for statements and a neural network is used to update episodic memory multiple times. • MemN2N: This is an extension of Memory Network with weak supervision as proposed in Sainbayar et al. (2015) . Here, an external memory module is used to encode the input statements and a recurrent attention mechanism is used to read the memory for answer prediction. • EncDec: We extend the encoder-decoder framework (Cho et al., 2014) to solve QA tasks as a baseline method. Specifically, EncDec uses a GRU to encode statements and questions, the end of hidden states is used as context representation, and another GRU to generate the output.

5.3. PERFORMANCE OF QUESTION ANSWERING:

In this section, we evaluate model's ability for answer prediction based on traditional QA dataset (i.e., bAbI-10k). For this task, our model (denoted as CAN+QA) does not use the interactive mechanism. As the output answers for this dataset only contain a single word, we adopt test error rate as evaluation metric. For DMN+ and MemN2N methods, we select the best performance over bAbI dataset reported in (Xiong et al., 2016) . The results of various models across 20 tasks are reported in Table 4 . We summarize the main observations as follows:

• Our approach is better than all baseline methods in each individual task. For example, it reduces the error by 4% compared to DMN+ in task 17, and compared to MemN2N, it reduces 18.4% and 4.8% error in task 17 and 18 respectively. We can achieve a better result primarily because our approach can model the semantic logic flow for statements. Table 5 shows two examples in task 17 and 18, where MemN2N predicts incorrectly while CAN+QA can make correct predictions. In these two examples, the semantic logic determines the relationship between two objects mentioned in the question, such as chest and suitcase. In addition, Kumar et al. (2016) has shown that memory networks with multiple hops are better than the one with single hop. Our strong results illustrate that our approach has more accurate context modeling even without multiple hops. • EncDec performs the worst amongst all models over all tasks. EncDec concatenates the statements and questions as a single input, resulting in the difficulty of training the GRU. For example, EncDec is not good on task 2 and 3 because these two tasks have longer inputs than other tasks. • The results of DMN+ and MemN2N are much better than EncDec. It is not surprising that they can outperform EncDec, because they are specifically designed for question answering and do not suffer from the problem mentioned above by treating input sentences separately. • All models perform poorly on task 16. Xiong et al. (2016) points out that MemN2N with a simple update for memory could achieve a near perfect error rate of 0.4 while a more complex method will lead to a much worse result. This shows that a sophisticated modeling method makes it

• Our method significantly outperforms all baseline methods. Specifically, we can achieve 0% test error rate in task 1 and task 4 with R IQA = 1.0 ; while the best result of baseline methods can only get 40.5% test error rate. CAN+IQA benefits from more accurate context modeling, which allows it to correctly understand when to output an answer or require additional information. For those QA problems with incomplete information, it is necessary to gather the additional information from users. Randomly guessing may harm model's performance, which makes conventional QA models difficult to converge. But our approach uses an interactive procedure to obtain user's feedback and allows the model to provide the correct answer. • For the baseline methods, DMN+ and MemN2N perform similarly and do better than EncDec.

Their similar performance (which are worse than our approach) is due to the limitation that they could not learn the accurate meaning of statements and questions with limited resource and then have trouble training the models. But they are superior over EncDec as they treat each input sentence separately instead of modeling very long inputs.

In addition, we also quantitatively evaluate the quality of supplementary question generated by our approach where the details can be found in Appendix A.

Methods RIQA RIQA RIQA RIQA RIQA RIQA RIQA RIQA = 1.0 = 0.9 = 0.8 = 0.7 = 0.6 = 0.5 = 0.4 = 0. 

5.5. QUALITATIVE ANALYSIS OF INTERACTIVE MECHANISM:

In this section, we qualitatively show the attention weights over input sentences generated by our model on both QA and IQA data. We train our model (CAN+IQA) on task 1 of ibAbI dataset with Q IQA = 0.9, and randomly select one IQA example from the testing data. Then we do the prediction on this IQA problem. In addition, we change this instance to a QA problem by replacing the question "Where is she?" with "Where is Sandra?", and then do the prediction as well. The prediction results on both QA and IQA problems are shown in Table 8 . From the results, we observe the following:

1) The attention that uses user's feedback focuses on the key relevant sentence while the attention without feedback only focuses on an unrelated sentence. This happens because utilizing user's feedback allows the model to understand a question better and locate the relevant input sentences. This illustrates the effectiveness of an interactive mechanism on addressing questions that require additional information.

2) The attention on both two problems can finally focus on the relevant sentences, showing the usefulness of our model for solving different types of QA problems. otherwise the corresponding answer is viewed as output. Similar to our approach, the EOS symbol is used to determine whether the generated sentence is question or not, where the question ends with question mark and the answer ends with period symbol.

6. CONCLUSION:

To test model's ability to generate supplementary question, we define some following metrics. Suppose the number of problems is N , and the number of problems having supplementary question is N s . Then N a = N -N s is the number of remaining problems. Let SQueAcc = Ns Ns is the fraction of IQA problems which can be correctly estimated, and AnsAcc = Na Na is the fraction of remaining problems which can be correctly estimated as QA problem. So SQueAnsAcc = Ns+ Na N is the overall accuracy. In addition, the widely used BLEU (Papineni et al., 2002) and METEROR (Banerjee & Lavie, 2005) are also adopted to evaluate the quality of generated supplementary question. The results of our method and baseline method are presented in Table 9 .

From the results, we can observe that 1) Two models can almost correctly determine whether it is time to output a question or not; 2) Two models are able to generate the correct supplementary questions whose contents exactly match with the ground truth. There is no surprise that EncDec * also performs well in generating question, because it is specifically designed for only outputting questions. Thus, if given enough training data, EncDec * could predict good questions. However, the limitation is that it cannot predict a supplementary question and an answer at the same time. Different from EncDec * , our approach can accurately know when to output an answer or when to generate a supplementary question. 

:



Figures and Tables:

Figure 1: An example of QA problem using CAN.

Figure 2: The illustration of the proposed model, consisting of a question module, an input module and an answer module. The question module maps the question sentence into a sentence level space.The input module generates a context representation based on input sentences. The answer module has a binary choice, either to generate an answer immediately or to take an interactive mechanism.

Performance comparison of various models in terms of test error rate (%) based on interactive question answering datasets with different IQA ratios.

Table content: <table><tr><td>3</td></tr></table>

In this paper, we present a self-adaptive model, CAN, which learns more accurate representations for statements and questions. More importantly, our model is aware what it knows and what it does not know within the context of the story, and takes an interactive mechanism to answer a question. Hence, our model takes an important step towards having a natural and intelligent conversation A SUPPLEMENTARY QUESTION ANALYSIS We quantitatively evaluate the quality of supplementary question generated by our model based on IQA dataset. All baseline methods are designed to only predict an answer, none of them can generate a question. Thus we design another baseline method to generate supplementary question based on EncDec, and denote it as EncDec * . Specifically, in training procedure, EncDec * takes statements and questions as input. If the supplementary question is available, it is used as output;

Table content: <table/>

Performance comparison of the generated supplementary question quality with R IQA as 0.8. Both two methods achieve 100% under all metrics in all tasks with other different R IQA values.

Table content: <table/>

Peer review (iclr2017):
This work describes 1: a two stage encoding of stories in bAbI like setups, where a GRU is used to encode a sentence, word by word, conditioned on a sentence level GRU, and the sentence level GRU keeps track of a sentence level encoding. Each is used 2: modifying the bAbI tasks so it is necessary to ask a question to correctly solve the problem I am not convinced by the papers results: 1: The new architecture does not do significantly better than DMN+, and in my view, is similar to DMN+. What problem with DMN+ does your architecture solve? 2: There are now several papers doing the second thing, for example *Dialog-based Language Learning* by Weston and *Learning End-to-End Goal-Oriented Dialog* by Bordes and Weston, and I think doing it more carefully and in more compelling ways. In the current work, the correct answer to the question seems given independent of the what the agent asks, so any model that can output *unknown* and then input the extra response has an advantage. Essentially all of the architectures that are used to solve bAbI can be modified to do this... Indeed, the enc-dec* accuracies in appendix A show that this sort of module can be appended to any other model. All of the standard models can be trained to output questions as a sequence of words. Furthermore, I suspect you could generate the questions in the authors* setting just by enumerating all the questions that occur in training, and taking a softmax over them, instead of generating word-by-word.