Venue: NAACL 2018

# Self-Training for Jointly Learning to Ask and Answer Questions

Mrinmaya Sachan &Eric P. Xing

School of Computer Science

Carnegie Mellon University

{mrinmays, epxing}@cs.cmu.edu

###### Abstract

Building curious machines that can answer as well as ask questions is an important challenge for AI. The two tasks of question answering and question generation are usually tackled separately in the NLP literature. At the same time, both require significant amounts of supervised data which is hard to obtain in many domains. To alleviate these issues, we propose a self-training method for jointly learning to ask as well as answer questions, leveraging unlabeled text along with labeled question answer pairs for learning. We evaluate our approach on four benchmark datasets: _SQUAD_, _MS MARCO_, _WikiQA_ and _TrecQA_, and show significant improvements over a number of established baselines on both question answering and question generation tasks. We also achieved new state-of-the-art results on two competitive answer sentence selection tasks: _WikiQA_ and _TrecQA_.

## 1 Introduction

Question Answering (QA) is a well-studied problem in NLP which focuses on answering questions using some structured or unstructured sources of knowledge. Alongside question answering, there has also been some work on generating questions (QG) Heilman (2011); Du et al. (2017); Tang et al. (2017) which focuses on generating questions based on given sources of knowledge.

QA and QG are closely related1 tasks. However, NLP literature views the two as entirely separate tasks. In this paper, we explore this relationship between the two tasks by jointly learning to generate as well as answer questions. An improved ability to generate as well as answer questions will help us build _curious_ machines that can interact with humans in a better manner. Joint modeling of QA and QG is useful as the two can be used in conjunction to generate novel questions from free text and then answers for the generated questions. We use this idea to perform self-training Nigam and Ghani (2000) and leverage free text to augment the training of QA and QG models.

Footnote 1: We can think of QA and QG as inverse of each other.

QA and QG models are typically trained on question answer pairs which are expensive to obtain in many domains. However, it is cheaper to obtain large quantities of free text. Our self-training procedure leverages unlabeled text to boost the quality of our QA and QG models. This is achieved by a careful data augmentation procedure which uses pre-trained QA and QG models to generate additional labeled question answer pairs. This additional data is then used to retrain our QA and QG models and the procedure is repeated.

This addition of synthetic labeled data needs to be performed carefully. During self-training, typically the most _confident_ samples are added to the training set Zhu (2005) in each iteration. We use the performance of our QA and QG models as a proxy for estimating the _confidence_ value of the questions. We describe a suite of heuristics inspired from curriculum learning Bengio et al. (2009) to select the questions to be generated and added to the training set at each epoch. Curriculum learning is inspired from the incremental nature of human learning and orders training samples on the _easiness_ scale so that _easy_ samples can be introduced to the learning algorithm first and _harder_ samples can be introduced successively. We show that introducing questions in increasing order of _hardness_ leads to improvements over a baseline that introduces questions randomly.

We use a seq2seq model with soft attention Sutskever et al. (2014); Bahdanau et al. (2014) for QG and a neural model inspired from _Attentive Reader_Hermann et al. (2015); Chen et al. (2016) for QA. However, these can be any QA

and QG models. We evaluate our approach on four datasets: _SQUAD_, _MS MARCO_, _WikiQA_ and _TrecQA_. We use a corpus of English Wikipedia as unlabeled text. Our experiments show that the self-training approach leads to significant improvements over a number of established approaches in QA and QG on these benchmarks. On the two answer sentence selection QA tasks: (_WiKiQA_ and _TrecQA_), we obtain state-of-the-art.

## 2 Problem Setup

In this work, we focus on the task of machine comprehension where the goal is to answer a question \({\bf q}\) based on a passage \({\bf p}\). We model this as an answer sentence selection task i.e., given the set of sentences in the passage \({\bf p}\), the task is to select the sentence \({\bf s}\in{\bf p}\) that contains the answer \(a\). Treating QA as an answer sentence selection task is quite common in literature (e.g. see Yu et al., 2014). We model QG as the task of transforming a sentence in the passage into a question. Previous work in QG Heilman and Smith (2009) transforms text sentences into questions via some set of manually engineered rules. However, we take an end-to-end neural approach.

Let \(\mathcal{D}^{0}\) be a labeled dataset of (passage, question, answer) triples where the answer is given by selecting a sentence in the passage. We also assume access to unlabeled text \(\mathcal{T}\) which will be used to augment the training of the two models.

## 3 The Question Answering Model

Since we model QA as the task of selecting an answer sentence from the passage, we treat each sentence in the corresponding passage as a candidate answer for every input question.

We employ a neural network model inspired from the _Attentive Reader_ framework proposed in Hermann et al. (2015); Chen et al. (2016). We map all words in the vocabulary to corresponding \(d\) dimensional vector representations via an embedding matrix \(E\in\mathbb{R}^{d\times V}\). Thus, the input passage \({\bf p}\) can be denoted by the word sequence \(\{p_{1},p_{2},\ldots p_{|{\bf p}|}\}\) and the question \({\bf q}\) can similarly be denoted by the word sequence \(\{q_{1},q_{2},\ldots q_{|{\bf q}|}\}\) where each token \(p_{i}\in\mathbb{R}^{d}\) and \(q_{i}\in\mathbb{R}^{d}\).

We use a bi-directional LSTM Graves et al. (2005) with dropout regularization as in Zaremba et al. (2014) to encode contextual embeddings of each word in the passage:

The final contextual embeddings \({\mathbf{h}}_{t}\) are given by concatenation of the forward and backward pass embeddings: \({\mathbf{h}}_{t}=[\vec{\mathbf{h}}_{t};\vec{\mathbf{h}}_{t}]\). Similarly, we use another bi-directional LSTM and encode contextual embeddings of each word in the question.

Then, we use attention mechanism Bahdanau et al. (2014) to compute the alignment distribution \(a\) based on the relevance among passage words and the question: \(a_{i}=\textit{softmax}\left({\mathbf{q}}^{T}{\mathbf{W}}{\mathbf{h}}_{i}\right)\). The output vector \(o\) is a weighted combination of all contextual embeddings: \({\mathbf{o}}=\sum\limits_{i}a_{i}{\mathbf{h}}_{i}\). Finally, the correct answer \(a^{*}\) among the set of candidate answers \(\mathcal{A}\) is given by: \(a^{*}=\operatorname*{arg\,max}_{a\in\mathcal{A}}{\mathbf{w}}^{T}{\mathbf{o}}\).

We learn the model by maximizing the log-likelihood of correct answers. Given the training set \(\{{\mathbf{p}}^{(i)},{\mathbf{q}}^{(i)},{\mathbf{a}}^{(i)}\}_{i=1}^{N}\), the log-likelihood is:

\[\mathcal{L}_{QA}=\sum\limits_{i=1}^{N}\textit{logP}\left({\mathbf{a}}^{(i)}|{ \mathbf{p}}^{(i)},{\mathbf{p}}^{(i)};\theta\right)\]

Here, \(\theta\) represents all the model parameters to be estimated.

## 4 The Question Generation Model

We use a seq2seq model Sutskever et al. (2014) with soft attention Bahdanau et al. (2014) as our QG model. The model transduces an input sequence \({\bf x}\) to an input sequence \({\bf y}\). Here, the input sequence is a sentence in the passage and the output sequence is a generated question. Let \({\bf x}=\{x_{1},x_{2},\ldots,x_{|{\bf x}|}\}\), \({\bf y}=\{y_{1},y_{2},\ldots,y_{|{\bf y}|}\}\) and \(\mathcal{Y}\) be the space of all possible output questions. Thus, we can represent the QG task as finding \(\hat{{\bf y}}\in\mathcal{Y}\) such that: \(\hat{{\bf y}}=\operatorname*{arg\,max}_{{\bf y}}P({\bf y}|{\bf x})\).

Here, \(P({\bf y}|{\bf x})\) is the conditional probability of a question sequence \({\bf y}\) given input sequence \({\bf x}\).

**Decoder:** Following Sutskever et al. (2014), the conditional factorizes over token level predictions:

\[P({\bf y}|{\bf x})=\prod\limits_{t=1}^{|{\bf y}|}P(y_{t}|{\bf y}_{<t},{\bf x})\]

Here, \({\bf y}_{<t}\) represents the subsequence of words generated prior to the time step \(t\). For the decoder, we again follow Sutskever et al. (2014): 

Here, \({\bf h}_{t}^{(d)}\) is the decoder RNN state at time step \(t\), and \({\bf c}_{t}\) is the attention based encoding of the input sequence \({\bf x}\) at decoding time step \(t\) (described later). Also \({\bf W}\) and \({\bf W}_{t}\) are model parameters to be learned. We use an LSTM with dropout Zaremba et al. (2014) as the decoder RNN. The LSTM generates the new decoder state \({\bf h}_{t}^{(d)}\) given the representation of previously generated word \(y_{t1}\) obtained using a look-up dictionary, and the previous decoder state \({\bf h}_{t-1}^{(d)}\).

**Encoder:** We use a bi-directional LSTM Graves et al. (2005) with attention mechanism as our sentence encoder. We use two LSTM's: one that makes a forward pass in the sequence and another that makes a backward pass as in the QA model described earlier. We use dropout regularization for LSTMs as in Zaremba et al. (2014) in our implementation. The final context dependent token representation \({\bf h}_{t}^{(e)}\) is the concatenation of the forward and backward pass token representations: \({\bf h}_{t}^{(e)}=[\vec{\bf h}_{t}^{(e)};\vec{\bf h}_{t}^{(e)}]\). To obtain the final context dependent token representation \({\bf c}_{j}\) at the decoding time step \(j\), we take a weighted average over token representations: \({\bf c}_{j}^{(d)}=\sum\limits_{i=1}^{|{\bf X}|}a_{ij}{\bf h}_{i}^{(e)}\). Following Bahdanau et al. (2014), the attention weights \(a_{ij}\) are calculated by bilinear scoring followed by softmax normalization:

\[{\bf a}_{ij}=\frac{\exp\left({{\bf h}_{j}^{(e)}}^{T}{\bf W}\:{\bf h}_{i}^{(d) }\right)}{\sum\limits_{i^{\prime}}\exp\left({{\bf h}_{j}^{(e)}}^{T}{\bf W}\:{ \bf h}_{i^{\prime}}^{(d)}\right)}\]

Learning and Inference:We train the encoder decoder framework by maximizing data log-likelihood on a large training set with respect to all the model parameters \(\theta\). Let \(\{{\bf x}^{(i)},{\bf y}^{(i)}\}_{i=1}^{N}\) be the training set. The log-likelihood can be written as:

\[\mathcal{L}_{QG} = \sum\limits_{i=1}^{N}\textit{logP}\left({{\bf y}^{(i)}|{\bf x}^{ (i)};\theta}\right)\] \[= \sum\limits_{i=1}^{N}\sum\limits_{j=1}^{|{\bf y}^{(i)}|}\textit {logP}\left({y}_{j}^{(i)}|{\bf x}^{(i)},{\bf y}_{<j}^{(i)};\theta\right)\]

We use beam search for inference. As in previous works, we introduce a \(<\)UNK\(>\) token to model rare words during decoding. These \(<\)UNK\(>\) tokens are finally replaced by the token in the input sentence with the highest attention score.

## 5 Self-training Framework for Joint Training of QA and QG models

In our self-training framework, we are given unlabeled text in addition to the labeled passages, question and answer pairs. Self-training Yarowsky (1995); Riloff et al. (2003), also known as self-teaching, is one of the earliest techniques for using unlabeled data along with labeled data to improve learning. During self-training, the learner keeps on labeling unlabeled examples and retraining itself on an enlarged labeled training set. We extend self-training to jointly learn two models (namely, QA and QG) iteratively. The QA and QG models are first trained on the labeled corpus. Then, the QG model is used to create more questions from the unlabeled text corpus and the QA model is used to answer these newly created questions. These new questions (carefully selected by an oracle - details later) and the original labelled data is then used to (stochastically) update these two models. This procedure can be repeated as long as both the two models continue to improve.

```
1\(\theta_{\textbf{QA}}^{(\textbf{Q})}\leftarrow\text{Train initial QA model.}\)
2\(\theta_{\textbf{QA}}^{(\textbf{Q})}\leftarrow\text{Train initial QG model.}\)
3Init: i = 0 whileperformance on dev set risesdo
4 CCq\({}_{\textbf{I}}\leftarrow\) Set of candidate questions generated using our QG model \(\theta_{\textbf{QA}}^{(i)}\) from the unlabeled text \(\mathcal{T}\) which are not in \(\mathcal{D}\).
5 Q\({}_{\textbf{I}}\gets k\times m^{i}\) questions drawn from CCq\({}_{\textbf{I}}\) using our question selector oracle \(\mathcal{QS}\).
6 A\({}_{\textbf{I}}\leftarrow\) Set of answers to questions Q\({}_{\textbf{I}}\) obtained using our QA model \(\theta_{\textbf{QA}}^{(i)}\).
7 Let D\({}^{*}\)be the set of chosen questions Q\({}_{\textbf{I}}\) and answers A\({}_{\textbf{I}}\).
8 Subsample S\({}_{\textbf{I}}\subset{}^{\mathcal{D}}\) of size \(k_{1}\) and S\({}_{2}\subset{}^{\mathcal{D}}\) of size \(k_{2}\). Let \(\mathcal{S}=\mathcal{S}_{1}\cup\mathcal{S}_{2}\)
9\(\theta_{\textbf{I}}^{(\textbf{I}+1)}\leftarrow\) Update QA model on \(\mathcal{S}\).
10\(\theta_{\textbf{I}}^{(\textbf{I}+1)}\leftarrow\) Update QG model on \(\mathcal{S}\).
11 i++
12 end while
```

**Algorithm 1**Self-training QA and QG.

Algorithm 1 describes the procedure in detail. In each succesive iteration, we allow the addition of more questions than that introduced in the previous iteration by a multiplicative factor. This theme adds fewer questions initially when the QA and QG models are weak and more questions thereafter when the two models have (hopefully) improved. We found that this scheme works better in practice than adding a fixed number of questions in each iteration. The two models are 

updated on a subsample of the newly generated datapoints and original unlabelled data.

Self-training has been seldom used in NLP. Most prominently, it has been used for WSD (Yarowsky, 1995), noun learning (Riloff et al., 2003) and AMR parsing and generation (Konstas et al., 2017). However, it has not been explored in this way for QA and QG.

### The Question Selection Oracle

A key challenge in self-training is selecting which unlabeled data sample to label (iwhich generated questions to add to the training set). The self-training process may erroneously generate some bad or incorrect questions which can sidetrack the learning process. Thus, we implement a question selection oracle which determines which questions to add among the potentially very large set of questions generated by the QG model in each iteration.

Traditional wisdom in self-training (Yarowsky, 1995; Riloff et al., 2003) advises selecting a subset of questions on which the models have the highest _confidence_. We experiment with this idea, proposing multiple self-training oracles which introduce questions in the order of how _confident_ the QA and QG models are on the new potential question:

* **QG:** The QG oracle introduces the question in the order of how confident the QG model is on generating the question. This is calculated by a number of heuristics (described later).
* **QA:** The QA oracle introduces the question in the order of how confident the QA model is on answering the question. This too is calculated by some heuristics (described later).
* **QA+QG:** The QA+QG oracle introduces a question when both QA and QG models are confident about the question. The oracle computes the minimum confidence of the QA and QG models for a question and introduces questions which have the the highest minimum confidence score.

Our question selection heurisitcs are based on the ideas of _curriculum learning_ and _diversity_:

1. _Curriculum learning_(Bengio et al., 2009; Sachan and Xing, 2016) requires ordering questions on the easiness scale, so that easy questions can be introduced to the learning algorithm first and harder questions can be introduced successively. The main challenge in learning the curriculum is that it requires the identification of easy and hard questions. In our setting, such a ranking of easy and hard questions is difficult to obtain. A human judgement of 'easiness' of a question might not correlate with what is easy for our algorithms in its feature and hypothesis space. We explore various heuristics that define a measure of easiness and learn the ordering by selecting questions using this measure.
2. A number of cognitive scientists (Cantor, 1946) argue that alongside curriculum learning, it is important to introduce diverse (even if sometimes hard) samples. Inspired by this, we introduce a measure of _diversity_ and show that we can achieve further improvements by coupling the curriculum learning heuristics with a measure for diversity.

**Curriculum Learning:** Studies in cognitive science (Skinner, 1958; Peterson, 2004; Krueger and Dayan, 2009) have shown that humans learn much better when the training examples are not randomly presented but organized in increasing order of difficulty. In the machine learning community, this idea was introduced with the nomenclature of _curriculum learning_(Bengio et al., 2009), where a curriculum is designed by ranking samples based on manually curated difficulty measures. A manifestation of this idea is self-paced learning (SPL) (Kumar et al., 2010; Jiang et al., 2014, 2015) which selects samples based on the local loss term of the sample. We extend this idea and explore the following heuristics for our various oracles:

**1) Greedy Optimal (GO):** The simplest greedy heuristic is to pick a question \(q\) which has the minimum expected effect on the QA and QG models. The expected effect on adding \(q\) can be written as:

\[\sum_{a\in\mathcal{A}}p(a^{*}=a)\mathbb{E}[\mathcal{L}_{QA/QG}]\]

Here, \(\mathcal{L}_{QA/QG}\) is \(\mathcal{L}_{QA}\), \(\mathcal{L}_{QG}\) or \(min\left(\mathcal{L}_{QA},\mathcal{L}_{QG}\right)\) depending on which oracle we are using. \(p(a^{*}=a)\) can be estimated by computing the scores of each of the answer candidates for \(q\) and normalizing them. \(\mathbb{E}[\mathcal{L}_{QA/QG}]\) can be estimated by retraining the model(s) after adding this question.

**2) Change in Objective (CiO):** Choose question \(q\) that causes the smallest increase in \(\mathcal{L}_{QA/QG}\). If 

there are multiple questions with the smallest increase in objective, pick one of them randomly.

**3) Mini-max (M\({}^{2}\)):** Choose question \(q\) that minimizes the expected risk when including the question with the answer candidate \(a\) that yields the maximum error.

\[\hat{q}=\operatorname*{arg\,min}_{q}\max_{a\in\mathcal{A}}\mathcal{L}_{QA/QG}\]

**4) Expected Change in Objective (ECiO):** In this greedy heuristic, we pick a question \(q\) which has the minimum expected effect on the model. The expected effect can be written as:

\[\sum_{a}p(a^{*}=a)\times\mathbb{E}\left[L_{QA/QG}\right]\]

Here, \(p(a^{*}=a)\) can again be achieved by computing the scores of each of the answer candidates for \(q\) and normalizing them and \(\mathbb{E}\left[L_{QA/QG}\right]\) can be estimated by evaluating the model.

* ECoIo):** We pick a question \(q\) which has the minimum value of the difference between the change in objective and the expected change in objective described above. Intuitively, the difference represents how much the model is surprised to see this new question.

_Time Complexity:_ GO and CiO require updating the model, M\({}^{2}\) and EciO require performing inference on candiate questions, and CiO - EciO requires retraining as well as inference. Thus, M\({}^{2}\) and EciO are computationally most efficient.

**Ensembling:** We introduce an ensembling strategy that combines the heuristics into an ensemble. We tried two ensembling strategies. The first strategy computes the average score over all the heuristics for all potential (top-K in beam) questions and picks questions with the highest average. The second strategy uses minimum instead of the average. Minimum works better than average in practice and we use it in our experiments. The use of minimum is inspired by agreement-based learning Liang et al. (2008), a well-known extension of self-training which uses multiple views of the data (described using different feature sets or models) and adds new unlabeled samples to the training set when multiple models agree on the label.

**Diversity:** The strategy of introducing easy questions first and then gradually introducing harder questions is intuitive as it allows the learner to improve gradually. Yet, it has one key deficiency. With curriculum learning, by focusing on easy questions first, our learning algorithm is usually not exposed to a diverse set of questions. This is particularly a problem for deep-learning approaches that learn representations during the process of learning. Hence, when a harder question arrives, it can be difficult for the learner to adjust to the new question as the current representation may not be appropriate for the new level of question difficulty. We tackle this by introducing an explore and exploit (**E&E**) strategy. _E&E_ ensures that while we still select easy questions first, we also want to make our selection as diverse as possible. We define a measure for diversity as the angle between the question vectors: \(\angle\mathbf{q}_{i},\mathbf{q}_{j}=Cosine^{-1}\left(\frac{|\mathbf{q}_{i} \mathbf{q}_{j}|}{||\mathbf{q}_{i}||||\mathbf{q}_{j}||}\right)\). _E&E_ picks the question which optimizes a convex combination (tuned on the dev set) of the curriculum learning objective and sum of angles between the candidate questions and the questions in the training set.

## 6 Experiments

**Implementation Details:** We perform the same preprocessing on all the text. We lower-case all the text. We use NLTK for word tokenization. For training our neural networks, we only keep the most frequent 50k words (including entity and placeholder markers), and map all other words to a special <UNK> token. We choose word embedding size d = 100, and use the 100-dimensional pretrained GloVe word embeddings Pennington et al. (2014) for initialization. We set \(k\), \(m\), \(k_{1}\) and \(k_{2}\) (hyperparameters for self-training) by grid search on a held-out development set.

**Datasets:** We report our results on four datasets: _SQUAD_Rajpurkar et al. (2016), _MS MARCO_Nguyen et al. (2016), _WikiQA_Yang et al. (2015) and _TrecQA_Wang et al. (2007). _SQUAD_ is a cloze-style reading comprehension dataset with questions posed by crowd workers on a set of Wikipedia articles, where the answer to each question is a segment of text from the corresponding reading passage. _MS MARCO_ contains questions which are real anonymized queries issued through _Bing_ or _Cortana_ and the documents are related web pages which may or help answer the question. _WikiQA_ is also a datset of queries taken from Bing query logs. Based on user clicks, each query is associated with a Wikipedia page. The summary paragraph of the page is taken as candidate answer sentences, with labels on whether the sentence is a correct answer to the question provided by crowd 

workers. Finally, _TrecQA_ is a QA answer sentence selection dataset from the TREC QA track.

While _WikiQA_ and _TrecQA_ are directly answer sentence selection tasks, the other two are not. Hence, we treat the _SQUAD_ and _MS MARCO_ tasks as the answer sentence selection task assuming a one to one correspondence between answer sentences and annotated correct answer spans. Note that only a very small proportion of answers (\(<0.2\%\) in training set) span two or more sentences. Since _SQUAD_ and _MS MARCO_ have a hidden test set, we only use the training and development sets for our evaluation purposes and we further split the provided development set into a dev and test set. This is also the data analysis setting used in previous works Du et al. (2017); Tang et al. (2017). In fact, we use the same setting as in Tang et al. (2017) for comparison. The statistics of the four datasets and the respective train, dev and test splits are given in Table 1. For _WikiQA_ and _TrecQA_ datasets, we use the standard data splits. We use a large randomly subsampled corpus of English Wikipedia and use the first paragraph of each document as unlabeled text for self-training.

**Evaluation Metrics:** Following Tang et al. (2017), we evaluate our QA system with three standard evaluation metrics: _Mean Average Precision_ (MAP), _Mean Reciprocal Rank_ (MRR) and _Precision@1_ (P@1). For QG, we follow Du et al. (2017) and use automatic evaluation metrics from MT and summarization: _BLEU-4_Papineni et al. (2002), _METEOR_Denkowski and Lavie (2014) and \({Rouge}_{L}\)Lin (2004) to measure the overlap between generated and ground truth questions.

**Baselines:** For _SQUAD_ and _MS MARCO_ datasets, we use four QA baselines that have been used in previous works Tang et al. (2017). The first two baselines, _WordCut_ and _NormWordCnt_, have been taken from Yang et al. (2015) and Yin et al. (2015), and are based on simple word overlap which have been shown to be strong baselines. These compute word co-occurrence between a question sentence and the candidate answer sentence. While _WordCut_ uses unnormalized word co-occurrence, _NormWordCnt_ uses normalized word co-occurrence. The third and fourth baselines are _CDSSM_Shen et al. (2014) and _ABCNN_Yin et al. (2015) which use a neural network approach to model semantic relatedness of sentence pairs. For the _WikiQA_ and _TrecQA_ dataset, we report results of various existing state-of-the-art approaches on the two datasets2.

Footnote 2: [https://aclweb.org/aclwiki/Question_Answering_](https://aclweb.org/aclwiki/Question_Answering_)(State_of_the_art)

For QG, we compare our model against the following four baselines used in previous work Du et al. (2017). The first baseline is a simple _IR_ baselines taken from Rush et al. (2015) which generates questions by memorizing them from the training set and uses edit distance Levenshtein (1966) to calculate distance between a question and the input sentence. The second baseline is a MT system - _MOSES_Koehn et al. (2007) which models question generation as a translation task where raw sentences are treated as source texts and questions are treated as target texts. The third baseline, _DirectIn_, uses the longest sub-sentence of the input sentence (using a set of simple sentence splitters) as the question. The fourth baseline, _H&S_ is a rule-based overgenerate-and-rank system proposed by Heilman and Smith (2010).

**The Question Selection Oracle:** The first question we wish to answer is: _Is careful question selection even necessary?_ To answer this, we plot MAP scores for our best QA model (QA+QG, Ensemble+E&E) when we do not have a curriculum learning based oracle (i.e. an oracle which picks questions to be added to the dataset randomly) in Figure 1 as a function of epochs. We observe that

Figure 1: MAP for our best self-trained QA model (with 10,000 Wikipedia paragraphs) without any curriculum learning (i.e. candidate questions are added randomly) vs epochs.

 

the MAP score degrades instead of improving with time. This supports our claim that we need to augment the training set by a more careful procedure.

We also plot MAP scores for our best QA model (Ensemble+E&E) when we use various question selection oracles as a function of the amount of unlabeled data in Figure 2. We can observe that when we do not have a curriculum learning based oracle, the MAP score degrades by having more and more unlabeled data. We also observe that the QA+QG oracle performs better than QA and QG which confirms that the best oracle is one that selects questions in increasing degree of hardness in terms of both question answering and question generation. This holds for all the experimental settings. Thus we only show results for the QA+QG strategies in our future experiments.

**Evaluating Question Answering:** First, we evaluate our models on the question answering task. _Ensemble+E&E(K)_ is the variant where we perform self-training using \(K\) Wikipedia paragraphs. Hence, _Ensemble+E&E(0)_ is the variant of our model without any self-training. We vary \(K\) to see the impact of the size of unlabeled Wikipedia paragraphs on the self-training model.

Table 2 shows the results of the QA evaluations on the _SQUAD_ and _MS MARCO_ datasets. We can observe that our QA model has competetive or better performance over all the baselines on both datasets in terms of all the three evaluation metrics. When we incorporate ensembling or diversity, we see a further improvement in the result.

Tables 3 and 4 show results of QA evaluations on the _WikiQA_ and _TrecQA_ datasets, respectively. We can again observe that our QA model is competitive to all the baselines. When we introduce ensembling and diversity while jointly learning the QA and QG models, we see incremental improvements. In both these answer sentence selection tasks, our approach achieves new state-of-the-art.

\begin{table}
\begin{tabular}{l|c c}  & **MAP** & **MRR** \\ \hline
**He and Liu (2016)** & 0.758 & 0.822 \\ He et al. (2015) & 0.762 & 0.830 \\ Tay et al. (2017) & 0.770 & 0.825 \\ Rao et al. (2016) & 0.780 & 0.834 \\ \hline
**Ensemble+E&E/E0(0)** & 0.742 & 0.813 \\
**Ensemble+E&E/E(100)** & 0.776 & 0.831 \\ Ensemble+E&E/E(100)** & 0.783 & 0.836 \\ \hline M* & 0.759 & 0.816 \\ ECOO & 0.762 & 0.822 \\ GO & 0.759 & 0.823 \\
**GO** & 0.762 & 0.826 \\ CO-ECIO & 0.767 & 0.830 \\
**ensemble** & 0.789 & 0.843 \\ \hline
**Ensemble+E&AE/(10000)** & **0.798** & **0.854** \\ \end{tabular}
\end{table}
Table 4: Performance of our models and the QA baselines on the _TreeQA_ dataset. Shaded part of the table shows the effect of various question selection heuristics when 10000 Wikipedia paragraphs are used as unlabeled data. Our model achieves the state-of-the-art.

Figure 2: MAP for the best models for the three oracles: QA, QG and QA+QG. Also on the same plot, MAP when we have no curriculum learning.

 



**Evaluating Question Generation:** Table 5 shows the results for QG on the four datasets on each of the three evaluation metrics on all the four datasets. We can observe that the QG model described in our paper performs much better than all the baselines. We again observe that self-training while jointly training the QA and QG models leads to even better performance. These results show that self-training and leveraging the relationship between QA and QG is very useful for boosting the performance of the QA and QG models, while additionally only using cheap unlabeled data.

**Human Evaluations:** We asked two people not involved with this research to evaluate 1000 (randomly selected) questions generated by our best QG model and our best performing baseline Du et al. (2017) on SQUAD for fluency and correctness on a scale of 1 to 5. The raters were also shown the passage sentence used to generate the question. The raters were blind to which system produced which question. The Pearson correlation between the raters' judgments was \(r=0.89\) for fluency and \(r=0.78\) for correctness. In our analyses, we used the averages of the two raters' judgments. The evaluation showed that our system generates questions that are more fluent and correct than those by the baseline. The mean fluency rating of our best system was 4.15 compared to 3.35 for the baseline, a difference which is statistically significant (t-test, \(p<0.001\)).

**Evaluating the Question Selection Oracle:** As discussed earlier, the choice of which subset of questions to add to our labeled dataset while self-training is important. To evaluate the various heuristics proposed in our paper, we show the effect of the question selection oracle on the final QA and QG performance in Tables 2, 3, 4 and 5. These comparisions are shown in the shaded grey portions of the tables for self-training with 10,000 Wikipedia paragraphs as unlabeled data.

We can observe that all the proposed heuristics (and ensembling and diversity strategies) lead to improvements in the final performance of both QA and QG. The heuristics arranged in increasing order of performance are: \(M^{2}\), _ECiO_, _GO_, _CiO_ and _CiO-EciO_. While the choice of which heuristic to pick seems to make a lesser impact on the final performance, we do see a much more significant performance gain by _ensembling_ to combine the various heuristics and using _E&E_ to incorporate diversity. The incorporatation of diversity is important because the neural network models which learnt latent representations of data usually find it hard to adjust to new level of difficulty of questions as the current representation may not be appropriate for the new level of difficulty.

**Low data scenarios:** A key advantage of our self-training approach is that it can leverage unlabeled text, and thus requires less labeled data. To test this, we plot MAP for our best self-training model and various QA baselines as we vary the proportion of labeled training set in Figure 3. However, we keep the unlabeled text fixed (10K Wikipedia paragraphs). We observe that all the baselines significantly drop in performance as we reduce the proportion of labeled training set. However, the drop happens at a much slower rate for our self-trained model. Thus, we can conclude that our approach requires less labeled data as compared to the baselines.

**Does more unlabeled text always help?:** Another important question is: _Does more unlabeled

\begin{table}
\begin{tabular}{l|c c c c c c c c c c c c}  & \multicolumn{6}{c}{**SQUAD**} & \multicolumn{6}{c}{**MS MACRO**} & \multicolumn{6}{c}{**WAGQA**} & \multicolumn{6}{c}{**TreedyA**} \\  & **B** & **M** & **R** & **B** & **M** & **R** & **B** & **M** & **R** & **B** & **M** & **R** \\ \hline
**H** & 1.07 & 77.28 & 0.85 & 0.81 & 5.42 & 15.78 & 0.93 & 6.99 & 19.88 & 0.83 & 5.73 & 16.34 \\ MOSES & 0.51 & 1.04 & 7.28 & 0.72 & 9.74 & 15.82 & 0.32 & 10.26 & 17.27 & 0.29 & 9.96 & 17.02 \\
**Direction** & 1.125 & 14.91 & 22.51 & 10.82 & 13.55 & 20.38 & 10.84 & 14.18 & 22.01 & 9.59 & 12.21 & 19.76 \\

- \\ Du et al. (2017) & 1.28 & 16.26 & 39.75 &
- \\ \hline
**Esn-AE+DEQ** & 12.31 & 16.67 & 97.78 & 11.14 & 15.60 & 37.26 & 11.38 & 16.08 & 38.42 & 10.96 & 14.25 & 27.27 \\
**Esn-AE+REJ000** & 14.14 & 18.70 & 42.46 & 13.25 & 71.10 & 40.28 & 11.30 & 17.00 & 40.93 & 11.63 & 15.08 & 29.07 \\
**En-CE+LE+L000** & 14.27 & 18.78 & 12.93 & 13.81 & 17.87 & **41.23** & 12.32 & 13.44 & 27.22 & 12.45 & 15.93 & 30.26 \\ MZ & 12.46 & 16.95 & 40.27 & 11.56 & 15.93 & 38.32 & 11.83 & 16.84 & 39.26 & 11.52 & 16.42 & 28.92 \\ ECG & 12.79 & 17.04 & 40.92 & 12.11 & 16.32 & 18.96 & 12.14 & 17.04 & 39.82 & 11.67 & 16.59 & 29.12 \\ GO & 13.12 & 17.73 & 41.24 & 12.25 & 17.66 & 39.47 & 17.56 & 17.62 & 40.31 & 11.16 & 16.52 & 29.10 \\
**CIO** & 13.59 & 17.94 & 41.57 & 13.00 & 16.53 & 40.02 & 12.86 & 18.13 & 40.97 & 11.97 & 16.68 & 29.89 \\
**CO-ECO** & 13.97 & 18.18 & 41.90 & 13.44 & 17.16 & 40.65 & 13.22 & 18.34 & 41.28 & 12.34 & 16.65 & 29.63 \\
**Escenario** & 13.47 & 18.57 & 42.73 & 13.56 & 17.40 & 40.92 & 14.26 & 18.91 & 43.26 & 13.23 & 16.76 & 30.12 \\ \hline
**Esn-AE+LE+L000** & 14.28 & **15.79** & **42.97** & **15.74** & **17.59** & 41.07 & **15.26** & **19.45** & **44.77** & **14.87** & **16.88** & **31.91** \\ \end{tabular}
\end{table}
Table 5: Performance (B: BLEU4, M: METEOR, and R: ROUGE) of our model variants and various QG baselines on _SQUAD_, _MS MARCO_ and _ WikiQA_ datasets. The shaded part of the table shows the effect of various question selection heuristics when 10000 Wikipedia paragraphs are used as unlabeled data. The performance numbers for Tang et al. (2017) and Du et al. (2017) were not reported for all the settings.



_text always improve our models?_ Will the performance improve if we add more and more unsupervised data during self-training. According to our results in Tables 2, 3, 4 and 5, the answer is "probably yes". As we can observe from these tables, the performance of the QA and QG models improves as we increase \(K\), the size of the unsupervised data during training of the various _Ensemble+E&E(K)_ models. Having said that, we do see a tapering effect on the performance results, so it is clear that the performance will be capped by some upper-bound and we will need better ways of modeling language and meaning to make progress.

## 7 Related Work

Our work proposes an approach for joint modeling QA and QG. While QA has recieved a lot of attention from the research community with large scale community evaluations such as _NTCIR_, _TREC_, _CLEF_ spurring progress, the focus on QG is much more recent. Recently, there has been a renewed interest in reading comprehensions (also known as _machine comprehension_ - a nomenclature popularized by Richardson et al. (2013)). Various approaches Sachan et al. (2015); Wang et al. (2015); Sachan and Xing (2016); Sachan et al. (2016); Narasimhan and Barzilay (2015) have been proposed for solving this task. After the release of large benchmarks such as _SQUAD_, _MS MARCO_ and _WikiQA_, there has been a surge in interest on using neural network or deep-learning models for QA Yin et al. (2015); Seo et al. (2016); Shen et al. (2016); Chen et al. (2017); Liu et al. (2017); Hu et al. (2017). In our work, we deal with the answer sentence selection task and adapt the _Attentive Reader_ framework proposed in Hermann et al. (2015); Chen et al. (2016) as our base model. While, all these models were trained on question answer pairs, we propose a self-training solution to additionally leverage unsupervised text.

Similarly, there have been works on QG. Traditionally, rule based approaches with post-processing Woo et al. (2016); Heilman and Smith (2009, 2010) were the norm in QG. However, recent papers build on neural network approaches such as seq2seq Du et al. (2017); Tang et al. (2017); Zhou et al. (2017), CNNs and RNNs Duan et al. (2017) for QG. We also choose the seq2seq paradigm in our work. However, we leverage unsupervised text in contrast to these models.

Finally, some very recent works have concurrently recognized the relationship between QA and QG and have proposed joint training Tang et al. (2017); Wang et al. (2017) for the two. Our work differs from these as we additionally propose self-training to leverage unlabeled data to improve the two models. Self-training has seldom been used in NLP. Most prominently, they have been used for word sense disambiguation Yarowsky (1995), noun learning Rikoff et al. (2003) and recently, AMR parsing and generation Konstas et al. (2017). However, it has not been explored in this way for QA and QG.

An important decision in the workings of our self-training algorithm was the question selection using curriculum learning. While curriculum learning has seldom been used in NLP, we draw some ideas for curriculum learning from Sachan and Xing (2016) who conduct a case study of curriculum learning for question answering. However, their work focuses only on QA and not QG.

## 8 Conclusion

We described self-training algorithms for jointly learning to answer and ask questions while leveraging unlabeled data. We experimented with neural models for question answering and question generation and various careful strategies for question filtering based on curriculum learning and diversity promotion. This led to improved performance for both question answering and question generation on multiple datasets and new state-of-the-art results on _WikiQA_ and _TreeQA_ datasets.

## Acknowledgment

We acknowledge the CMLH fellowship to MS and ONR grant N000141712463 for funding support.

Figure 3: MAP for the best self-training model and QA baselines as we vary the proportion of labeled training set but keep the unlabeled text fixed (10K Wikipedia paragraphs).

 



## References

* Bahdanau et al. (2014) Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2014. Neural machine translation by jointly learning to align and translate. _arXiv preprint arXiv:1409.0473_ .
* Bengio et al. (2009) Yoshua Bengio, Jerome Louradour, Ronan Collobert, and Jason Weston. 2009. Curriculum learning. In _Proceedings of the 26th annual international conference on machine learning_. ACM, pages 41-48.
* Cantor (1946) Nathaniel Freeman Cantor. 1946. _Dynamics of learning_. Foster and Stewart publishing corporation, Buffalo, NY.
* Chen et al. (2016) Danqi Chen, Jason Bolton, and Christopher D Manning. 2016. A thorough examination of the cnn/daily mail reading comprehension task. _arXiv preprint arXiv:1606.02858_ .
* Chen et al. (2017) Danqi Chen, Adam Fisch, Jason Weston, and Antoine Bordes. 2017. Reading wikipedia to answer open-domain questions. _CoRR_ abs/1704.00051.
* Denkowski and Lavie (2014) Michael Denkowski and Alon Lavie. 2014. Meteor universal: Language specific translation evaluation for any target language. In _Proceedings of the ninth workshop on statistical machine translation_. pages 376-380.
* Du et al. (2017) Xinya Du, Junru Shao, and Claire Cardie. 2017. Learning to ask: Neural question generation for reading comprehension. _arXiv preprint arXiv:1705.00106_ .
* Duan et al. (2017) Nan Duan, Duyu Tang, Peng Chen, and Ming Zhou. 2017. Question generation for question answering. In _Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, EMNLP 2017, Copenhagen, Denmark, September 9-11, 2017_. pages 866-874.
* Graves et al. (2005) Alex Graves, Santiago Fernandez, and Jurgen Schmidhuber. 2005. Bidirectional lstm networks for improved phoneme classification and recognition. _Artificial Neural Networks: Formal Models and Their Applications-ICANN 2005_ pages 753-753.
* He et al. (2015) Hua He, Kevin Gimpel, and Jimmy J. Lin. 2015. Multiperspective sentence similarity modeling with convolutional neural networks. In _Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, EMNLP 2015, Lisbon, Portugal, September 17-21, 2015_. pages 1576-1586.
* He and Lin (2016) Hua He and Jimmy J. Lin. 2016. Pairwise word interaction modeling with deep neural networks for semantic similarity measurement. In _NAACL HLT 2016, The 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, San Diego California, USA, June 12-17, 2016_. pages 937-948.
* Heilman (2011) Michael Heilman. 2011. _Automatic factual question generation from text_. Ph.D. thesis, Carnegie Mellon University.
* Heilman and Smith (2009) Michael Heilman and Noah A Smith. 2009. Question generation via overgenerating transformations and ranking. Technical report, CARNEGIE-MELLON UNIV PITTSBURGH PA LANGUAGE TECHNOLOGIES INST.
* Heilman and Smith (2010) Michael Heilman and Noah A Smith. 2010. Good question! statistical ranking for question generation. In _Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics_. Association for Computational Linguistics, pages 609-617.
* Hermann et al. (2015) Karl Moritz Hermann, Tomas Kocisky, Edward Grefenstete, Lasse Espeholt, Will Kay, Mustafa Suleyman, and Phil Blunsom. 2015. Teaching machines to read and comprehend. In _Advances in Neural Information Processing Systems_. pages 1693-1701.
* Hu et al. (2017) Minghao Hu, Yuxing Peng, and Xipeng Qiu. 2017. Mnemonic reader for machine comprehension. _CoRR_ abs/1705.02798.
* Jiang et al. (2014) Lu Jiang, Deyu Meng, Teruko Mitamura, and Alexander G Hauptmann. 2014. Easy samples first: Self-paced reranking for zero-example multimedia search. In _Proceedings of the ACM International Conference on Multimedia_. ACM, pages 547-556.
* Jiang et al. (2015) Lu Jiang, Deyu Meng, Qian Zhao, Shiguang Shan, and Alexander G Hauptmann. 2015. Self-paced curriculum learning. In _Twenty-Ninth AAAI Conference on Artificial Intelligence_.
* Koehn et al. (2007) Philipp Koehn, Hieu Hoang, Alexandra Birch, Chris Callison-Burch, Marcello Federico, Nicola Bertoldi, Brooke Cowan, Wade Shen, Christine Moran, Richard Zens, Chris Dyer, Ondrej Bojar, Alexandra Constantin, and Evan Herbst. 2007. Moses: Open source toolkit for statistical machine translation. In _Proceedings of the 45th Annual Meeting of the ACL on Interactive Poster and Demonstration Seessions_. Association for Computational Linguistics, Stroudsburg, PA, USA, ACL '07, pages 177-180.
* Konstas et al. (2017) Ioannis Konstas, Srinivasan Iyer, Mark Yatskar, Yejin Choi, and Luke Zettlemoyer. 2017. Neural amr: Sequence-to-sequence models for parsing and generation. _arXiv preprint arXiv:1704.08381_ .
* Krueger and Dayan (2009) Kai A Krueger and Peter Dayan. 2009. Flexible shaping: How learning in small steps helps. _Cognition_ 110(3):380-394.
* Kumar et al. (2010) M Pawan Kumar, Benjamin Packer, and Daphne Koller. 2010. Self-paced learning for latent variable models. In _Advances in Neural Information Processing Systems_. pages 1189-1197.
* Levenshtein (1966) VI Levenshtein. 1966. Binary Codes capable of Correcting Deletions, Insertions and Reversals. _Soviet Physics Doklady_ 10:707

* Liang et al. (2008) Percy S Liang, Dan Klein, and Michael I Jordan. 2008. Agreement-based learning. In _Advances in Neural Information Processing Systems_. pages 913-920.
* Lin (2004) Chin-Yew Lin. 2004. Rouge: a package for automatic evaluation of summaries. In _Workshop on Text Summarization Branches Out, Post-Conference Workshop of ACL, Barcelona, Spain_.
* Liu et al. (2017) Rui Liu, Junjie Hu, Wei Wei, Zi Yang, and Eric Nyberg. 2017. Structural embedding of syntactic trees for machine comprehension. _CoRR_ abs/1703.00572.
* Miao et al. (2016) Yishu Miao, Lei Yu, and Phil Blunsom. 2016. Neural variational inference for text processing. In _International Conference on Machine Learning_. pages 1727-1736.
* Miller et al. (2016) Alexander H. Miller, Adam Fisch, Jesse Dodge, Amir-Hossein Karimi, Antoine Bordes, and Jason Weston. 2016. Key-value memory networks for directly reading documents. _CoRR_ abs/1606.03126.
* Narasimhan and Barzilay (2015) Karthik Narasimhan and Regina Barzilay. 2015. Machine comprehension with discourse relations. In _Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing of the Asian Federation of Natural Language Processing, ACL 2015, July 26-31, 2015, Beijing, China, Volume 1: Long Papers_. pages 1253-1262.
* Nguyen et al. (2016) Tri Nguyen, Mir Rosenberg, Xia Song, Jianfeng Gao, Saurabh Tiwary, Rangan Majumder, and Li Deng. 2016. Ms marco: A human generated machine reading comprehension dataset. _arXiv preprint arXiv:1611.09268_ .
* Nigam and Ghani (2000) Kamal Nigam and Rayid Ghani. 2000. Analyzing the effectiveness and applicability of co-training. In _Proceedings of the ninth international conference on Information and knowledge management_. ACM, pages 86-93.
* Papineni et al. (2002) Kishore Papineni, Salim Roukos, Todd Ward, and Weijing Zhu. 2002. Bleu: A method for automatic evaluation of machine translation. In _Proceedings of the 40th Annual Meeting on Association for Computational Linguistics_. ACL '02, pages 311-318.
* Pennington et al. (2014) Jeffrey Pennington, Richard Socher, and Christopher Manning. 2014. Glove: Global vectors for word representation. In _Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP)_. pages 1532-1543.
* Peterson (2004) Gail B Peterson. 2004. A day of great illumination: Bf skinner's discovery of shaping. _Journal of the Experimental Analysis of Behavior_ 82(3):317-328.
* Rajpurkar et al. (2016) Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. 2016. Squad: 100, 000+ questions for machine comprehension of text. _CoRR_ abs/1606.05250. [http://arxiv.org/abs/1606.05250](http://arxiv.org/abs/1606.05250).
* Rao et al. (2016) Jinfeng Rao, Hua He, and Jimmy Lin. 2016. Noise-contrastive estimation for answer selection with deep neural networks. In _Proceedings of the 25th ACM International on Conference on Information and Knowledge Management_. ACM, New York, NY, USA, CIKM '16, pages 1913-1916. [https://doi.org/10.1145/2983323.2983872](https://doi.org/10.1145/2983323.2983872).
* Richardson et al. (2013) Matthew Richardson, Christopher J.C. Burges, and Erin Renshaw. 2013. MCTest: A challenge dataset for the open-domain machine comprehension of text. In _Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing_. Association for Computational Linguistics, Seattle, Washington, USA, pages 193-203. [http://www.aclweb.org/anthology/D13-1020](http://www.aclweb.org/anthology/D13-1020).
* Riloff et al. (2003) Ellen Riloff, Janyce Wiebe, and Theresa Wilson. 2003. Learning subjective nouns using extraction pattern bootstrapping. In _Proceedings of the seventh conference on Natural language learning at HLT-NAACL 2003-Volume 4_. Association for Computational Linguistics, pages 25-32.
* Rush et al. (2015) Alexander M Rush, Sumit Chopra, and Jason Weston. 2015. A neural attention model for abstractive sentence summarization. _arXiv preprint arXiv:1509.00685_ .
* Sachan et al. (2015) Mrinnaya Sachan, Avinava Dubey, Eric P Xing, and Matthew Richardson. 2015. Learning answer-entailing structures for machine comprehension. In _Proceedings of the Annual Meeting of the Association for Computational Linguistics_.
* Sachan et al. (2016) Mrinnaya Sachan, Kumar Dubey, and Eric P. Xing. 2016. Science question answering using instructional materials. In _Proceedings of ACL_.
* Sachan and Xing (2016a) Mrinnaya Sachan and Eric P. Xing. 2016a. Easy questions first? A case study on curriculum learning for question answering. In _Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, ACL 2016, August 7-12, 2016, Berlin, Germany, Volume 1: Long Papers_.
* Sachan and Xing (2016b) Mrinnaya Sachan and Eric P. Xing. 2016b. Machine comprehension using rich semantic representations. In _Proceedings of ACL_.
* Santos et al. (2016) Cicero dos Santos, Ming Tan, Bing Xiang, and Bowen Zhou. 2016. Attentive pooling networks. _arXiv preprint arXiv:1602.03609_ .
* Seo et al. (2016) Minjeon Seo, Aniruddha Kembhavi, Ali Farhadi, and Hannaneh Hajishirzi. 2016. Bidirectional attention flow for machine comprehension. _arXiv preprint arXiv:1611.01603_ .
* Shen et al. (2014) Yelong Shen, Xiaodong He, Jianfeng Gao, Li Deng, and Gregoire Mesnil. 2014. A latent semantic model with convolutional-pooling structure for information retrieval. In _Proceedings of the 23rd ACM International Conference on Conference on Information and Knowledge Management_. ACM, pages 101 

Yelong Shen, Po-Sen Huang, Jianfeng Gao, and Weizhu Chen. 2016. Reasonet: Learning to stop reading in machine comprehension. _CoRR_ abs/1609.05284. [http://arxiv.org/abs/1609.05284](http://arxiv.org/abs/1609.05284).
* Skinner (1958) Burhus F Skinner. 1958. Reinforcement today. _American Psychologist_ 13(3):94.
* Sutskever et al. (2014) Ilya Sutskever, Oriol Vinyals, and Quoc V Le. 2014. Sequence to sequence learning with neural networks. In _Advances in neural information processing systems_. pages 3104-3112.
* Tang et al. (2017) Duyu Tang, Nan Duan, Tao Qin, and Ming Zhou. 2017. Question answering and question generation as dual tasks. _arXiv preprint arXiv:1706.02027_ .
* Tay et al. (2017) Yi Tay, Anh Tuan Luu, and Siu Cheung Hui. 2017. Enabling efficient question answer retrieval via hyperbolic neural networks. _CoRR_ abs/1707.07847. [http://arxiv.org/abs/1707.07847](http://arxiv.org/abs/1707.07847).
* Wang et al. (2016a) Bingning Wang, Kang Liu, and Jun Zhao. 2016a. Inner attention based recurrent neural networks for answer selection. In _Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, ACL 2016, August 7-12, 2016, Berlin, Germany, Volume 1: Long Papers_.
* Wang et al. (2015) Hai Wang, Mohit Bansal, Kevin Gimpel, and David McAllester. 2015. Machine comprehension with syntax, frames, and semantics. In _Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 2: Short Papers)_. volume 2, pages 700-706.
* Wang et al. (2007) Mengqiu Wang, Noah A Smith, and Teruko Mitamura. 2007. What is the jeopardy model? a quasi-synchronous grammar for qa. In _EMNLP-CoNLL_. volume 7, pages 22-32.
* Wang and Jiang (2016) Shuohang Wang and Jing Jiang. 2016. A compare-aggregate model for matching text sequences. _arXiv preprint arXiv:1611.01747_ .
* Wang et al. (2017) Tong Wang, Xingdi Yuan, and Adam Trischler. 2017. A joint model for question answering and question generation. _CoRR_ abs/1706.01450.
* Wang et al. (2016b) Zhiguo Wang, Haitao Mi, and Abraham Ittycheriah. 2016b. Sentence similarity learning by lexical decomposition and composition. _CoRR_ abs/1602.07019.
* Woo et al. (2016) Simon Woo, Zuyao Li, and Jelena Mirkovic. 2016. Good automatic authentication question generation. In _Proceedings of the 9th International Natural Language Generation conference_. pages 203-206.
* Yang et al. (2015) Yi Yang, Scott Wen-tau Yih, and Chris Meek. 2015. Wikiqa: A challenge dataset for open-domain question answering. In _Proceedings of EMNLP_.
* Yarowsky (1995) David Yarowsky. 1995. Unsupervised word sense disambiguation rivaling supervised methods. In _Proceedings of the 33rd annual meeting on Association for Computational Linguistics_. Association for Computational Linguistics, pages 189-196.
* Yin et al. (2015) Wenpeng Yin, Hinrich Schutze, Bing Xiang, and Bowen Zhou. 2015. Abcnn: Attention-based convolutional neural network for modeling sentence pairs. _arXiv preprint arXiv:1512.05193_ .
* Yu et al. (2014) Lei Yu, Karl Moritz Hermann, Phil Blunsom, and Stephen Pulman. 2014. Deep learning for answer sentence selection. _arXiv preprint arXiv:1412.1632_ .
* Zaremba et al. (2014) Wojciech Zaremba, Ilya Sutskever, and Oriol Vinyals. 2014. Recurrent neural network regularization. _arXiv preprint arXiv:1409.2329_ .
* Zhou et al. (2017) Qingyu Zhou, Nan Yang, Furu Wei, Chuanqi Tan, Hangbo Bao, and Ming Zhou. 2017. Neural question generation from text: A preliminary study. _CoRR_ abs/1704.01792. [http://arxiv.org/abs/1704.01792](http://arxiv.org/abs/1704.01792).
* Zhu (2005) Xiaojin Zhu. 2005. Semi-supervised learning literature survey. Technical Report 1530, Computer Sciences, University of Wisconsin-Madison.