Ask to Learn: A Study on Curiosity-driven Question Generation (2020)

Abstract: We propose a novel text generation task, namely Curiosity-driven Question Generation. We start from the observation that the Question Generation task has traditionally been considered as the dual problem of Question Answering, hence tackling the problem of generating a question given the text that contains its answer. Such questions can be used to evaluate machine reading comprehension. However, in real life, and especially in conversational settings, humans tend to ask questions with the goal of enriching their knowledge and/or clarifying aspects of previously gathered information. We refer to these inquisitive questions as Curiosity-driven: these questions are generated with the goal of obtaining new information (the answer) which is not present in the input text. In this work, we experiment on this new task using a conversational Question Answering (QA) dataset; further, since the majority of QA dataset are not built in a conversational manner, we describe a methodology to derive data for this novel task from non-conversational QA data. We investigate several automated metrics to measure the different properties of Curious Questions, and experiment different approaches on the Curiosity-driven Question Generation task, including model pre-training and reinforcement learning. Finally, we report a qualitative evaluation of the generated outputs.

## introduction the growing interest in machine reading comprehension (mrc) has sparked significant research efforts on question generation (qg), the dual task to question answering (qa). in qa, the objective is to produce an adequate response given a query and a text; conversely, for qg, the task is generally defined as generating relevant questions given a source text and, optionally, a specific target answer included therein. to our knowledge, all works tackling qg have thus far exclusively focused on generating relevant questions which can be answered given the source text: for instance, given the 1st coling conference took place in 1965 as input, a question likely to be automatically generated would be when did the 1st coling conference take place?, where the answer 1965 is a span of the input. such questions are useful to evaluate reading comprehension for both machines @xcite and humans @xcite . however, the human ability of asking questions goes well beyond evaluation: asking questions is essential in education @xcite and has been proven to be fundamental for children cognitive development @xcite . curiosity is baked into the human experience: it allows to extend one's comprehension and knowledge by asking questions that, while being relevant to context, are not directly answerable by it, thus being inquisitive and curious. the significance of such kind of questions is two-fold: first, they allow for gathering novel relevant information, e.g. a student asking for clarification; second, they are tightly linked to one's understanding of the context, e.g. a teacher testing a student's knowledge by asking questions whose answers require a deeper understanding of the context and more complex reasoning. from an applicative point of view, we deem the ability to generate curious, inquisitive, questions as highly beneficial for a broad range of scenarios: i) in the context of human-machine interaction (e.g. robots, chat-bots, educational tools), where the communication with the users could be more natural; and ii) during the learning process itself, which could be partially driven in a self-supervised manner, reminiscent of how humans learn by exploring and interacting with their environment. to our knowledge, this is the first paper attempting to tackle curiosity-driven neural question generation. the contributions of this paper can be summarized as follows: deep learning models have been widely applied to text generation tasks such as machine translation @xcite , abstractive summarization @xcite or dialog @xcite , providing significant gains in performance. the state of the art approaches are based on sequence to sequence models @xcite . in recent years, significant research efforts have been directed to the tasks of machine reading comprehension (mrc) and question answering (qa) @xcite . the data used for tackling these tasks are usually composed of {context, question, answer} triplets: given a context and the question, a model is trained to predict the answer. following qa, research on question generation (qg) @xcite has also seen increasing interest from the community: the qg task @xcite can be considered as the dual task for qa @xcite : given a context and an answer span, the model is trained to generate the corresponding question. one of the main motivations is that an effective qg model can be used to generate synthetic data in order to augment existing qa datasets @xcite . for instance, @xcite proposed a reinforcement learning setup trained using a qa-based metric reward: given a paragraph and an answer, the model first generates questions; then, the paragraph and the corresponding generated questions are given to a pre-trained qa model which predicts an answer; finally, the reward is computed as the number of overlapping words between the ground truth answer and the predicted answer -in other words, the reward to maximize, for the qg model, corresponds to the qa score. for an extensive evaluation of models trained with different rewards we refer the reader to the work of @xcite . most of these works follow the approach by @xcite , who applied reinforcement to neural machine translation: first, a sequence to sequence model is trained under teacher forcing @xcite to optimize cross-entropy, hence helping to reduce the action space (i.e. the vocabulary size); then, the model is finetuned with a mix of teacher forcing and reinforce @xcite . for automatic evaluation, all previous works on qg resort to bleu metrics @xcite , originally developed and widely used in machine translation. however, how to evaluate text generation models remains an open research question: nema and khapra (2018) pointed out that, on qg tasks, the correlation between bleu and human evaluation was poor. a thorough investigation of the behavior of open-domain conversational agents has been recently presented by @xcite . using controllable neural text generation methods, the authors control important attributes for chit-chat dialogues, including question-asking behavior. among the take-away messages of this work, is that question-asking represents an essential component in an engaging chitchat pipeline: the authors find, via a large-scale human validation study, that agents with higher rates of question-asking obtain qualitative improvements in terms of inquisitiveness, interestingness and engagingness. indeed, in a conversational setting, it can be expected that the nature of follow-up questions significantly differs from those used as target in a traditional qg training setup: as mentioned earlier, qg has so far been framed as the dual task to qa, hence training models to generate questions whose answer is present in the input context. in contrast, we argue that in natural conversations the questions follow the input context but are rather a means to augment one's knowledge (as their answer is not explicit in the input context). in this work, we thus define the task as curiosity-driven question generation. ## dataset question answering datasets are usually composed of a set of questions associated with reading passages (the context) and the corresponding answers contained therein. the qa task is defined as finding the answer to a question given the context. as opposed, the question generation (qg) task is to generate the question given the input and (optionally) the answer. most previous efforts on the qg task have resorted to the widely used stanford question answering dataset (squad) @xcite . it contains roughly 100,000 questions posed by crowd-workers on a selected sample of wikipedia articles. several other qa datasets have also been recently published accounting for characteristic such as requiring multi-passage or discrete reasoning @xcite ; further, conversational qa datasets have been made available: coqa @xcite and quac @xcite have the desirable property to be in a dialogue-like setting. in our scenario, curiosity-driven qg, the reading passage associated with a question should not contain the answer, but rather pave the way for asking a new, curious, question -whose answer would eventually enrich the knowledge on the matter at hand. therefore, a natural choice to build qg data would be to rely on existing datasets for conversational qa. a detailed comparison of the above-mentioned coqa and quac datasets is provided by @xcite , who reports the proportion of topic error (i.e. questions unlikely to be asked in the context) and entity salad (i.e. questions unanswerable for any context): foot_0 compared to quac, coqa is found to include significantly more topic error and entity salad questions. for this reason, we resort to quac in order to derive data curiosity-driven qg. furthermore, recognizing the fact that the great majority of qa datasets available does not account for conversational characteristics, we propose a methodology to derive data for curiosity-driven question generation from standard qa datasets, and apply it to the popular squad @xcite . for both our data sources, and consistently with standard qa and qg tasks, we encode each sample as a triplet {p, q, a} where the paragraph p comprises n sentences [s 0 , ..., s n ], and a represents the answer to the question q. a canonical qg approach would thus use s a , i.e. the sentence of p that contains the answer, as source, and q as generation target. on the contrary, for curiosity-driven qg, any sentence s x from p can potentially be used as the source sequence, as long as it does not contain the answeri.e. under the necessary constraint of @xmath0. in the following subsections, we elaborate on additional constraints depending on the nature of the source data. ## metrics automatic evaluation of natural language generation (nlg) systems is a challenging task @xcite . for qg, n-gram based similarity metrics are commonly used. these measures evaluate how similar the generated text is to the corresponding reference(s). while they are known to suffer from several shortcomings @xcite @xcite , they allow to evaluate specific properties of the developed models. in this work, we use various automatic metrics detailed below, and we assess their quality for our task through a human evaluation -see section 6. bleu one of the most popular metrics for qg, bleu @xcite provides a set of measures to compare automatically generated texts against one or more references. in particular, bleu-n is based on the count of overlapping n-grams between the candidate and its corresponding reference(s). self-bleu within the field of computational creativity, diversity is considered a desirable property @xcite . indeed, generating always the same question such as "what is the meaning of the universe?" would be an undesirable behavior, reminiscent of the "collapse mode" observed in generative adversarial networks (gan) @xcite . therefore, we adopt self-bleu, originally proposed by @xcite , as a measure of diversity for the generated text sequences. self-bleu is computed as follows: for each generated sentence s i , a bleu score is computed using s i as hypothesis while the other generated sentences are used as reference. when averaged over all the references, it thus provides a measure of how diverse the questions are: low self-bleu scores indicate high diversity. ## experiments baseline model as baseline architecture we adopt the popular transformer @xcite , which proved to perform well on a wide range of text generation tasks. among these, neural machine translation @xcite , automatic summarization @xcite , and question generation @xcite . it can be briefly described as a sequence-to-sequence model with symmetric encoder and decoder based on a self-attention mechanism, which allows to overcome the inherent obstacles to parallelism present in recurrent models such as long short time memory (lstm) networks @xcite . the copy mechanism @xcite proved beneficial for qg @xcite ; indeed, the qg task is very sensitive to rare and out of vocabulary words such as named entities and this mechanism helps deal with it efficiently: more than 50% of the answers in squad, for instance, correspond to named entities -see for our experiments, we used the following hyper-parameters for the transformer: @xmath1; d @xmath2; d @xmath3; and, @xmath4. experiments run with the original hyper-parameters foot_5 as proposed by @xcite obtained consistent and numerically similar results. during training, we used mini-batches of size 64 and the adam optimizer @xcite . at generation time, the decoding steps are computed via beam search with @xmath5. ## discussion what is the impact of the pretraining? we observe that for pretrained models (i.e. pt and pt+rl) the correctness is significantly higher than the models without pretraining (i.e. base and rl). this is consistent with the higher bleu observed for these models in ## conclusions asking inquisitive questions allows humans to learn from each other and increase their knowledge. we thus proposed a new task: curiosity-driven question generation, which attempts to address such a key component for several human-machine interaction scenarios. in absence of data directly usable for this task, we proposed an automatic method to derive it from conversational qa datasets. further, recognizing that the great majority of qa datasets are not conversational, we also extended the method to standard qa data. our experiments, which include learning strategies such as pretraining and reinforcement, show promising results under both automatic and human evaluation. in future works, we plan to extend the approach to conditional generation of curiosity-driven questions.