How to Ask Good Questions? Try to Leverage Paraphrases (2020)

Abstract: Given a sentence and its relevant answer, how to ask good questions is a challenging task, which has many real applications. Inspired by human’s paraphrasing capability to ask questions of the same meaning but with diverse expressions, we propose to incorporate paraphrase knowledge into question generation(QG) to generate human-like questions. Specifically, we present a two-hand hybrid model leveraging a self-built paraphrase resource, which is automatically conducted by a simple back-translation method. On the one hand, we conduct multi-task learning with sentence-level paraphrase generation (PG) as an auxiliary task to supplement paraphrase knowledge to the task-share encoder. On the other hand, we adopt a new loss function for diversity training to introduce more question patterns to QG. Extensive experimental results show that our proposed model obtains obvious performance gain over several strong baselines, and further human evaluation validates that our model can ask questions of high quality by leveraging paraphrase knowledge.

## introduction question generation (qg) is an essential task for nlp, which focuses on generating grammatical questions for given paragraphs or sentences. it plays a vital role in various realistic scenarios. for educational purposes, qg can create reading comprehension materials for language learners @xcite . for business use, qg can bring benefits to conversation systems and chat-bots for effective communication with humans @xcite . besides, automatically-generated questions can be conversely used for constructing question answering datasets to enhance reading comprehension sys- * corresponding author. sentence: the next three drives of the game would end in punts. answer: punts reference question: what did the next three drives result in? question generated by the baseline model: the next three drives of the game would end in what? sentence: in ring theory, the notion of number is generally replaced with that of ideal. answer: ring theory reference question: in what theory is the idea of a number exchanged with that of an ideal? question generated by the baseline model: in what theory is the notion of number replaced with that of ideal? recent neural network-based methods have achieved promising results on qg, most of which are based on the seq2seq attention framework @xcite @xcite @xcite , enriched with lexical features @xcite @xcite or enhanced by copy mechanism @xcite @xcite . although much progress has been made for qg, existing approaches do not explicitly model the "notorious" lexical and syntactic gaps in the generation process. that is, some parts of two texts (e.g. the input sentence and reference question, the reference question and generated question) may convey the same meaning but use different words, phrases or syntactic patterns. in real communication, humans often paraphrase a source sentence to ask questions which are grammatical and coherent. take squad @xcite as an example, which is a popular reading comprehension dataset and has been widely used for qg, there is a large percentage of questions created by paraphrasing (33.3% of the questions contain synonymy variations and 64% of questions contain syntactic variations @xcite . two examples are shown in to address this issue, we introduce paraphrase knowledge in the qg process to generate humanlike questions. the sketch of our design is illustrated in figure 1 . to make our model easy to implement and train the model in an end-to-end fashion, we do not use any extra paraphrase generation (pg) dataset but just use a simple backtranslation method to automatically create paraphrases for both the input sentences and reference questions. based on the high-quality expanded data, we propose a two-hand hybrid model. on the left hand, using the expanded sentence paraphrase as the target of pg, we perform multi-task learning with pg and qg, to optimize the task-share encoder with the paraphrase knowledge. on the right hand, with the gold reference question and question paraphrase as qg's multi-targets, we adopt a new min-loss function, to enable the qg module to learn more diverse question patterns. we conduct extensive experiments on squad and marco @xcite . results show that both separate modules, the pg auxiliary task and the min-loss function, obviously improve the performances of qg task, and combing them achieves further improvements. furthermore, human evaluation results show that our hybrid model can ask better and more human-like questions by incorporating paraphrase knowledge. ## related work for current mainstream neural network-based methods on qg, most approaches utilize the seq2seq model with attention mechanism @xcite @xcite . to obtain better representations of the input sequence and answer, the answer position and token lexical features are treated as supplements for the neural encoder @xcite @xcite . similar to other text generation tasks, many works on qg also employ copy or pointer mechanism to overcome the oov problem @xcite @xcite . recently, @xcite employ language modeling (lm) as an auxiliary task to enrich the encoder representations. in this paper, we adopt this work as one of the baseline models, since their universal model is easy to implement and achieves promising results for qg. in order to make use of the context information of paragraphs, @xcite al. (2018b) paraphrase generation is also a challenging task for nlp. recent works usually obtain paraphrases by reordering or modifying the syntax or lexicon based on some paraphrase databases and rules @xcite , or by employing some neural generation methods @xcite . in this paper, we employ a simple and effective paraphrasing method to expand both input sentences and reference questions. our method also can be replaced with more sophisticated paraphrasing methods. paraphrase knowledge has been used to improve many nlp tasks, such as machine translation, ques-tion answering, and text simplification. @xcite use paraphrase techniques to deal with unknown phrases to improve statistical machine translation. @xcite and @xcite employ paraphrase knowledge to enhance question answering models. @xcite utilize paraphrase and context-based lexical substitution knowledge to improve simplification task. similarly, @xcite combine paraphrase rules of ppdb @xcite with transformer @xcite to perform sentence simplification task. @xcite al. (2018a) propose a multi-task learning framework with pg and simplification. in addition, @xcite and @xcite use paraphrase as data argumentation for their primary tasks. different from these works, we leverage paraphrase knowledge for question generation, by automatically constructing a built-in paraphrase corpus without using any external paraphrase knowledge bases. ## model description in this section, we first describe two baseline models we used: feature-enriched pointer-generator and language modeling enhanced qg. then we explain how to obtain paraphrase resources and show the quality statistics. furthermore, we describe in detail two modules of utilizing paraphrase knowledge: the pg auxiliary task and the min loss function, as well as their combination. the overall structure of our hybrid model is shown in figure 2 . @xcite model with rich features proposed by @xcite . they adopt a bidirectional lstm as the encoder, which takes the featureenriched embedding e i as input: ## conclusion and future work in this paper, we propose a two-hand hybrid model leveraging paraphrase knowledge for qg. the experimental results of independent modules and hybrid models prove that our models are effective and transferable. besides, human evaluation results demonstrate that the paraphrase knowledge benefits our model to ask more human-like questions of high quality. in the future, we will explore more diverse and advanced paraphrase expanding methods for both sentence and paragraph level qg. moreover, we will apply our methods to other similar tasks, such as sentence simplification.