Stay Hungry, Stay Focused: Generating Informative and Specific Questions in Information-Seeking Conversations (2020)

Abstract: We investigate the problem of generating informative questions in information-asymmetric conversations. Unlike previous work on question generation which largely assumes knowledge of what the answer might be, we are interested in the scenario where the questioner is not given the context from which answers are drawn, but must reason pragmatically about how to acquire new information, given the shared conversation history. We identify two core challenges: (1) formally defining the informativeness of potential questions, and (2) exploring the prohibitively large space of potential questions to find the good candidates. To generate pragmatic questions, we use reinforcement learning to optimize an informativeness metric we propose, combined with a reward function designed to promote more specific questions. We demonstrate that the resulting pragmatic questioner substantially improves the informativeness and specificity of questions generated over a baseline model, as evaluated by our metrics as well as humans.

## introduction conversations are a primary means to seek and communicate information between humans, where asking the right question is an important prerequisite for effective exchange of knowledge. learning to ask questions in conversations can help computer systems not only acquire new knowledge, but also engage human interlocutors by making them feel heard @xcite . previous work on question generation often falls into three classes: generating questions according to a discrete schema or end goal @xcite , transforming the answer statement into a question @xcite @xcite , or generating questions with data-driven systems by conditioning on the context where the answer comes background: spandau ballet (english band) spandau ballet were an english new wave band formed in islington, london, in 1979. inspired by … ## related work question generation. question generation has long been studied in the education and psychology communities as a means to assess and promote reading comprehension in humans @xcite . in natural language processing, question generation has been explored to improve the systems in various natural language processing tasks, e.g., the quality of question answering systems @xcite as well as information retrieval in an open-domain question answering system @xcite . some of the first question generation systems are rule-based @xcite @xcite , while large-scale question answering datasets, e.g., squad @xcite , have recently kindled research interest in data-driven approaches. @xcite and @xcite apply sequence-to-sequence (seq2seq) models to generate squad questions from wikipedia sentences containing the answers. the release of large conversational question answering datasets such as quac @xcite and coqa @xcite enabled @xcite , @xcite and @xcite to extend previous neural seq2seq question generators by conditioning them on the conversation history and the context that contains the answer, while @xcite remove answers to the reference question to generate curiosity-driven questions from the rest of the context. despite their success, most existing approaches to question generation are limited to either reading comprehension settings where potential answers are known a priori, or goal-oriented settings where the schema of knowledge is limited @xcite . this prevents them from being applied to an open-domain communication setting, where the purpose of questions is to acquire information that is unknown ahead of time. ## method in this section, we outline the setup for the communication problem we set out to address, present a baseline system, and lay out our approach to extending it to reason pragmatically to acquire information more efficiently. ## analysis we further analyze concrete examples of generated questions in conversations to understand the behavior of our informativeness and specificity metrics. ## conclusion in this paper, we presented a question generation system in information-seeking conversations. by optimizing our proposed automatic metrics for informativeness and specificity, the model is able to generate pragmatically relevant and specific questions to acquire new information about an unseen source of textual knowledge. our proposed method presents a practical if shallow implementation of pragmatics in an open-domain communication setting beyond simple reference games. we hope that our work brings the community's attention to this important problem of natural language communication under information asymmetry.