Controllable Open-ended Question Generation with A New Question Type Ontology (2021)

Abstract: We investigate the less-explored task of generating open-ended questions that are typically answered by multiple sentences. We first define a new question type ontology which differentiates the nuanced nature of questions better than widely used question words. A new dataset with 4,959 questions is labeled based on the new ontology. We then propose a novel question type-aware question generation framework, augmented by a semantic graph representation, to jointly predict question focuses and produce the question. Based on this framework, we further use both exemplars and automatically generated templates to improve controllability and diversity. Experiments on two newly collected large-scale datasets show that our model improves question quality over competitive comparisons based on automatic metrics. Human judges also rate our model outputs highly in answerability, coverage of scope, and overall quality. Finally, our model variants with templates can produce questions with enhanced controllability and diversity.

## introduction question-asking has long served as an effective instrument for knowledge learning @xcite ) and assessing learning progress @xcite @xcite . compared to the widely studied task of generating factoid questions that inquire about "one bit" of information @xcite @xcite , this work is interested in generating open-ended questions that require deep comprehension and long-form answers @xcite . such open-ended questions are valuable in education, e.g., to facilitate complex knowledge acquisition @xcite and nurture reasoning skills @xcite , as well as in other applications like improving search engines (han input: it's a difficult task to undertake. teenagers tend to identify gangs with "fitting" in. peer pressure plays a large part in it and sometimes teenagers have problems with their own identity being part of a gang deals with those issues. it also provides a little bit of respect on the street ... ## related work question generation has long been studied to reduce human efforts in constructing questions for knowledge learning evaluation @xcite . early work relies on syntactic transformation to convert declarative sentences to questions @xcite . recent advancements rely on sequence-to-sequence models to generate a question from a given sentence or paragraph by considering the focus, type, and general-specific relations of questions @xcite @xcite . in particular, question likelihoods and rewards are designed to steer them toward being addressed by the given answers @xcite . attempts are also made toward creating complex questions that require multi-hop reasoning over the given text, and graph-based representations have been an enabling tool to facilitate the access to both entities and relations @xcite . while our model also enhances the input with a semantic graph, it boasts a richer representation by including both dependency and semantic relations, with predicted question focuses highlighted via extra node embeddings. moreover, we create a separate layer of cross attentions that is dedicated to the semantic graph, while prior work uses the same set of attentions to attend to the concatenated text and graph representations. given the data-driven nature of question generation and answering tasks, recent studies take advantage of the availability of large-scale qa datasets, such as squad @xcite , ms marco @xcite , hotpotqa @xcite , drop @xcite , inter alia. these corpora mainly contain factoid questions, while our newly collected datasets are not only larger in size but also comprise significantly more open-ended questions for querying reasons and procedures. a dataset closer to ours is eli5 @xcite , which also obtains open-ended questionanswer pairs from reddit, while one of our datasets includes more reddit communities and thus covers a wider range of topics. our work is more inline with generating deeper questions with responses that span over multiple sentences, where manually constructed templates are found effective @xcite . for example, @xcite use crowdsourcing to collect question templates based on an ontology derived from wikipedia and freebase topics. different from the topic-based ontology, our question types are more aligned with cognitive levels. moreover, our templates are automatically learned from training data. recent work @xcite focuses on asking clarification questions based on both retrieval and generation models. as there has been no suitable framework for diverse types of questions, this work aims to fill the gap by introducing type-aware generation models which optionally leverage question templates for better controllability. generating diverse questions is much less studied, with existing approaches mainly focusing on entity replacement @xcite , sampling @xcite , and post-filtering @xcite . however, the produced diversity is driven by word choice and syntax variation, with little ability to control on question types, which is the focus of this work. ## type-aware open-ended question generation in this section, we present our type-aware question generation framework. as shown in figure 2 , our model takes in a multi-sentence text and a predicted question type. built on shared input representations, it first detects question focuses from a semantic graph, and then generates the question ( § 4.1). we also propose two model variants that consider automatically extracted template exemplars or generated templates to achieve controllability ( § 4.2), enabling the generation of diverse questions. ## conclusion we present a new question type ontology which better captures the nuances of questions to support the study of open-ended question generation. we further annotate a new dataset with 4,959 questions based on the proposed ontology. we describe a joint question focus detection and question generation framework with a novel semantic graphaugmented representation, which is directly built on large pre-trained models. based on this framework, we also enhance the controllability and diversity of generated questions by employing template exemplars or automatically generated templates. experiments on two large datasets show that questions generated by our models have better quality and higher diversity than non-trivial comparisons, with similar results rated by human judges.