Asking It All: Generating Contextualized Questions for any Semantic Role (2021)

Abstract: Asking questions about a situation is an inherent step towards understanding it. To this end, we introduce the task of role question generation, which, given a predicate mention and a passage, requires producing a set of questions asking about all possible semantic roles of the predicate. We develop a two-stage model for this task, which first produces a context-independent question prototype for each role and then revises it to be contextually appropriate for the passage. Unlike most existing approaches to question generation, our approach does not require conditioning on existing answers in the text. Instead, we condition on the type of information to inquire about, regardless of whether the answer appears explicitly in the text, could be inferred from it, or should be sought elsewhere. Our evaluation demonstrates that we generate diverse and well-formed questions for a large, broad-coverage ontology of predicates and roles.

## introduction soliciting information by asking questions is an essential communicative ability, and natural language question-answer (qa) pairs provide a flexible format for representing and querying the information expressed in a text. this flexibility has led to applications in a wide range of tasks from reading comprehension @xcite to information seeking dialogues @xcite . automatically generating questions can potentially serve as an essential building block for such applications. previous work in question generation has either required human-curated templates @xcite , limiting coverage and question fluency, or generated questions for answers already identified in the text (heilman and the plane took off in los angeles. the tourists will arrive in mexico at noon. entity in motion who will arrive in mexico? end point where will the tourists arrive? start point where will the tourists arrive from? manner how will the tourists arrive? cause why will the tourists arrive? temporal when will the tourists arrive? figure 1 : example role questions for "arrive". some questions are for explicit arguments (entity in motion,end point, temporal), some for implicit (start point, manner) ones, and some for arguments that do not appear at all (cause). smith, 2010; dhole and manning, 2020). openended generation of information-seeking questions, where the asker poses a question inquiring about a certain type of information, remains a significant challenge. in this work, we propose to resolve these difficulties by generating role questions. in particular, for any predicate expressed in a text and semantic role associated with that predicate, we show how to generate a contextually-appropriate question whose answer-if present-corresponds to an argument of the predicate bearing the desired role. some examples are shown in figure 1 . since the set of possible questions is scoped by the relations in the underlying ontology, this gives us the ability to ask it all: pose information-seeking questions that exhaustively cover a broad set of semantic relations that may be of interest to downstream applications. concretely, we generate questions derived from qa-srl @xcite for the semantic role ontology in propbank @xcite using a two-stage approach. in the first stage ( §4.1), we leverage corpus-wide statistics to compile an ontology of simplified, context-independent prototype questions for each propbank role. in the second stage ( §4.2), we contextualize the question using a learned translation from prototypes to their contextualized counterparts, conditioned on a sentence. to that end we present a new resource of framealigned qa-srl questions which are grounded in their source context. this setup decouples posing a question that captures a semantic role from fitting that question to the specific context of the passage. as we show in §5, the result is a system which generates questions that are varied, grammatical, and contextually appropriate for the passage, and which correspond well to their underlying role. 1 the ability to exhaustively enumerate a set of questions corresponding to a known, broadcoverage underlying ontology of relations allows for a comprehensive, interpretable, and flexible way of representing and manipulating the information that is contained in-or missing from-natural language text. in this way, our work takes an essential step towards combining the advantages of formal ontologies and qa pairs for broad-coverage natural language understanding. ## background question generation automatic question generation has been employed for use cases such as constructing educational materials @xcite , clarifying user intents @xcite , and eliciting labels of semantic relations in text @xcite @xcite . methods include transforming syntactic trees @xcite and srl parses @xcite , as well as training seq2seq models conditioned on the question's answer @xcite or a text passage containing the answer @xcite . by and large, these approaches are built to generate questions whose answers are already identifiable in a passage of text. however, question generation has the further potential to seek new information, which requires ask-ing questions whose answers may only be implicit, inferred, or even absent from the text. doing so requires prior specification of the kind and scope of information to be sought. as a result, previous work has found ways to align existing relation ontologies with questions, either through human curation @xcite -which limits the approach to very small ontologies -or with a small set of fixed question templates @xcite ) -which relies heavily on glosses provided in the ontology, sometimes producing stilted or ungrammatical questions. in this work, we generate natural-sounding information-seeking questions for a broad-coverage ontology of relations such as that in propbank @xcite . qa-srl integral to our approach is qa-srl @xcite , a representation based on question-answer pairs which was shown by @xcite and @xcite to capture the vast majority of arguments and modifiers in prop-bank and nombank @xcite . instead of using a pre-defined role lexicon, qa-srl labels semantic roles with questions drawn from a 7-slot template, whose answers denote the argument bearing the role (see qa-srl is useful to us because of its close correspondence to semantic roles and its carefully restricted slot-based format: it allows us to easily transform questions into context-independent prototypes which we can align to the ontology, by removing tense, negation, and other information immaterial to the semantic role ( §4.1). it also allows us to produce contextualized questions which sound natural in the context of a passage, by automatically aligning the syntactic structure of different questions for the same predicate ( §4.2). ## task definition our task is defined with respect to an ontology of semantic roles such as propbank. given a passage of text with a marked predicate and a chosen role of that predicate, we aim to generate a naturallyphrased question which captures that semantic relation. for example, consider figure 1 . for the predicate arrive.01 and role a0 (defined in propbank as entity in motion), we want to generthe contextualizer takes in a prototype question and a context, and outputs a frame-aligned qa-srl question. note how frame-aligned qa-srl questions preserve the role expressed in their respective prototypes. c: selecting prototype questions according to their ability to recover explicit arguments (see §4.1). we test each prototype (1) against a sample of arguments for each role (2). contextualized versions of each prototype question for each sampled sentence (3) are fed (with the sentences) to a qa model, and the predicted answer is evaluated against the gold argument. the highest ranking prototype (4) is selected for that role. ## method generating role questions requires mapping from an input sentence, a predicate, and role to a natural language question, regardless of whether the question is answerable in the text. however, we don't have training data of this nature: existing question-answer driven approaches to semantic annotation @xcite only elicit answerable questions from annotators, and existing resources with unanswerable questions @xcite , inter alia) do not systematically align to semantic roles in any clear way. indeed, we will show in §5.2 that training a seq2seq model to directly generate role questions from questionanswer pairs annotated with explicit semantic roles yields poor results for implicit or missing arguments, because the model overfits towards only asking answerable questions. instead, we adopt a two-stage approach: first, we leverage corpus-wide statistics to map each role in an ontology to a context-independent prototype question which provides the high-level syntactic structure of any question for that role. for example, bring.01's destination argument a2 may receive the prototype question where does something bring something? second, we employ a question contextualization step which aligns the tense, negation/modality, and entities in the question with those in the particular source text, e.g., where did the pilot bring the plane?. the result is a question about the source text which is amenable to traditional qa systems and whose semantically correct answer should relate to the predicate according to the given role. figure 3 shows example prototype and contextualized questions for the predicate change.01. foot_1 ## evaluation to assess our system, we perform an intrinsic evaluation against a seq2seq baseline ( §5.2) as well as comparisons to existing question generation systems ( §5.3). ## conclusion we presented an approach to produce fluent natural language questions targeting any predicate and semantic role in the context of a passage. by leveraging the syntactic structure of qa-srl questions in a two-stage approach, we overcome a lack of annotated data for implicit and missing arguments and produce questions which are highly specific to the desired roles. this enables the automatic generation of information-seeking questions covering a large, broad-coverage set of semantic relations, which can bring the benefits of qa-based representations to traditional srl and information extraction tasks. on the qa-srl bank 2.0, we aligned 190k qapairs with predicted srl labels out of 271k gold questions. for qanom, we aligned 8.3k out of 21.5k gold questions. we use the verbal srl parser in allennlp (gardner et al., 2018) which reimplements shi and lin (2019)'s srl parser. for nominal predicates, we re-train the same model on nombank (meyers et al., 2004), achieving 81.4 conll-f1 score on the development set.