Open-domain clarification question generation without question examples (2021)

Abstract: An overarching goal of natural language processing is to enable machines to communicate seamlessly with humans. However, natural language can be ambiguous or unclear. In cases of uncertainty, humans engage in an interactive process known as repair: asking questions and seeking clarification until their uncertainty is resolved. We propose a framework for building a visually grounded question-asking model capable of producing polar (yes-no) clarification questions to resolve misunderstandings in dialogue. Our model uses an expected information gain objective to derive informative questions from an off-the-shelf image captioner without requiring any supervised question-answer data. We demonstrate our model’s ability to pose questions that improve communicative success in a goal-oriented 20 questions game with synthetic and human answerers.

## introduction human-machine interaction relies on accurate transfer of knowledge from users. however, natural language input can be ambiguous or unclear, giving rise to uncertainty. a fundamental aspect of human communication is collaborative grounding, or seeking and providing incremental evidence of mutual understanding through dialog. specifically, humans can correct for uncertainty through cooperative repair @xcite @xcite which involves interactively asking questions and seeking clarification. making and recovering from mistakes collaboratively through question-asking is a key ingredient in grounding meaning and therefore an important feature in dialog systems @xcite . in this work, we focus on the computational challenge of generating clarification questions in visually grounded human-machine interactions. one popular approach is to train an end-to-end model to map visual and linguistic inputs directly the interaction ends with the model returning a guess for which image the answerer is referring to. to questions @xcite . this approach is heavily data-driven, requiring large annotated training sets of questions under different goals and contexts. another approach has drawn from work on active learning and optimal experiment design (oed) in cognitive science to search for questions that are likely to maximize expected information gain from an imagined answerer @xcite @xcite @xcite . much of this work has relied on large-scale question-answer datasets @xcite for training or retrieval to propose candidate questions or evaluate their expected utility. others, like @xcite , derive questions from attribute annotations for domain-specific systems. in this paper, we address an open-domain setting where one cannot rely on an immediate grounding of the meaning of questions in the target domain (in contrast to end-to-end approaches, which assume examples of questions to train on, or semantic parsing approaches, which assume a logical form for questions). our key contribution is a lightweight method to ground question semantics in the open image domain without observing question examples. instead, our framework builds a visually grounded question-asking model from image captioning data, deriving question selection and belief updating without existing semantics. our model generates candidate polar questions, arguably the most common form of clarification in dialogue @xcite , by applying rule-based linguistic transformations to the outputs of a pretrained image captioner. we then use self-supervision to train a response model that predicts the likelihood of different answers. given these predictions, we estimate the expected information gain of each question and select the question with the highest utility. we demonstrate our method's ability to pose questions that improve communicative success in a questiondriven communication game with synthetic and human answerers. ## 20 questions task we study interactions between questioners and answerers in a visually grounded 20-questions paradigm (see fig. 1 ). both agents are shown a set of k images as a context (@xmath0. 1 ). one of these images is privately indicated to the answerer as the target (e.g., bottom row, center), but remains unknown to the questioner. the questioner's goal is therefore to select questions that allow them to identify this target based on responses from the answerer. after a maximum of 20 questions, the questioner must make a guess (i.e., a kway classification). this task can be viewed as the most straightforward extension of a signaling game @xcite to allow for interactive clarification and repair. to approximate the setting of natural "clarification questions" we also consider games that begin with a description of the target. critically, the appropriate question changes depending on the context of objects and previous information provided by the answerer. ## model our model (fig. 2 ) maintains a belief distribution, p(y|x t ), about which image y in the set of images y is the target. this distribution is conditioned on the history of the interaction, x @xmath1, q 1 , ..., a t , q t ), which includes all questions, q, and answers, a, exchanged up to the current step, t. our model is defined in terms of three basic components. at each interaction step, it must generate a set of candidate questions, select one of these candidates based on expected information gain and finally update its beliefs based on the answer. question generator. to generate questions without question examples, we must derive suitable candidates using an alternative method. specifically, we suggest using a pre-trained image captioner to produce a list of candidate captions, which can then be programmatically transformed into question form. we begin by producing a list of captions for each image @xmath2, obtained using the berkeley neural parser @xcite . we then transform each noun phrase (np) subtree in each caption's constituency tree into a polar question ('are @xmath3. using this procedure, we generate an average of 10 candidate ## experiments we evaluated our question-asking framework in grounded interactions with both synthetic and human answerers. ## conclusions we introduce a question generation framework capable of producing open-domain clarification questions. instead of relying on specialized questionanswer training data or pre-specified question meanings, our model uses a pretrained image captioner in conjunction with expected information gain to produce informative questions for unseen images. we demonstrate the effectiveness of this method in a question-driven communication game with synthetic and human answerers. we found it important to generate questions varying in specificity by decomposing captioner utterances into component noun phrases. having generated this set of potential questions, selecting based on estimated information gain yielded useful questions. without seeing question examples, our framework demonstrates a capacity for generating effective clarification questions. future research should aim to generate more diverse question sets, allow for more expressive answers, and address abstract properties of objects within images. one approach, as demonstrated by our preliminary work with 'what'-questions, would be to extend our framework to incorporate additional types of wh-questions. integrating this clarification capacity more fully into collaborative, goal-directed dialog agents will allow them to engage in cooperative repair.