Asking the Right Question at the Right Time: Human and Model Uncertainty Guidance to Ask Clarification Questions (2024)

Abstract: Clarification questions are an essential dialogue tool to signal misunderstanding, ambiguities, and under-specification in language use. While humans are able to resolve uncertainty by asking questions since childhood, modern dialogue systems struggle to generate effective questions. To make progress in this direction, in this work we take a collaborative dialogue task as a testbed and study how model uncertainty relates to human uncertainty—an as yet under-explored problem. We show that model uncertainty does not mirror human clarification-seeking behavior, which suggests that using human clarification questions as supervision for deciding when to ask may not be the most effective way to resolve model uncertainty. To address this issue, we propose an approach to generating clarification questions based on model uncertainty estimation, compare it to several alternatives, and show that it leads to significant improvements in terms of task success. Our findings highlight the importance of equipping dialogue systems with the ability to assess their own uncertainty and exploit in interaction.

## introduction the ability to ask effective and informative questions is a peculiar feature of human intelligence. from an evolutionary perspective, it has been argued that being able to ask questions to reliable informants gives humans a key evolutionary advantage over other species @xcite . questions are also shown to play a fundamental role in cognitive development and language learning during childhood @xcite @xcite @xcite . both children and adults generally ask questions to resolve a state of uncertainty @xcite . deciding how to resolve uncertainties and maintaining mutual understanding are essential steps to achieve conversational grounding @xcite , the process through which humans negotiate the production and acceptance of their utterances. clarification questions are among the key strategies speakers use to achieve conversational grounding. however, while humans handle and resolve uncertainties through clarification questions almost effortlessly in everyday conversations, this is particularly challenging for modern dialogue systems based on deep neural networks @xcite . even the new generation of conversational agents, such as chatgpt, foot_0 seem to struggle to generate clarification questions when dealing with ambiguous instructions @xcite . in this paper, we investigate uncertainty-driven strategies for asking clarification questions. clarification requests are usually triggered when ambiguity and/or underspecification make it difficult for the receiver to follow up with enough confidence @xcite @xcite . it should come as no surprise then that there is a long-standing tradition of work highlighting the importance of incorporating clarification requests in dialogue systems @xcite @xcite @xcite . recent years have witnessed a renewed interest in this ambitious goal, especially in the context of collaborative dialogue tasks (e.g., @xcite @xcite @xcite ). yet, several key aspects remain under-explored. for example, some existing approaches use a dedicated neural network model trained on human clarification behaviour to decide when to ask @xcite . however, it is well known that humans may follow different confirmation-seeking strategies @xcite @xcite figure 1 : overview of our experimental setup in codraw. after receiving an instruction from the teller, the drawer agent selects the clipart(s) to draw, together with their attributes. if the entropy over an attribute exceeds a threshold θ (size in the figure), then the drawer asks a clarification question. the question-answer pair is added to the dialogue history before performing the next drawing action. different human players may react differently (bottom-left box); the agent decides whether to ask for clarification on the basis of its own uncertainty, independently from the clarification decisions of human players. we consider the codraw task and dataset @xcite -where a drawer player is asked to recreate a clipart scene following instructions from a teller player-as a testbed to study uncertainty-driven clarification strategies in collaborative agents. we propose different ways to represent model uncertainty in codraw, and then carry out a study on the relation between model uncertainty and human uncertainty, as measured by the decision to ask clarification requests in humanhuman dialogues. this study reveals that there is a poor relation between model and human uncertainty, which in part appears to stem from our finding that in codraw human players do not follow a consistent clarification strategy. this poor relation suggests that using human clarification decisions as supervision for deciding when to ask may not be the most effective way to resolve model uncertainty. we therefore propose a new drawer agent model equipped with an uncertainty-based clarification module (illustrated in figure 1 ), where we use a template-based approach to generate clarification questions triggered by the uncertainty state of the agent. this approach allows us to modulate the level of uncertainty that leads to posing a question and thus control for the number of questions asked per dialogue, which may be advantageous given the processing cost of questions for humans as well as models @xcite . our results and analyses show that our proposed model is significantly more effective in terms of task success than several strong baselines, including a version based on the human supervision model by @xcite . overall, our findings highlight the importance of equipping conversational agents with the ability to assess and act upon their own uncertainty by asking effective clarification questions. ## related work clarification requests have long been studied as a key repair mechanism in human dialogue @xcite @xcite @xcite @xcite . datasets of human conversations where spontaneous clarification questions are annotated have been an important resource for data-driven research on clarification strategies. for example, @xcite proposed an annotation scheme based on clark's four levels of communication @xcite to identify the form and function of clarification requests. these and similar annotation schemes have been applied to small datasets @xcite @xcite . at a larger scale, the teach dataset (padmakumar et al., 2022) has been annotated by @xcite with dialogue acts, including acts related to clarification, and the minecraft dialogue corpus (narayan-chen et al., 2019) and its extensions @xcite @xcite have been annotated with different types of clarification requests. however, in the latter case the dataset was augmented with further clarification questions a-posteriori, introduced asynchronously by other annotators who were asked to decide whether clarifications were needed. @xcite and @xcite adopt similar techniques for including clarification questions in dialogue data. recently, madureira and schlangen (2023b,a) released codraw-icr (v1 and v2), a fine-grained annotation of spontaneous clarification requests in the codraw dataset @xcite . this constitutes a worthwhile resource for analysing clarification behaviour in collaborative tasks. @xcite exploit these annotations to train a neural model for deciding when to clarify. in contrast, we shall use them to study the relation between human and model uncertainty in section 5 and then, in section 6, propose a model for the co-draw drawer agent that asks clarification questions on the basis of its own uncertainty. model uncertainty represents a valuable signal for natural language generation models to deal with the different sources of ambiguity in language use @xcite . earlier on, when the speech recogniser used to be a common source of errors in spoken dialogue systems, asr confidence scores were exploited to decide on error handling strategies such as confirmation and clarification @xcite . in more recent systems, where the focus is on the next action to take in a collaborative task, a common approach consists of using model uncertainty (as measured by the entropy over the model pre-diction on a set of actions) to signal ambiguity and trigger the generation of clarification questions @xcite @xcite . without explicitly estimating uncertainty, @xcite propose a beam search re-ranking algorithm guided by the model intermediate predictions about the target in a referential task. similar approaches use the probability difference between the top two predicted actions to estimate uncertainty @xcite . recently, @xcite investigated the alignment between predictive uncertainty and ambiguous instructions in visually-grounded communication tasks. they show that well-calibrated prediction probabilities benefit the detection of ambiguous instructions. differently from our work, the authors do not investigate the effectiveness of model uncertainty to generate follow-up clarification questions. instead of relying on model uncertainty, other proposed methods involve training external neural networks or sub-modules with the objective of deciding when to ask for clarification @xcite . @xcite propose to incorporate cognitive modeling in a reinforcement learning framework to generate context-sensitive clarification strategies in a referential game. while a few approaches generate clarification questions from scratch in an end-to-end fashion @xcite , heuristics are often applied at the generation stage @xcite @xcite . in this paper, we propose an approach to generating clarification questions based on model uncertainty estimation and compare it to alternative methods for deciding when to clarify by training on human clarification decisions. ## task & dataset codraw @xcite is a goal-driven instruction-following collaborative task between two players. the teller player has access to an abstract target scene containing a variable number of cliparts (on average, target scenes contain 6 cliparts), leveraging the abstract scene dataset @xcite . the teller provides written instructions in english to the drawer in order to reconstruct the scene in a turn-based fashion. the drawer has access to a gallery of 58 cliparts that they can place on a canvas. the drawer can remove, resize (small, medium, large), or flip (facing right or left) each object in the scene. after receiving an instruction, the drawer has to reply to the teller before proceeding to the following turn. typically, the drawer either replies with confirmation feedback or asks follow-up questions. the codraw dataset consists of around 10k dialogues, with an average length of 7.7 turns per dialogue. additional details can be found in appendix d. in codraw, task success is measured by comparing the target image with the one drawn by the drawer at the end of the conversation. to this aim, @xcite designed a similarity score metric, ranging from 0 to 5, tailored to the codraw task. this score takes into account different cliparts (by checking whether cliparts in the target scene appear in the reconstructed one) and their attributes (size, position, flipping, plus some additional attributes for cliparts representing people, i.e., facial expression and body pose) by assigning different weights to each component. the scenes drawn by human players reach a similarity score of 4.17 (@xmath0.64). in our work, we apply some changes to the similarity score metric to fix some flaws in its design and better capture the properties under analysis in the experimental setup, as described in appendix a. in the following, we report the results using the new version of the metric. ## model vs. human uncertainty in this section, we investigate the relation between model and human uncertainty. in the absence of clues such as eye gaze or reaction times (which could be indicative of uncertainty, but are not available in codraw), we take clarification questions as a signal of human uncertainty. we exploit the annotation of clarification questions released by @xcite , which consists of turns by the drawer player annotated as clarification requests plus the attributes mentioned in such requests (size, position, orientation, etc.). 2 around 40% of the dialogues in codraw contain at least one clarification question, with an average number of 2.2 clarification requests per dialogue in this subset. ## asking clarification questions we implement a questioning drawer model (qdrawer) by adapting the silent drawer described in section 4.1. we focus on the size attribute, because it applies to all cliparts and it is often un-t: in the bottom left of the image is a large pointed tree , half of it is cropped , it extends off the top of the frame qd: ok. t: in the visible portion of the tree is a bear , his legs are on the grass but entire body is in front of the tree . he faces right qd: what size is the bear? t: the bear has a witch's hat qd: what size is the hat? t: the hat is big. t: the bear is big. qd: ok. derspecified in the teller's instructions (see section 5). our goal is to test whether asking clarification questions triggered by size uncertainty results in (1) higher accuracy regarding the size attribute, and (2) higher overall similarity score. to isolate the effect of asking questions and avoid confounds related to language generation, we use a templatebased approach to generate clarification questions and their answers. ## conclusions different kinds of uncertainties characterize language use, and asking informative and effective questions is a crucial tool to address these uncertainties. while humans are able to ask strategic questions since early childhood to resolve a state of uncertainty, equipping modern dialogue systems with the ability to ask effective questions has proved challenging. some existing approaches propose employing dedicated neural network models trained on human behaviour to decide when to ask for clarification in a dialogue. however, the high variability in human clarification strategies casts doubt on the effectiveness of this approach. other proposed methods involve using model uncertainty to decide when and/or what to ask. yet, estimating and exploiting model uncertainty is not trivial, and the relationship between model and human uncertainty is still unclear. in our work, we considered the codraw collaborative dialogue task as a test bed. we proposed different ways of extracting model uncertainty and investigated its relationship to human uncertainty, as measured by the clarification decisions of human players. we found that human and model uncertainty are poorly correlated, possibly due to the wide diversity of human clarification strategies. we then presented an approach that makes use of model uncertainty to generate clarification questions, and compared it to other methods for deciding when clarification is needed. our approach allows us to vary the level of uncertainty that elicits the generation of clarification questions to control for the number of questions generated. the results of our experiments show that our proposal outperforms a number of baselines and other methods trained on human clarification decisions. more generally, our findings pave the way for further research on integrating model uncertainty into natural language generation approaches as an effective way to resolve ambiguities and address underspecification in language use @xcite .