Repairs in a Block World: A New Benchmark for Handling User Corrections with Multi-Modal Language Models (2024)

Abstract: In dialogue, the addressee may initially misunderstand the speaker and respond erroneously, often prompting the speaker to correct the misunderstanding in the next turn with a Third Position Repair (TPR). The ability to process and respond appropriately to such repair sequences is thus crucial in conversational AI systems. In this paper, we first collect, analyse, and publicly release BlockWorld-Repairs: a dataset of multi-modal TPR sequences in an instruction-following manipulation task that is, by design, rife with referential ambiguity. We employ this dataset to evaluate several state-of-the-art Vision and Language Models (VLM) across multiple settings, focusing on their capability to process and accurately respond to TPRs and thus recover from miscommunication. We find that, compared to humans, all models significantly underperform in this task. We then show that VLMs can benefit from specialised losses targeting relevant tokens during fine-tuning, achieving better performance and generalising better to new scenarios. Our results suggest that these models are not yet ready to be deployed in multi-modal collaborative settings where repairs are common, and highlight the need to design training regimes and objectives that facilitate learning from interaction. Our code and data are available at www.github.com/JChiyah/blockworld-repairs

## introduction unlike its formulation in much of the literature within nlp (see especially @xcite , natural language understanding (nlu) is not a unilateral, passive process, but an (inter)active one (see @xcite , for expansive discussion): in everyday conversation, people continuously work together to negotiate shared understanding and coordination in order to move the conversation forward @xcite @xcite @xcite figure 1 : example dialogue from blockworld-repairs: after predicting an incorrect response, vlms must accurately interpret the repair to produce the correct bounding box prediction, a critical skill for humanrobot collaboration tasks. @xcite ). one of the key interactional processes that enables this is called repair @xcite ) -see figure 1 : a set of universal and highly systematised corrective feedback mechanisms for dealing with miscommunication when it arises in conversation @xcite . therefore, the ability to interpret and generate effective repair sequences is crucial to robust and faithful conversational ai technology. this need is especially acute in settings where more finegrained levels of understanding are required, for example in embodied human-machine collaboration. indeed the frequency of miscommunication in human dialogue is known to vary with both the overarching task and the medium of communication @xcite . in this paper, we focus on a particular class of corrective feedback, so called, third position repairs (henceforth tpr; @xcite in multi-modal settings. these occur when the addressee initially misunderstands the speaker (see figure 1 at t1, the trouble source turn), responds based on this misunderstanding (at t2), which in turn reveals the misunderstanding to the addressee who then goes on to correct the misunderstanding (at t3). the addressee then provides a new response (at t4) to the original request (at t1) based on the correction. tprs have been largely neglected in the nlp community, likely due to underestimation of the importance of repair phenomena in dialogue, and thereby lack of appropriate data and frameworks for model training and evaluation. repair is of particular interest in (multi-modal) language modelling since it almost invariably involves highly focused / specific revisions to the representations already built by the model (e.g. revisions to the referents of referring expressions, like in our case here), based on the context-dependent structure of the corrective feedback. repairs can thus be used as probes into how models represent meaning and the structure of the overarching task (see @xcite ; @xcite ; @xcite for different implementations of this idea). and as we show here, this is also consequential for how language models should be trained to process tprs including the specific objective functions involved. our contributions are thus as follows: 1) we collect and release blockworld-repairs (bw-r), a dataset of collaborative dialogues in a tabletop manipulation task that, by design, has a high potential for referential ambiguity and is focused on complex multi-modal task instructions as well as tprs; 2) we establish a human baseline for the whole task through an in-person human study where participants try to solve the task using these dialogues; 3) we show that state-of-the-art vision language models (vlm) can learn to process tpr sequences through a specialised training regime, with some opportunity for generalisation; and 4) considering the substantial performance gap between the best models and humans, we present an in-depth error analysis comparing the two, revealing that models struggle with references that humans find easy. computational models of repair considerable attention has been paid to computational models for the interpretation and generation of self-repair (see @xcite @xcite @xcite : a class of repairs whereby the speaker corrects themselves on the fly within the same conversational turn (e.g. "user: i want to go to london uhm sorry paris"). similarly, the crucial role of generating and responding to crs (e.g. "pardon/what/who?") in conversational models has long been recognised (see @xcite @xcite @xcite , but existing systems either remain limited (e.g. @xcite ) or do not support this at all -see @xcite for an overview. recent work tries to identify when to pose a cr @xcite @xcite @xcite , but few evaluate the ability of models to process their responses @xcite . @xcite show that the ability of vision and language models (vlm) to effectively process clarificational exchanges depends on the level of granularity of the model's cross-modal representation, and therefore also the specific objectives used to train the models. our work here is closest in spirit to that of @xcite who provide the first large dataset of third position repairs, and use it to evaluate llms. however, they focus only on unimodal repairs in the context of conversational question answering, and do not perform any fine-tuning of the models they evaluate, like we do here. vlms for situated collaborative tasks thanks to recent advances in the development of llms, there have been many attempts to derive vision+language models that use pretrained visual encoders to solve vision-language tasks (e.g., @xcite , inter alia). it is important to note that most of these models are trained to follow instructions that can be specified in a single turn. many tasks can be included in this category such as visual question answering @xcite , image captioning @xcite , and referential expression resolution @xcite . to solve these tasks, models are trained to maximise the likelihood of the responses in the training data using supervised learning methods or using reinforcement learning from human feedback @xcite , ignoring the fact that a conversation is a process that unfolds over multiple turns where each of which matters to make correct decisions. for instance, in tasks such as visual dialogue @xcite , it is well-known that a very limited number of responses are dependent on the dialogue history @xcite , making them not a suitable benchmark for truly collaborative tasks requiring the ability to establish common ground. as highlighted by @xcite , many embodied ai benchmarks (e.g., alfred @xcite , simbot arena @xcite , etc.) also have similar issues considering that the level of ambiguity is reduced to the minimum, minimising the need for any form of correction. for this reason, bw-r represents the first benchmark that is aimed at assessing the ability of current vlms to resolve tprs-an important capability which is essential for establishing common ground. ## the blockworld-repair dataset we build on top of the block world dataset by @xcite , an instruction-following task where a robot manipulator has to move blocks on a virtual board. humans provided instructions in natural language referring to a block to pick up and a new location to drop it with complex visual and spatial descriptions (i.e., "the block on the top-right corner moves behind the middle-top block"). prior works have struggled to handle the most ambiguous setting with blank blocks @xcite @xcite . however, here we argue that singleturn instructions are insufficient to solve highly ambiguous environments or real-world situations. even the most detailed referring expressions in the block world may fail to uniquely identify a referent (see figure 1 ), which repairs could alleviate by narrowing down the candidate pool and introducing new information to identify the correct referent (t3 in figure 1 ). similar to how humans misinterpret and subsequently repair referential ambiguities with tprs, vlms must also be capable of handling multi-modal tprs in dialogues, as they are essential for mutual coordination. differences from the original block world the original dataset consists of pairs of images and single-turn instructions, requiring agents to resolve inherently ambiguous commands. however, it does not account for dialogue or clarificational exchanges as strategies to overcome environmental ambiguity, which humans commonly use to repair miscommunications. to address this, we extend the dataset by incorporating brief dialogues where humans interact with the agent and produce third position repairs. we preserve the original block world instructions and test set while adding our dataset as an extension to explore how humans resolve ambiguity through clarification and repair strategies. ## experiments this section evaluates the ability of vlms to process instructions and repairs in an instructionfollowing task within a situated context. the repair turns in the bw-r are meaningful only when interpreted alongside the robot's pointing position at the time of the repair, based on the first instruction; thus, they are incomplete on their own. consequently, we have dialogue triplets that are intrinsically connected and can only be comprehended as a whole: the initial instruction, the incorrect candidate prediction, and the repair. we argue that more general models must be able to process both initial instructions and any subsequent repairs. ## learning to process tprs during training, models encountered two types of entries: single-turn instructions or dialogues up to the tpr. in human conversations, we can effortlessly manage both scenarios; therefore it is reasonable to expect that models trained to predict bounding boxes from a single instruction should also be capable of handling repairs to their predictions. however, our previous results indicate that achieving both capabilities simultaneously is challenging, and that generalization incurs a cost, even when models address the same task. to handle repairs, models cannot rely only on the last turn but must consider the context from the initial user instruction and the following candidate output. the intermediate predicted bounding box, even if inaccurate, also provides crucial information within the context of the tpr, as repairs rarely repeat a fully-formed instruction (i.e., "the right one!"). in the case of bw-r, repairs are relative to where the robot is currently pointing and often provide partial information to correct the robot's action. during fine-tuning, following common approaches in vlm training @xcite , vlms calculate the cross-entropy loss for all the tokens in the input including both the intermediate (wrong) and last (correct) generation. thus, the models from §4.2 are learning from incorrect tokens, and therefore these tokens are somehow affecting the quality of the vlms generations. ## error analysis we want models that can collaborate with humans even when misunderstandings arise, and thus we need to understand the differences in how models and humans solve the same visual-grounding task. this section attempts to analyse models beyond performance metrics and provides insight into the models' behaviour, such as where mismatches arise compared to how humans process tprs. we use the best models from §5 trained on the whole data using completions-only loss. ## conclusion this paper explores vlm's capabilities of handling repairs with a new dataset, blockworld-repairs. we collect 795 collaborative dialogues in a highly ambiguous tabletop manipulation environment that requires complex multi-modal task instructions as well as repairs. in these scenarios, referential ambiguities are common and thus being able to process tprs becomes essential to complete the task. we validate these dialogues through an in-person human study where participants attempt to solve the task, providing a baseline for model performance. we then show that vlms struggle to process tprs alongside other dialogues out of the box and propose improving their fine-tuning regime with different loss criteria. our results indicate that vlms benefit from partially masking the input tokens when learning to process tprs along with the task, resulting in more generalisable models across single-turn instructions and tprs for one of the tasks. however, this needs to be balanced as masking too many tokens hurts models when data is smaller due to a lack of training signal. we finish with an error analysis comparing these models and human participants, finding stark differences in how they process repairs. surprisingly, models struggle with the dialogues that human participants found easiest, highlighting a large gap in their ability to process tprs, despite these being fundamental to effective dialogue coordination. we believe that future research should focus on two major improvements for vlms: 1) design training objectives that facilitate learning from interaction, including tprs; 2) equip models with more objectcentric visual representations that facilitate visual grounding tasks @xcite which are of paramount importance for situated collaborative tasks.