Toward the Automatic Detection of Word Meaning Negotiation Indicators in Conversation (2025)

Abstract: Word Meaning Negotiations (WMN) are sequences in conversation where speakers collectively discuss and shape word meaning. These exchanges can provide insight into conversational dynamics and word-related misunderstandings, but they are hard to find in corpora. In order to facilitate data collection and speed up the WMN annotation process, we introduce the task of detecting WMN indicators – utterances where a speaker signals the need to clarify or challenge word meaning. We train a wide range of models and reveal the difficulty of the task. Our models have better precision than previous regular-expression based approaches and show some generalization abilities, but have moderate recall. However, this constitutes a promising first step toward an iterative process for obtaining more data.

## introduction miscommunication is an inherent part of linguistic interaction. whether it stems from ambiguity, production errors, or differences in speaker knowledge, it provides a window into how language is processed and into the strategies used by speakers to resolve such situations @xcite . in this paper, we are interested in a specific type of miscommunication involving word meaning. these are cases where a speaker expresses the need to clarify the meaning of a word or phrase used previously in the conversation, engaging in a metalinguistic discussion of word meaning (see the example in figure 1 ). these interactions, known as word meaning negotiations @xcite or wmn, offer a lens on word-related misunderstandings and misalignment and can complete and complement the study of repair mechanisms in dialog @xcite , from signaling to resolution strategies. they can also support the detection of unclear, ambiguous or controversial word usages @xcite ; as well as contribute to the development of dialogue systems that can detect, avoid and resolve such lexico-semantic divergences as humans do. identifying and annotating wmn sequences in corpora, however, is challenging and costly in practice: existing annotated data is scarce, wmns are hard to find, and regular expression-based methodologies used up to now @xcite have low precision and require extensive manual filtering of irrelevant instances. furthermore, the form of wmn interactions can differ substantially depending on the conversational modality (spoken or written), speaker relationship and familiarity as well as the communicative goal. therefore, in order to study wmns in their full complexity and diversity, it is crucial to collect a large number of wmn examples from a variety of conversational situations. this represents an additional challenge, as conversational corpora can differ substantially both in format and in register. our goal is to optimize the data collection process for wmn annotation, increasing the proportion of relevant retrieved examples (precision) without compromising their variety (recall). this can improve the efficiency of manual annotation, increasing the speed at which relevant instances are found and decreasing human effort. to do so, we use the only existing dataset with wmn annotations to date to train and evaluate multiple models in a large number of settings on the task of identifying the part of a wmn that features the least variation: the indicator (in red in figure 1 ). the indicator of a wmn is a (part of) an utterance that questions or challenges word meaning. our evaluation and analysis highlight the difficulty of the task and the complementarity of our models with regular expression-based approaches. while our models have better precision and some incipient generalization abilities beyond previously considered patterns, we also identify and discuss several limitations. our main contributions are as follows: 1) we introduce the task of wmn indicator detection; 2) we benchmark supervised models and large language models (llms) on this task for english; 3) we conduct a thorough analysis, comparing our best models to existing regular expression-based approaches and manually examining their predictions on new data; and 4) we identify key challenges and promising directions of improvement. ## task and data we propose to address the task of wmn indicator detection, which we frame as a binary classification problem where a model needs to learn whether the input text is signaling the need to discuss or clarify a word's meaning. indicators are the most easily identifiable part of a wmn, as triggers and negotiation exhibit large variation. the dataset used in our experiments, the indicators dataset, consists of utterances extracted from newme. ## experimental setup in this section we describe how input is presented to the models ( §4.1); how we train supervised models ( §4.2) and prompt llms ( §4.3) for the task; and we present a regex-based baseline ( §4.4). ## results and analysis in this section we present the results obtained by all models on the indicators dataset (section 5.1) and compare our method with the original collection method based on regexes (section 5.2). ## model behavior on new data we now explore our models' ability to generalize beyond the data retrieved by regexes. specifically, we ask whether they can correctly identify and uncover indicators that are expressed in ways that were not previously considered by the previous regex-based methodology. 13 we use the bestperforming model for each corpus. 14 ## conclusion and future work we have introduced the task of word meaning negotiation indicator detection and presented a first exploration using supervised models and few-shot prompting with llms. our results show that this is a challenging problem which current open-source llms do not manage to solve, but for which supervised models show promise. our best models achieve a higher precision than the previous regexbased approach, but their limited recall and our closer look at specific regexes suggest that a hybrid method combining the strengths of both approaches could offer a better compromise between efficiency and coverage. our manual evaluation on challenging samples highlights that while the models have learned useful patterns and have some generalization abilities (i.e., they are able to correctly detect a few new indicators that were missed by the previous regex methodology), more data is needed; with special attention on the treatment of short sentences. this makes the task well-suited for an active learning approach @xcite , where the most informative instances would be used to help the model better learn the task with as little additional annotation as possible. more work and data is also needed to better incorporate conversational context to the models, and to explore their domain adaptation capability across corpora and dialogue settings, which can be challenging given the observed difference in results between corpora. in the longer term, future studies could explore jointly addressing indicator and trigger detection in a multi-task setting, as well as collecting more data and annotations from other modalities that also express communication issues (e.g., speech and gestures) to train multi-modal models. this study lays a foundation toward the computational modeling and semi-automatic annotation of this interesting linguistic phenomenon. improving its detection can support both discourse and linguistic analysis as well as conversational system development. we share our code to promote further research. 16