It Depends: Resolving Referential Ambiguity in Minimal Contexts with Commonsense Knowledge (2025)

Abstract: Ambiguous words or underspecified references require interlocutors to resolve them, often by relying on shared context and commonsense knowledge. Therefore, we systematically investigate whether Large Language Models (LLMs) can leverage commonsense to resolve referential ambiguity in multi-turn conversations and analyze their behavior when ambiguity persists. Further, we study how requests for simplified language affect this capacity. Using a novel multilingual evaluation dataset, we test DeepSeek v3, GPT-4o, Qwen3-32B, GPT-4o-mini, and Llama-3.1-8B via LLM-as-Judge and human annotations. Our findings indicate that current LLMs struggle to resolve ambiguity effectively: they tend to commit to a single interpretation or cover all possible references, rather than hedging or seeking clarification. This limitation becomes more pronounced under simplification prompts, which drastically reduce the use of commonsense reasoning and diverse response strategies. Fine-tuning Llama-3.1-8B with Direct Preference Optimization substantially improves ambiguity resolution across all request types. These results underscore the need for advanced fine-tuning to improve LLMs’ handling of ambiguity and to ensure robust performance across diverse communication styles.

## introduction natural language is inherently ambiguous. for example, pronouns may refer to multiple possible entities within a sentence. nevertheless, humans typically resolve such ambiguity by drawing on context, shared knowledge, and conversational history @xcite . consider the two conversations shown in figure 1 , where the user asks the question, "why can it fly?". without additional clues, the pronoun "it" is unclear and could refer to multiple entities. in the left conversation, the prior context mentions a helicopter and a drum; in provide me one sentence for each of the following: helicopter, owl, drum a helicopter is parked on the tarmac. an owl is a bird known for its distinctive call. a drum was placed on the table for the meeting. why can it fly? do you mean the helicopter, or the owl? what is the definition of 'bat'? the term "bat" can refer to several different things depending on the context: 1. animal: [..] 2. sports equipment: [..] 3. verb: [..] 4. slang: [..] if you have a specific context in mind, please let me know! what is the definition of 'bat' in simple language? a "bat" is a flying mammal that is active at night. it has wings made of skin stretched over its long fingers. bats are known for using echolocation to find food, which usually consists of insects. they are often found in caves or trees and play an important role in the ecosystem. ## methodology we evaluate whether llms can resolve referential ambiguity using common knowledge and how requests for simplified language affect this ability. each test instance consists of a short context passage introducing some entities (e.g., helicopter, owl, drum). the user then asks an ambiguous question referring to one of the entities without naming it directly (e.g., why can it fly?). for each instance, we define a set of positive entities as those for which the question makes sense, and negatives as those for which it does not (e.g., a drum cannot fly). we evaluate two setups: clearref, where one positive and one negative entity make the referent unambiguous with commonsense, and sharedref, where two positives and one negative leave ambiguity even with commonsense. this setup tests whether models consider multiple plausible candidates rather than relying on recency or default biases. we treat the pronoun "it" as equally applicable to all introduced positive entities. to assess the impact of recency, we perform an ablation in which the order of entities is permuted (see appendix d). ## discussion our results indicate that current models struggle to apply commonsense knowledge for ambiguity resolution. in the simpler clearref task, where only one entity fits the question, models are able to resolve the ambiguity with an accuracy ranging from 94.23% down to 21.15% depending on the model and setting. the more challenging share-dref task, which involves two plausible entities, sees direct responses ranging from just 36.56% down to 0.44%. this aligns with findings by @xcite . they observe that llms can retrieve commonsense facts, which in our case means realizing that an entity fits a relation when asked on its own. however, the models often fail to apply this knowledge when answering a specific question requiring such reasoning. in appendix c, we evaluate gpt-4o's performance in english under a chain-of-thought setting, prompting it of follow-up clarification questions, which require multi-turn interaction. moreover, annotators often favor verbose, catch-all answers for under-specified queries, even though such verbosity imposes cognitive costs on users @xcite . another important observation is that prompting models to use simpler language can harm response quality. interestingly, in clearref, there is no drop from normal to simple; in some models, simple responses are even slightly better. in contrast, for the more complex sharedref task, performance drops drastically in the simple setting. this confirms prior work showing that simplification often leads to omissions and vague phrasing @xcite @xcite @xcite . we argue that this behavior needs to change. for example, @xcite show that llms adapt the information they provide based on assumptions about the user. this is problematic, especially if requesting simple language causes models to produce less thoughtful responses or overlook important distinctions. again, rlhf may play a role, failing to capture the needs of diverse users and discouraging clarification and hedging in simplified contexts. taken together, we argue that resolving ambiguity requires a balance: infer as much as possible to avoid unnecessary elaboration, but clarify when uncertainty remains. our dpo-trained model moves in this direction. it not only improves on our main evaluation but also generalizes to the lexical ambiguity benchmark of @xcite . moreover, it reduces the performance drop commonly observed when models operate in simplified language settings. this suggests that clarification and hedging behaviors can be learned in a transferable and robust way. ## conclusion in this paper, we analyzed how llms handle textual referential ambiguity and to what extent they apply commonsense knowledge to resolve it. our findings show that llms have limited ability to do so effectively. they tend to commit to a single interpretation or cover all possible references, rather than hedging or seeking clarification. this tendency becomes even more pronounced when users request simple language, which reduces commonsense reasoning and different answering strategies. these results point to two core issues. first, there is a need for better fine-tuning to improve how llms deal with ambiguity. second, llms should better adapt to different user needs. it is especially concerning that a request for simpler language leads to less thoughtful responses and fewer clarifications, showing that current systems often fail to support users with varied communication styles. to support reproducibility and future research, we release our code foot_1 . further links to models and datasets are provided in the repository.