Can Vision-Language Models Infer Speaker’s Ignorance? The Role of Visual and Linguistic Cues (2025)

Abstract: This study investigates whether vision-language models (VLMs) can perform pragmatic inference, focusing on ignorance implicatures, utterances that imply the speaker’s lack of precise knowledge. To test this, we systematically manipulated contextual cues: the visually depicted situation (visual cue) and QUD-based linguistic prompts (linguistic cue). When only visual cues were provided, three state-of-the-art VLMs (GPT-4o, Gemini 1.5 Pro, and Claude 3.5 sonnet) produced interpretations largely based on the lexical meaning of the modified numerals. When linguistic cues were added to enhance contextual informativeness, Claude exhibited more human-like inference by integrating both types of contextual cues. In contrast, GPT and Gemini favored precise, literal interpretations. Although the influence of contextual cues increased, they treated each contextual cue independently and aligned them with semantic features rather than engaging in context-driven reasoning. These findings suggest that although the models differ in how they handle contextual cues, Claude’s ability to combine multiple cues may signal emerging pragmatic competence in multimodal models.

## introduction in recent years, many large language models (llms) have demonstrated the ability to solve a wide variety of tasks, contributing to their growing popularity. initially limited to text-based inputs, these models have been extended to incorporate visual inputs, paving the way for vision-language models (vlms). by bridging vision and language modalities, vlms have expanded the possibilities for ai applications and become central to the ongoing technological revolution @xcite @xcite . vlms have enabled various multimodal applications, such as object recognition @xcite @xcite , caption generation @xcite @xcite , and visual question answering @xcite . these tasks primarily focus on associations between visual and textual inputs by identifying objects, describing scenes, or responding to straightforward queries. while such capabilities are remarkable, they represent only the surface level of human-like understanding. in fact, real-world communication often requires reasoning about implicit meanings that emerge from the interplay between language and visual information @xcite . to move toward more humanlike multimodal intelligence, vlms must also be able to engage in this type of context-sensitive and inferential processing (see @xcite . this raises critical questions about vlms' capacity for context-sensitive reasoning, which underlies the pragmatic reasoning abilities required for realworld communication. pragmatics offers an ideal framework for investigating this question. in human communication, pragmatic inference plays a crucial role in understanding intended meanings beyond literal statement @xcite @xcite . contextual cues often provide disambiguating information that influences the interpretation of utterances, making pragmatic reasoning inherently multimodal @xcite @xcite . while a few studies on pragmatic reasoning have been explored in text-only llms @xcite @xcite @xcite , the visual modality enriches meaning construction through interaction with linguistic input. as visual context provides rich, implicit information that influences language interpretation, studying pragmatic phenomena through vlms presents an intriguing research opportunity. however, how well vlms can leverage visual information for pragmatic inference remains largely unexplored. therefore, we investigate whether vlms exhibit sensitivity to context, particularly focusing on ignorance implicatures-a pragmatic phenomenon in which a speaker's utterance implies a lack of precise knowledge, and whether this sensitivity can be modulated by a single cue or by the combination of multiple cues. by examining how vlms handle this phenomenon in comparison to human reasoning, we aim to better understand their strengths and limitations in processing context-dependent pragmatic meaning. ## ignorance implicatures to better understand pragmatic reasoning in realworld language use, we examine the phenomenon of ignorance implicatures. consider the examples in (1). four students passed the exam. b. (superlative modified numeral) at least four students passed the exam. c. (comparative modified numeral) more than three students passed the exam. when it comes to how many students passed the exam, the statement (1a) triggers 'exactly four' interpretation, whereas (1b) and (1c) do not. both (1b) and (1c) contain modified numerals, suggesting that the speaker may not know the exact number of students who passed the exam. this is known as ignorance implicatures, where the speaker's choice of modifier implies a lack of precise knowledge. however, not all modifiers give rise to ignorance implicatures to the same extent. previous studies have shown that superlative modifiers like at least tend to trigger ignorance implicatures more consistently than comparative ones like more than @xcite @xcite @xcite . in this regard, the likelihood of ignorance inferences typically follows the hierarchy: superlative modified @xmath0. this observation has prompted researchers to explore how such inferences arise, leading to two main perspectives. one approach suggests that ignorance inference is dependent on the words or phrases themselves @xcite ; also see @xcite . @xcite , for example, argued that the semantics of superlative modifiers are inherently more complex. while more than n expresses a simple meaning 'larger than n', at least n can convey both 'possible that there is a set of n' and 'certain that there is no smaller set of n.' according to @xcite , when someone has basic knowledge of geometry, (2a) gives the impression that the speaker lacks precise information, as compared to ( 2b ). this attributes the ignorance implicatures to a semantic property specific to at least. under the pragmatic account, on the other hand, ignorance implicatures for both at least and more than have been primarily explained through gricean reasoning, particularly the maxim of quantity @xcite , which holds that the speaker's choice to provide a lower-bound statement, rather than a more informative exact number, suggests that the speaker lacks precise knowledge @xcite @xcite . more recent studies have expanded this account by emphasizing the role of contextual factors @xcite @xcite @xcite . in these studies, contextual cues, including question under discussion (qud), preceding discourse, or accompanying visual input, were manipulated to modulate the likelihood of implicature. for instance, westera and brasoveanu (2014) investigated how different types of modified numerals give rise to ignorance implicatures depending on contextual demands and processing cost. in their experiments, participants read short dialogues or utterances containing modified numerals and judged how confident the speaker seemed about the exact quantity, as well as how natural the utterance was. to manipulate the informativeness required by the discourse, the authors introduced different quds, such as a 'how many' condition (how many of the diamonds did you find under the bed?), which demanded precise answers, and a 'polar' condition (did you find {at most | less than} ten of the diamonds under the bed?), which did not require numerically specific responses, as they could be answered with a simple yes or no. the results no. image text situation modifier these findings raise the question of whether and how visual and linguistic contextual information can enhance the pragmatic reasoning abilities of vlms. to address this question, the present study examines whether vlms exhibit sensitivity to ignorance implicatures across multiple contextual cues. ## discussion this study aimed to investigate the influence of contextual cues in the interpretation of ignorance inference within vlms. in experiment 1, we investigated how visually depicted situation (precise and approximate) and different types of modifiers (bare, superlative, and comparative) influenced appropriateness ratings of image-text pairs. results revealed that appropriateness ratings consistently followed the order of @xmath1, regardless of situation types. this pattern suggests that the models primarily relied on the semantic features of modifiers rather than incorporating contextual information into their judgments. building upon these findings, experiment 2 introduced an additional contextual cue, qud, with two conditions (howmany and polar). this experiment aimed to determine whether multiple contextual cues would facilitate more sophisticated pragmatic inference. interestingly, when presented with the howmany qud, both gpt and gemini models gave higher appropriateness ratings to bare numerals, which aligned with expectations for precise situation, but not for approximate situation. in our analysis, these models showed greater improvement in providing precise information than in engaging in pragmatic reasoning. while the influence of modifiers remained, there was a modest increase in sensitivity to contextual cues. in contrast, claude demonstrated a more integrated approach to contextual reasoning, using both situation and qud simultaneously. this integration pattern suggests claude may be moving closer to human-like pragmatic reasoning, which typically involves holistic consideration of multiple contextual factors. this leads to the assumption that claude may have benefited from cue combination, where the presence of two contextual cues, rather than a single cue, led to pragmatic interpretation. this pattern resonates with parker (2019)'s cue combination scheme, which posits the processing benefit of retrieval cues for anaphora in memory is not merely additive but emerges nonlinearly when multiple cues are jointly available. extending this idea to our findings, contextual cue combination for ignorance inference in vlms may similarly follow the nonlinear cue combination method: one contextual cue alone may not significantly affect the context-sensitive reasoning, but the addition of the second cue increases the "cue weight" enough to reach the threshold for pragmatic interpretation. this threshold effect is visualized in figure 4 , which contrasts linear and nonlinear cue integration patterns as a function of contextual cue number. then, do gpt and gemini follow the linear cue combination? it would be insufficient to characterize these models' behavior merely as examples of linear cue combination. rather, the observed pattern suggests a difference in how these models represent and utilize contextual information. while claude appears to engage in combining two contextual cues into a unified pragmatic representation, gpt and gemini exhibit a pattern of local alignment, in which modifiers are evaluated separately with each cue, but the cues themselves remain structurally unbound. in this sense, their responses are not limited because they combine cues linearly, but because their internal processing architecture does not support contextual cue combination in the first place. consequently, their outputs reflect a tendency to prioritize informational precision over pragmatic reasoning. as more cues become available, the models tend to converge on more semantically determinate interpretations, aiming to reduce uncertainty in a localized manner rather than resolving it through holistic, context-sensitive inference. taken together, these findings offer new insights into how current vlms differ in their capacity for contextual cue combination in pragmatic inference. by introducing multiple types of cues-both visual and linguistic-within a controlled experimental setting, this study provides empirical evidence that not all models process contextual information in the same way, and that the ability to integrate multiple cues holistically may serve as a crucial indicator of emerging pragmatic reasoning abilities in vlms. in doing so, this research contributes to the growing body of work on multimodal language processing by highlighting the need to evaluate not only what models generate and understand, but also how they integrate diverse contextual cues to infer meaning. ## conclusion this study examined whether and how current vlms engage in pragmatic inference, particularly focusing on ignorance implicatures, when provided with visual and linguistic contextual cues. through two experiments manipulating modifier types and contextual cues-including situation and quds-we found that not all vlms process such information in the same way. claude demonstrated the ability to integrate multiple contextual cues into a unified interpretation, exhibiting a threshold effect in pragmatic reasoning when both contextual cues were available. in contrast, gpt and gemini tended to treat these cues independently, prioritizing precision over context-sensitive inference. this suggests a fundamental difference not only in cue weighting tendencies but also in how models internally represent and combine contextual information. by systematically evaluating ignorance implicatures across vlms, this study contributes to our understanding of the mechanisms underlying pragmatic behavior in vlms. importantly, it highlights that the capacity for contextual cue combination may serve as one of the key indicators of emerging pragmatic reasoning abilities in vlms. these findings open new directions for evaluating and developing vlms that move beyond literal interpretation toward more human-like pragmatic inference.