Trick or Neat: Adversarial Ambiguity and Language Model Evaluation (2025)

Abstract: Detecting ambiguity is important for language understanding, including uncertainty estimation, humour detection, and processing garden path sentences. We assess language models’ sensitivity to ambiguity by introducing an adversarial ambiguity dataset that includes syntactic, lexical, and phonological ambiguities along with adversarial variations (e.g., word-order changes, synonym replacements, and random-based alterations). Our findings show that direct prompting fails to robustly identify ambiguity, while linear probes trained on model representations can decode ambiguity with high accuracy, sometimes exceeding 90%. Our results offer insights into the prompting paradigm and how language models encode ambiguity at different layers.

## introduction linguistic utterances often have ambiguous meanings, but our world knowledge helps us resolve them. consider the ambiguous sentence in (1), and the unambiguous alterations in (2) and ( 3): (1) the man saw the woman with the telescope. even though sentences of the form np v'ed np with np are always structurally ambiguous between an interpretation where the pp modifies the object np -the so-called low attachment reading exemplified by (2) -and a reading where the pp is the instrument of the vp -the high attachment reading exemplified by (3) -our world knowledge can help us resolve such ambiguities by ruling out unlikely interpretations. in particular, although it is arguably roughly equally conceivable to do both interpretations in (1) rendering both structurally and semantically ambiguity, in the absence of licensing context, the sentences in (2) and ( 3 semantically unambiguous, because it is implausible for someone to use a dress as a seeing device, and implausible for someone to walk around with another person's eyes. once supporting information is introduced, however, implausible sentences can become plausible @xcite . for example, a context in which someone calls a telescope a dress or mistakes a telescope for a dress or has a dress-shaped telescope can make the instrument-pp interpretation in (2) more likely, revealing the sentence's underlying structural ambiguity. in the absence of such licensing context, reading time studies show that humans have a general preference to attach an ambiguous pp to the vp rather than the np (high attachment over low attachment), with higher reading times around the pp region, when the high attachment reading is not semantically licensed @xcite , such as in (2). ambiguity in language poses a great challenge for language models (lms) (see figure 1 ; @xcite , as it can lead to hallucinations, biased completions, and misinterpretations. this can degrade performance on tasks such as factchecking, sentiment analysis, and information extraction @xcite . different types of ambiguity affect models in distinct ways: syntactic ambiguity (e.g., (1)) can lead to parsing errors and flawed summarization or translation; lexical ambiguity (e.g., "the speaker") can confuse question answering and retrieval systems, especially in low-context scenarios; and phonological ambiguity (e.g., "ice cream" vs. "i scream") complicates speech recognition and dialogue modeling. models that can detect and reason about ambiguity can help mitigate these issues by flagging unclear inputs, suggesting alternative interpretations, and improving user understanding-particularly in sensitive contexts like misleading news headlines or political claims @xcite . ambiguity-aware modeling is therefore essential for generating coherent, trustworthy outputs that align with human expectations @xcite . to systematically evaluate how well lms handle ambiguity, we introduce ambadv, a dataset with adversarial variations of ambiguous sentences, and find that most llms struggle to identify ambiguity, with some exhibiting a 'yes bias' when prompted, highlighting the need to account for class distribution in evaluation. we examine how llms encode ambiguous sentences and find that linear probes on layer-wise representations can reliably distinguish ambiguity. our analysis of model disambiguations and representations suggests that llms may partially rely on memorization to resolve ambiguity. ## related work ambiguity enables efficient communication by relying on context and minimizing processing load @xcite . previous works have already highlighted ambiguity as a challenge in a variety of nlp tasks such as multimodal machine translation @xcite , visual question answering @xcite , misleading claim detection @xcite speech-to-text transcription @xcite , humour style classification @xcite , sentiment analysis (buscemi and proverbio, 2024), semantic parsing @xcite , etc. there exist a few attempts to create ambiguity-inclusive datasets. @xcite introduce ambigqa, identifying all possible answers to questions and rephrasing/disambiguating them. @xcite create ambient, an nli benchmark for ambiguity detection and disambiguation. @xcite create a dataset of 1,000 scope-ambiguous sentences, showing that models may be sensitive to the meaning ambiguity. building on these challenges, we construct an adversarial ambiguity dataset to evaluate whether models can detect ambiguity in inputs resembling real-world user queries. while most prior datasets focus on lexical or syntactic ambiguity, phonological ambiguity remains underexplored. recent speech modeling studies show how transformer-based models like whisper and wav2vec2 handle phonological variation, including homophones @xcite , phonotactic patterns (de heer kloots and zuidema, 2024), and assimilation @xcite . these findings suggest that neural models can encode ambiguity-relevant distinctions in the speech domain, motivating our inclusion of phonological ambiguity and rhyme-based perturbations. we position ambadv relative to existing datasets in ## methodology dataset creation we construct a dataset based on 8 syntactically, 16 lexically, and 16 phonologically ambiguous sentences. these sentences were hand-picked based on existing textbooks, online linguistic studies, and ambiguity datasets @xcite @xcite . we modified the originally ambiguous base sentences to create sentence variants each using four different manipulation types: (i) word order. we swap the subject and the object of the sentence or, when not possible, create passive/active voice alternations. critically, this manipulation does not lead to a semantically implausible interpretation of either of the pp attachment possibilities. (ii) synonymous word. we exchange a key word in the sentence for a synonym. because we use synonyms, these sentence variants have the same ambiguity structure as the original example. (iii) random word. we exchange a keyword in the sentence for a random word of the same syntactic category. these sentence variants rule out one interpretation of the sentence (i.e., here: the pp can no longer be interpreted as introducing the instrument of the seeing action). (iv) rhyme word. we exchange a key word in the sentence for a word that rhymes in sound with the original word. ## conclusion we introduce the first adversarial dataset for linguistic ambiguity to evaluate whether llms can assess a sentence's ambiguity status under minimal adversarial attacks that may or may not alter its ground truth label. our results show that llms struggle to accurately interpret ambiguous sentences. while ambiguity-related information seems present in the models' representations, they fail to leverage it effectively in their outputs. our representational analysis shows that linear probes on layer-wise representations can reliably distinguish ambiguity, suggesting that llms may partially rely on memorization rather than ambiguity understanding.