When Format Changes Meaning: Investigating Semantic Inconsistency of Large Language Models (2025)

Abstract: Large language models (LLMs) have demonstrated remarkable capabilities in natural language processing tasks. However, they remain vulnerable to semantic inconsistency, where minor formatting variations result in divergent predictions for semantically equivalent inputs. Our comprehensive evaluation reveals that this brittleness persists even in state-of-the-art models such as GPT-4o, posing a serious challenge to their reliability. Through a mechanistic analysis, we find that semantic-equivalent input changes induce instability in internal representations, ultimately leading to divergent predictions. This reflects a deeper structural issue, where form and meaning are intertwined in the embedding space. We further demonstrate that existing mitigation strategies, including direct fine-tuning on format variations, do not fully address semantic inconsistency, underscoring the difficulty of the problem. Our findings highlight the need for deeper mechanistic understanding to develop targeted methods that improve robustness.

## introduction large language models (llms) have become the foundation of modern natural language processing, achieving state-of-the-art performance across a wide range of tasks @xcite @xcite . despite their impressive capabilities, llms frequently exhibit semantic inconsistency, where minor variations lead to inconsistent predictions for semantically equivalent inputs @xcite @xcite @xcite @xcite . as illustrated in figure 1 , even a frontier model like gpt-4o may produce different answers to the same question solely due to formatting changes. such inconsistencies raise concerns about the trustworthiness and reasoning stability of llms, suggesting that these models may not possess genuine semantic understanding but rather function as stochastic parrots, generating responses based on surfacelevel patterns @xcite @xcite . in this work, we systematically examine semantic inconsistency of llms, investigating whether they provide consistent predictions across semantically equivalent inputs. our comprehensive evaluation across diverse tasks, model and prompting strategies quantifies the pervasiveness of issue, revealing that even the state-of-the-art models such as gpt-4o remain vulnerable. this persistent brittleness presents a serious obstacle for deploying llms in high-stakes domains such as law, finance and healthcare, where consistency and trust are essential. to this end, our work aims to understand when and why semantic inconsistency emerges and how it might be controlled. we first provide a mechanistic diagnosis, tracing semantic inconsistency from superficial input perturbations to instability in internal embeddings, revealing a failure to form formatinvariant representations. this analysis suggests that form and meaning are not clearly separated in the embedding space. we then show that common mitigation strategies, such as fine-tuning on diverse format variations, are insufficient to eliminate inconsistency, indicating that the problem extends beyond the input-output level. finally, by connecting these failures to their internal representational causes, we motivate the need for representationaware solutions that address the root mechanisms of inconsistency. ## measuring semantic consistency under prompt format variations understanding how language models respond to semantically equivalent inputs with different surface forms is crucial for diagnosing their generalization and abstraction capabilities. in this work, we focus on semantic consistency, which captures whether a model produces stable predictions across semantic-preserving input variations @xcite . we specifically analyze prompt format variations @xcite , perturbations that alter input formatting (e.g., spacing, capitalization) while preserving the exact wording of the main content verbatim. this choice is motivated by the prevalence of such variations in real-world use, where users may unintentionally modify prompt formats without altering the semantic content. we propose a novel evaluation framework that combines: (1) setwise consistency, a stricter metric that requires identical outputs across all semanticpreserving variants of an input, and (2) prompt format variations, a class of overlooked yet practically important perturbations. ## tracing the propagation of semantic inconsistency we present a mechanistic analysis tracing how superficial format variations propagate through a model's internal representations, ultimately leading to output inconsistencies. first, we establish the primary propagation chain: from input-level format differences to instabilities in intermediate representations and finally to output-level inconsistency, as summarized in figure 4 (section 5.1). we then examine the context-dependent impact of specific format elements (section 5.2) and analyze how model confidence correlates with consistency (section 5.3). ## the limits of existing mitigation strategies our earlier analysis (section 5) diagnosed semantic inconsistency as a deep representational failure. this motivates an evaluation of existing mitigation strategies to assess their effectiveness and limitations. we examine (1) prompt-based techniques that do not modify model parameters and (2) a more direct approach using fine-tuning with diverse format variations. ## conclusion this study systematically investigates semantic consistency of large language models (llms), showing that even state-of-the-art models like gpt-4o remain vulnerable to minor prompt format variations. our mechanistic analysis moves beyond behavioral observation to diagnose the root cause, tracing the problem to a deep representational failure, where form and meaning are intertwined. we further demonstrate that widely used mitigation strategies, including prompt-based methods and fine-tuning, do not fully address the problem. taken together, our results motivate deeper mechanistic understanding to address the root causes of semantic inconsistency.