Position of Uncertainty: A Cross-Linguistic Study of Positional Bias in
  Large Language Models
Authors: name: Menschikov Mikhail; name: Alexander Kharitonov; name: Maiia Kotyga; name: Vadim Porvatov; name: Anna Zhukovskaya; name: David Kagramanyan; name: Egor Shvetsov; name: Evgeny Burnaev
Published: 2025-05-22 02:23:00

Abstract: Large language models exhibit positional bias -- systematic neglect of
information at specific context positions -- yet its interplay with linguistic
diversity remains poorly understood. We present a cross-linguistic study across
five typologically distinct languages (English, Russian, German, Hindi,
Vietnamese), examining how positional bias interacts with model uncertainty,
syntax, and prompting. Key findings: (1) Positional bias is model-driven, with
language-specific variations -- Qwen2.5-7B favors late positions, challenging
assumptions of early-token bias; (2) Explicit positional guidance (e.g.,
correct context is at position X) reduces accuracy across languages,
undermining prompt-engineering practices; (3) Aligning context with positional
bias increases entropy, yet minimal entropy does not predict accuracy. (4) We
further uncover that LLMs differently impose dominant word order in
free-word-order languages like Hindi.