Not Lost After All: How Cross-Encoder Attribution Challenges Position Bias Assumptions in LLM Summarization (2025)

Abstract: Position bias, the tendency of Large Language Models (LLMs) to select content based on its structural position in a document rather than its semantic relevance, has been viewed as a key limitation in automatic summarization. To measure position bias, prior studies rely heavily on n-gram matching techniques, which fail to capture semantic relationships in abstractive summaries where content is extensively rephrased. To address this limitation, we apply a cross-encoder-based alignment method that jointly processes summary-source sentence pairs, enabling more accurate identification of semantic correspondences even when summaries substantially rewrite the source. Experiments with five LLMs across six summarization datasets reveal significantly different position bias patterns than those reported by traditional metrics. Our findings suggest that these patterns primarily reflect rational adaptations to document structure and content rather than true model limitations. Through controlled experiments and analyses across varying document lengths and multi-document settings, we show that LLMs use content from all positions more effectively than previously assumed, challenging common claims about “lost-in-the-middle” behaviour.

## introduction large language models (llms) have significantly advanced summarization, often producing summaries that approach human-level quality @xcite . despite this performance, position bias, where models preferentially select summary content from certain document locations, typically the beginning and end, raises questions about whether these models truly understand content importance or simply exploit positional shortcuts. initially documented as "lead bias" in news summarization, this phenomenon was considered appropriate because of the standard "inverted pyramid" structure of news articles, which emphasizes early content @xcite . however, similar positional preferences have since been reported across various neural architectures @xcite and domains @xcite , suggesting broader implications beyond journalism. more recently, studies have identified a u-shaped attention pattern in which models disproportionately neglect middle sections of documents @xcite , raising concerns about their ability to faithfully process and summarize long-form content. however, characterizing these patterns as biases depends critically on accurately identifying how source content contributes to generated summaries. most existing evaluations rely on n-gram matching, which counts shared word sequences between summaries and sources @xcite . this approach is fundamentally inadequate for abstractive summaries, which involve extensive rephrasing. in fact, over 80% of bigrams in xsum and over 50% in cnn/dailymail summaries are novel (suhara and alikaniotis, 2024), demonstrating the limitations of lexical overlap methods. consequently, current evaluations may significantly underestimate how much source content models actually utilize. furthermore, treating position patterns as biases assumes they reflect model limitations rather than rational responses to document structure. many documents naturally emphasize important information in specific locations, meaning apparent positional preferences may instead reflect effective content selection. this raises the critical need for more reliable methods to distinguish between true model limitations and appropriate structural adaptations. to address these concerns, we introduce a crossencoder approach, a transformer-based model that jointly processes summary-source sentence pairs to explicitly measure semantic alignment. unlike n-gram methods that rely on surface-level word overlap, cross-encoders directly capture meaning relationships, enabling more accurate source attribution even when content is substantially rephrased. specifically, we investigate: 1. how improved semantic alignment alters interpretations of position bias. 2. which position patterns emerge under precise semantic alignment in standard-length documents. 3. how these patterns shift in controlled multidocument scenarios with manipulated positions. 4. whether biases persist in summarizing longer documents with extended context. through experiments with five state-of-the-art llms across six datasets, we demonstrate substantial deviations from previously reported position patterns. our findings suggest that observed positional preferences typically reflect rational alignment with document structures and content importance rather than inherent model limitations. contributions we make four main contributions: ## conclusion this paper fundamentally reframes position bias in llm summarization through improved semantic attribution. using cross-encoder methods, we demonstrate that reported position biases largely reflect rational content assessment rather than architectural limitations. we challenge these core assumptions across five models and multiple datasets. first, the widely-cited u-shaped attention pattern does not hold-models show rightward shifts toward more balanced content use compared to humans. second, controlled position manipulation reveals minimal systematic effects: 90% of comparisons show no significant differences in where models select content, even when sentence counts vary. third, extended context analysis refutes "lostin-the-middle" claimsmodels successfully extract from global middle positions (median ∼ 0.5) in multi-document settings while maintaining quality. most importantly, position patterns prove contextdependent rather than universal. models that struggle with scientific papers excel with government documents, demonstrating adaptive strategies that prioritize content structure over positional heuristics. this suggests that "bias" reflects sophisticated document-type recognition rather than processing limitations. these results shift the research focus from bias mitigation to content assessment enhancement. future work should develop semantic evaluation frameworks that reveal model capabilities obscured by traditional metrics. our cross-encoder approach provides such a foundation, showing that concerns about positional limitations may be overstated when models possess robust content evaluation mechanisms.