Behavioral Analysis of Information Salience in Large Language Models (2025)

Abstract: Large Language Models (LLMs) excel at text summarization, a task that requires models to select content based on its importance. However, the exact notion of salience that LLMs have internalized remains unclear. To bridge this gap, we introduce an explainable framework to systematically derive and investigate information salience in LLMs through their summarization behavior. Using length-controlled summarization as a behavioral probe into the content selection process, and tracing the answerability of Questions Under Discussion throughout, we derive a proxy for how models prioritize information. Our experiments on 13 models across four datasets reveal that LLMs have a nuanced, hierarchical notion of salience, generally consistent across model families and sizes. While models show highly consistent behavior and hence salience patterns, this notion of salience cannot be accessed through introspection, and only weakly correlates with human perceptions of information salience.

## introduction large language models (llms) significantly advanced text synthesis tasks, including text summarization, which they perform well even under zero-shot conditions @xcite . the nature of the summarization task requires models to do content selection: picking the most salient pieces of information for inclusion in a summary @xcite . however, it remains unclear what underlying notion of salience the models have internalized. prior work investigated information salience from several angles. theories of discourse structure have been used to induce content salience @xcite , and a large body of summarization research uses word distribution or centrality as the main signal for content selection @xcite . @xcite laid out a theoretical perspective for content salience in summarization, though the exact notion remains largely latent and aloof; rather, pre-llm summarization work uses human summaries as supervision signals to learn what to include @xcite @xcite . yet, none of these accounts explains why llm zero-shot summarization works so well on the one hand, while missing key elements on the other @xcite @xcite . to begin to make sense of this behavior, we need to understand how models internalize salience: whether it is a consistent notion within and across models, how they prioritize information, and whether llms' notion of salience aligns with prior theories or human intuitions. in this paper, we present a novel explainable framework to systematically derive and investigate llms' grasp of information salience through their summarization behavior. our method combines two key ideas. first, we can use length-constrained summarization @xcite as a behavioral probe into the content selection process of llms. intuitively, when there is a limited length budget for a summary, we posit that the least important information is dropped first. second, we can describe what is salient as the answerability of domain-relevant questions under discussion (quds; @xcite @xcite . quds can be thought of as representations of a coherent unit of information in the form of information-seeking questions, e.g., who are the participants of this study? we use such questions -and hence their answers extracted from documents, according to alternative semantics @xcite @xcite -as the figure 1 : framework overview, conceptualizing content salience as question answerability. left: given a corpus, we derive questions that are typically answered in summaries. length-controlled summarization acts as a probe into the content-selection process of llms. question paraphrases are clustered by semantic intent. middle: answerability is calculated as the fraction of document-answer claims entailed by the summary. right: the content salience map tracks answerability at each summary length. more salient questions remain answerable even in shorter summaries. taken together, by gradually decreasing the length budget available for a summary and by systematically tracing question answerability throughout, we can derive a proxy for how models prioritize information. see figure 1 for an overview. using this framework ( § 2), we empirically study llms' content selection behavior, and its alignment with perceived notions of salience. through experiments on 13 models and four datasets ( § 3), we aim to answer the following research questions: rq1 what notion of salience have llms learned in different domains? rq2 do llms of different families/sizes have a similar notion of salience? rq3 when models introspect, does their perceived notion of salience align with their summarization behavior? rq4 to what extent does model salience align with human perceived salience? we find that llms have a nuanced notion of salience prioritizing information hierarchically across summary lengths ( § 4.1). also the notion of salience is generally compatible between models even of different families and sizes, though more recent/bigger llms correlate more strongly with gpt-4o ( § 4.2). furthermore, models show highly consistent behavior and hence notions of salience, but it cannot be elicited through introspection (i.e., directly prompting for salience of topics; § 5.2). lastly, we find that model behavior only weakly aligns with human perceptions of salience ( § 5.2). ## method: analyzing content salience to analyze content salience we need a way to both observe what content models consider important ( § 2.1), and to describe it in an interpretable manner ( § 2.2). figure 1 illustrates the framework. ## experimental settings datasets. we analyze llm salience across several technical and scientific domains using four datasets ( summarization models. we experiment with 13 llms of different scales: olmo (7b; 02/24, 07/24; groeneveld et al., 2024), mistral (7b; v0.3; jiang et al., 2023), mixtral (8x7b; v0.1, jiang et al., 2024), llama 2 (7b, 13b, 70b; touvron et al., 2023), llama 3 (8b, 70b), and llama 3.1 (8b, 70b; grattafiori et al., 2024). for api-based models, we use gpt-4o-mini (07/24) and gpt-4o (08/24; openai et al. @xcite . we also include 3 baselines to contextualize results: lead-n, random and textrank @xcite , all adjusted to meet summary length budgets. to assess consistency across multiple rounds of decoding, we generate 5 summaries per document and target length with temperature τ = 0.3. we use a zero-shot summarization prompt (appendix g). before analyzing salience in these models, we validate two key assumptions: (i) generated summaries should approximately meet the target length, and (ii) longer summaries should expand on shorter ones ("incremental consistency"). additionally, we analyze how greater τ affect those criteria. our analysis confirms that models largely meet above criteria, with newer and bigger models showing better length control. higher τ results in stable q1. what is the main focus of the study? q2. which patient population is the study concerned with? q3. what condition is being addressed in the study? q4. what is the participant demographic or characteristics in the study? q5. what was the main intervention used in the study? q6. what are the significant benefits of the intervention? q7. what are the specific biological markers influenced by the intervention? q8. what specific treatments were compared in the study? q9. what specific metrics or outcomes were measured? q10. what was the study design or setting of the trial? q11. what are the detailed findings regarding adverse events or side effects? q12. what significant statistical results are reported? ... (9 additional rows) ... llama 3.1 (70b) average .84 10 20 50 100 200 .13 .13 .22 .44 .68 10 20 50 100 200 .26 .26 .32 .47 .70 10 20 50 100 200 .12 .23 .43 .62 .81 10 20 50 100 200 .11 .22 .42 .57 .71 to understand how llms prioritize different information, we consider average question answerability as a proxy for salience. we show the results for the rct dataset as a representative example in figure 2 , and include other datasets in appendix b. models prioritize information hierarchically. we observe a clear hierarchy in how information is prioritized across summary lengths. for example, fundamental aspects such as the focus of a study (q1), and the condition being treated (q3) consistently achieve higher scores, even at 10-word summaries. in contrast, more specific and technical information like the study design (q10) and the statistical significance of results (q12) are primarily discussed in longer summaries (≥ 100 words). information frequency is not in itself predictive of salience. when we consider how frequently a question is answered by documents in the corpus (leftmost column of figure 2 ), we find that even relatively rare questions such as biological markers and adverse effects (q7/11, prevalence 40%/26%) maintain a consistent representation in summaries. this suggests that llms do not simply prioritize information based on its frequency in a genre. summaries progressively get more detailed, and information density differs across models. as expected, longer summaries consistently include more information as shown by the higher average answerability (bottom row in figure 2 ). however, the absolute scores differ across models. gpt-4o has a notably higher answerability score than llama 3.1, particularly at longer summaries (0.81 vs. 0.71 at the 200-word length). given that both models generate summaries of similar lengths (cf. figure 5 ), this suggests that gpt-4o conveys information more efficiently. ## perceived salience and alignment in addition to the observational salience analysis, we elicit perceived salience by having humans and models directly rate the salience of each question. this study has two purposes: (1) to understand whether model behavior aligns with human expectations, and (2) to see if the summarization behavior of llms can be approximated by direct prompting. ## related work evaluating and interpreting summarization. recent work suggests that llms match or surpass human performance in news summarization @xcite . however, traditional evaluation protocols remain unreliable especially for llmgenerated summaries @xcite . this spurred interest in analyzing summarization model behavior. studies found biases towards content near the beginning/end of documents @xcite . others analyze training dynamics of summarization models to identify when skills like content selection are learned @xcite . extractthen-abstract pipelines @xcite aim for interpretable text summarization but this interpretability is limited to the document-level @xcite . our research complements prior work by providing a global in-terpretation of what topics llms consider important through the lens of text summarization. explainable topic modeling. our analysis method draws inspiration from the interpretable topic modeling literature. while classical topic models such as lda @xcite have long been used to explain latent themes in text corpora, they are often difficult to interpret @xcite . recent work showed that llms can effectively be used to generate natural language descriptions of latent themes in text mining, clustering and concept induction workflows @xcite @xcite . our framework uses llms to describe salient summary content in form of informationseeking quds. the use of quds as a representation of information units was shown successful in a wide range of tasks @xcite @xcite . finally, in the context of summarization, our work shares theoretical foundations with @xcite , who explore human curiosity through inquisitive quds. they observe that answering salient questions is a quality indicator for news summaries. ## conclusion we propose an interpretable framework to systematically derive and analyze llms' notion of information salience, a previously latent concept that is nonetheless crucial for text synthesis applications. our work builds on two key ideas: using length-controlled summarization as a behavioral probe for content selection, and describing what is salient as the answerability of questions. we found that llms have a highly consistent notion of salience which is largely compatible across models. we further found that llms cannot directly rate the salience of questions, and that model salience weakly aligns with human perceptions. our work opens new directions to study how llm salience emerges during training, and for diagnosing content selection challenges in text synthesis tasks.