I n F act: Informativeness Alignment for Improved LLM Factuality (2025)

Abstract: Factual completeness is a general term that captures how detailed and informative a factually correct text is. For instance, the factual sentence “Barack Obama was born in the United States” is factually correct, though less informative than the factual sentence “Barack Obama was born in Honolulu, Hawaii, United States”. Despite the known fact that LLMs tend to hallucinate and generate factually incorrect text, they might also tend to choose to generate factual text that is indeed factually correct and yet less informative than other, more informative choices. In this work, we tackle this problem by proposing an informativeness alignment mechanism. This mechanism takes advantage of recent factual informativeness benchmarks to propose an informativeness alignment objective. This objective prioritizes answers that are both correct and informative. We find that when training a model to maximize this objective or optimize its preference, we can improve not just informativeness but also factuality.

## introduction large language models (llms) are known to capture and store extensive amounts of factual knowledge @xcite @xcite @xcite , as they are trained on vast quantities of text, which includes a significant body of factual knowledge. however, they often hallucinate or generate factually incorrect text @xcite @xcite @xcite . although the way llms represent their knowledge remains unclear @xcite , it can be effectively accessed via prompting @xcite . for example, modern llms are likely to correctly complete the input prompt "barack obama was born in" with multiple different answers -united states, hawaii and kapiolani medical center for women & children. while these answers are all factually correct, they differ greatly in specificity and significantly in the level of their informativeness. this highlights an important gap: even when llms do not hallucinate, they often fail to provide the most informative answer. therefore, in this work, our focus extends beyond the factual correctness of the llms by examining the informativeness of correct responses among an llm's outputs. an answer is considered highly informative if it includes either the most specific response or all correct responses. 2 our main assumption and motivation for this work is that an llm might indeed have correct and informative parametric knowledge about a given query, yet generate a less informative answer -as illustrated in figure 1 . this might occur due to statistical or even spurious correlations between the input text (prompts or questions) and the correct yet less informative answer. for example, "barack obama" and "the united states" is more likely to substantially co-occur than "barack obama" and "kapiolani medical center for @xcite in the training data, leading to the llm preferring the less specific but more frequent answer. while the distinction between factuality and informativeness may appear subtle, its practical implications are substantial. underspecified answers, though factually correct, can introduce ambiguity or reduce the usefulness of responses in real-world applications. for instance, in the granola qa dataset, the question "where is knott rigg located?" could be answered with "lake district", which is correct but ambiguous given multiple "lake districts" worldwide. a more informative answer such as "lake district, north west england, united kingdom" resolves this ambiguity and enhances utility. similar considerations arise in high-stakes settings: in healthcare, an answer like "cancer drugs" to the question "what drugs target the egfr gene?" is factually true but insufficiently specific, whereas "tyrosine kinase inhibitors; gefitinib, erlotinib, afatinib, osimertinib" provides actionable information. likewise, in domains such as law, education, and tutoring, informativeness ensures clarity and reliability. this motivates our focus on aligning llms not only to be factually correct but also maximally informative. therefore, we aim to align the model to prefer the most informative or specific answer the llm knows. additionally, we also address the wellknown existing factual precision problem of the llms @xcite by incorporating this into our alignment mechanism. to this end, we begin by formulating the informativeness evaluation task and propose a novel framework to create a general informativeness dataset. in this dataset, each question or input text is paired with a set of answers associated with a hierarchical set of labels, representing different levels of informativeness. this dataset structure, which includes the informativeness metadata for each label is incorporated directly into the training. building on this dataset, we propose the novel training framework infact, to align a pre-trained or instruction-tuned llm model to generate more informative and complete facts. this procedure consists of two stages -structure tuning and informativeness alignment. the structure tuning phase aims to teach the model to consider the informativeness problem setup as well as to abstain from generating misinformation, whereas the informativeness alignment seeks to train the model to answer with the most informative and complete factual answers to the questions. our contributions can be summarized as follows: ## informativeness alignment in this section, we present a detailed overview of our proposed method, infact, which leverages the informativeness evaluation dataset introduced in the last section to improve both the informativeness and factual correctness of llm outputs. ## experimental setup in this section, we describe our experimental setup, including the investigated models, our baselines, and experimental details of our training procedure. ## results we present the main results obtained in our experiments, focusing on testing the influence of our method on the factual accuracy and informativeness of the answers generated by the models. ## related work informativeness evaluating the informativeness of dialogue agents and llms is a crucial research goal because of their widespread use. some studies focus on evaluating the informativeness of dialogue model responses @xcite @xcite , and some on factual sentence completions @xcite . more recent work proposes a benchmark for evaluating the factual granularity of model responses @xcite , which we use in this work for both training and evaluation. external tools are also used for better informativeness and correctness @xcite . in this work, we take a model alignment approach to tackle these open challenges. factuality factuality has been widely studied lately from various different perspectives @xcite . one way to approach it is via the task of factual error detection, where a binary prediction model is provided instead of a continuous probability. this is also related to the setting of selective prediction, where models can abstain from answering a query @xcite . another approach is to adopt model calibration @xcite . the goal is to provide a measure of the probability that a prediction is incorrect alongside the actual prediction. common techniques for calibration include performing various transformations on a model's output logits @xcite and measuring uncertainty (e.g., see @xcite . more recent work has studied the use of lms for providing calibration by training them on statements known to be factually correct or not. this "supervised" approach has been explored via fine-tuning @xcite , in-context learning @xcite , zero-shot instruction-oriented @xcite @xcite , and consistency sampling @xcite techniques. further recent studies @xcite use the internal state of the model for classifying whether it is certain or not, use a new token for unanswerable inputs @xcite or for uncertainty representation @xcite , or construct a specific dataset for effectively tuning the model for answering refusal @xcite . our work addresses the factuality problem by aligning an llm for better informativeness and correctness simultaneously. ## conclusion we propose a novel method infact, which aims to improve both the correctness and informativeness of llm's responses. this mechanism takes advantage of factual questions that can be correctly answered at various levels of informativeness and aligns the llm to exhibit a preference for more informative, yet still correct answers. an in-depth evaluation across diverse qa benchmarks suggests that this mechanism upgrades both the factual precision of the model's answers, by more effectively abstaining rather than generating wrong facts, as well as the informativeness of its answers, by generating a larger amount of correct answers (higher recall of multiple-answer questions), including answers that are more descriptive and of more appropriate granularity. this work has the potential to facilitate several intriguing follow-up studies. one of them is the curation of a new unified and qualitative dataset for informativeness evaluation, which may have the potential to further improve the factual consistency of the llms.