Title: Precise Information Control in Long-Form Text Generation
Venue: NeurIPS 2025
Accepted: yes
Keywords: faithful hallucination

faithful text generation

long-form generation evaluation

controllable language models

strict grounded generation
Abstract: A central challenge in language models (LMs) is faithfulness hallucination: the generation of information unsubstantiated by input context. To study this problem, we propose Precise Information Control (PIC), a new task formulation that requires models to generate long-form outputs grounded in a provided set of short self-contained statements, without adding any unsupported ones. PIC includes a full setting that tests a model’s ability to include exactly all input claims, and a partial setting that requires the model to selectively incorporate only relevant claims. We present PIC-Bench, a benchmark of eight long-form generation tasks (e.g., summarization, biography generation) adapted to the PIC setting, where LMs are supplied with well-formed, verifiable input claims. Our evaluation of a range of open and proprietary LMs on PIC-Bench reveals that, surprisingly, state-of-the-art LMs still hallucinate against user-provided input in over 70% of generations. To alleviate this lack of faithfulness, we introduce a post-training framework that uses a weakly supervised preference data construction method to train an 8B PIC-LM with stronger PIC ability—improving from 69.1% to 91.0% F1 in the full PIC setting. When integrated into end-to-end factual generation pipelines, PIC-LM improves exact match recall by 17.1% on ambiguous QA with retrieval, and factual precision by 30.5% on a birthplace fact-checking task, underscoring the potential of precisely grounded generation.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe manuscript proposes a novel task, \"Precise Information Control\" (PIC), which involves long-form generation that includes all or part of the suggested claims.\nBy adapting existing language generation benchmarks, the authors introduce PIC-Bench, a dataset consisting of (Instruction, Claims, and Gold-Response) tuples.\nBy applying SFT and DPO to this dataset, PIC-LM achieves the best or near-best performance among open-source models and demonstrates comparable results to closed-source models.\nThe authors also verify its viability in real-world scenarios, such as retrieval-augmented generation and self-verification.\n\nSTRENGTHS AND WEAKNESSES:\n- **Strenght**\n  - Overall, I like this paper. The manuscript is well written. It proposes a dataset, training method, model, and its performance on real-world scenarios. The experiments are thorough and well-motivated.\n\n- **Weakness**\n  - **Presentation**\n    - Please check the definitions of intrinsic and extrinsic hallucinations. As far as I know, intrinsic hallucination refers to hallucination that can be inferred from the given context, while extrinsic hallucination refers to hallucination that cannot be inferred from the context. ([1] – Sections 2.1 and 2.2 mention that intrinsic and extrinsic hallucinations are subcategories of factual hallucination.) Consider using the terms \"faithful\" instead of \"intrinsic\" and \"factual\" instead of \"extrinsic.\" For example, in the manuscript, the POPB-CF benchmark is faithful but not factual.\n    - It would be helpful to include some qualitative results showing (1) why a perfect match is so rare, especially in the PIC-Bench-Full setting, and (2) why instruction-following performance decreases in Table 4 after fine-tuning.\n  - **Experiment**\n    - I'm not fully convinced why XSUM data is used, as it is \"extreme summarization\" in a single sentence, as the name implies. Doesn't this conflict with the paper's focus on \"long-form text generation\"?\n    - Why did you sample the $y_{perturb}$ from $\\theta_{SFT}$, not superior models? It's a bit worrisome as $\\theta_{SFT}$'s capacity does not seem good from your results.\n    - I'm also not convinced by the strategy of using the last $L$ tokens for score computation, as described in lines 184–189. The authors mention that this approach is due to efficient computation and length bias, but couldn't the length bias be mitigated by using a different denominator?\n    - I read in the appendix, line 1259, that the train and test sets do not overlap. However, it would be better to ensure that the dataset does not have any overlap between the training set and the test benchmark. In other words, it would be better not to use FACTS and biography generation datasets for training. Since the training data uses the same (or similar) test instructions during the training procedure, this would definitely affect the final score, unlike other non-fine-tuned models.\n    - It would be helpful to compute the ChatGPT–Human agreement score, as the evaluation process heavily depends on ChatGPT's ability to extract and verify claims, at least for a few examples.\n  - **Misc.**\n    - It would be better to exclude POPB-CF from the average calculation, as this benchmark directly conflicts with some models' instructions to be factual. From Tables 2 and 4, the main source of the average score gap appears to come from this benchmark, which does not seem fair. (I'm also not sure whether this is an appropriate benchmark, as it could raise certain ethical issues.)\n\n- **Typos**\n  - Line 213: \"... 8B open-weight baseline (Llama 3.3 70B ...\"\n  - Check bold in Table 6 (Ministral 8B - QAMParI - Prec.)\n\n**References**\n[1] On Faithfulness and Factuality in Abstractive Summarization, ACL 2020\n\nQUESTIONS:\n- Please provide some qualitative results analyzing questions (1) and (2) above.\n- I am curious whether using all tokens versus only the last $L$ tokens for score computation (as mentioned in line 185) yields similar results.\n- Please provide evidence of ChatGPT–Human agreement in your evaluation process.\n- Please reconsider the presentation of the POPB-CF benchmark, as it may be misleading.\n\nLIMITATIONS:\nAlthough the authors mention their setting—where user-provided knowledge should supersede the LM's inherent priors—in lines 317–319, its implications are not thoroughly discussed. This approach could raise ethical concerns, as it may allow individuals to manipulate the LM to generate any content. This is precisely why several guardrails have been established in the AI community. It would be better if the authors addressed these concerns in more detail.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 4, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-06-19"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper introduces a task called Precise Information Control (PIC) in an attempt to measure the ability of LMs to generate long-form text grounded in a set of provided input statements, i.e. without intrinsic hallucinations. In this task, the LM is provided with a set of short verifiable claims and an instruction to generate the long form output in one of 2 settings - a *full setting* in which the model is expected to utilize all provided claims in the generation, and a *partial setting* in which the model can choose to select relevant claims to be included. A verifiable claim extractor from past work (VeriScore, Song et. al. 2024) is utilized for extracting claims from summarization, biography generation & QA datasets to construct the input data, which results in a set of 8 datasets curated in the PIC setting and (expected to be) released as a benchmark for this task. The claim extractor is also applied to the generated response in order to compute support with supplied input claims. \nApart from off the shelf LLMs, a Llama 3.1 8B Instruct model is fine tuned using a) SFT b) DPO and compared against baselines. The model finetuned with DPO shows better performance than most instruction tuned & reasoning models for both full & partial settings.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n1. Several types of datasets have been covered for this problem, i.e. summarization, biography generation & QA, which is commendable. \n2. The paper is fairly well-written and contains exhaustive experiments in several directions, e.g. analyses on instruction following capabilities of the LM, comparison with RAG settings, etc.\n\nWeaknesses:\n1. Precise Information Control has been presented as a new task. However, it is very similar to RAG & attributable generation (without the need for explicit citations) as the paper also states (Sec 6, 7), where the generated response is required to be grounded in provided source references. Both the task setting and the evaluation processes are similar between the two. The authors attempt to address this by stating that the full PIC setting requires inclusion of all input claims, but that does not suffice in terms of novelty since this would be achievable in the attributable generation setting by modifying the input instruction prompt to direct the LM to include all source refs as well. \n2. The task expects claim extraction as a necessary step to be conducted from typically noisy input source material that is generally present in RAG settings. This creates a large dependency on efficacy of the claim extractor model, and assumes that only relevant claims will make their way into the input. This seems to be an unrealistic setting to expect in the real-world.\n\nQUESTIONS:\n1. Given that the LLMs are sensitive to the instruction prompts, were there experiments conducted by varying the prompts used for the problem for the full & partial settings? Specific instruction prompts are likely to garner better performance from the models. \n2. Could you elaborate on how the gold response was obtained (line 111)?\n3. A few lines on the model used to compute support for eval (line 115) would be helpful. Support is typically posed as an NLI task in these cases, but it would help to understand what was used here. \n4. Which instruction-following model is referred to in line 184?\n5. It is good to see the DPO-finetuned Llama 8B model to be at par/outperform other baselines in Tables 2 & 3. However, it's curious to see the SFT model degrade in performance w.r.t the same base model in F1 for AskH, ELI5, ExpertQA & FACTS. Do you have any thoughts on why that might be the case?\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 5, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-06-28"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nPrecise Information Control (PIC) is a new generation task that eliminates intrinsic hallucinations by demanding outputs containing exactly and only a supplied list of factual claims. The authors construct PIC-Bench, eight datasets spanning biographies, QA, and news summaries in both “full” (all claims required) and “partial” (only relevant claims) modes, and show that even GPT-4 and Claude add or omit claims in over 70 % of cases. They fine-tune an 8B Llama-3 model with SFT + DPO to obtain PIC-LM, which delivers higher factual fidelity while preserving general instruction-following skills (judged by Promotheus). They also show that plugging PIC-LM into retrieval-augmented QA and self-verification pipelines yields noticeably more trustworthy answers.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\n\n* The paper introduces a distinct and granular formulation of the hallucination problem. Instead of treating intrinsic hallucination as a binary phenomenon, PIC enforces and measures faithfulness at the level of individual factual claims. This per-claim control is a novel perspective. It goes beyond prior work by requiring exact inclusion/exclusion of specific information units, a stricter criterion than traditional factuality metrics.  \n* PIC-Bench covers eight tasks spanning different domains and formats. These include: generating Wikipedia-style biographies (EntityBios, PopBios), open-ended Q\\&A from Reddit (ELI5, AskHistorians), expert-curated Q\\&A (ExpertQA), abstractive news summarization (XSUM), and a long-document grounded generation task (FACTS).  \n* An 8B PIC-LM lifts full-PIC F1 from 69.1% to 89.9% and delivers strong precision improvements in downstream QA/verification pipelines.\n\nWeaknesses\n\n* The fine-tuned PIC-LM is specialized to follow given facts religiously. A possible weakness is how this specialization affects its general language abilities or flexibility. The paper includes an alignment evaluation (“Prometheus score”) suggesting only a minor drop in general helpfulness, but a more extensive evaluation on non-PIC tasks would strengthen confidence in PIC-LM’s general usability.  \n* PIC penalizes any additional (even correct) information; the paper lacks discussion or ablation on whether such rigid constraints harm user value in contexts where minor extra facts are tolerable.  \n* The evaluation focuses almost entirely on factual inclusion metrics (F1, precision of claims) and Prometheus scores. There is little assessment of the linguistic quality or user-perceived quality of the outputs.   \n* The research focuses on an 8B-parameter model for fine-tuning. While this is pragmatic, it remains unclear how well this method would scale to bigger or smaller models.\n\nQUESTIONS:\n* Consider providing a breakdown of remaining errors (missing vs. unsupported claims, topical patterns) and discuss how those insights can guide future mitigation strategies.  \n* In the partial PIC setting, only a subset of provided claims should be used. It would help to know how the “relevant” subset is determined for evaluation. The paper could be more explicit on whether this is based on the original reference output or some information retrieval heuristic. For example, if multiple claims in the context could have been relevant but the model was penalized for including them (because the reference didn’t), that might unfairly penalize the model.  \n* How do the PIC-LM outputs do in terms of coherence and style, beyond just containing the required facts? It would be great to include a small qualitative analysis or human evaluation of output quality.   \n* By design, PIC penalizes any extra information not present in the input, even if that information is factually correct. This stringent criterion is appropriate for measuring hallucination, but it could be argued that in some applications a harmless additional detail (especially if true) might not be a failure. Could the authors provide more discussion on this nuance?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper focuses on the intrinsic hallucination and introduces Precise Information Control (PIC), a new task formulation for evaluating and improving language models’ ability to generate long-form text with no intrinsic hallucinations, which is strictly grounded in provided input claims without introducing unsupported information. There are two proposed evaluation settings: full PIC and partial PIC. They also develop PIC-bench, a benchmark with 8 long-form generation tasks, and introduce PIC-LM, an 8B parameter model trained through a two-stage approach combining supervised fine-tuning with preference optimization using a novel weakly-supervised data construction method. \nThey demonstrate that even SOTA models have intrinsic hallucination in over 70% of outputs, while PIC-LM achieves better performance in both settings.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n- Well-presented, good motivation: I like the motivation of the paper, PIC-Bench could become standard benchmark for intrinsic hallucination research. In real world, medical, legal, and journalistic domains urgently require controllable information generation. Also, intrinsic hallucination has been understudied compared to extrinsic hallucination; this work can serve as a foundation for future intrinsic hallucination research.\n- Precise framework for a more fine-grained task: this paper transforms the vague binary judgment evaluation into precise claim-level constraints, also why “verifiable claim constraints” rather than sentence and atomic is well supported. The authors define clear constraints through  rigorous set relationships, and also provide computable and optimizable objective functions. \n- Well-designed Benchmark: the authors provide 8 different long-term generation tasks, and also two different settings for different real-world applications. They also provide a complete and comprehensive technical implementation details. \n\nWeaknesses:\n- I find the evaluation pipeline depends a lot on the LLM, e.g. GPT 4o, if the LLM used has some biases in claim extraction and verification accuracy, then this could result in failure to generalize to evaluation with different LLM judges.\n\nQUESTIONS:\nSee weaknesses.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 2, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-07-06"}