Learning to Generate Answers with Citations via Factual Consistency Models (2024)

Abstract: Large Language Models (LLMs) frequently hallucinate, impeding their reliability in mission-critical situations. One approach to address this issue is to provide citations to relevant sources alongside generated content, enhancing the verifiability of generations. However, citing passages accurately in answers remains a substantial challenge. This paper proposes a weakly-supervised fine-tuning method leveraging factual consistency models (FCMs). Our approach alternates between generating texts with citations and supervised fine-tuning with FCM-filtered citation data. Focused learning is integrated into the objective, directing the fine-tuning process to emphasise the factual unit tokens, as measured by an FCM. Results on the ALCE few-shot citation benchmark with various instruction-tuned LLMs demonstrate superior performance compared to in-context learning, vanilla supervised fine-tuning, and state-of-the-art methods, with an average improvement of 34.1, 15.5, and 10.5 citation F _1 points, respectively. Moreover, in a domain transfer setting we show that the obtained citation generation ability robustly transfers to unseen datasets. Notably, our citation improvements contribute to the lowest factual error rate across baselines.

## introduction large language models (llms; @xcite have showcased remarkable capabilities in information-seeking tasks. however, concerns persist regarding their trustworthiness, stemming from their opaque nature and factual hallucinations in their responses @xcite . one proposed solution involves the production of citations to retrieved text passages alongside llm-produced statements @xcite . by integrating citations into generated responses, readers can easily verify * work done while interning at amazon web services. question: who has the highest goals in world football? retrieved passages: @xcite . . . pelé's 1281 goals are recognized by fifa as the highest total achieved by a professional footballer, although the soccer statistic foundation (rssf) recognizes only 767 goals in official mode, occupying the third place after josef bican (805) and romario (772) . . . figure 1 : an alce-asqa question with a generated answer prompted via in-context learning. two error classes are common: information not supported by the sources (red) and incorrect citation to the sources (blue). llms statements. the ability to accurately produce citations enables llms to generate responses more closely aligned with cited sources, alleviating hallucinations @xcite . despite its significance, accurate citation generation proves to be challenging. state-of-the-art llms, such as chatgpt @xcite , and commercial generative chat engines, such as bing chat, produce accurate citations only for less than 60% of generated statements @xcite . figure 1 illustrates typical citation errors, including hallucinated statements or citations associated with incorrect claims. hence, there is a necessity to train llms to generate citations accurately. this paper focuses on teaching llms to generate citations for retrieval-augmented longform question answering (lfqa), tackling two main challenges: the scarcity of high-quality labeled data at scale and the risk of compromising original language and generalization capacities during fine-tuning for citation generation. to address these challenges, we present calf (citation learning via factual consistency models), a fine-tuning strategy that enables llms to learn citation generation without sacrificing their language capacities. as illustrated in figure 2 , the cornerstone of our approach is factual consistency models (fcms; @xcite , initially introduced as a neural measure of consistency between a claim and its context. we use fcms to gauge whether cited passages support a generated statement. our method incorporates fcms in two designs. firstly, we propose a weakly-supervised training strategy, where an llm generates diverse responses with citations, an fcm filters high-quality citation data, and the llm is fine-tuned on the filtered data. secondly, we utilize focused learning to adjust the loss contribution of each answer token based on its factual relevance, as measured by an fcm. the intuition is to have the llm concentrate on tokens related to factual knowledge during fine-tuning, minimizing the impact on its original language capacities. we evaluate calf on various llms, including llama2 @xcite , mistral-instruct, and mistralorca @xcite . on the alce automatic few-shot citation evaluation benchmark @xcite , calf enhances citation metrics over the in-context learning and baseline finetuning, with an average improvement of 34.1 and 15.5 f 1 , respectively, while maintaining fluency and correctness. all llms trained via calf , outperform the state-of-the-art model self-rag @xcite and @xcite , with an average improvement of 24.8 and 10.5 citation f 1 points, respectively. domain transfer experiments, testing citation quality on a dataset different from the training dataset, highlight calf's ability to generalize across tasks and domains. additionally, on the factscore biography generation benchmark @xcite , calf demonstrates an improvement in factuality. finally, human evaluation results indicate that calf yields more preferable answers compared to the fine-tuning baseline. factual consistency models fcms assess whether all the factual information in a claim is consistent w.r.t to the information conveyed in its grounding text. the task shares strong similarities to natural language inference (nli) @xcite . however, in contrast to nli, factual consistency is not evaluated on subjec-tive or opinionated statements. work that explores fcms for improving the generation is mainly constrained to summarization. @xcite filter samples from a large-scale dataset according to an fcm while @xcite use an fcm to rerank and select source passages for crosslingual summarization. @xcite incorporate fcms to improve factuality via direct preference optimization @xcite . instead of using fcms, @xcite improve the factuality of generations by measuring the cosine similarity between a token's embedding and relevant knowledge to adjust a token's loss contribution. our work is the first to explore the use of model explainability mechanisms, namely shapley values @xcite , for incorporating fcms directly into the fine-tuning process of an llm. ## preliminaries task description. given an information-seeking question q, such as shown in figure 1 , the task is to generate a long-form answer ŷ = {s 1 , • • • , s n }, consisting of sentences s i , conditioned on passages p retrieved from a knowledge base. a long-form answer with citations needs to ensure that one or multiple relevant passages c i ⊆ p are cited in each generated sentence s i (indicated by brackets with a passage index, e.g. "[1]" for p 1 ), such that the generated information in s i follows from the cited passages c i . this task definition strictly requires all facts to originate from the retrieved passages, ensuring that ŷ is fully verifiable by p . we further assume the availability of few-shot training samples (q, p, y) ∈ d to learn citation production. ## citation learning via fcms calf is a fine-tuning strategy for producing longform answers with citations. calf's main assumption is that an fcm ϕ can be leveraged as a supervision signal to improve citation quality of an llm f via an iterative training procedure, as illustrated in figure 2 and algorithm 1. calf alternates between two modes in each iteration k. first, it generates weakly-supervised training data dk to enrich the training corpus by sampling diverse answers from a fine-tuned llm f k-1 and filters them via the fcm ϕ (sec. 4.1). second, it fine-tunes the llm f k-1 on dk + d with a modification of the nll objective, which re-weights the loss contribution of individual tokens according to their importance for ensuring factual consistency, as measured by the fcm ϕ (sec. 4.2). the number of iterations is determined dynamically by stopping when the proportion of filtered examples over the candidates does not improve between iterations or once the maximum number of iterations is reached. ## evaluation datasets & metrics. we conduct experiments on long-form qa datasets of the alce citation benchmark @xcite , namely their version of asqa @xcite and eli5 @xcite . we further evaluate models on bio @xcite . for our domain transfer experiments, we further consider hagrid @xcite as a source for training. since hagrid's answers are generated by gpt-4 without annotations for factual coverage, we do not use it for evaluation as a target. for training, alce contains | d |= 4 cited gold instances. for hagrid, we randomly sample 4 instances for calf. details are in appendix a.1. we follow the official metrics and nomenclature of @xcite to measure correctness (via em recall), fluency (via mauve @xcite ), and citation f 1 (via an nli-trained t5-11b model @xcite ). we further measure rouge-l @xcite and introduce a more strict variation of their correctness metric: passage-grounded correctness (correct. in p), which only considers information from responses which are supported by the retrieved passages p . subsequently, this metric ignores factual content produced by an llm's parametric memory if not explicitly derivable from the retrieved passages, isolating factual grounding from a model's parametric memory. we use factscore @xcite , based on t5-3b. ## conclusion this paper presented calf, a fine-tuning method for llms to produce accurate citations alongside generated text. it focuses on using fcms as a training signal by filtering candidate answers with citations and by re-weighting the llm's objective function according to the tokens' factual importance. calf outperforms all baselines in terms of citation quality and passage-grounded correctness while ensuring that the overall quality of responses remains high, measured via both automated and human evaluation. in a domain transfer setting we further validate the generalizability of calf. moreover, we discuss the benefits of accurate citation production for improving factuality and highlight the importance of each component of calf through a systematic ablation. future work looks at incorporating a learned mechanism into calfto abstain from answering if none of the retrieved passages are considered relevant to the question.