Learning Fine-Grained Grounded Citations for Attributed Large Language Models (2024)

Abstract: Despite the impressive performance on information-seeking tasks, large language models (LLMs) still struggle with hallucinations. Attributed LLMs, which augment generated text with in-line citations, demonstrate potential in mitigating hallucinations and improving verifiability. However, current approaches suffer from suboptimal citation quality due to their reliance on in-context learning. Furthermore, the practice of merely citing document identifiers complicates the process for users to pinpoint specific supporting evidence. In this work, we introduce FRONT, a training framework that teaches LLMs to generate Fine-gRained grOuNded ciTations. By initially grounding fine-grained supporting quotes, which then guide the generation process, these quotes not only provide supervision signals to improve citation quality but also serve as fine-grained attributions. Experiments on the ALCE benchmark demonstrate the efficacy of FRONT in generating superior grounded responses and highly supportive citations. With LLaMA-2-7B, the framework significantly outperforms all the baselines, achieving an average of 14.21% improvement in citation quality across all datasets, even surpassing ChatGPT1.

heading: 1 Introduction
text: The recent advent of large language models (LLMs) (Touvron et al., 2023; OpenAI, 2023) has taken the world by storm, fueling a paradigm shift in information acquisition (Zhu et al., 2023). Despite their compelling performance, LLMs still struggle with hallucinations (Ji et al., 2023; Huang et al., 2023), a tendency to fabricate non-existent facts or generate unfaithful content. This issue further poses a risk of misinformation dissemination(Chen and Shu, 2023), directly impacting the reliability and trustworthiness of LLMs.
*Corresponding Author 1Our data and code can be found at: https://github.
com/LuckyyySTA/Fine-grained-Attribution.
Such prevalence of hallucinations in LLM outputs has motivated the development of attributed systems (Nakano et al., 2021; Thoppilan et al., 2022; Menick et al., 2022), such as New Bing2 and Perplexity3, where LLMs are allowed to generate responses with in-line citations. Not only does it improve factuality and alleviate hallucinations, but it also simplifies user verification of model outputs, further enhancing the verifiability of LLMs.
Despite recent advancements, current attributed LLMs still expose significant limitations. Firstly, recent efforts in attributed LLMs predominantly rely on either in-context learning (Gao et al., 2023b) or post-hoc retrieval (Gao et al., 2023a), lacking
2https://www.bing.com/chat 3https://www.perplexity.ai
14095
an inherent capability for attributable generation, thereby resulting in compromised citation quality (Liu et al., 2023). Secondly, these citations are typically presented in the form of either document identifiers (Nakano et al., 2021) or URLs (Thoppilan et al., 2022). Such coarse attributionsn complicate the process for users to pinpoint exact supporting evidence, particularly in lengthy documents.
To this end, we aim to advance attributed text generation by empowering LLMs with fine-grained attribution ability. However, one challenge comes from the acquisition of high-quality attribution data for supervised fine-tuning, which is difficult and costly to annotate, and therefore scarce. Thus, we start with an automatic data generation pipeline tailored for collecting high-quality attribution data (§3.1). Given a user query, the pipeline automates data construction through document retrieval, relevance reranking, attributed answer generation, and data filtering to ensure the informativeness and attributability of the answers. Furthermore, to better unlock LLMs’ ability for fine-grained attribution, we introduce FRONT, a training framework that teaches LLMs to generate Fine-gRained grOuNded ciTations (§3.2). Specifically, the framework involves two stages: Grounding Guided Generation (G3) and Consistency-Aware Alignment (CAA). G3 first select supporting quotes from retrieved sources (grounding) and then condition the generation process on them (generation). Then, CAA utilizes preference optimization to further align the grounding process and generation process by automatically constructing preference signals. In this way, these quotes can serve as fine-grained citations and improve the efficiency of the verification process for users (see Figure 1).
We conduct extensive experiments to evaluate our framework on the ALCE Benchmark (Gao et al., 2023b). Our findings are as follows:
• FRONT demonstrates supervisor performance gains in citation quality compared to all baselines, achieving an average 14.21% improvement using LLaMA-2-7B.
• Human evaluation reveals that the quotes generated by our framework are of high quality and significantly benefit user verification.
• Analysis shows that FRONT generates less hallucination and demonstrates remarkable generalization across different base models.

heading: 2 Related Work
text: Retrieval Augmented Generation. Recently, retrieval augmented generation (RAG) (Karpukhin et al., 2020; Lewis et al., 2020; Feng et al., 2023; Gao et al., 2023c) has shown promise in knowledgeintensive tasks. By incorporating retrieved documents, LLMs are equipped with up-to-date information, significantly mitigating knowledge gaps. However, recent studies (Shi et al., 2023; Yoran et al., 2023; Xu et al., 2023a; Zhu et al., 2024) have revealed that existing retrieval-augmented LLMs struggle to handle irrelevant or contradictory retrieval documents and effectively utilize contextual evidence. These limitations can result in performance degradation or even hallucinations (Huang et al., 2023), highlighting the necessity for more factual and verifiable systems.
Attributed Large Language Models. The persistent challenge of hallucinations within LLMs has spurred the development of attributed LLMs (Bohnet et al., 2022; Li et al., 2023; Worledge et al., 2023), which seek to enhance information verifiability by generating responses with attribution to evidence sources. The way of providing attributions varies across studies. For example, Gao et al. (2023b) enables LLMs to generate text with in-line citations via in-context learning. Another line of research (Gao et al., 2023a; Xu et al., 2023b) explores post-hoc attribution, where LLMs first generate an initial response and then retrieve the most relevant evidence to achieve attribution. In this paper, we advance the research on attributed LLMs further. Unlike existing models that predominantly cite document identifiers, we delve into a more fine-grained form of attribution by pinpointing and citing specific extractive quotes.

heading: 3 Task Formulation and Methodology
text: Following (Liu et al., 2023; Gao et al., 2023b), the task is formalized as follows: given a user query q and a corpus of retrieved documents D as input, the LLM is required to produce a response S, which consists of statements with embedded in-line citations. We assume the response S comprising with n statements S = {s1, s2, . . . , sn} and each statement si ∈ S, cites a list of passage Ci = {ci1, ci2, . . .}, where cij ∈ D. Specifically, citations are presented in the form of [1][2].
Next, we present a comprehensive overview of our method, which consists of two primary compo-
nents: an automatic data generation pipeline (§3.1) and a two-stage training framework (§3.2).

heading: 3.1 Automatic Data Generation Pipeline
text: Equipping LLMs with the attribution capability necessitates training data that includes high-quality responses paired with precise citations, which is typically labor-intensive and costly. To address this challenge, we propose a pipeline designed for the automatic generation of high-quality attributed data. This pipeline comprises three core components: data collection, attributed answer generation, and data filtering, as outlined in Figure 2.
Data Collection. To simulate the real-world environment for information-seeking, we collect questions from the AQuAMuSe dataset (Kulkarni et al., 2020), which is derived from the Natural Question (NQ) dataset (Kwiatkowski et al., 2019). The NQ dataset comprises real user queries from the Google search engine, providing a robust basis for realistic question-answering scenarios. The dataset spans a range of diverse question types, demanding answers of varying lengths, from concise to detailed. To mimic the way a search engine might synthesize documents of high relevance in response to a user query, we employ Sphere (Piktus et al., 2021), a pre-processed and cleaned version of the Common Crawl corpus, serving as a proxy web search index. In particular, for a given user query sampled from the AQuAMuSe dataset, we initially retrieve the top 100 relevant documents from the Sphere corpus using sparse retrieval. These documents are subsequently re-ranked by RankVicuna (Pradeep
et al., 2023) considering its superior performance in listwise re-ranking, resulting in the top 5 most relevant documents for each query.
Attributed Answer Generation. Given the remarkable performance of ChatGPT in attributed question answering, we employ ChatGPT to generate answers with corresponding citations for given queries and the top 5 retrieved documents. We provide precise instructions and in-context demonstrations to ensure that ChatGPT produces informative responses and cites the sources accordingly.
Data Filtering. To guarantee the high quality of our synthetic training data, we employ a data filtering process guided by two key criteria derived from Kamalloo et al. (2023): (1) informativeness: assessing if the answer provides sufficient information to the question, and (2) attributability: determining if the answer is attributed to the cited documents. To mitigate the impact of nonsensical queries and irrelevant document retrieval that may lead to non-informative answers, we utilize ChatGPT for preliminary informativeness annotations. Responses categorized as non-informative are directly excluded. Furthermore, to ensure that answers are accompanied by highly supportive citations, we train a discriminator on human-labeled data from the comprehensive evaluation by Liu et al. (2023), where attributability is categorized into three levels: full support, partial support, or no support. We quantitatively map the discriminator’s outputs to an attributability score and ultimately derive an average score for each attributed answer. Answers falling below a defined threshold are sys-
tematically excluded to ensure the synthetic data’s reliability, which results in nearly 8,000 entries. For more details, please refer to Appendix A.

heading: 3.2 Two-Stage Training Recipe
text: In this section, we introduce FRONT, a two-stage training framework that aims at empowering LLMs with fine-grained attribution capability. Figure 3 illustrates the overview of our framework.

heading: 3.2.1 Grounding Guided Generation
text: To empower LLMs with fine-grained attribution capability, we propose Grounding Guided Generation (G3), which teaches LLMs to generate fine-grained citations. The cornerstone of G3 lies in enabling LLMs to extract supporting quotes from the source documents, each associated with its document identifier, which in turn guides the generation of attributed answers. Such a grounding format offers two primary benefits. Firstly, the direct extraction of quotes from sources significantly reduces the impact of the incorporation of irrelevant information and the risk of hallucinations in subsequent attributed answers. Secondly, the process naturally facilitates accurate attribution, with each document identifier serving as a clear supervised signal that delineates the origin of the extractive quotes, thus improving the citation quality.
However, the absence of specific grounding content for statements within our generated dataset poses additional challenges. To tackle this, we employ ChatGPT to meticulously extract segments from cited documents that support the corresponding statement. Hence, when given a query q and the top-5 retrieved documents D as input, the LLM is fine-tuned to generate a response S which consists
of two components: the extractive grounding G and the attributed answer A. Specifically, the extractive grounding G is delineated as follows:
G = {[GROUNDING], (i1, e1), . . . , (in, en)}, (1)
where [GROUNDING] denotes a special token indicating the start of grounding content. Each tuple within G, comprising a document identifier i and the corresponding extractive segment e, collectively forming an extractive grounding quote.
Similarly, the formulation of the attributed answer A is concisely presented as:
A = {[ANSWER], s1, s2 . . . , sm}, (2)
where [ANSWER] is a special token that signals the beginning of the attributed answer. Each statement si cites a list of passages Ci = {ci1, ci2, . . .}, where cij ⊆ {i1, i2, . . . , in}, as defined in Equation 2.
Thus, the training loss is formulated as:
L = − N∑
i=1
logP (yi|qi,Di; θ) (3)
where yi represents the combined output of grounding G and answer A for each given query qi and set of retrieved documents Di.

heading: 3.2.2 Consistency-Aware Alignment
text: While G3 unlocks the ability to first extract supporting quotes before generating attributed answers, it occasionally leads to inconsistencies between grounding quotes and attributed answers. Such discrepancies challenge the attempt to employ these grounding quotes as fine-grained verification. In
response, we propose a consistency-aware alignment stage specifically aimed at enhancing the consistency between the grounding process and the generation process.
The cornerstone of our approach involves contrasting a consistent answer with an inconsistent one under the guidance of the same oracle grounding quotes, which aligns with the concept of Reinforcement Learning from Human Feedback (RLHF) (Ouyang et al., 2022; Bai et al., 2022), where LLMs are further fine-tuned to distinguish between desirable and undesirable responses under preference feedback. However, such contrastive preference feedback typically comes from human annotation. Inspired by the weak-to-strong generalization (Burns et al., 2023; Zhao et al., 2024) where a weaker LLM is utilized to guide the training of more powerful LLMs, we introduce consistency-aware alignment (CCA) that employs smaller LLMs (e.g., 7B) to provide contrastive supervision signals. In this setting, the process not only encourages the LLM to generate attributed answers more consistent with the grounding quotes but also facilitates the identification and correction of nuanced errors present in smaller models.
Specifically, we adopt Direct Preference Optimization (Rafailov et al., 2023), a variant of RLHF known for its stability, for our contrastive alignment. Formally, for each instance, given the oracle grounding g(i) along with a consistent oracle answer y(i)w as well as an attributed answer y (i) l generated by a weaker LLM via in-context learning, we can simply construct a preference dataset:
D = { x(i), τ (i)w , τ (i) l }N i=1 , (4)
where τ (i)w = g(i)◦y(i)w denotes the concatenation of the oracle grounding with the consistent, attributed answer, τ (i)l = g
(i) ◦ y(i)l denotes the concatenation with the inconsistent attributed answer. Here, ◦ signifies the operation of string concatenation.
Finally, we can optimize the policy model πθ on the dataset D by minimizing the following loss:
LDPO(πθ;πref ;D)
=− E(x,τw,τl)∼D [ log σ ( β log
πθ(τw|x) πref(τw|x)
− β log πθ(τl|x) πref(τl|x)
)] ,
(5)
where πref represents the reference model, initialized from G3. The hyper-parameter β modulates
the divergence between the distribution from the policy model and the reference model. τw is the consistent answer, while τl is the inconsistent one.

heading: 4 Experimental Settings

heading: 4.1 Datasets
text: We conduct experiments on the ALCE benchmark (Gao et al., 2023b), designed for attributed text generation. The benchmark includes three long-form QA datasets that span various types of questions.
ASQA (Stelmakh et al., 2022) is a long-form factoid QA dataset characterized by inherently ambiguous questions that require multiple short answers to encapsulate different viewpoints.
ELI5 (Fan et al., 2019) features open-ended questions intended for simplification to the comprehension level of five-year-olds, requiring explanatory multi-sentence responses.
QAMPARI (Amouyal et al., 2022) is a factoid QA dataset derived from Wikipedia, where answers are structured as a compilation of entities.

heading: 4.2 Evaluation Metrics
text: Following the ALCE benchmark (Gao et al., 2023b), our evaluation primarily focuses on two key dimensions: Citation Quality and Correctness. Detailed descriptions of additional evaluation dimensions are presented in the Appendix B.
Citation Quality. Citation quality is critical for evaluating LLM attribution, assessed along two dimensions: (1) Citation Recall, determining if the output is entirely supported by the cited documents, and (2) Citation Precision, assessing if each citation supports its corresponding statement. Evaluation is conducted by TRUE (Honovich et al., 2022), a T5-11B model fine-tuned on a collection of NLI datasets to automatically examine the entailment of cited documents and the model generation. Additionally, to capture a holistic measure of citation quality, we also report the Citation F1, the harmonic mean of citation precision and recall:
F1 = 2 · citation precision · citation recall citation precision + citation recall , (6)
Correctness. For the ASQA dataset, correctness is quantified using exact match recall (EM Rec.) by checking whether the short answers are exact substrings of the generation. Regarding the ELI5
dataset, correctness is measured through claim recall (Claim), evaluating whether the model’s response entails the ground truth sub-claims. For the QAMPARI dataset, correctness is assessed using exact match precision (Prec.) and top-5 exact match recall (Rec.-5) — considered 100% if the prediction includes at least five correct answers.

heading: 4.3 Baselines
text: We compare our method with three types of baselines: prompting-based, post-hoc retrieval, and training-based.

heading: 4.3.1 Prompting-based Methods.
text: We directly prompt LLMs using few-shot demonstrations, each consisting of a query, the top 5 relevant retrieved documents, and an answer with inline citations. Our experiments encompass a range of LLMs, from foundational models to supervised fine-tuning (SFT) LLMs. For foundational LLMs, we select GPT-3.5-Turbo4 as the representative closed-source model, recognized for its notable performance. Among the open-source foundational LLMs, we focus on the LLaMA-2 series including LLaMA2-7B, LLaMA2-13B, and LLaMA2-70B, as well as the Mistral series, which spans from Mistral-7B (Jiang et al., 2023) to Mistral-8x7B-
4Specifically, we utilize gpt-3.5-turbo-1106 version
MoE (Jiang et al., 2024). Regarding SFT LLMs, we select the SFT counterparts of the open-source foundational LLMs we used. Detailed prompting settings can be found in Appendix C.

heading: 4.3.2 Post-hoc Retrieval Methods.
text: Following Gao et al. (2023b), we first instruct LLMs to answer the given query in a closed-book setting, and then integrate citations in a post-hoc manner. For each generated statement, we employ GTR (Ni et al., 2022) to identify and cite the most relevant document from the top 100 retrieved documents. We utilize the same models mentioned in prompting-based settings for this baseline.

heading: 4.3.3 Training-based Methods.
text: Self-RAG (Asai et al., 2023) Self-RAG trains the LLM to learn to adaptively retrieve passages ondemand and enable it to reflect on its generation to further improve generation quality and attributions.
VANILLA-SFT We directly employ supervised fine-tuning to train the LLM on our generated training data. Given a query and corresponding documents, the LLM is required to directly generate answers with citations.

heading: 4.4 Implement Details
text: We implement FRONT with different sizes of foundational models (LLaMA-2-7B and LLaMA2-13B) to evaluate its effectiveness. During the evaluation, FRONT utilize the same retrieval settings as those outlined by Gao et al. (2023b). Additional details of training and evaluation settings can be found in Appendix D.

heading: 5 Results and Analysis

heading: 5.1 Overall Results
text: Simply supervised fine-tuning can boost citation quality. As shown in Table 1, teaching LLMs to generate responses with citations via supervised fine-tuning significantly enhances citation quality, demonstrating substantial improvements over both prompt-based and post-hoc retrieval baselines across all datasets. Specifically, with LLaMA-27B, VANILLA-SFT led to substantial gains in citation F1 scores over prompting: ASQA (17.55 → 65.61), ELI5 (4.54 → 41.15), and QAMPARI (6.19 → 21.35). These gains highlight the effectiveness of our training data generation pipeline.
FRONT achieves significant performance gains and surpasses ChatGPT. While VANILLA-SFT demonstrates strong performance, it still shows notable discrepancies compared to leading opensource LLMs, such as Mixtral-8×7B-Instruct (e.g., 41.15 vs. 45.74) and ChatGPT (e.g., 41.15 vs. 48.22) on the ELI5 dataset. FRONT not only bridges these gaps but also establishes significant leads across all datasets. Specifically, using LLaMA-2-7B, FRONT comprehensively outperforms ChatGPT, achieving increases of 3.32%, 18.04%, and 21.28% in citation quality on the ASQA, ELI5, and QAMPARI datasets respectively. This performance underscores the effectiveness of FRONT in enhancing attribution capabilities.
FRONT exhibits scalability with model size. As illustrated at the bottom of Table 1, the performance of FRONT in terms of citation quality shows notable improvements when scaling from 7B to 13B. Specifically, we observe improvements of 3.23% in ASQA, 4.97% in ELI5, and 1.33% in QAMPARI. This upward trend underscores the scalability of FRONT with increasing model size, demonstrating the potential of FRONT in leveraging the increased capabilities of larger LLMs for further performance gains.
FRONT demonstrates remarkable generalization. Compared to the varied queries and answer types present in the ALCE benchmark, our training data, derived exclusively from the AQuAMuSe dataset (Kulkarni et al., 2020), exhibits outof-domain characteristics. Nonetheless, FRONT demonstrates superior citation quality, affirming its exceptional ability to generalize across diverse query types and retrieval documents. Additionally, while not specifically optimized for correctness, FRONT also showcases modest improvements in this metric over VANILLA-SFT on the ASQA and QAMPARI datasets. However, FRONT encounters lower Rec.-5 on the QAMPARI dataset, likely due to the nature of its answers, which consist of concatenated entities, diverging significantly from our training data.

heading: 5.2 Ablation Study
text: We conduct ablation studies to verify the effectiveness of different components proposed in FRONT.
Effects of Data Generation Pipeline. As illustrated in §5.1, simply SFT achieves strong performance, underscoring the high quality of our training data. Furthermore, data filtering, a crucial component of our data generation pipeline, plays a pivotal role in ensuring the quality of the generated data by filtering out queries that yield non-informative answers or fail to meet attribution criteria. To validate the effectiveness of our data filtering strategies, we conducted experiments comparing models fine-tuned on both pre-filtered and post-filtered data. The results, depicted in Figure 4, confirm that models trained on filtered data exhibit a notable improvement in citation quality over those trained on unfiltered data, achieving superior attribution performance with reduced data volume.
Effects of Grounding Guided Generation (G3). G3 empowers LLMs to first select relevant finegrained quotes, which subsequently guide the generation process. These quotes can provide finegrained supervision signals for attributed text generation. To evaluate the effectiveness of G3, we conduct an ablation study comparing it against two variants with distinct training recipes. Given that FRONT consists of two stages, we refer to the model trained only during the first stage (without consistency-aware alignment) as SELF-GUIDE. We first compare SELF-GUIDE against VANILLA-SFT (w/o Ground), which is trained to directly generate responses with citations, bypassing the grounding step. The ablation study, detailed in Table 2, reveals that models incorporating grounding guidance significantly outperform their VANILLA-SFT counterparts that lack such grounding mechanisms. This highlights the crucial role of grounding in bolstering attribution.
Moreover, we explore an alternative variant of grounding guidance. Considering that SELFGUIDE leverages the model itself to both select grounded quotes and generate attributed answers in an end-to-end paradigm, a natural variant involves breaking down this task into two distinct stages. In this variant, ChatGPT is tasked with extracting grounded quotes. Subsequently, a separate model is trained to utilize these grounded quotes, along with the query and retrieval documents, to directly output the response and citations. This variant, referred to as PROMPT-GUIDED, integrates grounded quotes into the prompt to guide the gen-
eration process. Experiments conducted on the ELI5 dataset using the LLaMA-2-7B model show that SELF-GUIDE outperforms PROMPT-GUIDE. Results depicted in Figure 5 indicate that training models to self-generate grounded quotes before generating attributed responses is more effective than simply incorporating these grounded quotes into the prompt.
Effects of Consistency-Aware Alignment (CCA). The primary goal of CCA is to enhance the consistency between grounded quotes and attributed answers, thereby alleviating hallucinations and achieving more precise attribution. To evaluate this, we compare models that underwent only the G3 stage (SELF-GUIDE) with those further enhanced through the CCA stage (FRONT). As illustrated in Table 2, FRONT significantly improves citation quality over SELF-GUIDE, demonstrating the effectiveness of the CCA stage in enhancing attribution.
Furthermore, to assess CCA’s impact on reducing hallucinations, we utilize QAFactEval (Fabbri et al., 2022), a widely used metric for factual consistency, which scores the consistency of model responses to given documents on a scale from 0 to 5, with higher scores indicating greater faithfulness. Specifically, we analyze the performance
of leading open-source models and two variants of FRONT and SELF-GUIDE on the ELI5 dataset. As shown in Figure 6, FRONT produces more faithful outputs than SELF-GUIDE, significantly reducing hallucinations.
Effects of Training Data Scale. We analyze the impact of the data scale on model performance across two training stages. In particular, we randomly sampled 2k, 4k, 6k, and 8k instances from our full training data across two distinct training stages. These subsets were then utilized to finetune various 7B model variants, enabling a comparative analysis of performance based on data scale. Results are shown in Figure 7, which indicates that increasing data size shows significant enhancements in citation quality, indicating a positive correlation between data size and model performance. As FRONT implements an automated procedure capable of generating high-quality attributed data and constructing contrastive supervision from weak and strong LLMs, it holds the potential for continuous performance improvements.

heading: 6 Human Evaluation
text: Given the significant impact of the quality of grounded quotes on fine-grained verification for users, we conducted a human evaluation to assess the quality of grounded quotes at different stages of our framework: (1) Quotes extracted by ChatGPT from 50 sampled data points during the G3 stage. (2) Quotes generated by FRONT-7B across three datasets, with 50 data points sampled from each.
We engaged four annotators, each with relevant expertise and holding at least a bachelor’s degree. The quality of quotes was evaluated on two dimensions: authenticity and helpfulness. Authenticity (a binary scale of 0/1) refers to whether the quotes genuinely originate from the corresponding documents (quotes that are hallucinated or mismatched with the corresponding document ID are considered inauthentic). Helpfulness (5-point Likert scale) refers to the degree to which the quotes are beneficial in addressing the query. The results in Table 3 represent the average scores for all quotes within each model response, with two annotators evaluating each response to ensure reliability.
Furthermore, to evaluate the consistency of quote quality annotations, we computed the interannotator agreement using Fleiss’ Kappa coefficient. The obtained Kappa coefficient of 0.82 indicates a high level of agreement among annota-
tors. The results of the human evaluation indicate that both quotes extracted by ChatGPT and those generated by FRONT are of high quality, further substantiating the effectiveness of our method.

heading: 7 Conclusion
text: In this work, we present FRONT, a two-stage training framework designed to equip LLMs with finegrained attribution capabilities. FRONT enables LLMs to initially select supporting quotes, which then guide the generation process. By further enhancing the consistency between the grounding and generation process via preference optimization, these supporting quotes can serve as fine-grained citations. Through comprehensive experiments, FRONT has demonstrated its ability to generate superior grounded responses and highly supportive citations. Further analysis shows that FRONT significantly reduces hallucinations and benefits user verification.

heading: 8 Limitation
text: Our study presents several limitations worth noting. Firstly, the validation of our framework is predominantly conducted on models of sizes 7B and 13B, leaving the exploration of larger models, such as LLaMA-2 70B due to computational constraints. Secondly, our framework relies on a prior retrieval process, wherein relevant documents are retrieved at one time. The incorporation of adaptive retrieval, enabling more dynamic interactions with LLMs, could potentially enhance performance. We leave it for future research. Lastly, evaluating the correctness of long-form question answering presents inherent challenges, leading our framework to primarily enhance citation quality, with modest advancements in correctness. Therefore, we advocate for the development of more robust metrics capable of accurately assessing the correctness of long-form QA responses, paving the way for future work.

heading: Acknowledgements
text: We appreciate Yuchun Fan for providing valuable suggestions. Xiaocheng Feng is the corresponding author of this work. We thank the anonymous reviewers for their insightful comments. This work was supported by the National Key R&D Program of China via grant No. 2021ZD0112905, the National Natural Science Foundation of China (NSFC) (grant 62276078, U22B2059), the Key R&D Program of Heilongjiang via grant 2022ZX01A32, the International Cooperation Project of PCL, PCL2022D01 and the Fundamental Research Funds for the Central Universities (Grant No.HIT.OCEF.2023018).

heading: A Details of Data Generation Pipeline
text: A.1 Data Statistic
Table 4 presents the statistics of the data automatically generated by our data generation pipeline. In total, we collected 8,098 questions from the Natural Questions (NQ) dataset, of which 5,667 questions were gathered from those with long-form answers, and 2,431 questions were collected from those with short-form factoid answers.
For questions requiring long-form answers, we initialized our query source with the AQUAMUSE dataset (Kulkarni et al., 2020), which consists of high-quality queries specifically designed for longform responses within the NQ dataset, recognized as “good” by the majority of NQ evaluators. In this way, utilizing a refined and superior quality query set laid a robust groundwork for our training data generation, streamlining the data filtering process. For factoid queries that necessitate short-form answers, we directly sampled from the original NQ dataset, leveraging its abundance and inherently high quality.
During the data generation process, our initial query set comprised 7,725 queries requiring longform answers and 4,000 queries necessitating shortform answers. After a two-stage data filtering
process, we retained 5,667 and 2,431 queries, respectively. Additionally, we calculated the average length of answers and the average number of citations generated for various types of queries within our dataset, as shown in Table 4.
A.2 Details of Data Filtering
We trained our Attributed Discriminator using the manually annotated data provided by Liu et al. (2023), which is sampled from real generative search engines. Each statement and its cited document have been meticulously annotated for attribution, categorized into three types: complete support, partial support, and no support. For training, we utilized a dataset of 8,834 instances, comprising 6,415 instances of complete support, 1,552 of partial support, and 867 of no support. The discriminator initialized with LLaMA-2-7B, was trained with a maximum sequence length of 512. We trained it for 3 epochs, with a total batch size of 128, and a peak learning rate of 2e-5, incorporating 3% warmup steps, followed by a linear decay.
During the data filtering stage, we first break down the automatically generated attributed answers into statement form and use the trained discriminator to annotate the attribution between each statement and its cited documents. Specifically, we assign different attribution scores to each statement s based on its attribution relationship with cited documents d, as shown in Equation 7. Consequently, for each attributed answer, we can calculate its average attribution score. Attributed answers with an average attribution score below 0.8 are filtered out. The threshold of 0.8 was determined through preliminary testing on the development set, for which we manually annotated 100 samples to ensure the effectiveness of our filtering criteria.
r(s)=    1, Dis(s, d) = complete support 0.5, Dis(s, d) = partial support
0, Dis(s, d) = no support (7)

heading: B Details of Evaluation Metrics
text: In addition to evaluating citation quality and correctness, the ALCE benchmark includes a broader set of dimensions, such as fluency, ROUGE-L, and generation length.
Fluency We evaluate the fluency of the generated response using MAUVE (Pillutla et al., 2021). Notably, we calculate fluency only for the ASQA and ELI5 datasets, omitting it for QAMPARI, as the response in QAMPARI typically consists of lists of short answers. A relatively high MAUVE score indicates that the generation is sufficiently fluent.
ROUGE-L In addition to evaluating the correctness of the model-generated content, we employ ROUGE-L to assess the overall quality and textual coherence of the responses.

heading: C Prompts
text: C.1 Prompts for Prompting-based Methods
Following Gao et al. (2023b), we adopt the vanilla prompting strategy for its simplicity and effectiveness. Specifically, the prompts vary according to the type of data within the ALCE benchmark. For long-form QA datasets such as ASQA and ELI5, the prompt format is detailed in Table 5. For the short-form QA dataset QAMPARI, the format is outlined in Table 6.
C.2 Instructions for FRONT
During the training process, we follow the instruction format of Alpaca5. Specifically, we employ varied instructions for different question types, as delineated in Table 7 for long-form questions and Table 8 for short-form questions.

heading: D Experimental Details
text: D.1 Training Details of FRONT
The training of all models is executed on 4 Nvidia A100 GPUs, each with 80GB of memory, leveraging the Deepspeed (Rasley et al., 2020) and HuggingFace Accelerate libraries (Gugger et al., 2022) to conduct multi-GPU distributed training. Given the long nature of the inputs, the maximum token length is set to 2,048 tokens.
During the grounding guide generation stage, models are trained for 5 epochs with a total batch size of 128, a peak learning rate of 2e-5 with 3% warmup steps followed by a linear decay. During the contrastive alignment stage, we set the β to 0.1 and continued training for two additional epochs. Specifically, During inference, we use the vllm
5https://github.com/tatsu-lab/stanford_alpaca/ tree/main
framework (Kwon et al., 2023) for efficient inference. The hyperparameters are set as illustrated in Table 9.
D.2 Retrieval Settings
During the evaluation, we adopt the same retrieval settings as specified by Gao et al. (2023b). For the ASQA and QAMPARI datasets, we use the dense retriever GTR (Ni et al., 2022). For the ELI5 dataset, we employ the sparse retriever BM25.

heading: E More detail about Ablation Study
text: E.1 The Effect of Training Data Scale.
We examine how model performance varies with changes in data scale, as depicted in Figure7. The upper part of the figure illustrates the impact of the training data scale on citation quality during the Grounding Guided Generation training stage, with datasets ASQA, ELI5, and QAMPARI represented from left to right. Similarly, the lower part of the figure describes the influence during the Consistency-Aware Alignment training stage.
E.2 The Generalization Across Model Architectures.
FRONT demonstrates exceptional generalization capabilities across various foundational model architectures. Specifically, transitioning the foundational model from LLaMA-2-7B to the stronger foundational model, Mistral-7B, results in even greater performance enhancements as shown in Figure 8. This further underscores the broad applicability and generalizability of FRONT.
E.3 The effect of β in Consistency-Aware Alignment Training Stage
In the Consistency-Aware Alignment Training Stage, the β parameter in Direct Preference Optimization (DPO) controls the strength of the Kullback-Leibler penalty, typically set within the range of 0.1 to 0.5. A higher β value indicates a preference for the policy model’s training process to remain closer to the initially referenced model. In extreme cases, as β → 0, we ignore the constraints imposed by the reference model. This setting aims to balance the model’s ability to adapt to new training signals while maintaining the stability of the learned behaviors from the reference model.
Subsequently, we trained five variants by adjusting β from 0.1 to 0.5 on the model previously
trained with G3 to explore the impact of the hyperparameter β on attribution quality. We evaluated these variants on the ASQA and ELI5 datasets, and the experimental results are shown in Figure 9.
The experimental results indicate that as β increases, the model’s performance on attribution gradually decreases. This observation suggests that the first stage of G3 might introduce a noticeable inconsistency between grounding and attribution. With higher β values, the model struggles to escape the constraints of inconsistent attributed answers, leading to a reduction in attribution quality as β increases.

heading: F Full Results
text: We present the comprehensive results of our experiments in Tables 10, 11, and 12. Beyond the evaluation metrics related to Correctness and Citation, we
adhere to the evaluation framework established in (Gao et al., 2023b). For long-form QA datasets like ASQA and ELI5, we also report metrics related to Fluency, ROUGE-L, and average response length. Specifically, we use MAUVE (Pillutla et al., 2021) to evaluate the fluency of the model response. For datasets like QAMPARI, where answers are composed of concatenated entities, we calculate the average number of predicted entities.

heading: Future Work
text: We leave it for future research. Lastly, evaluating the correctness of long-form question answering presents inherent challenges, leading our framework to primarily enhance citation quality, with modest advancements in correctness. Therefore, we advocate for the development of more robust metrics capable of accurately assessing the correctness of long-form QA responses, paving the way for future work