SelfCite: Self-Supervised Alignment for Context Attribution in Large Language Models
Authors: Yung-Sung Chuang
Benjamin Cohen-Wang
Zejiang Shen
Zhaofeng Wu
Hu Xu
Xi Victoria Lin
James R. Glass
Shang-Wen Li
Wen-tau Yih
Venue: ICML 2025 poster (2025)
Keywords: Large Language Models
LLMs
Alignment
Preference Optimization
Context Attribution
Citation
Primary area: deep_learning->large_language_models
TL;DR: We designed a self-supervised reward to align LLMs for generating better citations to attribute the context when answering to questions, without human supervision.

Abstract: We introduce SelfCite, a novel self-supervised approach that aligns LLMs to generate high-quality, fine-grained, sentence-level citations for the statements in their generated responses. 
Instead of only relying on costly and labor-intensive annotations, SelfCite leverages a reward signal provided by the LLM itself through *context ablation*: If a citation is necessary, removing the cited text from the context should prevent the same response; if sufficient, retaining the cited text alone should preserve the same response. 
This reward can guide the inference-time best-of-N sampling strategy to improve citation quality significantly, as well as be used in preference optimization to directly fine-tune the models for generating better citations. 
The effectiveness of SelfCite is demonstrated by increasing citation F1 up to 5.3 points on the LongBench-Cite benchmark across five long-form question answering tasks. The source code is available at https://github.com/facebookresearch/SelfCite.

SelfCite: Self-Supervised Alignment for Context Attribution in Large Language Models
Yung-Sung Chuang Xi Victoria Lin? 2 James Glass!
Benjamin Cohen-Wang Shannon Zejiang Shen Zhaofeng Wu Hu Xu² Shang- Wen Li Wen-tau Yih 2
Abstract
Even when provided with accurate context, models may misinterpret the data or include details that are not supported by the context (Shi et al., 2024; Chuang et al., 2024).
We introduce SelfCite, a novel self-supervised ap- proach that aligns LLMs to generate high-quality, fine-gree grained, sentence-level citations for the state- ments in their generated responses. Instead of only relying on costly and labor-intensive annota- tions, SelfCite leverages a reward signal provided by the LLM itself through context ablation: If a citation is necessary, removing the cited text from the context should prevent the same response; if sufficient, retaining the cited text alone should pre- serve the same response. This reward can guide the inference-time best-of-N sampling strategy to improve citation quality significantly, as well as be used in preference optimization to directly fine-tune the models for generating better cita- tions. The effectiveness of SelfCite is demon- strated by increasing citation F1 up to 5.3 points on the LongBench-Cite benchmark across five long-form question answering tasks. The source code is available at https: htt//ggtttub com/ facebook
Although completely eliminating hallucinations remains dif- ficult, existing approaches have sought to enhance the relia- comumioonnn-oooolly
bility of LLMs by providing context referred to as citations-which are fine-grained references to relevant evidences from the context, alongside generated responses for user verification (Menick et al., 2022; Slobod- kin et al., 2024; Zhang et al., 2024). While they have shown promise in generating citations, an outstanding challenge their reliance on annotated data either from human (Menick et al., 2022; Slobodkin al., 2024) costly proprietary APIs (Zhang et al., 2024) to train models to generate ci- tations. Collecting annotations can be time-consuming or costly, especially with long-context documents.
To address this challenge, we introduce SelfCite, a novel alignment approach designed to autonomously enhance the quality of citations generated by LLMs without the need for any amnotations in the alignment process. Drawing inspira- tion from model interpretability techniques (Lei et al., 2016; Cohen- Wang et al., 2024), SelfCite leverages the inherent capabilities of LLMs to provide feedback through context ablation a process to evaluate the necessity and sufficiency of a citation. If removing the cited text prevents the LLM from assigning high probability to the same response, we can infer that it necessary for the LLM. Conversely, if the response remains highly probable despite removing all context other than the cited text, this indicates that the ci- tation is sufficient for the LLM to make the claim. This self-evaluation mechanism enables SelfCite to calculate a reward signal without relying on the annotation processes.
1. Introduction
Assistants built using large language models (LLMs) have become ubiquitous in helping users gather information and acquire knowledge (OpenAI, 2022; 2023). For instance, when asked about recent news, an assistant can read through dozens of relevant articles potentally more than a user could comb through themselves-and use these articles as context provide a clear, specific answer the user's query. While this ability can greatly accelerate information gathering, LLMs often produce hallucinations- conteettttettteee that sounds plausible but is actually fabricated (Ji et al., 2023).
Building on this intuition, we design a reward that can be cheaply computed by the LLM itself, composed by prob- ability drop and probability hold in context ablation. By integrating this reward function into best-of-N sampling strategy. SelfCite achieves substantial improvements in ci- tation quality. Furthermore, we employ this reward for preference optimization using SimPO (Meng et al., 2024), which not only maintains these improvements but also elim- inates the need for the computationally expensive best-of-N
Massachusetts Institule of Technology, Cambridge, MA 02139, USA 'Meta FAIR, USA. Correspondence to: Yung-Sung Chuang mrngsuun@@mttedu...
Proceedings of the 42nd International Conference Machine Learning. Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

SelfCite: Self-Supervised Aligument for Context Attribution in LLMs
sampling. We outperform the previous state of the art on the
sampled from the LM PLM as follows:
LongBench-Cite benchmark (Zhang et al., 2024) by up to 5.3 points in Fl scores, and showing a promising direction to bootstrap the citation quality from LLMs via self-rewarding.
ri PLM (: -1:--., Gc): e,, P. - 0i-1,
6 ~ PLM (: [1....,.C.. Q, 01,
0,, 01,.... , 61-17
2. Method
The objective of optimizing the LM is ensure that the citation sequence e, accurately reflects the evidence from the context that supports the generation of rs. In the SFT setting (Zhang et al., 2024), the probability of a "ground truth" annotated responses and citations {fi, e,,,..,. ês} will be maximized, given the imput C' and Q. but it is not trivial to do further alignment with feedback after the SFT data is used up. To achieve this, we introduce SelfCite that can evaluate the quality of these citations based context ablation reward for further preference optimization.
In this section, we describe the SelfCite framework. We begin by introducing the task of generating responses with context attributions (2.1), referred to as citations for brevity. We then design reward for providing feedback on citation quality without human annotations (2.2) as illustrated in Fig. 1. Finally, we discuss two approaches for utilizing this reward to improve citation quality: best-of-N sampling (2.3) and preference optimization (2.4).
2.1. Problem Formulation
2.2. Sepe-Suue Reward via Context Ablation
We first formalize the task of generating responses with context attributions and the metrics to self-evaluate context attributions within the SelfCite framework, inspired by pre- vious papers (Zhang al., 2024; Cohen-Wait et al., 2024) but adapted to our proposed self-supervised reward.
We measure the quality of citation sequence es by the changes in the LM's probability of generating r; when the cited sentences are either removed from or isolated within the context. To simplify the notation, let all the cited context sentences be E, = {ce] Cej...., Ce." J. We define two key metries: necessity score and sufficiency score, and finally combine them into the final reward, as shown in Fig. 1.
Setup. Consider employing an autoregressive language model (LM) to generate a response to a specific query given a context of relevant information. Specifically, given an LM PLM, let (t, .ty...,,--1) denote its output distribution over the next token based on sequence of preceding tokens , Next, let C' represent the context of relevant information. This context is partitioned into |C| sentences: C1, ....... CiCI- Each sentence Cj is prepended with unique identifier (e.g., sentence index j) as a way for the model to reference the sentence when generating cita- tions. The context C' followed by query Q, question or instruction for the model. A response R is then sampled from the model PLM-
Necessity Score: Probability Drop. This metric quanti- fies the decrease in the probability of generating r; when the cited sentences in E, are all removed from the context operator). Formally, it is defined as:
(denoted as set minus \
Prob-Drop(e:) = log C) - log PLM (ra C ()))...
To keep the equation concise, we ignore Q and {ri, fi-, in the equation, but they are staying in the context history when computing the probabilities. larger probability drop indicates that the removal of E; sig- nificantly diminishes the likelihood of generating r; thereby validating the necessity of the cited evidence.
Generating Responses with Context Attributions. In SelfCite, following prior work on generating responses with context attributions (Zhang et al., 2024), each state- ment r; in the response R is followed by a citation se- quence e, consisting of the identifiers of sentences from the context C. Thus, the entire response sequence R is fri,e1, 72, c2,..., rr,,s,,)) e, where S is the total number of generated statements. The citation e, is intended to ref- erence sentences that support the generation of ri. For- mally, for each response statement r,, the model outputs a citation sequence e; = fej, ? .?....,c" e,"), where each of E (1,2,...,,CC)) corresponds to a specific sentence num- ber in the context C, and m sentences are cited. Note that this citation sequence may be empty. The entire response R consisting of statements r; followed by citations e; is
Sufficiency Score: Probability Hold. Conversely, this metric measures if the probability of generating r; is still kept large when only the cited sentences are kept in the context, effectively testing the sufficiency of the citation to support the response statement. Formally:
Prob-Hold(e), (e.)) = log PLM PLM (ra E.) log PLM CC).
A more positive value of probability hold indicates that the cited sentences alone are sufficient support the generation of ri. while removing all the other irrelevant context. Please note that the values of probability drop or hold can be either positive or negative. For example, if the citation is not relevant to r; or even distracting, it is possible for p(r, E,) to be lower than p(r: C).
2

---

SelfCite: Self-Supervised Alignment for Context Attribution in LLMs
Context (C)
cactos a member of the plane family Cettrose spines provide protection from bethiccess and cemmur lage
Query (Ch م
SelfCite Reward
Why do cacti have spines!
O ritel
Necessily Score Sufficiency Score
PeM P 0.97
noid ouler contavvaiion They...
Necosaily
Contect without cited sentences (CTVF)
Queey (()) رf
Prolubility day 0.97 00.01
the plant family
evval)) Prob, of statoment i with CVE S RPP
large drop means the citziion necessory
Why do cocti have spines!
response (8)
Paatge S. 0.01
spirers
mi multtple reasons... 35
Context with only clted sentences (A)
deffenee meciaanimm rpaine
Sufficiency Score:
Query ((2) }
horhtworrs and water conservatinn...
eeval]] Prob, of stteemnt ri with Q
Prolubility huld 0.96-
Spiness prodide
Why xpirae?
Ptacee p
More postiive value means the ciaation sufficient
horbhvores comouriage spectes
assis water conservttion
Ultation E is gererated for
expeeilic statement
Figure 1. The SelfCite framework calculates rewards based on two metrics: (probability hold). First, the full context is used to generate response. Then, the framework evaluates the probability of generating the same response after (I) removing the cited sentences from the context and (2) using only the cited sentences in the context. The probability drop and hold are computed from these probability differences, and their sum is used as the final reward.
necessity score (probability drop) and sufficiency score
Final Reward. To comprehensively evaluate the necessity and sufficiency of the generated citations, we add the two metrics together, where the opposing terms cancel out:
Algorithm 1 SelfCite Best-of-N Sampling for Citations
Require: LM PLM: context C, query Q, response R, # of candidates N, length limit m T(-) counts # of tokens in a text, #(-) counts # of sentences in a citation. for r; E R do
Reward(es) (es) = Prob-Drop(ee) + (rob-Hold(es)) -logPMM (ra [.) log (ri(C)) E.).
(1)
Reward(k) = - 0o for = I....,NN
for h 1,.... do
The combined reward measures if the citations are both necessary and sufficient for generating the response ri.
~ PIM( ri, C,Q)
if TTTEE <= Lmax or #(c(t)) = then
(kward(k)
2.3. Best-of-N Sampling
= loog PMM (TilE( log 1oggiM(P:/CC) )aa
end end for
To leverage the self-supervised reward computed via con- text ablation, we employ best-of-N sampling strategy, which is common way to test the effectiveness of a re- ward design (Gao et al., 2023a; Lightman et al., 2024) as a performance oracle without any confounders from train- ing. For convenience, we first generate the full response R = {F1, (1,.,,, TT, Eg} which includes a set of statements (r:) paired with citations (ei), and then locate the position of ei, i.e., where the citation tags <cite>... .rreee..../ctteeee </cit>> are generated. Within the citation tags of ex we re- sample N candidate citation sequences (e("), : ( by making the model to continue the generation from (C, Q, ei,......... ri). and then select the best citation (et) that maximizes the combined reward metric, Eq. (1). The corresponding procedure is shown in Algorithm 1. After obtaining all the selected citations (ei...., e..). we replace the original citation sequence e; with the optimal citation ot for each response statement rú, while keeping the re- sponse statements (ri..... (ri..... rs} unchanged. This process is repeated for each statement in the response R to obtain the final, citation-improved output R" = (ri,ei..... rs, es). To prevent the model from citing too many sentences, we exclude the candidate e, if the cited text (E₂) is longer than L 384 tokens in total, unless E, are all from a single long sentence.
k* arg max; Reward(k)
et (k*)
end for return R' = .r. rs.es)
..........T...
2.4. Preference Optimization
Best-of-N sampling is a straightforward way obtain better citations, but the additional inference cost of generating candidates and reranking. Thus, we try to internalize the ability of generating better citations back to the LM itself.
Given documents and queries, we can prompt the LM to generate the responses along with the citations R = {ri, e .e...... Ts, eg}. By further applying best-of-N sam- pling, we can obtain new responses of the same statements but with better citations R {ri, ef,, .i.,...,s.. e5). Such preference data can be used in direct preference optimiza- tion (DPO) (Rafailov et al., 2024) to align the model based on the preference between the original outputs and im- proved outputs. Instead of using DPO, we choose its variant SimPO (Meng et al., 2024) here, as SimPO does not require
3

---

SelfCite: Self-Superviee Alignment for Context Attribution in LLMS
reference model and allows 2x memory saving for 25.6K long-context fine-tuning. Through this self-supervised pro- cess, which does not require ground-truth answers human annotations, the model learns to generate more accurate and contextually grounded citations on its own.
cut generatin longer citations instead of improving cita- tion quality. To prevent this, we introduce length balancing: if an original response has a shorter citation length than the best-of-N response, we insert random citations from nearby sentences. This encourages the model to focus on where to cite rather than simply citing more. Details are provided in Appendix C, with an ablation study in Section 4.2.
3. Experiments
We evaluate the effectiveness of SelfCite by applying the best-of-N sampling and preference optimization methods to existing models that generate responses with citations.
3.3. Evaluation
Benchmark. We evaluate our approach on LongBench- Cite (Zhang et al., 2024), a comprehensive benchmark specifically designed for long-context QA with citations (I-QAC). Given a long context C and query Q, the model must produce multi-statement answer with each statement cites relevant supporting sentences in C. Unlike chunk-level citation schemes (Gao et al., 2023b) which cites short para- graphs, LongBench-CCr -htt adopts sentence-leve citations to ensure semantic integrity and finer-grained evidence track- ing. LongBench-Cri Citc assesses two main aspects:
3.1. Model Details
We use LongCite-8B. the Llama-3. 1-8B model (Dubey et al., 2024) fine-tuned on LongCite-45K SFT data (Zhang et al., 2024) as the start point for both best-of-N sampling and preference optimization. We adopt the same text segmen- tation strategy from Zhang et al. (2024); each document is split into individual sentences using NLTK (Bird, 2006) and Chinese punctuations, and each sentence is prepended with unique identifier in <c] format. These identifiers serve as the cilation indices, enabling the model to cite rel- evant context right after the statements with the format of <statement>> fcentent I <cite>[B [ú] - i2] lig- 41 11l..... ........ /tt////ttettttteeeeeeeeeee
Whether each statement is fully sup- ported by relevant and only relevant sentences. GPT-4o measures citation recall (extent to which statement is fully or partially supported by the cited text) and ci- tation precision (whether each cited text truly supports the statement). These are combined into a citation Fl score. Additionally, we track averaaee citation length (tokens per citation) to promote fine-grained citations over unnecessarily long passages.
Citation Quality:
This format allows the model cite a single sentence (e.g. i = i2) or a span (e.g. d i2) efficiently within several tokens. The responses generated via top-p sampling (Holtzman et al., 2020) with p=0.7 and temperature=0.95. We set p=0.9 and temper- ature=1.2 when doing best-of-N sampling for the citation strings to increase the diversity. We set N=10 in all the experiments considering the limited diversity in citations.' !
Correctness: How accurately and comprehensively the response answers the query disregarding the cita- tions. This is scored by GPT-4o in a zero-/few-shot fashion based on the query and reference answers.
3.2. Preference Optimization
The benchmark contains five datasets, including single-doc QA MuliiFieldQA-enfaa (Bai et al., 2023), multi-doc QA HotporQA (Yang et al., 2018) and DuReader (He et al., 2018), one summarization dataset GovReport (Huang et al., 2021), and LongBench-Cher (Bai et al., 2024) which cov- ers diverse real-world queries with long contexts such as document QA, summarization, and coding.
LongCite-45K. Best-of-N sampling (Section 2.3) requires no training, so no training data is used. For preference optimization with SimPO (Section 2.4), we use 2K doc- ument-questir pairs from LongCite-45K (Zhang et al., 2024) as the training set but we do not use its ground-truth responses with high-quality citations for SFT. Instead, we generate model responses from the documents and queries, then apply best-of-N to refine citations. We label the orig- inal responses as rejected and replace their citations with BoN-refined ones to create the chosen responses, forming preference pairs to build the dataset for SimPO.
Baselines. SelfCite is compared with these baselines.
Zhang et al. (2024) propose the base- line of prompting LLMS with an one-shot example. This can be applied to proprietary models including GPT-4o (OpenAI, 2023), Claude-3-sonne (Anthropic, 2024), and GLM-4 (GLM et al., 2024), as well as open-source models including GLM-4-9B-chat (GLM et al., 2024), Llama- (88,00 ( Instruct (Dubey et al., 2024), and Sostral-Large-Insttue (Mistral, 2024).
Prompting:
Data Construction and Length Balancing Since best-of- N responses tend to have slightly longer citations, directly fine-tuning on them can lead the model to adopt a short-
Contributive context attribution: Contributive con- text attribution seeks to directly identify the parts of the context that cause the model to generate a par-
After deduplicating repeated citation candidates, on average there are only 4.8 candidates left per statement in the BoN experi- ment on LongBench--Ctee, with a standard deviation 3.2.
4

---

SelfCite: Self-Superviee Alignment for Context Attribution in LLMs
Table 1. Citation recall (R), citation precision (P), citation Fl (FI), and citation length evaluated on Long Cite benchmark. The best of our results are bolded. The best of previous state of the underlined. 'indicates the results taken from Zhang al. (2024).
Longbench-Chat
MultificldQA
HotpptQA
Dureader P
GovReport
Avg. F1
Citation
Model
R
P
F1
P
F1
R
P
F1
R
F1
R
P
F1
Length
Proprietary models
GPT-4o
46.7
53.5
46.7
79.0
87.9
80.6
55.7
62.3
53.4
65.6
74.2
67.4
73.4
90.4
79.8
65.6
220
Maade-3-sonnet
52.0
67.8
55.1
64.7
85.8
71.3
46.4
65.8
49.9
67.7
89.2
75.5
77.4
93.9
84.1
67.2
132
GI.M-4
47.6
53.9
47.1
72.3
80.1
73.6
47.0
50.1
44.4
73.4
82.3
75.0
82.8
93.4
87.1
65.4
169
Open-source models
GLM-4-9B-chat
25.9
20.5
16.7
51.1
60.6
52.0
22.9
28.8
20.1
45.4
48.3
40.9
5.7
8.2
6.3
27.2
96
1-mmn-3.--BB-Instrree
14.1
19.5
12.4
29.8
44.3
3166
20.2
30.9
20.9
22.0
25.1
17.0
16.2
25.3
16.8
19.7
100
25.8
32.0
23.2
53.2
65.2
53.9
29.6
37.3
28.6
38.2
46.0
35.4
53.4
77.5
60.7
40.4
174
Mistral-Large- Instruct
19.8
23.9
19.0
71.8
80.7
73.8
34.5
40.9
32.1
58.3
67.0
60.1
67.9
79.6
72.5
51.5
132
Contributive context attribution (with Llama-3.
ContexiCite (32 calls)
56.7
76.8
58.0
76.1
87.2
78.9
40.5
54.7
43.9
58.0
82,4
65.0
67.1
88.8
75.6
64.3
92.7
ContextCite (256 calls)
63.5
83.1
64.7
78.8
89.8
81.8
46.5
60.8
49.2
61.7
89.1
70.1
69.1
93.5
78.8
68.9
100.8
Fine-tuned models
LongCite-9B' LongCite-888 8B' SinPO w/ NIJ Rewards
57.6
78.1
63.6
67.3
91.0
74.8
61.8
78.8
64.8
67.6
89.2
74.4
63.4
76.5
68.2
69.2
91
62.0
79.7
67.4
74.7
93.0
80.8
59.2
72.1
60.3
68.3
85.6
73.1
74.0
86.6
78.5
72.0
85
64.4
87.1
69.8
70.1
92.4
77.4
58.8
78.1
63.2
69.4
77.2
83.7
93
87.5
75.0
105.9
Ours: SelfCite
LongCite-8B (Our repro.)
67.0
78.1
66.6
74.8
90.7
79.9
60.8
77.9
64.1
67.1
87.2
73.7
81.6
89.3
84.5
73.8
83.5
BoN
68.4
81.3
71.2
76.1
92.8
81.2
67.2
81.0
68.8
76.9
87.6
92.4
89.3
77.5
93.4
SimPO
68.1
79.5
69.
75.5
92.6
81.0
69.4
82.3
71.5
72.7
91.6
78.9
86.4
92.9
89.1
77.9
105.7
+ SimPO then BoN
73,3
79.4
72.8
76.7
93.2
82.2
69.4
83.0
71.1
74.2
92.2
80.3
86.7
92.7
89.2
79.1
94.7
Llama-3.J 1.teu--..------- Inst-sstttt (fully self-supervixed setting)
+ SFT on ContextCite
52.3
70.6
56.5
79.1
90.5
82.0
54.5
72.3
56.3
54.9
79.0
61.6
63.7
84,9
72.3
65.7
83.0
BooN
54.8
67.6
58.1
80.4
90.5
83.0
58.3
70.0
57.5
57.6
79.0
63.1
67.2
84.8
74.6
67.3
80.4
SimPO
63.3
74.3
64.6
80.2
88.9
82.4
59.7
76.9
61.0
59.0
80.9
65 44
68.5
86.6
76.1
69.9
90.2
*SimPO then BoN
66.0
82.4
71.1
81.5
90.7
83.2
61.3
70.0
59.9
62.1
81.4
67.4
68.8
86.2
76.1
71.5
87.4
ticular statement. We consider ContextCite (Cohen- Wang et al., 2024), a contributive context attribution method that performs several random context ablations to model the effect of ablating different parts the con- text on generated statement. We use NLTK to split Llama-3.1 1--e-3..-------sse -nsttruttsssssssss s responses into statements, and then apply ContextCite with 32 and 256 times of ran- dom context ablations to get the citations, with the details described in Appendix B.
across the datasets, suggesting room for further gains. Our results surpass LongCite-8B/99 at similar citation lengths and outperform proprietary model prompting while produc- ing shorter citations.
To better contextualize the gains of our proposed reward, we additionally implement variant of SimPO using NLI-based citation precision/recall rewards from Huang et al. (2024a) by using the same training pipeline and initialization as our SimPO, modifying only the reward function (see details in Appendix E). As shown in row of SimPO w/ NLI Rewards, this baseline improves LongCite-8F on out of 5 datasets, but is still consistently outperformed by SelfCite. This result highlights that while NLI-based rewards are helpful, our SelfCite reward provides more accurate signal for optimizing citation quality.
Fine-tuned models: LongCite-8B and 9B released by Zhang et al. (2024), trained on LongCite-45K, fine- tuned from Ilama-3.1 1-B (Dubey et al., 2024) and GLM-4-9B (GLM et al., 2024), respectively. Addition- ally, we consider baseline of finetuning LongCite-88 using SimPO with the NLI rewards which resembles Huang et al. (2024a), with the details in Appendix E.
Besides the fine-tuned baselines, we additionally compare our method to ContextCite for reference, a method very dif- ferent from SelfCite-it does not directly generate citations, it estimates the importance scores of the context sentences after the response is generated (in Appendix B we show how to convert continuous importance scores into citations). Both SelfCite and ContextCite rely on the idea of context ablation, but our approach is significantly better. A key reason is that ContextCite estimates sentence importance from scratch using linear regression, while we rerank ex-
3.4. Main Results
Citation Quality. Table presents our main results. Our best-of-N sampling (BoN) consistently improves both cita- tion recall and citation precision across tasks, increasing the overall Fl score from 73.8 to 77.5. Using SimPO to internalize BoN's gains- gains-elinnnn eliminating the need for costly -aciiinvgaaaivesssseeeaaee a similar improvement, with an Fl of 77.9. Applying BoN again to the SimPO fine-tuned model further boosts F1 by 5.3 points to 79.1, the highest
BoN sampling- achieves
5

---

SelfCite: Self-Supervised Alignment for Context Attribution in LLMs
Table 2. Answer correctness when responding with or without ci-
is similar LongSFT-8B/9B (Zhang et al., 2024), which are ablation baselines fine-tuned LongCite-45k QA pairs but without the citation parts. The same observation still holds when starting from Llama-3, --8--Instuuu either SFT with ContextCite data or the further SimPO step do not change the answer correctness significantly. Under the same answer correctness, the additional "citations" can benefit the verifia- bility of the answers, enabling a user to easily double-check the answer, even in cases where the answers are wrong.
tations. ! indicates results taken from Zhang et al. (2024). The
header contains abbreviations for the same five datasets in Table I.
Model
Long.
Multi
Hot.
Dur.
Gov.
Avg
Answering without citations
LongSFT.8R'
68.6
83.6
69.0
62.3
54.4
67.6
LongSFT-9B
64.6
83.3
67.5
66.3
46.4
65.6
1--ma-3.1-8B-nnstree
66.0
83.7
65.8
62.8
66.1
68.9
Answering with citations
LongCite-BB (Our repro.)
67.6
86.7
69.3
64.0
60.4
69.6
SimPO
67.4
86.7
67.5
66.0
61.3
69.8
Llama-3.1-80 1.-mn.3.-8B--nttmmee + SFT on ContextCite
58.4
75.3
67.3
59.3
56.4
63.3
Chunk-level Citation Evaluation. Additionally, we eval- uate our methods on the traditional chunk-level citation benchmark ALCE (Gao et al., 2023b). However, due to the mismatch of data distributions and different task settings dur- ing training (sentence-level) and evaluation (chunk-level) we consider this as zero-shot evaluation, and the results are shown in Appendix F, due to the limited space.
58.8
83.4
65.8
57,8
57.5
64.6
SimPO
56.8
80.9
65.3
59.5
60.9
64.7
isting LLM-generated citation candidates, leading more efficient and accurate citation quality estimation.
Finally, we evaluate the latest released Claude Citations API, as shown in Appendix D that SelfCite achieves strong results very close to this commercial- API, validating the effectiveness of SelfCite.
4. Analysis
4.1. Ablation Study on Rewards
Fully Supervuue Setting. In our main experiment, we start from the Llama-3.1-8B model fine-tuned on the LongCite-45K SFT data, which effectively kick-starts its ability to generate structured citations for best-of-N sam- pling. The subsequent SimPO alignment stage is entirely self-supervised. We are also curious if it is possible to start from a fully self-supervised SFT model and then apply our self-supervised alignment after that. To begin with, we auto- matically generate IIK citation SFT data using ContextCite (see Appendix B for details) to replace the LongCite-45K annotations in the training data, shown in the results at the bottom of Table 1. We can see that SFT on ContextCite can achieve decent initial results (65.7 F1) but still far from LongCite-88 8B (73.8 F1). BoN helps improving F1 to 67.3. After SimPO training, it achieves 69.9 F1, and additionally applying BoN can boost its F1 by 5.8 to 71.5, significantly closing the gap to LongCite-SB, showing our alignment method not only improve the supervised models, but also enhance the models purely trained from self-supervision.
To better understand our final reward design, we explore var- ious reward strategies in the BoN sampling process. Here, all BoN candidates are pre-generated and fixed, the re- ward is the only factor affecting results. Table 3 presents our ablation results on HotpotQA, while citation lengths are computed across all LongBench-Cite datasets for di- rect comparison with Table 1. We evaluate four alterna- tive reward designs. BoN by LM log prob re-ranks can- didates simply by the probability of the citation string, <cite> [4] - i22 Lig - i4 ....... </cit>>, which is similar to beam search but less costly. We observe that this strategy slightly boosts recall while reducing precision, resulting in minor reduction in F1. BoN by max citation length always selects the candidates with the longest citations, i.e. citing the greatest number of sentences. Although it improves re- call, it significantly reduces precision from 77.9 to 73.6 and inflates the citation length from 83.5 to 139.8. By contrast, both BoN by Prob-Drop and BoN by Prob-Hold improve recall without sacrificing precision. Finally, by combining both Prob-Drop and Prob-Hold into our final SelfCite re- ward, we achieve the best outcome, increasing both recall and precision and a 4-point improvement in F1.
Answer Correctness. For best-of-N sampling, only the citation parts are modified, so the responses it generates to answer the questions are the same as those of the origi- nal LongCite-8B model, maintaining the same correctness. For the SimPO fine-tuned models, we test their answer cor- rectness by the evaluation in Zhang et al. (2024), which contains two settings: answering with/without citations. If answering with citations, the model will be prompted to generate answers with structured citations, making the task more complex, and the citation parts will be removed when evaluating the answer correctness. The results in Table 2 show that the SimPO fine-tuning does not change the cor- rectness of the LongCite-8B model much. The correctness
We also explored different token-length limits for citations in the bottom of Table 3, as discussed in Section 2.3. By default, we exclude candidates citing more than 384 tokens, unless the citation contains only a single sentence. Lowering the cap to 256 tokens slightly hurts FI. while raising it to 512 tokens has negligible impact. Completely removing length limits inflates citation length to 121.9 tokens and yields worse precision (79.3) but slightly improved recall (67.9). We also notice that the 256 length limit still outperforms the LongCite-8h baseline (66.4 vs 64. 1) while having almost
6

---

SelfCite: Self-Supervised Alignment for Context Attribution in LLMs
Table 3. Ablation study on HolpotQA citation recall, precision, and F1 (R, P. F1) and citation length for BoN decoding methods.
4.3. Training Size of SimPO
In prior study (Zhou et al., 2023), IK examples are suffi- cient to align user preferences effectively. Table 4 presents SimPO results with 1K to 8K examples. IK examples al- ready bring a moderate improvement, raising F1 from 64.1 I to 65.7. with gains in precision and recall. Using 2K exam- ples further boosts F1 to 71.5, while 4K leads to saturated improvement. However, at 8K examples, performance de- clines, and citation length rises 158.1. We attribute this to SimPO's 's off off-policy nature, especially because it lacks a reference model to constrain the output distributions to be similar to the collected data. As training steps grow, the model may drift from the collected data, potential overfitting to the biases in preference data. Thus, further fine-tuning may degrade citation quality. To address this, we show initial results from iterative SimPO in Section 4.6.
HA
Citation Length
Decoding Methods
R
P
F1
LongCite-Bee 8B (Our repro.)
60.8
77.9
64.1
83.5
+ BoN by LM log prob
62.7
75.5
63.4
74.6
+ BoN by max citation length
66.5
73.6
65.1
139,8
+ BoN by Proh-Drop
65.6
78.1
66.6
92.9
+ BoN by Prob-Hold
66.2
78.1
67.0
93.4
+ BoN by SelfCite
67.2
81.0
68.8
93.4
w/ lower length limit (256)
65.8
78.8
66.4
84.5
w/ higher length limit (512)
67.0
82.2
68.5
99.2
w/o length limit (oc)
67.9
79.3
68.1
121.9
Table Ablation study on HotpotQA citation recall, precision, and Fl (R. F1) and citation length for finetuned models.
HotpotQA
Citation Length
Fine-tuning Methods
R
F1
LongCite-8B 8B (Our repro.)
60.8
77.9
64.1
83,5
4.4. SimPO vs. SFT on Best-of-N responses
SimPO
69.4
82.3
71.5
105.7
SimPO BoN
72.0
82.7
72.9
126.9
We also show the effect of applying standard supervised fine-tuning (SFT) on the responses selected by best-of-N sampling, which is a simplified alternative of preference optimization. As the result shown in the last row in Table 4, SFT also improves the Fl score from 64.1 to 68.4, but it still falls behind 71.5 of SimPO. This result confirms that it is necessary to train the model via SimPO with preference data, which enables the model to distinguish between bad and good citations, and thus improve the citation quality.
SimPO w/ or w/o length balancing
w/ length balancing
69.4
82.3
71.5
105.7
w/o length balancing
64.4
62.9
60.5
152.9
+ SimPO w/ varying data sizes
IK examples
62.5
78.9
65.7
90.1
2K examples
69.4
82.3
71.5
105.7
4K examples
68.5
80.4
70.3
134.1
8K examples
64.6
79.5
65.9
158.1
+ SFT on BoN responses
68.8
77.3
68.4
98.7
SimPO by denoising perturbed citations
4.5. Off-policy Denoising Perturbed Citations
On original responses
40.5
50.5
41.6
88.8
We explored a purely off-policy alternative approach. Specif- ically, given model--geneaate response, we randomly shift its citation spans to create perturbed variants. SimPO train- ing pairs were then constructed by preferring the original ci- tation over the perturbed one, encouraging the model to "de- noise" citations by restoring their original spans. However, as shown at the bottom of Table 4, this approach degrades performance, both when applied to original and best-of-N responses. We attribute this to a mismatch between the train- ing data and the model's natural error distribution random shifts do not reflect typical citation errors, they fail to provide useful guidance for improvement.
On BoN responses
42.6
50.7
42.3
79.7
equally long citation length (84.5 vs 83.5), showing that the improvement of SelfCite correlates less with the citation length. Overall, using a 384-token limit achieves a good balance for short citation lengths and strong performance.
4.2. Citation Length Balance
As noted in Section 3.2, BoN selects slightly longer cita- tions, making it easy for model trained directly on BoN- preferred data to adopt the shortcut of generating longer ci- tations without improving quality. To counter this, we apply length balancing, injecting random citations into examples where length bias exists to equalize the number of cited sentences. Table 4 (see w/ vs. w/o length balancing) high- lights its critical role in length balaneing. Without length balancing, the model overextends citations (average length 152.9), leading to lower precision (62.9) and F1 (60.5). In contrast, enabling length balancing maintains high precision (82.3) and recall (69.4), achieving a better Fl of 71.5 while keeping citation length reasonable (105.7). These results confirm that length balancing prevents shorteut learning, ensuring the model truly learns to cite accurately.
4.6. Iterative Preference Optimization
It has been discussed that an on-policy alignment process can be beneficial to avoid reward exploitation (Bai et al., 2022) and maintains consistency between the generated data and the model's evolving output distribution. We thus experiment with iteratively performing SimPO, similar to the concepts of recent studies (Pang et al., 2024; Yasunaga et al., 2024). to maintain the consistency between the gen- erated data and the model's evolving output distribution.
7

---

SelsCite: Self-Supervised Alignment for Context Attribution in LLMs
LongCrte 60
Unma-3 1-80-neirad
Table 5. Average latency per example on LongRench-Cite (8
50
A100 GPUs, batch size 1, model parallel).
7B
70
Method
Avg Latency (s)
LongCite-8B
24.3
SelfCite BoN Sampling
149.0
SelfCite BoN Reranking
34.0
SelfCite SimPO model
26.2
(SFT
SaxPO
SinP
(SFT mmb)
San1O
Figure Iteratively applying SimPO for three iterations.
5. Related Work
Specifically, after fine-tuning with SimPO, we generate new dataset via BoN, which also 2K in size but not over- lapped with previous iterations. We continue training the model and repeat the process for three rounds. As shown in Figure 2, while the largest improvement occurs in the first round, improvements continue over three iterations, which further validates the reliability of our reward signal. Itera- tive SimPO is still not perfect since it remains an off- off-pooicy method. Given that our reward can he cheaply computed, we believe that on-policy methods like PPO (Schulman et al., 2017) could further enhance performance. We leave the exploration of such approaches for future work.
Citations for Language Models. Recent work has ex- plored various approaches to teaching language models to generate citations, including finettunigg with direct human feedback or annotations (Nakano et al., 2021; Menick et al., 2022; Slobodkin et al., 2024), rewards from external NLI models (Huang al., 2024a;b), and prompting-based meth- ods (Gao et al., 2022; 2023b) to explicitly incorporate rel- evant retrieved documents. Given the high cost of human annotation, Zhang et al. (2024) introduced CoF ("Coarse to Fine"), an automated multi-stage pipeline that simulates hu- man annotation. This approach leverages proprictary LLMs for query generation, chunk-level retrieval, and sentence- level citation extraction, achieving high citation quality through supervised fine-tuning. However, it depends larger proprietary models two proprietary APIs-GLM-4 for the LLM and Zhipu Embedding-v2 for retrieval with carefully designed prompting, effectively distilling the ca- pabilities these proprietary models into much smaller models in 8B/9B. In contrast, our SelfCite aims at com- pletely eliminating the reliance on annotations for citation, either from human proprietary APIs. Instead, our method enables small 8B model to assess citation quality itself using self-supervised reward signal from context ablation, effectively self-improving without external supervision. We additionally provide Table 9 to contrast the key differences between SelfCite and prior papers in Appendix G.
4.7. Latency Best-of-N
Table reports the average per-example latency on LongBench-Cite. As expected, Best-of-N (BoN) introduces additional latency due to the need to generate and rerank multiple citation candidates. In our setup, use N = 10 candidates, but the sampling time is not 10x longer than direct decoding. This is because we only re-sample short citation spans (typically 5-10 tokens), not the full responses, resulting in relatively lightweight sampling overhead.
However, the increased latency from BoN is not major concern, because SelfCite SimPO model also achieves the same performance as BoN in a single pass, without addi- tional latency. For scenarios requiring maximum efficiency, we recommend using the SimPO model directly.
Contributive Context Attribution. Besides being self- supervised, SelfCite also adopts the view that citations should reference the sources from the context that a model actually uses when generating a statement- known as con- tributive attribution (Worledge et al., 2023)-rather than any sources that merely support the claim. Our reward signal naturally aligns with this attribution framework, con- text ablation identifies the sources that cause the model to produce a statement. Existing contributive attribution methods for LLMs typically require extensive context abla- tions or other computationally expensive techniques, such as gradient-basee analysis during inference (Cohen- Waan et al., 2024; Qi al., 2024; Phukan et al., 2024). In contrast,
4.8. Qualitative Study
Finally, we examine an example that requires citing multi- ple context sentences to support a complex response. As shown in Table 6, the response integrates information from sentences 302, 303, and 306. Direct sampling (2) omits sen- tence 302 while incorrectly including 305. In contrast, the best-of-N candidate (1) correctly includes 302 and excludes 305, achieving slightly higher reward (0.578 vs. 0.547), demonstrating the effectiveness of our reward design. We also present candidates (3) and (4), which cite more irrele- vant sentences and miss key citations, leading to even lower rewards. Additional qualitative examples are provided in Appendix H.
pettpa:/ooperr bi giomddel www.ddel.ccc./ieeeeeee wwpppriciiinncc
8

---

SelfCite: Self-Supervised Aligument for Context Attribution in LLMs
Table 6. An example of differences in the citation from baseline vs BoN. Related information are highlighted in the context/response.
Sent. ID
Context Sentences (only showing a paragraph due to limited space)
302 (V)
In general, consumer advocates believe that any comprehensive federal privacy policy should complement, and not supplant, sector-specific privacy legislation or state-level legislation.
303 (V)
Finding a global consensus on how to balance open data flows and privacy protection may be key to maintaining trust in the digital environment and advancing interaational trade.
304 (X)
One study found that over 120 countries have laws related to personal data protection.
305 (X)
Divergent national privacy approaches raise the costs of doing business and make it harder for govermments to collaborate and share data, whether for scientific research, defense, or law enforcement.
306 (V)
A system for global interoperability in least trade-restrictive and nondiscriminatory way between different national systems could help minimize costs and allow entities in diffcrent jurisiitioons with varying online privacy regimes to share data via cross-border data flows.
Query
Please write a one-page summary of the above government report.
Response
[..]] The report concludes by noting that finding a global consensus on how to balance open data flows and privacy protection may be key to maintaining trust in the digital environment and advancing international trade, The report suggests that Congress may consider comprchensive privacy legislation and examine the potential challenges and implications of building system of interoperability between different national privacy regimes. [...]
(only single
statement due
space)
BoN Candidates
Citation Strings (green: correct; red: wrong)
Missing Citations
SelfCite Reward
(1) Best candidate
[302-303) [306-306]
0,578
(2) Direct sampling
[303-303] [305-306]
(302)
0.547
(3) Other candidate
[303-304] [308-308] 1310-311]
(302, 306)
0,461
(4) Other candidate
[303-303] [309-309] [311-311]
(302, 306)
0.375
SelfCite simply generate the citation tags, and refine citation candidates by preference optimization with reward signals from context ablations, effectively teaching the model to perform contributive context attribution itself.
6. Conclusion and Limitations
We present SelfCite, self- framework for align- ing large language models (LLMs) to generate more ac- curate and fine-grained citations. By leveraging LLMs' own output probabilities, SelfCite computes necessity and sufficiency rewards through context ablation, enabling pref- erence optimization without relying on external annotations from human or proprietary APIs. Applying such rewards in best-of-N (BoN) sampling and SimPO fine-tuning can signif- icantly improve the citation correctness on the LongBench- Cite benchmark, offering promising self-impnn improvingi direc- tion towards verifiable and trustworthy LLMs.
We also note that there is distinction between corrobora- sources that support claim,
tive Eraation-higglightet used in benchmarks like LongBench-Cite- Cetechh--itttann Cite-and contributive attribution, as emphasized in ContextCite. While SelfCite applies a contributive alignment method (via ablations) in the context of a corroborative evaluation framework, we find the two objectives to be at least partially aligned: cita- tions that genuinely influence the generation are often also semantically supportive. Although this alignment is not guaranteed, our empirical results show that enforcing con- tributive attribution leads to clear improvements on corrob- orative benchmarks, suggesting that current corroborative methods (e.g., LongCite) still have significant headroom for improvement- even under a slightly mismatched objective.
SelfCite also has limitations: 1) While achieving strong re- sults with SimPO, integrating other preference optimization or reinforcement learning (RL) algorithms, e.g., PPO (Schul- man al., 2017), remains under explored. However, prior work (Mudgal et al., 2024) shows that BoN closely approxi- mates the performance upper bound of RL, and we follow established practice (Gao et al., 2023a; Lightman et al., 2024) to mainly validate our rewards through BoN, and fur- ther verify it with SimPO fine-tuning. 2) SelfCite assumes access to model output probabilities, which may not be fea- sible for closed-source models. 3) While our framework improves the quality of citations already generated by LLMs, discovering unsupervised methods to kick-start L.MMs' abil- ity in generating structured citations from scratch remains an important direction for future research.
Self-Supervised Alignment and Reward Modeling. An- other relevant area is self- or weakly- supervisd approaches for aligning LLMs without human supervision (Kim al., 2023; Yuan et al., 2024), reducing the need for explicit hu- man feedback (Ouyang et al., 2022), or curating high-quality data for supervised fine-tuning (Zhou et al., 2023). SelfCite shares the same spirit by computing simple probability dif- ferences under context ablation as rewards, eliminating the need for additional annotation process.
9

---

SelfCite: Self-Supervise Alignment for Context Attribution in LLMs
Impact Statement
bilingual, multitask benchmark for long context under- standing. arXiv preprint arXiv:2308. 14508, 2023.
This paper introduces SelfCite, a self-supervised framework for improving citation accuracy in large language models (LLMs). Our method enhances the verifiability and trustwor- thiness of LLM-ggenrrted content by aligning citations with relevant supporting evidence in scalable manner, with- out relying on costly human annotations. By improving citation quality, SelfCite contributes to the broader goal of reducing misinformation and hallucinations in Al-generated responses. Ensuring that L.LMs provide accurate and prop- erly attributed information is particularly crucial in high- stakes domains such as healthcare, law, and journalism, where incorrect or unverified information can have signifi- cant real-world consequences. Overall, SelfCite aligns with the broader ethical goal of making machine learning sys- tems more transparent and accountable, reducing the risk of unchecked misinformation while maintaining the efficiency and scalability required for real-world applications.
Bai, Y. Lv, X., Zhang, J., He, Y., Qi, J., Hou, L., Tang, J., Dong, Y., and Li, J. Longalign: recipe for long con- alignment of large language models. arXiv preprint arXi:: 2024.
Bird, S. Nltk: the natural language toolkit. In Proceedings of the COLING/ACL 2006 Interactive Presentation Sessions, pp. 69 -2, 2006.
Chuang, Y.-S., Qiu, L.. Hsich. C.-Y., Krishna, R., Kim, Y., and Glass, J. Lookback lens: Detecting and mitigat- ing contextual hallucinations large language models using only attention maps. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pp.
2024.
Cohen- Cohen-Wang, B., Shah, H., Georgiev, K., and Madry, A. Contexteite: Attributing model generation to context. In The Thirty-eighth Annual Conference on Neural Informa- tion Processing Systems, 2024.
Acknowledgements
We thank Jiajie Zhang and Yushi Bai for their assistance in providing implementation details of LongCite. Special thanks to Pin-Lun (Byron) Hsu for his invaluable support and guidance with Liger-Kemnl. We are also grateful to Tianyu Gao for his timely help in setting up the ALCE benchmark during the rebuttal period. We also appreciate Andrei Barbu, Linlu Qiu, Weijia Shi for their valuable dis- cussions. Yung-Sung was sponsored by the Department of the Air Force Artificial Intelligence Accelerator and was ac- complished under Cooperative Agreement Number FA8750- 19-2-1000. The views and conclusions contained in this document are those of the authors and should not be inter- preted representing the official policies, either expressed or implied, of the Department of the Air Force or the U.S. Government. The U.S. Government is authorized to re- produce and distribute reprints for Government purposes notwithstanding any copyright notation herein.
Dubey, A., Jauhri, A., Pandey, A., Kadian, A., Al-Dahle, A., Letman, A., Mathur, A., Schelten, A., Yang, A., Fan, A., et al. The Ilama 3 herd of models. arXiv preprint 2407 2024.
Gao, L., Dai, Z., Pasupat, P., Chen, A., Chaganty, A. T., Fan, Y., Zhao, V. Y., Lao, N.,, Lee, H., Juan, D.-C., et al. Rarr: Researching and revising what language models say, using language models. arXiv preprint arX1v:2210.08726. 2022.
Gao, L., Schulman, J., and Hilton, J. Scaling laws for reward model overoptimization, In International Conference on Machine Learning, pp. 10835-10866. PMLR, 2023a.
Gao, T., Yen, H., Yu, J., and Chen, D. Enabling large language models to generate text with citations. In Pro- ceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pp. 6465 6488, 2023b.
References
Anthropic. Anthropic: Introducing claude 3.5 son- net, 2024. URL https:://// www. anth comm/ news/clauder -orodde------oe-e-
GLM, T., Zeng, A., Xu, B., Wang, B., Zhang, C., Yin, D., Rojas, D., Feng, G., Zhao, H., Lai, H., Yu, H., Wang, H., Sun, J., Zhang, J., Cheng, J., Gui, J., Tang, J., Zhang, J., Li, J., Zhao, L., Wu, L., Zhong, L., Liu. M., Huang, M.. Zhang, P., Zheng, Q. Lu, R., Duan, S., Zhang, S., Cao. S., Yang. S., Tam, W. L., Zhao, W., Liu, X., Xia, X., Zhang, X., Gu, X., Lv, X., Liu, X., Liu, X., Yang, X., Song, X., Zhang, X., An, Y., Xu, Y., Niu, Y., Yang, Y., Li, Y., Bai, Y., Dong, Y., Qi, Z., Wang, Z., Yang, Z., Du, Z., Hou, Z. and Wang, Z. Chatglm: A family of large language models from glm- 130b to glm-4 all tools, 2024.
Bai, Y., Jones, A., Ndousse, K., Askell, A., Chen, A., Das- Sarma, N., Drain, D., Fort, S., Ganguli, D., Henighan, T., et al. Training helpful and harmless assistant with rein- forcement learning from human feedback. arXiv preprint arXiv:2204.05882, 2022.
Bai, Y., Lv, X., Zhang, J., Lyu, H., Tang, J., Huang, Z., Du, Z.,, Liu, X., Zeng, A., Hou, L., et al. Longbench: A
10

---

SelfCite: Self-Supervised Alignment for Context Attribution in LLMs
He, W., Liu, K., Liu, J., Lyu, Y., Zhao, S., Xiao, X., Liu, Y., Wang, Y., Wu, H., She, Q., et al. Dureader: a chinese machine reading comprehension dataset from real-world applications. In Proceedings of the Workshop on Machine Reading for Question Answering, pp. 37-46, 2018.
Meng, Y., Xia, M., and Chen, D. SimPO: Simple preference optimization with reference-free reward. In The Thirty- eighth Annual Conference Neural Information Pro- cessing Systems, 2024. URL
net/ forum? i d LKb.
Holtzman, A., Buys, J., Du, L., Forbes, M.. and Choi, Y. The curious case of neural text degeneration. In International Conference on Learning Representations, 2020. URL https:t////........ wpwp//pperree..eeoe/ee/ee net/ forum? id ry=rygGQy.Evi.
Menick, J., Trebacz, M., Mikulik, V., Aslanides, J., Song, F. Chadwick, M., Glaese, M., Young, S., Campbell- Gillingham, L., Irving, G., et al. Teaching language models to support answers with verified quotes. arXiv preprint arXiv:2203 2022.
Hsu, P.-.., Dai, Y., Kothapalli, V., Song, Q. Tang, S., Zhu, S. Shimizu, S., Sahni, S., Ning, H., and Chen, Y. Liger kemel: Efficient triton kemells for Ilm train- ing. arXiv preprint arXiv:24/0. 10989, 2024. URL https:/ / /atxiv.vvvvv/////i////////aaaava www.iv..o//aaaaae aataaab//211 2210.4110.1 10989.
Mistral. Mistral large, 2024. URL https: a / nwws//ww///////e/e// /ttmmmmtttaalarreee///eeee
Mudgal, S., Lee, J., Ganapathy, H., Li, Y., Wang. T., Huang, Y., Chen, Z., Cheng, H.-T., Collins, M., Strohman, T. et al. Controlled decoding from language models. In In- ternational Conference on Machine Learning, pp. 36486- 36503. PMLR, 2024.
Huang, C., Wu, Z., Hu, Y., and Wang. W. Training language models generate text with citations via fine-grained rewards. arXiv preprint ar arXiv:2402.044: 04315, 2024a.
Nakano, R., Hilton, J., Balaji. S., Wu, J., Ouyang, L., Kim, C., Hesse, C., Jain, S., Kosaraju, V., Saunders, W., et al.
Huang, L.,, Cao,, S., Parulian, N., Ji, H., and Wang, L.. Effi- cient attentions for long document summarization. In Pro- ceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 1419-1436, 2021.
Webgpt: Browser-assisted question-answering with hu- man feedback. arXiv preprint arXiv:2//1.09332) 2021.
OpenAL. Introducing chatgpt, November 2022. URL chatgpt.
Huang, L.,, Feng, X., Ma, W., Zhao, L., Fan, Y.. Zhong, W., Xu, D., Yang, Q., Liu, H., and Qin, B. Advancing large language model attribution through self-improving. In Proceedings of the 2024 Conference on Empirical Meth- ods in Natural Language Processing, pp. 3822-3836, 2024b.
OpenAL. Gpt-4 technical report, 2023. URL https://
cdn. connai.....omm wttp:/pcoopppppppeeeeeee /ptt- 4 4.df.
Ouyang, L.,, Wu, J., Jiang, X., Almeida, D., L. Wainwright, C., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., Schulman, J., Hilton, J., Kelton, F., Miller, I.,, Simens, M., Askell, A., Welinder, P., Christiano, P., Leike, J., and Lowe, R. Training language models to follow instructions with human feedback. arXiv preprint 2203.02 2022.
Ji, Z., Lee, N., Frieske, R., Yu, T. Su, D., Xu, Y., Ishii, E., Bang, Y. J., Madotto, A., and Fung. P. Survey of halluci- nation in natural language generation. ACM Computing Surveys, 55(12): 2023.
Pang, R. Y., Yuan, W., Cho, K., He, H., Sukhbaatar, S., and Weston, J. Iterative reasoning preference optimization. arXiv preprint arXiv:2404. 19733, 2024.
Kim, S., Bae, S., Shin, J., Kang. S., Kwak, D., Yoo, K., and Seo, M. Aligning large language models through syn- thetic feedback. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pp. 13677-13700, 2023.
Phukan, A., Somasundaram, S., Saxena, A., Goswami, K., and Srinivasan, B. V. Peering into the mind of language models: An approach for attribution in contextual ques- tion answering. arXiv preprint arXiv:2405. 17980, 2024.
Lei, T.. Barzilay, R., and Jaakkola, T. Rationalizing neural predictions. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pp. 107-117, 2016.
Qi, J., Sarti, G., Fernández, R., and Bisazza, A. Model internals-based answer attribution for trustwor- thy retrieval- augmenntd generation. arXiv preprint arXiv:2406. 13663, 2024.
Lightman, H., Kosaraju, V., Burda, Y., Edwards, H., Baker, B., Lee, T., Leike, J., Schulman, J., Sutskever, I., and Cobbe, K. Let's verify step by step. In The Twelfth International Conference on Learning Representations, http://pppppeeeeec/////p/. www.:eww.eettttrt/e/// 1==88100N6E01.
Rafailov, R., Sharma, A., Mitchell, E., Manning, C. D., Er- mon, S., and Finn, C. Direct preference optimization: Your language model is secretly a reward model. Ad- vances in Neural Information Processing Systems, 36, 2024.
2024. URL

---

SelfCite: Self-Supervise Aligument for Context Attribution in LLMs
Schulman, J., Wolski, F., Dhariwal, P., Radford, A., and Klimov, O. Proximal poliey optimization algorithms. In arXiv preprint 177.:.707.06347, 2017.
M., Zetrlemoyer, L.,, and Levy, O. LIMA: Less is more for alignment. In Oh, A., Naumann, T., Globerson, A., Saenko, K., Hardt, M., and Levine, S. (eds.), Advances in Neural Information Processing Systems, volume 36, PP. 55006-55021, 2023.
Shi, W. Han, X., Lewis, M., Tsvetkov, Y., Zettlemoyer, L., and Yih, W.-t. Trusting your evidence: Hallucinate less with context-aware decoding. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 2: Short Papers), pp. 783-791, 2024.
Slobodkin, A., Hirsch, E., Cattan, A., Schuster, T., and Dagan, I. Attribute first, then generate: Locally- attributable grounded text generation. arXiv preprint arXiv:2403. 17104, 2024.
Wolf, T., Debut, L., Sanh, V., Chaumond, J., Delangue, C., Moi, A., Cistac, P., Rault, T., Louf, R., Funtowicz, M., Davison, J., Shleifer, S., von Platen, P., Ma, C., Jer- nite, Y., Plu, J., Xu, C., Scao, T. I.., Gugger, S., Drame, M.. Lhoest, Q. and Rush, A. M. Transformers: State- of-the-art natural language processing. In Proceedings of the 2020 Conference on Empirical Methods in Natu- ral Language Processing: System Demonstrations, pp- 38-45, Online, October 2020, Association for Compu- tational Linguistics. URL https: /www///www/////////////t aclweb. org/ ha//aoahhooogyy / 2200 emn 1 1-mmln.-emms...
Worledge, T., Shen, J. H., Meister, N., Winston, C., and Guestrin, C. Unifying corroborative and contributive attributions in large language models. arXiv preprint arXiv:2311.12233 2023.
Yang, 7., Qi, P., Zhang, S., Bengio, Y., Cohen, W. W., Salakhutdinov, R., and Manning, C. D. Hotpotqa: A dataset for diverse, explainable multi-hop question an- swering. arXiv preprint arXiv:1809.000 09600, 2018.
Yasunaga, M., Shamis, L., Zhou, C., Cohen, A., We- ston, J., Zettlemoyer, L., and Ghazvininejad, M. Alma: Alignment with minimal annotation. arXiv preprint arXiv:2412.04305. 2024.
Yuan, W. Pang, R. Y., Cho, Li, X., Sukhbaatar, S., Xu, J., and Weston, J. F. Self-rewarding language mod- els. In Forty-first International Conference on Machine Learning, 2024. URL https::/to/occ.cc. http:://////eeeeiiieeee / forum? ? i-d ONPh
Zhang, J., Bai, Y., Lv. X., Gu. W. Liu, D., Zou, M., Cao, S., Hou, L., Dong, Y., Feng, L.,, al. Longcite: Enabling Ilms to generate fine- mie-grained citations in long-context qa. arXiv preprint arXiv:2409.02897 2024.
Zhou, C., Liu, P. Xu, P., Iyer, S., Sun. J., Mao, Y., Ma, X., Efrat, A., Yu, P., YU, L.,, Zhang, S., Ghosh, G., Lewis,
12

---

SelfCite: Self-Supervised Alignment for Context Attribution in LLMs
A. Implementation Details
For SimPO fine-tuning, we randomly sample 2K document and question pairs from the LongCite-45k data, generate the best-of-N responses with our Algorithm to obtain the preference data, and train for one epoch. We sample another 100 examples as development set to pick the best learning rate from {le-7, 3c-7, 5e-7, 7c-73. We keep other hyperparameters the same the original SimPO (Meng et al., 2024). We follow the same prompt format used in Zhang et al. (2024)² to keep the comparison fair. For the iterative SimPO experiment, in each iteration, we sampled new, non-overlapping subset of 2K examples to ensure no data repetition across iterations. For self-supervised SFT, we generate 11K citation data unsupervisedly from ContextCite outputs as described in Appendix B, trained with larger learning rate 7c-6.
We use the SimPO source code built from Huggingface Transformers (Wolf et al., 2020) for the finetuning experiments, as well as Liger-Kernel (Hsu al., 2024)² to enable memory efficient training for long-context examples LongCite-45K without tensor parallelization. We run all the finetuning experiments on with 8x A10 GPUs of 80 GB memory on single node. The batch size is set to 1 per GPU due to the long context examples. We set our max context length to 25600 to prevent OOM. For the data examples longer than 25600, we perform truncation, start from truncating the sentences that are the most far away from the sentences cited by the ground truth annotation, so as to keep the impact of truncation to be minimum.
When evaluating the citation length, as well as calculating the token length limit of 384 for excluding long BoN candidates, we follow Zhang et al. (2024) to use GLM4-9B's tokenizer count tokens.
In the ablation study of off-policy denoising in Section 4.5, the citation examples for denoising are collected by randomly shifting existing citation spans by 3-10 positions in sentence indices.
B. Obtaining Citations from ContextCite
In this section, we first describe how the ContextCite method Glohen-Wang et al., 2024) estimates continuous attribution scores for each sentence in the context. We then explain a simple heuristic for extracting citations (i.e., selecting a subset of context sources) from these scores.
B.1. ContextCite
Given a language model PLM, a context C, a query Q and a generated response R. ContextCite aims to quantify how each source in the context C = {c1, ( c2,-.. : 9C1) contributes the generated response R (in our case, the sources sentences). To do so, ContextCite performs several random context ablations. We begin by introducing some notation to describe these ablations. Let u E {0, 13(C) be an ablation vector whose i-th entry toggles whether source e; is included (u; 1) or excluded (c; = 0). We write ABLATE(C,, v) to denote a modified version of the original context C' in which sources for which 1/ = 0 are omitted. ContextCite seeks to understand how the probability of generating the original generated response,
f(u) = PR ABLATE(C,U). Q).
changes as a function of the ablation vector v.
Attribution via Surrogate Modeling. Directly measuring f(v) for all 2| ablation vectors is infeasible for large |C]. Hence, ContextCite seeks identify surrogate model f((v) that is easy to understand and approximates f(vv well. To simplify this surrogate modeling task, ContextCite applies logit transform to f. which maps values in (0, 1) to (-o0, oo)):
g(v) :v)==o-)(((o)) = loe{ 1 f(u)
f(v)
ContextCite then approximates g(v) using sparse linear function,
g(v) i T +.
Notice that resulting weights á e CCl encode the importance of each source e, to the probability of generating the original response; they can be interpreted directly attribution scores (higher scores suggest greater importance).
//ttps://giaee thub. hhttge////ggthtt....//rrrreeeeee http.///rriieetooooo/////ome
chttps://gt wtw.bb..c.l.iii.icceet mat.
13

---

SelfCite: Self-Supervise Aligument for Context Attribution in LLMs
Finding Surrogate Model via LASSO. To learn the parameters v and b of the surrogate model, ContextCite randomly samples small number of ablation vectors and measures the corresponding probabilities of generating the original response. It then uses this "training dataset" to fit sparse linear model with LASSO. Concretely, it learns a surrogate model with the following three steps:
1. Sample n ablation vectors {uil==1 uniformly at random from {0, 10.111c1.
2. For each sample V/ compute g(e;) a a (f((e) by running the LM with only the sources specified by vs and measuring the (sigmoid) probability of R.
3. Solve Lasso regression problem to find in and b:
n E -w -wa - b)² A|[w|||,
ú b arg min w.
where À controls sparsity (larger drives more coefficients to zero).
In Cohen-Wang et al. (2024), typical choices n range from 32 to 256, balancing cost (requires n. LM forward passes) and accuracy. there are multiple statements {ri, r2,,., "1)))))) [| R, the same method can also be applied by focusing only on a
subset of tokens in R.
B.2. Heuristic Citation Extraction
In our setting, we would like a discrete list of cited sentences for each generated statement, rather than score for every sentence. We will now describe how to convert the attribution scores w into discrete subset C" C C of citations. Let f be threshold, p be a cumulative probability mass cutoff, and be maximum citation limit.
Thresholding and Merging.
1. Filtering: Include only those sources c, whose attribution score úi 2 t.
2. Merging Adjacent Sources: If multiple consecutive sources in the original text each exceed t, merge them into single "span" Sj. We assign this merged span the maximum score among its constituents:
(S,) max ú...
Here, adjacency is defined by the original ordering in C. For instance, if c2 and c3 both pass the threshold and appear consecutively, we merge them into a single span Sy.
Softmax Normalization. Let { be the set of spans (or single sources) that survived the threshold. We normalize their scores into probability distribution:
exp exp()((s))
w'(S,)
exp(
so that Σ, (S;) = 1.
Top-p Selection.
To avoid including too many low-value sources, we adopt a greedy approach:
Add spans in order of descending w' (5,)) stopping once Σ
á w'(S,) 2 p.
SYEC
Top-k Filtering. Finally, if |C"| > k. take only the k highest-scoring spans.
We set / = 1.5, p 0.7, A: = 4 in the experiment. When generating supervised fine-tuning (SFT) data, we discard any example for which more than 30% of its statements have no any citations that can survive threshold t. This ensures the dataset emphasizes cases where the LM's response can be tied to explicit context sources. We take the LongCite-45X document and question pairs to generate the responses by Llama-3.1-88 -.---ut-ntt itself, and then obtain citations with ContextCite (256 calls), tranffommed into the statement/citation format of LongCite-45K, Finally, we collest 11K examples used for SFT.
14

---

SelfCite: Self-Supervised Aligument for Context Attribution in LLMs
Table Citation recall (R), citation precision (P), citation F1 (FI), and citation length evaluated LongRench-CCte henchmark. The best
results are bolded. indicates the results taken from Zhang al. (2024).
Longhench-Chat
MultifieldQA
HotpotQA
Dureader
Gov GovReport
Avg.
Citation Length
Model
R
P
F1
R
F1
R
P
FI
Proprietary models
GPT-4o
46.7
53.5
46.7
79.0
87.9
80.6
55.7
62.3
53.4
65.6
74.2
67.4
73.4
90.4
79.8
65.6
220
Claude-3-somnet
52.0
67.8
55.1
64.7
85.8
71.3
46.4
65.8
49.9
67.7
89.2
75.5
77.4
93.9
84.1
67.2
132
GLM-4
47.6
53.9
47.1
72.3
80.1
73.6
47.0
50.1
44.4
73.4
82.3
75.0
82.8
93.4
87.1
65.4
169
Ours: SeltCite
LongCite-8B (Our repro.)
67.0
78.1
66.6
74.8
90.7
79.9
60.8
77.9
64.1
67.1
87.2
73.7
81.6
89.3
84.5
73.8
83.5
+
68.4
81.3
71.2
76.1
92.8
81.2
67.2
81.0
68.8
76.9
87.6
92.4
77.5
93.4
SimPO
68.1
79.5
69.1
75.5
92.6
81.0
69.4
82.3
71.5
72.7
91.6
78.9
86.4
92.9
89.1
77.9
105.7
+ SimPOO then BoN
73.3
79.4
72.8
76.7
93.2
82.2
69.4
83.0
71.1
74.2
92.2
80.3
86.7
92.7
89.2
79.1
94.7
Topline
Claude Citations
61.2
81.7
67.8
76.8
98.4
84.9
61.9
94.1
72.9
88.5
93.2
79.4
99.2
87.7
81.3
88.8
C. Length Balancing
To prevent the model from simply generating longer citations rather than focusing on citation correctness, we apply a length balancing procedure to align the total citation length in our two training responses: a chosen prediction and a reject prediction. First, we find the citation string (e.g.. 4435-437) enclosed <cite> </cite> tags for each statement. We then measure each string's ' total citation "coverage", which means the total number cited sentences in these intervals.
If reject prediction has total coverage lower than the corresponding chosen prediction, insert additional citations around nearby sentence indices to match the chosen coverage. Conversely, if the reject coverage is larger, we randomly remove some of its intervals. We ensure new or inserted citations do not overlap existing intervals and keep them within a small window of 5-10 sentences away from the original citations to maintain realism. Finally, the reject and chosen will have matched coverage. This approach discourages the model from trivially learning to cite more sentences, instead prompting it to learn where and how to cite evidence more accurately. Our ablation Section 4.2 shows that this length balancing technique significantly improves final citation quality.
D. Comparison with Claude Citations API
On January 23rd, 2025, Claude announced an API specialized for providing citations along with responses: Claude Citations We also try to evaluate this API on the LongBench-Cite benchmark. Since the implementation details and resource requirements (e.g., training data) Claude Citations are not publicly available yet, and it relies on a significantly larger and more powerful LLM, Claude-3.5. 1oteee-.---onneeee which potentially has over 100 billions of parameters, we consider it as topline of the benchmark rather than baseline.
When evaluating it on Chinese examples from LongBench-Cite, we found that the API does not split Chinese text properly. As result, it cites large passages when processing Chinese examples, leading to an average citation length of approximately 800 tokens per citation.
To address this issue, we pre-segment the text ourselves using exactly the same method as our approach following LongCite (Zhang al., 2024), which uses NLTK and Chinese punetuation segmentation. We then run the Claude Citations API, supports both non-segmented and pre-segmented document inputs. The evaluation was conducted using the latest version of
As shown in Table 7, Claude Citations achieves an overall F1 score of 81.3, which is higher than all other models we have tested. However, the performance of Claude Citations is not consistent over all datasets. For example, it is worse than SelfCite on Long results on other datasets are comparable to the results of SelfCite. Given the fact that SelfCite leverages a much smaller 8B model compared the Claude-3.55 3-5nntt model, the result of SelfCite very impressive, demonstrating its potential to serve strong alternative to proprietary solutions.
and GovReport. The main improvement of Claude is from the DuReader dataset, while the
"https://wwe http://wwww.anoooooooo////////e/
15

---

SelfCite: Self-Supervised Aligument for Context Attribution in LLMs
E. Baseline: SimPO with NLI Rewards
To provide a stronger fine-tuned baseline, we implement SimPO variant that adopts NLI-based citation rewards, fol- lowing the design proposed by Huang al. (2024a). For fair comparison, we keep our full SelfCite SimPO training pipeline-imittaizinng from LongCite-8B and training on the LongCite-45k datasct-and modify only the reward function as controlled experiment. This NLI-based reward combines two components:
Citation Recall Reward: This measures whether the full set of cited sentences entails the model-generated statement. is equivalent to the Citation Recall Reward proposed by Huang et al. (2024a).
This estimates whether each cited sentence is necessary by ablating one sentence at a time and testing whether the remaining span still entails the statement. If entailment fails after removing a sentence, it indicates that the sentence contributes uniquely to the justification. To reduce latency, we ablate all sentences when the citation contains makes reward of N. and finally all ablations sum up to 1.0. It resembles the Citation Precision Reward proposed by Huang al. (2024a).
Citation Precision Reward:
fewer; otherwise, we randomly sample for ablation. When there are ablations, each ablation
We make both rewards positive and capped at 1.0, effectively constructing preference pairs for SimPO. We do not consider the Correctness Recall Reward from Huang et al. (2024a), because the LongCite-45k training set does not contain ground-truth answers. All entailment scores are computed using the public NLI model google/t5 xx1 true nli mixture?.
F. Zero-shot Evaluation on Chunk-level Citation Benchmark ALCE
We additionally include the zero-shot evaluation the chunk-level citation benchmark ALCE (Gao al., 2023b) and report the results in Table 8. We find that our baseline model, LongCite-8B, although under zero-shot setting (it is trained on sentence-level citation but test on chunk-level citations), already outperforms the prompting-basee approach from Gao et al. (2023b) by a substantial margin in both citation recall and precision. Incorporating NI.--ased rewards from Huang et al. (2024a) into our SimPO training yields further improvements. Most notably, our method- with SelfCite achiees-achieves the best performance among models trained on the same LongCite-45k dataset.
The last row of the table presents the best result reported by Huang et al. (2024a), who fine-tuned their model using supervised data. However, this setting is not directly comparable to ours for several reasons:
1. They optimize directly for the ALCE evaluation metric by using the same NLI evaluator model wtw.gg/eett..xx.tue.........
to provide both training rewards and evaluation scores.
2. Their model is trained on the in-distribution QA training sets in ALCE, with exactly the same chunk-level format as the benchmark. In contrast, our SelfCite model is trained on out-of-distributtio sentence-level citations from LongCite-45k.
3. Their method involves distillation from ChatGPT in the first stage, whereas ours does not rely on external supervision.
Despite this domain and format mismatch, SelfCite demonstrates strong generalization and consistently outperforms both LongCite-8B and the NLI-based SimPO baseline. This highlights the robustness and effectiveness of our approach even in cross-domain, cross-format transfer settings.
G. Comparison with Prior Studies
We further provide a comparison table in Table 9 contrast the key differences between SelfCite and other prior studies on producing citations from LLMs. Among all methods, SelfCite is the only approach that supports sentence-lewe citation generation in single pass, leverages preference optimization, and scales to 128K-token contexts-al all without requiring additional supervision. In contrast, prior work such as ALCE (Gao et al., 2023b) and Huang et al. (2024a) use chunk-level citations for shorter context (<8K) and require prompt-based or supervised NLI signals. ContextCite (Cohen-Wang et al., 2024), while being sentence-leved relies on a computationally expensive (at least 32 inference calls) process for random context ablation and trains a linear model for estimating the importance scores. This comparison underscores the practical advantages and technical contributions of SelfCite.
hhttps://huggiinggacce wnaiinnfaaaee
_tr___rue_______ mww__mnmmmmmm_ mi xtuuee
16

---

SelfCite: Self-Supervised Aligument for Context Attribution in LLMs
Table Evaluation the chunk-level citation benchmark ALCE (Gao al., 2023b). Our model (SimPO w/ SelfCite) is trained
sentence-level, LongCite-45k data but still generalizes well to the chunk-level ALCE benchmark.
ASQA
EL15
Model
EM Rec.
Cite Ree.
Cite Prec.
Correct
Cite Rec.
Cite Prec.
Gao et al. (2023b) (Prompting)
Llama-2-13B-chat
34.66
37.48
39.62
12.77 13.63
17.13 34.66
17.05
Llama-3, 1--8-Innstruct
42.68
50.64
53.08
32.08
Finetuned on LongCite-45k
LongCite-8B
42.11 11
62.27
57.00
15.37
30.54
29.15
+ SimPO w/ NLJ Rewards
41,20
63.65
60.20
15.30
33.06
31.05
+ SimPO w/ SelfCite
42.57
71.68
62.05
15.17
37.09
35.62
Finetuned on ALCE train set (In-Distribution Supervision)
Huang et al. (2024a)
40.05
77.83
76.33
11.54
60.86
60.23
Table 9. Key differences among prior methods on producing citations from LLMs. CC stands for ContextCite.
Sentence-level citations?
One pass generation?
Preference
Handle 128K
External supervision?
Method
optimization?
long-context?
ALCE (Gao al., 2023b)
(chunk-level) (chunk-level)
X (prompting)
(8K)
2-shot prompting
Huang et al. (2024a)
(8K)
NLI ground truth
CC (Cohen- Wang et al., 2024)
X (at least 32 calls)
x (not generative) x (SFT only)
N/A SFT data
LongCite (Zhang et al., 2024)
SelfCite (Ours)
N/A
H. More Qualitative Examples
We further show more qualitative examples in Table 10,11, and 12, to represent the cases where SelfCite is better well as where the LongCite-8B direct sampling baseline is better. In Table 10, SelfCite BoN avoid the cited irrelevant sentence (42, 47-50) by the baseline, while further including a correct citation (23) that are not found by the baseline. In Table 11, both SelfCite BoN and the baseline cites too many irrelevant sentences (391-393) but SelfCite BoN's citation is slightly better. In Table 12, SelfCite BoN wrongly includes and misses 70, but the baseline is slightly better and only wrongly includes 71.
17

---

SelfCite: Self-Supervied Aligument for Context Attribution in LLMs
Table 10, An example of differences in the citation from baseline vs BoN. Related information are highlighted in the context/response.
Sent. ID
Context Sentences (only showing cited sentenees due to space)
23 (V)
We explored using a simple and cost-effective procedure to instruction finetune our continually pretrained long models without any human- annotated data.
42 (X)
Collecting human demonstration and preference labels for LLM alignment cumbersome and expensive process (Ouyang et al., 2022; Touvron et al., 2023).
45 (V)
In this work, we found that simple and cheap approach which leverages a pre-built large diverse short-prompl dataset works surprisingly well on long-context benchmarks.
46 (V)
Specifically, we take the RLHF dataset used in LLAMA 2 CHAT and augment it with synthetic self-instruct (Wang al., 2022) long data generated by LLAMA CHAT itself, the hope that the model can learn a diverse set skills through the large amount of RLHI data and transfer that knowledge to long-context scenarios via self-instruct data.
47-50 (X)
The data generation process focuses on QA forrfrt tasks: starting from long document in our pretraining corpus, we select a random chunk and prompt LLAMA CHAT to write question-answer pairs based on information in the text chunk. [...]
Query
What aspects of the LLAMA Long model proposed above have changed relative to the LLAMA- -asee- model? What improvements have been madc?
Response
The paper proposes a simple and cost-elfective procedure to instruction finetune the continually pretrained long models without any numan-annotate data, This involves using a pre-built large and diverse short-prompl dataset and augmenting it with synthetic self-instruct long data generated by LLAMA CHAT itself, [...]
[...] 3. Instruction Tuning:
(single state- ment due to space)
Citation Strings (green: corrects red: wrong)
Baseline
[42-42] [45-50]
SelfCite BoN
[23-23] [45-45] [46-46]
18

---

SelfCite: Sel--Superviee Alignment for Context Attribution in LLMs
Table 11. An example of differences in the citation from baseline vs BoN. Related information are highlighted in the context/response.
Sent. ID
Context Sentences (only showing cited sentences due to space)
299 (X)
Ribosomes link amino acids together the order specified by the codons messenger RNA (mRNA) molecules form polypeptide chains.
300 (V)
Ribosomes consist two major components: the small and large ribosomal subunits.
301 (V)
Each suhunit consists of more ribosomal RNA (rRNA) molecules and many ribosomal proteins (RPs 1-rooteins),
302 (V)
The ribosomes and associated molecules are also known as the translational apparatus.
383 (X)
Rihosomes are minute particles consisting of RNA and associated proteins that function to synthesize proteins.
384 (X)
Proteins are needed for many cellular functions such as repairing damage or directing chemical processes.
385 (X)
Rihosomes be found floating within the eytoplasm attached the endoplasmic reticulum.
390 (V)
Each subunit is made of one more rRNAs and many r-proteins.
391 (X)
The small subunit (30S in bacteria and archaea, 40S in eukaryotes) has the decoding function, whereas the large subunit (50S in bacteria and archaea, 60S in eukaryotes) catalyzes the formation of peptide bonds, referred to as the peptidyl-transferee transferase activity.
392 (X)
The bacterial (and archaeal) small subunit contains the 16S rRNA and 21 r-proteins (Escherichia coli), whereas the eukaryotic small subunit contains the 18S rRNA and r-proteins (Saccharomyces cerevisiae; although the numbers vary between species).
393 (X)
The bacterial large subunit contains the 5S and 23S rRNAs 34 r-proteins (E, coli), with the eukaryotic large subunit containing the 5S, 5.85, and 25S / 28S rRNAs and 46 r-proteins (S. cerevisiae; again, the exact numbers vary between species).
Query
The large subunit and small subunit that use two types of RNA are major components that make up what?
Response (single state- ment due to space)
[..]] The ribosome consists of two major components: small and large ribosomal subunits, Each subunit consists of one more ribosomal RNA (rRNA) molecules and many ribosomal proteins (RPs or r-proteins). The ribosomes and associated molccules are also known as the translational apparatus. [...]
Citation Strings (green: correct; red: wrong)
Baseline
[299-302] [383-385] [390-393] |300-302| [390-393]
SelfCite BoN
19

---

SelfCite: Self-Superviee Aligument for Context Attribution in LLMs
Table 12. An example of differences in the citation from baseline vs BoN. Related information are highlighted in the context/response
Sent. ID
Context Sentences (only showing cited sentences due to space)
28 (V)
The Aegis BMD system exists in several variants.
29 (V)
Listed in order of increasing capability, these include (but not necessarily limited 3.6.X variant, the 4.0.3 variant, the 4.1 variant (also known as the Aegis Baseline [BL] 5.4 variant), the 5.0 CU (Capability Upgrade) variant (also known the BI. 9.1 variant), the 5.1 variant (also known the BI. 9.2 variant), and the 6.X variant (also known as BL 10 variant).
30 (X)
Figure summarizes the capabilitiis of some of these variants (using their designations 2016) and correlates them with the phases of the European Phased Adaptive Approach (or EPAA; see discussion below) for European BMD operations.
61 (V)
The BMD interceptor missiles used by Aegis ships are the Standard Missile-3 (SM-3), the SM-2 Block IV, and the SM-6.
62 (V)
The SM-3 is designed to intercept ballistic missiles above the atmosphere (i.c., exo-atmospheric intercept), in the midcourse phase of enemy ballistic missile's flight.
70 (V)
The SM-2 Block IV designed to intercept ballistic missiles inside the atmosphere (i.e., endo-atmoshhrrii intercept), during the terminal phase an enemy ballistic missile's e's flight.
71 (X)
It is equipped with blast fragmentation warhead.
Query
Please write one- page summary of the above government report.
Response
[..]] The Aegis BMD system exists in several variants with increasing capability, including the 3.6.X, 4.0.3, 4.1, CU, and 6.X variants. The system uses the Standard Missile-3 (SM-3), SM-2 Block IV, SM-6 interceptor missiles. The SM-3 is designed to intercept ballistic missiles above the atmosphere in the midcourse phase, while the SM-2 Block IV and SM-6 are designed intercept ballistic missiles inside the atmosphere in the terminal phase. [...]
(single state- ment due to space)
Citation Strings (green: correcty red: wrong)
Baseline
[28-29] (110)]6616600)))) [70-71]
SelfCite BoN
[28-30] [61-62] (missing: 70)
20


Reviews:
[{"review_id": "FhbcULlUMH", "reviewer": "Reviewer_ci8i", "summary": "The paper presents SelfCite, a self-supervised method for improving citation accuracy in Large Language Models (LLMs). The key innovation lies in using context ablation to compute a self-rewarding signal based on necessity and sufficiency scores, which are then used to enhance citation quality through best-of-N sampling and preference optimization (SimPO). The method achieves up to 5.3 F1 improvement on the LongBench-Cite benchmark without requiring human annotations.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank Reviewer ci8i for the constructive comments!\n## A Comparison Table (Also for **Reviewer Cy66**)\n> The paper should more explicitly differentiate SelfCite from previous work… A comparison table summarizing key differences could be helpful.\n\nWe follow your suggestion to make a table and point out the key differences among the previous work. We will include it in our final version paper.\n|Method|Sentence-level citations?|One pass generation?|Preference optimization?|Handle 128K long-context?|External supervision?|\n|-|-|-|-|-|-|\n|ACLE|❌(chunk-level)|✅|❌(prompting)|❌(8K)|2-shot prompting|\n|Huang et al. 2024|❌(chunk-level)|✅|✅|❌(8K)|NLI + ground truth|\n|ContextCite|✅|❌(at least 32 calls)|❌(not generative)|✅|N/A|\n|LongCite|✅|✅|❌(SFT only)|✅|SFT data|\n|SelfCite (Ours)|✅|✅|✅|✅|N/A|\n## A Toy Example of Necessity and Sufficiency Scores\n> The necessity and sufficiency scores are well-motivated, but additional theoretical justification or a toy example demonstrating their individual impact could strengthen the argument.\n\nThe necessity and sufficiency scores are designed based on human’s common preference: a citation has to be necessary and sufficient, which also matches the metrics of citation precision (necessary) and recall (sufficient) commonly exist in the evaluation benchmarks. Here we follow your advice to show a simple toy example demonstrating their individual impacts:\n\n**Document:**\n\n[1] Alice traveled to France in 2020.  \n[2] Bob visited the famous National Museum in Tokyo, Japan in 2019.  \n[3] Chloe visited the Louvre Museum in Paris, France in 2018.  \n…\n\n**Query:**  \n\"Which famous museum could Alice have visited?\"\n\n**Response:**  \n\"Alice could have visited the Louvre Museum.\"\n\n**Citation Candidates:**\n\n- **[1,2] (Incorrect)**:  \n  - *Necessity*: Probability drops (since removing [1] prevents “Alice traveled to Paris”). (✅ high necessity due to [1])  \n  - *Sufficiency*: Probability doesn’t hold. [1,2] alone cannot fully support “visited the Louvre Museum” since [2] is irrelevant and [3] is missing. (❌ low sufficiency)\n\n- **[1,3] (Correct)**:  \n  - *Necessity*: Probability drops more; removing [1,3] loses essential details (“Alice traveled to Paris,” “visited Louvre”). (✅ high necessity)  \n  - *Sufficiency*: Probability holds. [1,3] Fully supports the response. (✅ high sufficiency)\n\nThis toy example clearly shows the individual contributions of necessity and sufficiency scores.\n\n## Discussion on Efficiency\n> Since best-of-N sampling increases inference-time costs, a discussion on its efficiency trade-offs and potential ways to reduce overhead (e.g., pruning low-quality candidates early) would be beneficial.\n\nWe calculate the latency per example on the LongBench-Cite dataset. On average, direct decoding from LongCite-8B/SelfCite SimPO models have similar latency. When using SelfCite BoN, the sampling + reranking steps in total take roughly 7x longer time compared to the direct decoding methods. All experiments are done with 8*A100 GPUs with batch size 1 and model parallel.\n|Method|Avg latency (s)|\n|-|-|\n|LongCite-8B|24.3|\n|SelfCite BoN sampling|149.0|\n|SelfCite BoN reranking|34.0|\n|SelfCite SimPO model|26.2|\n\nAlso, because we only sample the citation sequence, not the whole responses, the number of generated tokens is very limited, usually within 5~10 tokens. The strategy of pruning low-quality candidates early may not help, as they are mainly for saving time in generating long responses. \n### Latency of BoN is not a major concern\nIn fact, we do not have concerns on the longer latency or extra inference cost of BoN, because we also have the SelfCite SimPO model that can achieve the same performance of BoN in one pass generation, without any additional inference cost. If the users have the need for best efficiency, the best solution is to directly use our SimPO model, instead of using our BoN and trying to optimize BoN.\n## Exploring Hyperparameters\n> The effect of hyperparameters like N in best-of-N sampling and the choice of probability thresholds for necessity/sufficiency scores should be explored more systematically.\n\nThere are no “probability thresholds” for necessity/sufficiency scores in SelfCite. We use the raw probability changes (probability drop and probability hold) during context ablation directly as the reward. There is no need to tune any thresholds in our reward design.\n\nFor N in best-of-N sampling, as we mentioned in Line 217 (left column), after deduplicating repeated citation candidates, on average there are only 4.8 candidates (std=3.2) left per statement. It is due to the fact that we only do sampling within the citation sequences, and keep the statements in the response unchanged. When generating citations, usually only a limited number of relevant sentences can support the statements, resulting in a limited possibility of citations to be generated. Given the low diversity of citation candidates, increasing N to be larger than 10 would have a very limited impact on the BoN results.", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "Strength:\n  1. The paper introduces a novel self-supervised approach for citation alignment in LLMs, eliminating the need for human annotations. The combination of necessity and sufficiency scores to derive a reward function is well-motivated and provides a principled way to improve citation quality.\n  2. The method is designed to be lightweight, leveraging a model’s own probability estimates rather than requiring expensive external annotations. This makes it applicable to large-scale citation tasks in real-world settings, such as research assistants or fact-checking systems.\n3. Unlike previous methods that rely on human annotations or costly API calls, SelfCite autonomously improves citation quality using a reward function derived from context ablation, making it highly scalable and cost-efficient.\nWeakness:\n  1. The paper should more explicitly differentiate SelfCite from previous work, especially in how it improves over ContextCite and other contributive context attribution methods. A comparison table summarizing key differences could be helpful.\n  2. The necessity and sufficiency scores are well-motivated, but additional theoretical justification or a toy example demonstrating their individual impact could strengthen the argument.\n  3. Since best-of-N sampling increases inference-time costs, a discussion on its efficiency trade-offs and potential ways to reduce overhead (e.g., pruning low-quality candidates early) would be beneficial.\n  4. The effect of hyperparameters like N in best-of-N sampling and the choice of probability thresholds for necessity/sufficiency scores should be explored more systematically.", "theoretical_claims": "Strength:\n  1. The paper introduces a novel self-supervised approach for citation alignment in LLMs, eliminating the need for human annotations. The combination of necessity and sufficiency scores to derive a reward function is well-motivated and provides a principled way to improve citation quality.\n  2. The method is designed to be lightweight, leveraging a model’s own probability estimates rather than requiring expensive external annotations. This makes it applicable to large-scale citation tasks in real-world settings, such as research assistants or fact-checking systems.\n3. Unlike previous methods that rely on human annotations or costly API calls, SelfCite autonomously improves citation quality using a reward function derived from context ablation, making it highly scalable and cost-efficient.\nWeakness:\n  1. The paper should more explicitly differentiate SelfCite from previous work, especially in how it improves over ContextCite and other contributive context attribution methods. A comparison table summarizing key differences could be helpful.\n  2. The necessity and sufficiency scores are well-motivated, but additional theoretical justification or a toy example demonstrating their individual impact could strengthen the argument.\n  3. Since best-of-N sampling increases inference-time costs, a discussion on its efficiency trade-offs and potential ways to reduce overhead (e.g., pruning low-quality candidates early) would be beneficial.\n  4. The effect of hyperparameters like N in best-of-N sampling and the choice of probability thresholds for necessity/sufficiency scores should be explored more systematically.", "experimental_designs_or_analyses": "Strength:\n  1. The paper introduces a novel self-supervised approach for citation alignment in LLMs, eliminating the need for human annotations. The combination of necessity and sufficiency scores to derive a reward function is well-motivated and provides a principled way to improve citation quality.\n  2. The method is designed to be lightweight, leveraging a model’s own probability estimates rather than requiring expensive external annotations. This makes it applicable to large-scale citation tasks in real-world settings, such as research assistants or fact-checking systems.\n3. Unlike previous methods that rely on human annotations or costly API calls, SelfCite autonomously improves citation quality using a reward function derived from context ablation, making it highly scalable and cost-efficient.\nWeakness:\n  1. The paper should more explicitly differentiate SelfCite from previous work, especially in how it improves over ContextCite and other contributive context attribution methods. A comparison table summarizing key differences could be helpful.\n  2. The necessity and sufficiency scores are well-motivated, but additional theoretical justification or a toy example demonstrating their individual impact could strengthen the argument.\n  3. Since best-of-N sampling increases inference-time costs, a discussion on its efficiency trade-offs and potential ways to reduce overhead (e.g., pruning low-quality candidates early) would be beneficial.\n  4. The effect of hyperparameters like N in best-of-N sampling and the choice of probability thresholds for necessity/sufficiency scores should be explored more systematically.", "relation_to_broader_scientific_literature": "Strength:\n  1. The paper introduces a novel self-supervised approach for citation alignment in LLMs, eliminating the need for human annotations. The combination of necessity and sufficiency scores to derive a reward function is well-motivated and provides a principled way to improve citation quality.\n  2. The method is designed to be lightweight, leveraging a model’s own probability estimates rather than requiring expensive external annotations. This makes it applicable to large-scale citation tasks in real-world settings, such as research assistants or fact-checking systems.\n3. Unlike previous methods that rely on human annotations or costly API calls, SelfCite autonomously improves citation quality using a reward function derived from context ablation, making it highly scalable and cost-efficient.\nWeakness:\n  1. The paper should more explicitly differentiate SelfCite from previous work, especially in how it improves over ContextCite and other contributive context attribution methods. A comparison table summarizing key differences could be helpful.\n  2. The necessity and sufficiency scores are well-motivated, but additional theoretical justification or a toy example demonstrating their individual impact could strengthen the argument.\n  3. Since best-of-N sampling increases inference-time costs, a discussion on its efficiency trade-offs and potential ways to reduce overhead (e.g., pruning low-quality candidates early) would be beneficial.\n  4. The effect of hyperparameters like N in best-of-N sampling and the choice of probability thresholds for necessity/sufficiency scores should be explored more systematically.", "essential_references_not_discussed": "Strength:\n  1. The paper introduces a novel self-supervised approach for citation alignment in LLMs, eliminating the need for human annotations. The combination of necessity and sufficiency scores to derive a reward function is well-motivated and provides a principled way to improve citation quality.\n  2. The method is designed to be lightweight, leveraging a model’s own probability estimates rather than requiring expensive external annotations. This makes it applicable to large-scale citation tasks in real-world settings, such as research assistants or fact-checking systems.\n3. Unlike previous methods that rely on human annotations or costly API calls, SelfCite autonomously improves citation quality using a reward function derived from context ablation, making it highly scalable and cost-efficient.\nWeakness:\n  1. The paper should more explicitly differentiate SelfCite from previous work, especially in how it improves over ContextCite and other contributive context attribution methods. A comparison table summarizing key differences could be helpful.\n  2. The necessity and sufficiency scores are well-motivated, but additional theoretical justification or a toy example demonstrating their individual impact could strengthen the argument.\n  3. Since best-of-N sampling increases inference-time costs, a discussion on its efficiency trade-offs and potential ways to reduce overhead (e.g., pruning low-quality candidates early) would be beneficial.\n  4. The effect of hyperparameters like N in best-of-N sampling and the choice of probability thresholds for necessity/sufficiency scores should be explored more systematically.", "other_strengths_and_weaknesses": "Strength:\n  1. The paper introduces a novel self-supervised approach for citation alignment in LLMs, eliminating the need for human annotations. The combination of necessity and sufficiency scores to derive a reward function is well-motivated and provides a principled way to improve citation quality.\n  2. The method is designed to be lightweight, leveraging a model’s own probability estimates rather than requiring expensive external annotations. This makes it applicable to large-scale citation tasks in real-world settings, such as research assistants or fact-checking systems.\n3. Unlike previous methods that rely on human annotations or costly API calls, SelfCite autonomously improves citation quality using a reward function derived from context ablation, making it highly scalable and cost-efficient.\nWeakness:\n  1. The paper should more explicitly differentiate SelfCite from previous work, especially in how it improves over ContextCite and other contributive context attribution methods. A comparison table summarizing key differences could be helpful.\n  2. The necessity and sufficiency scores are well-motivated, but additional theoretical justification or a toy example demonstrating their individual impact could strengthen the argument.\n  3. Since best-of-N sampling increases inference-time costs, a discussion on its efficiency trade-offs and potential ways to reduce overhead (e.g., pruning low-quality candidates early) would be beneficial.\n  4. The effect of hyperparameters like N in best-of-N sampling and the choice of probability thresholds for necessity/sufficiency scores should be explored more systematically.", "other_comments_or_suggestions": "Strength:\n  1. The paper introduces a novel self-supervised approach for citation alignment in LLMs, eliminating the need for human annotations. The combination of necessity and sufficiency scores to derive a reward function is well-motivated and provides a principled way to improve citation quality.\n  2. The method is designed to be lightweight, leveraging a model’s own probability estimates rather than requiring expensive external annotations. This makes it applicable to large-scale citation tasks in real-world settings, such as research assistants or fact-checking systems.\n3. Unlike previous methods that rely on human annotations or costly API calls, SelfCite autonomously improves citation quality using a reward function derived from context ablation, making it highly scalable and cost-efficient.\nWeakness:\n  1. The paper should more explicitly differentiate SelfCite from previous work, especially in how it improves over ContextCite and other contributive context attribution methods. A comparison table summarizing key differences could be helpful.\n  2. The necessity and sufficiency scores are well-motivated, but additional theoretical justification or a toy example demonstrating their individual impact could strengthen the argument.\n  3. Since best-of-N sampling increases inference-time costs, a discussion on its efficiency trade-offs and potential ways to reduce overhead (e.g., pruning low-quality candidates early) would be beneficial.\n  4. The effect of hyperparameters like N in best-of-N sampling and the choice of probability thresholds for necessity/sufficiency scores should be explored more systematically.", "questions_for_authors": "Strength:\n  1. The paper introduces a novel self-supervised approach for citation alignment in LLMs, eliminating the need for human annotations. The combination of necessity and sufficiency scores to derive a reward function is well-motivated and provides a principled way to improve citation quality.\n  2. The method is designed to be lightweight, leveraging a model’s own probability estimates rather than requiring expensive external annotations. This makes it applicable to large-scale citation tasks in real-world settings, such as research assistants or fact-checking systems.\n3. Unlike previous methods that rely on human annotations or costly API calls, SelfCite autonomously improves citation quality using a reward function derived from context ablation, making it highly scalable and cost-efficient.\nWeakness:\n  1. The paper should more explicitly differentiate SelfCite from previous work, especially in how it improves over ContextCite and other contributive context attribution methods. A comparison table summarizing key differences could be helpful.\n  2. The necessity and sufficiency scores are well-motivated, but additional theoretical justification or a toy example demonstrating their individual impact could strengthen the argument.\n  3. Since best-of-N sampling increases inference-time costs, a discussion on its efficiency trade-offs and potential ways to reduce overhead (e.g., pruning low-quality candidates early) would be beneficial.\n  4. The effect of hyperparameters like N in best-of-N sampling and the choice of probability thresholds for necessity/sufficiency scores should be explored more systematically.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Oe9xjtpldo", "reviewer": "Reviewer_WRpv", "summary": "This paper proposes a method (\"Self-cite\") to automatically evaluate cited text using context ablation -- i.e. changing the context and compare probability of generating a given sentence. It then proposes to use this signal to enhance citation quality as the reward model for two approaches: (1) Best-of-N sampling and (2) preference learning. Experiments show that it is possible to enhance the citation F1 for both LongCite-8B and Llama-3.1-8B-Instruct.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank Reviewer WRpv for the constructive comments!\n## A Better Baseline: SimPO with NLI Rewards (Also for **Reviewer Cy66**)\n> …ContextCite baseline is not appropriate. … [1] has been proposed to leverage NLI models to measure citation precision / recall, which seems to bit a more appropriate baseline\n\nWe agree that ContextCite’s mechanism is much different from SelfCite, so we will change our framing and treat its scores mainly for reference. We follow your advice to adopt NLI rewards from Huang et al. 2024 [1] as baseline. For fair comparison, we reuse our SelfCite SimPO training pipeline (initializing from LongCite-8B + trained with LongCite-45k data), but only change the reward function to be the NLI-based citation recall/precision proposed in Huang et al. 2024 [1]. We ignore the correctness reward in [1] as we don’t have ground truth answers from LongCite-45k.\n\nWe compare this method (SimPO w/ NLI Rewards) with ours (SimPO w/ SelfCite) on both LongBench-Cite (table below) and ALCE (table in the next section). Both results show SimPO w/ NLI Rewards improves citation quality over LongCite (except for MultifieldQA & HotpotQA), but still consistently outperformed by SelfCite, further verifying the effectiveness of SelfCite. We will include this baseline in our final version paper.\n|Metric: Citation F1|Longbench-Chat|MultifieldQA|HotpotQA|Dureader|GovReport|Avg|\n|-|-|-|-|-|-|-|\n|LongCite-8B|66.6|79.9|64.1|73.7|84.5|73.8|\n|+ SimPO w/ NLI Rewards|69.8|77.4|63.2|77.2|87.5|75.0|\n|+ SimPO w/ SelfCite|69.1|81.0|71.5|78.9|89.1|77.9|\n## Evaluation on Chunk-level Citation Benchmark ALCE (Also for **Reviewer Cy66**)\n> previous method [1] can be adopted to chunk-level citation. Therefore, it would be better to also include experiments are datasets such as ALCE [0].\n\nWe follow your advice to test our models on ALCE and show the results in the table below. We found that our baseline LongCite-8B already achieves much better citation recall/precision than the prompting method of Gao et al. (2023). The baseline “SimPO w/ NLI Rewards” (using the rewards from Huang et al., (2024) above) performs slightly better than LongCite-8B. Our method, “SimPO w/ SelfCite”, further brings substantial improvements over both baselines.\n\nThe bottom row is the best result from the supervised method of Huang et al. (2024). It differs from other model settings and was trained on in-distribution data. Its numbers are thus incomparable with the other rows and we only include them for reference. Specifically, the differences are:\n1. They train the models only on the “in-distribution” training sets of QA datasets in ALCE, with the exact same chunk-level setting of ALCE, while SelfCite was trained on “out-of-distribution” LongCite-45k data with sentence-level citations.\n2. They directly use the same NLI evaluator used in ALCE benchmark (`google/t5_xxl_true_nli_mixture`) to provide rewards for citation recall/precision, essentially optimizing the benchmark scores of ALCE directly.\n3. They also do distillation from ChatGPT.\n\nDespite the fact of a **cross-domain** & **cross-setting** transfer setting, SelfCite still achieves good performance much better than baselines: LongCite-8B & SimPO w/ NLI Rewards, showing its effectiveness. We will include this result in our paper.\n||ASQA|||ELI5|||\n|-|-|-|-|-|-|-|\n||EM Rec.|Cite Rec.|Cite Prec.|Correct|Cite Rec.|Cite Prec.|\n|**Gao et al. 2023**|\n|Llama-2-13B-chat|34.66|37.48|39.62|12.77|17.13|17.05|\n|Llama-3.1-8B-Instruct|42.68|50.64|53.08|13.63|34.66 |32.08|\n|**Finetuned on LongCite-45k**|\n|LongCite-8B|42.11|62.27|57.00|15.37|30.54|29.15|\n|+ SimPO w/ NLI Rewards|41.20|65.65|60.20|15.30|33.06|31.05|\n|+ SimPO w/ SelfCite|42.57|**71.68**|**62.05**|15.17|**37.09**|**35.62**|\n|**Finetuned on ALCE train set**|\n|Huang et al. 2024|40.05|77.83|76.33|11.54|60.86 |60.23|\n## How does SelfCite handle unfaithful answers?\n> are there cases where the generated answer is not faithful (and thus can't be supported by the context) and how does SelfCite deal with that?\n\nWe did sometimes (but not often) find that the answer can be a slight misunderstanding of the cited information, and SelfCite will still cite such text that it is based on. Stepping back, this is a common theme for all methods that post-hoc generate citations, where the prevailing philosophy is that “citations” are for the traceability and verifiability of answers, which enable a user to double-check the answer correctness easily, in case the answer is wrong. We will include this discussion in our final version paper.", "claims_and_evidence": "Experiments results on LongBench-Cite with two models (LongCite and Llama-3.1-8B) demonstrate that the proposed reward signal does improve citation quality.", "methods_and_evaluation_criteria": "* **Proposed method**: The propose method leverages probability difference in generating a given answer when a piece of cited text is included / not included in the context, which is an intuitive method to approximate citation quality. \n* **Dataset** : The experiments are mainly conducted on the LongBench-Cite benchmark, which adopts sentence-level citations. However, my understanding is that proposed method is not limited to sentence-level citation by design and can be adopted to chunk-level citation. Therefore, it would be better to also include experiments are datasets such as ALCE [0].\n\n\n[0] Enabling Large Language Models to Generate Text with Citations. Gao et al., EMNLP 2023.", "theoretical_claims": "N/A", "experimental_designs_or_analyses": "* **Baseline chosen**: Experiments are conducted against three baselines: a prompting baseline, ContextCite and Fine-tuned models. I think the ContextCite baseline is not appropriate. More specifically, ContextCite is a method that is used to attribute generated response towards the context (i.e. **generating** citation), and thus it is applied on Llama-3.1-8B-Instruct, while the proposed method is a method that is used to **improve** citation quality, and is applied to model that is already fine-tuned to generate citation (either using the LongCite-SFT data or data generated with ContextCite in the experiment setting). Thus, it is a bit unfair to compare these two methods. On the other hand, previous method [1]  has been proposed to leverage NLI models to measure citation precision / recall, which seems to bit a more appropriate baseline for the proposed method. While it is true that this method requires an external NLI model whereas the proposed method is \"self-supervised\", it would be helpful to compare these two approaches (if there is a gap, how big is it?).\n\n\n[1] Training Language Models to Generate Text with Citations via Fine-grained Rewards. Huang et al., 2024 ACL.", "relation_to_broader_scientific_literature": "The proposed method contributes to the line of work that enables language model to generate citation / attribution to its generation, which is an important research direction. The proposed \"self-supervised\" method is interesting in that it leverages context ablation to improve citation quality.", "essential_references_not_discussed": "N/A", "other_strengths_and_weaknesses": "* Strength: Overall I think the proposed method is an effective approach to improve citation quality given an answer generated. The  `SimPO then BoN` results are pretty strong on both LongCite-8B and Llama-3.1-8B SFT on ContextCite.\n* Weakness: While the proposed method is \"self-supervised\", it would be helpful to compare it with some \"supervised\" methods -- for instance, the NLI model as reward as mentioned before, or SFT with the data from LongCite that is used to create the preference-pair. It is ok if SelfCite does not out-perform these methods, but it will be helpful to understand the gap (if there is any).", "other_comments_or_suggestions": "N/A", "questions_for_authors": "* Since the method (especially with BoN) aims to improve citation quality while keeping the answer unchanged, are there cases where the generated answer is not faithful (and thus can't be supported by the context) and how does SelfCite deal with that?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "viq8HBxsSf", "reviewer": "Reviewer_Cy66", "summary": "This paper proposes an attributable response generation strategy, SelfCite, which cites relevant sentences in the context supportive of the generated response. SelfCite can both operate during inference or during training. For inference, SelfCite pick the Best-of-N using a newly designed reward/score composed by probability drop and probability hold via ablating the reference. For training, the reward signal can be leveraged to curate DPO preference data to train the generation model. Experimental results on LongBench-Cite demonstrate the effectiveness of the proposed strategy in terms of citation quality.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank Reviewer Cy66 for the constructive comments!\n## Ours vs ContextCite\n> The proposed method makes sense, but not entirely novel, ablation technique has been proposed by ContextCite\n\nWhile inspired by ContextCite (CC)’s context ablation (L029, right column), our key contribution differs notably: **SelfCite enables an LLM to directly generate its own citations, while CC does not.**\n\nCC is post-hoc, relying on external linear models trained from scratch for each new example, requiring heavy inference per example (32 to 256 ablated generations) after a response was produced.\n\nIn contrast, SelfCite directly teaches LLMs to produce accurate citations in responses with a few tokens in one pass. The context ablation signals become internalized capabilities of LLMs after SimPO; no context ablations needed at inference.\n\nExtra distinctions:\n- CC measures citation “necessity” by “prob drop”; SelfCite adds “sufficiency” by “prob hold” to catch missing citations.\n- CC’s linear model assumes context sentences independently impact responses; SelfCite directly learns from nonlinear “prob drop/hold” rewards, handling sentence interactions.\n\nCC’s details were in Appendix B due to space. We'll move them to the main text.\n## Answer Accuracy Drops?\n> Table 2, it seems answer accuracy drops on Llama-3 which is concerning. (avg 71.7 vs 64.6)\n\nGood catch! We carefully checked our experiments and found the high score (avg 71.7) of Llama3 baseline under “Answering with citations” was mistakenly copied from our ContextCite (CC) experiment. We reran the experiments to get a corrected baseline of avg 63.3 in row (4) of the table below. Compared to this correct baseline, both our `+ SFT on CC` (avg 64.6) and `+ SimPO (Ours)` (avg 64.7) in fact show slightly higher accuracy. We apologize for the mistake. \n||Long.|Multi.|Hot.|Dur.|Gov.|Avg|\n|-|-|-|-|-|-|-|\n|**Answering without citations**|\n|(1) Greedy (CC: wrong baseline)|67.4|87.9|73.5|67.8|62.1|71.7|\n|(2) Sampling|66.0|83.7|65.8|62.8|66.1|68.9|\n|**Answering with citations**|\n|(3) Greedy|61.2|79.0|68.8|60.0|54.9|64.8|\n|(4) Sampling (true baseline)|58.4|75.3|67.3|59.3|56.4|63.3|\n|**Table 2 from paper**|\n|+ SFT on CC|58.8|83.4|65.8|57.8|57.5|64.6|\n|+ SimPO (Ours)|56.8|80.9|65.3|59.5|60.9|64.7|\n### Why is CC higher? (avg 71.7)\nCC’s higher score (71.7) comes from:\n1. CC uses greedy decoding; we follow LongCite [1] to use sampling (top_p=0.7; temp=0.95). See rows (1) vs (2): avg 71.7 vs 68.9\n2. CC’s citations are post-hoc, so it’s “answering without citations”, making answer generation easier and more accurate. See rows (1) vs (3): avg 71.7 vs 64.8 \n\nThe correct baseline to be used is Sampling + Answering with citations in row (4): avg 63.3, which is slightly better than the same results from LongCite ([1], Table 3, row Llama-3.1-8B, column C’s). And it confirms no accuracy drop after our fine-tuning (avg 64.6 & 64.7). We’ll update Table 2 in our paper.\n\n[1] LongCite: https://arxiv.org/pdf/2409.02897\n## More RL algorithms?\n> …beneficial to investigate these RL algorithms...\n\nOur main goal is to validate a novel \"reward\" for citation. Prior work ([2], Figures 4 & 5) shows Best-of-N (BoN) closely approximates the upper-bound scores of RL without training artifacts to make comparisons more confounded. Following established practices [3, 4], we used BoN as main evaluation, and further verified it using training-based alignment, SimPO, achieving the same improvement of BoN. While extra RL algorithms may offer improvements, we believe they wouldn’t qualitatively change our observations given BoN results.\n\nWe also acknowledge SelfCite doesn’t aim to boost answer accuracy (but not to decrease it either). Combining it with answer-matching rewards to jointly improve citations & answers is an exciting direction we want to explore in future work!\n\n[2] Controlled Decoding from Language Models, Mudgal et al., ICML 2024\n\n[3] Scaling laws for reward model overoptimization, Gao et al., ICML 2023\n\n[4] Let’s Verify Step by Step, Lightman et al., ICLR 2024\n## More Benchmarks/Baselines?\n> …other benchmark datasets, such as ALCE...\n> …both Huang et al., 2024 and Gao et al., 2023 have proposed … there are no comparisons in experiments.\n\nFollowing your advice, we evaluated on ALCE and compared to Huang et al. 2024 & Gao et al. 2023. Due to space, please see our rebuttal to Reviewer WRpv. Spoiler: SelfCite outperforms baselines on ALCE even on cross-domain transfer!\n## More Discussion?\n> lack of detailed discussion on comparison against existing approaches such as Zhang et al., 2024 (LongCite) & Cohen-Wang et al., 2024 (ContextCite).\n\nComparison with ContextCite is at the top of this reply; will be added to our paper.\n\nComparison with LongCite is in our Section 5. Briefly, LongCite uses data from proprietary APIs for SFT only. SelfCite performs further alignment steps without external supervision.\n\nWe also made a table to contrast key distinctions among prior works; see our rebuttal to Reviewer ci8i.", "claims_and_evidence": "Yes, the claim that \"using the combination of probability drop and probability hold during inference and training can boost the citation quality\" is supported by the experimental results.", "methods_and_evaluation_criteria": "1. The proposed method makes sense, but not entirely novel, as the ablation technique has been proposed by Cohen-Wang et al. in ContextCite. The authors adopt this technique to produce a reward or score to guide the selection of the best generation or to be used in preference optimization.\n2. The evaluation mostly focuses on citation quality, but overlooks the answer accuracy which is also quite important. From Table 2, it seems the answer accuracy drops substantially on Llama-3 which is concerning.\n3. Given many RL algorithms can be explored to improve the generation quality (both citation and answer accuracy), it is more beneficial to investigate these RL algorithms for this problem.\n4. In the field of attributable generation, there are also other benchmark datasets, such as the ALCE benchmark (Gao et al., 2023) and ExpertQA (Malaviya et al., 2023). Is the proposed method applicable in these benchmarks?\n\nGao et al., 2023: \"Enabling large language models to generate text with citations\". \nMalaviya et al., 2023: \"Expertqa: Expert-curated questions and attributed answers\".", "theoretical_claims": "There is no theoretical claim", "experimental_designs_or_analyses": "Yes, I have checked the experimental designs and analyses. There are no issues.", "relation_to_broader_scientific_literature": "The idea is quite related to the field of attributable generation in general. The methodology is inspired from the work of ContextCite.", "essential_references_not_discussed": "No.", "other_strengths_and_weaknesses": "Strengths:\n- The proposed approach using self rewards instead of manual annotations is meaningful. The results in terms of citation quality show the effectiveness of the proposed reward.\n- Extensive experiments are conducted over a well-known benchmark on both inference-only and training-based scenarios. Comprehensive analysis on model components, length balancing, data size and iteration is provided.\n\nWeaknesses:\n- There is lack of detailed discussion on the comparison against existing approaches such as Zhang et al., 2024 and Cohen-Wang et al., 2024. How does SelfCite differentiate itself from these studies?\n- Given that many existing works have already investigated attributable generation, there is a lack of empirical comparison against these baselines. From related work, both Huang et al., 2024 and Gao et al., 2023, for example, have proposed methods to tackle this problem, but there are no comparisons in experiments.\n- Besides LongBench-Cite, what about other benchmarks such as the ALCE (Gao et al., 2023) datasets?\n- One key limitation is the degradation of answer accuracy. Admitting that the citation quality is crucial for reliable generation, we should not expect trading enhanced citation quality with decreased answer capability.\n\nGao et al., 2023: \"Enabling large language models to generate text with citations\".", "other_comments_or_suggestions": "NA", "questions_for_authors": "Refer to the above sections", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "yztcoJqKyF", "reviewer": "Reviewer_PWPX", "summary": "This submission proposes SelfCite, an approach for LLMs to improve their citation of sentences from the input context to support their responses. SelfCite evaluates the quality of a citation in terms of the LLM's probability of generating the same response without the cited sentences and with only the cited sentences. This self-supervised reward is used for best-of-N sampling, which is in turn used to generate training data for preference optimization to improve the LLM's intrinsic citation capability. SelfCite is evaluated on the LongBench-Cite benchmark (composed of 5 datasets) against baselines that use prompting, fine-tuning, or contributive context attribution, showing superior citation F1 scores. The paper also presents several studies of ablations/alternatives.\n\n### Update after rebuttal\n\nThe rebuttal provided additional discussions, clarifications, and a latency table that I trust will be added to further improve the paper.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank Reviewer PWPX for the constructive comments!\n\n> …different notions of \"citation\" targeted by ContextCite on the one hand and evaluated by LongBench-Cite (and similar benchmarks) on the other hand.\n\nThanks for pointing out the mismatch between the objectives of corroborative (sources that support a statement, e.g., LongCite & LongBench-Cite benchmark) and contributive context attribution (e.g., ContextCite). As you said, SelfCite applies contributive alignment (using context ablations) to a method for corroborative evaluation (LongBench-Cite). Our intuitions are:\n1. Among citation candidates proposed by LongCite, the candidates actually “used” by the model are also more likely to “support” the statement—other candidates may just be semantically related. Although such support is certainly not always guaranteed, the two objectives are still aligned to some extent.\n2. Current corroborative methods (LongCite) have a big room to improve, even by a contributive method, despite their discrepancy in goals. In other words, if LongCite were already near perfect, enforcing it to be “more contributive” may not help much.\n\nWe will discuss this nuanced point more clearly throughout our paper!\n> …why the text implies that best-of-N sampling has to be done \"after generating the full response\" (lines 154-155). Could it not also be done after generating each statement $r_i$?\n\nYes, BoN only needs each statement $r_i$ before sampling citation $e_i$. Our implementation here is mostly for convenience. We generate a full response to get $r_1, ..., r_S$ first, then re-sample each $e_i$, considering the weak dependence between $r_{>i}$ and $e_i$.\n> In re-sampling citation sequences in position $e_i$, are the future statements and citations $r_{i+1}, e_{i+1}$,… removed? The notation in Algorithm 1 suggests yes?\n\nYes, future statements aren't used when sampling citation $e_i$.\n> Lines 132-134, right column, \"additional inference cost of generating candidates and re-ranking\": This inference cost should be quantified…\n\nWe measured latency per example (8*A100 GPUs, batch size 1, model parallelism) on LongBench-Cite. Direct decoding of LongCite-8B vs. SelfCite SimPO shows similar latency. BoN sampling+reranking is ~7x slower. We'll add this to our paper.\n|Method|Avg latency (s)|\n|-|-|\n|LongCite-8B|24.3|\n|SelfCite:|\n|BoN sampling|149.0|\n|BoN reranking|34.0|\n|SimPO model|26.2|\n> What constitutes a statement $r_i$ in the response? How is the response divided into statements?\n\nIn LongCite-45k, statements are split based on semantic integrity in data generation, so fine-tuned models will naturally learn to produce statements. We'll clarify this in our paper.\n> Lines 112-113, right column, \"we exclude the BoN candidates that cite more than 384 tokens in total\": I believe this refers to the set of context sentences $c_{e_i^1}, ..., c_{e_i^m}$, not the sequence of identifiers $e_i^1, ..., e_i^m$…\n\nYes, the length limit was applied on the cited texts, not identifiers. We'll clarify this in our paper.\n> Equation (1): Were weighted sums of the probability drop and probability hold metrics also considered?\n\nGood suggestion! We only tested 1:1 weights for simplicity but will explore this further.\n> Table 2: Are the numbers in the second Llama-3.1-8B-Instruct row better than the ones in the first Llama-3.1-8B-Instruct row because the second one is for answering with citations?\n\nAfter carefully checking our experiments, we found the second Llama-3.1-8B-Instruct row (avg 71.7) was  actually mistakenly taken from the ContextCite result, which uses greedy decoding and answering without citations, thus is not directly comparable.\n\nWe reran the experiments and show full results in the table below. The first Llama-3.1-8B-Instruct row in Table 2 of our paper should be updated with row (2) below (avg 68.9). Its original scores with \"†\" (avg 60.2) in Table 2 of our paper are taken from Table 3 in LongCite paper and thus have some prompt/implementation differences (they didn't open-source this part of code). We will update it to our own results for now. The second Llama-3.1-8B-Instruct row should be updated with row (4) (avg 63.3). \n\nIn summary, answering with citations hurts accuracy (68.9 -> 63.3), which is expected and consistent with the same trend from Table 3 in the LongCite paper (all non-LongCite models have such degradation when asked to answer with citations.) We'll update Table 2 of our paper.\n||Long.|Multi.|Hot.|Dur.|Gov.|Avg|\n|-|-|-|-|-|-|-|\n|**Answering without citations**|\n|(1) Greedy (CC)|67.4|87.9|73.5|67.8|62.1|71.7|\n|(2) Sampling|66.0|83.7|65.8|62.8|66.1|68.9|\n|**Answering with citations**|\n|(3) Greedy|61.2|79.0|68.8|60.0|54.9|64.8|\n|(4) Sampling|58.4|75.3|67.3|59.3|56.4|63.3|\n> The Limitations section could also note that SelfCite assumes access to the LLM's predicted probabilities…\n\n> Additional references on contributive attribution: …\n\nWe'll add these points and references to our paper. Thanks again for the valuable suggestions!", "claims_and_evidence": "Yes, I think the experimental design and results are sound.", "methods_and_evaluation_criteria": "The proposed approach makes sense as it judges the quality of cited sentences based on their contribution to the LLM's probability of generating the given response, which is mostly aligned with the degree to which the cited sentences support the response in a general sense (not from the LLM's point of view). The evaluation is done on LongBench-Cite, which is a recognized benchmark for citation quality.", "theoretical_claims": "The submission does not make theoretical claims.", "experimental_designs_or_analyses": "I read the experimental sections in full and did not find issues with soundness.\n\nMy only major comment is that the discussion of experimental results should also address the different notions of \"citation\" targeted by ContextCite on the one hand and evaluated by LongBench-Cite (and similar benchmarks) on the other hand. The former is a contributive context attribution method, i.e., one that aims to find the sources \"that a model actually uses when generating a statement\" (lines 423-424), whereas the latter is based on GPT-4o annotations of whether context sentences \"support\" a statement in a general sense. I think the paper in general could discuss this distinction more clearly or more often. Regarding Section 3.4 specifically, given these somewhat different objectives, I think it is expected that the LongCite fine-tuned models in Table 1 are already slightly better than ContextCite in terms of LongBench-Cite performance. Similarly, this might explain the decent but inferior results of SFT (supervised fine-tuning) on ContextCite. What I find interesting and a bit unexpected is that using contributive attribution to re-rank citations can then improve LongBench-Cite performance, despite the (small?) mismatch in objectives.", "relation_to_broader_scientific_literature": "Section 5 discusses the relationships with 1) other work on teaching LLMs to generate citations as well as with 2) contributive context attribution and 3) self-supervised alignment in general. The main distinction with respect to 1) is the use of the techniques of 2) to further improve citation quality in a self-supervised manner. A longer-term goal, which the submission reports some results on, is to produce high-quality citations (as measured by benchmarks like LongBench-Cite) in a completely self-supervised manner.", "essential_references_not_discussed": "I cannot think of essential references that were not discussed, but please see \"Other Comments or Suggestions\" for additional references on contributive context attribution.", "other_strengths_and_weaknesses": "### Strengths\n- I think (as mentioned above) that it is a very interesting idea and finding that leveraging a somewhat different notion of \"citation\", namely importance in *causing* a model to generate a certain statement, can improve performance with respect to the evaluated notion of citation, namely logically supporting a statement in a general sense.\n- The paper reports on many ablations and alternatives: the fully self-supervised case of SFT on ContextCite, and all the subsections of Section 4 (different rewards, citation length balancing, preference optimization vs. SFT, etc.). The set of baselines considered (shown in Table 1) is also comprehensive.\n\n### Weaknesses\nSome aspects of best-of-N sampling (Section 2.3) are not clear to me:\n1. I am wondering why the text implies that best-of-N sampling has to be done \"after generating the full response\" (lines 154-155). Could it not also be done after generating each statement $r_i$?\n1. In re-sampling citation sequences in position $e_i$, are the future statements and citations $r_{i+1}, e_{i+1}, ...$ removed? The notation in Algorithm 1 suggests yes?\n1. Lines 132-134, right column, \"additional inference cost of generating candidates and re-ranking\": This inference cost should be quantified more precisely somewhere.", "other_comments_or_suggestions": "Minor questions and comments:\n- What constitutes a statement $r_i$ in the response? How is the response divided into statements?\n- Lines 112-113, right column, \"we exclude the BoN candidates that cites more than 384 tokens in total\": I believe this refers to the set of context sentences $c_{e_i^1}, ..., c_{e_i^m}$, not the sequence of identifiers $e_i^1, ..., e_i^m$, but it is not completely clear.\n- Equation (1): Were weighted sums of the probability drop and probability hold metrics also considered?\n- Table 2: Are the numbers in the second Llama-3.1-8B-Instruct row better than the ones in the first Llama-3.1-8B-Instruct row because the second one is for answering with citations (i.e. generating citations also improves answer correctness)?\n- The Limitations section could also note that SelfCite assumes access to the LLM's predicted probabilities, which may not always be available.\n\nAdditional references on contributive context attribution:\n1. G. Sarti et al. \"Inseq: An Interpretability Toolkit for Sequence Generation Models.\" ACL 2023.\n1. L. Monteiro Paes et al. \"Multi-Level Explanations for Generative Language Models.\" https://arxiv.org/abs/2403.14459", "questions_for_authors": "Please see Weaknesses under Other Strengths and Weaknesses.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: SelfCite is a novel self-supervised framework designed to improve the accuracy of citations made by Large Language Models (LLMs) when referencing provided context. The core idea involves using context ablation—measuring the change in response probability when cited sentences are removed or isolated—to generate necessity and sufficiency scores, creating a self-reward signal without human labels. This reward signal is then effectively utilized both during inference via best-of-N sampling and for training via preference optimization (like SimPO), leading to significant improvements in citation quality on benchmarks such as LongBench-Cite. Reviewers generally agreed on the merits of the proposed method. We highly recommend authors incorporate these feedbacks into the revision.
Decision: Accept (poster)