SelfCite: Self-Supervised Alignment for Context Attribution in Large
  Language Models
Authors: name: Yung-Sung Chuang; name: Benjamin Cohen-Wang; name: Shannon Zejiang Shen; name: Zhaofeng Wu; name: Hu Xu; name: Xi Victoria Lin; name: James Glass; name: Shang-Wen Li; name: Wen-tau Yih
Published: 2025-02-13 18:55:13

Abstract: We introduce SelfCite, a novel self-supervised approach that aligns LLMs to
generate high-quality, fine-grained, sentence-level citations for the
statements in their generated responses. Instead of only relying on costly and
labor-intensive annotations, SelfCite leverages a reward signal provided by the
LLM itself through context ablation: If a citation is necessary, removing the
cited text from the context should prevent the same response; if sufficient,
retaining the cited text alone should preserve the same response. This reward
can guide the inference-time best-of-N sampling strategy to improve citation
quality significantly, as well as be used in preference optimization to
directly fine-tune the models for generating better citations. The
effectiveness of SelfCite is demonstrated by increasing citation F1 up to 5.3
points on the LongBench-Cite benchmark across five long-form question answering
tasks.