Attribute or Abstain: Large Language Models as Long Document Assistants (2024)

Abstract: LLMs can help humans working with long documents, but are known to hallucinate. *Attribution* can increase trust in LLM responses: The LLM provides evidence that supports its response, which enhances verifiability. Existing approaches to attribution have only been evaluated in RAG settings, where the initial retrieval confounds LLM performance. This is crucially different from the long document setting, where retrieval is not needed, but could help. Thus, a long document specific evaluation of attribution is missing. To fill this gap, we present LAB, a benchmark of 6 diverse long document tasks with attribution, and experiments with different approaches to attribution on 5 LLMs of different sizes. We find that *citation*, i.e. response generation and evidence extraction in one step, performs best for large and fine-tuned models, while additional retrieval can help for small, prompted models. We investigate whether the “Lost in the Middle” phenomenon exists for attribution, but do not find this. We also find that evidence quality can predict response quality on datasets with simple responses, but not so for complex responses, as models struggle with providing evidence for complex claims. We release code and data for further investigation. [Link](https://github.com/UKPLab/arxiv2024-attribute-or-abstain)

## introduction recent llms can process long documents @xcite , showing great potential as long document assistants. for example (fig. 1 ), such an assistant could answer a researcher's questions about a paper. however, due to llm hallucinations @xcite , the researcher must verify responses, which is difficult with lengthy papers. to improve verifiability and trust, the assistant should either attribute @xcite or abstain @xcite example from qasper @xcite . it finds the necessary information, it should provide a response and point to the evidence in the paper (attribute). if not, it should clearly communicate this (abstain). we investigate the capabilities of llms to fulfill these requirements, and the relation between response quality (i.e. correctness) and evidence quality (i.e. the relevance of the evidence to the response). thus far, attribution has only been investigated in retrieval augmented generation (rag) settings, where evidence comes from a large corpus (e.g. wikipedia) that does not fit the llm context. this means that some form of retrieval is required to provide the llm with a limited number of relevant passages or to retrieve evidence post-hoc. performance depends on retrieval quality, and the best-performing approach is unclear @xcite @xcite . in contrast, when the potential evidence is a single long document that fits the llm context window, the confounding retrieval can be omitted, as llms can cite from their input @xcite . still, it is possible that separating response generation and evidence retrieval improves performance, as shown in related works on task decomposition @xcite where @xcite found positive effects, while @xcite and @xcite did not. these works did not consider attribution, so there is a lack of knowledge on the effect of separating response generation and evidence retrieval, and the optimal approach to attribution on long documents. the document length poses additional challenges: recent works have found that llm performance on long-input tasks depends on the position of the information in the context @xcite @xcite . whether this can also be observed for attribution has not yet been investigated. evidence quality can be automatically evaluated without external reference @xcite @xcite . if evidence quality were positively correlated with response quality, bad responses could be filtered by identifying responses with bad evidence quality, thereby improving abstaining. if not, this would lead to abstaining from potentially helpful responses with insufficient evidence. current research on the relation of response quality and evidence quality is inconclusive: @xcite and @xcite reranked multiple sampled responses by evidence quality. while the former found an improvement in response quality, the latter did not. neither provide an analysis, so we lack understanding if and how evidence quality correlates with response quality. to close these gaps, we compile lab, a long-document attribution benchmark of 6 longdocument datasets with diverse tasks (qa, classification, fact checking, nli and summarization) and domains @xcite . we conduct experiments using approaches to attribution with and without additional retrieval on 5 llms of varying sizes, prompted and fine-tuned, to answer three research questions: rq1: what are optimal approaches for attribution in long document tasks? we find that large and fine-tuned models reach best evidence quality by directly citing from input documents, but small prompted llms can benefit from posthoc evidence retrieval. rq2: do llms exhibit positional biases in evidence attribution? concerning evidence retrieval, except for govreport, we find no particular bias, as the predicted and gold evidence distributions are mostly similar. however, we find that response quality generally decreases as evidence appears later in the document. rq3: what is the relation between evidence quality and response quality? we find that evidence quality can predict response quality on datasets with single-fact responses, but not so for multi-fact responses, as models struggle with providing evidence for complex claims. to evaluate evidence quality automatically, most works have used true, flan-t5-xxl fine-tuned on several nli datasets @xcite @xcite @xcite . more recently, attrscore @xcite and @xcite were proposed specifically for the evaluation of attributability. we compare these models and employ the best-performing for attributability evaluation. ## formalization we first define the task of producing attributed responses. based on this, we define the approaches to attribution compared in this work. ## methods we first introduce the datasets ( §4.1) and evaluation metrics ( §4.2) used in lab, our long document attribution benchmark. we then describe our experimental setup for generation ( §4.3) and retrieval ( §4.4). ## conclusion in our experiments on lab, we found that citation is a promising approach to attribution for large or fine-tuned models, while for small prompted models, post-hoc extraction can improve performance. we did not find a "lost in the middle" effect, but negative correlation between evidence position and response quality in some cases. finally, we showed that evidence quality can predict response quality for responses with low complexity. we hope that our results, code and data spur further research on long document assistants, most prominently: (1) improving the citation capabilities of llms for complex responses. (2) combining attributability evaluation models and iterative self-refinement @xcite to try to improve abstained responses.