Unsupervised Token-level Hallucination Detection from Summary Generation By-products (2022)

Abstract: Hallucinations in abstractive summarization are model generations that are unfaithful to the source document. Current methods for detecting hallucinations operate mostly on noun phrases and named entities, and restrict themselves to the XSum dataset, which is known to have hallucinations in 3 out of 4 training examples (Maynez et al., 2020). We instead consider the CNN/DailyMail dataset where the summarization model has not seen abnormally many hallucinations during training. We automatically detect candidate hallucinations at the token level, irrespective of its part of speech. Our detection comes essentially for free, as we only use information the model already produces during generation of the summary. This enables practitioners to jointly generate a summary and identify possible hallucinations, with minimal overhead. We repurpose an existing factuality dataset and create our own token-level annotations. The evaluation on these two datasets shows that our model achieves better precision-recall tradeoffs than its competitors, which additionally require a model forward pass.

## introduction large pretrained transformers @xcite have considerably advanced the state of the art in abstractive summarization @xcite @xcite . however, model hallucinations -where the information in the generated summary is not faithful to the source document -are a prominent remaining failure mode of these models. a lot of recent work has addressed this problem, predominantly on the xsum dataset @xcite . xsum is an outlier, however, in that over 75% of its reference summaries contain hallucinations @xcite . models trained (or finetuned) on this dataset are consequently prone to hallucinate themselves when summarizing an article. additionaly, current work focuses on detecting hallucinations for noun phrases and named entities @xcite @xcite , sometimes with the addition of dates and numbers @xcite . recent work has shown, however, that summarization models also make mistakes in other parts of speech, such as predicates @xcite . in this paper, we aim to expand the current line of research to a different dataset, and to remove the restriction to entities. we use the diagonal crossattention patterns present in transformer-based abstractive summarization models (see figure 1 ) to align the summary with the source document. we detect hallucinations in an unsupervised fashion for segments of aligned and unaligned tokens by computing statistics from the encoder's self-attentions and the decoder's next-word probabilities. these by-products arise when generating a summary with any transformer model. in this paper, we use bart @xcite . we evaluate our approach on two datasets. 1 we repurpose the factuality dataset frank @xcite , but only 0.4% of tokens turn out to be hallucinations. therefore, we additionally create our own dataset called tlhd-cnndm, which contains token-level annotations on examples heuristically selected to have a higher chance of containing a hallucination. indeed 14.2% of tokens in tlhd-cnndm are hallucinations. our method demonstrates good results compared to its competitors, while at the same time requiring negligible additional computation. at the same time, hallucination detection proves to be a difficult task, in particular on intrinsic hallucinations (defined in section 3), where all models struggle to detect any hallucinations. ## hallucination detection definition. we adopt the definition from @xcite , and define intrinsic hallucinations as combinations of information from the source document that cannot be inferred from it, and extrinsic hallucinations as information that is not present in the source document. paraphrases and information that can be directly inferred from the source document, however, do not constitute hallucinations. furthermore, whether some information is a hallucination is an orthogonal problem to whether that information is factually correct, a question we do not consider in this paper. ## experiments we study cnn/dailymail @xcite , a summarization dataset known to be highly extractive @xcite and therefore less likely to contain a lot of hallucinations. ## conclusion we have presented bart-gbp, a method to detect hallucinations from the by-products of summary generation of a bart abstractive summarization model, trained and evaluated on cnn/dailymail. we first aligned the segments of the summary and source document using cross-attentions, and then used encoder self-attentions and decoding probabilities to detect intrinsic and extrinsic hallucinations, respectively. this happens with minimal computational overhead, compared to prior work that uses external models that require an additional model forward pass. our evaluations show that this is a difficult task, and especially intrinsic hallucination detection needs to be addressed by future work. we hope to contribute to this endeavor with our method and our token-level annotated dataset, tlhd-cnndm.