Investigating Hallucinations in Pruned Large Language Models for Abstractive Summarization (2024)

Abstract: Despite the remarkable performance of generative large language models (LLMs) on abstractive summarization, they face two significant challenges: their considerable size and tendency to hallucinate. Hallucinations are concerning because they erode reliability and raise safety issues. Pruning is a technique that reduces model size by removing redundant weights, enabling more efficient sparse inference. Pruned models yield downstream task performance comparable to the original, making them ideal alternatives when operating on a limited budget. However, the effect that pruning has upon hallucinations in abstractive summarization with LLMs has yet to be explored. In this paper, we provide an extensive empirical study across five summarization datasets, two state-of-the-art pruning methods, and five instruction-tuned LLMs. Surprisingly, we find that hallucinations are less prevalent from pruned LLMs than the original models. Our analysis suggests that pruned models tend to depend more on the source document for summary generation. This leads to a higher lexical overlap between the generated summary and the source document, which could be a reason for the reduction in hallucination risk.1

## introduction abstractive summarization is the task of distilling the key information from a document into a summary that may contain novel text not present in the original document @xcite @xcite . generative large language models (llms) have demonstrated strong performance on abstractive summarization @xcite @xcite @xcite . however, they face two significant challenges: their substantial risk has yet to be explored. given the appeal of greater efficiency with comparable downstream performance it is important to establish how trustworthy summaries generated from pruned models are. therefore, we seek to answer the following question: are hallucinations more or less prevalent in llms after pruning? to this end, we empirically investigate the risk of generating hallucinated content in pruned models across five llms, two state-of-the-art pruning methods, and five summarization datasets. surprisingly, our results show that pruned models are less prevalent in hallucinations compared to the original llm. to understand this phenomenon, we further investigate the impact of different sparsity levels on hallucination patterns. our analysis shows that hallucination risk decreases as sparsity increases, regardless of the pruning methods tested. furthermore, our results suggest that pruning encourages the model to rely more on the source document during generation, resulting in summaries that are lexically more similar to the source document. ## impact of pruning sparsity on hallucination risk to better understand previous observations and test our hypothesis (i.e., sparsity likely encourages models to focus more on the source document during generation), we analyze hallucination risk across different sparsity levels. we additionally track the lexical overlap (using rouge-1/2/l) and semantic overlap (using bertscore) between the generated summary and the source document. our hypothesis is: if lexical overlap positively correlates with sparsity levels, it suggests that pruned models may rely more on the source document for generation. figure 2 shows the summarization performance ratio (rouge-1/2/l and bertscore; ratio computed as pruned over original) and hrr (↓) for five llms and two pruning methods, across increasing levels of unstructured sparsity (10% to 50%). we only consider unstructured sparsity, since the 2:4 semi-structured pattern enforces a fixed sparsity level of 50%. the ratio for each metric is averaged across datasets for brevity, with error bars indicating standard deviation. for summarization performance, a ratio higher than 1.0 indicate that the pruned model performs better than the original, whereas a hrr lower than 1.0 indicates that summaries from the pruned model have a lower hallucination risk. ## qualitative analysis following the human evaluation (see sections 3.7 and 4.4), we review specific cases, highlighting issues with the summaries generated by pruned models in hallucinations. our analysis of the human evaluation task results suggests that hallucinations in the summaries from both llama-2 7b and mistral 7b are either: (a) additional information not supported by the source document, or (b) modified or misplaced information from the source document (e.g., factcc #205). omissions. omission is a category where we found a few instances of disagreement between the participants. in general, participants agree in clear cases like summeval #86 (e.g., @xcite . comparatively in disagreements, omitted information is more nuanced and difficult to detect, such as important details from the source document (e.g., missing dates). repetitions. interestingly, we find that summaries containing repetitions occur when the source document also contains repeating information (e.g., the price range ''@xmath0600,000'' duplicated in summeval #33). alignment. the generated summaries that are less aligned to the source document do not necessarily contain any hallucinations, omissions, or repetitions. however, we found that they do not entirely convey the original meaning of the source document. for example in factcc #136, the source describes deion sanders jr. being publicly scolded by his father for downplaying his wealthy lifestyle. however, this particular piece of information is not conveyed in the generated summary. ## conclusion we conducted an extensive study to assess the hallucination risk of llms after pruning. we experimented with two state-of-the-art pruning methods applied to five instruction-tuned llms. we measured the hallucination risk using three established automatic metrics, in addition to a human evaluation. our results show that as models are pruned to moderately high sparsity levels, the risk of generating hallucinating content decreases. our analysis suggests that pruned models tend to generate summaries that have a greater lexical overlap with the source document, offering a possible explanation for the lower hallucination risk. in future work, we plan to explore the relationship between hallucination risk and model quantization @xcite and also expand to tasks such as open-book question answering @xcite and machine translation @xcite . finally, an interesting direction is to investigate the relationship between hallucination risk and explanation faithfulness @xcite .