Better Late Than Never: Model-Agnostic Hallucination Post-Processing Framework Towards Clinical Text Summarization (2024)

Abstract: Clinical text summarization has proven successful in generating concise and coherent summaries. However, these summaries may include unintended text with hallucinations, which can mislead clinicians and patients. Existing methods for mitigating hallucinations can be categorized into task-specific and task-agnostic approaches. Task-specific methods lack versatility for real-world applicability. Meanwhile, task-agnostic methods are not model-agnostic, so they require retraining for different models, resulting in considerable computational costs. To address these challenges, we propose MEDAL, a model-agnostic framework designed to post-process medical hallucinations. MEDAL can seamlessly integrate with any medical summarization model, requiring no additional computational overhead. MEDAL comprises a medical infilling model and a hallucination correction model. The infilling model generates non-factual summaries with common errors to train the correction model. The correction model is incorporated with a selfexamination mechanism to activate its cognitive capability. We conduct comprehensive experiments using 11 widely accepted metrics on 7 baseline models across 3 medical text summarization tasks. MEDAL demonstrates superior performance in correcting hallucinations when applied to summaries generated by pre-trained language models and large language models.1

heading: 1 Introduction
text: Given the widespread use of electronic health records (EHR) (e.g., health questions, radiology reports, and doctor-patient dialogues), medical text summarization enhances healthcare efficiency (Van Veen et al., 2023b). However, manual summarization is notably laborious and time-consuming (Rink et al., 2023), leading to potential errors under
∗Corresponding author. 1Our code is available at https://github.com/lisdarr
/MEDAL.
lighted in red, signifying sections that should remain “unchanged” and “hemothorax”. Extrinsic errors are highlighted in green, indicating the issues of imposing causality and introducing additional details.
overwhelming workloads. Encouragingly, there is a rising focus on clinical text summarization (He et al., 2021; Dai et al., 2021; Joshi et al., 2020) to alleviate clinicians’ burden and enable them to refocus on patient-centered care.
However, clinical text summarization may engender hallucinations, resulting in intrinsic hallucinations (facts that contradict the source text) or extrinsic hallucinations (facts that cannot be directly inferred from the source text) (Maynez et al., 2020; Zhang et al., 2023), as illustrated in Figure 1. Hallucinations may mislead clinicians and patients, posing significant medical risks. Mitigating hallucinations is imperative for the seamless integration of automated summarization into clinical workflows.
Numerous methods have been proposed to address hallucinations in general summarization tasks (Ji et al., 2023; Li et al., 2022). However, these methods lack tailored designs for medical contexts, limiting their direct application to the medical domain. Recently, specific techniques for tackling hallucinations in medical summarization have been introduced. Adams et al. (2022) suggest selectively
995
rewrite unsupported reference sentences to improve the training set. Meanwhile, Zhang et al. (2020) and Alambo et al. (2022) focus on enhancing summarization models via reinforcement learning and knowledge-guided multi-objective optimization, respectively. Nevertheless, these methods are unsuitable for clinical deployment as they are confined to a specific type of medical text. To address this challenge, Zhang et al. (2023) introduce a finetuning strategy with wide applicability. However, its contrastive learning component demands highquality training data and substantial computational costs. Moreover, the rise of large language models (LLMs) escalates fine-tuning costs due to the increasing model parameters.
We propose MEDAL, a model-agnostic postprocessing framework designed to address medical hallucinations in various summarization tasks. MEDAL comprises two components: 1) a hallucination correction model for post-processing, and 2) a medical infilling model to generate training data for the correction model. The hallucination correction model is built upon Flan-T5 (Chung et al., 2022), a model fine-tuned with instructions. Our hallucination correction model fully leverages the cognitive capabilities of Flan-T5 through selfexamination. The model is guided to examine hallucinations in a model-generated summary and then correct the summary based on its self-examination. Specifically, we introduce a self-examination token that triggers the hallucination correction model to refine its generation process. Furthermore, we employ the medical infilling model to create nonfactual summaries with errors aligned with those found in model-generated summaries. This approach aims to enhance the correction model’s capacity to rectify common errors in automated medical summaries. Specifically, we enhance a traditional infilling model (Donahue et al., 2020) by constructing medical Cloze questions. These medical Cloze questions help incorporate medical knowledge into the infilling model during training and are utilized to generate non-factual summaries during inference.
We conduct extensive experiments to evaluate the performance of MEDAL across three tasks: health question summarization, radiology report summarization, and doctor-patient dialogue summarization. First, MEDAL has consistently achieved optimal or near-optimal performance using widely accepted general quality metrics and faithfulness metrics across three medical text sum-
marization tasks. Second, MEDAL effectively rectifies hallucinations in summaries generated by LLMs, demonstrating its potential for future application. Third, we demonstrate MEDAL’s robustness with the imbalanced training set. Finally, further studies are conducted to comprehensively explain the effectiveness of self-examination.

heading: 2 Related Work
text: Medical text summarization Medical text summarization encompasses three key tasks: 1) health question summarization condenses an inquiry into a concise question to elicit accurate responses; 2) radiology report summarization condenses a detailed “findings” section into an “impression” section, capturing the most pertinent and actionable information; and 3) doctor-patient dialogue summarization aims to encapsulate full conversations into a succinct paragraph.
Health question summarization is introduced by Ben Abacha and Demner-Fushman (2019). Recent models have achieved performance improvements using reinforcement learning (Yadav et al., 2021a), contrastive learning (Zhang et al., 2022), transfer learning (Yadav et al., 2021b), multi-task learning (Mrini et al., 2021), and question-aware transformer models (Yadav et al., 2022). For radiology report summarization, most prior works leverage pre-extracted knowledge like ontologies and word graphs (MacAvaney et al., 2019; Sotudeh Gharebagh et al., 2020; Hu et al., 2021). Hu et al. (2022b) and Karn et al. (2022) advocate for contrastive learning and reinforcement learning, respectively. Recently, attention has shifted towards employing LLMs (Van Veen et al., 2023a; Karn et al., 2023). Doctor-patient dialogue summarization is early studied with pointer generator networks (Joshi et al., 2020). Krishna et al. (2021) explore summarization methods, while Navarro et al. (2022) focus on the few-shot setting. Zhang et al. (2021) focus on handling long conversations by fine-tuning BART. MedDialog (Zeng et al., 2020) and ACI-BENCH (Yim et al., 2023) introduce datasets to advance the research.
Differing from the task-specific methods mentioned above, we propose a model-agnostic framework suitable for all these tasks, showcasing consistent strong performance across these tasks.
Mitigating hallucinations in medical text summarization Researches on hallucinations in medical text summarization primarily focus on training
data and training method optimization. Adams et al. (2022) selectively rewrite unsupported reference sentences to better reflect source data. Zhang et al. (2020) utilize reinforcement learning, while Xie et al. (2023) propose FactReranker which selects the best summary based on an estimated factual consistency score. Krishna et al. (2021) propose an algorithm that extracts and clusters pertinent utterances to generate one summary sentence per cluster. Zhang et al. (2023) introduce a fine-tuning strategy using contrastive learning. However, its resourceintensive nature and requirement for high-quality training data pose challenges for application.
To the best of our knowledge, we are the first to propose a post-processing framework to rectify summaries generated by medical summarization models. Moreover, our framework is versatile and lightweight compared to previous methods.

heading: 3 Method
text: In this section, we first provide an overview of our proposed framework MEDAL (§3.1). Sequentially, we detail the construction of medical Cloze questions (§3.2). Finally, we delve into the architectures of the medical infilling model (§3.3) and the hallucination correction model (§3.4).

heading: 3.1 Overview
text: For the medical text summarization task, the summarization model generates a summary s′ based on a given medical source text m. However, the model-generated summary s′, namely a dafted summary, may contain hallucinations. Our objective is to rectify hallucinations and align the corrected summary s more closely with the reference summary r.
MEDAL is a model-agnostic hallucination postprocessing framework without retraining summarization models, resulting in low costs and wide application. As illustrated in Figure 2, the MEDAL framework involves three essential steps.
First, we mask terms that are error-prone during generation to construct medical Cloze questions. These masked terms are selected based on the taxonomy of faithfulness errors in medical summaries (Zhang et al., 2023).
Second, the medical infilling model, denoted as MI , utilizes the medical Cloze questions to generate non-factual summaries. During the training phase, the model’s primary objective is to acquire medical knowledge. Consequently, it is trained
to complete the medical Cloze task. During the inference phase, motivated by Goyal and Durrett (2020), we select lower-ranked token candidates as output to create non-factual summaries. These non-factual summaries manifest subtle deviations from the reference summaries.
Third, the hallucination correction model, denoted asMC , is built upon Flan-T5 (Chung et al., 2022). To activate the cognitive capability of FlanT5, we guide it towards a self-examination process. This innovative addition allows the model to initially examine hallucinations in the summary. Subsequently, the model refines its generation process. MC is trained using both the non-factual summaries generated byMI and reference summaries, thereby enhancing its capacity to rectify hallucinations in summaries.

heading: 3.2 Construction of medical Cloze questions
text: We employ diverse tools to identify error-prone terms in a medical text. Common errors appear in entity, entity relation, quantifier, and negation. We outline them in Appendix A. We utilize MedCAT2 to extract medical entities. Additionally, we leverage Stanford OpenIE3 to detect the relation triples. Quantifiers are identified through regular expression matching techniques, while negations are recognized based on a predefined set of negation terms {“no”, “nope”, “doesn’t”, “don’t”, “not”}.
Given a medical text p, we use the mentioned tools to identify all error-prone terms in the text, forming the set T . Subsequently, each identified error-prone term is replaced with a [MASK] token to form a Cloze question. This process is formalized as the function GENCLOZE(p, T ), which yields the set of medical Cloze questions, denoted as Q:
Q =GENCLOZE(p, T )
={(clz, t) | clz = replace(p, t, [MASK]), t ∈ T}
(1)
where the funtion replace(p, t, [MASK]) replaces the term t in the medical text p with “[MASK]” and returns the modified text.
2MedCAT is an open-source medical concept annotation toolkit proposed by Kraljevic et al. (2021).
3Stanford OpenIE is an open information extraction tool proposed by Angeli et al. (2015). Open information extraction (OpenIE) aims to extract relation tuples from text without predefined relations. The relation name is just the text linking two arguments.

heading: 3.3 Medical infilling model
text: The medical infilling modelMI is based on BioBART (Yuan et al., 2022). The input is the concatenation of the sentence with the [MASK] token ŝ and its context c, while the target output is the original error-prone term t replaced by the [MASK] token.
Training The training set is generated from medical source texts, where error-prone terms are identified and replaced with the [MASK] token. Algorithm 1 shows the method for generating the training set. During training, the model learns to predict the masked terms based on their context. By optimizing this objective, the model acquires a deeper understanding of medical terminology and context.
Inference The inference phase aims to produce non-factual summaries that manifest subtle deviations from the reference summaries. Therefore, we generate the input dataset utilizing the reference summaries. The detailed procedure is outlined in Algorithm 2.
During inference, the trained medical infilling modelMI employs beam-search decoding to generate various token candidates. An example is depicted in the Decoding Example section of Step 2 in Figure 2. Lower-ranked candidates often exhibit semantics or parts of speech similar to the original
term but lack factual accuracy (Zhang et al., 2023). Therefore, we select lower-ranked candidates as non-factual replacements for the masked term. The method for selecting the lower-ranking range is detailed in Appendix B. The non-factual summary, denoted as r′, is produced by replacing the [MASK] token in ŝ with the token candidate.

heading: 3.4 Hallucination correction model
text: The backbone of the hallucination correction model MC is Flan-T5 (Chung et al., 2022), known for its abilities in reasoning and generalization. To engage the cognitive capabilities of Flan-T5, we design an instruction (shown in Step 3 of Figure 2), directing the model to examine hallucinations before generating the corrected summary. We introduce a self-examination token, where [YES] indicates the presence of hallucinations and [NO] indicates their absence.
The input of the hallucination correction model comprises a prompt sentence, a medical source text m, and a summary, while the target output is a combination of either [NO] or [YES] along with the corrected summary s. The detailed input and output formats are listed in Appendix D.5.
Training To prevent the model from modifying factual summaries, we structure the training set
Algorithm 1 Training Set Generation Input: The medical source text, m; the set of error-
prone terms identified in the medical source text, Tm;
Output: The training set, Dtrain; 1: Q← GENCLOZE(m, Tm) 2: Dtrain ← empty set 3: for each (cloze, term) in Q do 4: c← "" 5: sents← sentence_tokenize(cloze) 6: for each sent in sents do 7: if "[MASK]" in sent then 8: ŝ← sent 9: t← term 10: else 11: c = c+ sent 12: end if 13: end for 14: add (ŝ, c, t) to set Dtrain 15: end for 16: return Dtrain
with both positive samples and negative samples. Positive samples account for 40% of the dataset, while negative samples consist of the remaining 60%. The summaries in positive samples are from reference summaries, whereas the summaries in negative samples are non-factual summaries generated byMI .
Inference During inference, the summary of each input is the model-generated summary s′. We extract the corrected summaries from the output of the correction model.

heading: 4 Experiment Setup

heading: 4.1 Datasets
text: To illustrate MEDAL’s efficacy in medical text summarization, we conduct experiments on three distinct types of medical texts: health questions, radiology reports, and doctor-patient dialogues. The statistics of the datasets are listed in Table 1.
Health questions We employ Health Question Summarization (HQS) from MEDIQA 2021 shared task 1 (Ben Abacha et al., 2021). The training set is from the MeQSum Dataset (Ben Abacha and Demner-Fushman, 2019). The validation and test sets include diverse consumer health questions received by the U.S. National Library of Medicine, along with expert-generated gold summaries.
Algorithm 2 Inference Dataset Generation Input: The medical source text, m; the reference
summary, r; the set of error-prone terms identified in the reference summary, Tr;
Output: The inference set, Dinfer; 1: Q← GENCLOZE(r, Tr) 2: Dinfer ← empty set 3: for each (cloze, term) in Q do 4: ŝ← cloze 5: c← m 6: t← term 7: add (ŝ, c, t) to set Dinfer 8: end for 9: return Dinfer
Radiology reports We utilize Radiology Report Summarization (RRS) from MEDIQA 2021 shared task 3 (Ben Abacha et al., 2021). The source of RRS is the MIMIC-CXR chest X-ray report dataset4 and the Indiana University chest X-ray dataset5. The validation set is selected from the Indiana dataset, and the test set combines the Indiana dataset with newly released reports drawn from the Stanford Health Care system.
Doctor-patient dialogues Most datasets are private due to ethical limitations. Therefore, we utilize the public ACI-BENCH dataset (Yim et al., 2023) with full doctor-patient conversations and associated clinical notes.6 Van Veen et al. (2023b) select 126 samples containing an “assessment and plan” section from the dataset. We follow their work and divide the 126 samples according to the proportions of each set in the original ACI-BENCH dataset.

heading: 4.2 Baselines
text: Pre-trained language models We fine-tune Pegasus-large (Zhang et al., 2020), a abstractive summarization model; and BioBART-large (Yuan et al., 2022), a biomedical generative model.
Task-agnostic medical summarization model We evaluate MEDAL against the state-of-the-art (SOTA) model FAMESUMM (Zhang et al., 2023), which performs contrastive learning to encourage the faithful generation of medical terms.
4https://physionet.org/content/mimic-cxr/2.0.0/ 5https://openi.nlm.nih.gov/faq#collection 6A clinical note contains four sections: 1) subjective information reported by the patient; 2) objective observations; 3) assessments made by the doctor; and 4) a plan for future care (Krishna et al., 2021).

heading: Task-specific medical summarization models
text: We select the SOTA methods for each medical summarization task. For health question summarization, we select QFCL7 (Zhang et al., 2022) which employs contrastive learning. For radiology report summarization, we choose WGSum+CL (Hu et al., 2022b) combining graphs and contrastive learning. For doctor-patient dialogue summarization, taskspecific methods are fine-tuned on large private datasets. Given the paucity of public training data, these methods are not suitable as baselines.
Large language models We select two opensource large language models: 1) Llama-2 (7B) (Touvron et al., 2023) and 2) Med-Alpaca (Han et al., 2023) specifically fine-tuned for medical question-answering and dialogue applications.

heading: 4.3 Metrics
text: We adopt comprehensive evaluation metrics.8 In addition to the general quality metrics and the faithfulness metrics, we incorporate the latest evaluation metrics for automated medical note generation proposed by Ben Abacha et al. (2023).
General quality metrics Two reference-based metrics, ROUGE-1/2/L and BERTSCORE, are used to evaluate the general quality of summaries. ROUGE-1/2/L (Lin, 2004) is used to measure the count of overlapping units, while BERTSCORE (Zhang* et al., 2020) calculates a similarity score between the generated and reference summary.
General faithfulness metrics QUESTEVAL (Scialom et al., 2021) evaluates whether the generated and reference summaries contain the same information based on question answering. SUMMAC (Laban et al., 2022) measures factual consistency by natural language inference (NLI).
7QFCL is a competitive method for health question summarization. ProphetNet+QTR+QFR (Yadav et al., 2021a) is not selected because the method requires manually labeled question focuses and question types.
8Calculation details are described in Appendix C.
Task-specific faithfulness metrics FaR (Shing et al., 2021) and Concept F1 (C F1) (Joshi et al., 2020) are computed based on medical entities. Recently, Ben Abacha et al. (2023) introduce three metrics, MedBERTScore, MedBARTScore, and ClinicalBLEURT.

heading: 4.4 Implementation details
text: We use biobart-large (Yuan et al., 2022) and flan-t5-base (Chung et al., 2022) as the backbones of the infilling modelMI and the correction modelMC , respectively. Notably, the length of the doctor-patient dialogues exceeds Flan-T5’s maximum context length. We select relevant supporting sentences as the source input of the correction model. More details are presented in Appendix D.

heading: 5 Results and Analysis
text: In this section, we aim to answer the following research questions: • RQ1: Does MEDAL perform well across differ-
ent medical text summarization tasks on general quality metrics and faithfulness metrics (§5.1)? • RQ2: Does the self-examination process activate MEDAL’s cognitive capability (§5.2), and if affirmative, what contributes to its efficacy (§5.4)? • RQ3: Does MEDAL exhibit robustness when trained with imbalanced samples (§5.3)?

heading: 5.1 Overall results
text: The overall results for the three tasks are shown in Table 2, 3, and 4, respectively.
Improvements over medical summarization models First, MEDAL achieves SOTA or nearSOTA performance in three medical text summarization tasks. Compared with the competitive baseline FAMESUMM, MEDAL requires fewer computational costs. FAMESUMM is a fine-tuning method which needs retraining when applied to different summarization models. Besides, FAMESUMM involves learning from contrastive summaries, leading to high requirements on GPU memory. In con-
best are in blue. Baselines are reproduced using the official implementation for metric computation. We do not incorporate FAMESUMM into LLMs due to the substantial computational costs and time required.

heading: Model ROUGE-L FaR C F1 ClinicalBLEURT MedBERT
text: trast, MEDAL is model-agnostic and can directly optimize various summarization models.
Second, MEDAL exhibits significant improvements on SUMMAC, FaR, and C F1 metrics across all baselines, indicating that summaries generated by MEDAL include more crucial medical entities. Furthermore, the general quality metrics also show improved results after the application of MEDAL, implying an enhancement in coherence and fluency.
Improvements over large language models First, MEDAL maintains its effectiveness when dealing with summaries generated by LLMs. Notably, the performance of Llama-2 obtains significant improvement with the correction by MEDAL. Take the health question summarization as an example. There is an increase of approximately 6% on the SUMMAC metric and around 7% on the FaR metric. Furthermore, the performance of Med-
Alpaca also improves further. Experiments on LLMs illustrate the capability of MEDAL to identify and rectify hallucinations in LLMs’ outputs.
Second, Med-Alpaca exhibits superior performance compared to Llama-2 on health question summarization and radiology report summarization. Take radiology report summarization as an example. Med-Alpaca outperforms Llama-2 by approximately 18% on the SUMMAC metric and demonstrates an improvement of around 9% on the FaR metric. MEDAL further improves the MedAlpaca’s performance by about 2% on the SUMMAC metric and 3% on the FaR metric.
Third, for doctor-patient dialogue summarization, Med-Alpaca performs significantly worse than Llama-2, highlighting the limitations of MedAlpaca in handling long texts. To address the context length limitation of our correction model, the source text is not directly used as input. Instead, we select relevant supporting sentences as the source text for the correction model.9 Experimental results indicate that this approach effectively corrects erroneous content in long-text summarization, thereby enhancing the faithfulness of the summaries.
9The selection method is detailed in Appendix D.4.

heading: 5.2 Ablation study
text: Self-examination is a crucial component of our method. To substantiate its effectiveness, we conduct an ablation study.10 The experimental results are shown in Table 5. Without self-examination, the performance of MEDAL exhibits varying degrees of decline. Remarkably, the results without self-examination are even worse than those before correction. This indicates that without selfexamination, the model may incorrectly alter parts of the summary that are originally correct. Therefore, self-examination plays a pivotal role, prompting the model to consider the existence of hallucinations before rectifying the summaries.

heading: 5.3 Robustness of MEDAL
text: We conduct experiments to investigate the impact of the balance between positive and negative samples on the hallucination correction model. The results depicted in Table 6 indicate MEDAL’s superior performance on balanced datasets (e.g., 40%, 50%, and 60% positive samples). Notably, MEDAL exhibits resilience even in scenarios of dataset imbalance. These findings highlight MEDAL’s robustness to maintain efficacy across varied proportions of positive and negative samples in the training dataset.
10Implementation details can be found in Appendix D.5.

heading: 5.4 Further study
text: To delve deeper into the effectiveness of selfexamination, we employ heatmap to visualize the attention matrix of the correction model. Figure 3 illustrates that the self-examination process prompts the model to allocate more attention to important medical entities. Additionally, we provide some examples of outputs from MEDAL and baselines in Appendix D.6 to show the effectiveness of MEDAL.

heading: 6 Conclusion
text: In this paper, we propose MEDAL, a modelagnostic post-processing framework designed to rectify hallucinations in medical summaries. MEDAL comprises a medical infilling model and a hallucination correction model. The correction model is trained using synthetic datasets generated by the infilling model. A self-examination mechanism is incorporated into the correction model to enhance its performance. Experimental results demonstrate MEDAL’s superiority across
11 metrics in 3 medical text summarization tasks. MEDAL is model-agnostic and can post-process summaries to enhance their reliability. Furthermore, MEDAL exhibits potential in rectifying summaries generated by LLMs, highlighting its prospective application in the era of LLMs.

heading: Acknowledgements
text: We appreciate the valuable insights provided by the anonymous reviewers. This work is supported by National Key Research and Development Program of China (No. 2022YFC3302600).

heading: Limitations
text: The effectiveness of MEDAL’s correction model relies on the quality and diversity of its training data, particularly the non-factual summaries tailored for hallucination correction. However, limitations arise as the errors in these non-factual summaries predominantly pertain to the entity levels and relation levels. While MEDAL has made significant improvements in mitigating hallucinations related to medical concepts, relations, and negations, it may not fully address more complex issues like contextual misunderstandings or semantic ambiguities. These complex errors pose challenges in detection due to the need for logical reasoning. Moreover, generating non-factual summaries containing these errors becomes more challenging.
Another consideration is the text length limitation of MEDAL’s correction model. We select relevant supporting sentences as the source text for the doctor-patient dialogue summarization task. However, the rules of selecting relevant supporting sentences are based on ROUGE-L score and lack of the consideration of semantic information.
We will investigate new approaches capable of addressing a wider range of errors and long clinical text to improve the overall performance of medical text summarization.

heading: Ethics Statement
text: Our research rigorously adheres to ethical standards by using publicly available datasets devoid of personally identifiable information. The study aims to mitigate hallucinations in medical summaries by a post-processing framework. Despite the exceptional performance of MEDAL in mitigating hallucinations, it is important to acknowledge the possibility of residual errors in its outputs. Therefore, prior to implementation, it is imperative
for medical practitioners to undertake thorough assessments of faithfulness and accuracy.

heading: A Error Types
text: The common error types in medical text summarization are illustrated in Table 7.

heading: B Selection of The Lower-Ranking Range
text: During the inference phase, the decoding beam size is set to 15 for predicting token candidates. We select 200 medical Cloze questions and compute the ranking at which the generated tokens are inconsistent with the original tokens. Subsequently, we establish a threshold of 5 to ensure non-factual replacements occur over 95% of the time. Finally, token candidates ranked between 5 and 15 (i.e., range=[5,15]) are chosen as the non-factual replacements.

heading: C Metrics

heading: C.1 General quality metrics
text: ROUGE-1/2/L Following Zhang et al. (2023), we select ROUGE-N (N ∈ {1, 2}) and ROUGE-L as evaluation metrics. Lin (2004) propose ROUGE to measure the similarity between summaries. Given a candidate summary s and a reference summary r, ROUGE-N refers to the overlap of n-gram between s and r. ROUGE-L is computed based on the longest common subsequence (LCS) between s and r. We adopt the pyROUGE11 to compute the ROUGE scores and select the f_score as the reported result.
BERTSCORE BERTSCORE computes the similarity of two sentences as a sum of cosine similarities between their tokens’ embeddings (Zhang* et al., 2020). We select allenai/scibert_scivocab_cased12 as the BERT model to generate contextual embeddings.

heading: C.2 General faithfulness metrics
text: QUESTEVAL QUESTEVAL (Scialom et al., 2021) is proposed to evaluate summarization systems without requiring reference summaries. It consists of a question generation model QG and a question answering model QA. For the question answering model QA, QA(a|M, q) refers to the probability of the answer a to question q on a text M , and QA(M, q) is the answer greedily
11pyROUGE is a Python wrapper for the ROUGE summarization evaluation package. It can be installed from https://pypi.org/project/pyrouge/
12https://huggingface.co/allenai/scibert_scivocab_cased
generated from the model. For the question generation model QG, QG(M) is denoted as the set of question-answer pairs (q, a) for a text M such that QA(M, q) = a. QUESTEVAL is computed as:
Prec(M,S) = 1 |QG(S)| ∑
(q,a)∈QG(S) F1(QA(M, q), a)
(2)
Rec(M,S) =
∑ (q,a)∈QG(M) W (q,M)(1−QA(ϵ|S, q)) ∑
(q,a)∈QG(M) W (q,M)
(3)
FScore(M,S) = 2 · Prec(M,S) ·Rec(M,S) Prec(M,S) +Rec(M,S) (4)
where M and S are two sequences of tokens with M denoting the source text and S representing the corresponding evaluated summary, W (q,M) is the weight of query q for text M , ϵ is the unanswerable token denoting that a summary contains no answer.
Following Scialom et al. (2021), the final QUESTEVAL score is the FScore(M,S). We calculate it using the released package.13
SUMMAC SUMMAC enables NLI models to be successfully used for inconsistency detection. Laban et al. (2022) first split the source text into M blocks and split the summary into N blocks. Then, an out-of-the-box NLI model is applied to generate an M × N NLI pair matrix, where each value in the matrix is the entailment score of the source text block and the corresponding summary block. Finally, they present two models, SUMMACConv and SUMMACZS. The model produces a single consistency score for a given summary by processing the pair matrix. Following Zhang et al. (2023), we adopt SUMMACZS in our evaluation by calling its package.14

heading: C.3 Task-specific faithfulness metrics
text: FaR Shing et al. (2021) illustrate the relationship between source text, reference summary, and model-generated summary using a Venn diagram as shown in Figure 4. The Faithfulness-adjusted Recall (FaR) measures the amount of faithful and relevant information that has been included by the model-generated summary. Therefore, FaR is defined as CB+C . Following Shing et al. (2021), we measure the rigions in Figure 4 by medical named
13https://pypi.org/project/questeval/ 14https://pypi.org/project/summac/

heading: Error Type Description Example
text: entity recognition (NER) system in scispaCy.15
Concept F1 Similar to FaR, Concept F1 is also computed based on the medical entities and we obtain the medical entities using the same method like FaR. According to the Venn diagram in Figure 4, Concept F1 is defined as:
Concept Rec = C + F
B + C + E + F (5)
Concept Prec = C + F
C +D + F +G (6)
Concept F1 = 2 · Concept Rec · Concept Prec Concept Rec + Concept Prec
(7)

heading: MedBERTScore and MedBARTScore
text: Ben Abacha et al. (2023) update the scoring policy of two popular evaluation metrics, BERTSCORE (Zhang* et al., 2020) and BARTSCORE (Yuan
15The scispaCy (Neumann et al., 2019) NER identifies any span in a given text which might be an entity in UMLS (a large biomedical database) and returns the identified medical entities.
et al., 2021), by providing a higher weight to the words in the summaries that have medical meaning. They identify the medical entities defined in UMLS by MedCAT toolkit16. We evaluate the summaries using the official instructions for MedBERTScore17 and MedBARTScore18.
ClinicalBLEURT ClinicalBLEURT relies on fine-tuning a model-based metric on a large collection of clinical notes including family medicine and orthopaedic notes. Ben Abacha et al. (2023) fine-tune BLEURT (Sellam et al., 2020) using a quality score. We use their provided fine-tuned model19 to complete evaluation.
D Implementation Details
We implement the experiments with a single NVIDIA GeForce RTX 4090 GPU. For each task, we report the average results across 3 runs. Further implementation details can be found in our code.
D.1 Experiments on pre-trained language models
We fine-tune two pre-trained language models, namely Pegasus-large (Zhang et al., 2020) and BioBART-large (Yuan et al., 2022). The hyperparameter settings are shown in Table 8. We select the best models based on the loss on the validation set for all tasks. Notably, these pre-trained language models are not applied to the doctor-patient dialogue summarization task because the length of
16https://pypi.org/project/medcat/ 17https://github.com/abachaa/EvaluationMetricsACL23/tree/main/MedBERTScore 18https://github.com/abachaa/EvaluationMetricsACL23/tree/main/MedBARTScore 19https://github.com/abachaa/EvaluationMetricsACL23/tree/main/ClinicalBLEURT
the doctor-patient dialogues exceeds the context length of Pegasus-large and BioBART-large.

heading: D.2 Experiments on LLMs
text: Following Van Veen et al. (2023b), the prompts for medical summarization tasks are depicted in Table 9. We fine-tune LLMs with LoRA (Hu et al., 2022a) using the Adam optimizer. Table 10 displays the hyper-parameters for fine-tuning in different tasks. The final model used for testing is obtained at the end of training, based on the specified number of epochs.

heading: D.3 Experiments on MEDAL
text: For the medical infilling modelMI and the hallucination correction modelMC , we select the best models based on the loss on the validation set. The hyper-parameter settings are shown in Table 11 and 12.
D.4 Selection of relevant supporting sentences
To adapt MEDAL to doctor-patient dialogue summarization, we select the most relevant sentences from the source text as input context instead of using the entire source text. For each sentence i in the model-generated summary, we choose a sentence from the source text that yields the highest ROUGE-L score computed with i. These selected sentences are then concatenated as the input source text.

heading: D.5 Ablation study
text: We show the input and output formats of the correction model in Table 13.
Besides, we implement further experiments to explore whether clear instructions can help the model perform better. The experiment results are shown in Table 14. The clear instructions indeed improve model performance.

heading: D.6 Examples of outputs
text: Figure 5 illustrates MEDAL’s effectiveness in correcting errors found in model-generated summaries. In the first two examples, the original outputs generated by summarization model contains the wrong entities and quantifiers. MEDAL accurately captures the key medical entities and generates a complete summary. Similarly, in the last example, Pegasus overlooks the negation “no”, whereas MEDAL generates the correct summary. These examples highlight MEDAL’s capability to correct hallucinations within summaries. We design medical Cloze questions based on common errors and employ an infilling model to generate non-factual summaries for training the correction model. Additionally, we incorporate a self-examination mechanism into the correction model, further enhancing its performance.

heading: Future Work
text: gov/faq#collection 6A clinical note contains four sections: 1) subjective information reported by the patient; 2) objective observations; 3) assessments made by the doctor; and 4) a plan for future care (Krishna et al. , 2021)