Fast Evidence Extraction for Grounded Language Model Outputs (2024)

Abstract: Summarizing documents with Large Language Models (LLMs) warrants a rigorous inspection of the resulting outputs by humans. However, unaided verification of generated outputs is time-intensive and intractable at scale. For high-stakes applications like healthcare where verification is necessary, expediting this step can unlock massive gains in productivity. In this paper, we focus on the task of evidence extraction for abstractive summarization: for each summary line, extract the corresponding evidence spans from a source document. Viewing this evidence extraction problem through the lens of extractive question answering, we train a set of fast and scalable hierarchical architectures: EarlyFusion, MidFusion, and LateFusion. Our experiments show that (i) our method outperforms the state-of-the-art by 1.4% relative F1-Score; (ii) our model architecture reduces latency by 4x over a RoBERTa-Large baseline; and (iii) pretraining on an extractive QA corpus confers positive transfer to evidence extraction, especially in low-resource regimes.

## introduction suppose we train an llm to summarize a doctorpatient conversation into a clinical note. such models could save physicians hours each day. however, an auditing step is still requisite. this auditing involves repeatedly diving through a long transcript to find relevant information for every detail that appears in the note (see fig. 1 ). without an automated mechanism that makes this process efficient, can we really say that we've saved a clinician any time? workflows that involve grounded tasks that operate on top of a source document (e.g. summarization, dialogue and translation) @xcite @xcite @xcite are well suited for llms @xcite @xcite . however, owing to the lingering limitations of these models, humans have remained firmly in the loop, providing last-mile verification of the model's outputs. in these setups, an individual may spend a significant amount of time on verification of llmgenerated first drafts. for grounded tasks, verifying each generated sentence can be broken down into two steps (i) locating a span of text from the much larger source document that has information related to that sentence; (ii) using the obtained span to form conclusions about correctness. with long sources (e.g. hour-long conversation transcripts), it's likely that carrying out the first step of extracting the right span of evidence proves more cumbersome than using the extracted evidence to make conclusions. furthermore, this problem is exacerbated as the transcript grows in length. therefore, we present automated evidence extraction (ee) as an efficient and scalable way to reduce verification time and fully realize the benefits of workflow automation. cold? yes, since 9a take advil source document patient has cold encoder encoder tile transformer cls embedding lstm + dense concatenate query-sentence cross attention sentence embeddings are cached and reused for speed cold? yes,since 9a take advil source document patient has cold encoder encoder cls embedding cls embedding concatenate into sequence concatenate embeddings lstm + dense logits ## related work innovations in better llm generations are plenty @xcite @xcite @xcite . however, our work is situated among post-hoc methods that serve to increase trust in these generations. with the tendency of llms to hallucinate @xcite there has been growing interest in post-hoc evaluation of the factuality of llm generations @xcite @xcite @xcite . our work considers applications where the aim is not to automatically evaluate each generation but to retrieve supporting material from the source to aid a human with verification. thus, while scoring the extent of factuality is useful, they cannot replace human spot-checking when an llm is deployed in a low-risk setting. while there are similarities with the line of work in @xcite ; @xcite ; @xcite that highlight regions of the input that have correlation with model predictions, they are closer to explaining predictions than explicitly retrieving supporting material. similar ideas also appear in multihop qa works @xcite al. (2023); tu et al. (2020); nishida et al. (2019 ), but differ in our focus on scale and domain adaptation. the methods in pruthi et al. (2020) tackle the ee problem in deep nlp, as we framed it, although they are limited to classification tasks. further, kryściński et al. (2019) builds ee and factuality verification models with weak supervision, but their method does not handle cross-sentence dependencies or coreference resolution. more recently, stammbach (2021); dehaven and scott (2023); krishna et al. (2023); wadden et al. (2021, 2020) all tackle the ee task, but are distinct given our focus on scalability, speed, and establishing the benefits of qa pretraining for ee. an open-source benchmark for domain # of examples train valid test biographies 3740 1875 3642 landmarks 0 0 211 disasters 247 122 256 newspapers 0 0 137 companies 162 75 156 schools 220 123 235 ee is introduced in @xcite along with the state-of-the-art methods on this dataset which we compare against. ## methodology we have a source document d made up of components @xmath0. unless mentioned otherwise, u is a sentence (we make explicit when u is a token). an operation (e.g. summarization) on d results in an output o. for each sentence @xmath1.g summary sentence) we need to find an evidence span e ⊂ u . we refer to q as query. intuitively, e should have sentences u that contain information relevant to q. ## experiment results naive baselines are not competitive from more data helps reduce performance gap in qa pretraining helps evidence extraction it is easy to see from gpt-4 generated synthetic data is useful from wide gap between ha and pr-metrics in in appendix d.1 we consider adding documentwide context using a transformer instead of an lstm. despite having fewer parameters, lstms seem to do better than transformers. method p(m) r(m) ha(m) p(am) r(am) ha(am) absence of an entity consider the following line inserted in an llm generated soap note: "extremities: no clubbing or cyanosis" appearing under physical exam (pe) section. the pe section is populated this way by default and then changed if an issue is discussed. here, we need to surface evidence that discussion about clubbing/cyanosis is not part of the conversation. this is a failure mode, perhaps for the problem setup itself, since the complete evidence is the entire transcript. when wrong is it really wrong? often the predicted evidence is reasonable but does not score since it is an alternate source of evidence: query: the patient to continue with the lower dosage of trulicity if it alleviates the symptoms. predicted evidence: "it doesn't cause that but it can make it worse. so, let's change trulicity to 0.75 mg. it's going to be a dose change. so, use what you have and then we'll go ahead and lower the dosage to make sure that you're doing okay. ground truth evidence: "what you can do is, um, alternate the 1.5 with a 0.75 and you can see if you see a difference in how you feel. and i can give you some of the 0.75 and we'll switch you to the lower dosage because it is true that trulicity can give you more reflux and if you do have something in your stomach, the bowel issue, it will worsen. ## feedback from clinicians with the help of abridge we made our ee model available to two clinicians to aid them in finding evidence for verifying llm-generated clinical notes from transcripts. we asked them to randomly assign 50 percent of their notes for enabling the aid of our ee model and to carry out the remaining half as usual without this aid (refer to appendix c for more details on the exact instructions). we then collected feedback: feedback.1: ee dramatically reduces the amount of time required to verify the contents of the ai generated note. without it, i tend to skim the contents, do keyword searches, and struggle to identify the evidence; this process is frustrating and often negates the time that i saved by not drafting the note myself. i estimate that ee finds the appropriate @xmath2, and reduces the amount of time needed to review a note from 5 min to 1 min. moreover, i am more likely to do a comprehensive review of the note when using ee. feedback.2: the time saved by using ee was consistently 1-2m, almost half the time for a given length, and takes extra cognitive effort without it. having to scan the whole transcript vs just 3-7 lines of a transcript -huge efficiency booster. i estimate i used ee about a total of 55 times, with 2-3 that may have been close but not quite correct mapping, but minor and corrected when extending the query. in particular, ee makes it easier to check medical terms, reported symptoms, and doses. foot_1 ## conclusions in this paper, we described a setup that extracts evidence spans for language model outputs on grounded tasks. we presented three hierarchical architectures focused on speed and scalability to long documents, while looking to qa pretraining strategies for boosting performance. we showed that tapping into extractive qa datasets allows positive transfer even if the curated data is synthetic.