The student becomes the master: Outperforming GPT 3 on Scientific Factual Error Correction (2023)

Abstract: Due to the prohibitively high cost of creating error correction datasets, most Factual Claim Correction methods rely on a powerful verification model to guide the correction process. This leads to a significant drop in performance in domains like Scientific Claim Correction, where good verification models do not always exist. In this work we introduce SciFix, a claim correction system that does not require a verifier but is able to outperform existing methods by a considerable margin — achieving correction accuracy of 84% on the SciFact dataset, 77% on SciFact-Open and 72.75% on the CovidFact dataset, compared to next best accuracies of 7.6%, 5% and 15% on the same datasets respectively. Our method leverages the power of prompting with LLMs during training to create a richly annotated dataset that can be used for fully supervised training and regularization. We additionally use a claim-aware decoding procedure to improve the quality of corrected claims. Our method outperforms the very LLM that was used to generate the annotated dataset — with FewShot Prompting on GPT3.5 achieving 58%, 61% and 64% on the respective datasets, a consistently lower correction accuracy, despite using nearly 800 times as many parameters as our model.

## introduction the widespread adoption of the internet has led to the distribution of more written content than ever before in human history, and recent advances in generative ai models are expected to push this trend even further @xcite @xcite . as this decade has seen, this revolution comes with its demerits because with more content comes more inaccurate or false content @xcite @xcite . a reliable way to automatically flag these incorrect claims or, more ambitiously, to automatically correct incorrect claims would do wonders for our ability to manage this risk. researchers have identified and been working on factual claim verification with some success. this is not the case, however, for factual error correction, where the prohibitively high cost of manually annotating corrections of incorrect claims means there is currently no available dataset for this task @xcite . the few methods that tackle this problem use claim verification datasets for distant supervision and try to use claim verification models to provide signals that can guide the correction process. this exposes the correction methods to flaws in the verification models -one of which is that current verification methods often make either an implicit or explicit domain assumption -focusing on news and political domains @xcite . due to this, today's most potent claim verification methods fail to transfer well to scientific claims, which is especially worrying when we consider that scientific reports and claims are challenging for people without domain expertise to verify or correct. this also adversely impacts the best claim correction methods, with these methods struggling to perform satisfactorily on claim correction tasks from the scientific domain. in this paper, we introduce scifix, a factual claim correction system that makes no domain assumptions, does not require a claim verification model, and is shown to work well on scientific claims. our method leverages the power of large language models (llms) like gpt @xcite to generate a claim correction dataset from a claim verification dataset by corrupting 'correct' claims into 'incorrect' ones. this dataset not only allows us to learn a mapping between incorrect claims and their associated correction but also allows us to generate rich annotations via explanations for why this mapping is correct. we use this figure 1 : full description of the scifix system. during training a fact verification database is converted to a well annotated error correction database using prompting with llms, this database is used to train a seq2seq correction lm and a seq2label semantic difference model. during prediction the semantic difference model helps guide the generative model using claim aware decoding to generate the corrected claim dataset to train a conditional generation language model to map evidence and incorrect claims to the appropriate correction, and the explanations serve as a useful guide during the learning process. finally, we introduce a claim-aware decoding procedure to guide the generation process. while this component does not require a verifier, any verifier can be easily integrated into our procedure if available for the domain at hand. scifix can achieve 84.73% correction accuracy on the scifact dataset @xcite , 77.77% @xcite on the scifact-open dataset, and 72.75% on the covidfact dataset, which is an order of magnitude better than competing methods, with the best contemporary method achieving 7.6%, 5% and 15.45% on the datasets, respectively. more impressively, our method outperforms the pretrained llms that generated the dataset. despite using around 800 times as many parameters as our model, few-shot prompting on gpt3.5 achieves 58.74%, 61.11%, and 64.58% on the datasets, respectively, which is consistently outperformed by our method. our work presents an alternative route forward for claim correction efforts that do not rely on having access to a powerful verification model and, more generally, shows that general llms can be used effectively as part of the model training pipeline to create a more compact yet more powerful model. in summary, our contributions include: factual error correction: one of the first approaches for this problem was developed by @xcite . their strategy was to first use a pretrained fact verification model to identify and mask out the components of a sentence that cause the given claim to be incorrect and then use a separate model to fill in the masked sentence while remaining consistent with the provided evidence. building on these ideas, @xcite adopts the same masking and infilling paradigm but advances the choice of how to use the fact verification model in the masker and the infilling model. most recently, @xcite considered fact correction as an iterative sampling problem and sampling editing actions with respect to a target density function. they also used a pre-trained verification model to guide the sampling process. these methods have all made significant advances in factual error correction. however, none are expected to perform reasonably well if their verification model is poor for the task at hand. these methods work around the fever dataset, a well-studied fact verification dataset with a suite of powerful verification models that can be plugged into the method. this is not the case in the less studied sub-field of scientific claim verification domains @xcite , where datasets like scifact-open prove to be harder to tackle from a verification perspective @xcite . zerofec @xcite explores a verifier-free, zero-shot approach to error correction. this method formulates questions about input claims, looks for correct answers in the given evidence, and assesses the faithfulness of each correction based on its consistency evidence: ... incorrect claim: a breast cancer patient's capacity to metabolize tamoxifen does not influence treatment outcome. correct claim: a breast cancer patient's capacity to metabolize tamoxifen influences treatment outcome. explanation: the evidence mentions that compared with the extensive metabolizers, those with decreased cyp2d6 activity (hetorozygous extensive/intermediate and poor metabolism) had worse event-free survival and desease-free survival, which suggests that capacity to metabolize tamoxifen influences treatment outcome and the claim is true. with the evidence. factual consistency of summarizations: similar problems have arisen in making the summary of a paragraph consistent with the facts of the paragraph. the relevant approaches here are the post-editing processes, which can detect and correct factual errors in the summary. some methods in this domain are @xcite , @xcite and @xcite , which try to manually introduce corruptions into correct claims and reconstruct the correct claim using a seq2seq model @xcite . our method extends this approach by discarding the need for manually defined entity swapping, back translation, or other labor-intensive methods of introducing corruptions by using an llm to provide a diverse set of corrupted claims. we also provide a way to generate a set of rich annotations that is fundamentally impossible via the corruption approaches @xcite @xcite . prompting with llms: it has been shown recently that an llm can achieve high performance on specific few-shot tasks with a few examples in the context window of the input @xcite . chain-of-thought prompting improves on simple prompting approaches by providing examples in the prompt, which contain question-answer pairs and some stated reasoning for the provided answer. we use these powerful methods with general llms to generate data for our more compact and task-dependent model. ## method first, we specify how we interpret the claim verification and error correction tasks. below, we use notation similar to @xcite . given a claim c and evidence e(c) such that the evidence contradicts c, that is, e(c) ̸ |= c, our task is to generate a corrected claim c ′ , such that c ′ is supported by the claim: e(c ′ ) = e(c) |= c ′ , and c ′ makes claims about similar ideas, concepts, objects, etc. as c. in our work, we assume that we have access to a domain-specific claim verification dataset instead of a correction dataset itself. our system has three key components: i) llm driven data generation, ii) domain adaptive language modeling, and iii) a claim aware decoding procedure. llm driven data generation: first, we identify something fundamental about the nature of the claim correction problem. one direction, mapping incorrect claims to correct claims, is challenging since it requires a deep understanding of the semantics of both the evidence and claim to perform. however, the reverse direction, mapping correct claims to incorrect claims, is much easier and requires only a partial understanding of the concepts and words in the correct claim, often not requiring any evidence at all. a concrete example of this is shown in figure 2 , where it is possible to generate an incorrect claim from a correct claim without seeing the evidence or understanding any of the medical concepts in the sentence. however, to recover the correct claim, one must comprehend the evidence, identify the error, and correct it. there is a similar pattern in generating explanations for why a claim is valid or rewording a correct claim to maintain the same meaning overall. we exploit this property using a pretrained llm (gpt3.5): first, we take the evidence and supported claims from the verification datasets and then produce a correction dataset with annotations and explanations for why the correct claim is true. after these steps, we create an 'augmented correct claim' (an alternate correct claim with the same meaning) for each example. a sample from this dataset is shown in figure 2 , and more are provided in the appendix figure 7 . domain adaptive language modeling: we use domain adaptive pretraining @xcite 2020) on the evidence passages to give the llm a better understanding of the kinds of words and concepts that are unique to the specific domain we are interested in. using this adapted lm, we learn a 'claim correction model'. the claim correction model is trained to map the evidence and incorrect claim to a correct claim and explain why that is a correct claim. unlike previous methods that interpret error correction as a masking and infilling problem, we interpret it as a conditional generation task. this allows us to tackle a significantly more diverse set of incorrect claims because many incorrect claims are harder to correct by swapping tokens, as they may require the sentence structure to change considerably. claim aware decoding procedure: @xcite introduced an effective way to perform constrained decoding for generative llms. during beam search decoding, instead of scoring a partial sequence as just the probability of generating that sequence of tokens, the constrained decoding method performs a look-ahead search to make a greedy estimate of what the complete sequence is likely to be. then it incorporates the goodness of the entire sequence into the score of the partial sequence. we utilize the fact that the corrected claim should not have the same meaning as the incorrect claim to guide the decoding process. specifically, we use the same adapted lm from the previous component to learn a 'semantic difference model'. the semantic difference model is a classifier trained to identify when two claims are semantically similar, giving a score of 1 when they have a different meaning and 0 when they are identical. we then use this as the scoring function for the look-ahead estimate, incentivizing the decoding process to avoid solutions with the same meaning as the incorrect claim. in the following sections, we showcase the power of this method on scientific claim verification datasets. components: for the llm driven data generation step, we used few-shot prompting on gpt3.5. for the domain adaptive language modeling, we trained a t5-base model @xcite ) (220m params) on the abstracts of scifact and scifact-open. after these steps, we fine-tuned this model for the claim correction model and semantic difference model. variation: we use the components mentioned above to train two models: the first of which is trained on all available datasets (scifix all). for the second model (scifix bio), we separate our datasets into two distinct 'domains'. scifact and scifact-open are biomedical datasets that use abstracts as their evidence passages. covidfact is a dataset from news reporting on covid-19, with selected sentences as evidence passages. we split these into the 'biomed' and 'covid news' domains, respectively, and observe the performance when the training data is taken from the biomed domain, and the prediction task is on the covid news domains. this tests the ability of the methods to generalize to shifting domains. all software implementations are publically available at: https: //github.com/dhananjayashok/scifix explanation | incorrect prediction: the evidence mentions that self-harm rates were more than ten times higher in female prisoners than in male inmates claim: the risk of male prisoners harming themselves is 10 times that of female prisoners. metric zerofec vence gpt zs gpt fs scifix bio scifix all diffmodel 39.78 28.33 50.66 49.67 74.69 18.72 sari -9.49 10.66 46.19 57.44 48.58 62.66 gptscore 73.68 68.14 62.21 43.83 92.45 81.12 annotator 1 86.13 87.62 71.31 87.09 95.42 86.29 annotator 2 85.3 90.23 87 87.92 88.95 83.75 annotator 3 87.22 84.44 87.76 84.12 95.05 87.34 5 experiment 1: correction dataset in the first experiment, we use human annotation to measure the accuracy of the claim correction datasets generated from the claim verification datasets. we conduct a survey of using 200 examples from each model, with each example being seen by three distinct annotators. annotators are asked to select whether the predicted claim is both 1) true given the evidence provided and 2) makes a claim related to the original, incorrect claim. we compute the correction accuracy based on these responses. the annotators were recruited through the personal networks of the authors and were not provided any compensation for the task. to mitigate potential bias in annotation, the annotators were given no information regarding how the predicted claims were generated. annotators were provided random samples of output from all the models without any indication of which model produced which output, additionally the annotators were not even aware that there could be multiple models responsible for different outputs that they observed. finally, the annotators were given no details about any of the methods they evaluated, only a description of the claim correction task. the annotation task itself does not involve any graphic descriptions, disturbing language pertaining to sensitive topics (including but not limited to hate speech, violence, sexual content etc.) or personal information. additionally, all the annotators were shown examples of the task text beforehand and were informed that they were free to decline the task at any time, even after starting. more details available in the appendix (figure 6 ). we use these scores to measure the correlation between human evaluation and several automatic metrics. the metrics used are (i) semantic difference model: the scores of the semantic difference model we trained to use for claim aware decoding, (ii) few shot prompting on gpt3.5: a binary indicator of whether gpt3.5 classifies the prediction as correct or not, see the appendix for the exact prompt used (figure 9 ) and (iii) sari @xcite , the most commonly used metric in the claim correction literature, which has its origins in summarization. baselines: we compare our method against the following approaches -zerofec @xcite , vence @xcite , zero-shot prompting on gpt3.5, and few-shot prompting on gpt3.5 (2 examples) @xcite . results: surprisingly, we find that despite never being explicitly guided to not alter an input claim if it is already true, we can observe this behavior in the model. when the input to our model is the set of all the claims labeled 'supported' in the verification datasets (i.e., already true claims), 65.82% of the time the model does not alter the input claim (i.e., string equality with the input claim which was correct), this shows our model is not simply negating or flipping the meaning of input sentences but making more guided corrections to align with the evidence provided. a qualitative inspection of our model's performance shows that it can make a wide variety of edits and corrections, including swapping incorrect entities, correcting erroneous numerical figures, flipping incorrect adjectives or quantifiers, as well as swapping incorrectly placed concepts(figures 5 , 12 ). however, the system's mistakes are equally diverse, with no clearly discernable trend in which types of examples are more complex for the model to correct appropriately (see the appendix for examples figure 11 ). ## experiment 2: verification dataset in the second experiment, we measure the ability of the methods to successfully correct the incorrect claims present in the claim verification dataset itself, and this shows that our results are not only applicable to the specific correction dataset we generated but are more generally applicable to incorrect scientific claims. we use as benchmarks only the best methods from the previous experiments. results: the results are shown in ## ablation and analysis we next vary components of the pipeline to investigate their individual contribution to the system's success. specifically, we remove the explanations and the claim-aware decoding procedure from the pipeline. the explanations show an interesting trend, with the explanation's coherence and faithfulness being empirically correlated with the correctness of the predicted claim. this can be seen with an example in figure 3 , where an explanation is incoherent, preempting an incorrect prediction. more examples of explanations having this effect are provided in the appendix figure 10 scifact scifact our method is verifier free. however, there is considerable scope for improvement if we can devise a method to integrate a verifier into the method in a 'soft' way, perhaps with a parameter to balance the tradeoff between how much the method will trust the predictions of the verifier. additionally, our method does not use the supported claims from the claim verification datasets during training. future work may study how to use this data split to improve our method efficiently. this could make the model robust to incorrect and correct claims inputs. more immediately, our method could be improved by employing prompt engineering to optimize the diversity and quality of the datasets generated and using more powerful semantic difference models or score signals during the claim-aware decoding procedure. there are several interesting directions for future work using this method of leveraging the power of prompting to generate synthetic datasets. there are potentially several problems where the key bottleneck has been the prohibitive cost of human annotation and data creation -the success of this method on factual error correction suggests there may be other subtasks where this method would perform well. ## conclusion in this work, we put forward scifix, a new way to utilize existing claim verification datasets to perform claim correction effectively. we show that our method is not only more powerful than existing methods on scientific claim correction but also outperforms prompting on gpt3 -one of the largest language models accessible today. factual claim correction and scientific claim correction, in specific, are vital yet nascent subfields of nlp. we hope this work expands the range of possible solutions by offering an effective way to perform correction without access to a powerful verification model.