Probing Factually Grounded Content Transfer with Factual Ablation (2022)

Abstract: Despite recent success, large neural models often generate factually incorrect text. Compounding this is the lack of a standard automatic evaluation for factuality–it cannot be meaningfully improved if it cannot be measured. Grounded generation promises a path to solving both of these problems: models draw on a reliable external document (grounding) for factual information, simplifying the challenge of factuality. Measuring factuality is also simplified–to factual consistency, testing whether the generation agrees with the grounding, rather than all facts. Yet, without a standard automatic metric for factual consistency, factually grounded generation remains an open problem. We study this problem for content transfer, in which generations extend a prompt, using information from factual grounding. Particularly, this domain allows us to introduce the notion of factual ablation for automatically measuring factual consistency: this captures the intuition that the model should be less likely to produce an output given a less relevant grounding document. In practice, we measure this by presenting a model with two grounding documents, and the model should prefer to use the more factually relevant one. We contribute two evaluation sets to measure this. Applying our new evaluation, we propose multiple novel methods improving over strong baselines.

## introduction large pretrained models have shown impressive effectiveness at longstanding tasks and benchmarks. one exciting example is gpt-3 @xcite , which completes tasks with remarkable clarity and knowledge-without supervision-simply by writing what might come next. yet significant challenges prevent these models from helping humans write real documents. for example, in figure 1 gpt-3 attempts to auto-complete the next sentence of a prompt regarding auto racer ralph context speedway management would maintain their pol-icy to not race on sundays until 1974 indianapolis 500-1974. after a heartbreaking loss in 1912 indianapolis 500-1912, ralph depalma succeeds in victory for 1915. depalma was accompanied by riding mechanic louis fontaine. gpt-3 → the 1915 race was the first to have a post-race distance of more than 500 miles gpt-2 @xmath0, 1915, at his home in south pasadena, california, of cancer. grounding south pasadena, calif., march 31 (ap)-ralph depalma, pioneer auto racer who probably won more races than any other driver in history, died today of cancer. his age was 72. gpt-2 @xmath1. gpt-2 pmi-@xmath2, california, at the age of 72. figure 1 : generation with different models continuing a wikipedia article. gpt-3 has no grounding, while the other 3 models use one document as grounding. the table highlights factual and false information. de palma; gpt-3 suggests the 500-mile indy-500 race had an impressive-yet impossible-finishing distance of "more than 500 miles." such factual hallucinations limit the usability of existing models @xcite . issues are exacerbated by the black-box nature of memorized knowledge that these models draw from, which may have factual gaps or be out-of-date. this motivates explicitly controlling the information models generate with, by textual grounding. summarization is a good example of this: all information needed for the summary comes from the source document (grounding). besides assuring models draw on factual knowledge, introducing grounding simplifies the challenge of evaluating factuality. rather than verifying generations against all facts, the problem is reduced to testing factual consis-tency with information in the grounding. however, measuring this automatically is an open problem. in this work, we study factual consistency in the setting of figure 1 : generating the next sentence with grounded information. we refer to this as content transfer @xcite -transferring knowledge from a source document to continue a target document. factual consistency has largely been studied in summarization, but content transfer introduces an exciting notion of control (the document being extended) which affects style, content, and factual selection. central to any study of factual consistency is defining a way to measure it. in this work, we introduce factual ablation, which asserts that an output y should be more likely when grounding g is more relevant. in particular, if grounding g entails y but g ′ does not, p(y|g) should be greater than p(y|g ′ ); the closer g and g ′ , the more challenging the example. an evaluation set for factual ablation is constructed by collecting such grounding pairs to test models with. content transfer is particularly suited for this: due to continuous edits in the underlying wikipedia data, there are many instances of document pairs g, g ′ which are relevant to the same target document, but result in different continuations. following a similar intuition to factual ablation, we propose both training-time and inference-time approaches that measure the effect grounding has on generation, to keep models on-topic and factually consistent with grounding. overall, our contributions bring the study of factual consistency to a new domain: content transfer. we propose factual ablation, then use this to generate evaluation data (both synthetic and natural). we propose multiple methods to improve factual consistency, carrying out a wide evaluation of models using lexical metrics, factual ablation, and human annotation, finding the superior model by factual ablation also achieves the best human-measured factual consistency. as natural generation models see increasing deployment, it is more important than ever to make sure they are factual and well controlled ( §7). studying this in highly applicable domains, like content transfer, is an important step in keeping models accountable. ## methodology here, we bring factual consistency to a new domain, content transfer, which is the task of extending context c with content from a grounding document g. we discuss the task ( §3.1), and our major contributions: novel methods for judging ( §3.2) and improving ( §3.3) factual consistency in this setting. ## experimental setup we probe factual consistency for an array of models tuned on the training set for content transfer from @xcite ( §3.1). we generate on the validation set, assessing the generations of each model with lexical and human metrics; then, we compare generative properties to the factual ablation of each model, measured on our synthetic ( §3.2.1) and natural ( §3.2.2) evaluation sets. ## results and analysis lexical overlap lexical overlap metrics for model generations are reported in ## conclusions in this work, we introduce the study of factual consistency to the content transfer domain by proposing factual ablation, a measure of factual consistency that uniquely fits this setup. we test multiple training-time and inference-time methods for improving factual consistency in this domain, carrying out a wide study of lexical metrics, factual ablation, and pairwise human comparison. we find the same model is superior at both factual ablation and human-judged factual consistency; this supports factual ablation as a useful measure in developing more consistent models, extending the already rich and promising vein of methods studied here.