SlowGuess/NAACL_2021_OCR huggingface dataset:
We evaluate the quality of  $m$  by comparing it in terms of F1 to both (1)  $m_{\mathrm{edit}}$ , the non-stopwords removed or replaced in the true revision (i.e., edit prediction), and (2)  $m_{\mathrm{manual}}$ , a manually annotated "human" reference, (i.e., rationale prediction). We implement the following two baselines:
Unsupervised. As in Shah et al. (2020), we optimize a Lagrangian relaxation of Eq. 1, where
$$
\mathcal {L} _ {\mathrm {u s}} := - \log p (\operatorname {r e l} (c, s \odot m) = \mathrm {N E I}) + \frac {\lambda}{n} \sum_ {i = 1} ^ {n} m _ {i}.
$$
We keep the rel classifier (from §4.2) fixed, and train a separate ALBERT model to predict the mask  $m$  using a Gumbel softmax (Jang et al., 2017).
Distantly Supervised. By leveraging opposing claims present in VITAMINC, we are able to identify  $m_{\mathrm{edit}} = \text{diff}(s_{t-1}, s_t)$  —i.e., the non-stopwords that are deleted or replaced in  $s_{t-1}$  when compared to  $s_t$ . We then use  $m_{\mathrm{edit}}$  as distant supervision for  $m$ , where  $\mathcal{L}_{\mathrm{ds}} = -\frac{\gamma}{n} \sum_{i=1}^{n} \log p(m_i = m_{\mathrm{edit}_i})$ . We combine both the  $\mathcal{L}_{\mathrm{us}}$  and  $\mathcal{L}_{\mathrm{ds}}$  losses.
4.4 Factually Consistent Generation
As facts change, the sources reporting them must change as well to reflect the most recent information. In VITAMINC, this is reflected via the active revisions to Wikipedia. We simulate automating this process by considering two generation tasks:
Automatic Revisions. Given an outdated context  $s_{t-1}$  and an updated claim  $c$ , we learn
$f_{\mathrm{revise}}\colon \mathcal{S}\times \mathcal{C}\to \mathcal{S}$  to produce a new context  $s_t$  that minimally modifies  $s_{t - 1}$  to agree with  $c$ . For example, one can change  $s_{t - 1}$  in Figure 2 to state "before December" in order to agree with the claim.
Claim Extraction. Given a revision  $(s_{t - 1}, s_t)$ , we learn  $f_{\mathrm{extract}} \colon S \times S \to \mathcal{C}$  to produce a short claim  $c$  that expresses the factual change.
In both tasks, the output should satisfy  $\mathrm{rel}(c,s_t) = \mathrm{SUP}$ , while  $\mathrm{rel}(c,s_{t - 1}) = \mathrm{REF}$ . We use  $f_{\mathrm{verdict}}$  (§4.2) to evaluate this requirement. We experiment with both BART-base (Lewis et al., 2020a) and T5-base (Raffel et al., 2020) sequence-to-sequence transformer-based generators. For the revision task, we concatenate  $s_{t - 1}$  and  $c$  with a separator and train the model to predict  $s_t$ . For the claim extraction task, we combine the input pair  $(s_{t - 1},s_t)$  into a single sentence that visualizes the revision (e.g., "sales of  $\{4.7\to 5.4\}$  million").
We present and analyze results for the models described in Section 4. Our analysis attempts to evaluate several questions: (1) How well can the current state-of-the-art models perform on the VITAMINC tasks? (2) Does VITAMINC increases the robustness of models against adversarial examples? (3) Can VITAMINC improve interpretability by providing supervision for anchoring words?
5.1 Related Datasets
In addition to VITAMINC, we train and evaluate on several related datasets, which we briefly describe:
FEVER (Thorne et al., 2018): A popular fact verification dataset based on Wikipedia. We use the provided SUP and REF claim-evidence pairs. For NEI claims, we randomly sample neutral evidence from the article with the highest BM25 score (Fisch et al., 2021).
MNLI (Williams et al., 2018): A large and diverse dataset for natural language inference. The three-way sentence-pair entailment prediction is similar to fact verification. We use the hypothesis as the claim and the premise as the evidence and evaluate on the "mismatched" evaluation set.
Symmetric (Schuster et al., 2019): A set of challenging symmetric, synthetic extensions to FEVER's evaluation set that avoid claim-only bias.
Adversarial (Thorne et al., 2019c): Adversarial examples created by participants of the FEVER 2.0 shared task. Teams were asked to create claims that
Model	Train data	AUC	Prec.	Rec.	F1
Edit dist.	-	71.34	64.90	63.18	63.56
ALBERT	PAWS-full	72.20	65.27	60.61	60.48
BOW	VitC-diff	79.87	70.85	67.84	68.55
ALBERT	VitC-diff	89.87	80.69	82.06	81.18
ALBERT	VitC-full	91.97	82.63	84.49	83.18
Table 3: Factual revision flagging scores for models aware of the full sentence-pair (full) and aware only of the modified words (diff). We use ALBERT-base.
break FEVER-trained models. We take all SUP and REF claims and their gold evidence sentences.
Triggers (Atanasova et al., 2020): A set of 186 FEVER claims paraphrased adversarily to contain universal adversarial triggers (Wallace et al., 2019). Its small size leads to high variance results.
ANLI (Nie et al., 2020): An adversarial dataset for MNLI- and FEVER-based models. The creation was performed in three iterative rounds in which a model was trained, and then crowdworkers devised adversarial inputs, and the process repeated.
PAWS (Zhang et al., 2019): A dataset of altered Wikipedia sentences using word swapping and back-translation. Human annotators labeled whether the modified sentence is a paraphrase or not. We evaluate whether a PAWS-trained classifier can be used for our factual revision flagging task.
5.2 Factual Revision Flagging
Table 3 shows the results of our baseline models on the factual revision flagging task. First, we notice that a model trained on the PAWS dataset (reaching 93.42 F1 score on PAWS test) does not transfer well to the flagging task, and performs on par with a simple edit distance heuristic. We hypothesize that this is a result of the entity scrambling technique used to synthetically revise sentences in PAWS, which is different from the edits introduced by real, factual Wikipedia revisions in practice.
Second, we see that the performance of neural models trained on the VITAMINC flagging task increases with richer inputs and more advanced models—demonstrating the complexity of the task. The ALBERT (diff) model that uses only the modified word sequences from each sentence (i.e., contextual within a subspan) improves the AUC by 10 points over a BOW model that gets a similar input. The ALBERT (full) model that receives the full sentences as input (i.e., has access to even more context), further improves the AUC by 2 points. Nevertheless, the best model still only reaches 83
Figure 3: Test accuracy of models trained on a dataset of 100K combined SUP and REF examples from VITAMINC and FEVER. The higher the ratio of VITAMINC in the training data, the better the performance on adversarial evaluation sets (solid lines). The shaded areas represent standard error across three runs.
macro-F1, indicating the difficulty of this task.
5.3 Fact Verification
Table 4 summarizes the results for classifiers trained on fact verification and NLI datasets. Verifying claims against real revisions proves to be the hardest. The best model achieves  $89\%$  accuracy, lower than that on either VITAMINC's synthetic cases or the original FEVER examples. Including VITAMINC examples in the training data drastically increases models' sensitivity to contrastive examples (rightmost column)—while preserving the in-domain accuracy (only  $-0.42\%$  for FEVER and  $+0.12\%$  for MNLI with ALBERT-xlarge). Another evidence for the generalization properties conferred by VITAMINC is its zero-shot performance to both other datasets. An ALBERT-xlarge model trained only on VITAMINC reaches  $76\%$  and  $79\%$  accuracy on FEVER and MNLI, respectively. In contrast, the transfer accuracy for MNLI  $\rightarrow$  FEVER is  $70\%$  and for FEVER  $\rightarrow$  MNLI is only  $38\%$ .
Most importantly, models trained with VITAMINC perform better on challenging adversarial datasets. On the other hand, simply augmenting FEVER data with MNLI data has a limited effect on adversarial examples.[10] We conjecture that the contrastive nature of VITAMINC helps models better learn the relations between the claims and evidences—and to avoid relying on certain artifacts that do not generalize well.
To further probe the value of VITAMINC examples compared to FEVER ones (SUP and REF
10We've also tried augmenting FEVER with ANLI for an ALBERT-xlarge model and find it to achieve only  $73\%$ ,  $91\%$ , and  $34\%$  on Adver., Sym., and Triggers, respectively.
Model	Train dataset	VitC real	VitC syn	FEVER	MNLI	Adver.	Sym.	Triggers	ANLI	Contrast
ALBERT-base	FEVER	54.78	79.18	95.07	58.45	62.01	81.18	3.33	32.94	55.53
MNLI	44.93	69.67	65.70	85.22	49.61	72.89	68.82	30.63	56.67
FEVER + MNLI	55.93	82.26	95.62	85.58	63.97	85.67	38.71	30.59	61.90
VitC	86.16	89.78	74.56	69.18	71.41	90.17	86.67	37.25	84.11
VitC + MNLI	86.68	91.18	77.70	85.95	69.58	91.15	70.97	34.31	85.61
VitC + FEVER	86.26	91.05	94.24	68.90	68.80	91.57	72.04	38.50	85.89
ALBERT-xlarge	FEVER	58.56	84.22	96.47	38.33	72.58	87.08	32.80	36.03	64.59
MNLI	49.26	74.58	70.47	88.91	53.52	78.65	72.58	36.38	63.41
FEVER + MNLI	61.21	86.81	96.81	89.04	69.97	89.75	43.55	37.66	70.44
VitC	88.64	93.84	75.99	78.89	82.51	94.80	67.20	42.66	89.57
VitC + MNLI	88.69	93.92	76.80	89.03	80.81	94.80	67.20	40.09	89.80
VitC + FEVER	88.87	94.01	96.05	62.30	80.94	94.80	65.59	40.31	90.94
Table 4: Test accuracy of fact verification and NLI models. VITAMINC-trained models are more robust to adversarial examples and more sensitive to contrastive contexts. The rightmost column shows the percent of FEVER claims in which the prediction flipped when presented with contrastive contexts.
Token labels	Edit prediction	Word-level rationales
Prec.	Rec.	F1	Prec.	Rec.	F1
-	31.93	30.56	31.23	31.11	46.64	37.33
✓	55.93	62.68	59.11	36.92	66.03	47.36
Table 5: The distant token-level supervision of VITAM-INC improves the edit prediction, and as result identifies the anchoring words (rationales) more accurately.
only), we compose training sets of 100K examples using different ratios of the two datasets. As shown in Figure 3, including more VITAMINC pairs continuously improves the performance on the challenging adversarial and symmetric evaluation sets.
As an additional qualitative experiment, given the recent successes of huge language models such as GPT-3 (Brown et al., 2020), we explore whether such models develop sufficient context sensitivity on their own. Appendix C shows the results of classifying several claims using a few-shot GPT-3 model. We find that GPT-3 still largely underperforms our VITAMINC-trained models in terms of sensitivity—demonstrating the importance of using VITAMINC's unique structure during training.
5.4 Word-level Rationales
Table 5 shows the results of our baseline models for identifying word-level rationales (i.e., anchoring words in the evidence). While our unsupervised model is able to uncover some patterns, directly leveraging the structure of VITAMINC to obtain distant supervision for likely anchoring words (i.e., token labels) improves both the edit prediction and the word-level rationale prediction performance.11 Example predictions are provided in Appendix E.
5.5 Factually Consistent Generation
Table 6 presents the results on factually consistent generation. We find BART to perform better in both of our generation tasks (though we only tried the default setting). The BLEU score (Papineni et al., 2002) is lower in the claim extraction task since there is freedom in how to phrase the claims, which can result in greater differences between the outputs and the references. The BERT-based BLEURT score (Sellam et al., 2020) shows a similar trend. Still, the claim extraction model succeeds in updating the facts that reflect the true revision  $86\%$  of the time, as measured by the fact verification model's verdict  $(f_{\text{verdict}})$ .
The revision generator aims to modify sentences so that they agree with a given claim. According to our fact verification model's verdict, it succeeds in doing so  $76\%$  of the time. Furthermore, revisions should resemble real ones, and preserve the remaining content that is unrelated to the claim. The SARI KEEP F1 (Xu et al., 2016) of 75 shows that the model and the reference mostly agree on parts of the sentence that should be kept unchanged.
We find that the token-based measures and our  $f_{\text{verdict}}$  metric agree well with human (manual) evaluation scores. We randomly sampled 100 generated and human-written sentences per task, and asked workers on Amazon MTurk to rate their grammaticality and whether the evidence  $s_t$  supports the claim. The scores of the generated sentences were on par with the human-written ones, indicating the high-quality of our outputs.
Table 7 presents two example generations for the claim extraction task (we provide additional qualitative examples in Appendix E). Our model is able to efficiently extract a self-contained claim that expresses the correct fact after the edit. As in §5.3,
<sup>11</sup>We evaluate rationales using a manually annotated test set of 300 examples (150 each from VitC real and VitC synthetic).
Target	Model	ROUGE2	BLEU	KEEP	SARI scores	AVG	BLEURT	fverdict	Manual evaluation
ADD	DEL	Grammar	SUP
Revision	T5	77.63	47.46	72.61	13.32	43.04	42.99	0.38	64.52	81.00	71.80
BART	85.23	54.86	75.36	18.31	47.95	47.21	0.67	76.26	84.80	83.20
Claim	T5	35.19	13.95	44.36	20.59	87.54	50.83	-0.12	75.39	71.33	72.22
BART	40.38	16.14	52.91	23.62	91.37	55.97	0.16	85.83	75.78	74.22
Table 6: Factually consistent generation results. Higher is better for all scores and the max value is 100 (except for BLEURT).  $f_{\text{verdict}}$  is the score of our VITAMINC-trained ALBERT-base model on the outputs. For manual evaluation, outputs were rated by their grammaticality and by how much the evidence supports the claim (SUP). For reference, human-written pairs received 75.75 and 76.0 average scores for Grammar and SUP, respectively.
(st-1,st)	2020 coronavirus pandemic in Germany: there have been{*|349->444}confirmed cases and 16 recoveries.
BART (VitC)	More than 400 people have tested positive for COVID-19 in Germany. | fverdict(c,st) =	SUP
GPT-3 T=0	As of 14 March, there have been more than 350 confirmed cases of the virus in Germany	SUP
GPT-3 T=0.7	As of March 12, there have been more than 400 confirmed cases and 20 reported deaths	NEI
Reference	There have been more than 400 confirmed coronavirus cases in Germany.	SUP
(st-1,st)	Diego Corrales: Corrales was born to a{*| Puerto Rican -> African American}father and a {*| Dominican -> Mexican}mother.
BART (VitC)	Diego Corrales'father is African American and his mother is Mexican.	SUP
GPT-3 T=0	Corrales was born to a Puerto Rican father and a Mexican mother	REF
GPT-3 T=0.7	Corrales was born to a father from Puerto Rico and a mother from the Dominican Republic	REF
Reference	Diego Corrales' father was African American and his mother Mexican.	SUP
Table 7: Example outputs for expressing claims that reflect the factual changes in a single Wikipedia revision. The BART-base model is trained on VITAMINC data, while GPT-3 is applied in a 2-shot setting with a temperature of 0 or 0.7 (see Appendix C). The revision  $(s_{t - 1}, s_t)$  is given to the model as a single sentence, where the edits are between curly brackets. The human-written claim is provided for reference. The rightmost column contains the prediction of our ALBERT-xlarge  $f_{\text{verdict}}(c, s_t)$  model (trained on VITAMINC) when using the generated claim.
we also explore how GPT-3 handles this task (we provide two demonstrations in the prompt). Compared to the BART model trained on VITAMINC, GPT-3 appears to make more factually inconsistent or unsupported generations (see Appendix C for more details). Encouragingly, our  $f_{\text{verdict}}$  classifier is still able to pick up on this—as demonstrated by the predictions in the rightmost column of Table 7. For example, classifying the report about 20 deaths as NEI since it is not part of the source. Once again, this serves to qualitatively demonstrate the effectiveness of leveraging VITAMINC.
We presented VITAMINC, a large-scale dataset for training and evaluating fact verification models using contrastive contexts. Our novel method of leveraging factual revisions to Wikipedia enabled us to create challenging examples in which a claim is paired with contexts that are lexically similar, yet factually opposing. Our results illustrated that training on VITAMINC improves classifier sensitivity to subtle changes in evidence, and increases their robustness to adversarial examples.
Furthermore, we formulated several new, important tasks for fact verification that VITAMINC allows us to test. We showed how the dataset's unique "before and after" structure lends itself to training classifiers to flag factual revisions. In addition, for factual revisions, the edits reveal which words in the evidence are the most critical—which helps supervise word-level rationale models for better interpretability. Finally, we demonstrated that VITAMINC can help with factually consistent text generation. We hope that this work and the range of tasks it presents will motivate and support the fact verification field in developing reliable models that can adapt to dynamically changing evidence.
We thank the TransPerfect team, Darsh J Shah and Enrico Santus for helpful discussions, as well as the members of the MIT NLP group and Andreas Vlachos for valuable feedback. This work is supported in part by the Facebook Online Safety Benchmark Award. TS is supported in part by DSO grant DSOCL18002. AF is supported in part by a NSF Graduate Research Fellowship.
Pepa Atanasova, Jakob Grue Simonsen, Christina Lioma, and Isabelle Augenstein. 2020. Generating fact checking explanations. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 7352-7364, Online. Association for Computational Linguistics.
Isabelle Augenstein, Christina Lioma, Dongsheng Wang, Lucas Chaves Lima, Casper Hansen, Christian Hansen, and Jakob Grue Simonsen. 2019. MultiFC: A real-world multi-domain dataset for evidence-based fact checking of claims. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 4685-4697, Hong Kong, China. Association for Computational Linguistics.
Omer Benjakob and Stephen Harrison. 2019. From anarchy to wikiality, glaring bias to good cop: Press coverage of wikipedia's first two decades. Wikipedia @ 20. https://wikipedia20.pubpub.org/pub/tw83imoy.
Piotr Bojanowski, Edouard Grave, Armand Joulin, and Tomas Mikolov. 2017. Enriching word vectors with subword information. Transactions of the Association for Computational Linguistics, 5:135-146.
Samuel R. Bowman, Gabor Angeli, Christopher Potts, and Christopher D. Manning. 2015. A large annotated corpus for learning natural language inference. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 632-642, Lisbon, Portugal. Association for Computational Linguistics.
Ronan Le Bras, Swabha Swayamdipta, Chandra Bhagavatula, Rowan Zellers, Matthew Peters, Ashish Sabharwal, and Yejin Choi. 2020. Adversarial filters of dataset biases.
Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. 2020. Language models are few-shot learners.
Oana-Maria Camburu, Tim Rocktäschel, Thomas Lukasiewicz, and Phil Blunsom. 2018. e-snli: Natural language inference with natural language explanations. In Advances in Neural Information Processing Systems, volume 31. Curran Associates, Inc.
Wenhu Chen, Hongmin Wang, Jianshu Chen, Yunkai Zhang, Hong Wang, Shiyang Li, Xiyou Zhou, and
William Yang Wang. 2020. Tabfact: A large-scale dataset for table-based fact verification. In International Conference on Learning Representations.
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 4171-4186, Minneapolis, Minnesota. Association for Computational Linguistics.
Jay DeYoung, Sarthak Jain, Nazneen Fatema Rajani, Eric Lehman, Caiming Xiong, Richard Socher, and Byron C. Wallace. 2020. ERASER: A benchmark to evaluate rationalized NLP models. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 4443-4458, Online. Association for Computational Linguistics.
Angela Fan, Aleksandra Piktus, Fabio Petroni, Guillaume Wenzek, Marzieh Saeidi, Andreas Vlachos, Antoine Bordes, and Sebastian Riedel. 2020. Generating fact checking briefs. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 7147-7161, Online. Association for Computational Linguistics.
Manaal Faruqui, Ellie Pavlick, Ian Tenney, and Dipanjan Das. 2018. WikiAtomicEdits: A multilingual corpus of Wikipedia edits for modeling language and discourse. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 305-315, Brussels, Belgium. Association for Computational Linguistics.
Adam Fisch, Tal Schuster, Tommi S. Jaakkola, and Regina Barzilay. 2021. Efficient conformal prediction via cascaded inference with expanded admission. In International Conference on Learning Representations.
JL Fleiss. 1971. Measuring nominal scale agreement among many raters. Psychological bulletin, 76(5):378-382.
Matt Gardner, Yoav Artzi, Victoria Basmov, Jonathan Berant, Ben Bogin, Sihao Chen, Pradeep Dasigi, Dheeru Dua, Yanai Elazar, Ananth Gottumukkala, Nitish Gupta, Hannaneh Hajishirzi, Gabriel Ilharco, Daniel Khashabi, Kevin Lin, Jiangming Liu, Nelson F. Liu, Phoebe Mulcaire, Qiang Ning, Sameer Singh, Noah A. Smith, Sanjay Subramanian, Reut Tsarfaty, Eric Wallace, Ally Zhang, and Ben Zhou. 2020. Evaluating models' local decision boundaries via contrast sets. In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 1307-1323, Online. Association for Computational Linguistics.
Suchin Gururangan, Swabha Swayamdipta, Omer Levy, Roy Schwartz, Samuel Bowman, and Noah A. Smith. 2018. Annotation artifacts in natural language inference data. In Proceedings of the 2018