SlowGuess/NAACL_2021_OCR huggingface dataset:
We expect GPT-3 to improve with longer prompts or fine-tuning and leave this to future research due to our limited quota.
GPT-3 for Fact Verification. We also experiment with using GPT-3 few-shot classification capabilities for the fact verification task. We follow the ANLI few-shot format of Brown et al. (2020) and compose prompts with 6 examples (2 from each class) with random examples from VITAMINC
training set. We use only numerical examples to evaluate numerical claims (Figure C.3), and mixed examples for other claims (Figure C.2). We set  $\mathcal{T} = 0$  as recommended for classification.
Table C.1 summarizes the results. Even with only six examples, GPT-3 seems to perform significantly better than random. Yet, its verdict is wrong in several cases that can be easily classified by humans. For example, we find it to refrain from predicting a True/False verdict even when the evidence is clear. We observe this both for a date-based (line 3.2 in Table C.1), numerical (lines 4.1-4.2), and entity-focused claims (line 5.2).
To experiment with the sensitivity of the model to the provided context, we manually modified some of the examples to provide even stronger evidence. For example, while GPT-3's prediction for line 5.2 is acceptable as actually, Turner Broadcasting System merged with WarnerMedia in 1996, changing the evidence to another disconnected entity (The Walt Disney Company) did not change the prediction (line 5.3) as expected. Even when explicitly stating that there is no other owner GPT-3 didn't modify its verdict (line 5.4). Similarly, when evaluating the claim about the population of Beaverton being less than  $90\mathrm{K}$ , GPT-3 ignores the supporting evidence and outputs a false verdict (lines 1.4-1.5). Changing the claim to state "approximately  $86\mathrm{K}$ " instead of "less than 90,000" modified the prediction to "Neither" (line 1.6). Only repeating the exact same number as the evidence led to a true verdict (line 1.7).
D Complementary Experiments
We report fact verification results with a fine-tuned BERT-base (Devlin et al., 2019) model in Table D.1. We find ALBERT-base to outperform BERT-base on most of the evaluated datasets. ALBERT-xlarge performed better than the two base models in all datasets except for Triggers. The Triggers dataset is very small (186 examples) and contains some unnaturally looking claims, which could explain the high variance across models.
E Example Outputs
We provide examples of predicted word-level rationales in Table E.1 and of outputs for the two generation tasks in Tables E.2 and E.3.
Figure C.1: The prompt used for GPT-3 few-shot claim extraction.
Figure C.2: The prompt used for GPT-3 few-shot fact verification predictions on non-numerical claims (examples 2 and 4 in Table C.1. We follow the few-shot setting of Brown et al. (2020) for ANLI.
Figure C.3: The prompt used for GPT-3 few-shot fact verification predictions on numerical claims (examples 1 and 3 in Table C.1.
Model	Train dataset	VitC real	VitC syn	FEVER	MNLI	Adversarial	Symmetric	Triggers	ANLI	Contrast
BERT-base	FEVER	60.55	71.35	87.16	61.90	52.09	73.60	69.89	34.53	54.05
MNLI	46.31	69.01	70.06	83.80	50.13	73.88	65.05	26.88	51.92
FEVER + MNLI	56.24	81.80	95.59	85.06	63.05	85.11	37.63	29.63	60.63
VitC	85.80	90.63	74.21	66.66	76.24	90.17	63.98	33.19	72.49
VitC + MNLI	84.47	91.00	74.88	83.70	63.05	84.55	66.13	31.00	84.88
VitC + FEVER	84.72	89.16	87.55	69.28	64.75	90.73	72.58	34.06	84.01
Table D.1: Fact verification Complementary results for Table 4 with a BERT-base model.
Claim
Evidence	the youtube channel chuchu tv is placed 42nd and has more than 25 million subscribers .
chuchu tv is the 43rd most subscribed youtube channel in the world , with over 20 million subscribers .
Claim
Evidence	the ramus has sold less than 4.5 million albums worldwide .
the rasmus has sold 5 million albums worldwide , 310,000 copies in their native finland alone .
Claim
Evidence	darren randolph is spanish .
humes dated irish footballer darren randolph in 2005 .
Claim
Evidence	astravyets is near vilnius .
his father may have migrated to the us in the 1860s from astravyets near vilnius .
Claim
Evidence	the pace controlling stamina meter is a new feature in the game series .
new to the series is a three-tier stamina meter which controls the pace of a match .
Claim
Evidence	the movie will be released on 25 november 2015 .
[...] are producing the film which columbia pictures will release on november 25 , 2015 .
Table E.1: Example masks produced by the word-level rationale model for identifying anchoring words in the evidence that are responsible for the classifiers verdict regarding the claim. Masking these words leads the classifier to predict NEI instead of what would have been SUP or REF.
Claim
st-1	Stephen Bruner was born on October 19, 1984.
Stephen Bruner ( born October 18, 1984 ) , better known by his stage name Thundercat , is an American multi-genre bass player , producer and singer fr om Los Angeles , California .
Output
st	Stephen Bruner ( born October 19, 1984 ) , better known by his stage name Thundercat , is an American multi-genre bass player , producer and singer from Los Angeles, California.
Stephen Bruner ( born October 19, 1984 ) , better known by his stage name Thundercat , is an American multi-genre bass player , producer and singer from Los Angeles , California.
Claim
st-1	The House that Jack Built has an approval rate of over 45 % and over 13 critics .
On Rotten Tomatoes , the film has an approval rating of 42 % , based on 12 critics , and an average rating of 5.9/10
Output
st	On Rotten Tomatoes, the film has an approval rating of 47 %, based on 14 critics, and an average rating of 5.9/10.
On Rotten Tomatoes , the film has an approval rating of 47 %, based on 15 critics , and an average rating of 6.3/10.
Claim
st-1	Cartoon Network is owned by Warner Bros. Entertainment .
Cartoon Network is an American pay television channel owned by Turner Broadcasting System , a subsidiary of AT & T ’s WarnerMedia .
Output
st	Cartoon Network is an American pay television channel owned by Warner Bros. Entertainment, a subsidiary of AT & T’s WarnerMedia.
Cartoon Network is an American pay television channel owned by Warner Bros. Entertainment , a subsidiary of AT & T’s WarnerMedia .
Claim
st-1	The New York Times reviewed the film Mad Max : Fury Road .
Lindsay Bahr of The Associated Press wrote , “ Miller has reminded us that blockbusters have the potential to not only be art , but radically visionary – even the fourth in a series .
Output
st	Lindsay Bahr of The New York Times wrote, “ Miller has reminded us that blockbusters have the potential to not only be art , but radically visionary – even the fourth in a series .
The New York Times wrote , “ Miller has reminded us that blockbusters have the potential to not only be art , but radically visionary – even the fourth in a series .
Table E.2: Example outputs of the BART-base used for generating factually consistent revisions given the old version  $s_{t-1}$  and the updated claim we wish to support. The "ground-truth"  $s_t$  is provided for reference.
(st-1,st)	WWE 2K15: As of August 2015, WWE 2K15 has shipped over{*| six -> 7.5 |}million units.
BART (VitC)	WWE 2K15 sold more than 7 million units.	| fverdict(c,st) = SUP
GPT-3 T=0	As of August 2015, WWE 2K15 has shipped over seven million units	SUP
GPT-3 T=0.7	As of August 2015, WWE 2K15 has shipped over seven million units	SUP
Reference	By August 2015, WWE 2K15 shipped over 7.5 million units.	SUP
(st-1,st)	Pat Jennings: He has played for League of Ireland clubs UCDDerry and is now at{*| Shamrock Rovers -> Dublin |}.
BART (VitC)	Pat Jennings is currently playing for Dublin club UCDDerry.	SUP
GPT-3 T=0	He has played for League of Ireland clubs UCD and is now at Shamrock Rovers	REF
GPT-3 T=0.7	He played for Shamrock Rovers and is now at Dublin	REF
Reference	Pat Jennings currently plays for the Dublin club.	SUP
Table E.3: Additional examples for Table 7. Example outputs for extracting claims that express the factual change in a Wikipedia revision. The BART-base model is trained on VITAMINC data and GPT-3 is applied in a 2-shot setting with a temperature of 0 or 0.7. The revision  $(s_{t - 1}, s_t)$  is given to the model as a single sentence visualization where the edits are between curly brackets, preceded by the article's title. The human-written claim is provided for reference. The prediction of our ALBERT-xlarge VITAMINC-trained model  $f_{\text{verdict}}(c, s_t)$  on the generated claim against  $s_t$  is also reported in the rightmost column.