SlowGuess/NAACL_2021_OCR huggingface dataset:
Byeongchang Kim* Hyunwoo Kim* Seokhee Hong Gunhee Kim
Seoul National University, Seoul, Korea
{byeongchang.kim,hyunw.kim,seokhee.hong}@vl.snu.ac.kr，gunhee@snu.ac.kr
https://vl.snu.ac.kr/projects/colloquial-claims
Knowledge is now starting to power neural dialogue agents. At the same time, the risk of misinformation and disinformation from dialogue agents also rises. Verifying the veracity of information from formal sources are widely studied in computational fact checking. In this work, we ask: How robust are fact checking systems on claims in colloquial style? We aim to open up new discussions in the intersection of fact verification and dialogue safety. In order to investigate how fact checking systems behave on colloquial claims, we transfer the styles of claims from FEVER (Thorne et al., 2018) into colloquialism. We find that existing fact checking systems that perform well on claims in formal style significantly degenerate on colloquial claims with the same semantics. Especially, we show that document retrieval is the weakest spot in the system even vulnerable to filler words, such as "yeah" and "you know". The document recall of WikiAPI retriever (Hanselowski et al., 2018) which is  $90.0\%$  on FEVER, drops to  $72.2\%$  on the colloquial claims. We compare the characteristics of colloquial claims to those of claims in formal style, and demonstrate the challenging issues in them.
Recently, knowledge has been starting to power neural dialogue agents (Moghe et al., 2018; Zhou et al., 2018b; Ghazvininejad et al., 2018; Qin et al., 2019; Gopalakrishnan et al., 2019), being equipped with Wikipedia (Dinan et al., 2019b), news (Gopalakrishnan et al., 2019), domain specific knowledge-base (Eric and Manning, 2017), and commonsense (Zhou et al., 2018a; Young et al., 2018; Wu et al., 2020). However, the use of knowledge inevitably put dialogue agents in new jeopardy. For example, recent workshop on safety for conversational AI (Dinan et al., 2020b) introduced
an example of such risk: Bickmore et al. (2018) asked participants to query conversational agents for advice in situations where medical information is needed. Then, internist and pharmacist judged the actions that the participants would take based on the advice. Assessments revealed that agents often deliver incorrect medical information that may cause lethal consequences.
A bigger threat may be the abuse of dialogue agents to deliberately distribute disinformation. What would happen if knowledge-powered agents are tweaked to massively generate false claims on online communities? The impact of such fake news can be critical as they quickly spread through social media (Shu et al., 2017). The chatbot Tay's shut down due to malicious attempts show the imminent danger of abuse (Wolf et al., 2017).
Verifying the integrity of a given piece of information has been studied in the field of computational fact checking. Thorne et al. (2018) introduce an annotated dataset FEVER for fact checking based on Wikipedia. Augenstein et al. (2019) collect claims on fact checking websites and release the MultiFC dataset. Jiang et al. (2020) collect a dataset requiring many-hop evidence extraction from Wikipedia. Wadden et al. (2020) collect a dataset of scientific claims to be verified.
Most claims of existing datasets are taken from formal texts, such as news, academic papers, and Wikipedia. These claims tend to be concise and structured: "Beautiful was number two on the Billboard Hot 100 in 2003". On the other hand, claims or information that we encounter in dialogues are more unstructured and informal: "The song Beautiful is great! It even reached number two on the Hot 100 in 2003, you know?". For improving the applicability of fact checking systems, they must also be robust for verifying the claims in dialogues.
Unfortunately, threats regarding misinformation and disinformation from dialogue agents remain understudied. Research on dialogue safety mainly
Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 1535-1548 June 6-11, 2021. ©2021 Association for Computational Linguistics
has focused on making dialogue agents robust to adversarial attacks (Dinan et al., 2019a), and preventing dialogue agents from generating offensive or biased responses (Henderson et al., 2018; Sap et al., 2019; Xu et al., 2020).
In this work, we aim to investigate how fact checking systems behave when verifying claims in dialogue style, rather than claims from news outlets, scientific articles, or Wikipedia. Colloquial claims are different in several aspects compared to claims from formal sources. (i) They tend to also include filler words, casual comments, or personal feelings which do not require verification. (ii) Since claims in colloquial language are less precise than formal claims, correctly using the context in claims becomes important to disambiguate them. We demonstrate that these features make existing fact checking systems have difficulties in verifying colloquial claims. We use English datasets for the investigation in this work. Our major contributions of this work can be outlined as follows:
(1) We open up new discussions in the intersection of fact verification and dialogue safety; how to verify claims in colloquial language, compared to previous works that solely focus on the claims in formal style (e.g. news, academic papers, Wikipedia).
(2) For this study, we curate colloquial claims by transferring the styles of claims in existing fact checking dataset of FEVER (Thorne et al., 2018). For style transfer, we finetune a pretrained dialogue model with a knowledge-grounded dialogue dataset and apply additional filtering to compensate for the quality of output.
(3) We show that the existing fact checking systems that perform well on claims in formal style significantly degenerate on colloquial ones with the same semantics. We analyze the performance drop and show document retrieval is the weakest spot in the system.
(4) We identify the challenging characteristics of colloquial claims; (i) they often involve expressions that are not verifiable (e.g. filler words or personal feeling) and (ii) they include ambiguity inside the claim that necessities better understanding of the context. We release the code and the curated colloquial claims set.
FEVER (Thorne et al., 2018)
Claim: The iPhone 4 is a dial telephone.
Wikipedia Document: [iPhone]
Evidence Sentence: The iPhone 4 is a smartphone that was designed and marketed by Apple Inc..
Label: REFUTED
Wizard of Wikipedia (Dinan et al., 2019b)
Topic: [The Hershey Company]
Wikipedia Knowledge: Headquarters are in Hershey,
Pennsylvania, which is home to Hershey's Chocolate World.
Apprentice (i.e. dialogue context):
I love chocolate, my favorite is Hershey. What's yours?
Wizard: I love Hershey too! Do you know that Hershey's HQ is actually located in Hershey, Pennsylvania?
Table 1: Example of FEVER and Wizard of Wikipedia.
2.1 Fact Checking Pipeline
FEVER (Thorne et al., 2018) is a fact checking benchmark dataset based on Wikipedia. Its fact checking pipeline has become one of the standard followed by many (Hanselowski et al., 2018; Nie et al., 2019; Zhou et al., 2019; Liu et al., 2020; Zhong et al., 2020; Jiang et al., 2020). The pipeline comprises three stages: document retrieval, evidence selection, and claim verification. For a given claim to be verified, the system first retrieves the related documents from the pool. Next, among the returned documents, the system selects the most suitable sentences for evidence. Finally, based on the evidence sentences the system classifies the claim's veracity with three classes: SUPPORTED, REFUTED (contradicted by the evidence), and NOTENOUGHINFO (cannot be determined by the evidence). An example from the FEVER is shown in Table 1.
2.2 Wizard of Wikipedia
The Wizard of Wikipedia (WoW) (Dinan et al., 2019b) may be the closest dialogue dataset to existing fact checking datasets. It is a knowledge-based open-domain dialogue dataset involving two speakers discussing on a given topic. An example is presented in Table 1. One speaker (referred as apprentice) is eager to learn about the topic, while the other speaker (the wizard) delivers knowledge-grounded responses based on both dialogue context and Wikipedia documents for the topic. In this dataset, the gold "knowledge sentence" from Wikipedia is provided for each wizard's response. Hence, we can regard the gold knowledge sentence as the evidence for the Wizard's response.
However, WoW only provides pairs of (knowledge sentence, grounded response), hence those responses are all SUPPORTED by Wikipedia. There are no REFUTED or NOTENOUGHINFO responses in the dataset. Such limitation makes it difficult to directly adopt WoW as a fact checking dialogue dataset. Nonetheless, its knowledge-grounding property makes it a useful resource for training dialogue models to generate colloquial utterances grounded on claims.
3 Transferring to Colloquial Claims
Our goal is to curate colloquial claims by transferring the style of each claim sentence in the FEVER dataset into colloquial style. We first finetune a dialog model with the WoW dataset so that it learns to transfer knowledge sentences from Wikipedia into conversational utterances (section 3.1). We then apply the finetuned model to transfer each claim in FEVER (sourced from Wikipedia) into colloquial style, and perform filtering process to warrant the integrity of this style transfer (section 3.2). Figure 1 overviews the whole pipeline of style transfer.
3.1 Finetuning a Dialogue Model
We first finetune BART-large (Lewis et al., 2020) to generate the wizard's response given only the corresponding knowledge sentence from WoW, without the dialogue context. Take the example in Table 1, when the knowledge sentence is given as "Hershey's headquarters are in Hershey, Pennsylvania", BART is finetuned to generate the wizard's response "I love Hershey too! Do you know that Hershey's HQ is actually in Hershey?" We exclude the dialogue context during fine-tuning in order to enforce the dialogue model to exclusively focus on knowledge contents. The finetuned BART shows a low perplexity of 10.51 on WoW's validation set. This indicates that BART can generate information-grounded utterances when given knowledge sentences.
Then, we apply the finetuned BART to transfer each claim in FEVER to a colloquial one. Our expectation is that since claims in FEVER are based on Wikipedia too and similar to knowledge sentences in WoW in many aspects, the finetuned model may be able to produce utterances while preserving the semantics of claims from FEVER.
However, naively using the generated claims as is has several issues, including (i) copy-and-paste, (ii) pronoun overwrite, (iii) semantic discrepancy, and (iv) lack of colloquialism. We carefully mitigate these issues through a filtering pipeline.
3.2 Oversampling and Filtering
We first oversample  $n$  colloquial candidates  $Q_{i} = \{q_{i,j}\}_{j=1}^{468}$  per claim  $c_{i}$  in FEVER, using BART through Nucleus Sampling (Holtzman et al., 2020) ( $p = 0.95$ ).
Preventing Copy-Paste. We observe the dialogue model sometimes simply copies the input claim as output. Since copy-pasted candidates are not colloquial, we remove the ones whose F1 scores are higher than 0.9, in respect to the original claim.
Preserving Named Entities. Utterances in dialogues tend to refer entities with pronouns rather than their original word. As a result, we observe that dialogue models also convert entities in claims to pronouns. For example, given the input claim "Tetris has sold millions of physical copies", BART outputs "Yeah it's fun even today, no wonder it sold millions of physical copies". Since there are no previous contexts for claims in FEVER, it is not possible to recognize that pronoun "it" is referring to "tetris".
In order to preserve the entities, we leverage the named entity recognition (NER) module from Stanza (Qi et al., 2020), which shows 88.8 F1-score on OntoNotes (Weischedel et al., 2013) test set. We extract a set of named entities  $\mathcal{E}_i^c$  from claim  $c_{i}$  and compare it with the named entity set  $\mathcal{E}_{i,j}^{q}$  of each  $q_{i,j}$  in  $Q_{i}$ . We remove candidates with less than two matching named entities. For claims with single named entity, we remove candidates having no named entities.
Preserving Semantic Equivalence. It is well known that neural dialogue models lack consistency (Li et al., 2016) and can hallucinate irrelevant content (Roller et al., 2020). As a result, there can be semantic difference between the original FEVER claim and the generated one.
To preserve the original semantics, we leverage natural language inference (NLI), which is a task of determining whether a hypothesis sentence can be inferred from the given premise sentence. The hypothesis sentence is classified into three categories: ENTAILMENT (true), CONTRADICTION (false), and NEUTRAL (undetermined). A sound colloquial claim should be entailed by the original
<sup>1</sup>We verified that FEVER is released under a Creative Commons (CC BY-SA 3.0) license.
Figure 1: Illustration of the transfer pipeline for our Colloquial Claims.
F1-score	NER	NLI	AF
Avg. Cumulative Survival Rate (%)	96.2	46.4	6.3	top-k
Table 2: The average cumulative survival rate of candidates after each filtering. We apply filters in the order of F1, NER, NLI, and AF.
Figure 2: The recall for AFLITE linear classifiers for our Colloquial Claims.
claim and it also must not contradict the original. Suppose the original claim is "Apple Inc. designed and manufactured iPhone 4" and the generated claim is "I heard Apple is also famous for designing the iMac computer". This claim is removed because "designing iMac" cannot be inferred from the fact "Apple manufactured iPhone 4".
We conduct bidirectional NLI between the original claim and the generated one using RoBERTa (Liu et al., 2019) trained on MNLI (Williams et al., 2018). The RoBERTa model shows  $90.59\%$  accuracy on MNLI validation set. For each candidate  $q_{i,j}$ , we conduct  $\mathrm{NLI}(c_i, q_{i,j})$  and  $\mathrm{NLI}(q_{i,j}, c_i)$  with the original claim  $c_i$ . We only preserve the candidates that result in ENTAILMENT for the former and do not result in CONTRADICTION for the latter.
Ensuring Colloquialism. Although the candidates are generated by a dialogue model, they may still resemble the style of the original claims, rather than colloquial style. To ensure colloquialism, we select the top- $k$  candidate claims which are most
difficult to discriminate from responses in Wizard of Wikipedia (WoW) (Dinan et al., 2019b), through an iterative adversarial filtering method AFLITE (Sakaguchi et al., 2020; Bras et al., 2020). We first embed the candidates with RoBERTa and train an ensemble of binary linear classifiers to determine each candidate whether it is from WoW or our colloquial claims. We eliminate candidates that are easily classified as our colloquial claims after each iteration. We continue the iteration until  $k$  candidates remain in each  $Q_{i}$ . We set  $k = 3$ . Since only candidates that are hard to discriminate from WoW responses survive, they resemble the styles of dialogue utterances. We defer the detailed algorithm for adversarial filtering to Appendix.
Filtering Statistics. Table 2 shows the average survival rate of candidates after each filtering step. We observe that the NER and NLI filter effectively remove large amounts of candidates. On average, 29 out of 486 candidates survive after the NLI filtering stage. Then, adversarial filtering is used for selecting  $k$  candidates among the remainders.
Figure 2 shows the recall for our colloquial claims by the binary classifiers used in AFLITE. As only indistinguishable candidate claims from the WoW responses survive, the recall drops after each iteration. We also compare the qualitative traits of candidates before and after the filtering in Section 4.2.
3.3 Manual Quality Check on Test set
Finally, we manually check all SUPPORTED and REFUTED instances in the test set of our Colloquial Claims dataset. Three human annotators choose the best suitable claim for each colloquial claim set  $(|Q_{i}| \leq k)$  for the given label and evidence. If there are no suitable claim in the set, we recover the set before top- $k$  selection. As a last resort, we let annotators rewrite the colloquial claim when no eligible candidate exists. The proportion that requires manual rewriting is less than  $1\%$  of 5,615
#Claims	#Words/Claim
Train	Valid	Test
FEVER	145.4K	10K	10K	8.2
Colloquial Claims	410.0k	28.9K	8.4K	11.1
Figure 3: Comparison of the claim sentence length distributions between FEVER and our Colloquial Claims.
instances.
4 Properties of Colloquial Claims
We first discuss the characteristics of our Colloquial Claims with quantitative analysis, compared to FEVER (Thorne et al., 2018) and Wizard of Wikipedia (WoW) (Dinan et al., 2019b).
Diverse Claims. We provide basic statistics of our Colloquial Claims in Table 3. In FEVER, only a single claim exists per evidence set, whereas our Colloquial Claims provide up to three claims. As a result, the number of data instances of our dataset is larger than FEVER.
Due to the wordy nature of colloquial language, the our transferred claims are longer and more diverse in length than those in FEVER. Figure 3 plots the density of the claim sentence lengths of FEVER and our dataset.
Colloquial Style. The claims in our Colloquial Claims have similar styles to the utterances in dialogues. Following Yang et al. (2020), we gauge the style of sentences by measuring the perplexity with a pretrained DialoGPT (Zhang et al., 2019). The perplexity of the sentence becomes high if its style is far from a dialogue. Table 4 compares the perplexity of responses from WoW, claims from FEVER and our Colloquial Claims. The perplexity of claims in FEVER is high, whereas our Colloquial Claims have closer perplexity to WoW.
Table 3: Statistics of the Colloquial Claims compared to FEVER (Thorne et al., 2018).
WoW	FEVER	Colloquial Claims
DiaLoGPT Perplexity	471.9	1381.5	575.8
Table 4: The perplexity measured by DialoGPT (Zhang et al., 2019) for the responses in Wizard of Wikipedia (WoW) (Dinan et al., 2019b), claims in FEVER (Thorne et al., 2018) and our Colloquial Claims. Higher perplexity implies that sentences are far from the styles of dialogues.
FEVER	Colloquial Claims
film	american	yes	know
born	released	actually	yeah
series	award	movie	film
stars	won	called	american
actor	united	born	like
directed	starred	heard	yea
television	album	released	played
world	worked	won	oh
movie	states	actor	award
john	played	people	world
written	appeared	series	album
role	character	great	directed
Table 5: Comparison between the top-20 tokens of FEVER and our Colloquial Claims.
Table 5 also compares the top-20 frequent tokens in the claims from FEVER and our dataset. The most frequent tokens in FEVER's claims are mostly fact-related words, such as "american", "released", and "born". On the other hand, the claims in our Colloquial Claims also have tokens that frequently appear in conversations, such as "know", "actually", "like", and "oh".
4.2 Qualitative Comparisons
We conduct human evaluation via Amazon Mechanical Turk to investigate the effectiveness of our filtering pipeline. We random sample 100 data instances from our Colloquial Claims and compare between survived and removed candidates. Each instance is rated by three unique human annotators.
To evaluate the overall quality of our generated claims, we ask human users to evaluate humanness in 4-point scale: "Do you think this sentence is from a bot or a human?" We compare them with responses from WoW and FEVER on humanness.
We also conduct NLI on the claims from our Colloquial Claims and FEVER to evaluate the label mappings. Users are instructed to classify claims into three veracity labels given the gold evidence: SUPPORTED, REFUTED, NOTENOUGHINFO.
Humanness	Human NLI
Wizard of Wikipedia	3.12	-
FEVER	2.57	0.96
Removed Claims	2.95	0.50
Survived Claims	2.94	0.82
Table 6: Human evaluation results comparing the humanness and NLI performance between responses from Wizard of Wikipedia, claims from FEVER, our removed claims and survived claims.
Table 6 summarizes the averaged humanness and human NLI scores. Since the responses in WoW are from real dialogues, we can observe they have the highest humanness score. Interestingly, our generated claims are evaluated to be better than human-generated claims in FEVER, in terms of humanness. We suspect that this is due to the colloquialism of our generated claims.
The survived claims have more accurate label mappings with the evidence, compared to removed candidates. It is thanks to the bidirectional NLI filter that removes the candidate claims that are semantically different from the original claims. Table 7 shows some examples comparing our generated claims to the original FEVER claims.
We conduct experiments on our curated colloquial claims to see how they impact existing fact checking systems.
Datasets. FEVER (Thorne et al., 2018) consists of three steps of fact checking pipeline: document retrieval, evidence selection, and claim verification. Based on selected evidence, the claims are classified into three classes of veracity: SUPPORTED, REFUTED, NOTENOUGHINFO. The Colloquial Claims is our generated dataset based on FEVER with claims in the colloquial style.
Metrics. FEVER fact checking uses two performance scores: label accuracy and FEVER-score. Label accuracy is the claim verification performance of the fact checking system. The FEVER-score is a more complicated evaluation regarding the whole pipeline. Following the FEVER challenge², a claim verification is evaluated as correct if the system retrieves at least one complete set of ground-truth evidence sentences and also classifies
FEVER: Google Search displays movie showtimes.
Colloquial Claim:
I can try google search to see
what movie to watch and get show times!
FEVER:
Unison (Celine Dion album) was originally released by Atlantic Records.
I remember the Celine Dion album titled Unison.
It was released by Atlantic Records.
FEVER: Firefox is a desktop browser.
Yes, I use something calledfirefox for my desktop browser.
FEVER: Kung Fu Panda was released in theaters in 2006.
Have you watched Kung Fu Panda? It came out in 2006.
FEVER: San Francisco Bay Area contains many airports.
Sure, and yes there are lots of Bay Area airports!
Birthday Song's (2 Chainz song) producer was Mike Dean.
Do you listen to Birthday Song by 2 Chainz?
It was produced by Mike Dean.
Table 7: Examples of generated colloquial claims for the original FEVER claims.
the claim correctly. For the evidence sentences, we evaluate the first 5 sentences retrieved from the system. We also report the recall for retrieved documents and selected evidence sentences.
5.2 Fact-Checking Baselines
We run experiments on six combinations of the fact-checking system according to the steps. For each dataset evaluation, we finetuned the system on the respective dataset.
Document Retrieval. We test three types of approaches: (1) oracle, (2) term-matching, and (3) similarity search with dense representation. First, the oracle always returns five evidence sentences including the gold evidence. Second, the WikiAPI<sup>3</sup>, following Hanselowski et al. (2018), retrieves Wikipedia documents by matching words in the claim through a python library. Third, Dense Passage Retrieval (DPR) (Karpukhin et al., 2020) retrieves documents via similarity search with BERT embeddings trained by metric learning.
<sup>3</sup>We adopt the implementation by Hanselowski et al. (2018) at https://github.com/UKPLab/fever-2018-team-athene
2https://fever.ai/2018/task.html
Dataset	Document Retrieval +Evidence Selection	Document Recall	Evidence Recall	Veracity Classification
KGAT(BERT)	KGAT(CorefBERT)
Label Accuracy	FEVER score	Label Accuracy	FEVER score
FEVER	Evidence Oracle	-	-	69.7	-	77.5	-
WikiAPI + BERT	90.0	85.3	67.5	62.4	73.8	69.5
Dense Passage Retrieval + BERT	84.0	81.8	62.9	55.4	61.1	52.4
Colloquial Claims	Evidence Oracle	-	-	57.3	-	67.7	-
WikiAPI + BERT	72.2	73.4	53.2	43.6	60.9	52.4
Dense Passage Retrieval + BERT	79.6	77.4	51.2	41.5	61.0	55.4