Fool Me Twice: Entailment from W ikipedia Gamification (2021)

Abstract: We release FoolMeTwice (FM2 for short), a large dataset of challenging entailment pairs collected through a fun multi-player game. Gamification encourages adversarial examples, drastically lowering the number of examples that can be solved using “shortcuts” compared to other popular entailment datasets. Players are presented with two tasks. The first task asks the player to write a plausible claim based on the evidence from a Wikipedia page. The second one shows two plausible claims written by other players, one of which is false, and the goal is to identify it before the time runs out. Players “pay” to see clues retrieved from the evidence pool: the more evidence the player needs, the harder the claim. Game-play between motivated players leads to diverse strategies for crafting claims, such as temporal inference and diverting to unrelated evidence, and results in higher quality data for the entailment and evidence retrieval tasks. We open source the dataset and the game code.

## introducing a game of challenging claims given a statement-and a large collection of textual knowledge-how do you find evidence that shows a reader that the statement is true or false? this problem takes on multiple forms in the natural language processing (nlp) community. given only a single statement and a single sentence, this decision process is called recognizing textual entailment @xcite or natural language inference @xcite . given a single statement and a vast pool of possible evidence (e.g., all of wikipedia), this problem is called verification @xcite . stage 2: players mark evidence that entails/refutes the claim on earth it is caused by wind and rain erosion. on venus, about 85% of the craters are in pristine condition. venusian craters range from 3 to 280 km. stage 3: players spot the refuted claim in stages 1 to 4, players write challenging claims either entailed or refuted by evidence from wikipedia (section 3.1). they are then tasked to spot the refuted claim among a group (section 3.2). the claims and evidence are available for download. we review existing resources for the latter task in section 2 and how they have spawned a vibrant subcommunity around related tasks. however, these datasets fail to challenge modern nlp models such as bert @xcite or t5 @xcite that achieve "super-human performance" despite also exhibiting "annotation artifacts" that hurt their generalization potential @xcite . our goal is twofold: (1) to build a new, challenging dataset (statistics for foolmetwice in not only does this create more realistic claimsthe best must withstand human scrutiny-it also creates a way to better evaluate the evidence that support or refute claims. as we surface the evidence, humans use that evidence to decide which claims are true or false; these signals can further improve our systems (figure 1 ). we apply baseline models for retrieval and classification to our dataset (section 4) and examine how their ability to detect wrong statements differs from humans' (section 5). ## related work entailment is a key task in natural language understanding. @xcite describe it as an ai-complete task: solve it, and you can solve all of artificial intelligence. typically, entailment is presented as: given a premise ("brooklyn is the most populous of new york city's boroughs"), decide whether a hypothesis ("manhattan has more residents than brooklyn") is entailed-supportedby the premise. even simple examples show the promise (and complexity) of this task. to recognize that this hypothesis is contradicted, a model must: know that manhattan is a borough of new york, recognize that "x is the most populous borough" entails "x has more residents than any other borough", and correctly combine this knowledge to recognize the contradiction. ## foolmetwice game mechanics this section outlines the two phases of the game: authoring claims (section 3.1) and voting on those claims (section 3.2). while these sections present the game in its final form, this is the reflection of an iterative process. we first began with a paper version @xcite of the game, which showed that a time constraint made the game feel more fun and encouraged people to not read individual pieces of evidence too intently. without the timer, people tried to look for tiny clues in text that probably were not there @xcite . we then moved onto a version of the game presented via slides where we experimented with design choices such as the number of claims players distinguish between, and the number of evidence sentences they see while doing that. examples of the final web interface are shown in appendix b. ## methods: subtasks and models each of the instances in foolmetwice is a tuple (c, e, l): a natural language claim c, evidence e from a knowledge corpus k (in our case wikipedia), and a binary label l (entailment / contradiction). 5 from this we define two sub-tasks, following @xcite . the first sub-task, retrieval, requires systems to select candidate evidence from k (including, perhaps, the gold evidence e). the second sub-task is entailment, where systems given claim c and the gold evidence e need to make a final prediction for the label l. we also consider an end-to-end setting. instead of the gold evidence, systems only have access to the retrieved evidence ê at test time. in the rest of this section we define baseline models for each of the sub-tasks. ## experiment results: machines spotting false claims this section studies the performance of existing automatic methods on fm2 for both the retrieval of evidence (section 5.1) and for entailment once the results are retrieved (section 5.2). ## dataset analysis: humans spotting and writing false claims while the previous section focuses on how well automatic methods can detect false claims, this section focuses on human ability. voters are usually right and were fooled 20.40% of the time. this section addresses how players are fooled and how this compares to computers. to provide a better picture of the strategies players use to craft challenging claims, we manually sample fifty instances from the development set that both models and humans answer incorrectly. we focus on these examples because they are the most difficult and are the emphasis of our adversarial technique. two claims were mislabeled and two more lacked a necessary evidence span. temporal many of the most challenging claims require an inference about time: whether one event happened before another, how long an event happened, or whether an event happened during a period. while many of these are based on years, centuries, or other explicit markers of time, some authors use narrative time. for example, the page for the novel as i lay dying describes the plot in order, so it's difficult for either a system or a human given sentences (without knowing where they appear in the original page) to know when addie bundren dies. this shows some of the limitations of the setup: not only must voters reason across multiple pieces of evidence, this reasoning is only possible if they know the order in the underlying evidence. other markers of time include "the pilot" for the first episode of the office; readers must realize that if kelly kapoor was introduced in the episode diversity day, that implies mindy kaling's character did not appear in the pilot. reasoning a related, but more general, strategy requires the reader to reason: mathematically, applying definitions, or understanding hyponomy. for example, knowing that the child of your cousin is your second cousin or recognizing that "this mirrors the disney parks east regional division consisting of shanghai disney resort, hong kong disneyland and walt disney attractions japan. . . " implies that there are more than two walt disney resorts outside of the united states. ## limitations and conclusion while this paper seeks to advance the ability of humans and computers to support or refute statements entailed from a static, reliable source, the goal of examining arbitrary statements remains elusive. by construction, we have focused on statements that are incorrect because of factual errors. other datasets that use human-sourced obfuscations or deception are more nuanced and use framing or shading @xcite , which models trained on this dataset cannot detect. our goal is to focus on clear facts that can be recognized by computers, which is already challenging enough. further improving verification likely requires creating targeted datasets that focus on specific strategies for creating statements that are refuted by evidence, perhaps selecting different explanations for particular users @xcite . likewise, a more complicated task likely requires more nuanced incentives and instructions for authors. however, this dataset provides a foundation to build these richer, more challenging datasets for entailment.