Introduction
Misinformation and disinformation are two of the most pertinent and difficult challenges of the information age, exacerbated by the popularity of social media. In an effort to counter this, a significant amount of manual labour has been invested in fact checking claims, often collecting the results of these manual checks on fact checking portals or websites such as politifact.com or snopes.com. In a parallel development, researchers have recently started to view fact checking as a task that can be partially automated, using machine learning and NLP to automatically predict the veracity of claims. However, existing efforts either use small datasets consisting of naturally occurring claims (e.g. BIBREF0 , BIBREF1 ), or datasets consisting of artificially constructed claims such as FEVER BIBREF2 . While the latter offer valuable contributions to further automatic claim verification work, they cannot replace real-world datasets.
Datasets
Over the past few years, a variety of mostly small datasets related to fact checking have been released. An overview over core datasets is given in Table TABREF4 , and a version of this table extended with the number of documents, source of annotations and SoA performances can be found in the appendix (Table TABREF1 ). The datasets can be grouped into four categories (I–IV). Category I contains datasets aimed at testing how well the veracity of a claim can be predicted using the claim alone, without context or evidence documents. Category II contains datasets bundled with documents related to each claim – either topically related to provide context, or serving as evidence. Those documents are, however, not annotated. Category III is for predicting veracity; they encourage retrieving evidence documents as part of their task description, but do not distribute them. Finally, category IV comprises datasets annotated for both veracity and stance. Thus, every document is annotated with a label indicating whether the document supports or denies the claim, or is unrelated to it. Additional labels can then be added to the datasets to better predict veracity, for instance by jointly training stance and veracity prediction models.
Methods not shown in the table, but related to fact checking, are stance detection for claims BIBREF14 , BIBREF15 , BIBREF16 , BIBREF17 , BIBREF18 , BIBREF19 , BIBREF20 , satire detection BIBREF21 , clickbait detection BIBREF22 , conspiracy news detection BIBREF23 , rumour cascade detection BIBREF24 and claim perspectives detection BIBREF25 .
Claims are obtained from a variety of sources, including Wikipedia, Twitter, criminal reports and fact checking websites such as politifact.com and snopes.com. The same goes for documents – these are often websites obtained through Web search queries, or Wikipedia documents, tweets or Facebook posts. Most datasets contain a fairly small number of claims, and those that do not, often lack evidence documents. An exception is BIBREF2 , who create a Wikipedia-based fact checking dataset. While a good testbed for developing deep neural architectures, their dataset is artificially constructed and can thus not take metadata about claims into account.
Contributions: We provide a dataset that, uniquely among extant datasets, contains a large number of naturally occurring claims and rich additional meta-information.
Methods
Fact checking methods partly depend on the type of dataset used. Methods only taking into account claims typically encode those with CNNs or RNNs BIBREF3 , BIBREF4 , and potentially encode metadata BIBREF3 in a similar way. Methods for small datasets often use hand-crafted features that are a mix of bag of word and other lexical features, e.g. LIWC, and then use those as input to a SVM or MLP BIBREF0 , BIBREF4 , BIBREF13 . Some use additional Twitter-specific features BIBREF26 . More involved methods taking into account evidence documents, often trained on larger datasets, consist of evidence identification and ranking following a neural model that measures the compatibility between claim and evidence BIBREF2 , BIBREF27 , BIBREF28 .
Contributions: The latter category above is the most related to our paper as we consider evidence documents. However, existing models are not trained jointly for evidence identification, or for stance and veracity prediction, but rather employ a pipeline approach. Here, we show that a joint approach that learns to weigh evidence pages by their importance for veracity prediction can improve downstream veracity prediction performance.
Dataset Construction
We crawled a total of 43,837 claims with their metadata (see details in Table TABREF1 ). We present the data collection in terms of selecting sources, crawling claims and associated metadata (Section SECREF9 ); retrieving evidence pages; and linking entities in the crawled claims (Section SECREF13 ).
Selection of sources
We crawled all active fact checking websites in English listed by Duke Reporters' Lab and on the Fact Checking Wikipedia page. This resulted in 38 websites in total (shown in Table TABREF1 ). Ten websites could not be crawled, as further detailed in Table TABREF40 . In the later experimental descriptions, we refer to the part of the dataset crawled from a specific fact checking website as a domain, and we refer to each website as source.
From each source, we crawled the ID, claim, label, URL, reason for label, categories, person making the claim (speaker), person fact checking the claim (checker), tags, article title, publication date, claim date, as well as the full text that appears when the claim is clicked. Lastly, the above full text contains hyperlinks, so we further crawled the full text that appears when each of those hyperlinks are clicked (outlinks).
There were a number of crawling issues, e.g. security protection of websites with SSL/TLS protocols, time out, URLs that pointed to pdf files instead of HTML content, or unresolvable encoding. In all of these cases, the content could not be retrieved. For some websites, no veracity labels were available, in which case, they were not selected as domains for training a veracity prediction model. Moreover, not all types of metadata (category, speaker, checker, tags, claim date, publish date) were available for all websites; and availability of articles and full texts differs as well.
We performed semi-automatic cleansing of the dataset as follows. First, we double-checked that the veracity labels would not appear in claims. For some domains, the first or last sentence of the claim would sometimes contain the veracity label, in which case we would discard either the full sentence or part of the sentence. Next, we checked the dataset for duplicate claims. We found 202 such instances, 69 of them with different labels. Upon manual inspection, this was mainly due to them appearing on different websites, with labels not differing much in practice (e.g. `Not true', vs. `Mostly False'). We made sure that all such duplicate claims would be in the training split of the dataset, so that the models would not have an unfair advantage. Finally, we performed some minor manual merging of label types for the same domain where it was clear that they were supposed to denote the same level of veracity (e.g. `distorts', `distorts the facts').
This resulted in a total of 36,534 claims with their metadata. For the purposes of fact verification, we discarded instances with labels that occur fewer than 5 times, resulting in 34,918 claims. The number of instances, as well as labels per domain, are shown in Table TABREF34 and label names in Table TABREF43 in the appendix. The dataset is split into a training part (80%) and a development and testing part (10% each) in a label-stratified manner. Note that the domains vary in the number of labels, ranging from 2 to 27. Labels include both straight-forward ratings of veracity (`correct', `incorrect'), but also labels that would be more difficult to map onto a veracity scale (e.g. `grass roots movement!', `misattributed', `not the whole story'). We therefore do not postprocess label types across domains to map them onto the same scale, and rather treat them as is. In the methodology section (Section SECREF4 ), we show how a model can be trained on this dataset regardless by framing this multi-domain veracity prediction task as a multi-task learning (MTL) one.
Retrieving Evidence Pages
The text of each claim is submitted verbatim as a query to the Google Search API (without quotes). The 10 most highly ranked search results are retrieved, for each of which we save the title; Google search rank; URL; time stamp of last update; search snippet; as well as the full Web page. We acknowledge that search results change over time, which might have an effect on veracity prediction. However, studying such temporal effects is outside the scope of this paper. Similar to Web crawling claims, as described in Section SECREF9 , the corresponding Web pages can in some cases not be retrieved, in which case fewer than 10 evidence pages are available. The resulting evidence pages are from a wide variety of URL domains, though with a predictable skew towards popular websites, such as Wikipedia or The Guardian (see Table TABREF42 in the appendix for detailed statistics).
Entity Detection and Linking
To better understand what claims are about, we conduct entity linking for all claims. Specifically, mentions of people, places, organisations, and other named entities within a claim are recognised and linked to their respective Wikipedia pages, if available. Where there are different entities with the same name, they are disambiguated. For this, we apply the state-of-the-art neural entity linking model by BIBREF29 . This results in a total of 25,763 entities detected and linked to Wikipedia, with a total of 15,351 claims involved, meaning that 42% of all claims contain entities that can be linked to Wikipedia. Later on, we use entities as additional metadata (see Section SECREF31 ). The distribution of claim numbers according to the number of entities they contain is shown in Figure FIGREF15 . We observe that the majority of claims have one to four entities, and the maximum number of 35 entities occurs in one claim only. Out of the 25,763 entities, 2,767 are unique entities. The top 30 most frequent entities are listed in Table TABREF14 . This clearly shows that most of the claims involve entities related to the United States, which is to be expected, as most of the fact checking websites are US-based.
Claim Veracity Prediction
We train several models to predict the veracity of claims. Those fall into two categories: those that only consider the claims themselves, and those that encode evidence pages as well. In addition, claim metadata (speaker, checker, linked entities) is optionally encoded for both categories of models, and ablation studies with and without that metadata are shown. We first describe the base model used in Section SECREF16 , followed by introducing our novel evidence ranking and veracity prediction model in Section SECREF22 , and lastly the metadata encoding model in Section SECREF31 .
Multi-Domain Claim Veracity Prediction with Disparate Label Spaces
Since not all fact checking websites use the same claim labels (see Table TABREF34 , and Table TABREF43 in the appendix), training a claim veracity prediction model is not entirely straight-forward. One option would be to manually map those labels onto one another. However, since the sheer number of labels is rather large (165), and it is not always clear from the guidelines on fact checking websites how they can be mapped onto one another, we opt to learn how these labels relate to one another as part of the veracity prediction model. To do so, we employ the multi-task learning (MTL) approach inspired by collaborative filtering presented in BIBREF30 (MTL with LEL–multitask learning with label embedding layer) that excels on pairwise sequence classification tasks with disparate label spaces. More concretely, each domain is modelled as its own task in a MTL architecture, and labels are projected into a fixed-length label embedding space. Predictions are then made by taking the dot product between the claim-evidence embeddings and the label embeddings. By doing so, the model implicitly learns how semantically close the labels are to one another, and can benefit from this knowledge when making predictions for individual tasks, which on their own might only have a small number of instances. When making predictions for individual domains/tasks, both at training and at test time, as well as when calculating the loss, a mask is applied such that the valid and invalid labels for that task are restricted to the set of known task labels.
Note that the setting here slightly differs from BIBREF30 . There, tasks are less strongly related to one another; for example, they consider stance detection, aspect-based sentiment analysis and natural language inference. Here, we have different domains, as opposed to conceptually different tasks, but use their framework, as we have the same underlying problem of disparate label spaces. A more formal problem definition follows next, as our evidence ranking and veracity prediction model in Section SECREF22 then builds on it.
We frame our problem as a multi-task learning one, where access to labelled datasets for INLINEFORM0 tasks INLINEFORM1 is given at training time with a target task INLINEFORM2 that is of particular interest. The training dataset for task INLINEFORM3 consists of INLINEFORM4 examples INLINEFORM5 and their labels INLINEFORM6 . The base model is a classic deep neural network MTL model BIBREF31 that shares its parameters across tasks and has task-specific softmax output layers that output a probability distribution INLINEFORM7 for task INLINEFORM8 : DISPLAYFORM0
where INLINEFORM0 , INLINEFORM1 , INLINEFORM2 is the weight matrix and bias term of the output layer of task INLINEFORM3 respectively, INLINEFORM4 is the jointly learned hidden representation, INLINEFORM5 is the number of labels for task INLINEFORM6 , and INLINEFORM7 is the dimensionality of INLINEFORM8 . The MTL model is trained to minimise the sum of individual task losses INLINEFORM9 using a negative log-likelihood objective.
To learn the relationships between labels, a Label Embedding Layer (LEL) embeds labels of all tasks in a joint Euclidian space. Instead of training separate softmax output layers as above, a label compatibility function INLINEFORM0 measures how similar a label with embedding INLINEFORM1 is to the hidden representation INLINEFORM2 : DISPLAYFORM0
where INLINEFORM0 is the dot product. Padding is applied such that INLINEFORM1 and INLINEFORM2 have the same dimensionality. Matrix multiplication and softmax are used for making predictions: DISPLAYFORM0
where INLINEFORM0 is the label embedding matrix for all tasks and INLINEFORM1 is the dimensionality of the label embeddings. We apply a task-specific mask to INLINEFORM2 in order to obtain a task-specific probability distribution INLINEFORM3 . The LEL is shared across all tasks, which allows the model to learn the relationships between labels in the joint embedding space.
Joint Evidence Ranking and Claim Veracity Prediction
So far, we have ignored the issue of how to obtain claim representation, as the base model described in the previous section is agnostic to how instances are encoded. A very simple approach, which we report as a baseline, is to encode claim texts only. Such a model ignores evidence for and against a claim, and ends up guessing the veracity based on surface patterns observed in the claim texts.
We next introduce two variants of evidence-based veracity prediction models that encode 10 pieces of evidence in addition to the claim. Here, we opt to encode search snippets as opposed to whole retrieved pages. While the latter would also be possible, it comes with a number of additional challenges, such as encoding large documents, parsing tables or PDF files, and encoding images or videos on these pages, which we leave to future work. Search snippets also have the benefit that they already contain summaries of the part of the page content that is most related to the claim.
Our problem is to obtain encodings for INLINEFORM0 examples INLINEFORM1 . For simplicity, we will henceforth drop the task superscript and refer to instances as INLINEFORM2 , as instance encodings are learned in a task-agnostic fashion. Each example further consists of a claim INLINEFORM3 and INLINEFORM4 evidence pages INLINEFORM5 .
Each claim and evidence page is encoded with a BiLSTM to obtain a sentence embedding, which is the concatenation of the last state of the forward and backward reading of the sentence, i.e. INLINEFORM0 , where INLINEFORM1 is the sentence embedding.
Next, we want to combine claims and evidence sentence embeddings into joint instance representations. In the simplest case, referred to as model variant crawled_avg, we mean average the BiLSTM sentence embeddings of all evidence pages (signified by the overline) and concatenate those with the claim embeddings, i.e. DISPLAYFORM0
where INLINEFORM0 is the resulting encoding for training example INLINEFORM1 and INLINEFORM2 denotes vector concatenation. However, this has the disadvantage that all evidence pages are considered equal.
The here proposed alternative instance encoding model, crawled_ranked, which achieves the highest overall performance as discussed in Section SECREF5 , learns the compatibility between an instance's claim and each evidence page. It ranks evidence pages by their utility for the veracity prediction task, and then uses the resulting ranking to obtain a weighted combination of all claim-evidence pairs. No direct labels are available to learn the ranking of individual documents, only for the veracity of the associated claim, so the model has to learn evidence ranks implicitly.
To combine claim and evidence representations, we use the matching model proposed for the task of natural language inference by BIBREF32 and adapt it to combine an instance's claim representation with each evidence representation, i.e. DISPLAYFORM0
where INLINEFORM0 is the resulting encoding for training example INLINEFORM1 and evidence page INLINEFORM2 , INLINEFORM3 denotes vector concatenation, and INLINEFORM4 denotes the dot product.
All joint claim-evidence representations INLINEFORM0 are then projected into the binary space via a fully connected layer INLINEFORM1 , followed by a non-linear activation function INLINEFORM2 , to obtain a soft ranking of claim-evidence pairs, in practice a 10-dimensional vector, DISPLAYFORM0
where INLINEFORM0 denotes concatenation.
Scores for all labels are obtained as per ( EQREF28 ) above, with the same input instance embeddings as for the evidence ranker, i.e. INLINEFORM0 . Final predictions for all claim-evidence pairs are then obtained by taking the dot product between the label scores and binary evidence ranking scores, i.e. DISPLAYFORM0
Note that the novelty here is that, unlike for the model described in BIBREF32 , we have no direct labels for learning weights for this matching model. Rather, our model has to implicitly learn these weights for each claim-evidence pair in an end-to-end fashion given the veracity labels.
Metadata
We experiment with how useful claim metadata is, and encode the following as one-hot vectors: speaker, category, tags and linked entities. We do not encode `Reason' as it gives away the label, and do not include `Checker' as there are too many unique checkers for this information to be relevant. The claim publication date is potentially relevant, but it does not make sense to merely model this as a one-hot feature, so we leave incorporating temporal information to future work.
Since all metadata consists of individual words and phrases, a sequence encoder is not necessary, and we opt for a CNN followed by a max pooling operation as used in BIBREF3 to encode metadata for fact checking. The max-pooled metadata representations, denoted INLINEFORM0 , are then concatenated with the instance representations, e.g. for the most elaborate model, crawled_ranked, these would be concatenated with INLINEFORM1 .
Experimental Setup
The base sentence embedding model is a BiLSTM over all words in the respective sequences with randomly initialised word embeddings, following BIBREF30 . We opt for this strong baseline sentence encoding model, as opposed to engineering sentence embeddings that work particularly well for this dataset, to showcase the dataset. We would expect pre-trained contextual encoding models, e.g. ELMO BIBREF33 , ULMFit BIBREF34 , BERT BIBREF35 , to offer complementary performance gains, as has been shown for a few recent papers BIBREF36 , BIBREF37 .
For claim veracity prediction without evidence documents with the MTL with LEL model, we use the following sentence encoding variants: claim-only, which uses a BiLSTM-based sentence embedding as input, and claim-only_embavg, which uses a sentence embedding based on mean averaged word embeddings as input.
We train one multi-task model per task (i.e., one model per domain). We perform a grid search over the following hyperparameters, tuned on the respective dev set, and evaluate on the correspoding test set (final settings are underlined): word embedding size [64, 128, 256], BiLSTM hidden layer size [64, 128, 256], number of BiLSTM hidden layers [1, 2, 3], BiLSTM dropout on input and output layers [0.0, 0.1, 0.2, 0.5], word-by-word-attention for BiLSTM with window size 10 BIBREF38 [True, False], skip-connections for the BiLSTM [True, False], batch size [32, 64, 128], label embedding size [16, 32, 64]. We use ReLU as an activation function for both the BiLSTM and the CNN. For the CNN, the following hyperparameters are used: number filters [32], kernel size [32]. We train using cross-entropy loss and the RMSProp optimiser with initial learning rate of INLINEFORM0 and perform early stopping on the dev set with a patience of 3.
Results
For each domain, we compute the Micro as well as Macro F1, then mean average results over all domains. Core results with all vs. no metadata are shown in Table TABREF30 . We first experiment with different base model variants and find that label embeddings improve results, and that the best proposed models utilising multiple domains outperform single-task models (see Table TABREF36 ). This corroborates the findings of BIBREF30 . Per-domain results with the best model are shown in Table TABREF34 . Domain names are from hereon after abbreviated for brevity, see Table TABREF1 in the appendix for correspondences to full website names. Unsurprisingly, it is hard to achieve a high Macro F1 for domains with many labels, e.g. tron and snes. Further, some domains, surprisingly mostly with small numbers of instances, seem to be very easy – a perfect Micro and Macro F1 score of 1.0 is achieved on ranz, bove, buca, fani and thal. We find that for those domains, the verdict is often already revealed as part of the claim using explicit wording.
Our evidence-based claim veracity prediction models outperform claim-only veracity prediction models by a large margin. Unsurprisingly, claim-only_embavg is outperformed by claim-only. Further, crawled_ranked is our best-performing model in terms of Micro F1 and Macro F1, meaning that our model captures that not every piece of evidence is equally important, and can utilise this for veracity prediction.
We perform an ablation analysis of how metadata impacts results, shown in Table TABREF35 . Out of the different types of metadata, topic tags on their own contribute the most. This is likely because they offer highly complementary information to the claim text of evidence pages. Only using all metadata together achieves a higher Macro F1 at similar Micro F1 than using no metadata at all. To further investigate this, we split the test set into those instances for which no metadata is available vs. those for which metadata is available. We find that encoding metadata within the model hurts performance for domains where no metadata is available, but improves performance where it is. In practice, an ensemble of both types of models would be sensible, as well as exploring more involved methods of encoding metadata.
Analysis and Discussion
An analysis of labels frequently confused with one another, for the largest domain `pomt' and best-performing model crawled_ranked + meta is shown in Figure FIGREF39 . The diagonal represents when gold and predicted labels match, and the numbers signify the number of test instances. One can observe that the model struggles more to detect claims with labels `true' than those with label `false'. Generally, many confusions occur over close labels, e.g. `half-true' vs. `mostly true'. We further analyse what properties instances that are predicted correctly vs. incorrectly have, using the model crawled_ranked meta. We find that, unsurprisingly, longer claims are harder to classify correctly, and that claims with a high direct token overlap with evidence pages lead to a high evidence ranking. When it comes to frequently occurring tags and entities, very general tags such as `government-and-politics' or `tax' that do not give away much, frequently co-occur with incorrect predictions, whereas more specific tags such as `brisbane-4000' or `hong-kong' tend to co-occur with correct predictions. Similar trends are observed for bigrams. This means that the model has an easy time succeeding for instances where the claims are short, where specific topics tend to co-occur with certain veracities, and where evidence documents are highly informative. Instances with longer, more complex claims where evidence is ambiguous remain challenging.
Conclusions
We present a new, real-world fact checking dataset, currently the largest of its kind. It consists of 34,918 claims collected from 26 fact checking websites, rich metadata and 10 retrieved evidence pages per claim. We find that encoding the metadata as well evidence pages helps, and introduce a new joint model for ranking evidence pages and predicting veracity.
Acknowledgments
This research is partially supported by QUARTZ (721321, EU H2020 MSCA-ITN) and DABAI (5153-00004A, Innovation Fund Denmark).
Appendix
Summary statistics for claim collection. “Domain” indicates the domain name used for the veracity prediction experiments, “–” indicates that the website was not used due to missing or insufficient claim labels, see Section SECREF12 .
Comparison of fact checking datasets. Doc = all doc types (including tweets, replies, etc.). SoA perform indicates state-of-the-art performance. INLINEFORM0 indicates that claims are not naturally occuring: BIBREF6 use events as claims; BIBREF7 use DBPedia tiples as claims; BIBREF9 use tweets as claims; and BIBREF2 rewrite sentences in Wikipedia as claims. INLINEFORM1 denotes that the SoA performance is from other papers. Best performance for BIBREF3 is from BIBREF40 ; BIBREF2 from BIBREF28 ; BIBREF10 from BIBREF42 in English, BIBREF12 from BIBREF26 ; and BIBREF13 from BIBREF39 .

Question: What were the baselines?
Answer: a BiLSTM over all words in the respective sequences with randomly initialised word embeddings, following BIBREF30

Question: What metadata is included?
Answer: besides claim, label and claim url, it also includes a claim ID, reason, category, speaker, checker, tags, claim entities, article title, publish data and claim date

Question: How many expert journalists were there?
Answer: Unanswerable