Introduction
Recent years have seen the proliferation of deceptive information online. With the increasing necessity to validate the information from the Internet, automatic fact checking has emerged as an important research topic. It is at the core of multiple applications, e.g., discovery of fake news, rumor detection in social media, information verification in question answering systems, detection of information manipulation agents, and assistive technologies for investigative journalism. At the same time, it touches many aspects, such as credibility of users and sources, information veracity, information verification, and linguistic aspects of deceptive language.
In this paper, we present an approach to fact-checking with the following design principles: (i) generality, (ii) robustness, (iii) simplicity, (iv) reusability, and (v) strong machine learning modeling. Indeed, the system makes very few assumptions about the task, and looks for supportive information directly on the Web. Our system works fully automatically. It does not use any heavy feature engineering and can be easily used in combination with task-specific approaches as well, as a core subsystem. Finally, it combines the representational strength of recurrent neural networks with kernel-based classification.
The system starts with a claim to verify. First, we automatically convert the claim into a query, which we execute against a search engine in order to obtain a list of potentially relevant documents. Then, we take both the snippets and the most relevant sentences in the full text of these Web documents, and we compare them to the claim. The features we use are dense representations of the claim, of the snippets and of related sentences from the Web pages, which we automatically train for the task using Long Short-Term Memory networks (LSTMs). We also use the final hidden layer of the neural network as a task-specific embedding of the claim, together with the Web evidence. We feed all these representations as features, together with pairwise similarities, into a Support Vector Machine (SVM) classifier using an RBF kernel to classify the claim as True or False.
Figure FIGREF1 presents a real example from one of the datasets we experiment with. The left-hand side of the figure contains a True example, while the right-hand side shows a False one. We show the original claims from snopes.com, the query generated by our system, and the information retrieved from the Web (most relevant snippet and text selection from the web page). The veracity of the claim can be inferred from the textual information.
Our contributions can be summarized as follows:
The remainder of this paper is organized as follows. Section SECREF2 introduces our method for fact checking claims using external sources. Section SECREF3 presents our experiments and discusses the results. Section SECREF4 describes an application of our approach to a different dataset and a slightly different task: fact checking in community question answering forums. Section SECREF5 presents related work. Finally, Section SECREF6 concludes and suggests some possible directions for future work.
The Fact-Checking System
Given a claim, our system searches for support information on the Web in order to verify whether the claim is likely to be true. The three steps in this process are (i) external support retrieval, (ii) text representation, and (iii) veracity prediction.
External Support Retrieval
This step consists of generating a query out of the claim and querying a search engine (here, we experiment with Google and Bing) in order to retrieve supporting documents. Rather than querying the search engine with the full claim (as on average, a claim is two sentences long), we generate a shorter query following the lessons highlighted in BIBREF0 .
As we aim to develop a general-purpose fact checking system, we use an approach for query generation that does not incorporate any features that are specific to claim verification (e.g., no temporal indicators).
We rank the words by means of tf-idf. We compute the idf values on a 2015 Wikipedia dump and the English Gigaword. BIBREF0 suggested that a good way to perform high-quality search is to only consider the verbs, the nouns and the adjectives in the claim; thus, we exclude all words in the claim that belong to other parts of speech. Moreover, claims often contain named entities (e.g., names of persons, locations, and organizations); hence, we augment the initial query with all the named entities from the claim's text. We use IBM's AlchemyAPI to identify named entities. Ultimately, we generate queries of 5–10 tokens, which we execute against a search engine. We then collect the snippets and the URLs in the results, skipping any result that points to a domain that is considered unreliable. Finally, if our query has returned no results, we iteratively relax it by dropping the final tokens one at a time.
Text Representation
Next, we build the representation of a claim and the corresponding snippets and Web pages. First, we calculate three similarities (a) between the claim and a snippet, or (b) between the claim and a Web page: (i) cosine with tf-idf, (ii) cosine over embeddings, and (iii) containment BIBREF1 . We calculate the embedding of a text as the average of the embeddings of its words; for this, we use pre-trained embeddings from GloVe BIBREF2 . Moreover, as a Web page can be long, we first split it into a set of rolling sentence triplets, then we calculate the similarities between the claim and each triplet, and we take the highest scoring triplet. Finally, as we have up to ten hits from the search engine, we take the maximum and also the average of the three similarities over the snippets and over the Web pages.
We further use as features the embeddings of the claim, of the best-scoring snippet, and of the best-scoring sentence triplet from a Web page. We calculate these embeddings (i) as the average of the embeddings of the words in the text, and also (ii) using LSTM encodings, which we train for the task as part of a deep neural network (NN). We also use a task-specific embedding of the claim together with all the above evidence about it, which comes from the last hidden layer of the NN.
Veracity Prediction
Next, we build classifiers: neural network (NN), support vector machines (SVM), and a combination thereof (SVM+NN).
The architecture of our NN is shown on top of Figure FIGREF7 . We have five LSTM sub-networks, one for each of the text sources from two search engines: Claim, Google Web page, Google snippet, Bing Web page, and Bing snippet. The claim is fed into the neural network as-is. As we can have multiple snippets, we only use the best-matching one as described above. Similarly, we only use a single best-matching triple of consecutive sentences from a Web page. We further feed the network with the similarity features described above.
All these vectors are concatenated and fully connected to a much more compact hidden layer that captures the task-specific embeddings. This layer is connected to a softmax output unit to classify the claim as true or false. The bottom of Figure FIGREF7 represents the generic architecture of each of the LSTM components. The input text is transformed into a sequence of word embeddings, which is then passed to the bidirectional LSTM layer to obtain a representation for the full sequence.
Our second classifier is an SVM with an RBF kernel. The input is the same as for the NN: word embeddings and similarities. However, the word embeddings this time are calculated by averaging rather than using a bi-LSTM.
Finally, we combine the SVM with the NN by augmenting the input to the SVM with the values of the units in the hidden layer. This represents a task-specific embedding of the input example, and in our experiments it turned out to be quite helpful. Unlike in the SVM only model, this time we use the bi-LSTM embeddings as an input to the SVM. Ultimately, this yields a combination of deep learning and task-specific embeddings with RBF kernels.
Dataset
We used part of the rumor detection dataset created by BIBREF3 . While they analyzed a claim based on a set of potentially related tweets, we focus on the claim itself and on the use of supporting information from the Web.
The dataset consists of 992 sets of tweets, 778 of which are generated starting from a claim on snopes.com, which ma2016detecting converted into a query. Another 214 sets of tweets are tweet clusters created by other researchers BIBREF4 , BIBREF5 with no claim behind them. ma2016detecting ignored the claim and did not release it as part of their dataset. We managed to find the original claim for 761 out of the 778 snopes.com-based clusters.
Our final dataset consists of 761 claims from snopes.com, which span various domains including politics, local news, and fun facts. Each of the claims is labeled as factually true (34%) or as a false rumor (66%). We further split the data into 509 for training, 132 for development, and 120 for testing. As the original split for the dataset was not publicly available, and as we only used a subset of their data, we had to make a new training and testing split. Note that we ignored the tweets, as we wanted to focus on a complementary source of information: the Web. Moreover, ma2016detecting used manual queries, while we use a fully automatic method. Finally, we augmented the dataset with Web-retrieved snippets, Web pages, and sentence triplets from Web pages.
Experimental Setup
We tuned the architecture (i.e., the number of layers and their size) and the hyper-parameters of the neural network on the development dataset. The best configuration uses a bidirectional LSTM with 25 units. It further uses a RMSprop optimizer with 0.001 initial learning rate, L2 regularization with INLINEFORM0 =0.1, and 0.5 dropout after the LSTM layers. The size of the hidden layer is 60 with tanh activations. We use a batch of 32 and we train for 400 epochs.
For the SVM model, we merged the development and the training dataset, and we then ran a 5-fold cross-validation with grid-search, looking for the best kernel and its parameters. We ended up selecting an RBF kernel with INLINEFORM0 and INLINEFORM1 0.01.
Evaluation Metrics
The evaluation metrics we use are P (precision), R (recall), and F INLINEFORM0 , which we calculate with respect to the false and to the true claims. We further report AvgR (macro-average recall), AvgF INLINEFORM1 (macro-average F INLINEFORM2 ), and Acc (accuracy).
Results
Table TABREF14 shows the results on the test dataset. We can see that both the NN and the SVM models improve over the majority class baseline (all false rumors) by a sizable margin. Moreover, the NN consistently outperforms the SVM by a margin on all measures. Yet, adding the task-specific embeddings from the NN as features of the SVM yields overall improvements over both the SVM and the NN in terms of avgR, avgF INLINEFORM0 , and accuracy. We can see that both the SVM and the NN overpredict the majority class (false claims); however, the combined SVM+NN model is quite balanced between the two classes.
Table TABREF22 compares the performance of the SVM with and without task-specific embeddings from the NN, when training on Web pages vs. snippets, returned by Google vs. Bing vs. both. The NN embeddings consistently help the SVM in all cases. Moreover, while the baseline SVM using snippets is slightly better than when using Web pages, there is almost no difference between snippets vs. Web pages when NN embeddings are added to the basic SVM. Finally, gathering external support from either Google or Bing makes practically no difference, and using the results from both together does not yield much further improvement. Thus, (i) the search engines already do a good job at generating relevant snippets, and one does not need to go and download the full Web pages, and (ii) the choice of a given search engine is not an important factor. These are good news for the practicality of our approach.
Unfortunately, direct comparison with respect to BIBREF3 is not possible. First, we only use a subset of their examples: 761 out of 993 (see Section SECREF17 ), and we also have a different class distribution. More importantly, they have a very different formulation of the task: for them, the claim is not available as input (in fact, there has never been a claim for 21% of their examples); rather an example consists of a set of tweets retrieved using manually written queries.
In contrast, our system is fully automatic and does not use tweets at all. Furthermore, their most important information source is the change in tweets volume over time, which we cannot use. Still, our results are competitive to theirs when they do not use temporal features.
To put the results in perspective, we can further try to make an indirect comparison to the very recent paper by BIBREF6 . They also present a model to classify true vs. false claims extracted from snopes.com, by using information extracted from the Web. Their formulation of the task is the same as ours, but our corpora and label distributions are not the same, which makes a direct comparison impossible. Still, we can see that regarding overall classification accuracy they improve a baseline from 73.7% to 84.02% with their best model, i.e., a 39.2% relative error reduction. In our case, we go from 66.7% to 80.0%, i.e., an almost identical 39.9% error reduction. These results are very encouraging, especially given the fact that our model is much simpler than theirs regarding the sources of information used (they model the stance of the text, the reliability of the sources, the language style of the articles, and the temporal footprint).
Application to cQA
Next, we tested the generality of our approach by applying it to a different setup: fact-checking the answers in community question answering (cQA) forums. As this is a new problem, for which no dataset exists, we created one. We augmented with factuality annotations the cQA dataset from SemEval-2016 Task 3 (CQA-QA-2016) BIBREF7 . Overall, we annotated 249 question–answer, or INLINEFORM0 - INLINEFORM1 , pairs (from 71 threads): 128 factually true and 121 factually false answers.
Each question in CQA-QA-2016 has a subject, a body, and meta information: ID, category (e.g., Education, and Moving to Qatar), date and time of posting, user name and ID. We selected only the factual questions such as “What is Ooredoo customer service number?”, thus filtering out all (i) socializing, e.g., “What was your first car?”, (ii) requests for opinion/advice/guidance, e.g., “Which is the best bank around??”, and (iii) questions containing multiple sub-questions, e.g., “Is there a land route from Doha to Abudhabi. If yes; how is the road and how long is the journey?”
Next, we annotated for veracity the answers to the retained questions. Note that in CQA-QA-2016, each answer has a subject, a body, meta information (answer ID, user name and ID), and a judgment about how well it addresses the question of its thread: Good vs. Potentially Useful vs. Bad . We only annotated the Good answers. We further discarded answers whose factuality was very time-sensitive (e.g., “It is Friday tomorrow.”, “It was raining last week.”), or for which the annotators were unsure.
We targeted very high quality, and thus we did not use crowdsourcing for the annotation, as pilot annotations showed that the task was very difficult and that it was not possible to guarantee that Turkers would do all the necessary verification, e.g., gathering evidence from trusted sources. Instead, all examples were first annotated independently by four annotators, and then each example was discussed in detail to come up with a final label. We ended up with 249 Good answers to 71 different questions, which we annotated for factuality: 128 Positive and 121 Negative examples. See Table TABREF26 for details.
We further split our dataset into 185 INLINEFORM0 – INLINEFORM1 pairs for training, 31 for development, and 32 for testing, preserving the general positive:negative ratio, and making sure that the questions for the INLINEFORM2 – INLINEFORM3 pairs did not overlap between the splits. Figure FIGREF23 presents an excerpt of an example from the dataset, with one question and three answers selected from a longer thread. Answer INLINEFORM4 contains false information, while INLINEFORM5 and INLINEFORM6 are true, as can be checked on an official governmental website.
We had to fit our system for this problem, as here we do not have claims, but a question and an answer. So, we constructed the query from the concatenation of INLINEFORM0 and INLINEFORM1 . Moreover, as Google and Bing performed similarly, we only report results using Google. We limited our run to snippets only, as we have found them rich enough above (see Section SECREF3 ). Also, we had a list of reputed and Qatar-related sources for the domain, and we limited our results to these sources only. This time, we had more options to calculate similarities compared to the rumors dataset: we can compare against INLINEFORM2 , INLINEFORM3 , and INLINEFORM4 – INLINEFORM5 ; we chose to go with the latter. For the LSTM representations, we use both the question and the answer.
Table TABREF27 shows the results on the cQA dataset. Once again, our models outperformed all baselines by a margin. This time, the predictions of all models are balanced between the two classes, which is probably due to the dataset being well balanced in general. The SVM model performs better than the NN by itself. This is due to the fact that the cQA dataset is significantly smaller than the rumor detection one. Thus, the neural network could not be trained effectively by itself. Nevertheless, the task-specific representations were useful and combining them with the SVM model yielded consistent improvements on all the measures once again.
Related Work
Journalists, online users, and researchers are well aware of the proliferation of false information on the Web, and topics such as information credibility and fact checking are becoming increasingly important as research directions. For example, there was a recent 2016 special issue of the ACM Transactions on Information Systems journal on Trust and Veracity of Information in Social Media BIBREF9 , there was a SemEval-2017 shared task on Rumor Detection BIBREF10 , and there is an upcoming lab at CLEF-2018 on Automatic Identification and Verification of Claims in Political Debates BIBREF11 .
The credibility of contents on the Web has been questioned by researches for a long time. While in the early days the main research focus was on online news portals BIBREF12 , BIBREF13 , BIBREF14 , the interest has eventually shifted towards social media BIBREF4 , BIBREF15 , BIBREF6 , BIBREF16 , which are abundant in sophisticated malicious users such as opinion manipulation trolls, paid BIBREF17 or just perceived BIBREF18 , BIBREF19 , sockpuppets BIBREF20 , Internet water army BIBREF21 , and seminar users BIBREF22 .
For instance, BIBREF23 studied the credibility of Twitter accounts (as opposed to tweet posts), and found that both the topical content of information sources and social network structure affect source credibility. Other work, closer to ours, aims at addressing credibility assessment of rumors on Twitter as a problem of finding false information about a newsworthy event BIBREF4 . This model considers user reputation, writing style, and various time-based features, among others.
Other efforts have focused on news communities. For example, several truth discovery algorithms are combined in an ensemble method for veracity estimation in the VERA system BIBREF24 . They proposed a platform for end-to-end truth discovery from the Web: extracting unstructured information from multiple sources, combining information about single claims, running an ensemble of algorithms, and visualizing and explaining the results. They also explore two different real-world application scenarios for their system: fact checking for crisis situations and evaluation of trustworthiness of a rumor. However, the input to their model is structured data, while here we are interested in unstructured text as input.
Similarly, the task defined by BIBREF25 combines three objectives: assessing the credibility of a set of posted articles, estimating the trustworthiness of sources, and predicting user's expertise. They considered a manifold of features characterizing language, topics and Web-specific statistics (e.g., review ratings) on top of a continuous conditional random fields model. In follow-up work, BIBREF26 proposed a model to support or refute claims from snopes.com and Wikipedia by considering supporting information gathered from the Web. They used the same task formulation for claims as we do, but different datasets. In yet another follow-up work, Popat:2017:TLE:3041021.3055133 proposed a complex model that considers stance, source reliability, language style, and temporal information.
Our approach to fact checking is related: we verify facts on the Web. However, we use a much simpler and feature-light system, and a different machine learning model. Yet, our model performs very similarly to this latter work (even though a direct comparison is not possible as the datasets differ), which is a remarkable achievement given the fact that we consider less knowledge sources, we have a conceptually simpler model, and we have six times less training data than Popat:2017:TLE:3041021.3055133.
Another important research direction is on using tweets and temporal information for checking the factuality of rumors. For example, BIBREF27 used temporal patterns of rumor dynamics to detect false rumors and to predict their frequency. BIBREF27 focused on detecting false rumors in Twitter using time series. They used the change of social context features over a rumor's life cycle in order to detect rumors at an early stage after they were broadcast.
A more general approach for detecting rumors is explored by BIBREF3 , who used recurrent neural networks to learn hidden representations that capture the variation of contextual information of relevant posts over time. Unlike this work, we do not use microblogs, but we query the Web directly in search for evidence. Again, while direct comparison to the work of BIBREF3 is not possible, due to differences in dataset and task formulation, we can say that our framework is competitive when temporal information is not used. More importantly, our approach is orthogonal to theirs in terms of information sources used, and thus, we believe there is potential in combining the two approaches.
In the context of question answering, there has been work on assessing the credibility of an answer, e.g., based on intrinsic information BIBREF28 , i.e., without any external resources. In this case, the reliability of an answer is measured by computing the divergence between language models of the question and of the answer. The spawn of community-based question answering Websites also allowed for the use of other kinds of information. Click counts, link analysis (e.g., PageRank), and user votes have been used to assess the quality of a posted answer BIBREF29 , BIBREF30 , BIBREF31 . Nevertheless, these studies address the answers' credibility level just marginally.
Efforts to determine the credibility of an answer in order to assess its overall quality required the inclusion of content-based information BIBREF32 , e.g., verbs and adjectives such as suppose and probably, which cast doubt on the answer. Similarly, BIBREF33 used source credibility (e.g., does the document come from a government Website?), sentiment analysis, and answer contradiction compared to other related answers.
Overall, credibility assessment for question answering has been mostly modeled at the feature level, with the goal of assessing the quality of the answers. A notable exception is the work of BIBREF34 , where credibility is treated as a task of its own right. Yet, note that credibility is different from factuality (our focus here) as the former is a subjective perception about whether a statement is credible, rather than verifying it as true or false as a matter of fact; still, these notions are often wrongly mixed in the literature. To the best of our knowledge, no previous work has targeted fact-checking of answers in the context of community Question Answering by gathering external support.
Conclusions and Future Work
We have presented and evaluated a general-purpose method for fact checking that relies on retrieving supporting information from the Web and comparing it to the claim using machine learning. Our method is lightweight in terms of features and can be very efficient because it shows good performance by only using the snippets provided by the search engines. The combination of the representational power of neural networks with the classification of kernel-based methods has proven to be crucial for making balanced predictions and obtaining good results. Overall, the strong performance of our model across two different fact-checking tasks confirms its generality and potential applicability for different domains and for different fact-checking task formulations.
In future work, we plan to test the generality of our approach by applying it to these and other datasets in combination with complementary methods, e.g., those focusing on microblogs and temporal information in Twitter to make predictions about rumors BIBREF27 , BIBREF3 . We also want to explore the possibility of providing justifications for our predictions, and we plan to integrate our method into a real-world application.
Acknowledgments
This research was performed by the Arabic Language Technologies group at Qatar Computing Research Institute, HBKU, within the Interactive sYstems for Answer Search project (Iyas).

Question: Do they report results only on English data?
Answer: Yes

Question: Does this system improve on the SOTA?
Answer: Unanswerable

Question: How are the potentially relevant text fragments identified?
Answer:  Generate a query out of the claim and querying a search engine, rank the words by means of TF-IDF, use IBM's AlchemyAPI to identify named entities, generate queries of 5–10 tokens, which execute against a search engine, and collect the snippets and the URLs in the results, skipping any result that points to a domain that is considered unreliable.

Question: What algorithm and embedding dimensions are used to build the task-specific embeddings?
Answer:  task-specific embedding of the claim together with all the above evidence about it, which comes from the last hidden layer of the NN

Question: What data is used to build the task-specific embeddings?
Answer: embedding of the claim, Web evidence