r/Fakeddit: A New Multimodal Benchmark Dataset for Fine-grained Fake News Detection

Abstract: Fake news has altered society in negative ways as evidenced in politics and culture. It has adversely affected both online social network systems as well as offline communities and conversations. Using automatic fake news detection algorithms is an efficient way to combat the rampant dissemination of fake news. However, using an effective dataset has been a problem for fake news research and detection model development. In this paper, we present Fakeddit, a novel dataset consisting of about 800,000 samples from multiple categories of fake news. Each sample is labeled according to 2-way, 3-way, and 5-way classification categories. Prior fake news datasets do not provide multimodal text and image data, metadata, comment data, and fine-grained fake news categorization at this scale and breadth. We construct hybrid text+image models and perform extensive experiments for multiple variations of classification.

Within our progressively digitized society, the spread of fake news and misinformation has enlarged, leading to many problems such as an increasingly politically divisive climate. The dissemination and consequences of fake news are exacerbating partly due to the rise of popular social media applications with inadequate fact-checking or third-party filtering, enabling any individual to broadcast fake news easily and at a large scale BIBREF0. Though steps have been taken to detect and eliminate fake news, it still poses a dire threat to society BIBREF1. As such, research in the area of fake news detection is essential.
To build any machine learning model, one must obtain good training data for the specified task. In the realm of fake news detection, there are several existing published datasets. However, they have several limitations: limited size, modality, and/or granularity. Though fake news may immediately be thought of as taking the form of text, it can appear in other mediums such as images. As such, it is important that standard fake news detection systems detect all types of fake news and not just text data. Our dataset will expand fake news research into the multimodal space and allow researchers to develop stronger fake news detection systems.
Our contributions to the study of fake news detection are:
We create a large-scale multimodal fake news dataset consisting of around 800,000 samples containing text, image, metadata, and comments data from a highly diverse set of resources.
Each data sample consists of multiple labels, allowing users to utilize the dataset for 2-way, 3-way, and 5-way classification. This enables both high-level and fine-grained fake news classification.
We evaluate our dataset through text, image, and text+image modes with a neural network architecture that integrates both the image and text data. We run experiments for several types of models, providing a comprehensive overview of classification results.
A variety of datasets for fake news detection have been published in recent years. These are listed in Table TABREF1, along with their specific characteristics. When comparing these datasets, a few trends can be seen. Most of the datasets are small in size, which can be ineffective for current machine learning models that require large quantities of training data. Only four contain over half a million samples, with CREDBANK and FakeNewsCorpus being the largest with millions of samples BIBREF2. In addition, many of the datasets separate their data into a small number of classes, such as fake vs. true. However, fake news can be categorized into many different types BIBREF3. Datasets such as NELA-GT-2018, LIAR, and FakeNewsCorpus provide more fine-grained labels BIBREF4, BIBREF5. While some datasets include data from a variety of categories BIBREF6, BIBREF7, many contain data from specific areas, such as politics and celebrity gossip BIBREF8, BIBREF9, BIBREF10, BIBREF11. These data samples may contain limited styles of writing due to this categorization. Finally, most of the existing fake news datasets collect only text data, which is not the only mode that fake news can appear in. Datasets such as image-verification-corpus, Image Manipulation, BUZZFEEDNEWS, and BUZZFACE can be utilized for fake image detection, but contain small sample sizesBIBREF12, BIBREF13, BIBREF14. It can be seen from the table that compared to other existing datasets, Fakeddit contains a large quantity of data, while also annotating for three different types of classification labels (2-way, 3-way, and 5-way) and comparing both text and image data.
Many fake news datasets are crowdsourced or handpicked from a select few sources that are narrow in size, modality, and/or diversity. In order to expand and evolve fake news research, researchers need to have access to a dataset that exceed these current dataset limitations. Thus, we propose Fakeddit, a novel dataset consisting of a large quantity of text+image samples coming from large diverse sources.
We sourced our dataset from Reddit, a social news and discussion website where users can post submissions on various subreddits. Each subreddit has its own theme like `nottheonion', where people post seemingly false stories that are surprisingly true. Active Reddit users are able to upvote, downvote, and comment on the submission.
Submissions were collected with the pushshift.io API. Each subreddit has moderators that ensure submissions pertain to the subreddit theme and remove posts that violate any rules, indirectly helping us obtain reliable data. To further ensure that our data is credible, we filtered out any submissions that had a score of less than 1. Fakeddit consists of 825,100 total submissions from 21 different subreddits. We gathered the submission title and image, comments made by users who engaged with the submission, as well as other submission metadata including the score, the username of the author, subreddit source, sourced domain, number of comments, and up-vote to down-vote ratio. 63% of the samples contains both text and images, while the rest contain only text. For our experiments, we utilize these multimodal samples. The samples span over many years and are posted on highly active and popular pages by tens of thousands of diverse individual users from across the world. Because of the variety of the chosen subreddits, our data also varies in its content, ranging from political news stories to simple everyday posts by Reddit users.
We provide three labels for each sample, allowing us to train for 2-way, 3-way, and 5-way classification. Having this hierarchy of labels will enable researchers to train for fake news detection at a high level or a more fine-grained one. The 2-way classification determines whether a sample is fake or true. The 3-way classification determines whether a sample is completely true, the sample is fake news with true text (text that is true in the real world), or the sample is fake news with false text. Our final 5-way classification was created to categorize different types of fake news rather than just doing a simple binary or trinary classification. This can help in pinpointing the degree and variation of fake news for applications that require this type of fine-grained detection. The first label is true and the other four are defined within the seven types of fake news BIBREF3. We provide examples from each class for 5-way classification in Figure SECREF3. The 5-way classification labels are explained below:
True: True content is accurate in accordance with fact. Eight of the subreddits fall into this category, such as usnews and mildlyinteresting. The former consists of posts from various news sites. The latter encompasses real photos with accurate captions. The other subreddits include photoshopbattles, nottheonion, neutralnews, pic, usanews, and upliftingnews.
Satire/Parody: This category consists of content that spins true contemporary content with a satirical tone or information that makes it false. One of the four subreddits that make up this label is theonion, with headlines such as “Man Lowers Carbon Footprint By Bringing Reusable Bags Every Time He Buys Gas". Other satirical subreddits are fakealbumcovers, satire, and waterfordwhispersnews.
Misleading Content: This category consists of information that is intentionally manipulated to fool the audience. Our dataset contains three subreddits in this category: propagandaposters, fakefacts, and savedyouaclick.
Imposter Content: This category contains the subredditsimulator subreddit, which contains bot-generated content and is trained on a large number of other subreddits. It also includes subsimulatorgpt2.
False Connection: Submission images in this category do not accurately support their text descriptions. We have four subreddits with this label, containing posts of images with captions that do not relate to the true meaning of the image. These include misleadingthumbnails, confusing_perspective, pareidolia, and fakehistoryporn.
Multiple methods were employed for text and image feature extraction. We used InferSent and BERT to generate text embeddings for the title of the Reddit submissions BIBREF15, BIBREF16. VGG16, EfficientNet, and ResNet50 were utilized to extract the features of the Reddit submission thumbnails BIBREF17, BIBREF18, BIBREF19.
We used the InferSent model because it performs very well as a universal sentence embeddings generator. For this model, we loaded a vocabulary of 1 million of the most common words in English and used fastText as opposed to ELMO embeddings because fastText can perform relatively well for rare words and words that do not appear in the vocabulary BIBREF20, BIBREF21. We obtained encoded sentence features of length 4096 for each submission title using InferSent.
The BERT model achieves state-of-the-art results on many classification tasks, including Q&A and named entity recognition. To obtain fixed-length BERT embedding vectors, we used the bert-as-service tool, which maps variable-length text/sentences into a 768 element array for each Reddit submission title BIBREF22. For our experiments, we utilized the pretrained BERT-Large, Uncased model.
We utilized VGG16, ResNet50, and EfficientNet models for encoding images. VGG16 and ResNet50 are widely used by many researchers, while EfficientNet is a relatively newer model. For EfficientNet, we used the smallest variation: B0. For all three image models, we preloaded weights of models trained on ImageNet and included the top layer and used its penultimate layer for feature extraction.
For our experiments, we excluded submissions that did not have an image associated with them and solely used submission image and title data. We performed 2-way, 3-way, and 5-way classification for each of the three types of inputs: image only, text only, and multimodal (text and image).
Before training, we performed preprocessing on the images and text. We constrained sizes of the images to 224x224. From the text, we removed all punctuation, numbers, and revealing words such as “PsBattle” that automatically reveal the subreddit source. For the savedyouaclick subreddit, we removed text following the “” character and classified it as misleading content.
When combining the features in multimodal classification, we first condensed the features into 256-element vectors through a trainable dense layer and then merged them through four different methods: add, concatenate, maximum, average. These features were then passed through a fully connected softmax predictor.
The results are shown in Tables TABREF17 and SECREF3. We found that the multimodal features performed the best, followed by text-only, and image-only in all instances. Thus, having both image and text improves fake news detection. For image and multimodal classification, ResNet50 performed the best followed by VGG16 and EfficientNet. In addition, BERT generally achieved better results than InferSent for multimodal classification. However, for text-only classification InferSent outperformed BERT. The “maximum” method to merge image and text features yielded the highest accuracy, followed by average, concatenate, and add. Overall, the multimodal model that combined BERT text features and ResNet50 image features through the maximum method performed most optimally.
In this paper, we presented a novel dataset for fake news research, Fakeddit. Compared to previous datasets, Fakeddit provides a large quantity of text+image samples with multiple labels for various levels of fine-grained classification. We created detection models that incorporate both modalities of data and conducted experiments, showing that there is still room for improvement in fake news detection. Although we do not utilize submission metadata and comments made by users on the submissions, we anticipate that these features will be useful for further research. We hope that our dataset can be used to advance efforts to combat the ever growing rampant spread of misinformation.
We would like to acknowledge Facebook for the Online Safety Benchmark Award. The authors are solely responsible for the contents of the paper, and the opinions expressed in this publication do not reflect those of the funding agencies.

Figure and table captions:
Table 1: Comparison of various fake news detection datasets. IA: Individual assessments.
Figure 1: Dataset examples with 5-way classification labels.
Table 2: Results on fake news detection for 2, 3, and 5-way classification with combination method of maximum.
Table 3: Results on different multi-modal combinations for BERT + ResNet50

Questions about this paper:
What classification tasks do they experiment on?

Answers:
They experiment on 3 types of classification tasks with different inputs:
2-way: True/False
3-way: True/False news with text true in real world/False news with false text
5-way: True/Parody/Missleading/Imposter/False Connection