# Collecting a Large Scale Dataset for Classifying Fake News Tweets Using Weak Supervision
## 3 Datasets
### Large-Scale Training Dataset  
We create our training dataset by first collecting trustworthy and untrustworthy sources. Then, for each of the sources, we collect Tweets using the Twitter API. Each tweet from a trustworthy source is labeled as real news, each tweet from an untrustworthy source is labeled as fake news. While this labeling can be done automatically at large scale, it is far from perfect. Most untrustworthy sources spread a mix of fake and real news. The reverse (i.e., a trustworthy source spreading fake news, e.g., by accident) may also occur, but we assume that this case very rare, and hence do not consider it any further. For collecting fake news sources, we use lists from different Web pages:  
* [[http://mashable.com/2012/04/20/twitter-parodies-news/#IdNx6](http://mashable.com/2012/04/20/twitter-parodies-news/#IdNx6) sIG.Zqm]([http://mashable.com/2012/04/20/twitter-parodies-news/#IdNx6](http://mashable.com/2012/04/20/twitter-parodies-news/#IdNx6) sIG.Zqm) (accessed on 24 February 2017)
* [[https://www.dailydot.com/layer8/fake-news-sites-list-facebook/](https://www.dailydot.com/layer8/fake-news-sites-list-facebook/)]([https://www.dailydot.com/layer8/fake-news-sites-list-facebook/](https://www.dailydot.com/layer8/fake-news-sites-list-facebook/)) (accessed on 24 February 2017)
* [[https://www.cbsnews.com/pictures/dont-get-fooled-by-these-fake-news-sites/](https://www.cbsnews.com/pictures/dont-get-fooled-by-these-fake-news-sites/)]([https://www.cbsnews.com/pictures/dont-get-fooled-by-these-fake-news-sites/](https://www.cbsnews.com/pictures/dont-get-fooled-by-these-fake-news-sites/)) (accessed on 24 February 2017)
* [[http://fakenewswatch.com/](http://fakenewswatch.com/)]([http://fakenewswatch.com/](http://fakenewswatch.com/)) (accessed on 24 February 2017)
* [[https://www.snopes.com/2016/01/14/fake-news-sites/](https://www.snopes.com/2016/01/14/fake-news-sites/)]([https://www.snopes.com/2016/01/14/fake-news-sites/](https://www.snopes.com/2016/01/14/fake-news-sites/)) (accessed on 24 February 2017)
* [[https://www.thoughtc.com/guide-to-fake-news-websites-3298824](https://www.thoughtc.com/guide-to-fake-news-websites-3298824)]([https://www.thoughtc.com/guide-to-fake-news-websites-3298824](https://www.thoughtc.com/guide-to-fake-news-websites-3298824)) (accessed on 24 February 2017)
* [[https://newrepublic.com/article/118013/satire-news-websites-are-cashing-gullible-outraged-readers](https://newrepublic.com/article/118013/satire-news-websites-are-cashing-gullible-outraged-readers)]([https://newrepublic.com/article/118013/satire-news-websites-are-cashing-gullible-outraged-readers](https://newrepublic.com/article/118013/satire-news-websites-are-cashing-gullible-outraged-readers)) (accessed on 24 February 2017)
* [[http://www.opensources.co/](http://www.opensources.co/)]([http://www.opensources.co/](http://www.opensources.co/)) (accessed on 24 February 2017). In total, we collected 65 sources of fake news. For collecting trustworthy news sources, we used a copy of the recently shut down DMOZ catalog ([[http://dmoztools.net/](http://dmoztools.net/)]([http://dmoztools.net/](http://dmoztools.net/)) (accessed on 25 February 2017), as well as those news sites listed as trustworthy in _opensources_, and filtered the sites to those which feature an active Twitter channel. In order to arrive at a balanced dataset, we collected 46 trustworthy news sites. That number is incidentally chosen lower than that of fake news sources, since we could collect more tweets from the trustworthy sites. In the next step, we used the Twitter API ([[https://developer.twitter.com/en/docs](https://developer.twitter.com/en/docs)]([https://developer.twitter.com/en/docs](https://developer.twitter.com/en/docs)) (accessed on 26 April 2021) to retrieve tweets for the sources. The dataset was collected between February and June 2017.