# Collecting a Large Scale Dataset for Classifying Fake News Tweets Using Weak Supervision
## 4 Approach
### Sentiment Features  
The polarity of a tweet can be assessed in terms of positivity, neutrality, or negativity. Additionally, a lemmatization was performed to get a base form of the words. The meaning of words depend on the semantic of the sentence, and thus the context. Hence, the first step was to identify this meaning. In NLP, this task is called _Word Sense Disambiguation_. One of the most popular algorithms for the disambiguation of word senses is the _Lesk_ algorithm. This algorithm uses word dictionaries with explanations for each word's sense. Words from the sense definition are then crosschecked for overlaps with words in the sense definition of the other words in context. The sense with the highest overlap is then chosen for each word in context [38]. In the NLTK implementation, which was used in this paper, _WordNet_ is used as a dictionary. In that implementation, part-of-speech tags can be included for disambiguation as well. To determine the sentiment of a tweet, _SentiWordNet_ was then  
Figure 6: F1 scores using different numbers of topics for LDA with different learning algorithms (in blue), and the results for HDP (in red). used to get a positive and negative score for every single word in the context. _SentiWordNet_ is a publicly available lexicon with the annotation of all synsets in _WordNet_ according to their positivity, negativity, and neutrality [39]. All nouns, verbs, adjectives, and adverbs were evaluated, and the difference of positive and negative sentiment was calculated as a sentiment score for each word. On Twitter, sentiment is not only expressed in words, but also by using emoticons. At this point, emoticons have not yet been considered. Therefore, an emoticon dictionary for Unicode emojis [40], as well as one for ASCII emoticons [41], were used here to look up the sentiment of the emoticons. After all tokens in the sentence have been scored, the tweet's score was calculated by summing up the individual scores. Through summing up the sentiments, the sum's lower and upper bound is limited by the number of words in a tweet, and thus the 140 character limitation. In a tweet with 20 sentiment words of which all are positive, i.e., each word has a score of +1, the sentiment sum would be 20. If in another tweet there is only one positive word, the sum would be much lower. It is clear that in this example the tweet with more sentiment words also gets higher scores even if both are completely positive. Therefore the sum was divided by the number of sentiment words in the tweet to get the average sentiment score. Sentiment is usually given in a range of \(-1\) to 1. However, since many machine learning classifiers operate better on features in a \([0,1]\) interval, the sentiment score was normalized to a range from 0 to 1, i.e., tweets with a score of 0.5 are completely neutral. In addition to the previous approach, an implementation from the _TextBlob_ ([[http://textblob.readthedocs.io/en/dev/index.html](http://textblob.readthedocs.io/en/dev/index.html)]([http://textblob.readthedocs.io/en/dev/index.html](http://textblob.readthedocs.io/en/dev/index.html)), accessed on 27 April 2021) library was also considered. This implementation is a lexicon approach as well. It is based on the web mining module _Pattern_, which was proposed by [42]. Compared to _SentiWordNet_, _Pattern_ does not only provide scores for polarity, but also a degree of subjectivity. Polarity scores span from \(-1\), which is very negative, to 1, which is very positive. Objective words get a score of 0, whereas highly subjective words like _handsome_ get a score of 1. In _Pattern_, words were manually annotated based on the triangle representation, which is shown in Figure 7. The authors claim that \"[...] more positive/negative adjectives are also more subjective [but] not all subjective adjectives are necessarily positive or negative [...]\" [43]. Although the paper is primarily about Dutch language, the proposed English word annotation showed good results on their test data, too.