## 2 Materials and Methods
### Sampling and Models Generation  
Learning from imbalance data is still a challenge for machine learning methods nowadays [57]. A classifier, when trained with an imbalance distribution dataset, tends to be biased towards the more frequent class. Therefore, efforts to avoid such types of skewed distributions are of paramount importance to ensure the aforementioned influence is not learned by the model. Pre-processing or training methods should then focus on alleviating this disadvantage. The aim is to create a learning system that is able to predict over the minority class but without sacrificing the performance on the majority one [58]. As this is not a trivial challenge and has a major relevance in the generated model accuracy, multiple approaches to solve it have been proposed in the literature [59; 60]. These types of approaches are called sampling methods. There are two main ways of balancing an imbalanced class set: downsizing the large class (also known as under-sampling) or upsizing the small class (also known as over-sampling). Generally, over-sampling is preferred [61]. Both can be performed on a random basis by randomly adding or dropping instances of the minority or majority class, respectively. However, this approach may cause overfitting or loss of information. Thus, other techniques aim to overcome such limitations. One of them [62] introduced a cluster-based under-sampling approach, where it was proposed that clusters in the dataset that have more majority class samples and less minority class samples will behave like the majority class samples, and vice versa. Therefore, it would be reasonable to select a suitable number of majority class samples from each cluster by considering the ratio of the number of majority class samples to the number of minority class samples in the cluster. Regarding over-sampling, SMOTE [63] (synthetic minority over-sampling technique) was developed to generate synthetic minority class examples by selecting new samples close to the existing ones in the feature space. On the other hand, ADASYN [64] (adaptative synthetic sampling) also generates synthetic sample points for the minority class but considers a density distribution to decide the number of synthetic samples to be generated for a particular point. The main objective of the current research work was to generate different classification models by means supervised machine learning techniques. Such models were obtained starting from the different datasets mentioned in the previous subsection: considering the different numbers of evaluators (3 or 5) and discretizing or not the inputs (original, adapted or both). The variables in such datasets were the following: the output values from (i) IBM (originally a numerical score), (ii) Google Cloud (originally a numerical score) and (iii) Meaning Cloud (originally a discrete class) tools, and the (iv) manually annotated class. On the other hand, different sampling techniques were performed in order to balance the number of negative and non-negative classes associated to tweets. As discussed in Section 2, imbalanced datasets lead to poor quality classification models. There are two main ways of addressing this challenge: under- or down-sampling (i.e., reducing the number of samples of the majority class) and over- or up-sampling (i.e., increasing the number of samples of the majority class). For down-sampling, two methods were performed: random sampling and clustering; and for up-sampling, three methods were implemented: random sampling, SMOTE and ADASYN (see Section 2 for detailed references). In both cases of random sampling, the method was iterated over 10 times to ensure the randomness. For clustering, k-means was implemented. Both clustering and ADASYN were only applied to the original data not discretized, as they cannot handle categorized input variables. There is a wide variety of machine learning methods that are used to generate classification models. This kind of machine learning is also known as supervised machine learning.