## The Investigation Framework
### Dimensions of the Investigation
#### 4.1.2 Classification Techniques  
The second dimension of investigation is to find out the classification technique that demonstrates to be more suitable for the application scenario. Recall from Problem 1 that the classifier has to discover the topic \(tp_{j}\) that an unlabeled message \(umt_{j}\) talks about. Hereafter, we briefly introduce the four classification techniques we considered in our investigation framework.  
* **Naive Bayes (NB).** The Naive Bayes classifier [40] is a simple, fast, efficient, easy to implement and popular classification technique for texts: in fact, this technique is quite efficient as far as computation time is concerned; however, it performs well when features behave as statistically independent variables. In short, it is a probabilistic classification technique, which completely depends on the probabilistic value of features. For each single feature, the probability that it falls into a given class is calculated. It is widely used to address many different problems, such as for predicting social events, for denoting personality traits, for analyzing social crime, and so on.
* **Support Vector Machine (SVM).** Support Vector Machine classifiers are widely used for classification of short texts. This classification technique is based on the principle of structured risk minimization [41]: given the hyper-space of features, in which each point represents a document, it creates a hyper-plane \(h\) that divides the data into two sets (i.e., the hyper-space is divided into two semi-spaces by the hyper-place); the algorithm tries to identify the hyper-plane that maximizes the distance from each point (the distance is called _margin_) because the greater the margin, the lower the risk that a point falls into the wrong semi-space. In the test phase, a data point is categorized depending on the semi-space it falls into. The technique has been extended and adapted to support multi-label classification [42].
* **K-Nearest Neighbors (kNN).** K-Nearest Neighbors (kNN) classifiers are widely used for classification of short texts. During the test phase, given a new document \(\overline{d}\), the distance between document \(\overline{d}\) and all the documents in the training corpus is measured, by employing a similarity or a dissimilarity measure. Then, the set \(N(\overline{d})\) that contains the \(K\) nearest neighbors among the entire training corpus is obtained; the class label having the largest number of documents in \(N(\overline{d})\)becomes the class of \(\overline{d}\)[43]. For example, if \(N(\overline{d})\) contains 15 nearest neighbors to document \(\overline{d}\), where seven documents are labeled with the \"politics\" class, four documents are labeled with the \"sports\" class, three documents are labeled with the \"weather\" class and one document is labeled with the \"health\" class, then \(\overline{d}\) is labeled with the \"politics\" class.
* **Random Forest (RF).** It is a known supervised learning method for classification devised by [PERSON] [44]. It is an evolution of classical tree-based classifiers. The name of the technique is motivated by the fact that, during the training phase, many classification trees are generated: we can say that the classification model is a _forest_ of classification trees. During the test phase, all the classification tress are independently used to classify the unclassified case: the class assigned by the majority of trees is chosen as class label assigned to the unclassified case. This technique is very general and widely used in many application contexts, not only for text classification [45].