# An Empirical Approach for Extreme Behavior Identification through Tweets Using Machine Learning
## 4 Experiments
### Results  
In this work, our objective was to classify tweets having extreme or non-extreme (neutral) views. We adopted to apply machine learning approaches to demonstrate the efficacy of classifying tweets having extreme views or not. Firstly, it is important to mention that an automated labelling procedure of text data require improvement in order to get more accurate predictive ability using machine learning methods. This was the reason for us to use human assigned class labels, particularly in the context-based sentiment analysis, as a better viable solution (see Section 4.1). In terms of analysis, we present results in a two-step approach. The first approach in our analysis is applying PCA, as an EDA tool, which is a key to better understand data and in making/refining hypotheses (see Section 4.2). Applying classification models straightaway without understanding natural patterns in the data seems not to be a sensible way of analyzing data. Though we observed that PCA faced a challenge in terms of capturing variance in the first few dimensions for a very high-dimensional dataset. We believe that this has given us better clue of the hidden patterns in the data. The visualization through PCA for our dataset has demonstrated a better separation between clusters of neutral and extreme class. But cluster separation between sub-classes of extremes, as pro-Afghan government and pro-Taliban, were not obvious. These visualization results helped us in making hypothesis that using simplest features like TF-IDF and \(n\)-gram for tweets dataset can only help to differentiate between tweets either neutral or extreme. Whereas, discrimination between sub-classes of extreme may require some semantic knowledge in order to get discrimination between them. The second approach in our analysis was to evaluate our hypothesis of identifying tweets having extreme or neutral views. For this purpose, we opted to apply various classifiers like naive Bayes, SVM, decision tree, random forest, KNN and ensemble classification models (see Section 4.3). The classification performance is presented in the form of evaluation measures like predictive accuracy, precision, recall, F-score. Classification models often face complexity in terms of training process when working with high-dimensional data. Therefore, we apply PCA to reduce the dimensions considering the assumption that the reduced set of features preserves maximum structure (in terms of variance preservation) of the data and this reduced set of features will be used by the classification models. The classification results, in term of evaluation measures, with various set of features (i.e., unigram, bigram, PCA-based reduced feature set) are presented in Table 5. The results illustrate that the proposed framework has encouraging outcomes for radical/extreme content identification and aspect-based sentiment analysis of tweets. It can be observed from the results that naive Bayes' has shown a better result with 83.16% accuracy on unigram features. The SVM has demonstrated an improvement in the predictive accuracy (with 84.71% accuracy) with bigram features in comparison with unigram features. The ensemble techniques (i.e., boosting and bagging) are also comparable due to the high and stable accuracy over \(n\)-grams features. All other individual models (decision tree, KNN, random forest) have maintained less than 80% accuracy. Similarly, all the classifiers are also tested on a reduced feature set obtained through PCA. We present here the results obtained using a reduced feature set preserving 95% variance in the dataset. The experiments were also performed with a reduced feature set ensuring 90% variance in the dataset, but the classification predictive accuracy was not much different. PCA helped to reduce the high dimension features into low dimensions to minimize the overall analysis time required by the classification model training process. The outcome of classifiers on these reduced features is also depicted in Table 5 reporting the random forest by achieving highest accuracy (76%).