# An Empirical Approach for Extreme Behavior Identification through Tweets Using Machine Learning
## 4 Experiments
### Applying PCA  
The PCA is considered as a linear method for projecting a high-dimensional dataset onto a low-dimensional space. PCA can also be defined as an orthogonal projection of high-dimensional data to a low-dimensional space in such a way that there will be a maximum variance in the projected data [49; 50; 51]. We apply PCA, to our dataset consisting of TF-IDF features extracted from the tweets, to project a high-dimensional data (i.e., TF-IDF features) onto a two-dimensional space. The low-dimensional space is usually considered as 2-D or 3-D with the objective of visualizing it onto a scatter plot to observe \begin{table}
\begin{tabular}{c c c} \hline \hline
**Class** & **Sub-Class** & **Description** \\ \hline \multirow{2}{*}{_Extreme_} & _pro-Afghan_ & Afghan government supporting accounts or users against Taliban s \\  & _pro-Taliban_ & Taliban supporting accounts or users against the Afghan government \\ \multirow{2}{*}{_Neutral_} & & a user having a neutral statement about an incident (i.e., user views are \\  & & not in favor of any party (i.e., Afghan government or Taliban). \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Description of assigned classes. Figure 4: Words distribution. similarity between observed samples (tweets in our case). Figure 5 represents such a visualization plot where two sub-graphs are presented. These sub-figures are similar in terms of data projection but are different in the context of labelling (i.e., colors): The Figure 5a represents a visualization with three classes assigned to observed samples (tweets) whereas Figure 5b represents a graph with two classes. As observed in Figure 5a, a class having _neutral_ views (appeared as blue dots) is spread across the whole plot whereas observed samples of classes such as _pro-Taliban_ (appeared as red dots) and _pro-Afghan government_ (appeared as green dots) are all around the center of the plot with some samples of _Neutral_ class in overlapping positions. The result here suggests that tweets having radical/extreme views all share similar characteristics not only in high-dimensional space but also onto a low-dimensional space. This also suggest that using a simple bag-of-words approach, it is difficult to get clear separation in clusters of _pro-Taliban_ and _pro-Afghan government_ views from tweets. However, it is obvious that tweets having similar keywords but with neutral views are often differentiable than from radical/extreme views. Concluding from the results presented in Figure 5a, we combined pro-Taliban and pro-Afghan government observed tweet samples as one group (appeared as red dots) in Figure 5b. This Exploratory Data Analysis using PCA explains that it will be easier to classify between neutral and radical/extreme views sharing similar keywords in the tweets. We, therefore, decided to focus in our work to identify tweets either as neutral or as radical/extreme. PCA is known to be a linear data projection model and attempts to preserve maximum variance in the lower-dimensional space. The variance preservation through PCA is dependent on the type of data in a dataset. It is usually observed that for the dataset having thousands of features, the first few principal components preserve very small variance [26]. Analysis of PCA results for our Twitter data having more than 3000 dimensions (in terms of TF-IDF features), it was observed that projection using first two principal components preserved less than 10% variability suggesting that PCA is not much trustworthy when working with high-dimensional data. Results for our dataset through PCA also suggests that cluster separation between two classes (neutral and extreme) is better than the three classes (i.e., neutral and two subclasses of extreme: pro-Afghan and pro-Taliban). One objective of applying PCA was to get visualization on a 2-D or 3-D scatter plot. We do not present a 3-D plot here, because results were not much different in terms of separation between classes and sub-classes.