# Aggression Detection in Social Media from Textual Data Using Deep Learning Models
## 4 Results
### Experiments with Combined Word2 vec and Emotional Features  
In these experiments, eight emotional features were extracted manually from annotated training data and added to the Word2 vec features to perform aggression classification.  
Table 5 shows the performance measures of the machine learning classifiers with combined features. Similarly, as in Section 4.1 experiments, the GB classifier recorded the highest metric values compared to other machine learning models. However, the F1-score of the GB with the combined features was 86%. This observation highlights the significance of integrating the emotional features with the Word2 vec features to improve classification performance.  
The classification performance of the deep learning classifiers fed with the Word2 vec and emotional features combined set is depicted in Table 6. The proposed DNN performed the best over all other deep learning models with the highest F1-score value of 97%. It was also noticed that the performances of all the deep learning models improved when the classifiers were fed with the combined feature set over that when provided with the standalone Word2 vec features. The same observation was reported for the classical machine learning classifiers. This observation reveals that the emotional features contributed to improving the ability of the algorithms to distinguish between aggressive and non-aggressive tweets.  
\begin{table}
\begin{tabular}{c c c c} \hline \hline
**Models** & **Precision** & **Recall** & **F1-Score** & **Accuracy** \\ \hline LSTM & 80.25 \(\pm\) 0.24 & 82.15 \(\pm\) 0.47 & 80.65 \(\pm\) 0.85 & 82.15 \(\pm\) 0.15 \\ BiLSTM & 83.17 \(\pm\) 0.74 & 85.84 \(\pm\) 0.21 & 84.76 \(\pm\) 0.86 & 85.17 \(\pm\) 0.48 \\ CNN & 81.14 \(\pm\) 0.47 & 80.38 \(\pm\) 0.16 & 79.96 \(\pm\) 0.84 & 82.14 \(\pm\) 0.37 \\ DNN & 86.28 \(\pm\) 0.08 & 87.74 \(\pm\) 0.19 & 87.11 \(\pm\) 0.83 & 88.34 \(\pm\) 0.17 \\ \hline \hline \end{tabular}
\end{table}
Table 4: Evaluation measures of the used deep learning models fed with the Word2 vec features. The measures are presented as percentages.  
\begin{table}
\begin{tabular}{c c c c} \hline \hline
**Models** & **Precision** & **Recall** & **F1-Score** & **Accuracy** \\ \hline SVM & 67.21 \(\pm\) 0.23 & 67.74 \(\pm\) 0.85 & 64.24 \(\pm\) 0.74 & 62.17 \(\pm\) 0.36 \\ LR & 62.36 \(\pm\) 0.18 & 49.24 \(\pm\) 0.73 & 44.96 \(\pm\) 0.19 & 58.14 \(\pm\) 0.90 \\ NB & 62  
To compare the performance of the machine and deep learning models, a bar graph of the F1-score of all models trained using the combined feature set is shown in Figure 3. As the F1-score took the recall and precision into account, and the data classes were unbalanced, comparing the machine learning and deep learning models was considered. It is clear that the proposed DNN model trained by the combined features outperformed all other machine and deep learning models.  
The significant finding of this research work is that the combination of emotional features with word embedding features improves the classification performance of aggressive and non-aggressive classes on the Cyber-Troll Twitter dataset. The classification performance measures on the combined features demonstrate that emotional features play prominent roles when identifying aggressive tweets. Furthermore, the experimental study on the used dataset shows that manually extracted emotional features, when combined with Word2 vec, improved the classification capability of the classifiers. Therefore, this finding suggests that features such as anger, anxiety, disgust, and anticipation play important roles in identifying aggressive content. To reinforce this finding, hypothesis testing was conducted to test the significance of the emotional features.