## 4 Proposed Hate Speech Detection Architecture
### Proposed Odia-English Text Preprocessing
#### 4.1.1 Removing (Cleaning) Irrelevant Character, Punctuations Symbol, and Emoji's
\begin{table}
\begin{tabular}{c c c c c c c c c c} \hline \hline \multirow{2}{*}{**Extracted Feature**} & \multicolumn{2}{c}{**SVM**} & \multicolumn{2}{c}{**NB**} & \multicolumn{2}{c}{**RF**} \\ \cline{2-10}  & **P** & **R** & **F1** & **P** & **R** & **F1** & **P** & **R** & **F1** \\ \hline Word unigram & 0.51 & 0.51 & 0.51 & 0.43 & 0.42 & 0.42 & 0.51 & 0.51 & 0.50 \\ Word bigrams & 0.51 & 0.51 & 0.51 & 0.45 & 0.44 & 0.44 & 0.51 & 0.51 & 0.50 \\ Word trigrams & 0.51 & 0.51 & 0.51 & 0.45 & 0.44 & 0.44 & 0.51 & 0.51 & 0.50 \\ Combined n-grams & 0.52 & 0.52 & 0.52 & 0.52 & 0.50 & 0.50 & 0.50 & 0.51 & 0.50 \\ TF-IDF & 0.49 & 0.49 & 0.49 & 0.50 & 0.47 & 0.48 & 0.50 & 0.51 & 0.50 \\ TF-IDF+ combined n-grams & 0.49 & 0.49 & 0.49 & 0.50 & 0.48 & 0.49 & 0.50 & 0.51 & 0.50 \\ Word2 vec & 0.53 & 0.53 & 0.53 & 0.51 & 0.50 & 0.49 & 0.57 & 0.54 & 0.50 \\ \hline \hline \end{tabular}
\end{table} 
Table 14: Classification performance result of ternary models. Figure 11: Confusion matrix of sample ternary NB models based on extracted features. Figure 12: Confusion matrix of sample ternary RF models based on extracted features. trigram features. Classification performances of LR, DT, GB, and KNN by P, R and F1 score are listed in Table 15.

For accuracy, it is quite evident from Table 11 that the SVM model with the word trigram features has an accuracy of 51.4%, which is higher than the average accuracies reported by any of the four standard ML techniques. From Table 14, it can be seen that the SVM model has a precision of 0.51, which is a slight improvement on the precision of any of the methods shown in Table 15. It signifies that the false-positive cases recorded by SVM are less compared to other models. The recall value for SVM model stands at 0.51, which is also marginally higher than the precision value reported by all of the four standard techniques. This reveals that the false-negative case conveyed by our model is less compared to classic ML techniques, which signifies a drop in misclassification instances. ### Discussion

The defining characteristics of the few hate detection models published in the last four years are summarized in Table 16. The first column shows the paper, and the year of publishing, and columns two to five reflect the details of the datasets used in the experiments. The subsequent columns exhibit the features considered (the features used in the best model are in bold), and the classification models tested (the best classification model is in bold). Finally, the performance of the best model found is presented using the accuracy (A) and F1-scores (F1). A dash (-) indicates that the value is not available. Binary detection models were developed by [PERSON] and [PERSON] [22] using NB and RF with the extracted features word2 vec and TF-IDF. They reported performance accuracy results of 79% and 73% for NB, and 65% and 63% for RF with both features, respectively. They used a total of 6120 instances as dataset. Among those, 1821 were labeled posts and comments and a dictionary of hateful word and phrases was extracted from the annotated dataset. The dataset contains 3296 non-hate and 2824 hate speech instances, and the model used 80% of the dataset for training and 20% for testing. In contrast, the RF models of our study perform better than the RF models of [PERSON] and [PERSON]'s study [22], by a margin of 10% and 9% for both features, respectively. The NB model with TF-IDF feature of this study also performs equally with [PERSON] and [PERSON]; however, our NB model with word2 vec falls behind by 8%. A CNN-based model for HS detection from Twitter on word2 vec embeddings reported by [PERSON] and [PERSON] [25] showed the values of the P, R and F1 scores as 0.85, 0.72 and 0.78, respectively. Furthermore, the P and F1 score values of [25] are marginally higher than our results, which were P = 0.76, R = 0.73 and F1 score = 0.73. This small marginal difference of accuracy results obtained by the same feature extraction method shows that the size of neither the dataset nor the setup used is the core problem.