## 4 Proposed Hate Speech Detection Architecture
### Proposed Odia-English Text Preprocessing
#### 4.1.1 Removing (Cleaning) Irrelevant Character, Punctuations Symbol, and Emoji's
The problems are the ambiguity of dataset labels, which resulted in a larger percentage of non-hate class being misclassified as hate by the models. However, the SVM models with \begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Model** & **Extracted Features** & **P** & **R** & **F1** & **Average Accuracy (\%)** \\ \hline \multirow{3}{*}{LR} & Word unigram & 0.50 & 0.50 & 0.50 & 50.01 \\  & Word bigrams & 0.50 & 0.50 & 0.50 & 50.02 \\  & Word trigrams & 0.50 & 0.50 & 0.50 & 49.65 \\ \hline \multirow{3}{*}{DT} & Word unigram & 0.45 & 0.43 & 0.44 & 43.34 \\  & Word bigrams & 0.47 & 0.45 & 0.46 & 43.09 \\  & Word trigrams & 0.49 & 0.45 & 0.47 & 42.17 \\ \hline \multirow{3}{*}{GB} & Word unigram & 0.50 & 0.50 & 0.50 & 49.11 \\  & Word bigrams & 0.49 & 0.50 & 0.49 & 48.23 \\ \cline{1-1}  & Word trigrams & 0.50 & 0.50 & 0.50 & 48.52 \\ \hline \multirow{3}{*}{KNN} & Word unigram & 0.44 & 0.44 & 0.44 & 41.32 \\ \cline{1-1}  & Word bigrams & 0.45 & 0.45 & 0.45 & 42.31 \\ \cline{1-1}  & Word trigrams & 0.45 & 0.45 & 0.45 & 42.89 \\ \hline \hline \end{tabular}
\end{table} 
Table 15: Results using conventional machine learning methods. word2 vec used in this study show a better classification performance than the NB and RF models. This is the first time in the current research that a dataset on Odia-English mixed representation has been prepared. It is prepared using Facebook posts, which are in English alphabets but representing Odia phonetics. The datasets are annotated and labeled properly. ## 6 Conclusions

This paper proposes a solution for detecting hate speech on social media using machine learning techniques. The research attempts to develop, implement and compare machine learning and text feature extraction methods specifically for hate speech detection for the Odia-English mixed code language. To successfully execute the research, it is essential to understand and define hate and offensive speech on social media, explore the various existing techniques used to tackle the problem, and understand the Odia language. In addition, it is important to identify the different method followed to implement and design the models that have the capability of detecting hate speech. These methods include: collecting posts and comments for building the dataset; developing annotation guidelines; pre-processing and features extraction using n-gram; TF-IDF, and word2 vec, models training using SVM, NB, and RF; and models testing. Finally, comparisons of the models based on 5-fold CV evaluation metric results were performed. In this paper, the authors manually annotated the posts and comments into three classes of hate (HS), offensive (OFS), and neither (OK) speeches. The annotated dataset was converted into two class labelled datasets by converting all OFS to HS classes. This resulted in two datasets with 5000 instances of posts and comments; one with binary classes and the other with ternary class dataset. Based on the two datasets, the models were developed using SVM, NB, and RF, along with seven feature extraction methods, and the models were then executed. The experiment performed using these two datasets resulted in 21 binary and ternary models for each dataset. On one hand, binary models using RF with word2 vec resulted in better accuracy than both the SVM and NB models. On the other hand, the SVM model with word2 vec brought about a classification with a 73% F1-score, demonstrating a