## 4 Proposed Hate Speech Detection Architecture
### Proposed Odia-English Text Preprocessing
#### 4.1.1 Removing (Cleaning) Irrelevant Character, Punctuations Symbol, and Emoji's
### Feature Extraction Result

The feature extraction process uses three methods, N-gram, TD-IDF, and word2 vec, and produces seven different sets of features vectors for the dataset. For word2 vec, the window is set to 10, the embedding size is set to 150, and the min count is taken as two. This paper uses skip-gram. To implement word2 Vec, this paper utilizes a python Gensim module that is used to implement different embedding methods. It includes both skip-gram and CBOW, but the authors chose skip-gram for this experiment. Feature modeling with Gensim word2 Vec is straightforward. First, the word2 Vec class is imported and instantiated with the necessary parameters and the vocabulary is built. The word2 Vec model is then trained using the posts and comments that are gathered from the selected pages. The resulting feature vector is also known as the embeddings. Embeddings are the features that describe the target word. The resulting word2 vec model is then used to extract feature by computing the similarity for a word in the dataset and using it as a feature to train the machine learning models. The authors conducted the experiment with the embedding size varying from 100 to 300, but the optimal result occurs at the embedding size of 150, hence, the authors performed the experiment with 150. \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Label** & **Annotator 1** & **Annotator 2** & **Annotator 3** & **Final Class by [PERSON]** \\ \hline Offensive (OPS) & 251 & 227 & 221 & 264 \\ Hate (HS) & 93 & 114 & 151 & 95 \\ Neither (OK) & 156 & 159 & 128 & 141 \\ Total & 500 & 500 & 500 & 500 \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Annotation result of common posts and comments. \begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Label** & **Annotator 1** & **Annotator 2** & **Annotator 3** & **Annotator 4** & **Total Unique Annotated** \\ \hline Offensive (OPS) & 484 & 628 & 304 & 619 & 2035 \\ Hate (HS) & 281 & 202 & 198 & 417 & 1098 \\ Neither (OK) & 235 & 170 & 498 & 464 & 1367 \\ Total annotated & 1000 & 1000 & 1000 & 1500 & 4500 \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Annotation result of unique posts and comments. \begin{table}
\begin{tabular}{c c} \hline \hline
**Label or Class** & **Number of Post and Comments** \\ \hline Offensive (OFS) & 2299 \\ Hate (HS) & 1193 \\ Neither (OK) & 1508 \\ Total & 5000 \\ \hline \hline \end{tabular}
\end{table} 
Table 5: The three-class distribution of the dataset. These feature vectors are used in the training of the SVM, NB and RF models. For SVM, the parameters are multilabel classifications, *, n_jobs = None. For multinomial NB, the parameters are alpha = 1.0, fit_prior = True, class_prior = None. For RF, the parameters are estimators = 100, criterion = 'gini', max_depth = None, min_samples_split = 2, min_samples_leaf = 1, min_weight_fraction_leaf = 0.0, max_features = 'auto', max_leaf_nodes = None, min_impurity_decrease = 0.0, min_impurity_split = None, bootstrap = True, oob_score = False, n_jobs = None, random_state = None, verbose = 0, warm_start = False, class_weight = None, ccp_alpha = 0.0, max_samples = None. Table 6 shows the extracted feature vector size for the dataset. ### Models Evaluation Results

The experiment results in twenty-one different models based on seven features and three classifiers for both binary and ternary classification. These trained models are tested by 5-fold cross-validation. This method randomly splits the dataset into five equal sized datasets or folds. For each unique fold, the fold is taken to be the test dataset, and the models are trained using the remaining datasets. This process is iterated for each unique fold. The results are presented in binary and ternary classification models below. ### Binary Classification Models Evaluation Results

These classification models are built using the two-class dataset that is converted from the annotated three-class dataset, which means that the target classes are HS and OK. The trained models are tested using 5-fold CV.