## 3 Materials and Methods
### Data Collection
This step allows us to find the top-scoring features. Later, only the p% top-scoring features were used to train the ML classifiers. The process of selecting features for FV-ANOVA is presented in Algorithm 2. ```
Input:\(D\)-dataset, \(V\) features extracted as numeric representation by TF-IDF, C-class label and \(p\%\) percentile. Output:\(D_{FS}\) subset of top-scoring features based on the given \(p\%\) \(k\leftarrow\) number of classes in \(D\) \(N\leftarrow\) number of features in \(D\) For each pair \(f_{j}\in(V,\,C)\)do: Count number of samples per class  Compute (mean, standard deviation, standard error) of each \(f_{j}\) with respect to \(C_{i}\)  Compute degree of freedom between/within classes (\(SS_{B},\,\,\,SS_{w}\))  Compute sum of square of \((SS_{B},\,\,\,SS_{w})\)  Find mean square \(MS_{B}\) between groups as \(MS_{B}=SS_{B}\) / \((k-1)\)  Find mean square \(MS_{W}\) between groups as \(MS_{W}=SS_{W}\) / \((N-k)\) Endfor \(F\_value\leftarrow\frac{MS_{B}}{MS_{W}}\) Sort \(F\_value\) in ascending order \(D_{FS}\leftarrow\) Select the top-scoring features based on \(p\%\) Return\(D_{FS}\)
```

**Algorithm 2** Pseudocode for selecting features-based FV-ANOVA method. ### Feature Selection

Six ML classifiers were implemented: Random Forest (RF), Logistic Regression with Stochastic Gradient Descent (SGD), Support Vector Machine (SVM), Logistic Regression (LR), Multinomial Naive Bayes (NB), and k-Nearest Neighbor (k-NN). To explore the effectiveness of the proposed feature selection method, we conducted different experiments and employed these classifiers on different subsets of features based on F-values. For tuning hyper-parameters of the used ML classifiers, the grid search algorithm with k-fold cross-validation is used. Subsequently, the values of hyper-parameters that yield the highest performance measure are set to be the final values of hyper-parameters for each classifier. The set of values of hyper-parameters used in this work is presented in Table 7. \begin{table}
\begin{tabular}{c c c} \hline \hline
**ML Classifier** & **Hyper-Parameters Used for Tuning the Model** & **Best Values of Hyper-Parameters** \\ \hline \multirow{5}{*}{RF} & Criterion = [entropy, gini] & Criterion = gini \\  & max\_depth = [10-1200] + [None] & max\_depth = 142 \\  & min\_samples\_leaf = [3–13] & min\_samples\_leaf = 3 \\  & min\_samples\_split = [5–10] & min\_samples\_split = 7 \\  & n\_estimators = [150–1200] & n\_estimators: 300 \\ \hline \multirow{5}{*}{SGD} & alpha = [\(1\times 10^{-5}\), \(1\times 10^{-4}\), \(1\times 10^{-3}\), \(1\times 10^{-2}\), 0.1, 1] & alpha= \(1\times 10^{-4}\) \\  & loss = [log, hinge] & loss = log \\  & max\_iter = [10–1000] & max\_iter = 1000 \\  & Penalty = [12’, /11’,’elastic’] & Penalty = 12 \\ \hline \multirow{5}{*}{SVM} & C = [0.1, 1, 10, \(1\times 10^{2}\), \(1\times 10^{3}\)] & C = 10 \\  & Gamma = [\(1\times 10^{-4}\), \(1\times 10^{-3}\), \(1\times 10^{-2}\),0.1, 1, 10, \(1\times 10^{2}\)] & Gamma = \(1\times 10^{-3}\) \\  & Kernel = [sigmoid, linear, rbf] & Kernel = rbf \\ \hline \multirow{2}{*}{LR} & C = [\(1\times 10^{-3}\), \(1\times 10^{-2}\), 0.1, 1, 10, 100], fit\_intercept = [True, False] & C = \(1\times 10^{-3}\) \\  & fit\_intercept = True & fit\_intercept = True \\ \hline \multirow{2}{*}{NB} & alpha = [\(1\times 10^{-5}\), \(1\times 10^{-4}\), \(1\times 10^{-3}\), \(1\times 10^{-2}\), 0.1, 1]fit\_prior = & alpha = 0.1 \\  & [True, False] & fit\_prior = True \\ \hline \multirow{2}{*}{K-NN} & n\_neighbors = [1, 40] & Number of neighbours = 7 \\ \cline{2-2}  & & \\ \end{tabular}
\end{table} 
Table 7: List of optimized hyper-parameters of each classifier. ### Model Evaluation

To evaluate the performance of classifiers, we computed the accuracy (\(Acc\)), recall (\(R\)), precision (\(P\)), and f1-score (\(F1\)) metric of each classifier with those features that were selected by the proposed F-values of the one-way ANOVA test. The descriptions of these metrics are shown in Equations (4)-(7) respectively.