## 3 Materials and Methods
### Data Collection
\[Acc.=\frac{TP+TN}{D} \tag{4}\]

where (\(TP+TN\)) is the accurately predicted content either clickbait or not, \(D\) is the total number of samples in the dataset. \[P=\frac{TP}{TP+FP} \tag{5}\]

where (\(TP+FP\)) is the total number of predicted clickbait content. \[R=\frac{TP}{TP+FN} \tag{6}\]

where (\(TP+FN\)) is the total number of actual clickbait content. \[F1=2\times\frac{P\times R}{P+R} \tag{7}\]

## 4 Experimental Design

The experiments in this study were performed on Python 3.8 with Windows 10 operating system. We used numerous Python packages including sklearn 0.22.2 for implementing the classifiers, nltk 3.6.2 for pre-processing Arabic text and Beautiful soup 4.9.0 for scraping data from external web pages. The user-based features and content-based features were fed into classifiers separately. Later, we merged both types and measured the performance of ML classifiers based on top-scoring features p% that were selected based on f-values of one-way ANOVA. For ensuring fair comparison between classifiers, the same pre-processing steps and the same set of features were used for each classifier. In addition, we considered four experimental scenarios per feature type, as illustrated in Table 8.

## 5 Results and Findings

This section describes and discusses the results for each experiment shown in Table 8. First, we present the findings that were obtained when only the user-based features were used. The accuracy of each classifier is presented in Table 9. The second type of features, \begin{table}
\begin{tabular}{c c c} \hline \hline \# & **Type of Experiment** & **Number of Features** \\ \hline \multirow{4}{*}{\(UCFs\)} & Baseline: & 15 \\  & F\_values\_5\%: 5\% of features & 4 \\  & F\_values\_10\%: 10\% of features & 7 \\  & F\_values\_15\%: 15\% of features & 9 \\ \hline \multirow{4}{*}{\(NCFs\)} & Baseline: Including (TF-IDF) extracted features & 10,236 \\  & F\_values\_5\%: 5\% of features & 732 \\  & F\_values\_10\%: 10\% of features & 2187 \\  & F\_values\_15\%: 15\% of features & 5867 \\ \hline \multirow{4}{*}{\(UCFs+NCFs\)} & Baseline: & 10,251 \\  & F\_values\_5\%: 5\% of the extracted features & 736 \\ \cline{1-1}  & F\_values\_10\%: 10\% of the extracted features & 2194 \\ \cline{1-1}  & F\_values\_15\%: 15\% of the extracted features & 5876 \\ \hline \hline \end{tabular}
\end{table} 
Table 8: Number of features per each experiment. content-based features, were then investigated, as shown in Table 10. Finally, we combined both types of features and the performance of classifiers is presented in Table 11.

Based on the results presented in Table 9-11, the following findings are observed and can be summarized as follows:

* When the content-based features were used, the classifiers performed well and SVM, NB, and RF achieved notable results using 10% of top-scoring features compared to their results in the baseline experiment. Among these methods, SVM obtained the best accuracy (91.83%) for content-based features. * In most cases of experiments with content-based features, all classifiers showed good results when the one-way ANOVA method was used as feature selection, except k-NN and LR. It is notable that k-NN had the worst performance when the number of selected features increased to 10% and 15%. * Increasing the percentage of the top-scoring features to more than 10% leads to a reduction in the performance of the ML classifiers. * RF and SVM benefited more when the user-based features were used, compared to their results in the baseline experiment. * The result for LR remained constant, and no change was observed when user-based features were fed into the classifier. * The k-NN and SGD do not benefit from the ANOVA-based feature selection at all for user-based features.