# Automatic Detection of Clickbait Headlines Using Semantic Analysis and Machine Learning Techniques
## 5 Modeling
### Individual Modeling  
Six individual classification models are tested: decision tree, logistic regression, naive Bayes, support vector machine (svm), k-nearest neighbor (knn) and gradient-boosted decision tree (GBDT). The dataset is randomly split 80:20 into 25,600 training headlines and 6400 test headlines. In order to optimize the performance of the six machine learning models, we conducted a thorough search for the best hyperparameters. The hyperparameters of each model are selected through a randomized search, which is a probabilistic method for hyperparameter tuning. The randomized search was conducted for 100 iterations for each model and the best hyperparameters were chosen based on the highest performance metric score on the validation set. The hyperparameters selected for each machine learning model are displayed in Table 4.  
After selecting the hyperparameters for each of the six machine learning models, the next step is to evaluate their effectiveness in categorizing clickbait headlines. The headline dataset was partitioned into two subsets--a training set comprising 25,600 headlines and a test set comprising 6400 headlines, using an 80:20 ratio. Before evaluating the models, it is important to understand the key metrics used to assess the models' performance. The selected evaluation metrics are accuracy, precision and recall. 1. **Accuracy** measures the proportion of correct predictions out of all predictions made and is calculated as: \[\textbf{Accuracy}=\frac{TP+TN}{TP+TN+FP+FN}\] (5) \begin{table}
\begin{tabular}{l l} \hline \hline
**Model** & **Selected Hyperparameters** \\ \hline Decision Tree & Maximum Depth = 10 \\ Logistic Regression & Regularization Parameter = 0.01 \\ Naive Bayes & None \\ Support Vector Machine & Regularization Parameter = 0.1, Kernel = Polynomial \\ K-Nearest Neighbor & Number of Neighbors = 5 \\ Gradient Boosting & Learning Rate = 0.1, Maximum Depth = 5 \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Hyperparameters selected for each machine learning algorithm. \begin{table}
\begin{tabular}{l l} \hline \hline \multicolumn{2}{c}{**The Top-10 Features Correlated with “Clickbait” Headlines**} \\ \hline (1) Uppercase2 & \(0.93\) \\ (2) Possessive Pronouns & \(0.53\) \\ (3) Determiner/Superlative & \(0.45\) \\ (4) Begins w/Number & \(0.40\) \\ (5) Demonstrative & \(0.36\) \\ (6) Word Count2 & \(0.27\) \\ (7) Begins w/Det Sup & \(0.26\) \\ (8) Includes “this” & \(0.23\) \\ (9) Includes Number & \(0.23\) \\ (10) Contraction & \(0.22\) \\ \hline \multicolumn{2}{c}{**The Top-5 features Correlated with “Non-Clickbait” Headlines**} \\ \hline (1) Stop Words2 & \(-0.83\) \\ (2) F Score2 & \(-0.46\) \\ (3) Word Length2 & \(-0.33\) \\ (4) CL Score2 & \(-0.29\) \\ (5) LX Index2 & \(-0.29\) \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Top-15 features. where TP (true positive) is the number of actual clickbait headlines that were correctly classified as clickbait, TN (true negative) is the number of actual non-clickbait headlines that were correctly classified as non-clickbait, FP (false positive) is the number of actual non-clickbait headlines that were incorrectly classified as clickbait and FN (false negative) is the number of actual clickbait headlines that were incorrectly classified as non-clickbait. 2. **Precision** measures the proportion of correct clickbait predictions out of all predictions made as clickbait and is calculated as: \[\textbf{Precision}=\frac{TP}{TP+FP}\] (6)
3. **Recall** measures the proportion of actual clickbait headlines that were correctly classified as clickbait out of all actual clickbait headlines and is calculated as: \[\textbf{Recall}=\frac{TP}{TP+FN}\] (7)  
In addition to the full 30 classification features, the models were trained and tested using less than the 30 features (i.e., Top-25/20/15/10/5/2 features). The goal is to find the simplest model that produces the best accuracy utilizing the least number of features. Table 5 shows the validation performance of each of the models across the different combined feature sets.