# Detecting Web-Based Attacks with SHAP and Tree Ensemble Machine Learning Methods
## 3 Proposed Method
### Preprocessing
#### 3.1.2 Sample JS Attack Codes
Next, we visualized and investigated each AST-JS feature's contribution to the malicious JS code detection model's performance using SHAP values. This analysis indicated the need to define more concrete features. Using sample JS attack codes, we added ten more AST-JS features based on combinations of the original ones. These features represent JS-based attacks and contribute to the model's detection performance. We then used association rule mining and a confidence metric to measure how often AST-JS nodes appear together in benign and malicious JS codes. This resulted in 33 features with parameters \(min\_support=0.4\) for malicious JS codes, \(min\_support=0.53\) for benign codes, and \(confidence\)'s \(min\_threshold=1\) for both benign and malicious codes. This selection reduced our original features and formed the second input set. Finally, we selected a final set of AST-JS features using the SHAP values. This selection resulted in forty-four features, comprising thirty-four AS-JS nodes and ten node combinations. Performance evaluation was conducted using the three-input feature sets, i.e., the 48 AST-JS features obtained after the initial preprocessing, the 33 features obtained using association rule mining, and the 44 features obtained by feature selection using the SHAP value. The parameters obtained using \(GridSearchCV\) for the two best-performing models, XGBClassifier and LGBMClassifier, using the three AST-JS feature sets, are listed in Table 1.

We found that these parameters are optimal for malicious JS code content detection using the three-input feature sets. We performed 10-fold cross-validation to evaluate the performance of our approach and computed precision, recall, and F1-score evaluation metrics. Precision is the classifier's ability to not label a benign JS code as malicious, and recall is the classifier's ability to find all malicious JS codes. The F1-score can be interpreted as the weighted harmonic mean of precision and recall. Given that \(TP\) is the number of malicious JS codes correctly classified as malicious, \(TN\) is the number of benign JS codes correctly classified as benign, \(FN\) is the number of malicious JS codes classified as benign, and \(FP\) is the number of benign JS codes classified as malicious. _Precision_, \(recall\), and \(F1\)-_score_ are given by:

\[Precision=\frac{TP}{TP+FP} \tag{1}\] \begin{table}
\begin{tabular}{c c c c} \hline \hline
**Parameter** & **AST-JS Nodes** & **Association Rule** & **SHAP Value** \\ \hline XGBClassifier & & & \\ \(colsample\_bytree\) & 0.5 & 0.7 & 0.4 \\ \(gamma\) & 0.0 & 0.1 & 0.0 \\ \(learning\_rate\) & 0.2 & 0.2 & 0.2 \\ \(max\_depth\) & 10 & 8 & 13 \\ \(min\_child\_weight\) & 1 & 1 & 1 \\ \hline LGBMClassifier & & & \\ \(lambda\_1\) & 0 & 0 & 0 \\ \(lambda\_12\) & 0 & 0 & 0 \\ \(max\_depth\) & 9 & 6 & 8 \\ \(n\_estimators\) & 520 & 520 & 520 \\ \(num\_leaves\) & 10 & 10 & 10 \\ \hline \hline \end{tabular}
\end{table} 
Table 1: XGBClassifier and LGBMClassifier parameters. \[Recall=\frac{TP}{TP+FN} \tag{2}\]

\[F1\text{-}score=2\frac{Precision\times Recall}{Precision+Recall} \tag{3}\]

### Performance Comparisons

The results of these experiments are presented in this section. Table 2 presents the precision, recall, and F1-score values obtained using features defined as AST-JS nodes and 10-fold cross-validation. Each model performed well on the recall metric, especially the tree ensemble methods. The best-performing model, XGBoost, could detect malicious JS code content with an error rate of 0.0019 for recall, 0.0302 for precision, and 0.0162 for F1. The lower precision metric revealed that the model misclassified some benign JS codes. Even though this scenario is not harmful, it may lead to threat-alert fatigue if users and security analysts receive many false alarms. Therefore, there is a need to improve this model further. We analyzed the AST-JS node features using the SHAP values. Figure 2 presents a SHAP summary plot that shows the relative impacts of AST-JS features on the JS code dataset.