# Investigating the Influence of Feature Sources for Malicious Website Detection
## 5 Experimental Results--GAWAIN
### Feature Contributions  
The insights were at their most valuable when the feature combinations are divided into a combination of lexical, host, and content-based features, with and without including the embeddings. The specific feature importance/contribution for the classification task are outlined in Tables 5-7 for the XGBoost model. We are interested in which features are most important for the _Baseline_ category, and how the introduction of different embeddings affects the feature contributions. Since we have two dimensionality reduction methods tested, we present two sets of feature contributions, one for PCA and the other for Chi-square test. The XGBoost model offered insight into which features had been the most contributing factors for segmenting the training data.  
Several observations can be made regarding the feature contributions of each of these categories. The simplest observation is that as the embeddings are added to the features, the contribution to the classification is split more equally. We observe that URL embeddings play a major role and replace certain Lexical features altogether, and this replacement contributes slightly to the overall test accuracy shown in Table 4. There are many possible reasons for this change. One could be that the embeddings are compensating for other characteristics of the URLs and content that the other features may not be highlighting. These feature contributions may be a better justification than test set accuracy for the usage of the embeddings alongside the extracted features, since they provide a deeper explanation for the influence of these embeddings on the ML. However, it is worth mentioning that this has only been tested with XGBoost on GAWAIN, so it is necessary to expand and give justification for this observation in future work. Moreover, the suggestion of missing features given by this addition of embeddings suggests that this classification problem may benefit from the gathering of additional features.  
\begin{table}
\begin{tabular}{l c} \hline \hline
**Feature** & **Contribution(\%)** \\ \hline URL (Distilbert) PCA embedding 1 & 6.21 \\ \hline Number of ampersands in URL & 2.89 \\ \hline URL (Longformer) PCA embedding 4 & 2.79 \\ \hline DCD Color 1 & 2.74 \\ \hline WHOIS & 2.56 \\ \hline URL (Longformer) PCA embedding 2 & 2.56 \\ \hline HTTPS & 2.47 \\ \hline Top-level domain & 2.33 \\ \hline Is in Alexa’s top 1 million & 2.33 \\ \hline \hline \end{tabular}
\end{table}
Table 6: Top 10 Contributing Features (with PCA 10 components embeddings).  
\begin{table}
\begin{tabular}{l c} \hline \hline
**Feature** & **Contribution(\%)** \\ \hline WHOIS & 13.97 \\ \hline Top-level domain & 4.53 \\ \hline Unique URL Chars count & 4.36 \\ \hline Having @ in URL & 4.29 \\ \hline HTTPS & 4.13 \\ \hline Unique Nums in URL count & 3.75 \\ \hline Location & 2.84 \\ \hline Number of @’s in URL & 2.79 \\ \hline Number of ampersands in URL & 2.74 \\ \hline \hline \end{tabular}
\end{table}
Table 5: Top 10 Contributing Features (without embeddings).  
We also notice the absence of _Content-based_ features altogether from the tables. The information they carry does not seem to be relevant enough to push the accuracy scores higher. One of the main reasons to consider content is to look for suspicious code. However, in our data processing pipeline we have not encountered obfuscated JavaScript code which definitely could play a major role.