# Detecting Web-Based Attacks with SHAP and Tree Ensemble Machine Learning Methods
## 2 Related Work  
Researchers have put forward many prevention techniques to keep up with the increase in attacks and new attack methods. Blocklists are repositories of known malicious URLs and have long been employed to detect malicious URLs and their JS codes. Heuristic and signature-based systems search for signs of standard attack patterns. The weakness of these approaches is that new or variant URLs and their JS codes and zero-day attacks can easily evade detection. Further, cybercriminals widely use obfuscation to perpetrate attacks and evade detection. Some studies [12; 13; 14] have proposed embedding the bag-of-words model in a URL or features of a URL and its JS code content. [PERSON] et al. [12; 13; 14] proposed an approach to analyzing a URL to predict the maliciousness of websites using an online confidence-weighted algorithm for lexical feature learning. Individual feature vectors were manually engineered. [PERSON] et al. [12] acknowledged that other potentially useful sources of information for features could improve classification accuracy. However, they did not examine a web page's actual content, citing reasons such as user safety, the model's operation costs, the classifier's applicability to the URL context, and poor reliability when obtaining a malicious web page version. They also used bag-of-words features and therefore did not preserve token order. Conversely, our proposed approach uses JS code content features. Since AST-JS is resistant against perturbations in JS code content, it makes our model applicable for various JS code contexts and varying content. Other systems [2, 3, 5, 21, 22] have been proposed that use constructed features or feature engineering. [PERSON] et al. 's [2, 3] EXPOSURE conducted a passive DNS analysis to identify malicious domains. It is a dynamic reputation system based on passive recursive DNS monitoring. They extracted 15 behavioral features to train the J48 decision tree classifier. Their study assumed that large volumes of DNS data requests should exhibit sufficient behavioral differences to distinguish between benign and malicious domains. In another study, a system to detect malicious domains using DNS records and domain name features was proposed by [PERSON] et al. [5] using J48, a C4.5 decision tree. This approach studied prior DNS activities for each domain and the relationship between defective and legitimate domains' physical behavior [21], combining several existing DNS-based and domain name-based features from previous work [2, 3, 22]. They assumed that eight unique behavioral features could accurately identify malicious websites and proactively detect an attack. However, feature engineering is challenging, even for domain experts. We performed automatic feature selection to address these challenges. [PERSON] et al. [23, 24] sought to detect targeted attacks by monitoring the communication between the Command and Control (C&C) server and the computers in the local area network. Their proposed method identified new C&C servers using supervised machine learning; extracted the WHOIS feature points and DNS information; and searched the site. For performance comparison, the malicious domains were detected separately using a neural network and a support vector machine. Since manual feature selection is a tedious task, even for domain experts, we propose an autonomous feature-selection method using SHAP values and tree ensemble methods. Other studies have attempted to detect specific URL attack types, such as phishing [25, 26, 27, 28, 29, 30], malicious advertisements, and click fraud [1, 31, 32, 33, 34, 35, 36]. These approaches are suited for specific attack types and do not generalize well to other attacks. [PERSON] et al. [1] proposed a system for the automatic classification and detection of malicious advertisements. They used VirusTotal [37], URLVoid [38], and TrendMicro [39]. Their study relied heavily on other tools that employ signatures and blocklist services. Therefore, it was prone to problems inherent to signature and blocklist-based systems [4].