# A Web Shell Detection Method Based on Multiview Feature Fusion
## 3 Methodology
### Data Preprocessing  
After collecting the relevant web shells from github, all the PHP scripts are extracted first; a total of 1489 PHP web shells were obtained. After that, various techniques for data preprocessing are adopted to improve the quality and formalization of the data samples. Data preprocessing mainly includes the following three categories:  
1. Denoise.  
First, a script whose length exceeds a certain value is considered. Existing work [22] shows that scripts that are too large, often contain a lot of semantically irrelevant content and noise. We removed scripts with a size of more than 3 MB (it was found through extensive analysis that scripts exceeding this size generally cannot be effectively run) as noise, and a total of 1208 effective scripts were obtained. In addition, some PHP scripts will have a lot of obfuscation and BASE64 (a group of binary-to-text encoding schemes that represent binary data in an ASCII string format by translating it into a radix-64 representation [29]) encoding, which makes the detection method based on static analysis useless. In order to get a clean dataset, UnPHP technology [30] was used to deobfuscate the obfuscation code and BASE64-encoded web shells, and it has also been used in previous work for web shell analysis [21; 31]. Finally, 1192 web shells that could be effectively run without timeout were retained.
2. Deredundancy.  
In order to remove duplicate web shell scripts, all the scripts are standardized first (i.e., all comments and blank lines are deleted). Secondly, the term frequency-inverse document frequency (TF-IDF) [32] model is used to calculate the text similarity, which is widely adopted in previous work [33]. Additionally, only one copy of all scripts, with a similarity exceeding 95%, is retained. In the end, 1056 effective and executable script files were retained.  
Figure 1: The overview of our method.  
3. Clustering by various families. [PERSON] et al. [3] divided web shells into three categories according to the function and size of the scripting language: Big Trojan, One Word Trojan, and Small Trojan. However, in this paper, we only considered the Big Trojan for the following two reasons:  
1. Small Trojan usually has an upload function only. Attackers generally first obtain upload permission through the Small Trojan and then upload the Big Trojan to the website to perform key functions. The source of malicious behavior is Big Trojan.
2. One Word Trojan needs to be used in conjunction with other tools to perform malicious behavior. Additionally, One Word Trojan is often used at the beginning of the attack. Unlike the Big Trojan, One Word Trojan is rarely used for persistence attacks.  
There are a series of variations between web shells. For example, the c99 web shell is renamed c999 after adding the privilege escalation function. At the same time, some function modules in c99 can be found in the Fx29 web shell. It can be found that there is an inheritance relationship between the functional modules between them. Therefore, the idea of transitive closure [34] is used to cluster the similar web shells into one family. That is, if scripts m1 and m2 have a part of the same functional modules and scripts m2 and m3 have a part of the same functional modules, it can be inferred that m1, m2, and m3 are all subordinate to one family. This transfer feature makes it possible to get information about all members of the same family. At the same time, in this process, the corresponding function of the web shell in a family is gradually clear.  
To ensure the utility of the provided method, 1056 effective web shells were divided into seven different families, with a clustering threshold of 30%. That is, if 30% of the functional modules are identical between the two web shells, they can be clustered into one family. The final clustering results are shown in Table 1.