# BERT-Embedding-Based JSP Webshell Detection on Bytecode Level Using XGBoost

Authors: Ao Pu; Xia Feng; Yuhan Zhang; Xuelin Wan; Jiaxuan Han; Cheng Huang
Journal: Security and Communication Networks
 (2022)

---

## Abstract

Webshell is a malicious program that might result in data theft, file modification, or other damaging behaviors once uploaded to a server. Detecting webshells is a key security concern for website administrators. In recent years, techniques such as obfuscation and encryption have been deployed on webshell technology, and classic detection approaches such as static feature matching are gradually underperforming on webshell detection. Meanwhile, there are variations between languages such as JSP and PHP, and researchers have proposed webshell detection methods primarily for languages such as PHP. At the same time, there are fewer detection techniques for JSP webshells. In this case, a detection approach for the JSP webshells is needed. This paper provides a novel webshell detection model for the JSP language. The model’s fundamental premise is that it introduces the BERT-based word vector extraction method, which has been shown in experiments to be more effective at detecting obfuscation, encryption, and other means of evading detection than the traditional Word2vec word vector extraction method. Meanwhile, we introduce the XGBoost algorithm as the model classifier. The experimental results reveal that present model has achieved 99.14% accuracy, 98.68% precision, 98.03% recall, and 98.35% f1 score, and the overall effect is better than the already existing JSP webshell detection approaches.


---

## Body

## 1. Introduction

Numerous web applications have brightened people’s lives in recent years as a result of the ongoing growth of web technology. Nonetheless, cyberattacks have the potential to harm the Internet’s environment and jeopardize societal security. As a result, preventing web attacks is a critical issue that we must address immediately. Among the numerous web attacks, webshell, a type of backdoor application for websites, is a frequently used attack method that has a significant detrimental effect. According to the report released by CNCERT/CC in 2022 [1], the number of webshell-attacked websites is still on the rise, and even some government websites have been attacked. After deploying a webshell on a web server, hackers can read, alter, delete, and even directly execute system instructions on the web server in order to carry out the next step of the attack.At this stage, hackers frequently employ obfuscation techniques to avoid webshell detection, granting webshells a measure of invisibility. As a result, we are developing new ways for successfully detecting concealed webshells. The majority of susceptible websites at this level are those that have been online for an extended period of time and heavily utilize PHP and JSP. Current research on PHP-type webshell attack [2–7] is enough, but research on JSP-type webshell detection is much lower than that on PHP-type webshell detection, so we need to propose a detection technique for JSP-type webshell. At the moment, webshell research is concentrated on static and dynamic detection. Static detection is typically accomplished by extracting static information from the file, such as the longest string, which may not be as successful as expected due to the JSP language’s specificities and the usage of encryption and other measures. Dynamic detection is typically dependent on network traffic; however it consumes a large amount of system resources and is therefore inefficient. We offer a detection approach tailored to the JSP webshell that identifies the bytecode extracted from the original JSP file following a series of transformations and for the first time applies BERT to the word vector extraction of JSP bytecode, resulting in good results. The main work of this paper is as follows.(1)
We extract features from JSP compiled bytecodes using a BERT-based word embedding approach and then utilize the machine learning XGBoost method to detect JSP webshells. According to the experimental results, this method performs pretty well, and its detection effect is superior to that of other JSP webshell detection methods now available.(2)
The detection results of the widely used Word2vec word vector extraction approach are compared to those of the BERT-based word vector extraction method. The results indicate that the BERT-based word vector extraction method outperforms the Word2vec extraction method, with an accuracy of 99.14%.(3)
We obtained 2073 benign JSP samples and 830 JSP webshell samples from the Internet, including several obfuscated and encrypted JSP webshells. Additionally, we produced some JSP webshells artificially to augment the real dataset, increase the number of data samples, and boost the detection effect.(4)
We analyze and detect JSP files on the bytecode level to effectively identify obfuscated and encrypted webshells.Next, we present the related work in Section2. Section 3 details the whole framework of our model and the main methods used. Section 4 presents our experimental procedure and the results of the experiments. Finally, Section 5 concludes the research of this paper.


## 2. Background and Related Work

### 2.1. Outline

Webshell is a typical backdoor program left behind after a web server is compromised, usually in the form of a dynamic web script that is a command execution environment. Concealment and flexibility are the characteristics of webshells, so users may be unaware of hackers using webshells to control the system steadily over time. More importantly, a minimal study has been conducted on JSP webshells.Zhang et al. [8] proposed a research method for JSP webshells. They used TF-IDF to count the importance of JSP compiled bytecode and used XGBoost to classify them. Their method utilizes statistical features without considering contextual information between texts, so detection is ineffective. Liu et al. [9] directly divided the original JSP code, extracted word vectors using Word2vec, and then used a bidirectional GRU with attention for classification. While some benign JSP code now employs antitheft methods like obfuscation and encryption, this method of recognizing the original JSP code directly has limited efficiency in detecting JSP webshells that also employ obfuscation and encryption. Additionally, we employ a more advanced method of word vector extraction than Word2vec, namely, BERT word embedding.Since JSP and PHP webshells have distinct properties, we cannot simply apply the PHP webshell detection approach to JSP for the following reasons.(1)
The opcode of the PHP files and the bytecode of the JSP files are compiled differently. Typically, the PHP files’ opcode is compiled using the Zend engine, whereas the JSP files’ bytecode is compiled using Tomcat.(2)
Compared to standard PHP code samples, PHP webshells often contain shorter text. However, the JSP webshells have the same amount of text as the standard sample but a significantly greater number of words. It indicates that most webshell files of JSP type include malicious code with several functions and complexities.(3)
There are ready libraries to extract opcode for PHP, and many freely available PHP webshell datasets on the Internet exist. Webshells written in the JSP language lack this convenience.As a result, the next discussion will concentrate on webshell identification for popular programming languages such as PHP.


### 2.2. Classification of Webshell

There are various ways to classify webshells. According to the programming language used, they can be classified as PHP, JSP, ASP webshells, etc. According to the size of the webshell, they can be classified as big Trojans, small Trojans, and one-word Trojans. Among them, the big Trojan not only connects to the database but also has some graphical operability interface. The code presentation of webshells can be divided into noncoded webshells written directly in ordinary files, coded webshells with encoding such as base64 or some obfuscated encoding, and fileless webshells that launch attacks by directly using dangerous command execution functions in web application code.


### 2.3. Method of Webshell Detection

In general, webshell detection can be classified into two categories: static detection and dynamic detection.Dynamic detection is usually based on the detection of web traffic [10]. For instance, Wu et al. [11] employ reinforcement learning to determine whether the web traffic originates from a webshell. However, while dynamic detection can give real-time detection, it also uses a significant amount of system resources. As a result, it is difficult to implement in real applications.The research is generally based on text features and some static features for static detection. Information entropy, longest string, compression rate, etc. are included in static features. Due of the inherent limits of static features, their detection methods are frequently integrated with text information [12–14]. In addition, some research approaches for PHP language [15–17] focus on using machine learning methods to classify the opcode obtained after processing the PHP language directly. Yong et al. [18] transform PHP language into opcodes and use statistical methods such as TF-IDF and bag-of-words model to extract features and use deep learning models for classification. Wu et al. [19] perform the detection of logs on the server. Since webshell programs run with different network traffic than normal, webshells can be accurately detected.Fass et al. [20] build abstract syntax trees from code to extract features. To detect malicious code in Javascript, Ndichu et al. [21] use abstract syntax trees to extract features and exploit the contextual information in the text. Abstract syntax trees [22–24] is an abstract representation of the source code structure to represent the syntactic structure of the programming language in a tree-like structure, which is good at extracting textual information and has many applications in the Javascript language.Machine learning and deep learning methods [25–27] are used in webshell detection. Among the deep learning methods used, CNN [28–30] is often used for webshell detection. Tian et al. [31] first applied CNN to webshell detection. Recurrent Neural Networks are good at extracting text sequence information, so they are also used for webshell detection [32]. Zhou et al. [33] use a single-layer LSTM to extract text sequence information from PHP opcode and conclude that using a two-layer or more-layer LSTM will reduce the detection effectiveness.Webshell can be transformed into sequential text information by some means. Word2vec is a tool that can transform text information into word vectors and is used for webshell detection [34]. Tong [35] extracted word vectors from the native webshell using BERT. This method is advantageous since it eliminates the requirement to execute targeted compilation for various programming languages. Nevertheless, it is ineffective against encryption and obfuscation when utilizing the native webshell. Min [36] used BERT to PHP-compiled opcodes and employ a modified Bi-LSTM network for detection. This method yields excellent results, but it continues to prioritize the PHP language above the JSP language. This paper proposes a new method for extracting word vectors from transformed bytecode from JSP webshells. We are the first study to use BERT to extract word vectors from transformed bytecode from JSP webshells to the best of our knowledge.


## 2.1. Outline

Webshell is a typical backdoor program left behind after a web server is compromised, usually in the form of a dynamic web script that is a command execution environment. Concealment and flexibility are the characteristics of webshells, so users may be unaware of hackers using webshells to control the system steadily over time. More importantly, a minimal study has been conducted on JSP webshells.Zhang et al. [8] proposed a research method for JSP webshells. They used TF-IDF to count the importance of JSP compiled bytecode and used XGBoost to classify them. Their method utilizes statistical features without considering contextual information between texts, so detection is ineffective. Liu et al. [9] directly divided the original JSP code, extracted word vectors using Word2vec, and then used a bidirectional GRU with attention for classification. While some benign JSP code now employs antitheft methods like obfuscation and encryption, this method of recognizing the original JSP code directly has limited efficiency in detecting JSP webshells that also employ obfuscation and encryption. Additionally, we employ a more advanced method of word vector extraction than Word2vec, namely, BERT word embedding.Since JSP and PHP webshells have distinct properties, we cannot simply apply the PHP webshell detection approach to JSP for the following reasons.(1)
The opcode of the PHP files and the bytecode of the JSP files are compiled differently. Typically, the PHP files’ opcode is compiled using the Zend engine, whereas the JSP files’ bytecode is compiled using Tomcat.(2)
Compared to standard PHP code samples, PHP webshells often contain shorter text. However, the JSP webshells have the same amount of text as the standard sample but a significantly greater number of words. It indicates that most webshell files of JSP type include malicious code with several functions and complexities.(3)
There are ready libraries to extract opcode for PHP, and many freely available PHP webshell datasets on the Internet exist. Webshells written in the JSP language lack this convenience.As a result, the next discussion will concentrate on webshell identification for popular programming languages such as PHP.


## 2.2. Classification of Webshell

There are various ways to classify webshells. According to the programming language used, they can be classified as PHP, JSP, ASP webshells, etc. According to the size of the webshell, they can be classified as big Trojans, small Trojans, and one-word Trojans. Among them, the big Trojan not only connects to the database but also has some graphical operability interface. The code presentation of webshells can be divided into noncoded webshells written directly in ordinary files, coded webshells with encoding such as base64 or some obfuscated encoding, and fileless webshells that launch attacks by directly using dangerous command execution functions in web application code.


## 2.3. Method of Webshell Detection

In general, webshell detection can be classified into two categories: static detection and dynamic detection.Dynamic detection is usually based on the detection of web traffic [10]. For instance, Wu et al. [11] employ reinforcement learning to determine whether the web traffic originates from a webshell. However, while dynamic detection can give real-time detection, it also uses a significant amount of system resources. As a result, it is difficult to implement in real applications.The research is generally based on text features and some static features for static detection. Information entropy, longest string, compression rate, etc. are included in static features. Due of the inherent limits of static features, their detection methods are frequently integrated with text information [12–14]. In addition, some research approaches for PHP language [15–17] focus on using machine learning methods to classify the opcode obtained after processing the PHP language directly. Yong et al. [18] transform PHP language into opcodes and use statistical methods such as TF-IDF and bag-of-words model to extract features and use deep learning models for classification. Wu et al. [19] perform the detection of logs on the server. Since webshell programs run with different network traffic than normal, webshells can be accurately detected.Fass et al. [20] build abstract syntax trees from code to extract features. To detect malicious code in Javascript, Ndichu et al. [21] use abstract syntax trees to extract features and exploit the contextual information in the text. Abstract syntax trees [22–24] is an abstract representation of the source code structure to represent the syntactic structure of the programming language in a tree-like structure, which is good at extracting textual information and has many applications in the Javascript language.Machine learning and deep learning methods [25–27] are used in webshell detection. Among the deep learning methods used, CNN [28–30] is often used for webshell detection. Tian et al. [31] first applied CNN to webshell detection. Recurrent Neural Networks are good at extracting text sequence information, so they are also used for webshell detection [32]. Zhou et al. [33] use a single-layer LSTM to extract text sequence information from PHP opcode and conclude that using a two-layer or more-layer LSTM will reduce the detection effectiveness.Webshell can be transformed into sequential text information by some means. Word2vec is a tool that can transform text information into word vectors and is used for webshell detection [34]. Tong [35] extracted word vectors from the native webshell using BERT. This method is advantageous since it eliminates the requirement to execute targeted compilation for various programming languages. Nevertheless, it is ineffective against encryption and obfuscation when utilizing the native webshell. Min [36] used BERT to PHP-compiled opcodes and employ a modified Bi-LSTM network for detection. This method yields excellent results, but it continues to prioritize the PHP language above the JSP language. This paper proposes a new method for extracting word vectors from transformed bytecode from JSP webshells. We are the first study to use BERT to extract word vectors from transformed bytecode from JSP webshells to the best of our knowledge.


## 3. Methodology

### 3.1. Architecture

In this paper, we propose an effective JSP webshell detection model. Our model is divided into three parts: data preprocessing, word vector extraction, and the machine learning classification model. The architecture designed is shown in Figure1. In data preprocessing, we convert the original JSP file into a servlet in the form of a Java class through Tomcat Server and then compile it to generate bytecode for extracting text features. In the word vector extraction stage, we use BERT’s tokenizer to convert the bytecode into its corresponding token id and mask and then input the obtained token id and mask into the pretrained BERT model for fine-tuning to obtain the word vector. Finally, we input the word vector into our machine learning model, which will be able to determine whether the output word vector belongs to a JSP webshell or not.Figure 1
The architecture of our model.


### 3.2. Data Preprocessing

JSP serves user requests using the Java language as a script and can collaborate with other Java programs on the server to address sophisticated business requirements. Java code is embedded in a static JSP page and used as a template to dynamically generate the parts of web pages. Some webshells, in addition to Java code, also contain some HTML code for displaying visual interfaces. Certain JSP webshells may also contain obfuscated or encrypted code, making it difficult to extract functionality straight from the source JSP files. We must process the original JSP file in order to convert it to a text file and then extract the text information directly from the text file.After deploying the JSP to the server, the user can view the rendered JSP page by navigating to the server’s JSP file. When a user first accesses a JSP file, the server converts it to an executable servlet in the form of a Java class. Since Java is a platform-independent language, the same JSP code can be written once and run on a multitude of different operating systems. After the first request, the servlet is saved on the server, and subsequent requests for the JSP file do not require recompilation.Javap is a Java class file decomposer that is frequently used for decomposition of class files. It can decompile and inspect the bytecode generated by the Java compiler. We can use the commandjavap−c to get the bytecode in the Java class generated by the servlet after compilation. Following compilation, we can obtain a text file containing bytecode. To extract successive bytecodes from text files, regular expressions are utilized. The bytecode before and after regular expression extraction is shown in Figure 2.Figure 2
The file before extraction with regular expressions.


#### 3.2.1. Tomcat Server

We use Tomcat Server to convert raw JSP files into Java class in our model. Tomcat Server is the preferred development and debugging environment for JSP applications, particularly for small and medium-sized systems and applications with few concurrent users. We set up Tomcat Server on port 8080 of the local server and then put the original JSP files intowebapps/jspproject under the Tomcat Server directory. Following that, we connect to the local server’s port 8080. If the compilation succeeds, we can find the compiled Java class file inside the Tomcat Server directory in the /work/Catalina/localhost/jspproject/org/apache/jsp directory under the Tomcat Server directory.


#### 3.2.2. Bytecode

JSP code usually contains two parts, one is HTML code, which is usually used to display the web page, and the other is Java code contained by <%> tags that can be used to handle the interaction of the web page with the server. The bytecode processing flow is shown in Figure3. Due to the fact that the original JSP code lacks substantial textual features, some transformation of the JSP code is required. Tomcat transforms the Java class file into hexadecimal data that can be translated, and the translated bytecode provides some semantic information. Bytecode is cross-platform because it still needs to be compiled in the JVM (Java Virtual Machine).Figure 3
Bytecode processing flow.


### 3.3. Feature Extraction

#### 3.3.1. Word2vec-Based Word Embedding

Word2vec [37] is a tool for converting words into real-valued vectors and is one of the language models widely used in natural language processing for learning semantic knowledge from large text corpora unsupervised. Word2vec makes use of both the Skip-Gram and the Continued Bag-of-Words (CBOW) models. The Skip-Gram approach is used to determine the context of the current word, whereas the CBOW model predicts the target value based on the context, which is the method we utilize here. In our experiments, we obtain the bytecode, from which we can extract the word vector. Since the length of bytecode varies, the length of the word vector extracted from bytecode also varies. Therefore, a suitable threshold is chosen as the maximum length of the word vector. If the bytecode is too long, a part of it will be truncated, and conversely, if the bytecode is shorter than the threshold value, it will be filled with 0 to the threshold length. The vector of each word is high-dimensional.


#### 3.3.2. Word Segmentation

Before inputting the BERT model, the bytecode must be converted to token ID, and creating token ID requires word segmentation of the input text. There are numerous algorithms for segmenting words. Wordpiece is employed in BERT as a tool for word segmentation, and its main algorithm is the greedy matching algorithm. The Wordpiece algorithm’s fundamental concept is that each word in the bytecode is traversed forward from the last letter, and each traversal determines whether the first letter to the currently traversed position is a word in the vocabulary. If yes, the current word is divided into two subwords from the traversed point, and the preceding procedure is repeated for each subword. If the split subword does not begin the unsplit word, it should be preceded by a “##” indication to aid in assembly during decoding. This algorithm is shown in Algorithm 1.Algorithm 1: Word segmentation algorithm of BERT.
Input: A word to be split, WOutput: Lists of subwords, LL←EmptylistwhileWnotinvocabdoS←FindthelongestinvocabfromW.ifL is emptythenL.push(S)elseL.push(strcat(“##,”S))endW←GettheremainingsubwordfromWafterremovingS.endL.pushW


#### 3.3.3. BERT-Based Word Embedding

Traditional word vector models, such as Word2vec, are ineffective at explaining polysemous words, and it is difficult to distinguish their various meanings in various contexts. In 2018, Google proposed the BERT (Bidirectional Encoder Representation from Transformers) model, a pretrained linguistic representation model that employs encoders in a bidirectional transformer for feature extraction to generate dynamic word vectors [38], which can solve the problem of multiple meanings of a word. Simultaneously, BERT can be pretrained to obtain various levels of features on various network layers in order to better represent the complex properties of words. When applied to some downstream tasks, BERT usually requires simple fine-tuning to obtain good results. This paper proposes a method to apply BERT’s word embedding technique to JSP webshell detection. In the BERT structure, the CLS and SEP are two unique symbols. In the binary classification task, CLS is added to the beginning of the sentence to indicate that it is a single-sentence classification task, and the SEP token will not be added to the sentence in the binary classification task.After retrieving the bytecode for the original JSP file, we can use it to generate the word vector using BERT. BERT understands the input appropriately by utilizing the unique tokensCLS. The CLS token is unique. The output vector corresponding to the CLS symbols can be used as the semantic representation of the entire text in the classification task, as it contains the semantic information for each word in the text. Token embedding is a term that refers to the addition of tokenized sentences, which is combined with positional and segment embedding as the final input to BERT. After passing the bytecode to BERT, the classifier receives the output vector corresponding to our CLS symbols. The procedure of word vector extraction is shown in Figure 4. In the figure, the final output T is the feature vector. C represents the output vector corresponding to the input CLS. The characters on the same line as the CLS token are the bytecode after word segmentation, and these characters are processed to get the corresponding code; for example, CLS corresponds to the code 102.Figure 4
The process of word vector extraction.


### 3.4. Classification Model

Since Word2vec and BERT generate word vectors with sequential contextual data, they can be classified using the Bi-LSTM classifier. In comparison, for webshell detection, which is a binary classification problem, XGBoost may be utilized as an excellent classification model. As a result, we compare the performance of the two classifiers.


#### 3.4.1. XGBoost

XGBoost is short for Extreme Gradient Boosting. XGBoost is a toolkit for parallelizing the Boost tree with a focus on speed, effectiveness, and portability. It is an upgrade on the standard GBDT (Gradient Boosting Decision Tree) algorithm by using a loss function, regularization, and several other techniques to enhance the result.XGBoost achieves excellent prediction accuracy by adding a new tree to fit the difference between the previous tree’s forecast result and the actual value and establishing a new model as the foundation for subsequent model learning. If we setF to be the set of all decision tree models, yc to be the c th prediction of the model, and fkuc to be the output of the k th decision tree model corresponding to the c th input feature uc, K to be the total number of decision trees, then the prediction of the model is(1)yc=∑k=1Kfkuc,fk∈F.Similarly, if we setYm to be the objective function for iterations up to the m th round, qyc,y^cm is the error function between the true value yc and the predicted value y^cm of the m th iteration, and the error function gradually makes the prediction result keep approximating the true value by continuous learning. Then Ym can be expressed as follows:(2)Ym=qyc,y^cm+Ω,where Ω is the penalty term; let γ, λ be the coefficient matrix of the number of leaves and the fraction of leaf nodes, respectively, and let T be the number of leaves in each tree and ω be the set consisting of the fraction of leaf nodes in each tree; then Ω can be expressed as follows:(3)Ω=γT+12λω2.The word vectors are extracted from the JSP file, and we need a capable classifier with good performance and accuracy. XGBoost is fast, accurate, and adaptable and often performs well in handling binary classification tasks. Therefore, we chose XGBoost as the classifier for word vectors. The word vectors extracted from BERT are used to input to XGBoost. Before we acquire the final prediction results, we need to use the input word vectors to train XGBoost. First, the word vectors are input to the first tree, and after we get the output results of the first tree, we add a new tree, which makes the word vectors obtained from BERT input to these two trees such that the objective function obtained is minimized. And so on, the training of XGBoost can be completed. The word vectors are input to each tree of XGBoost, and each tree is given the probability score that its word vector belongs to a webshell. The predicted probability scores of all the trees are added to yield the probability that the original JSP file is a webshell.


### 3.5. Bi-LSTM

While RNNs are effective in extracting sequence features, they may encounter issues such as exploding gradient and vanishing gradient. The forgetting gate, input gate, and output gate are all included in the LSTM to successfully solve the above concerns. The following formula is used to calculate a standard LSTM unit:(4)X=ht−1wt,σz=11+e−z,ft=σWfX+bf,it=σWiX+bi,ot=σWoX+bo,ct=ft⊗ct−1+it⊗WcX+bc,ht=ot⊗tanhct,where ⊗ denotes the dot product of the corresponding elements, σ denotes the sigmoid function in the neural network, and Wf, Wi, Wo, and Wc are the weight matrices of the hidden layers, and bf, bi, bo, and bc are the bias vectors. ft, it, and ot are the forgetting gate, input gate, and output gate, respectively, ht is the output at moment t, and ct is the state of the final memory unit.The LSTM has a long-time memory function and is well-suited for processing long sequences. However, since the LSTM is a simple one-way propagation neural network, it is incapable of connecting the information contained in preceding and subsequent texts for the purpose of learning. Bi-LSTM usually has a strong performance in classifying word vectors. When word vectors are input to the Bi-LSTM, the Bi-LSTM can efficiently learn the relationship between word vectors corresponding to operations such as pop from the stack and push to the stack. Through the intrinsic connection of these operations, Bi-LSTM can effectively analyze which word vectors belong to the webshell word vectors. Therefore, we employ Bi-LSTM as a control group of the classifier in our experiments to evaluate the effect of our experiments.


## 3.1. Architecture

In this paper, we propose an effective JSP webshell detection model. Our model is divided into three parts: data preprocessing, word vector extraction, and the machine learning classification model. The architecture designed is shown in Figure1. In data preprocessing, we convert the original JSP file into a servlet in the form of a Java class through Tomcat Server and then compile it to generate bytecode for extracting text features. In the word vector extraction stage, we use BERT’s tokenizer to convert the bytecode into its corresponding token id and mask and then input the obtained token id and mask into the pretrained BERT model for fine-tuning to obtain the word vector. Finally, we input the word vector into our machine learning model, which will be able to determine whether the output word vector belongs to a JSP webshell or not.Figure 1
The architecture of our model.


## 3.2. Data Preprocessing

JSP serves user requests using the Java language as a script and can collaborate with other Java programs on the server to address sophisticated business requirements. Java code is embedded in a static JSP page and used as a template to dynamically generate the parts of web pages. Some webshells, in addition to Java code, also contain some HTML code for displaying visual interfaces. Certain JSP webshells may also contain obfuscated or encrypted code, making it difficult to extract functionality straight from the source JSP files. We must process the original JSP file in order to convert it to a text file and then extract the text information directly from the text file.After deploying the JSP to the server, the user can view the rendered JSP page by navigating to the server’s JSP file. When a user first accesses a JSP file, the server converts it to an executable servlet in the form of a Java class. Since Java is a platform-independent language, the same JSP code can be written once and run on a multitude of different operating systems. After the first request, the servlet is saved on the server, and subsequent requests for the JSP file do not require recompilation.Javap is a Java class file decomposer that is frequently used for decomposition of class files. It can decompile and inspect the bytecode generated by the Java compiler. We can use the commandjavap−c to get the bytecode in the Java class generated by the servlet after compilation. Following compilation, we can obtain a text file containing bytecode. To extract successive bytecodes from text files, regular expressions are utilized. The bytecode before and after regular expression extraction is shown in Figure 2.Figure 2
The file before extraction with regular expressions.


### 3.2.1. Tomcat Server

We use Tomcat Server to convert raw JSP files into Java class in our model. Tomcat Server is the preferred development and debugging environment for JSP applications, particularly for small and medium-sized systems and applications with few concurrent users. We set up Tomcat Server on port 8080 of the local server and then put the original JSP files intowebapps/jspproject under the Tomcat Server directory. Following that, we connect to the local server’s port 8080. If the compilation succeeds, we can find the compiled Java class file inside the Tomcat Server directory in the /work/Catalina/localhost/jspproject/org/apache/jsp directory under the Tomcat Server directory.


### 3.2.2. Bytecode

JSP code usually contains two parts, one is HTML code, which is usually used to display the web page, and the other is Java code contained by <%> tags that can be used to handle the interaction of the web page with the server. The bytecode processing flow is shown in Figure3. Due to the fact that the original JSP code lacks substantial textual features, some transformation of the JSP code is required. Tomcat transforms the Java class file into hexadecimal data that can be translated, and the translated bytecode provides some semantic information. Bytecode is cross-platform because it still needs to be compiled in the JVM (Java Virtual Machine).Figure 3
Bytecode processing flow.


## 3.2.1. Tomcat Server

We use Tomcat Server to convert raw JSP files into Java class in our model. Tomcat Server is the preferred development and debugging environment for JSP applications, particularly for small and medium-sized systems and applications with few concurrent users. We set up Tomcat Server on port 8080 of the local server and then put the original JSP files intowebapps/jspproject under the Tomcat Server directory. Following that, we connect to the local server’s port 8080. If the compilation succeeds, we can find the compiled Java class file inside the Tomcat Server directory in the /work/Catalina/localhost/jspproject/org/apache/jsp directory under the Tomcat Server directory.


## 3.2.2. Bytecode

JSP code usually contains two parts, one is HTML code, which is usually used to display the web page, and the other is Java code contained by <%> tags that can be used to handle the interaction of the web page with the server. The bytecode processing flow is shown in Figure3. Due to the fact that the original JSP code lacks substantial textual features, some transformation of the JSP code is required. Tomcat transforms the Java class file into hexadecimal data that can be translated, and the translated bytecode provides some semantic information. Bytecode is cross-platform because it still needs to be compiled in the JVM (Java Virtual Machine).Figure 3
Bytecode processing flow.


## 3.3. Feature Extraction

### 3.3.1. Word2vec-Based Word Embedding

Word2vec [37] is a tool for converting words into real-valued vectors and is one of the language models widely used in natural language processing for learning semantic knowledge from large text corpora unsupervised. Word2vec makes use of both the Skip-Gram and the Continued Bag-of-Words (CBOW) models. The Skip-Gram approach is used to determine the context of the current word, whereas the CBOW model predicts the target value based on the context, which is the method we utilize here. In our experiments, we obtain the bytecode, from which we can extract the word vector. Since the length of bytecode varies, the length of the word vector extracted from bytecode also varies. Therefore, a suitable threshold is chosen as the maximum length of the word vector. If the bytecode is too long, a part of it will be truncated, and conversely, if the bytecode is shorter than the threshold value, it will be filled with 0 to the threshold length. The vector of each word is high-dimensional.


### 3.3.2. Word Segmentation

Before inputting the BERT model, the bytecode must be converted to token ID, and creating token ID requires word segmentation of the input text. There are numerous algorithms for segmenting words. Wordpiece is employed in BERT as a tool for word segmentation, and its main algorithm is the greedy matching algorithm. The Wordpiece algorithm’s fundamental concept is that each word in the bytecode is traversed forward from the last letter, and each traversal determines whether the first letter to the currently traversed position is a word in the vocabulary. If yes, the current word is divided into two subwords from the traversed point, and the preceding procedure is repeated for each subword. If the split subword does not begin the unsplit word, it should be preceded by a “##” indication to aid in assembly during decoding. This algorithm is shown in Algorithm 1.Algorithm 1: Word segmentation algorithm of BERT.
Input: A word to be split, WOutput: Lists of subwords, LL←EmptylistwhileWnotinvocabdoS←FindthelongestinvocabfromW.ifL is emptythenL.push(S)elseL.push(strcat(“##,”S))endW←GettheremainingsubwordfromWafterremovingS.endL.pushW


### 3.3.3. BERT-Based Word Embedding

Traditional word vector models, such as Word2vec, are ineffective at explaining polysemous words, and it is difficult to distinguish their various meanings in various contexts. In 2018, Google proposed the BERT (Bidirectional Encoder Representation from Transformers) model, a pretrained linguistic representation model that employs encoders in a bidirectional transformer for feature extraction to generate dynamic word vectors [38], which can solve the problem of multiple meanings of a word. Simultaneously, BERT can be pretrained to obtain various levels of features on various network layers in order to better represent the complex properties of words. When applied to some downstream tasks, BERT usually requires simple fine-tuning to obtain good results. This paper proposes a method to apply BERT’s word embedding technique to JSP webshell detection. In the BERT structure, the CLS and SEP are two unique symbols. In the binary classification task, CLS is added to the beginning of the sentence to indicate that it is a single-sentence classification task, and the SEP token will not be added to the sentence in the binary classification task.After retrieving the bytecode for the original JSP file, we can use it to generate the word vector using BERT. BERT understands the input appropriately by utilizing the unique tokensCLS. The CLS token is unique. The output vector corresponding to the CLS symbols can be used as the semantic representation of the entire text in the classification task, as it contains the semantic information for each word in the text. Token embedding is a term that refers to the addition of tokenized sentences, which is combined with positional and segment embedding as the final input to BERT. After passing the bytecode to BERT, the classifier receives the output vector corresponding to our CLS symbols. The procedure of word vector extraction is shown in Figure 4. In the figure, the final output T is the feature vector. C represents the output vector corresponding to the input CLS. The characters on the same line as the CLS token are the bytecode after word segmentation, and these characters are processed to get the corresponding code; for example, CLS corresponds to the code 102.Figure 4
The process of word vector extraction.


## 3.3.1. Word2vec-Based Word Embedding

Word2vec [37] is a tool for converting words into real-valued vectors and is one of the language models widely used in natural language processing for learning semantic knowledge from large text corpora unsupervised. Word2vec makes use of both the Skip-Gram and the Continued Bag-of-Words (CBOW) models. The Skip-Gram approach is used to determine the context of the current word, whereas the CBOW model predicts the target value based on the context, which is the method we utilize here. In our experiments, we obtain the bytecode, from which we can extract the word vector. Since the length of bytecode varies, the length of the word vector extracted from bytecode also varies. Therefore, a suitable threshold is chosen as the maximum length of the word vector. If the bytecode is too long, a part of it will be truncated, and conversely, if the bytecode is shorter than the threshold value, it will be filled with 0 to the threshold length. The vector of each word is high-dimensional.


## 3.3.2. Word Segmentation

Before inputting the BERT model, the bytecode must be converted to token ID, and creating token ID requires word segmentation of the input text. There are numerous algorithms for segmenting words. Wordpiece is employed in BERT as a tool for word segmentation, and its main algorithm is the greedy matching algorithm. The Wordpiece algorithm’s fundamental concept is that each word in the bytecode is traversed forward from the last letter, and each traversal determines whether the first letter to the currently traversed position is a word in the vocabulary. If yes, the current word is divided into two subwords from the traversed point, and the preceding procedure is repeated for each subword. If the split subword does not begin the unsplit word, it should be preceded by a “##” indication to aid in assembly during decoding. This algorithm is shown in Algorithm 1.Algorithm 1: Word segmentation algorithm of BERT.
Input: A word to be split, WOutput: Lists of subwords, LL←EmptylistwhileWnotinvocabdoS←FindthelongestinvocabfromW.ifL is emptythenL.push(S)elseL.push(strcat(“##,”S))endW←GettheremainingsubwordfromWafterremovingS.endL.pushW


## 3.3.3. BERT-Based Word Embedding

Traditional word vector models, such as Word2vec, are ineffective at explaining polysemous words, and it is difficult to distinguish their various meanings in various contexts. In 2018, Google proposed the BERT (Bidirectional Encoder Representation from Transformers) model, a pretrained linguistic representation model that employs encoders in a bidirectional transformer for feature extraction to generate dynamic word vectors [38], which can solve the problem of multiple meanings of a word. Simultaneously, BERT can be pretrained to obtain various levels of features on various network layers in order to better represent the complex properties of words. When applied to some downstream tasks, BERT usually requires simple fine-tuning to obtain good results. This paper proposes a method to apply BERT’s word embedding technique to JSP webshell detection. In the BERT structure, the CLS and SEP are two unique symbols. In the binary classification task, CLS is added to the beginning of the sentence to indicate that it is a single-sentence classification task, and the SEP token will not be added to the sentence in the binary classification task.After retrieving the bytecode for the original JSP file, we can use it to generate the word vector using BERT. BERT understands the input appropriately by utilizing the unique tokensCLS. The CLS token is unique. The output vector corresponding to the CLS symbols can be used as the semantic representation of the entire text in the classification task, as it contains the semantic information for each word in the text. Token embedding is a term that refers to the addition of tokenized sentences, which is combined with positional and segment embedding as the final input to BERT. After passing the bytecode to BERT, the classifier receives the output vector corresponding to our CLS symbols. The procedure of word vector extraction is shown in Figure 4. In the figure, the final output T is the feature vector. C represents the output vector corresponding to the input CLS. The characters on the same line as the CLS token are the bytecode after word segmentation, and these characters are processed to get the corresponding code; for example, CLS corresponds to the code 102.Figure 4
The process of word vector extraction.


## 3.4. Classification Model

Since Word2vec and BERT generate word vectors with sequential contextual data, they can be classified using the Bi-LSTM classifier. In comparison, for webshell detection, which is a binary classification problem, XGBoost may be utilized as an excellent classification model. As a result, we compare the performance of the two classifiers.


### 3.4.1. XGBoost

XGBoost is short for Extreme Gradient Boosting. XGBoost is a toolkit for parallelizing the Boost tree with a focus on speed, effectiveness, and portability. It is an upgrade on the standard GBDT (Gradient Boosting Decision Tree) algorithm by using a loss function, regularization, and several other techniques to enhance the result.XGBoost achieves excellent prediction accuracy by adding a new tree to fit the difference between the previous tree’s forecast result and the actual value and establishing a new model as the foundation for subsequent model learning. If we setF to be the set of all decision tree models, yc to be the c th prediction of the model, and fkuc to be the output of the k th decision tree model corresponding to the c th input feature uc, K to be the total number of decision trees, then the prediction of the model is(1)yc=∑k=1Kfkuc,fk∈F.Similarly, if we setYm to be the objective function for iterations up to the m th round, qyc,y^cm is the error function between the true value yc and the predicted value y^cm of the m th iteration, and the error function gradually makes the prediction result keep approximating the true value by continuous learning. Then Ym can be expressed as follows:(2)Ym=qyc,y^cm+Ω,where Ω is the penalty term; let γ, λ be the coefficient matrix of the number of leaves and the fraction of leaf nodes, respectively, and let T be the number of leaves in each tree and ω be the set consisting of the fraction of leaf nodes in each tree; then Ω can be expressed as follows:(3)Ω=γT+12λω2.The word vectors are extracted from the JSP file, and we need a capable classifier with good performance and accuracy. XGBoost is fast, accurate, and adaptable and often performs well in handling binary classification tasks. Therefore, we chose XGBoost as the classifier for word vectors. The word vectors extracted from BERT are used to input to XGBoost. Before we acquire the final prediction results, we need to use the input word vectors to train XGBoost. First, the word vectors are input to the first tree, and after we get the output results of the first tree, we add a new tree, which makes the word vectors obtained from BERT input to these two trees such that the objective function obtained is minimized. And so on, the training of XGBoost can be completed. The word vectors are input to each tree of XGBoost, and each tree is given the probability score that its word vector belongs to a webshell. The predicted probability scores of all the trees are added to yield the probability that the original JSP file is a webshell.


## 3.4.1. XGBoost

XGBoost is short for Extreme Gradient Boosting. XGBoost is a toolkit for parallelizing the Boost tree with a focus on speed, effectiveness, and portability. It is an upgrade on the standard GBDT (Gradient Boosting Decision Tree) algorithm by using a loss function, regularization, and several other techniques to enhance the result.XGBoost achieves excellent prediction accuracy by adding a new tree to fit the difference between the previous tree’s forecast result and the actual value and establishing a new model as the foundation for subsequent model learning. If we setF to be the set of all decision tree models, yc to be the c th prediction of the model, and fkuc to be the output of the k th decision tree model corresponding to the c th input feature uc, K to be the total number of decision trees, then the prediction of the model is(1)yc=∑k=1Kfkuc,fk∈F.Similarly, if we setYm to be the objective function for iterations up to the m th round, qyc,y^cm is the error function between the true value yc and the predicted value y^cm of the m th iteration, and the error function gradually makes the prediction result keep approximating the true value by continuous learning. Then Ym can be expressed as follows:(2)Ym=qyc,y^cm+Ω,where Ω is the penalty term; let γ, λ be the coefficient matrix of the number of leaves and the fraction of leaf nodes, respectively, and let T be the number of leaves in each tree and ω be the set consisting of the fraction of leaf nodes in each tree; then Ω can be expressed as follows:(3)Ω=γT+12λω2.The word vectors are extracted from the JSP file, and we need a capable classifier with good performance and accuracy. XGBoost is fast, accurate, and adaptable and often performs well in handling binary classification tasks. Therefore, we chose XGBoost as the classifier for word vectors. The word vectors extracted from BERT are used to input to XGBoost. Before we acquire the final prediction results, we need to use the input word vectors to train XGBoost. First, the word vectors are input to the first tree, and after we get the output results of the first tree, we add a new tree, which makes the word vectors obtained from BERT input to these two trees such that the objective function obtained is minimized. And so on, the training of XGBoost can be completed. The word vectors are input to each tree of XGBoost, and each tree is given the probability score that its word vector belongs to a webshell. The predicted probability scores of all the trees are added to yield the probability that the original JSP file is a webshell.


## 3.5. Bi-LSTM

While RNNs are effective in extracting sequence features, they may encounter issues such as exploding gradient and vanishing gradient. The forgetting gate, input gate, and output gate are all included in the LSTM to successfully solve the above concerns. The following formula is used to calculate a standard LSTM unit:(4)X=ht−1wt,σz=11+e−z,ft=σWfX+bf,it=σWiX+bi,ot=σWoX+bo,ct=ft⊗ct−1+it⊗WcX+bc,ht=ot⊗tanhct,where ⊗ denotes the dot product of the corresponding elements, σ denotes the sigmoid function in the neural network, and Wf, Wi, Wo, and Wc are the weight matrices of the hidden layers, and bf, bi, bo, and bc are the bias vectors. ft, it, and ot are the forgetting gate, input gate, and output gate, respectively, ht is the output at moment t, and ct is the state of the final memory unit.The LSTM has a long-time memory function and is well-suited for processing long sequences. However, since the LSTM is a simple one-way propagation neural network, it is incapable of connecting the information contained in preceding and subsequent texts for the purpose of learning. Bi-LSTM usually has a strong performance in classifying word vectors. When word vectors are input to the Bi-LSTM, the Bi-LSTM can efficiently learn the relationship between word vectors corresponding to operations such as pop from the stack and push to the stack. Through the intrinsic connection of these operations, Bi-LSTM can effectively analyze which word vectors belong to the webshell word vectors. Therefore, we employ Bi-LSTM as a control group of the classifier in our experiments to evaluate the effect of our experiments.


## 4. Evaluation

In this section, we will present the details of our dataset, the experimental setting, the procedure and details of the experiment, and the experiment’s conclusions.


### 4.1. Dataset

Due to the fact that the number of webshell files is significantly less than the number of benign files on a typical website, the number of benign files in our dataset exceeds the number of JSP webshells. We gathered 2903 JSP samples from open-source projects, which included 830 webshells and 2073 benign JSP files. Among them, we collected innocuous JSP samples from Github’s open-source projects, and owing to a dearth of JSP samples, we personally generated several JSP webshell samples, which included some JSP webshell samples that were obfuscated and encrypted to verify the correctness of model testing. The JSP webshell projects collected from Github are shown in Table1. After deduplicating the JSP samples gathered, including benign and webshell samples, we ended up with 2903 samples. In our trials, we converted them to Java classes and then turned them into bytecode files using the Tomcat Server.Table 1
Github projects lists.
NoSource#0https://github.com/ysrc/webshell-sample#1https://github.com/xl7dev/WebShell#2https://github.com/tutorial0/WebShell#3https://github.com/tennc/webshell#4https://github.com/tanjiti/webshellSample#5https://github.com/oneoneplus/webshell#6https://github.com/threedr3am/JSP-WebShells#7https://github.com/fr4nk404/Webshell-Collections


### 4.2. Experiment Design

To examine the effectiveness of our derived methods, we used both Word2vec and BERT word embedding methods for feature extraction. For the classification model, we used Bi-LSTM to compare it against XGBoost, which excels at solving classification issues, in the hope of obtaining the best performing model. In addition, we also used the Support Vector Machine algorithm and the more generalized RandomForest algorithm as comparison experiments. A total of eight sets of experiments were used to test the effectiveness of the model.Our experiments were all run on Kaggle [39], which offers machine learning competitions and provides a Jupyter environment with GPUs. We used an Intel(R) Xeon(R) CPU @ 2.00 GHz with 16 GB of RAM and a GPU of Tesla P100-PCIE. Pytorch is used to implement Bi-LSTM, and the XGBoost package is used to compute XGBoost in Python.The experiments were conducted using 5-fold cross validation. The dataset is partitioned into five equal portions, with one serving as the test set and the other as the training set. To reduce the effect of randomness in dataset splitting, we run the experiment five times and take the average of the five results as our experimental result. Each experiment has 2322 JSP samples in the training set and 581 JSP samples in the testing set. To eliminate the randomness effect when the dataset is disrupted, we used a random number seed 42 to ensure that the training and test sets for each division are the same when comparing the trials.Through the Tomcat Server, the original JSP file samples were converted to Java class files. Following that, the Java class file was compiled into a bytecode-containing text file. Regular expressions were employed to extract the bytecode. Four distinct sets of experiments were used to compare and confirm our model’s performance.


### 4.3. Evaluation Metrics

The model was evaluated using four metrics: accuracy, precision, recall, and f1 score. Additionally, we displayed receiver operating characteristic (ROC) curves to evaluate the model’s detection performance on the test set.After determining the model with the best performance based on the aforementioned measures, we create the model’s ROC (receiver operating characteristics) curve to visually assess its performance and determine the AUC value (area under curve). The AUC can be stated as follows:(5)AUC=∑i∈positiveclassranki−M1+M/2M×N,where M denotes the number of positive samples and N denotes the number of negative samples. ranki denotes the sum of the rankings of the probability of predicting all positive samples as positive (the sample with the highest probability is rankM+N).


### 4.4. Experiment Result

Using the aforementioned methodologies, we designed tests and evaluated each combination of feature vector extraction methods and classification models. The results are shown in Table2.Table 2
The evaluation metrics results.
Feature extractionClassification algorithmEvaluation metricsAccuracyPrecisionRecallF1 scoreWord2vecBi-LSTM0.98000.96270.96850.9654BERTBi-LSTM0.98520.97030.97890.9746Word2vecXGBoost0.98590.98050.97010.9753BERTXGBoost0.99140.98680.98030.9835Word2vecRandomForest0.98440.99010.95620.9729BERTRandomForest0.98420.98650.95890.9724Word2vecSupport vector machine0.98520.99260.95610.9740BERTSupport vector machine0.98280.96520.97510.9700The evaluation results in the table represent the maximum values that can be obtained using the model’s various parameters. The evaluation results in the table represent the maximum values that can be obtained using the model’s various parameters. Model 1, Model 3, Model 5, and Model 7 all use Word2vec for word vector extraction, while Model 2, Model 4, Model 6, and Model 8 employ BERT as a tool for word vector extraction. They are classified by four machine algorithms, including XGBoost, which is strong at binary classification tasks, and Bi-LSTM algorithm, which is good at classifying word vectors, so we can achieve the best classification results. Our model achieves 99.14% accuracy, 98.68% precision, 98.03% for recall, and 98.35% for the final f1 score, all of which are greater than the accuracy, precision, recall, and final f1 score of other models, demonstrating that the BERT word embedding strategy is perfect for webshell detection. The hyperparameters of XGBoost are set as follows:max_depth=6, min_child_weight=1, n_estimators=150, gamma=0. The rest of the hyperparameters are set by default.From the above experimental results, we can conclude that BERT performs better than Word2vec as a word vector extraction technique for bytecode of JSP files, allowing for a more accurate classification of the original JSP files as webshell and normal files. We have analyzed the two most crucial points: First, Word2vec has only one representation for each token, so it cannot understand some cases of polysemy. Some words in bytecode do not exist or have incorrect meanings in Word2vec’s vocab, which may reduce the accuracy of the word vectors extracted from bytecode by Word2vec. BERT can successfully comprehend polysemous words in context, allowing word vectors derived from bytecode to be comprehended more precisely in contextual relationships and with fewer errors. Second, Word2vec’s vocabulary is fairly extensive, containing up to one million words. In this situation, performance and accuracy must be compromised. However, BERT as a pretrained model and the Byte-Pair Encoding tokenizer it possesses contain appropriate levels of vocabulary. Therefore, in most cases, BERT can be used with a high degree of accuracy.To visualize the classification results, we plotted the receiver operating characteristic (ROC) curves for several feature extraction and classification models, as shown in Figure5. As can be observed, the model performs best when BERT is used to extract the word vectors and then XGBoost is used as the classification model. Thus, we use this model to recognize JSP webshells.Figure 5
ROC curves of the different models.Zhang et al. extracted statistical features from bytecode using the TF-IDF technique and classified them using the XGBoost classifier. However, this technique does not take into account bytecode context information. Liu et al. extract word vectors from the source JSP files and classify them using Bi-GRU with attention. It is worth noting that this straight word vector extraction of raw JSP files is insufficiently powerful to discover obfuscated and encrypted code files. Due to the possibility that our dataset differs from theirs, we conducted trials using our dataset, the results of which are displayed in Table3.Table 3
Model performance with related works.
AccuracyPrecisionRecallF1 scoreZhang et al. [8]0.98480.97750.97060.9739Liu et al. [9]0.98730.98440.97180.9780Our model0.99140.98680.98030.9835


### 4.5. Case Study

Today’s webshells employ obfuscation, encryption, and a variety of bypass mechanisms, and conventional webshell detection models are not as efficient as they could be at detecting such webshells. To evaluate our model’s effectiveness against such webshells, a total of 393 webshells were used to check whether the obfuscated and encrypted webshells could be detected correctly. The main sources of the encrypted and obfuscated webshells are shown in Table4 below. The project #0 is a collection of several webshells that can bypass the detection of professional tools. The projects #1 and #2 are generators of JSP webshells. The project #1 can generate different JSP webshells with obfuscation each time. The webshell generator in project #2 has a variety of features, including eleven bypass methods such as dynamic compilation using javac, loading bytecode with ClassLoader, dynamically generating bytecode with ASM, and loading BCEL bytecode, and Unicode encoding of parameters based on any of the bypass methods. In order to check the robustness of our model, we use our model and other models dedicated to detecting JSP webshells mentioned in the references for these 393 special JSP webshell files, respectively.Table 4
The main source of encrypted or obfuscated JSP webshells.
IndexGithub source#0LandGrey/webshell-detect-bypass#1pureqh/webshell#2Sec-Fork/JSPHorseThe original JSP webshell files were translated to bytecode using the same method and then evaluated using our model. Due to the fact that all created files are webshell files, we chose accuracy as the evaluation metric. The results of the test are shown in Table5. From the experimental results, we see that our model can still have excellent detection results for encrypted and obfuscated JSP webshell files, and we can conclude that the means to combat obfuscation and encryption are weaker when using the source code directly for detection. The experimental results show that our model still works quite well for various encrypted and obfuscated JSP webshells with 97.96% accuracy, indicating our proposed method’s good robustness.Table 5
Results of webshell detection with encryption and obfuscation.
TP (true positive)AccuracyZhang et al. [8]3790.9644Liu et al. [9]3560.9059Our model3850.9796


## 4.1. Dataset

Due to the fact that the number of webshell files is significantly less than the number of benign files on a typical website, the number of benign files in our dataset exceeds the number of JSP webshells. We gathered 2903 JSP samples from open-source projects, which included 830 webshells and 2073 benign JSP files. Among them, we collected innocuous JSP samples from Github’s open-source projects, and owing to a dearth of JSP samples, we personally generated several JSP webshell samples, which included some JSP webshell samples that were obfuscated and encrypted to verify the correctness of model testing. The JSP webshell projects collected from Github are shown in Table1. After deduplicating the JSP samples gathered, including benign and webshell samples, we ended up with 2903 samples. In our trials, we converted them to Java classes and then turned them into bytecode files using the Tomcat Server.Table 1
Github projects lists.
NoSource#0https://github.com/ysrc/webshell-sample#1https://github.com/xl7dev/WebShell#2https://github.com/tutorial0/WebShell#3https://github.com/tennc/webshell#4https://github.com/tanjiti/webshellSample#5https://github.com/oneoneplus/webshell#6https://github.com/threedr3am/JSP-WebShells#7https://github.com/fr4nk404/Webshell-Collections


## 4.2. Experiment Design

To examine the effectiveness of our derived methods, we used both Word2vec and BERT word embedding methods for feature extraction. For the classification model, we used Bi-LSTM to compare it against XGBoost, which excels at solving classification issues, in the hope of obtaining the best performing model. In addition, we also used the Support Vector Machine algorithm and the more generalized RandomForest algorithm as comparison experiments. A total of eight sets of experiments were used to test the effectiveness of the model.Our experiments were all run on Kaggle [39], which offers machine learning competitions and provides a Jupyter environment with GPUs. We used an Intel(R) Xeon(R) CPU @ 2.00 GHz with 16 GB of RAM and a GPU of Tesla P100-PCIE. Pytorch is used to implement Bi-LSTM, and the XGBoost package is used to compute XGBoost in Python.The experiments were conducted using 5-fold cross validation. The dataset is partitioned into five equal portions, with one serving as the test set and the other as the training set. To reduce the effect of randomness in dataset splitting, we run the experiment five times and take the average of the five results as our experimental result. Each experiment has 2322 JSP samples in the training set and 581 JSP samples in the testing set. To eliminate the randomness effect when the dataset is disrupted, we used a random number seed 42 to ensure that the training and test sets for each division are the same when comparing the trials.Through the Tomcat Server, the original JSP file samples were converted to Java class files. Following that, the Java class file was compiled into a bytecode-containing text file. Regular expressions were employed to extract the bytecode. Four distinct sets of experiments were used to compare and confirm our model’s performance.


## 4.3. Evaluation Metrics

The model was evaluated using four metrics: accuracy, precision, recall, and f1 score. Additionally, we displayed receiver operating characteristic (ROC) curves to evaluate the model’s detection performance on the test set.After determining the model with the best performance based on the aforementioned measures, we create the model’s ROC (receiver operating characteristics) curve to visually assess its performance and determine the AUC value (area under curve). The AUC can be stated as follows:(5)AUC=∑i∈positiveclassranki−M1+M/2M×N,where M denotes the number of positive samples and N denotes the number of negative samples. ranki denotes the sum of the rankings of the probability of predicting all positive samples as positive (the sample with the highest probability is rankM+N).


## 4.4. Experiment Result

Using the aforementioned methodologies, we designed tests and evaluated each combination of feature vector extraction methods and classification models. The results are shown in Table2.Table 2
The evaluation metrics results.
Feature extractionClassification algorithmEvaluation metricsAccuracyPrecisionRecallF1 scoreWord2vecBi-LSTM0.98000.96270.96850.9654BERTBi-LSTM0.98520.97030.97890.9746Word2vecXGBoost0.98590.98050.97010.9753BERTXGBoost0.99140.98680.98030.9835Word2vecRandomForest0.98440.99010.95620.9729BERTRandomForest0.98420.98650.95890.9724Word2vecSupport vector machine0.98520.99260.95610.9740BERTSupport vector machine0.98280.96520.97510.9700The evaluation results in the table represent the maximum values that can be obtained using the model’s various parameters. The evaluation results in the table represent the maximum values that can be obtained using the model’s various parameters. Model 1, Model 3, Model 5, and Model 7 all use Word2vec for word vector extraction, while Model 2, Model 4, Model 6, and Model 8 employ BERT as a tool for word vector extraction. They are classified by four machine algorithms, including XGBoost, which is strong at binary classification tasks, and Bi-LSTM algorithm, which is good at classifying word vectors, so we can achieve the best classification results. Our model achieves 99.14% accuracy, 98.68% precision, 98.03% for recall, and 98.35% for the final f1 score, all of which are greater than the accuracy, precision, recall, and final f1 score of other models, demonstrating that the BERT word embedding strategy is perfect for webshell detection. The hyperparameters of XGBoost are set as follows:max_depth=6, min_child_weight=1, n_estimators=150, gamma=0. The rest of the hyperparameters are set by default.From the above experimental results, we can conclude that BERT performs better than Word2vec as a word vector extraction technique for bytecode of JSP files, allowing for a more accurate classification of the original JSP files as webshell and normal files. We have analyzed the two most crucial points: First, Word2vec has only one representation for each token, so it cannot understand some cases of polysemy. Some words in bytecode do not exist or have incorrect meanings in Word2vec’s vocab, which may reduce the accuracy of the word vectors extracted from bytecode by Word2vec. BERT can successfully comprehend polysemous words in context, allowing word vectors derived from bytecode to be comprehended more precisely in contextual relationships and with fewer errors. Second, Word2vec’s vocabulary is fairly extensive, containing up to one million words. In this situation, performance and accuracy must be compromised. However, BERT as a pretrained model and the Byte-Pair Encoding tokenizer it possesses contain appropriate levels of vocabulary. Therefore, in most cases, BERT can be used with a high degree of accuracy.To visualize the classification results, we plotted the receiver operating characteristic (ROC) curves for several feature extraction and classification models, as shown in Figure5. As can be observed, the model performs best when BERT is used to extract the word vectors and then XGBoost is used as the classification model. Thus, we use this model to recognize JSP webshells.Figure 5
ROC curves of the different models.Zhang et al. extracted statistical features from bytecode using the TF-IDF technique and classified them using the XGBoost classifier. However, this technique does not take into account bytecode context information. Liu et al. extract word vectors from the source JSP files and classify them using Bi-GRU with attention. It is worth noting that this straight word vector extraction of raw JSP files is insufficiently powerful to discover obfuscated and encrypted code files. Due to the possibility that our dataset differs from theirs, we conducted trials using our dataset, the results of which are displayed in Table3.Table 3
Model performance with related works.
AccuracyPrecisionRecallF1 scoreZhang et al. [8]0.98480.97750.97060.9739Liu et al. [9]0.98730.98440.97180.9780Our model0.99140.98680.98030.9835


## 4.5. Case Study

Today’s webshells employ obfuscation, encryption, and a variety of bypass mechanisms, and conventional webshell detection models are not as efficient as they could be at detecting such webshells. To evaluate our model’s effectiveness against such webshells, a total of 393 webshells were used to check whether the obfuscated and encrypted webshells could be detected correctly. The main sources of the encrypted and obfuscated webshells are shown in Table4 below. The project #0 is a collection of several webshells that can bypass the detection of professional tools. The projects #1 and #2 are generators of JSP webshells. The project #1 can generate different JSP webshells with obfuscation each time. The webshell generator in project #2 has a variety of features, including eleven bypass methods such as dynamic compilation using javac, loading bytecode with ClassLoader, dynamically generating bytecode with ASM, and loading BCEL bytecode, and Unicode encoding of parameters based on any of the bypass methods. In order to check the robustness of our model, we use our model and other models dedicated to detecting JSP webshells mentioned in the references for these 393 special JSP webshell files, respectively.Table 4
The main source of encrypted or obfuscated JSP webshells.
IndexGithub source#0LandGrey/webshell-detect-bypass#1pureqh/webshell#2Sec-Fork/JSPHorseThe original JSP webshell files were translated to bytecode using the same method and then evaluated using our model. Due to the fact that all created files are webshell files, we chose accuracy as the evaluation metric. The results of the test are shown in Table5. From the experimental results, we see that our model can still have excellent detection results for encrypted and obfuscated JSP webshell files, and we can conclude that the means to combat obfuscation and encryption are weaker when using the source code directly for detection. The experimental results show that our model still works quite well for various encrypted and obfuscated JSP webshells with 97.96% accuracy, indicating our proposed method’s good robustness.Table 5
Results of webshell detection with encryption and obfuscation.
TP (true positive)AccuracyZhang et al. [8]3790.9644Liu et al. [9]3560.9059Our model3850.9796


## 5. Conclusion

In this paper, we present a model for detecting JSP webshells and demonstrate the usage of BERT in transformed bytecode from JSP webshell for the first time. Finally, our model achieves 99.14% accuracy and significantly greater precision, recall, and f1 score than prior JSP webshell identification models. The Tomcat Server is used to convert JSP files to Java class files, which are subsequently converted to bytecode. The bytecode is processed using BERT-based word vector extraction, and the extracted word vectors are classified using XGBoost. Additionally, we compare our model to the conventional webshell detection model. Previously, Word2vec was frequently used to extract word vectors for opcode or bytecode extraction, and the model suggested in this study may provide researchers with a fresh way of thinking.This study still contains certain limitations. First, there are few publicly available JSP webshell datasets on the Internet. The experimental results could be more convincing if we could get more data. Second, the method in this paper needs to compile JSP webshell into the corresponding bytecode; hence porting this method to other programming languages is challenging. In the future, we can work on the following areas.(1)
We can conduct experiments on the methods mentioned in this manuscript on webshells of other programming languages to check the generality of the method in this manuscript.(2)
The literal meaning of the bytecode we extracted is not obvious, while BERT can understand the context effectively. Therefore, we may be able to find a way to semanticize bytecode to improve the method in this manuscript.The webshell detection method, which uses BERT to extract word vectors, was demonstrated to be effective experimentally. We can attempt to extend this method in the future to detect webshells in other programming languages.



---

*Source: 4315829-2022-08-31.xml*