# Lightweight Detection Method of Obfuscated Landing Sites Based on the AST Structure and Tokens
## 4 Experiment
### Lightweight AST
#### 4.2.3 Classification Accuracy of Obfuscation Type Using Lightweight AST  
We here conduct an experiment in which we compare the accuracies when obfuscation classification is respectively based on the conventional AST and the lightweight AST. First, we prepare an experimental JavaScript set that consists of both a non-obfuscated JavaScript set (1000 JavaScript) and obfuscated JavaScript sets of four types (125 JavaScript sets per type by using four different obfuscation tools). Then, we randomly select a representative JavaScript from the type 1 obfuscated JavaScript set and convert it into a conventional AST. Next, we convert all JavaScript from the experimental JavaScript set into conventional ASTs and measure their similarities using a representative JavaScript. We set a threshold to classify the JavaScript that has a structural similarity exceeding specific criteria as obfuscated JavaScript of type 1. We repeat the same process for types 2, 3, and 4. In this way, we can measure the accuracies when the obfuscation classification is based on the conventional AST. We can likewise measure the accuracies when the obfuscation classification is based on the lightweight AST. Figure 3 shows the structural similarities measured between the representative type 1 obfuscated JavaScript and all JavaScript in the conventional AST. The structural similarity ranges from 0 to 1. The larger the number is, the higher the similarity is. Even though the representative JavaScript is selected from the type 1 obfuscated JavaScript set, the latter does not show comparatively higher similarities than the others. At this point, we must set a threshold to classify the type 1 obfuscated JavaScript, which is a difficult task. If we set the threshold too high, we cannot detect all type 1 obfuscated JavaScript. On the other hand, if we set it too low, we might misclassify non-type 1 obfuscated JavaScript as type 1. Figure 4 shows the classification results according to the threshold of obfuscation type 1 in the conventional AST. In general, it is difficult to determine an optimal threshold value because the accuracy and false negative rate (FNR) usually change together. For example, if we use a higher threshold for a higher accuracy, the false positive rate (FPR) decreases, but the FNR increases. If we use a lower threshold for a lower FNR, the accuracy decreases. \begin{table}
\begin{tabular}{c c c c} \hline \hline \multirow{2}{*}{**JavaScript Set**} & **Structure Similarity** & **Structure Similarity** & **Reduction** \\  & **Conventional AST** & **Lightweight AST** & **Rate** \\ \hline
1000 unobfuscated benign samples & 8.404 s & 7.264 s & 13.56\% \\ \hline
1000 obfuscated benign samples & 1.252 s & 0.661 s & 47.20\% \\ \hline
1000 obfuscated malicious samples & 2.293 s & 1.139 s & 50.32\% \\ \hline Average & 3.983 s & 3.021 s & 24.15\% \\ \hline \hline \end{tabular}
\end{table} 
Table 5: Structure similarity computation time. Figure 3: Structure similarity of data compared with obfuscation type 1 with a conventional AST. Figure 5 shows the structural similarities measured between the representative type 1 obfuscated JavaScript and all JavaScript when the lightweight AST is used. For type 1, the structural similarity is measured as 1, because all obfuscation type 1 JavaScript are the same lightweight AST by Theorem 1. This applies the same to type 2, 3, and 4, respectively. Note that their similarities are not 1 because they are not type 1. Figure 6 shows the classification result according to the threshold of obfuscation type 1 in a lightweight AST. In this case, we can easily select an optimal threshold because the FNR does not increase, even if the accuracy is increased by the change in threshold. For example, if the threshold is 0.76, then the accuracy is 1, and the FPR and FNR are 0. Even if the threshold increases, the accuracy is already the maximum and the FNR does not increase. Figure 4: Classification result according to the obfuscation type 1 threshold in the conventional AST (FPR: False Positive Rate, FNR: False Negative Rate).