# Lightweight Detection Method of Obfuscated Landing Sites Based on the AST Structure and Tokens
## Proposed System
### Revised TF-IDF  
Obfuscation is performed to protect the internal information or structure of an algorithm from web crawlers. Obfuscation usually divides words and then recomposes them, which makes it difficult to detect a redirection JavaScript. We classified the obfuscation type in the previous step, whereby we could detect the redirection JavaScript by detecting the divided redirection words according to the obfuscation type. We refer to the divided redirection JavaScript as the redirection token. Note that we cannot determine if the JavaScript, including the redirection token, is malicious because the redirection token can be generated by other words that are not redirection code.  
Therefore, we use the TF-IDF to compare how many redirection tokens are included in each JavaScript. Let us analyze the total number of JavaScript, represented as _javascripttotal_, and calculate the TF-IDF for redirection token \(t\). When we analyze the \(i\)-th JavaScript, represented as _javascript\({}_{i}\)_, the conventional TF-IDF is calculated as  
\[\text{tf}-\text{idf}=\text{the number of $t$ in \emph{javascript}${}_{i}$}\times\frac{\emph{ javascript}_{total}}{\text{the number of javascript including $t$}}.\]  
The conventional TF-IDF represents how many more \(t\)'s are included in the \(i\)-th JavaScript than in the other JavaScript. However, the obfuscated code of a benign page is generally long, unlike the obfuscated redirection JavaScript, which is short because attackers aim to hide. Obfuscation algorithms generate a number of redirection tokens, even if a benign page does not have redirection code. Therefore, it mistakenly outputs higher values because the number of redirection tokens is extremely large. To avoid this problem, we revise the conventional TF-IDF by dividing the number of redirection tokens to be analyzed by the total number of all tokens using the following definition:  
**Definition 4**.: _Revised TF-IDF._  
\[\text{revised tf}-\text{idf}=\frac{\text{Number of $t$ in \emph{javascript}${}_{i}$}}{\text{Number of all \emph{tokens} in \emph{javascript}${}_{i}$}}\times\frac{\emph{ javascript}_{total}}{\text{Number of javascript including $t$}}.\]  
**Property 1.**_The proposed TF-IDF effectively distinguishes the redirection JavaScript compared to the original TF-IDF._  
The number of all tokens in _javascript_, is used to filter out obfuscated benign JavaScript. We obtain TF by normalizing the ratio of the total number of JavaScript to the number of JavaScript containing redirection tokens. Accordingly, we can obtain lower revised TF-IDF values by the given formula while analyzing the obfuscated benign JavaScript. This is because, even if the obfuscation algorithms generate several redirection tokens, they also generate several other tokens, excluding redirection tokens. In obfuscated redirection JavaScript, we obtain large revised TF-IDF values because they have many redirection tokens compared to their sizes. Hence, we can obtain TF-IDF values that can distinguish between benign JavaScript and obfuscated redirection JavaScript with higher accuracy.