# Code Obfuscation: A Comprehensive Approach to Detection, Classification, and Ethical Challenges
## 6 Working Process
### Feature Calculations and Formulas  
**The String Entropy**: Measures the randomness in code strings. This metric helps assess how uniformly the characters are distributed within the string literals found in the code.  
\[String\ Entropy=\frac{1}{n}\sum_{i=1}^{n}Entropy(s_{i}) \tag{1}\]  
where \(s_{i}\) is the string literal, and \(n\) is the number of string literals.  
And the entropy of each string literal \(s_{i}\) is calculated by  
\[Entropy(s_{i})=-\sum_{j}p_{j}log_{2}\left(p_{j}\right) \tag{2}\]  
where \(p_{j}\) is the probability of character \(j\) occurring in the string literal \(s_{i}\).  
**The Average Token Length:** The mean length of tokens in the code gives insight into the typical size of the identifiers and literals.  
\[\textit{Average Token Length}=\frac{1}{n}\sum_{i=1}^{n}\textit{len}(t_{i}) \tag{3}\]  
where \(t_{i}\) are the tokens, and \(n\) is the number of tokens.  
**The Standard Deviation of Token Length:** Reflects the variability in token lengths, indicating how consistently sized the tokens are.  
\[\textit{Standard Deviation of Token Length}=\sqrt{\frac{1}{n}\sum_{i=1}^{n}( len(t_{i})-avg)^{2}} \tag{4}\]  
where \(avg\) is the average token length.  
**The Median Token Length:** Identifies the middle value of token lengths when sorted, providing a robust measure of central tendency that is less sensitive to outliers than the mean.  
\[\textit{Median Token Length}=\textit{Median}(\{len(t_{1}),len(t_{2}),\dots,\ len(t_{n})\}) \tag{5}\]  
**The Overall Code Entropy:** Measures the overall randomness or complexity of the entire codebase, giving a single figure that summarizes the predictability of the code's content.  
\[\textit{Overall Code Entropy}=-\sum_{i}p_{i}log_{2}(p_{i}) \tag{6}\]  
where \(p_{i}\) is the probability of character \(i\) occurring in the text.  
**The Comment Density:** The proportion of lines in the code that are comments, which can indicate how well-documented the code is.  
\[\textit{Comment Density}=\frac{\textit{Number of Comment Lines}}{\textit{Total Number of Lines}} \tag{7}\]  
**The Unique Token Ratio:** The proportion of unique tokens relative to the total number of tokens, which can indicate code diversity or repetitiveness.  
\[\textit{Unique Token Ratio}=\frac{\textit{Number of Unique Tokens}}{\textit{Total Number of Tokens}} \tag{8}\]  
**The Keyword Count:** Counts the occurrence of specific keywords, which can be indicative of the control structures and logic used within the code.  
\[\textit{Keyword Count}=\sum_{k\ \in\ \textit{keywords}}\textit{count}(k) \tag{9}\]  
where _keywords_ is the list of specific keywords.  
Each feature was carefully selected based on its potential to capture the nuances introduced by different obfuscation techniques. The feature extraction process involved detailed analysis and computation to ensure that the selected features effectively captured the characteristics of obfuscated code.  
The final dataset consisted of 6476 non-obfuscated files and obfuscated files from various sources, including Jlaive (733 files), Oxyry (139 files), PyObfuscate (149 files), Pyarmor (706 files), and py-obfuscator (190 files). These additional features allowed our models to reach high accuracy in identifying obfuscated files and distinguishing the specific obfuscator used. After 10 iterations, the mean accuracies for the models were as follows: Support Vector Machine (SVM) achieved 97.07%; Gradient Boosting achieved 97.30%;and Random Forest achieved 97.25%, with Gradient Boosting demonstrating the highest accuracy among the models. The model training and evaluation process involved rigorous cross-validation to ensure the robustness of our results. We also conducted extensive hyperparameter tuning to optimize the performance of each model.