# A Framework to Quantify the Quality of Source Code Obfuscation
## 6 Experiment
### Experimental Process
#### 6.1.2 Detailed Protocol  
1. Experimental Environment:  
* Hardware: Intel i9-13900K @ 5.80 GHz CPU, 64 GB RAM, 32 cores
* Software: LLVM version 13.0.0 compiler, Ubuntu 22.04 LTS, Kernel version 5.15.0
2. Obfuscation Tool Configuration:  
* Stunnix C/C++ Obfuscator: applied formatting changes and comment removal. * Semantic Designs C-GCC4 Obfuscator: applied identifier scrambling, formatting changes, comment removal, and data encoding. * Tigress Obfuscator:  
* Level 1: Mixed Boolean Arithmetic
* Level 2: Mixed Boolean Arithmetic + Opaque Predicates
* Level 3: Mixed Boolean Arithmetic + Opaque Predicates + Control Flow Flattening
* Obfuscator LLVM: applied control flow flattening, instruction substitution, and bogus control flows. 3. Data Collection:  
* Potency Metrics: calculated using CFG Analyzer integrated with LLVM. * Resilience Metrics: measured using the KLEE symbolic execution tool, Clang static analyzer, and LLVM opt optimizer. Cost Metrics: assessed using Binary Analyzer developed to measure runtime overhead, memory usage, and binary file size. 4. Analysis:
5. The results were analyzed to identify the impact of each obfuscation technique on the selected metrics. Comparative analysis was performed to evaluate the effectiveness of different techniques. Figure 3 illustrates the experimental process for quantifying results using our framework. In the experiment, we established the NIST Juliet Test Suite v1.3 codes and Banescusebi's obfuscation benchmark codes as the baseline for 130 codes. Generate an obfuscated test code set by applying each obfuscation tool to the established baseline code set. We employed the most robust data obfuscation options like Stunnix C/C++ Obfuscator v4.9 on the baseline code set and executed layout and data obfuscation using the +PrintAsls+Obfuscate+ObfuscateLiterals options of the Semantic Designs C-GCC4 Obfuscator. Figure 3: Obfuscation quality quantification experiment for commercial and open-source obfuscation tools. Unlike previous commercial obfuscation tools, Obfuscator LLVM v4.0.1 can apply potent control flow obfuscation to LLVM intermediate representations (Il files). We applied Control Flow Flattening, Instruction Substitution, and Bogus Control Flows options of Obfuscator LLVM on the baseline code set. Furthermore, we decompiled the obfuscated.Il files to create a dataset comprising.c files. Tigress Obfuscator v3.1 offers a total of 32 obfuscation options. Of these, we selected three that we considered to be the most prevalent and effective for obfuscation research. We created three variations of the test code set to examine the changes in obfuscation quality quantification as we applied additional obfuscation options. The first Tigress data set exclusively utilized the Enc.Arithmetic option, a Mixed Boolean Arithmetic obfuscation technique renowned for its effectiveness among data obfuscation methods (Level 1). The second Tigress data set added the Add Opaque option to Enc.Arithmetic, and we also included the Opaque Predicates obfuscation technique (Level 2). The third Tigress data set added the Flatten option to the Level 2 data set, and we applied a Control Flow Flattening obfuscation technique (Level 3). To assess the obfuscation quality of each tool, we evaluated six obfuscation datasets that our framework generated. The quality of obfuscation can be estimated based on the size of an increase or decrease in quantitative values compared to the baseline. We anticipated that the quantitative values for layout obfuscation and data obfuscation--which is typically regarded as less robust--would not show significant deviation from the baseline. Furthermore, we expected control flow obfuscation, deemed more potent, to significantly influence all 12 measurement indicators. We represented the results of the 12 indicator measurements for each data set, organizing them into the categories of Potency, Resilience, and Cost. We separately displayed the results for Stunnix C/C++ Obfuscator and Semantic Designs C-GCC4 Obfuscator to minimize scale discrepancies in the graphs.