Self-Constructed Context Decompilation with Fined-grained Alignment Enhancement (2024)

Abstract: Decompilation transforms compiled code back into a high-level programming language for analysis when source code is unavailable. Previous work has primarily focused on enhancing decompilation performance by increasing the scale of model parameters or training data for pre-training. Based on the characteristics of the decompilation task, we propose two methods: (1) Without fine-tuning, the Self-Constructed Context Decompilation (sc ^2 dec) method recompiles the LLM’s decompilation results to construct pairs for in-context learning, helping the model improve decompilation performance. (2) Fine-grained Alignment Enhancement (FAE), which meticulously aligns assembly code with source code at the statement level by leveraging debugging information, is employed during the fine-tuning phase to achieve further improvements in decompilation. By integrating these two methods, we achieved a Re-Executability performance improvement of approximately 3.90% on the Decompile-Eval benchmark, establishing a new state-of-the-art performance of 52.41%. The code, data, and models are available at https://github.com/AlongWY/sccdec.

## introduction decompilation is the process of converting compiled machine code or bytecode back into a highlevel programming language. this process is typically used to analyze how the software works when the source code is not accessible @xcite @xcite @xcite @xcite . many tools have been developed for decompilation, like ghidra @xcite and ida pro @xcite . however, these tools often struggle to generate human-readable code. the main challenge in decompilation is that it's hard to fully reconstruct the source code, especially details such as variable names @xcite and primary structures @xcite , which are frequently lost during the compilation process. recent advances in large language models (llms) have prompted researchers to view programming languages as distinct linguistic systems, utilizing pre-trained code llms to accomplish various coding tasks @xcite @xcite . these models have demonstrated significant performance improvements over traditional techniques @xcite , making it feasible to apply llms to decompilation challenges. for instance, transformer-based models such as slade @xcite and nova @xcite have shown potential in using language models to translate binary code back into more readable and structured source code. recently, @xcite developed and released the first open-source llm focused on decompilation, alongside the construction of the first decompilation benchmark to evaluate the decompilation capabilities of models. these efforts primarily treat the decompilation task as a translation task, training a robust decompilation model on extensive synthetic data @xcite @xcite . by analyzing the characteristics of decompilation tasks, we have found two features: based on the characteristics of the decompilation task, we propose two approaches: (1) self-constructed context decompilation (sc 2 dec), a tuning-free approach, leverages the decompilation results from the model and specific compiler to derive ground-truth (assembly code, source code) pair closely related to the test sample. by incorporating the obtained pair as a demonstration example within the context, we can fully leverage the model's in-context learning capabilities to enhance performance. (2) fine-grained alignment enhancement (fae) introduces a new step-by-step decompilation training objective in addition to the end-to-end decompilation training objective that relies on large-scale pre-training for implicit alignment. by leveraging debugging information designed for debuggers, we can align assembly code with high-level code at a finer statement level. this allows the model to more faithfully preserve the original functionality after decompilation. combining these two methods, we have achieved an approximately 3.90% improvement in reexecutability performance based on llm4decompile model @xcite , reaching a new state-ofthe-art performance of 52.41% re-executability. to summarize, our contributions are as follows: ## related works decompilation has a long history, centered on converting binary files into human-readable source code. traditional tools such as hex-rays ida pro (hex-rays, 2024) and ghidra @xcite predominantly rely on analyzing a program's control flow graph (cfg) to perform this task @xcite . these tools operate by scrutinizing instructions within assembly code to construct cfgs and identify common programming structures like if-else statements and loops @xcite . however, they rely on intricate and errorprone expert-built rule systems that frequently fail with optimized binary code, a common practice in commercial software development. moreover, traditional tools often generate outputs that closely resemble assembly code representations, translating variables into registers and utilizing low-level operations such as goto statements. this output is not only challenging to comprehend but also occasionally cannot be recompiled @xcite . inspired by neural machine translation, researchers have begun to conceptualize decompilation as a translation task, wherein machine-level instructions are converted into readable source code using neural networks. initial endeavors employing recurrent neural networks (rnns) demonstrated limited success @xcite . however, recent advancements in natural language processing (nlp) have facilitated the application of pre-trained language models (lms) in decompilation. notable examples include btc @xcite , slade @xcite , and nova @xcite , which mark significant strides in the field. recently, @xcite created and released the first open-source large language models specifically for decompilation and established an evaluation benchmark that considers the re-compilability and re-executability of decompiled code. ## method in this section, we introduce the self-constructed context decompilation method and fine-grained alignment enhancement fine-tuning. figures 2 and 3 illustrate the methods. <func0>: mov %edi,%eax and @xmath00x1,%eax ret decompile compile this is the assembly code: <func0>: mov %edi,%eax add @xmath10x1,%eax ret what is the source code? should be & int func0(int x) { return x & 1; } 1 2 3 3 2 1 self construct pair ① the assembly to decompile ② initial decompilation result form ① ③ compiled result from ② figure 2 : the pipeline for self-constructed context decompilation operates as follows: when the llm decompiles and generates compilable code, we compile this code to construct the context, and then use it to decompile the initial assembly code again. ## conclusion in this paper, we introduce two methods based on the characteristics of the decompilation task: self-constructed context decompilation, which leverages the compilability of decompilation results to construct better contexts, and fine-grained alignment enhancement, which fine-tunes the model using fine-grained alignment data between assembly code and source code derived from debug information. by integrating these methods, we achieved a 3.90% improvement in decompilation performance, reaching a new state-of-the-art level of 52.41%.