LLM 4 D ecompile: Decompiling Binary Code with Large Language Models (2024)

Abstract: Decompilation aims to convert binary code to high-level source code, but traditional tools like Ghidra often produce results that are difficult to read and execute. Motivated by the advancements in Large Language Models (LLMs), we propose LLM4Decompile, the first and largest open-source LLM series (1.3B to 33B) trained to decompile binary code. We optimize the LLM training process and introduce the LLM4Decompile-End models to decompile binary directly. The resulting models significantly outperform GPT-4o and Ghidra on the HumanEval and ExeBench benchmarks by over 100% in terms of re-executability rate. Additionally, we improve the standard refinement approach to fine-tune the LLM4Decompile-Ref models, enabling them to effectively refine the decompiled code from Ghidra and achieve a further 16.2% improvement over the LLM4Decompile-End. LLM4Decompile demonstrates the potential of LLMs to revolutionize binary code decompilation, delivering remarkable improvements in readability and executability while complementing conventional tools for optimal results.

## introduction decompilation, the reverse process of converting machine code or binary code into a high-level programming language, facilitates various reverse engineering tasks such as vulnerability identification, malware research, and legacy software migration @xcite @xcite @xcite @xcite . decompilation is challenging due to the loss of information inherent in the compilation process, particularly finer details such as variable names @xcite source code and fundamental structures like loops and conditionals @xcite . to address these challenges, numerous tools have been developed for @xcite and ida pro (hex-rays, 2024) being the most commonly used. although these tools have the capability to revert binary code to high-level pseudo-code, the outputs often lack readability and re-executability @xcite , which are essential for applications like legacy software migration and security instrumentation tasks @xcite . figure 1 illustrates the transformation from the source c code to a binary file, assembly code (asm), and pseudo-code decompiled from ghidra. in this pseudo-code, the original nested for structure is replaced with a less intuitive combination of a do-while loop inside another while loop. furthermore, array indexing like num[i] is decompiled into complicated pointer arithmetic such as *(float *)(param_2 + (long)local_24 * 4). the decompiled output also exhibits syntactical errors, with the function return type being converted to undefined4. overall, traditional decompilation tools often strip away the syntactic clarity provided by high-level languages and do not ensure the correctness of syntax, posing significant challenges even for skilled developers to reconstruct the algorithmic logic @xcite . recent advancements in large language models (llms) have greatly improved the process of decompiling code. there are two primary approaches to llm-based decompilation-refined-decompile and end2end-decompile. in particular, refined-decompile prompts llms to refine the results from traditional decompilation tools @xcite @xcite . however, llms are primarily optimized for high-level programming languages and may not be as effective with binary data. end2end-decompile fine-tunes llms to decompile binaries directly. nevertheless, previous open-source applications of this approach were limited by the use of smaller models with only around 200 million parameters and restricted training corpus @xcite @xcite , in contrast, utilizing larger models trained on broader datasets has proven to substantially improve the performance @xcite @xcite . to address the limitations of previous studies, we propose llm4decompile, the first and largest open-source llm series with sizes ranging from 1.3b to 33b parameters specifically trained to decompile binary code. to the best of our knowledge, there's no previous study attempts to improve the capability of llm-based decompilation in such depth or incorporate such large-scale llms. based on the end2end-decompile approach, we introduce three critical steps: data augmentation, data cleaning, and two-stage training, to optimize the llm training process and introduce the llm4decompile-end models to decompile binary directly. specifically, our llm4decompile-end-6.7b model demonstrates a successful decompilation rate of 45.4% on humaneval @xcite and 18.0% on exebench @xcite , far exceeding ghidra @xcite or gpt-4o (openai, 2023) by over 100%. additionally, we improve the refined-decompile strategy by examining the efficiency of ghidra's decompilation process, augmenting and filtering data to fine-tune the llm4decompile-ref models, which excel at refining ghidra's output. experiments suggest a higher performance ceiling for the enhanced refined-decompile approach, with 16.2% improvement over llm4decompile-end. additionally, we assess the risks associated with the potential misuse of our model under obfuscation conditions commonly used in software protection. our findings indicate that neither our approach nor ghidra can effectively decompile obfuscated code, mitigating concerns about unauthorized use for infringement of intellectual property. in summary, our contributions are as follows: • we introduce the llm4decompile series, the first and largest open-source llms (ranging from 1.3b to 33b parameters) fine-tuned on 15 billion tokens for decompilation. ## related work the practice of reversing executable binaries to their source code form, known as decompilation, has been researched for decades @xcite @xcite . traditional decompilation relies on analyzing the control and data flows of program @xcite , and employing pattern matching, as seen in tools like hex-rays ida pro (hex-rays, 2024) and ghidra @xcite . these systems attempt to identify patterns within a program's controlflow graph (cfg) that corresponding to standard programming constructs such as conditional statements or loops. however, the output from such decompilation processes tends to be a source-codelike representation of assembly code, including direct translations of variables to registers, use of gotos, and other low-level operations instead of the original high-level language constructs. this output, while often functionally similar to the original code, is difficult to understand and may not be re-executable @xcite . drawing inspiration from neural machine translation, researchers have reformulated decompilation as a translation exercise, converting machine-level instructions into readable source code @xcite . initial attempts in this area utilized recurrent neural networks (rnns) @xcite for decompilation, complemented by errorcorrection techniques to enhance the outcomes. motivated by the success of large language models @xcite @xcite , researchers have employed llms for decompilation, primarily through two approaches-refined-decompile and end2end-decompile. in particular, refined-decompile prompts the llms to refine results from traditional decompilation tools like ghidra or ida pro. for instance, degpt @xcite ) enhances ghidra's readability by reducing cognitive load by 24.4%, while @xcite increases ida pro's re-executability rate to over 75% by integrating error messages into its refinement process. these approaches, however, largely ignore the fact that llms are designed primarily for high-level programming languages @xcite @xcite , and their effectiveness with binary files is not well-established. end2end-decompile, on the other hand, fine-tunes llms to decompile binaries directly. early open-source models like btc @xcite and recent development slade @xcite adopt the language model with around 200 million parameters @xcite to fine-tune for decompilation. while nova @xcite , which is not open-sourced, develops a binary llm with 1 billion parameters and fine-tunes it for decompilation. consequently, the largest open-source model in this domain is limited to 200m. whereas utilizing larger models trained on broader datasets has proven to substantially improve the performance @xcite @xcite . therefore, our objective is to present the first and most extensive open-source llm4decompile series, aiming at comprehensively advancing the decompilation capability of llms. initially, we optimize the end2end-decompile approach to train the llm4decompile-end, demonstrating its effectiveness in directly decompiling binary files. subsequently, we enhance the refined-decompile frameworks to integrate llms with ghidra, augmenting traditional tools for optimal effectiveness. ## llm4decompile first, we introduce our strategy for optimizing llm training to directly decompile binaries, the resulting models are named as llm4decompile-end. following this, we detail our efforts for enhancing the refined-decompile approach, the corresponding fine-tuned models are referred to as llm4decompile-ref, which can effectively refine the decompiled results from ghidra. ## experiments in this section, we discuss the experimental setups and results for llm4decompile-end and llm4decompile-ref respectively. ## obfuscation discussion the process of decompilation aims at revealing the source code from binaries distributed by developers, presenting a potential threat to the protection of intellectual property. to resolve the ethical concerns, this section accesses the risks of the possible misuse of our decompilation models. in software development, engineers typically implement obfuscation techniques before releasing binary files to the public @xcite . this is done to protect the software from unauthorized analysis or modification. in our study, we focus on two fundamental obfuscation techniques as suggested in obfuscator-llvm @xcite : control flow flattening (cff) and bogus control flow (bcf). these techniques are designed to disguise the true logic of the software, thereby making decompilation more challenging to protect the software's intellectual property. we present the details of these two techniques in the appendix e. results summarized in ## conclusions we propose llm4decompile, the first and largest open-source llm series with sizes ranging from 1.3b to 33b trained to decompile binary code. based on the end2end-decompile approach, we optimize the llm training process and introduce the llm4decompile-end models to decompile binary directly. the resulting 6.7b model shows a decompilation accuracy of 45.4% on humaneval and 18.0% on exebench, surpassing existing tools like ghidra and gpt-4o over 100%. additionally, we improve the refined-decompile strategy to fine-tune the llm4decompile-ref models, which excel at refining the ghidra's output, with 16.2% improvement over llm4decompile-end. finally, we conduct obfuscation experiments and address concerns regarding the misuse of llm4decompile models for infringement of intellectual property.