N-Bref : A High-fidelity Decompiler Exploiting Programming Structures 

1 INTRODUCTION . Decompilation , which is a process of recovering source code from binary , is useful in many situations where it is necessary to analyze or understand software for which source code is not available . For example , decompilation is highly valuable in many security and forensics applications ( Lin et al . ( 2010 ) ; Lee et al . ( 2011 ) ; Brumley et al . ( 2011 ) ) . Given a binary executable , an ideal decompiler generates the high-level program that preserves both the semantics and the functionality of the source code . However , this process is difficult as the data structure and semantics are largely destroyed or obfuscated during the compilation . Inspired by remarkable performance in neural machine translation ( NMT ) tasks ( Liu et al . ( 2019 ) ; Vaswani et al . ( 2017 ) ; Dai et al . ( 2019 ) ; Devlin et al . ( 2018 ) ; Dong & Lapata ( 2016 ) ) , recent works ( Fu et al . ( 2019 ) ; Katz et al . ( 2019 ) ) leverage NMT model for neural-based decompilation and achieve promising performance on small code snippets . To make neural-based decompilation useful in practice , many challenges remain : ( C1 ) Current stateof-the-art neural architectures for machine translation – transformer ( Vaswani et al . ( 2017 ) ) or its variants ( Dai et al . ( 2019 ) ; Devlin et al . ( 2018 ) ; Liu et al . ( 2019 ) ) – focused on sequential data ( e.g. , language ) , while neural decompilers deal with data with intrinsic structures ( e.g. , tree/graph ) and long-range dependencies . ( C2 ) The main decompilation task consists of many sub-tasks ( e.g. , datatype recovery , control/dataflow recovery ) . Training one neural network can not solve them all . ( C3 ) Practical data types ( e.g. , pointers ) are not modeled and compiling configurations need to be known beforehand ( Fu et al . ( 2019 ) ) . ( C4 ) Due to a lack of unification in terms of library usage , variable type , and/or control-flow complexity , a simple crawling from public repositories does not 1 N-Bref is the abbreviation for “ neural-based binary reverse engineering framework ” work well . Source code of different styles can be compiled into identical binary code ( i.e. , “ expression collision ” or EC ) and yield issues when evaluating decomplied code against original source code . To our best knowledge , no code generation toolkit with configurable code complexity exists . In this paper , we present N-Bref , an end-to-end neural-based decompiler framework that learns to decompile the source code to assembly . For ( C1 ) , we design a back-bone structural transformer by incorporating inductive Graph Neural Networks ( GNNs ) ( Hamilton et al . ( 2017 ) ) to represent the low-level code ( LLC ) as control/dataflow dependency graphs and source code as Abstract Syntax Tree ( AST ) . To better model long-range correlations in the structural representations , we add a graph neural network after each of the self-attention layers in the transformer . The AST decoder expands the AST of the source code in a tree fashion to better capture the dependency of each predicted node . Also , we adopt memory augmentation ( Cornia et al . ( 2019 ) ) and new tokenizing methods to improve the scalability of our neural networks with the growing size of programs . The backbone network is learned to iteratively generate AST for source code from structured representation of assembly . For ( C2 ) and ( C3 ) , we decouple decompilation into two sub-tasks : data type solver ( DT-Solver ) and source code generator ( SC-Gen ) , both use the same backbone structural transformer with different parameters . The output of the data type solver is used as the decoder input of the source code generation . For ( C4 ) , we design a dataset generator to generate training data , test and analyze the performance of different design principles across configurable code complexity . Different from conventional dataset generators ( Yang et al . ( 2011 ) ; IntelC++compiler ( 2017 ) ) used in programming language studies , our generator produces similar code styles as those written by human programmers , has unified source code representation that avoids EC , has configurable complexity and data types to facilitate factor analysis , and is specifically designed for learning-based methodologies . Extensive experiments show that on our new metrics , N-Bref outperforms transformer baseline/previous neural-based decompiler ( Fu et al . ( 2019 ) ) by 3.5 % /6.1 % and 5.5 % /8.8 % in data type recovery and source code generation tasks , respectively . Furthermore , on 5 human-written Leetcode solutions , N-Bref shows 4.1 % /6.0 % and 6.0 % /9.7 % margins over transformer/previous neural decompiler in data type recovery and source code generation , respectively . We also perform a comprehensive study of the design component in neural-based decompiler across different dataset configurations . In summary , this paper makes the following contributions : We construct an end-to-end decompilation system by integrating a LLC Encoder , an AST encoder , an AST decoder , and a set of novel embedding methods in a holistic manner . Our new architectures bridge the gap between low-level code and high-level code by transforming both of them into a graph space . We perform a comprehensive analysis of the influence of each neural-based decompiler design component to the overall program recovery accuracy across different dataset configurations . We corroborate the design performance on various generated benchmarks and Leetcode tasks . We boost decompilation performance by decomposing the decompilation process into separate tasks , data type recovery and AST generation . In addition , we present corresponding new metrics to evaluate data type recovery and source code generation . We develop the first dataset generation tool for neural-based decompiler development and testing . It randomly generates programs with configurable complexity and data types ; it also unifies source code representation to prevent ” expression collision ” . 2 PRELIMINARIES OF DECOMPILERS . Decompilation takes an executable file as input and attempts to create high-level source code that are more semantically meaningful and can be compiled back . Figure 1 shows a low-level code snippet disassembled from a stripped binary and the corresponding high-level program . A commonly used low-level code ( LLC ) is assembly ( ASM ) . An assembly program is a sequence of instructions that can be executed on a particular processor architecture ( e.g . MIPS , x86-64 ) . The first token for each instruction is called an ” opcode ” , which specifies the operation to be performed by the instruction . Many instructions in a program operate on processor registers ( a small amount of fast storage in the processor ) or instant values to perform arithmetic operations , such as shifting ( e.g.shl , shr ) , floating-point multiplications ( e.g . mulss ) , etc . Other instructions include ( 1 ) 2 Complete assembly code and graph are shown in Appendix H & I. memory instructions that load ( Figure 1 ( b ) Line 1 ) or store ( Line 9 ) data from memory/register to register/memory ; ( 2 ) branch instructions that conditionally ( Line 6 ) or unconditionally ( Line 10 ) redirect program execution to a different sequence . Each instruction has a certain internal structure , depending on the opcode . For example , in Line 8 of Figure 1 ( b ) , the first operand is a floating-point value in the memory and multss multiplies the value with the destination register ( xmm0 ) and stores the value back to xmm0 . Besides , connections also exist between instructions : ( i ) branch instructions ( e.g. , je , jmp ) reveal the ‘ control flow ’ of the high-level program ; ( ii ) the register which stores the new value of multss ( Line 8 ) is consumed later as a source register ( Line 9 ) . These data movements reveal the ’ data flow ’ of the program . In this paper , we formulate the low-level instructions as a graph using the instruction structure , control-flow and data-flow between each nodes as shown in Figure 1 ( b ) . High-level programming languages can be represented in its equivalent abstract syntax tree ( AST ) ( Baxter et al . ( 1998 ) ) during code generation ( Figure 1 ( a ) ) . This representation has many advantages over its sequential representations : ( i ) adjacent nodes are logically closer in AST compared with sequential representations , ( ii ) error propagation in sequential expansion can be alleviated in a tree decoder , and ( iii ) AST grammar helps prevent error predictions . 3 N-BREF OVERVIEW . In this section , we provide an overview of our design components with an illustrative example . Figure 2 shows an example of the prediction procedures . The Backbone Structural Transformer . Our structural transformer has three components : ( 1 ) LLC encoder , ( 2 ) AST encoder , and ( 3 ) AST decoder ( Detailed in Sec . 4 ) . The LLC encoder takes the low-level instructions converted from binaries using disassembler as input . AST encoder takes the input of a previous ( partial ) AST , and the predictions of AST decoder are AST nodes , which can be equivalently converted to the high-level program . As mentioned earlier , we formulate input low-level code into graphs and high-level code into tree structures . As the AST of the data declaration is very distinct from the rest of the code ( Figure 1 ( a ) ) in highlevel program , we decompose decompilation into two tasks : data type solver ( DT-Solver ) and source code generator ( SC-Gen ) . Both have the backbone structural transformer . Prediction Procedure . Figure 1 shows an example of the code recovery process of N-Bref . The assembly graph and high-level AST is the input of LLC encoder and AST encoder . The input of the AST decoder is the tree path from the root node to the expansion node . Initially , a single root-node is fed into the AST encoder/decoder . Once a new node is generated from decoder in each step , we update the AST and use it as the AST encoder input in the next prediction step . We expand the AST in a breadth-first ( BFS ) fashion . AST contains explicit terminal nodes , which are tokens with no child , such as registers , numerics , variable references and variable types . Non-terminal nodes ( e.g . binary operator ‘ = ’ ) must have children , otherwise there is a syntax error . The branch stop expansion when its leaf nodes are all terminal nodes . Note that during training , we apply ‘ teacher forcing ’ by attaching the correct node label into the AST encoder at each step . ( See Appendix E for formal algorithm ) Cascading DT-Solver and SC-Gen. As shown in Figure 1 , we divide the AST into two parts : ( i ) AST of data type and ( ii ) AST of main code body . Each part is generated using DT-Solver and SC-Gen respectively . This method allows the network to focus on each task individually and resolves more complicated data types . During testing , DT-Solver first generates the left part of the AST in Figure 1 , then the SC-Gen will continue the expansion from this intermediate results . During training , the initial data type input to the SC-Gen is the program golden .

Summary: The authors present a neural-based decompilation framework. They generate synthetic input data in order to make sure source code examples have a consistent code style that can be more easily learned. They predict types of variables and the actual code in two separate steps. For both steps, they employ a custom transformer architecture where evey second layer of the encoder is a graph neural network. There are two separate encoders of this kind, one conditions on a CFG obtained from static analysis of the input assembly, while the other one conditions on the partial output AST that has been generated thus far.