Exploring General Intelligence of Program Analysis for Multiple Tasks

1 INTRODUCTION . With the development of information technology , computer programs are used in an increasingly wide range of fields . As the number and variety of programs continue to expand , the task of analyzing programs becomes more significant and complex . Common program analysis tasks include program classification , duplicate code detection , making programming suggestions ( syntax and algorithms ) , program vulnerability analysis , bug detection , cross-language program translation , code annotation , etc . Program analysis is not a simple problem and many factors bring complexity to it . First , program functions are not implemented in a unique way ; apparently completely different codes may implement the same functions , and similar codes may implement quite different functions . Second , programs may be written in different languages and run on various platforms . The same code compiled by different compilation configurations may also have outstanding differences in terms of correctness and performance . All of these are the major factors to consider during program analysis . In recent years , more and more researchers have introduced machine learning approaches to solve complex program analysis problems . Earlier approaches treat codes as shallow textual structures , e.g. , sequences of tokens ( Allamanis et al. , 2016a ; Hindle et al. , 2016 ; Allamanis et al. , 2016b ; Pradel & Sen , 2018 ) , and analyze them with natural language processing models . These approaches ignore the rich and explicit structural information that exists in a program . In addition , since many variables are defined and used not adjacent to each other or even far apart , this “ def-use ” relations are difficult to capture by this kind of model . However , such long-distance dependencies , can be reflected in the structural information of the program . By abstracting programs as graphs and using graph neural networks ( GNN ) for program analysis tasks , better use can be made of these structure information that are readily available in programs . Graphs such as control flow graph , call graph , and data flow graph reflect the control flow , function calls and data flow relations of a program . Previous approaches based on graph neural networks have used only one graph structure ( Xu et al. , 2017 ; Ben-Nun et al. , 2018 ; Wang et al. , 2020 ) . However , different graph structures respond to different program features , so it is necessary to combine multiple graphs together to obtain richer structural information . Existing methods explore program embedding through different inputs , e.g. , source code , abstract syntax tree , assembly code and so forth . Constructing embeddings via source code or AST makes good use of semantic information in program variables . However , they lack compile-time information and can not solve compilation- and architecture-related analysis tasks . In order to build embed- dings that can solve both source-code level tasks ( program classification ) and compilation-related tasks ( program vulnerability analysis ) , we choose to build embeddings through the assembly code of a program . The embeddings we construct not only contain semantic and structural information of the program code , but also reflect differences in compilation configurations , so they can be applied to a wider range of tasks . In summary , this paper makes the following contributions : • We abstract a program into multiple graphs , extract different program structure information , and perform multi-dimensional analysis of the program . • We build program embedding based on the assembly code , offering the possibility to analyze both source code related and compile related analysis tasks . • We perform procedure-level and program-level analysis , capturing both local and global features . • We measure on two program analysis tasks , program classification and binary similarity detection , and achieved an accuracy of up to 82.58 % and 83.25 % , 8.6 % and 84.4 % of improvement over the state-of-art approaches . 2 CHALLENGES OF PROGRAM EMBEDDING . 2.1 MULTIPLE PROGRAM REPRESENTATIONS . From the perspective of language syntax , a program may be represented in many different formats , such as source code , abstract syntax tree ( AST ) , compiler IR , assembly code , etc . Fig . 1 shows the four formats with one code example . As seen from Fig . 1 , different formats share similar information but also have their unique characteristics and complexities . For example , only the source code and AST contain variable literals . Therefore , which level of code to embed with deserves a serious study . Program embedding from source code certainly has its own merits . A programmer uses English words with clear meaning for variable or function names when coding . Direct embedding over these names can capture the literal semantics for program analysis . Also , source code is small sized , often within a few kilobytes . However , the downside is having to maintain a vocabulary for all possible variable names , which can be an incredibly large number considering programmers can name variables as they want . Moreover , embedding over the source code lacks information with respect to program structure semantics . Another line of program embedding is over AST , an abstract representation of program syntactic structure in the form of tree . Unfortunately , embedding over AST also suffers from the problem of oversized vocabulary lists and the shortage of program structure information . This work chooses to embed a program based on the assembly code for several reasons . An assembly code can be compiled from a source code . It is specific to a processor architecture , for example x86-64 . It is typically less stylish and tight to program semantics . For example , programs that are syntactically different but semantically equivalent tend to correspond to similar assembly codes . Moreover , the assembly code often embraces more compilation-specific characteristics than the program intermediate representation or AST ( I. Neamtiu & Hicks , 2005 ) . This work aims to solve the task of binary similarity detection , so it is necessary to distinguish between different architectures and compilers . This compilation-related information is not available in the source code or AST . Therefore , embedding over the assembly code can solve both source code related problems such as program classification , and compilation-related problems such as binary similarity detection . 2.2 CHALLENGES BY NUMEROUS GRAPHS OF A PROGRAM . There are plenty of graph structures implicitly inside a program , including but not limited to control flow graph ( CFG ) , call graph ( CG ) , and data flow graph ( DFG ) and more . A CFG is the graphical representation of the control flow of a code , mostly used in static analysis . It shows all execution paths that can be traversed during a program execution . Edges in a CFG portray conditional branches and nodes represent basic blocks . It contains loops , branches , and other structures , which is difficult to obtain directly from the source code of a program . A CG represents calling relationships between functions . Each node represents a function and each edge ( a , b ) indicates that the function a is calling b . Thus , a cycle in the graph indicates recursive procedure calls . A DFG represents def-use chains in a program . A “ def ” means an assignment to a variable and a “ use ” is a reference . A program consists of a large number of ordered operations forming a partial-order graph . The data flow graph reflects data dependencies between these operations . Different graphs represent quite different program structures and semantics . Program embedding requires to embed as much semantic information as possible . How to embed these graphs together in an effective and efficient way is a challenging problem . Furthermore , these graphs are relative large for the efficient graph-based deep learning models ( e.g. , graph neural networks ) to run . As tested , the number of nodes and edges of the DFG can be as high as 10847 nodes and 26910 edges . These large-scale graphs incur tediously long training time using graph neural networks . 3 METHODOLOGY . In this section , we propose a program embedding methodology that embeds multiple graph structures in a program as a whole from the level of assembly code . For the best practice , we adopt the deep learning models of graph neural networks ( Scarselli et al. , 2009 ) to represent the semantics of these three types of graphs . The proposed methodology has two standalone training processes consisting of two models , BERT+ and Gated GNN . We fine-tune the pretrained BERT model to derive the initial embedding of each node of a CFG or DFG . Since the BERT model is specifically fine-tuned for program understanding , we name the model as BERT+ , to distinguish . The second model we use is Gated GNN ( GGNN ) to derive the whole embedding of the CFG , CG and DFG , as GGNN is good at memorizing long-term data dependencies between nodes . Depending on the specific tasks , program analysis may be in either procedure or program level . Procedure-level analysis is usually conducted to analyze a program in the unit of procedure , therefore for local function-level tasks . Program-level analysis is for program-level tasks . In this work , we focus on two program-level analysis tasks of program classification and binary similarity detection . However , procedure-level analysis tasks such as function name prediction can also be solved by the proposed model , which we will leave as future work . For procedure-level analysis , we construct a control flow graph for each procedure and combine it with instruction semantics to complete the embedding of each procedure . Then the embedding of each procedure is used to do procedure-level analysis . For program-level analysis , we represent the calling relations between different procedures by call graphs and represent the data dependencies between all variables in the program by data flow graphs . We embed every call graph and data flow graph for each procedure and combine the embeddings of all procedures to give rise to the embedding of the entire program . 3.1 GRAPH NEURAL NETWORKS . The objective of Graph Neural Network is to learn the node representation and graph representation for predicting node attributes or attributes of the entire graph . A Graph Neural Network ( GNN ) ( Scarselli et al. , 2009 ) structure G = ( V , E ) consists of a set of node V and a set of edge E. Each node v ∈ V is annotated with an initial node embedding by x ∈ RD and a hidden state vector htv ∈ RD ( h0v often equals to x ) . A node updates its hidden state by aggregating its neighbor hidden states and its own state at the previous time step . In total , T steps of state propagation are applied onto a GNN . In the t-th step , node v gathers its neighbors ’ states to an aggregation as mtv , as shown in Eq.1 . Then the aggregated state is combined with node v ’ s previous state ht−1v through a neural network called g , as shown in Eq.2 . f can be an arbitrary function , for example a linear layer , representing a model with parameters θ. mtv = ∑ ( u , v ) ∈E f ( htv ; θ ) ( 1 ) h t v = g ( m t v ; h t−1 v ) ( 2 ) h t v = GRU ( m t v ; h t−1 v ) ( 3 ) Gated Graph Neural Network ( GGNN ) ( Li et al. , 2016 ) is an extension of GNN by replacing g in Eq.2 with the Gated Recurrent Unit ( GRU ) ( Chung et al. , 2014 ) function as shown in Eq.3 . The GRU function lets a node memorize history long-term dependency information , as it is good at dealing with long sequences by propagating the internal hidden state additively instead of multiplicatively .

Summary: This paper presents a graph neural network-based approach to solve two binary analysis tasks (program classification and binary similarity detection). The key idea of the paper is to merge different forms of representation of binary code (compiler IR, assembly code, etc.).