Machine Learning-Assisted Binary Code Analysis

Binary code analysis is a foundational technique in the areas of computer security, performance modeling, and program instrumentation. In computer security, such analysis can provide the basis for detecting, understanding and controlling malicious code. Any analysis of malicious program requires as a first step precisely locating the Function Entry Points (FEPs, the starting byte of each function) within the binary. When full symbol information is available this is a trivial step. Malicious software authors, however, are not known for helpfully providing debugging symbols along with virus payloads. In addition, commodity software is often distributed without symbols, such as in many Linux distributions. In this paper, we consider the machine learning problem of identifying FEPs in binaries where symbols indicating function location are stripped. Our work is targeted at the processing of binaries on a large scale such as is needed in both networkand host-based analysis tools. As a result, we must keep false positive rates extremely low while trying to maximize recall. We consider binaries for both Linux and Windows on the Intel IA32 architecture. Existing techniques have used recursive disassembly parsing and heuristics to locate code in stripped binaries [1–3, 6, 7]. Recursive disassembly parsing follows program control (branches and calls) and finds all functions reachable from the main program entry point. However, it breaks down in the presence of indirect control flow (pointer based) transfers that cannot be resolved statically: on a large set of binaries on our department Linux server, approximately 40% of functions were unrecoverable through recursive disassembly. These functions lie in gaps between discovered functions. However, in these gaps also lie jump tables, numeric and string constants, padding bytes (both fixed-valued and random) and code fragments. Functions may be aligned on a byte boundary or on word or larger boundaries. Several tools, including Dyninst [3] and IdaPro [2], use a small number of manually created patterns to identify FEPs. These patterns are initial instruction sequences commonly found in known FEPs. For example, Dyninst searches Intel IA32 binaries for a common function preamble pattern that sets up a stack frame: (push ebp | mov esp,ebp). While effective on some binaries, these heuristic patterns cannot adapt to variations in compiler, optimization level, and post-compilation optimization tools, all of which may significantly perturb the initial instruction sequence in an FEP. This work extends the Dyninst tool, adding a probabilistic model that classifies FEPs in gaps. Let P be the program binary code. Let x1:n represent all the byte offsets within P’s gaps. For each xi, we can generate the disassembly starting at that byte offset. Our task is binary classification: predicting xi as an FEP (yi = 1) or not (yi = −1). Note the labels y1:n can be correlated for several reasons: i) The instruction at xi can span several bytes. If so, xi and xi+1 represent conflicting (overlapping) parses, and are unlikely to be both FEPs; ii) The disassembly starting from xi might contain a call instruction that calls byte offset xj . If we believe xi is an FEP, then xj should be one too. We therefore use both “local” information (instruction sequence starting from xi) and “global” information (instruction overlap, control flow graph, call graph) in making a global inference over the entire set y1:n.