Learning Cross-Architecture Instruction Embeddings for Binary Code Analysis in Low-Resource Architectures (2024)

Abstract: Binary code analysis is indispensable for a variety of software security tasks. Applying deep learning to binary code analysis has drawn great attention because of its notable performance. Today, source code is frequently compiled for various Instruction Set Architectures (ISAs). It is thus critical to expand binary analysis capabilities to multiple ISAs. Given a binary analysis task, the scale of available data on different ISAs varies. As a result, the rich datasets (e.g., malware) for certain ISAs, such as x86, lead to a disproportionate focus on these ISAs and a negligence of other ISAs, such as PowerPC, which suffer from the “data scarcity” problem. To address the problem, we propose to learn cross-architecture instruction embeddings (CAIE), where semantically-similar instructions, regardless of their ISAs, have close embeddings in a shared space. Consequently, we can transfer a model trained on a data-rich ISA to another ISA with less available data. We consider four ISAs (x86, ARM, MIPS, and PowerPC) and conduct both intrinsic and extrinsic evaluations (including malware detection and function similarity comparison). The results demonstrate the effectiveness of our approach to generate high-quality CAIE with good transferability.

## introduction binary code analysis, which allows one to analyze binary code without access to the corresponding source code, plays a critical role in a wide range of different tasks, including code plagiarism detection @xcite , malware classification @xcite , function similarity detection @xcite , and vulnerability discovery @xcite . today, software is frequently cross-compiled for various instruction set architectures (isas). for example, hardware vendors often use the same code base to compile firmware for different devices that operate on varying isas (e.g., x86 and arm), which causes a single vulnerability at source-code level to spread across binaries of diverse devices. as a result, cross-architecture binary code analysis has become an emerging problem that draws great attention @xcite @xcite . analysis of binaries across isas, however, is non-trivial: such binaries differ greatly in instruction sets, calling conventions, general-and special-purpose cpu register usages, memory addressing modes, and more. recently, we have witnessed a surge of research efforts that leverage deep learning to tackle various binary code analysis tasks. deep learning has demonstrated its strengths on code analysis, and shown noticeably better performances over traditional program analysis-based methods in terms of both accuracy and scalability. however, training a deep learning model usually requires massive amount of data. as a result, most deep learningbased binary analysis models have been dedicated to a high-resource isa, such as x86, where largescale labeled datasets exist for training their models. but for many other isas, such as powerpc, there are few or even no labeled dataset, resulting in a negligence focus on those low-resource isa. moreover, it is labor intensive and time-consuming to collect data samples and manually label them to build datasets for such low-resource isas. our approach. a binary, after being disassembled, is expressed in an assembly language. given this insight, innereye @xcite proposed to adapt deep learning techniques developed for natural language processing (nlp) to binary code analysis. since then a surge of nlp-inspired binary analysis approaches have been proposed @xcite @xcite @xcite . in many nlp tasks, words are often converted into word embeddings, which capture the semantic meaning of words, to facilitate further processing @xcite @xcite . to analyze binary code, we regard instructions as words and basic blocks as sentences. inspired by cross-lingual word embeddings in nlp @xcite , we propose a novel approach to tackle the challenge of data scarcity in binary code analysis. our approach learns cross-architecture instruction embeddings (caie), where semantically-similar instructions, regardless of their isas, have close embeddings in a shared space. equipped with such a shared space, we can transfer knowledge from one isa to another, especially in low-resource scenarios. specifically, by projecting instructions disassembled from binaries in different isas into the shared space, we can train a model using only the data in a high-resource isa, and transfer it to a low-resource isa. unlike nlp, where obtaining cross-lingual signals can be difficult, obtaining cross-architecture signal for binary code analysis across isas is not a challenging task (see section 4.1). we thus design a supervised model by borrowing the idea and technique from the bi-sent2vec algorithm @xcite in nlp to learn caie. we first build a dataset consisting of a large number of semantically-equivalent basic block pairs and use them to train a joint model which learns to predict both the instruction and context in the source and target basic blocks. the model explores rich semantic relationships among instructions within its own isa as well as across a different isa. it is also fully self-supervised, which helps overcome the limitation of unsupervised linear transformation. we have implemented our model, called crossins2vec, and evaluated it on four isas: x86, arm, mips and powerpc (ppc). (1) in intrinsic evaluation, we conduct the instruction similarity task to evaluate the quality of caie-whether they capture the syntax and semantic information of instructions across isas. (2) in extrinsic evaluation, we conduct two downstream binary analysis tasks to evaluate the transferability of caie, including function similarity comparison and malware detection. we also compare crossins2vec to the baseline methods. the results show that crossins2vec can effectively generate highquality caie with better transferability. below summarizes our contributions: ## model design in contrast to the challenges faced in nlp, where obtaining cross-lingual signals can be a difficult task, acquiring cross-architecture signals for binary analysis across isas is straightforward (section 4.1). moreover, studies have shown that supervised methods typically exhibit superior performance compared to unsupervised ones @xcite . we thus design a supervised model, called crossins2vec, to learn cross-architecture instruction embeddings (caie). figure 3 shows the model architecture. as we consider instructions as words and basic blocks as sentences, the input is a pair of semanticallyequivalent basic blocks, b 1 and b 2 , in different isas. initially, each instruction is assigned a random vector. during the joint learning process, crossins2vec effectively learns caie for each instruction. below we present the detailed process. ## evaluation we evaluate our model in terms of the quality and transferability of caie. we have two questions: (q1) quality: how well can caie tolerate architectural differences and capture code semantics across isas? (q2) transferability: whether caie can transfer knowledge from one isa to another? to answer q1, we conduct the intrinsic evaluation, including the instruction similarity task. to answer q2, we conduct the extrinsic evaluation, including two critical binary analysis tasks: function similarity comparison and malware detection. ## conclusion applying deep learning to binary code analysis has drawn great attention. limited availability of data on low-resource isas, however, hinders deep learning-based binary code analysis. in this work, we propose to learn cross-architecture instruction embeddings (caie), where semantically-similar instructions, regardless of their isas, have close embeddings in a shared space. as a result, we can transfer a model trained on a data-rich isa to another isa with less available data. we conducted experiments to evaluate the quality and transferability of the learned caie. in the downstream tasks, when a model trained on x86 is transferred to arm, mips and ppc, the prediction accuracies keep high. our approach significantly outperforms the prior work. therefore, our approach can generate caie with high quality and transferability, and resolve the data scarcity problem in low-resource isas for binary code analysis tasks.