o-glassesX: Compiler Provenance Recovery with
Attention Mechanism from a Short Code Fragment
Yuhei Otsubo∗† , Akira Otsuka† , Mamoru Mimura‡† , Takeshi Sakaki§ , and Hiroshi Ukegawa∗∗ National Police Agency, Tokyo, Japan† Institute of information Security, Kanagawa, Japan‡ National Defense Academy, Kanagawa, Japan§ The University of Tokyo, Tokyo, Japan

Abstract—Program binaries are often the focus of forensic
investigations. Extracting compiler provenance is important indigital forensics, as it provides crucial information about theprocess by which a malware binary is produced. Object files
and libraries, which are components of an executable file, areoften compiled in various environments. Thus the executable filebears important information about the development environmentof the authors. It is expected to reveal valuable information of
the authors if we could extract those compiler provenances withhigh accuracy enough to resolve each component.
In this paper, we investigate the feasibility of compiler
identification from a very short code fragment. Surprisingly,
we found that a randomly sampled code fragment consistingof only 16 sequential instructions is sufficient to identify the
program provenance, such as the source compiler family and the
optimization level of machine code efficiently. Additionally, ourmethod can visualize how much each instruction contributed tothe classification result. The model uses the attention mechanism,
widely used in natural language processing, and CNN with alocal receptive field specialized for recognition of one instruction.
Moreover, our method takes both opcodes and operands as input,
while many of the existing methods had omitted operands.
We used a dataset that consists of 28,074 binaries developedin 19 different classes. Each class corresponds to a combinationof five different compilers, two different optimization levels(none/max), and two different CPU architectures (x86/x86-64).
With this dataset, our method identifies one of the 19 classes
with 0.956 accuracy, which is a competitive performance withthe best-known results, from only 16 instructions or an equivalentbinary sequence of length 30-60 bytes. This result is satisfactory
in resolving the component-wise compiler provenance of an
executable binary. Additionally, the attention mechanism allows
us to extract typical instructions generated by each compiler
automatically. The attention mechanism is useful not only for improving the classification performance but also for understanding
the characteristics of each class.

I.

I NTRODUCTION

Program binaries are often the focus of forensic investigations [25], [29], [26], covering numerous issues from copyright
infringement [31] to malware analysis [3], [2]. Program binary
analysis is a challenging task due to the absence of highlevel information, which is found in source code, and the

Network and Distributed Systems Security (NDSS) Symposium 202023-26 February 2020, San Diego, CA, USA
ISBN 1-891562-61-4
https://dx.doi.org/10.14722/ndss.2020.23xxx
www.ndss-symposium.org

myriad variations in compiler provenance. The more generalterm, program provenance refers to information on how thetarget binary is developed, such as tools, libraries used, andtheir versions. Compiler provenance is a part of program
provenance, consists of compiler family, compiler version,
optimization level, and other compiler-related information.
Compiler provenance gives us many clues on the program
provenance. Extracting compiler provenance is important inprogram provenance digital forensics, as it provides crucial
information about the process by which a program (e.g.,
malware binary is produced. When presented with only aprogram binary or a snippet of binary code, the details ofprogram provenance are not readily apparent. The black box
between the program author and the program binary affords
little foothold for simple tools or analyses.
According to existing research [28], [21], [4], they canidentify hundreds of authors with high accuracy from program
binaries. Although those results are scientifically impressive,
this approach faces a big problem if they are applied in somereal forensics scenario such as malware analysis. First, theyrequire to prepare many binary code samples authored by all
possible malware developers with true identity as they arebased on supervised learning. Second, single-author identification is not enough as malware is usually composed of manybinary snippets authored by different developers. Thus, author
identification must be improved with a combination of extra
information such as binary chunk identification with authorshipdifference. Compiler provenance is one of the crucial clues to
identify such binary chunks.
In this paper, we propose a novel method, named
o-glassesX, that incorporates the attention mechanism toachieve high performance for identifying the source compilerof program provenance. Our method can calculate how mucheach instruction contributed to the identification result andrequires only a sequence of code for compiler provenance.
Hence, our method does not need meta-data or other details ofprogram headers. It is applicable even when such information
has been stripped or is otherwise unavailable. Additionally, it
can be used even when codes produced by multiple compilers
coexist within a program binary, such as statically linked
library code.
In summary, the main contributions of our approach are as
follows:•

Fine-grained compiler provenance from strippedbinary. Our classifier does not use any meta-data,architecture and find typical matching instruction pattern called“idioms” to predict the compiler families. Their early result isonly for the relatively easy compiler family identification with
3 classes. Rosenblum et al. have improved the above method
and proposed a tool named ORIGIN [26]. ORIGIN’s SVM(linear support vector machines [6]) takes the feature of an
independent “function” as input to predict the optimizationlevel and the version in addition to the compiler family but
faces difficulty in identifying the compiler versions. ORIGIN’s
CRF takes the same features of multiple adjacent functions as
input for the prediction and performs with better accuracy of
0.9 and above despite that the number of classes has increased
from 3 to 18. However, the size of the input data required forcompiler provenance is larger.

but it extracts the compiler provenance from a shortsequence of code in a stripped binary. Thus, it can extracts compiler provenance with high accuracy even insuch complex programs. Our result shows that it bears
enough information to identify compiler provenancethat only 30-60 bytes of consecutive binary sequencesampled from any part of the executable code.
•

Proposal of a model that can calculate how much
input data contributes to output in units of instructions. In this paper, we apply an attention mechanism,
widely used in natural language processing in recent
years, to code fragment recognition. This model cancalculate the degree of the contribution to the identification result on each instruction.

•

Knowledge gained from model judgment grounds.
By counting the appearance frequency of instructions
that contributed to the compiler identification results,
we can extract the typical instructions in each classautomatically. These instructions help us to understandthe characteristics of each class.

Rahimian et al. developed BinComp [25], an approachin which they analyze the syntax, structure, and semantics
of disassembled functions to extract the compiler provenance.
BinComp has an identification accuracy of 0.801 in 8 classes
classification in their experiments.
The common point in these existing studies is that itis necessary to analyze by paying attention to the function
structure rather than mere instruction sequence when trying to
identify the optimization level and version in addition to thecompiler family. We show that we can identify not only thecompiler family but also the optimization level and version
with high accuracy by merely analyzing instruction sequences
without paying attention to functions.

The remainder of this paper is structured as follows. In
Sec. II, we first briefly describe a review of related work. In
Sec. III, we describe some essential techniques for our method.
In Sec. IV, we present our approach. In Sec. V, we evaluate theproposed approach and provide comparisons against existingwork. In Sec. VI, we discuss about our approach. In Sec. VII,
we finally present some concluding remarks on this work witha discussion of future research.
II.

B. Binary Analysis using attention
Yakura et al. proposed a method for detecting a regionspecific to the family from malware by applying a CNN incorporating an attention mechanism to the imaged binary [37].
The input data is an image scaled to a resolution of 64 × 64,
and an attention level map indicating the region essential forclassification in the image is output. As a result, the attention
level map shows the region-specific to the malware family,
and it is expected to help analysis. However, the purpose ofthis method is malware family identification, and the input of
the method requires the whole of an executable file. Thus,
the method cannot achieve our goal of performing compiler
provenance recovery from binary fragments.

R ELATED W ORK

Binary Analysis allows software engineers to analyzebinary executables directly without access to source code.
Tesauro et al. [33] first introduced a neural network-based
method for recognizing virus in application binaries. Since
then, machine learning-based binary analysis has become an
important research topic in vulnerability detection [24], [38],[7], [11], [36], [5], [8], function recognition [27], [30], [2],[32], [35], and other areas.
In this section, we describe some methods about compiler
provenance and a binary analysis method using attention
mechanisms.

III.

A. Compiler provenance

P RELIMINARIES

o-glassesX is an extension of o-glasses [23], which
is a program snippet detection tool based on a convolutional
neural network (CNN [18]). This method classifies the input
block as either program-code or non-code. o-glasses canclassify whether the input binary sequence is a program witha very high accuracy rate. o-glasses is designed to capturefeatures of a single instruction by a corresponding multiple
local receptive fields in the first CNN layer. o-glassesX follows this design strategy as well but with attention mechanisms
so that it can tackle the more difficult problem of program
provenance recovery.

Several compiler identification tools are already available(e.g., IDA Pro1 , PEiD2 , and RD3 ). These tools are roughly
signature-based and typically relies on meta-data or otherdetails in the program header. Their exact matching algorithm
may fail if even a slight difference between signatures ispresent or if the header information has been stripped or isotherwise unavailable.
Rosenblum et al. used a conditional random field(CRF [20]) and set up a classifier which takes binary data as
input to identify one of the three compiler families with 0.925
accuracy rate [29]. They disassemble the binary with IA-32

For the sake of this, we used a Natural Language Processing (NLP) techniques called the attention mechanism ino-glassesX. In this section, we describe some preliminaries; First, we describe the essence of x86/x86-64 architecture,
which is our target architecture. Next, we explain CNN with

1 https://www.hex-rays.com/products/ida/
2 https://www.aldeid.com/wiki/PEiD
3 http://www.rdgsoft.net/

2

Fig. 1.

1) Pre-processing: o-glasses disassembles the input
binary assuming they consist of x86/x86-64 instructions. Each
instruction is converted into N -bit fixed-length instructions
padded with constant bits. They serialized a set of 16 fixedlength instructions into an array of 2048 bit values as input
data.

Softmax

FFN

BN

FNN

BN

FFN

128-bitlengthinstructions

CNN

Binaryx86
instructions

CNN

ConvertOutline of o-glasses

2) CNN for recognizing instruction: Generally, the local
receptive field successfully captures the local features in image
recognition, where the values of adjacent pixels are strongly
correlated. In the case of o-glasses, Otsubo et al. designed
the first convolutional layer to capture the feature of thewhole bits in every single instruction and the second layer
to capture the local distribution of instruction features. This
design principle worked surprisingly well against program
provenance recovery as empirically shown in the later sections.

a particular focus on the local receptive field. Then, we describe o-glasses briefly. Finally, we describe the attention
mechanism.
A. x86/x86-64 Architecture
In our experiment discussed in later sections, we mainly
targeted x86 and x86-64 architectures [13]. Note that, however,
our method is not restricted to x86/x86-64 architectures but canbe applied to other architectures such as arm64.

The first convolutional layer (Bit-CNN) takes 2048 bits
as input; each unit has 1-dimensional 128-unit kernel with thestride of 128 and the depth of 96. The kernel field size and thestride of the first layer are set to N so that each kernel covers
a single instruction. The second layer is also a convolutional
layer (Instruction-CNN). They applied 256 2-filters to a 16×96
input volume with a stride of 1. They expect that the second
layer will obtain the features of the relationship between two
adjacent instructions.

x86/x86-64 instruction sets are rich and complex, and mostimportantly, they support instructions of varying length. According to the specification of the x86/x86-64 architecture [13],
instruction lengths range from just one byte (i.e., instructions
comprising just a one-byte opcode) to 15 bytes. Although
15 bytes is the basic maximum length of instruction, longer
instructions could appear in theory (particularly when the file
being interpreted as x86/x86-64 machine code is non-code) 4

3) Other network configurations: The 3rd to the 5th layers
are fully connected MLP (multi-layer perceptron). Their output
volumes are 400, 400, and K, respectively. K is the number ofclasses. Batch normalization [14] layers are inserted after the
1st and 2nd fully connected layers to speed up and stabilize thelearning process. The activation function in each intermediate
layer is ReLU [9], and that of the final layer of MLP issoftmax.

B. CNN
Tools based on CNN have now led to great results in a wide
range of vision tasks [17]. Each unit in CNN has speciallylocal connections to the input units, called a kernel or a local
reception field. Every kernel shares the weight parameters withthe others in the same layer so that one can greatly reduce thenumber of parameters in the network.

They used the well-known stochastic gradient descent(SGD) method to minimize the error function in the backpropagation algorithm.

Several hyperparameters control the size of the output
volume of the convolutional layer: the kernel size, depth, andstride. The depth (D) of the output volume controls the numberof neurons in a layer that connect to the same region of the
input volume. The stride (S) controls how depth columns
around the spatial dimensions (width and height) are allocated.

D. Existing NLP techniques
1) Attention: Bahdanau et al. first proposed the attention
mechanism [1].The attention mechanism uses a vector (attention weight, AttW) that focuses only on a portion of the
input information, not the entire input information. In 2017,
Vaswani et al. proposed Transformer [34], which uses only
the attention mechanism without using Recurrent Neural Network (RNN [12]), and showed surprisingly good performancein translation with a little training. Since this proposal, researchusing the attention mechanism has become very active in thefield of NLP. Additionally, the model applying the attention
mechanism is not only high in performance, but also is possible
to infer essential input data that is the basis of the output result
by visualizing AttW.

The spatial size of the output volume can be computed asa function of the input volume W , the kernel size K, and thestride S. The output volume is given by (W − K)/S + 1.
C. o-glasses
o-glasses is written in Python, and uses the Chainer5
4.0.0 framework, which is a flexible framework for neuralnetworks.
The whole network is shown in Fig. 1. The 1-bit change
often causes the behavior of the machine code to be anotherone. Thus, we should design the neural network carefully,
in the case of treating discrete data such as machine code;e.g., pre-processing, parameter. The details of o-glasses’
implementation is as follows.

There are two types of attention mechanisms, depending on
how to obtain AttW. One is Additive Attention [1], and anotheris Dot-Product Attention (or Multiplicative Attention) [19].
Additive Attention calculates the AttW in the hidden layer
Feed-Forward Network (FFN). Dot-Product Attention calculates the AttW by an inner product. Dot-Product Attention is
generally faster because it does not require parameters. Thus,
we use Dot-Product Attention.

4 The following sentence appears in the specification.

Exceeding the instruction length limit of 15 bytes (this only canoccur when redundant prefixes are placed before an instruction).
5 https://chainer.org/

3InputQuery[q_length, depth][q_length, depth]

language instructions. This idea is the same as what was called
bit-CNN by the authors of o-glasses[23].
matmulSoftmax

The parameters for implementing PFFN in CNN is, forexample, if the N bit fixed-length instructions are arranged in
one dimension, the size of the kernel and stride are N , whichis the same as the length of one instruction. In the case of
PFFN’s input is a two-dimensional vector in which multiple
instruction vectors, we can implement PFFN by setting the sizeof the kernel and stride to 1. In our proposed model, CNN isimplemented by adjusting parameters and implementing PFFN.

Key
Att. W[m_length, depth][q_length, m_length]

Fig. 2.

MemoryValue[m_length, depth][m_length, depth]

matmulOutput[q_length, depth]

Outline of Dot-Product Attention.

3) Positional Encoding: The above mechanism alone canuse combinations of instructions in the code for learning, butcannot use the order of instructions for learning. We introduce
Positional Encoding (PE) to add information on the orderof instructions (relative or absolute position of instructions).
There are various implementation methods for PE. In thispaper, we adopt a method built into Transformer, and we
expect the method to suitable with FFN. PE function adds
the constant matrix PE to PE’s input. The following equation
defines the processing of PE.

Fig. 2 shows the basic structure of Dot-Product Attention.
The squares in the figure indicate processing such as FFN
described later, and the rounded squares indicate tensors. This
processing, such as FFN, transform the attention input into
Query (Q, search query). There is Memory in the hidden
layer in the attention mechanism. The processing, such as FFN,
transforms the Memory into Key (K) and Value (V ). The
following equation defines the attention mechanism.

Att(Q, K, V ) = softmax Q · K T · V ,(1)

PE(X) = X + αPE ,(3)where α is the addition rate of PE . The followings are eachcomponent of PE .

where Q is a matrix (generally a tensor) composed of dk dimensional vectors qi (i = 1, . . . ). Similarly, K and V
are tensors composed of vectors of dk and dv dimensions,
respectively. In particular, the output of softmax(·) when theexpression (1) is applied to one search query vector q ∈Query is called Attention Weight (AttW) . The following
equation defines AttW.

AttW(q, K) = softmax q · K T .
(2)

PE (pos,2i)
PE (pos,2i+1)=

sin(pos/100002i/dmodel )(4)=

2i/dmodel(5)

cos(pos/10000),

where pos is the position of the instruction, dmodel is thechannel depth of the PE’s input X, 2i and 2i + 1 are thecomponent dimensions (0 to dmodel − 1). This expression can
represent PE pos+k with the linear function of PE pos . In FFN,
the linear function of the previous layer can express the nextlayer. Therefore, we expect the PE method is well suitablewith FFN that expresses.

At this time, Memory functions as an array of pairs inwhich each k and each v have a one-to-one correspondence,that is, a dictionary object. The dot product of q and K
measures the similarity between q and each k ∈ K. The
Attention Weight (AttW) normalized by softmax represents
the position of k that is most similar to q. The inner product ofAttW and V is an operation to extract the value corresponding
to the position of k as a weighted sum. In other words,
the attention mechanism is an operation that searches thekey (k) that matches the search query (q) and retrieves thecorresponding value (v). This behavior is the same as thefunction of a dictionary object.

IV.

P ROPOSED METHOD

In this paper, we propose the following three.

Additionally, there are two types of Dot-Product Attention, depending on where Memory comes from. When the
Attention input (Input) and Memory match, it is called
Self-Attention, and when it does not match, it is called
Source Target Attention. Encoder (Classifier) often uses SelfAttention, and Decoder often uses Source Target Attention.
Thus, we use Self-Attention to classify binary sequences.

•

A deep learning network that can classify machine
language instruction sequences with high accuracy
and calculate the instruction-wise contribution to theclassification results (IV-A)•

Carefully designed the attention mechanism that calculates the contribution of each instruction in a sequence to the classification result (IV-B)•

Automatic extraction of representative instructions
which are typically found in each class based on theabove the attention mechanism (IV-C)

A. Network model
We tried to design our attention model as simple as possible
so that it can increase the effect of Attention Weight (AttW) on
the Attention Output (Att. Output). Fig. 3 gives an overview of
our attention model. The input data is as same as o-glasses’
one, a 128 × L bit value array which contains L instructions
converted to 128-bit fixed-length instructions.

2) Position-wise FFN (CNN): Position-wise Feed-Forward
Network (PFFN) is a network used in Transformer. PFFN
performs FFN processing independently for each position ofa word sequence. PFFN can be implemented even by CNN
with parameter adjustment, and it is expected to form a local
receptive field specialized for recognition of one word. In thispaper, it is expected to form a local receptive field specialized
in recognition of one instruction by applying it to machine

The first layer is CNN, which parameters are the same as
o-glasses’ parameters. We expect this layer to learn the4L2-norm

FFN

Softmax

BN

Att. Output

Att. WmatmulSoftmax

matmul

CNN

Query
Key

CNN

Input

CNN

Value

CNN
CNN
CNN

Att. Input

128-bitlengthinstructions

PE

x86
instructions

CNN

ConvertBinaryFig. 3.

Outline of our model.

feature of instruction structure by setting the kernel field size
and the stride size to 128, which is the same as the instruction
length. The output channel depth is 96.

ɥɬɬɯřɥɯƃʫɥ ƄřƃʫɥɨɥƄřɥɯɥ
ɥɫɥɥɥɨɮ
řƃʫɥɯƄřƃʫɥ ƄƃŞɥɯƄřřƃŞɥɯƄ řɥɨɥ
řɥ
řƃƋɫʫɥɥƄřɥɥɥɥɥɥɥ
řƃʫɥɯƄřƃƄřƃŞɥɯƄ řɥɯ

Fig. 4.

The second layer is Positional Encoding (PE), and we
described it in detail at III-D3. PE adds information on theorder of instructions.

ɯ

ɥ

ř

ɯ
ɬ
ɬ

ɥ











ř
Ƅ

ɥ
ʫ



ƃ





ɥ
ɯ

ɥ

ř
Ƅ
ɥ
ɨ

ɥ
ʫ



ƃ











Memory

ɮ

ɨ
ɥ
ɥ
ɥ
ɫ

ɥ




Ƅ
ɯ

ɥ
ʫ



ƃ

ř








Ƅ

ɥ
ʫ



ƃ

ř












ř
Ƅ
ɯ

ɥ
Ş



ƃ





Ƅ
ɯ

ɥ
Ş



ƃ

ř








ɥ
ɨ

ɥ

ř










ɥ

ř








ɥƄ
ɥʫ
ɫƋ

ƃ
ř




ɥɥ
ɥɥ
ɥɥ

ɥ
ř




ɯƄ
ɥʫ

ƃ
ř




Ƅ
ƃ
ř




ɯƄ
ɥŞ

ƃ
ř




ɯɥ
ř





.222 .020 .141 .067 .105 .109 .178 .102 .188 .029 .082 .153 .058 .130 .088 .190 .037
.115 .059 .038 .045 .134 .112 .048 .085 .117 .037 .047 .128 .078 .091 .066 .117 .035
.029 .040 .048 .025 .041 .044 .048 .036 .038 .033 .036 .050 .053 .040 .058 .038 .036
.016 .033 .033 .055 .008 .033 .042 .028 .026 .029 .016 .014 .017 .035 .037 .026 .034
.019 .038 .020 .022 .031 .009 .015 .006 .013 .048 .063 .019 .060 .007 .041 .013 .054
.079 .110 .036 .058 .123 .054 .026 .046 .056 .080 .080 .083 .084 .042 .048 .056 .069
.036 .052 .033 .028 .032 .018 .030 .008 .017 .066 .091 .023 .087 .013 .055 .017 .070
.041 .066 .036 .044 .021 .017 .027 .013 .018 .096 .075 .019 .050 .012 .052 .018 .103
.063 .021 .073 .059 .063 .086 .076 .066 .065 .021 .018 .073 .039 .095 .084 .065 .023
.255 .040 .108 .110 .159 .158 .129 .216 .154 .030 .028 .123 .121 .153 .112 .155 .030
.153 .111 .091 .138 .039 .075 .083 .058 .071 .160 .105 .056 .088 .071 .126 .071 .131
.044 .100 .061 .066 .024 .056 .064 .034 .032 .061 .021 .050 .016 .057 .033 .031 .063
.062 .070 .034 .035 .074 .022 .025 .013 .035 .078 .146 .047 .073 .016 .051 .035 .087
.207 .151 .134 .117 .065 .122 .113 .206 .087 .113 .094 .077 .079 .139 .028 .086 .099
.039 .064 .035 .042 .020 .016 .026 .012 .017 .093 .072 .018 .050 .012 .051 .017 .100
.063 .024 .079 .089 .060 .071 .070 .070 .066 .027 .024 .067 .049 .086 .069 .066 .028

Example of visualizing a AttW.

Algorithm 1 Automatic feature extraction procedure
1: InputFile ← Compile(SourceFile,Optimization).pullCode()
2: Offset ← 0
3: while Offset < InputFile.EndOffset do
4:
Input ← InputFile.pull(Offset,L)
5:
Result ← model.predictor(Input)
6:
if Result.Label = RealLabel then7:
if Result.Confidence > 0.99 then8:i ← argsmax(L2(Result.AttentionWeight))
9:Count[input[i]] ++10:
end if11:
end if
12:
Offset ++13: end while
14: Print(Ranking(Count))

The third layer is the attention block, which is the block
surrounded by the dotted line in Fig. 3. CNN transform theattention input to Query, Key, and Value. We set these
CNN’s parameters in line with the idea of Position-wise FeedForward Network (PFFN). We described PFFN in detail at
III-D2. Each CNN has an independent weight. We set the depthof the first and second channel to dmodel ×4 and dmodel , where
dmodel is the depth of the attention input, 96 in this case.
Immediately after the output of the attention block, weplace the Batch Normalization (BN [14]) layer to stabilize andspeed up the learning process. The last layer is fully-connected,
and the number of nodes is K, where K is the number ofclasses to classify. The activation function of the intermediate
network layer is ReLU [9], and the activation function of theoutput layer is the softmax function.

simplicity, in this paper, we use L2 -norm for calculating each
instruction’s output contribution. Denoted by L2i we visualize
the magnitude of the contribution of the i-th instruction to theclassification result, and L2i is given by the following equation.

B. Calculating ‘Why’ with the attention mechanism
What is important to know in compiler provenance recovery is instruction-wise occurrences and their order which are
typically found in each compiler’s output codes. For calculating the contribution to the identification result one instruction
by one instruction, we designed our network carefully. As
described in III-D1, the attention weight AttW is computedby softmax of the product of a search query matrix Q anda memory matrix M , where Q is a tensor composed of
dk dimensional vectors qi . The PFFN (Position-wise Feed
Forward Network) transforms the input vector of machine
instructions to a search query matrix Q such that its eachsearch query vector (qi ) only depends on one instruction
additionally with its position information. Hence, each elementin the attention weight matrix AttW corresponds to only one
pair of machine instructions in its search query vector and in its
memory vector in a different position. Therefore, the softmax
operation takes the most significant machine instruction in the
input vector contributed to the identification result.

L2i= || AttW (qi , K) ||22
= || softmax(qi · K T ) ||22 .

(6)
(7)

In the case of Fig. 4, it can be seen that the value of thesquared L2 -norm of the 10th instruction “AND EDX, 0xff”
is the largest at 0.255 and contributes the most to the output.
C. Automatic feature extraction for each class
It is possible to automatically extract the typical instructions of each class by calculating the output contribute, as described above. Algorithm 1 shows the automatically extracting
procedure for compiler-typical instructions.
The outline of Algorithm 1 is as follows. First, we preparea binary made by the compiler we want to know its typicalinstructions. Next, we pull out the first L instruction fromthe binary. We identify the compiler by inputting this codefragment into our model. If the classified result is correctand has high confidence (0.99 or higher), then we determine
the instruction that contributes to the identification using themethod described in Sec. IV-B. The offset address for compiler

An example visualization of AttW is shown in Fig. 4.
AttW is a two-dimensional map corresponding to the combination of instructions of Input and Memory. Memoryis the same as Input because we use the Self-Attention. For
5

TABLE I.

identification is shifted one byte at a time, and the appearance
frequency of instructions that contributed to the identification iscounted in the same procedure as described above. Finally, we
display typical instructions in descending order of appearance
frequency.

OVERVIEW OF OUR DATASET. (C ODE ): THE NUMBER OF THE
SHORT CODE FRAGMENTS .

How the performance change with the number of
instructions input to our method? (Sec. V-B)

3

Does our method perform compiler identification better than existing methods? (Sec. V-B)

4

Does our method explain characteristic of each class?
(Sec. V-C)

5

Can our method analyze an executable file in spite oflearning from object files? (Sec. V-D)

6

Can our method explain characteristic of an APTgroup? (Sec. V-E)

x86
x86-64x86#Binaries#Code

VC17,32,none(Od)1,170

369,605

VC17,32,max(Ox)1,147

255,143

VC17,64,none(Od)1,456

540,568

VC17,64,max(Ox)1,242

542,020

VC03,32,none(Od)1,350

292,277

VC03,32,max(Ox)1,306

270,743

-

-

-

-

-

-

GCC,32,none(O0)2,111227,004x86-64

VC2003What is the size of the learning sample required forour method? (Sec. V-B)

2

x86
x86

ICC

In this section, we conduct several experiments to address
the following questions:
1

GCC
Clang

E VALUATION

x86-64V.

ProgramOur model contains the Positional Encoding (PE) we
described in Sec. III-D3. When additional rate (α) was 1.0, theinfluence of PE in the initial stage of the learning often have
let the learning result a local solution. Then, the generalization
performance did not improve. Note that the value of the PE
input (X) is optimized as learning progresses, so it seems
that the value of α does not change the result of the optimal
solution. Therefore we set the α 0.01 temporally for improving
the generalization performance.

x86-64o-glassesX implemented our model is written in Python
and uses the Chainer 4.0.0 framework.

x86

D. o-glassesX’s implementation

x86-64

VC2017

Label

GCC,32,max(O3)1,844

239,821

GCC,64,none(O0)1,582283,276

GCC,64,max(O3)1,580

287,775

Clang,32,none(O0)1,205101,024

Clang,32,max(O3)1,19686,521Clang,64,none(O0)1,892332,278

Clang,64,max(O3)1,883

246,500

ICC,32,none(Od)1,761

1,494,677

ICC,32,max(Ox)1,724

1,161,499

ICC,64,none(Od)1,796

1,419,705

ICC,64,max(Ox)1,728

1,046,958

Others101

912,855

Total

28,074

10,110,249“Program” and comprises various sets of x86/x86-64 machinecode generated from randomly sampled source code from
Github7 . The other category is called “Others” and consistsof various document files and portions of data extracted fromthem. Document files contain data of various kinds: metadata,
text, and packed data. All of these file areas differ not only insize but also in the level of information entropy. On the other
hand, benign document files rarely contain machine code. For
each category and source or file type, we constructed two typesof the dataset: the whole files, and (L × 128)-bit segments ofcode extracted from these files.

In the following, we first describe our dataset, and then we
describe the details of our experiments. The research artifacts
are available at the following URL:

The methods for making each of our types of dataset are
as follows.

https://github.com/yotsubo/o-glassesX1) Binary: The following procedure is conducted for making the “Binary” dataset in the “Program” category.

A. Dataset
We chose C/C++ for the experiment from the various
program languages. According to the TIOBE Index 6 forAugust 2019, among the languages that output native code,these languages are at the top. Among all languages, these havegained more than 20 % share. Additionally, we treat binaries
as x86/x86-64 architecture code for evaluation experimentbecause the architecture occupies the PC market.

•

Gather various C/C++ source code files from GitHub•

Compile these files into x86/x86-64 object files byusing various compilers (TABLE II)•

Extract only the machine code from these object files.

We did not compile the source code files into the executable
files but into the object files. The object file is an intermediate
data representation file including a machine code which is
generated as a result of the compiler processing the sourcecode. On the other hand, the machine code in the executable
files contains various library code; e.g., C run-time libraries.
It is hard to know the compiler condition of these libraries.

Our model input data needs the L fix-length instructions.
The scale of the dataset changes with the value of L. TABLE I
shows an outline of our dataset in the case of L = 16.
We prepared two categories of dataset for our examination,
both of which can be gathered easily. One category is labeled
6 https://www.tiobe.com/tiobe-index/

7 https://github.com/

6

TABLE II.

Compilerfamily
VC
GCC
Clang
ICC

DATASET COMPILATION SETTINGS .

Version
2003
2017
6.3.0
5.0.2
19.0.0.117

TABLE III.

Architecture
x86
x86-64
X
X
X
X
X
X
X
X
X

B. Recognition Performance
We confirmed classification accuracy of compilers by our
proposed method using the dataset described in Sec. V-A.

Optimization Level
Low
High
-Od
-Ox
-Od
-Ox
-O0
-O3
-O0
-O3
-O0
-O3

Evaluation experiments were carried out using the dataset
described in the previous section. No compiler has been able
to compile all source code. As a result, our dataset has a poor
balance between sample numbers of each label. Therefore, weset (S) as the upper limit sample number for each label usedin the evaluation experiment. Accuracy was obtained by 4-foldcross validation, and here is our parameter configuration:

T HE KEYWORD LIST FOR EACH LABEL

Others

CFB
OOXML
PDF

keyword list
“test”,“.doc”“test”,“.docx”“test”,“.pdf”

In addition, we should know function boundaries to divide
the library code and the code generated from the well-known
source code. All the machine code in the object file is the code
derived from the source code. We can obtain ground truth fromthe object files without these difficulties.

•

input length(L) = 16•learning rate (η) = 0.01•

mini-batch size = 1000•

epochs = 50

TABLE IV shows a comparison between the existing
methods and our proposed method. In TABLE IV, theseaccuracy stated the following values. Rosenblum’s accuracy
is the average of the accuracy described in Table 2 of thepaper [29]. ORIGIN’s SVM’s accuracy is the value of “Allcomponent” in Table 2 of the paper [26]. ORIGIN’s CRF’s
accuracy is the value of “Joint” in Table 3 of the paper [26].
Accuracy of BinComp is the average of accuracy describedin Table 6 of the paper [25].

Next, to make the “Binary” datasets in the “Others” category, we used a search engine to gather various open-source
document files. TABLE III shows the keywords used for thissearch. The “Others” category contains “CFB,”“OOXML,” and“PDF” files. CFB stands for compound file binary [22], and
it is used as a container like the FAT16 file system. CFB isused in files with the extensions “.doc,” “.xls,” “.ppt,” and soon. OOXML stands for Office Open XML [16], which is a zip
container in reality. OOXML is used in “.docx,” “.xlsx,” and“.pptx” files. PDF stands for portable document format [15],which has the extension “.pdf.” We downloaded document
files from the beginning of the list of search results. We thenchecked these downloaded files using VirusTotal8 , and weremoved suspicious files that were detected as malware. We
adjusted the number of the document files so that the numberof “Code” (described later) does not differ greatly from themof other labels.

The number of labels included in our dataset is 19, whichis the largest number compared with existing researches.
However, in the paper [26], Rosenbulm et al. are experimenting
with datasets consisting of four versions of GCC and three
versions of VC. Regarding the compiler version differences,
our dataset includes two versions (VC2017/VC2003). On theversion of the compilers, their experiments are more challenging than our experiments.
The increase of input instructions improves the prediction accuracy. The effect is massive impact on our proposalmodel than o-glasses. The existing method o-glasses’
prediction error (1 − Accuracy) changes 0.0677 to 0.0579
when the number of input instructions increases from 16 to
64. The error has decreased by 15%. On the other hand, ourmodel’s prediction error changes 0.0445 to 0.0116 when thenumber of input instructions increases from 16 instructions
to 64 instructions. The error is reduced by about 74%. Oneof the biggest reason for this difference between our model’serror and the o-glasses’ error is that the model contains
PE or not. o-glasses only considered a combination of
instructions. On the other hand, our model containing PE
increased the amount of input information, such as instruction
context and distance. As a result, the effect of the increasing
input instructions led to the prediction accuracy increase.

2) Code: The following procedure is used to make “Code”
datasets. First, we treat the files of the “Binary” dataset
as x86/x86-64 machine code files, whether they come fromthe “Program” category or the “Others” category. Second,
we separate these files into “instructions” (i.e., disassembled
the real or pretended x86/x86-64 machine code). Third, weconvert each instruction into a 128-bit fixed-length instruction
by padding it with “0.” Finally, packing randomly selected L
fixed-length instructions into one set, we make a (L × 128)-bit
sequence.
The reason we padded instructions to 128 bits (16 bytes)
is the following. Fifteen bytes is basically the maximum
length of one instruction as we described in Sec. III-A. The
average of the lengths of each “instruction” is 3.69 bytes
for the“ Program” category and 2.38 bytes for the“ Others”
category. We did not find any instruction longer than 15 bytes
in our experiment. However, theoretically, the length of one
instruction maybe 16 bytes or more. Therefore we set the sizeof fixed-length instructions to 16 bytes (one byte larger thanthe maximum instruction length).

C. Automatic compiler feature extraction
In this section, we list the typical instructions of eachcompiler using Algorithm 1 and consider the knowledge obtained from them. We selected aes.c collected from GitHub
for creating InputFile because all compilers can compile thissource code successfully with all optimization levels.
Fig. 5 shows a list of the top five typical instructions in eachcompiler automatically extracted by the procedure described

8 https://www.virustotal.com/

7

TABLE IV.

#Variations

DatasetAccuracy

All components
Compiler family
Optimization
Architecture
Code/Non-code
ML modelFeatures#Samples#Binaries
VC
GCC
Clang
ICC
Non-code
K-fold cross-validation

C OMPARISON OF PERFORMANCE OF RELATED WORK . T HE NUMBERS IN PARENTHESES DENOTE THE NUMBER OF LABELS .
o-glassesX(L=64)(L=16).9884 (19).9555 (19).9886 (4).9670 (4).9989 (2).9943 (2).9997 (2).9985 (2).9999 (2).9995 (2)Attention
Attention
64 Instructions
16 Instructions
1,793,478
1,900,000
28,07428,074
6
6
4
4
4
4
4
4
1
1
4
4

o-glasses[23](L=64)(L=16).9421 (19).9323 (19).9140 (4).9271 (4).9830 (2).9864 (2).9959 (2).9940 (2).9991 (2).9987 (2)CNN
CNN
64 Instructions
16 Instructions
471,124
1,886,521
28,07428,074
6
6
4
4
4
4
4
4
1
1
4
4

in Sec. IV-C. Considering to Fig. 5, it is possible to catchthe characteristics of each class. Followings are examples ofknowledge obtained in the case of aes.c.

Rosenblum’s[29].924 (3).924 (3)–––
CRF
Byte Seq.
81,886,169
1,119
1
1
1
(Anomalous)

ORIGIN[26]
(SVM)
(CRF)
.604 (18).918 (18).983 (3).999 (3).971 (2).999 (2)
–
–
–
–
SVM
CRF
1 Function
Function Seq.
955,000
955,000
2,686
2,686
6
6
8
8
4
4
(Anomalous)(Anomalous)

BinComp[25].801 (6)–.917 (2)––(k-means)
1 File
1,177
1,177
2
2
2
10

The leftmost Bit-Image in the figure is a visualization of a executable file in the same way as the binary
editor Stirling [10]. One pixel corresponds to one byte,
Bit-Image corresponds to one byte, NULL (0x00) corresponds to white, control character string (0x01 to 0x1F) tolight blue, readable character string (0x20 to 0x7F) to red,and others to black. The remaining three parts of the figure
are colored results of 19 classes classified by our method. On
the left is coloring focusing on the type of compiler, coloring
focusing on Visual C++ in the middle, coloring not focusing
on version differences focusing on Visual C++ on the right.

In the case of low optimization level, many compilers use
IMUL. On the other hand, in the case of high optimizationlevel, rarely compilers use this instruction. IMUL is a multiplication instruction. This instruction takes more than ten times
longer to execute than shift instructions and add instructions.
In the case of a constant multiple such as [eax × 3], a
replaced code which is a combination of shift instruction andaddition instruction such as [(eax << 1) + eax] will increase
the processing speed several times. Therefore, in the case of
maximum optimization, many compilers may not use much
IMUL.

As shown in the figure, the trend of the estimation resultof the optimization option greatly differs in the first half andthe second half of the execution code part. The first half isthe execution code generated without optimization and thesecond half is the execution code generated with the maximum
optimization. As a result of static analysis, the first half was theexecutable code which the author of the executable file thought
to have created, the second half was the static link library. It
was confirmed that the execution code compiled with multiple
optimization options was mixed in the same section due tothe influence of the linked static link library. In addition, ourmethod was able to visualize the situation well.

On the other hand, focusing on VC2003 and VC2017
without optimization, it can be seen that VC2017 prefers
IMUL, and prioritizes the readability of machine language
instructions for debugging. When VC2003 was released, theprocessing time of IMUL was too long to prefer the readability.
When VC2017 was released, the operating frequency of the
CPU was high enough to be able to prefer the readability.
Thus, we consider that Microsoft preferred the readability than
the running speed at the time debugging.

Therefore, when estimating the compiler and optimization
option, it can be inferred that there is a limit to the accuracy ofthe method which does not assume that a plurality of compilers
and optimization options are mixed in the same section. In the
proposed method, input data required for estimation is as small
as 16 instructions, so there is a high possibility that the input
data contains only the execution code generated with a single
compiler and optimization option.

In the case of ICC, we focused on SSE2 instructions;
i.e., PXOR. The SSE2 was first available in Intel Pentium 4(2001) and became popular within a few years. However, atthat time, it had taken time for the code generated by thecompiler to refer to the newly released instruction set becauseof prioritizing compatibility. At present, there is almost no
problem even if we do not consider the processor that cannot
use the SSE2 instruction. Intel, designed x86/x86-64 architectures, developed ICC in order to be more prevalent thelatest instruction set. Even if we are not familiar with thisbackground, the proposed method extracts SSE2 instructions
as a feature that distinguishes compilers.

E. Case Study 2: Tracking the change of Emdivi RATs indevelopment environment
In this section, we describe the change of Emdivi creation
environment by arranging the analysis results of these malware
in order of compilation time. In order to accurately infer thecreation environment of malware by our method, instructions
created by the same compiler used to create the malware
needs to be included in the learning dataset. It is difficult toexactly identify, because the compiler types and options arediverse. However, by analyzing malware in the same family
and arranging the analysis results in chronological order, we
can find changes in the environment used for malware creation.

D. Case Study 1: Various optimization levels in a .text
segment
Fig. 6 shows a result obtained by sliding the position ofa code fragment to be extracted from a certain executable
file one byte from the beginning and visualizing the resultof classification by the learner used in the experiment.
8

VC03,32,none(Od)VC17,32,none(Od)

GCC,32,none(O0)Clang,32,none(O0)ICC,32,none(Od)1 řƃƋɫʫɥɥƄ

1 řřɥɥ

1 řɥɥɥɥɥɥɥ

1 řɥɥɥɥɥɥɥ

1 řřɥɫ

2 řɥɥɥɥɥɥɥ

2 řřɥɰ

2 ř

2 ƃʫʫɥɥƄ

2 řɥɥɥɥɥɥɥ

3 řƃƋɫʫɥɥƄ

3 řřɥ

3 řƃƋɫʫɥɥƄ

3 řɥɥɥɥɥ

3 řřɥɫ

4 řɥɥɥɥɥ

4 ř

4 ƃŞɥɯƄřɥɨ

4 řƃƋɫʫɥɨɥɩɯƄ

4 ɥɫɥɥɪ ɮɭ

5 řƃƋɫʫɥɥƄ

5 řřɥɪ

5 řɥɥɥɥɥ

5 ƃʫɥɥƄ

5 řɥ

VC03,32,max(Ox)VC17,32,max(Ox)

GCC,32,max(O3)Clang,32,max(O3)

ICC,32,max(Ox)1  řɥɥɥɥɥ

1 

1  řƃ ʫɥɥƄ

1 ƃśʫʫɥɥƄ

1 ƃʫɥɥƄ

2 řɥɥɥɥɥ

2 řƃʫƄ

2  řƃ ʫɥɥƄ

2 řɥɥɥɥɥɥɥ

2 řƃʫƄ

3  řɥ

3  ƃʫɥɩɫƄ

3 řɥɥɥɥɥɥɥ

3 řɥɥɥ

3 řƃ ʫ Ƅ

4 ř

4 ƃʫɥɩ Ƅřɥɨ

4 řɥɥɥɥɥ

4 řƃʫɥɨɬƄ

4 řƃ ʫ Ƅ

5 řɥ

5 ř

5 řɥɥɥɥɥ

5  řɥɥɥ

5 ɥřƃʫɥɩɥƄ

Fig. 5.

Typical instructions for each compiler against aes.c.

Fig. 6.

An executable file visualized by our method

Emdivi is a malware family of RAT (Remote AccessTrojan), which were observed many times in Japan from 2014
to 2015. We obtained 162 hashes of Emdivi RATs from the report9 created by MACNICA NETWORKS. We got 148 Emdivi
RATs from Virus Total based on this hash list. We visualized
these files in the same way as in Sec. V-D. As a result, they
are all executable files of the 32 Bit architecture created in
Visual C++, and the optimization option was found to be
the maximum optimization. Analysing of compiler versions,malware development environments were largely classified
into three (Type A, B and C.)

Fig. 7. The result of visualizing Emdivi RATs. Type A is created by arelatively new compiler, and linked libraries are also created by relatively
new compilers. Type B is created by a relatively old compiler, but the linked
libraries are created by a relatively new compiler. Type C is created by arelatively new compiler, but some of the linked libraries are created by arelatively old compiler.

Figure 7 shows the result of visualizing three representative
files by our method. Type A is created by a relatively newcompiler, and linked libraries are also created by relatively
new compilers. Type B is created by a relatively old compiler,
but the linked libraries are created by a relatively new compiler.
Type C is created by a relatively new compiler, but some of
the linked libraries are created by a relatively old compiler.

smaller than 109, they are not described in the table because
Type was all A.
Based on the above results, the change of the structure of
the attack group was inferred by the following procedure.
1) Premise: Characteristics of attack method of attack
group using Emdivi include the following.

Table V shows the results of classifying each file into
the three types mentioned above. “No.” is a number assigned
sequentially from the top of the already-mentioned hash list.
“Version” is a version of Emdivi, inferred from communication
contents. “Compile Times” is a compile date and time, which
is based on header information. For types whose “No.” are9 https://www.macnica.net/security/report 01.html/

9•

Attach to the e-mail just before sending the targeted
mail t17 series malware is compiled.

•

After successful intrusion, another malware (t20 series) is sent to another terminal in the attack target
system network in order to expand the intrusion.

TABLE V.

T RANSITION OF DEVELOPMENT ENVIRONMENT OF E MDIVI

No. Version

Compile Times

Type

No. Version

Compile Times

109 t17.08.27

2015/03/19 15:03:19

A

137 t20.22

2015/07/06 15:32:47

B

110 t17.08.27

2015/03/20 12:04:19

A

138 t17.08.31

2015/07/06 10:34:56

B

111 t17.08.27

2015/03/20 10:44:49

A

139 t17.08.31

2015/07/10 09:58:16

B

113 t17.08.27

2015/03/24 12:07:23

A

140 t20.22.1

2015/07/10 08:49:45

A

115 t17.08.29

2015/04/22 11:29:48

B

141 t17.08.31

2015/07/10 08:40:15

B

117 t17.08.29

2015/05/08 10:20:53

A

142 t20.22.1

2015/07/10 09:10:41

A

118 t20.19

2015/05/20 17:25:17

A

143 t17.08.31

2015/07/13 00:23:13

B

119 t20.19

2015/05/20 17:38:52

A

144 t17.08.31

2015/07/13 10:46:27

B

120 t17.08.30

2015/05/20 15:42:48

A

145 t17.08.31

2015/07/14 09:57:44

B

121 t20.19

2015/05/20 17:00:39

A

146 t17.08.31

2015/07/14 10:16:54

B

Type

122 t17.08.30

2015/05/20 11:52:28

A

147 t17.08.31

2015/07/14 17:44:14

B

123 t20.19

2015/05/21 15:10:03

A

148 t17.08.31

2015/07/16 09:10:07

B

124 t20.19

2015/05/21 14:08:10

A

149 t17.08.31

2015/07/28 12:56:35

B

125 t17.08.30

2015/05/21 15:38:39

A

150 t20.23.1

2015/07/31 17:35:52

A

126 t17.08.30

2015/05/21 15:38:39

A

151 t20.23.1

2015/07/31 17:03:49

A

127 t17.08.30

2015/05/22 11:51:18

A

152 t17.08.31

2015/08/05 08:51:31

B

128 t17.08.30

2015/05/22 11:51:18

A

153 t17.08.34

2015/08/07 09:23:11

C

129 t17.08.30

2015/05/22 11:51:18

A

154 t20.25.1

2015/08/07 13:11:08

A

130 t20.20

2015/05/27 11:07:55

A

155 t17.08.34

2015/08/10 14:47:52

C

131 t17.08.30

2015/05/28 12:48:14

A

156 t17.08.34

2015/08/13 08:48:01

C

132 t20.20

2015/05/29 11:19:00

A

157 t17.08.34

2015/08/13 09:35:15

C

133 t17.08.30

2015/06/02 11:15:26

A

158 t20.26

2015/08/13 13:21:57

A

134 t20.20

2015/06/03 11:50:00

A

159 t20.26

2015/08/13 13:29:34

A

135 t20.20

2015/06/04 15:12:36

A

160 t17.08.34

2015/08/19 09:16:01

C

136 t17.08.31

2015/06/18 10:15:02

B

161 t17.08.34

2015/08/19 09:16:01

C

136 t17.08.31

2015/06/18 10:15:02

B

162 t17.08.34

2015/10/13 09:52:52

C•2) Proposal of a model that can calculate how much
input data contributes to output in units of instructions: The
attention mechanism is rapidly spreading in natural language
processing in recent years. In the field of machine translation,
the attention model without RNN has obtained a translation
score that exceeds the existing research using RNN [34].
Additionally, the model with the attention mechanism cancalculate the degree of contribution to the decision.
In this paper, we applied the attention mechanism tocode fragment recognition. At the best of our knowledge,
o-glassesX is the first example of applying the attention
mechanism to machine code classification. For calculating thecontribution to the identification result one instruction by one
instruction, we designed our network carefully. Our model cancalculate the degree of the contribution to the identification
result on each instruction.
3) Knowledge gained from model judgment grounds: Ourmodel can identify the most important instruction of the input.
Therefore, by counting the appearance frequency of instructions that contributed to the compiler identification results, we
can extract the typical instructions in each class automatically.
We listed the compiler-typical instructions generated whencompiling aes.c with various compilers. As a result, wecaught some of the features of each compiler. Our method
helps us to understand the characteristics of each class.

t20 series of malware is compiled just before sending.

B. The limitation of our method

2) Assumption: The compilation time and the time totransmit malware are very close, so we made the following
assumptions. The person who compiles, the person who sends
malware, and the person who uses malware are highly likely
to be the same person.

o-glassesX still suffers from some limitations. For
instance, like most existing methods, o-glassesX works
under the assumption that the machine code is already deobfuscated. In practice, de-obfuscated of malware can be very
demanding. How to extract compiler provenance directly overobfuscated code is an important but very challenging future
avenue of research. On the other hand, when we success deobfuscate code snippet, we can identify source compiler sinceo-glassesX does not require meta-data.

3) Activities of A, B and C:ABoth initial invasion and invasion expansion staff.
After June 2015, mainly invasion expansion staff.

B

Initial invasion staff (activity period: mid-June 2015
to early August)C

Additionally, our method cannot support two or more CPUarchitectures at the same time. When dealing with two or
more CPU architecture inputs at the same time, first we need
identifying the CPU architecture from the input binary, andthen it is necessary to input the binary to o-glassesX
specialized for each architecture.

Initial invasion staff (activity period: after early August 2015)

4) Guess: Although the organization before June 2015 isunclear, there is a possibility that from 2016 June onwards,
responsible for intrusion (targeted mail sending) and explicit
role sharing of intrusion expansion after successful entry isthere.
VI.

VII.

C ONCLUSION

In this paper, we proposed a novel method for binary analysis. To the best of our knowledge, this is the first applicationof the attention mechanism to machine code recognition. Our
method can identify the source compiler family and optimization levels of machine code, an important element of program
provenance. Additionally, our method can demonstrate howmuch one instruction by one in 16 or 64 instruction windows
contributed to the decision of the compiler family and the
optimization level.

D ISCUSSION

A. Benefits of our method
1) High Recognition Rate for Stripped Machine Code: Itis applicable even when such information has been stripped
or is otherwise unavailable because our approach relies only
on characteristics of the binary code and not on meta-data
or other details of program headers. Furthermore, it can beused even when codes produced by multiple compilers coexist
within a program binary, such as statically linked library code
because our method classifies sequences of code instead ofwhole binaries. Our approach extracts compiler provenancewith high accuracy even in such complex programs.

We evaluated a large set of test binaries. Our result showedthat our method could identify the source compiler family andoptimization levels of machine code with about 0.99 accuracy
rate almost perfect, even though the size of the input binary
fragment is only 64 instructions. Additionally, our case study
10showed that our method provides new knowledge; e.g., thecharacteristic of instruction choices by each compiler.

[18]

It is left as an open problem to find a solution to moregeneral program provenance and author identification frombinary codes (malware) using the compiler provenance foundby o-glassesX

[19]

[20]

R EFERENCES
[21]
[1]

D. Bahdanau, K. Cho, and Y. Bengio, “Neural machine translation by
jointly learning to align and translate,” ICLR2015, 2015.

[2]

T. Bao, J. Burket, M. Woo, R. Turner, and D. Brumley,“BYTEWEIGHT: Learning to recognize functions in binary code,” in23rd USENIX Security Symposium (USENIX Security 14). San Diego,
CA: USENIX Association, 2014, pp. 845–860.

[22]
[23]

[3]

J. Caballero, N. M. Johnson, S. McCamant, and D. Song, “Binary
code extraction and interface identification for security applications,”in Proceedings of n Proceedings of the 17th Network and Distributed
System Security Symposium. The Internet Society, 2010.

[24]

[4]

A. Caliskan, F. Yamaguchi, E. Dauber, R. Harang, K. Rieck, R. Greenstadt, and A. Narayanan, “When coding style survives compilation:
De-anonymizing programmers from executable binaries,” in the 2018
Network and Distributed System Security Symposium (NDSS), 02 2018.

[25]

[5]

L. Cheng, Y. Zhang, Y. Zhang, C. Wu, Z. Li, Y. Fu, and H. Li, “Optimizing seed inputs in fuzzing with machine learning,” in Proceedings ofthe 41st International Conference on Software Engineering: Companion
Proceedings. IEEE Press, 2019, pp. 244–245.

[6]

C. Cortes and V. Vapnik, “Support-vector networks,” Machine learning,
vol. 20, no. 3, pp. 273–297, 1995.

[7]

Q. Feng, R. Zhou, C. Xu, Y. Cheng, B. Testa, and H. Yin, “Scalable
graph-based bug search for firmware images,” in Proceedings of the2016 ACM SIGSAC Conference on Computer and Communications
Security. ACM, 2016, pp. 480–491.

[8]

Q. Feng, R. Zhou, Y. Zhao, J. Ma, Y. Wang, N. Yu, X. Jin, J. Wang,
A. Azab, and P. Ning, “Learning binary representation for automatic
patch detection,” in 2019 16th IEEE Annual Consumer Communications& Networking Conference (CCNC). IEEE, 2019, pp. 1–6.

[29]

[9]

X. Glorot, A. Bordes, and Y. Bengio, “Deep sparse rectifier neuralnetworks,” in Proceedings of the fourteenth international conference
on artificial intelligence and statistics, 2011, pp. 315–323.

[30]

[10]

K. Goto, “Stirling,” https://www.vector.co.jp/soft/win95/util/se079072.
html, 1998.

[11]

G. Grieco, G. L. Grinblat, L. Uzal, S. Rawat, J. Feist, and L. Mounier,“Toward large-scale vulnerability discovery using machine learning,”
in Proceedings of the Sixth ACM Conference on Data and Application
Security and Privacy. ACM, 2016, pp. 85–96.

[12]

J. J. Hopfield, “Neural networks and physical systems with emergent
collective computational abilities,” Proceedings of the national academyof sciences, vol. 79, no. 8, pp. 2554–2558, 1982.

[13]

[26]

[27]

[28]

[31]

[32]

[33]
[34]

Intel, “Intel 64 and ia-32 architectures software developer manuals,”2016. [Online]. Available: https://software.intel.com/en-us/articles/
intel-sdm[14]

S. Ioffe and C. Szegedy, “Batch normalization: Accelerating deep
network training by reducing internal covariate shift,” arXiv preprint
arXiv:1502.03167, 2015.

[15]

ISO, “ISO32000–1:2008 Document management – Portable document
format – Part 1 : PDF1.7.” [Online]. Available: http://www.iso.org/iso/
iso catalogue/catalogue tc/catalogue detail.htm?csnumber=51502[16]——, “ISO/IEC 29500:2012:Information technology – Document description and processing languages – Office Open XML File Formats,”
2012.

[17]

A. Krizhevsky, I. Sutskever, and G. E. Hinton, “Imagenet classification
with deep convolutional neural networks,” in Advances in Neural Information Processing Systems 25, F. Pereira, C. J. C. Burges, L. Bottou,
and K. Q. Weinberger, Eds. Curran Associates, Inc., 2012, pp. 1097–
1105.

[35]

[36]

[37]

[38]

11

Y. LeCun, B. Boser, J. S. Denker, D. Henderson, R. E. Howard,
W. Hubbard, and L. D. Jackel, “Backpropagation applied to handwritten
zip code recognition,” Neural Computation, vol. 1, no. 4, pp. 541–551,
1989.
M.-T. Luong, H. Pham, and C. D. Manning, “Effective approaches to attention-based neural machine translation,” arXiv preprint
arXiv:1508.04025, 2015.
A. McCallum, “Efficiently inducing features of conditional random
fields,” in Proceedings of the Nineteenth conference on Uncertainty in
Artificial Intelligence. Morgan Kaufmann Publishers Inc., 2002, pp.
403–410.
X. Meng, B. P. Miller, and K.-S. Jun, “Identifying multiple authors ina binary program,” in ESORICS, 2017.
Microsoft, “[ms-cfb]: Compound file binary file format.” [Online].
Available: https://msdn.microsoft.com/ja-jp/library/dd942138.aspx
Y. Otsubo, A. Otsuka, M. Mimura, T. Sakaki, and A. Goto, “o-glasses:
Visualizing x86 code from binary using a 1d-cnn,” arXiv preprint
arXiv:1806.05328, 2018.
B. M. Padmanabhuni and H. B. K. Tan, “Buffer overflow vulnerability
prediction from x86 executables using static analysis and machine learning,” in 2015 IEEE 39th Annual Computer Software and Applications
Conference, vol. 2. IEEE, 2015, pp. 450–459.
A. Rahimian, P. Shirani, S. Alrbaee, L. Wang, and M. Debbabi,“Bincomp: A stratified approach to compiler provenance attribution,”
Digital Investigation, vol. 14, pp. S146–S155, 2015.
N. Rosenblum, B. P. Miller, and X. Zhu, “Recovering the toolchain
provenance of binary code,” in Proceedings of the 2011 International
Symposium on Software Testing and Analysis. ACM, 2011, pp. 100–
110.
N. Rosenblum, X. Zhu, B. Miller, and K. Hunt, “Machine learningassisted binary code analysis,” in NIPS Workshop on Machine Learning
in Adversarial Environments for Computer Security, Whistler, British
Columbia, Canada, December, 2007.
N. Rosenblum, X. Zhu, and B. P. Miller, “Who wrote this code?
identifying the authors of program binaries,” in European Symposium
on Research in Computer Security. Springer, 2011, pp. 172–189.
N. E. Rosenblum, B. P. Miller, and X. Zhu, “Extracting compiler
provenance from program binaries,” in Proceedings of the 9th ACM
SIGPLAN-SIGSOFT workshop on Program analysis for software tools
and engineering. ACM, 2010, pp. 21–28.
N. E. Rosenblum, X. Zhu, B. P. Miller, and K. Hunt, “Learning toanalyze binary computer code.” in AAAI, 2008, pp. 798–804.
A. Sæbjørnsen, J. Willcock, T. Panas, D. Quinlan, and Z. Su, “Detectingcode clones in binary executables,” in Proceedings of the EighteenthInternational Symposium on Software Testing and Analysis, ser. ISSTA’09. ACM, 2009, pp. 117–128.
E. C. R. Shin, D. Song, and R. Moazzezi, “Recognizing functions inbinaries with neural networks,” in 24th USENIX Security Symposium(USENIX Security 15), 2015, pp. 611–626.
G. J. Tesauro, J. O. Kephart, and G. B. Sorkin, “Neural networks forcomputer virus recognition,” IEEE expert, vol. 11, no. 4, pp. 5–6, 1996.
A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez,
Ł. Kaiser, and I. Polosukhin, “Attention is all you need,” in Advancesin neural information processing systems, 2017, pp. 5998–6008.
S. Wang, P. Wang, and D. Wu, “Semantics-aware machine learning
for function recognition in binary code,” in 2017 IEEE International
Conference on Software Maintenance and Evolution (ICSME). IEEE,
2017, pp. 388–398.
Y. Wang, Z. Wu, Q. Wei, and Q. Wang, “Neufuzz: Efficient fuzzing
with deep neural network,” IEEE Access, vol. 7, pp. 36 340–36 352,
2019.
H. Yakura, S. Shinozaki, R. Nishimura, Y. Oyama, and J. Sakuma,“Malware analysis of imaged binary samples by convolutional neuralnetwork with attention mechanism,” in Proceedings of the Eighth
ACM Conference on Data and Application Security and Privacy, ser.
CODASPY ’18. ACM, 2018, pp. 127–134.
F. Yamaguchi, A. Maier, H. Gascon, and K. Rieck, “Automatic inference of search patterns for taint-style vulnerabilities,” in 2015 IEEE
Symposium on Security and Privacy. IEEE, 2015, pp. 797–812.