Original: M AXIMAL D IVERGENCE S EQUENTIAL A UTOENCODER FOR B INARY S OFTWARE V ULNERABILITY D ETECTION
Revised: M AXIMAL D IVERGENCE S EQUENTIAL A UTOENCODER FOR B INARY S OFTWARE V ULNERABILITY D ETECTION

Original: ∗
Revised: ∗

Original: A BSTRACT
Revised: A BSTRACT

Original: Due to the sharp increase in the severity of the threat imposed by software vulnerabilities, the detection of vulnerabilities in binary code has become an important concern in the software industry, such as the embedded systems industry, and in the ﬁeld of computer security.
Revised: Due to the sharp increase in the severity of the threat imposed by software vulnerabilities, the detection of vulnerabilities in binary code has become an important concern in the software industry, such as the embedded systems industry, and in the ﬁeld of computer security.

Original: However, most of the works in binary code vulnerability detection has relied on handcrafted features which are manually chosen by a select few domain experts.
Revised: However, most of the works in binary code vulnerability detection has relied on handcrafted features which are manually chosen by a select few domain experts.

Original: In this paper, we attempt to alleviate this severe binary vulnerability detection bottleneck by leveraging recent advances in deep learning representations and propose the Maximal Divergence
Revised: In this paper, we attempt to alleviate this severe binary vulnerability detection bottleneck by leveraging recent advances in deep learning representations and propose the Maximal Divergence

Original: Sequential Auto-Encoder.
Revised: Sequential Auto-Encoder.

Original: In particular, latent codes representing vulnerable and non-vulnerable binaries are encouraged to be maximally divergent, while still being able to maintain crucial information from the original binaries.
Revised: In particular, latent codes representing vulnerable and non-vulnerable binaries are encouraged to be maximally divergent, while still being able to maintain crucial information from the original binaries.

Original: We conducted extensive experiments to compare and contrast our proposed methods with the baselines, and the results indicate that our proposed methods outperform the baselines in all performance measures of interest.
Revised: We conducted extensive experiments to compare and contrast our proposed methods with the baselines, and the results indicate that our proposed methods outperform the baselines in all performance measures of interest.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: Software vulnerabilities are speciﬁc ﬂaws or oversights in a piece of software that allow attackers to perform malicious acts including exposing or altering sensitive information, disrupting or destroying a system, or taking control of a computer system or program (Dowd et al., 2006).
Revised: Software vulnerabilities are speciﬁc ﬂaws or oversights in a piece of software that allow attackers to perform malicious acts including exposing or altering sensitive information, disrupting or destroying a system, or taking control of a computer system or program (Dowd et al., 2006).

Original: Due to the ubiquity of computer software, the growth and the diversity in its development process, a great amount of computer software potentially includes software vulnerabilities.
Revised: Due to the ubiquity of computer software, the growth and the diversity in its development process, a great amount of computer software potentially includes software vulnerabilities.

Original: This fact makes the problem of software security vulnerability identiﬁcation an important concern in the software industry and in the ﬁeld of computer security.
Revised: This fact makes the problem of software security vulnerability identiﬁcation an important concern in the software industry and in the ﬁeld of computer security.

Original: Although a great effort has been made by the security community, the severity of the threat from software vulnerabilities has gradually increased over the years.
Revised: Although a great effort has been made by the security community, the severity of the threat from software vulnerabilities has gradually increased over the years.

Original: Numerous exist of examples and incidents in the past two decades in which software vulnerabilities have imposed signiﬁcant damages to companies and individuals (Ghaffarian & Shahriari, 2017).
Revised: Numerous exist of examples and incidents in the past two decades in which software vulnerabilities have imposed signiﬁcant damages to companies and individuals (Ghaffarian & Shahriari, 2017).

Original: For example, vulnerabilities in popular browser plugins have threatened the security and privacy of millions of Internet users (e.g., Adobe Flash Player (US-CERT 2015; Adobe Security Bulletin 2015) and Oracle Java (US-CERT 2013)), vulnerabilities in popular and fundamental open-source software have also threatened the security of thousands of companies and their customers around the globe (e.g., Heartbleed (Codenomicon 2014) and ShellShock (Symantec Security Response 2014).
Revised: For example, vulnerabilities in popular browser plugins have threatened the security and privacy of millions of Internet users (e.g., Adobe Flash Player (US-CERT 2015; Adobe Security Bulletin 2015) and Oracle Java (US-CERT 2013)), vulnerabilities in popular and fundamental open-source software have also threatened the security of thousands of companies and their customers around the globe (e.g., Heartbleed (Codenomicon 2014) and ShellShock (Symantec Security Response 2014).

Original: Software vulnerability detection (SVD) can be categorized into source code and binary code vulnerability detection.
Revised: Software vulnerability detection (SVD) can be categorized into source code and binary code vulnerability detection.

Original: Source code vulnerability detection has been widely studied in a variety of works (Shin et al., 2011; Neuhaus et al., 2007; Yamaguchi et al., 2011; Li et al., 2016;
Revised: Source code vulnerability detection has been widely studied in a variety of works (Shin et al., 2011; Neuhaus et al., 2007; Yamaguchi et al., 2011; Li et al., 2016;

Original: Kim et al., 2017; Li et al., 2018).
Revised: Kim et al., 2017; Li et al., 2018).

Original: Most of the previous work in source code vulnerability detection (Neuhaus et al., 2007; Shin et al., 2011; Yamaguchi et al., 2011; Li et al., 2016;
Revised: Most of the previous work in source code vulnerability detection (Neuhaus et al., 2007; Shin et al., 2011; Yamaguchi et al., 2011; Li et al., 2016;

Original: Kim et al., 2017) has been based on handcrafted features which are manually chosen by a limited number of domain experts.
Revised: Kim et al., 2017) has been based on handcrafted features which are manually chosen by a limited number of domain experts.

Original: To mitigate the dependency on handcrafted features, the use of automatic features in SVD has been studied recently in (Dam et al., 2017; Li et al., 2018; Lin et al., 2018).
Revised: To mitigate the dependency on handcrafted features, the use of automatic features in SVD has been studied recently in (Dam et al., 2017; Li et al., 2018; Lin et al., 2018).

Original: In particular, Dam et al.
Revised: In particular, Dam et al.

Original: ; Lin et al.
Revised: ; Lin et al.

Original: (2018) employed a Recurrent Neural Network (RNN) to transform sequences of code tokens to vectorial features, which are further fed to a separate classiﬁer, while Li et al.
Revised: (2018) employed a Recurrent Neural Network (RNN) to transform sequences of code tokens to vectorial features, which are further fed to a separate classiﬁer, while Li et al.

Original: combined learning the vector representation and the training of the classiﬁer in a deep network.
Revised: combined learning the vector representation and the training of the classiﬁer in a deep network.

Original: Compared with source code vulnerability detection, binary code vulnerability detection is signiﬁcantly more difﬁcult because much of the syntactic and semantic information provided by high-level programming languages is lost during the compilation process.
Revised: Compared with source code vulnerability detection, binary code vulnerability detection is signiﬁcantly more difﬁcult because much of the syntactic and semantic information provided by high-level programming languages is lost during the compilation process.

Original: The existence of such syntactic and semantic information makes it easier to reason how data and inputs drive the paths of execution.
Revised: The existence of such syntactic and semantic information makes it easier to reason how data and inputs drive the paths of execution.

Original: Unfortunately, a software binary, such as proprietary binary code (with no access to source code) or embedded systems code, is generally all that is made available for code analysis (together perhaps with the processor architecture such as x86 etc.).
Revised: Unfortunately, a software binary, such as proprietary binary code (with no access to source code) or embedded systems code, is generally all that is made available for code analysis (together perhaps with the processor architecture such as x86 etc.).

Original: The ability to detect the presence or absence of vulnerabilities in binary code, without getting access to source code, is therefore a major importance in the context of computer security.
Revised: The ability to detect the presence or absence of vulnerabilities in binary code, without getting access to source code, is therefore a major importance in the context of computer security.

Original: Some work has been proposed to detect vulnerabilities at the binary code level when source code is not available, notably work based on fuzzing, symbolic execution (Cadar & Sen, 2013; Avancini & Ceccato, 2013; Meng et al., 2016), or techniques using handcrafted features extracted from dynamic analysis (Grieco et al., 2016; Cozzie et al., 2008; White & L ¨ uttgen, 2013).
Revised: Some work has been proposed to detect vulnerabilities at the binary code level when source code is not available, notably work based on fuzzing, symbolic execution (Cadar & Sen, 2013; Avancini & Ceccato, 2013; Meng et al., 2016), or techniques using handcrafted features extracted from dynamic analysis (Grieco et al., 2016; Cozzie et al., 2008; White & L ¨ uttgen, 2013).

Original: To the best of our knowledge, there has been no work studying the use of automatically extracted features for binary code vulnerability detection, though there has been some work using automatic features in conjunction with deep learning methods for malware detection, notably (Saxe & Berlin, 2015; Raff et al., 2017).
Revised: To the best of our knowledge, there has been no work studying the use of automatically extracted features for binary code vulnerability detection, though there has been some work using automatic features in conjunction with deep learning methods for malware detection, notably (Saxe & Berlin, 2015; Raff et al., 2017).

Original: It is worth noting that binary code vulnerability detection and malware detection are two different tasks.
Revised: It is worth noting that binary code vulnerability detection and malware detection are two different tasks.

Original: In particular, binary code vulnerability detection aims to detect speciﬁc ﬂaws or oversights in binary code, while malware detection aims to detect if a given binary is malicious or not.
Revised: In particular, binary code vulnerability detection aims to detect speciﬁc ﬂaws or oversights in binary code, while malware detection aims to detect if a given binary is malicious or not.

Original: The former is arguably harder in the sense that vulnerable and non-vulnerable binaries might be only slightly different, while there might be a clearer difference in general between malware and benign binaries.
Revised: The former is arguably harder in the sense that vulnerable and non-vulnerable binaries might be only slightly different, while there might be a clearer difference in general between malware and benign binaries.

Original: In addition, a signiﬁcant constraint in research on binary code vulnerability detection is the lack of suitable binaries labeled as either vulnerable or non-vulnerable.
Revised: In addition, a signiﬁcant constraint in research on binary code vulnerability detection is the lack of suitable binaries labeled as either vulnerable or non-vulnerable.

Original: Although we have some source code datasets for software vulnerability detection, to the best of our knowledge, there exists no large public binary dataset for the purpose of binary code vulnerability detection.
Revised: Although we have some source code datasets for software vulnerability detection, to the best of our knowledge, there exists no large public binary dataset for the purpose of binary code vulnerability detection.

Original: The reason is that most source code in source code vulnerability detection datasets is not compilable due to incompleteness, and they have important pieces missing (e.g., variables, data types) and relevant libraries – making the code compilable take a large effort in ﬁxing a vast volume of source code.
Revised: The reason is that most source code in source code vulnerability detection datasets is not compilable due to incompleteness, and they have important pieces missing (e.g., variables, data types) and relevant libraries – making the code compilable take a large effort in ﬁxing a vast volume of source code.

Original: This arises from the nature of the process that involves collecting and labeling source code wherein we start from security reports in CVE 1 and navigate through relevant websites to obtain code snippets of vulnerable and non-vulnerable source code.
Revised: This arises from the nature of the process that involves collecting and labeling source code wherein we start from security reports in CVE 1 and navigate through relevant websites to obtain code snippets of vulnerable and non-vulnerable source code.

Original: In this work, we leverage recent advances in deep learning to derive the automatic features of binary code for vulnerability detection.
Revised: In this work, we leverage recent advances in deep learning to derive the automatic features of binary code for vulnerability detection.

Original: In particular, we view a given binary as a sequence of machine instructions and then use the theory of Variational Auto-Encoders (VAE) (Kingma &
Revised: In particular, we view a given binary as a sequence of machine instructions and then use the theory of Variational Auto-Encoders (VAE) (Kingma &

Original: Welling, 2013) to develop the Maximal Divergence Sequential Auto-Encoder (MDSAE) that can work out representations of binary code in such a way that representations of vulnerable and nonvulnerable binaries are encouraged to be maximally different for vulnerability detection purposes, while still preserving crucial information inherent in the original binaries.
Revised: Welling, 2013) to develop the Maximal Divergence Sequential Auto-Encoder (MDSAE) that can work out representations of binary code in such a way that representations of vulnerable and nonvulnerable binaries are encouraged to be maximally different for vulnerability detection purposes, while still preserving crucial information inherent in the original binaries.

Original: In contrast to the original
Revised: In contrast to the original

Original: VAE wherein the data prior is kept ﬁxed, we propose using two learnable Gaussian priors, one for each class.
Revised: VAE wherein the data prior is kept ﬁxed, we propose using two learnable Gaussian priors, one for each class.

Original: Based on the VAE principle, latent codes (i.e., data representations) are absorbed (or compressed) into the data prior distribution, we further propose maximizing a divergence (e.g., Wasserstein (WS) distance or Kullback-Leibler (KL) divergence) between these two priors to separate representations of vulnerable and non-vulnerable binaries.
Revised: Based on the VAE principle, latent codes (i.e., data representations) are absorbed (or compressed) into the data prior distribution, we further propose maximizing a divergence (e.g., Wasserstein (WS) distance or Kullback-Leibler (KL) divergence) between these two priors to separate representations of vulnerable and non-vulnerable binaries.

Original: Our MDSAE can be used to produce data representations for another independent classiﬁer (e.g., Support Vector Machine or Random Forest) or incorporated with a shallow feedforward neural network built on top of the latent codes for simultaneously training both the mechanism to generate data representations and the classiﬁer.
Revised: Our MDSAE can be used to produce data representations for another independent classiﬁer (e.g., Support Vector Machine or Random Forest) or incorporated with a shallow feedforward neural network built on top of the latent codes for simultaneously training both the mechanism to generate data representations and the classiﬁer.

Original: The former is named MDSAE-R and the latter is named MDSAE-C.
Revised: The former is named MDSAE-R and the latter is named MDSAE-C.

Original: We summarize our contributions in this paper as follows
Revised: We summarize our contributions in this paper as follows

Original: : • We propose a novel method named Maximal Divergence Sequential Auto-Encoder (MDSAE) that leverages recent advances in deep learning representation (namely, VAE) for binary code vulnerability detection.
Revised: : • We propose a novel method named Maximal Divergence Sequential Auto-Encoder (MDSAE) that leverages recent advances in deep learning representation (namely, VAE) for binary code vulnerability detection.

Original: • One of our most signiﬁcant contributions is to create a labeled dataset for use in binary code vulnerability detection.
Revised: • One of our most signiﬁcant contributions is to create a labeled dataset for use in binary code vulnerability detection.

Original: In particular, we used the source code in the published NDSS18 dataset used in (Li et al., 2018) and then extracted vulnerable and non-vulnerable functions.
Revised: In particular, we used the source code in the published NDSS18 dataset used in (Li et al., 2018) and then extracted vulnerable and non-vulnerable functions.

Original: We developed a tool that can automatically detect the syntactical errors in a given piece of source code, ﬁx them, and ﬁnally compile the ﬁxed source code into binaries for various platforms (both Windows OS and Linux OS) and architectures (both x86 and x86-64 processors).
Revised: We developed a tool that can automatically detect the syntactical errors in a given piece of source code, ﬁx them, and ﬁnally compile the ﬁxed source code into binaries for various platforms (both Windows OS and Linux OS) and architectures (both x86 and x86-64 processors).

Original: Speciﬁcally, after preprocessing and ﬁltering out identical functions from the NDSS18 source code dataset, we obtain 13 , 000 functions of which 9 , 000 are able to be ﬁxed and compiled to binaries.
Revised: Speciﬁcally, after preprocessing and ﬁltering out identical functions from the NDSS18 source code dataset, we obtain 13 , 000 functions of which 9 , 000 are able to be ﬁxed and compiled to binaries.

Original: By compiling the source code of these functions under the various platform and architecture options, we obtain 32 , 281 binary functions including 17 , 977 binaries for Windows and 14 , 304 binaries for Linux.
Revised: By compiling the source code of these functions under the various platform and architecture options, we obtain 32 , 281 binary functions including 17 , 977 binaries for Windows and 14 , 304 binaries for Linux.

Original: • We conducted extensive experiments on the NDSS18 binary dataset.
Revised: • We conducted extensive experiments on the NDSS18 binary dataset.

Original: The experimental results show that the two variants MDSAE-R and MDSAE-C outperform the baselines in all performance measures of interest.
Revised: The experimental results show that the two variants MDSAE-R and MDSAE-C outperform the baselines in all performance measures of interest.

Original: It is not surprising that MDSAE-C achieves higher predictive performances compared with MDSAE-R, but the fact that MDSAE-R achieves good predictive performances conﬁrms our hypothesis of encouraging the separation in representations of data in different classes so that a simple linear classiﬁer subsequently trained on these data representations can obtain good predictive results.
Revised: It is not surprising that MDSAE-C achieves higher predictive performances compared with MDSAE-R, but the fact that MDSAE-R achieves good predictive performances conﬁrms our hypothesis of encouraging the separation in representations of data in different classes so that a simple linear classiﬁer subsequently trained on these data representations can obtain good predictive results.

Original:  R ELATED B ACKGROUND 2. T HE V ARIATIONAL A UTO - ENCODER
Revised:  R ELATED B ACKGROUND 2. T HE V ARIATIONAL A UTO - ENCODER

Original: The Variational Auto-Encoder (VAE) (Kingma & Welling, 2013) is a probabilistic auto-encoder that takes into account both the reconstruction of true samples and generalization of samples generated from a latent space.
Revised: The Variational Auto-Encoder (VAE) (Kingma & Welling, 2013) is a probabilistic auto-encoder that takes into account both the reconstruction of true samples and generalization of samples generated from a latent space.

Original: The underlying idea is to learn a probabilistic decoder p θ ( x | z ) , z ∼ N ( 0 , I ) that can mimic the true data sample x 1 , . . .
Revised: The underlying idea is to learn a probabilistic decoder p θ ( x | z ) , z ∼ N ( 0 , I ) that can mimic the true data sample x 1 , . . .

Original: , x N drawn from an existing but unknown data distribution p d ( x ) .
Revised: , x N drawn from an existing but unknown data distribution p d ( x ) .

Original: VAE is developed based on the following lower bound:
Revised: VAE is developed based on the following lower bound:

Original: [Equation]
Revised: [Equation]

Original: where q φ ( z | x ) is the approximate posterior distribution.
Revised: where q φ ( z | x ) is the approximate posterior distribution.

Original: We need to maximize the log likelihood at each training example x .
Revised: We need to maximize the log likelihood at each training example x .

Original: Therefore the objective function is of the following form:
Revised: Therefore the objective function is of the following form:

Original: [Equation]
Revised: [Equation]

Original: (1) where x is drawn from the empirical data distribution.
Revised: (1) where x is drawn from the empirical data distribution.

Original: To reduce the variance when using Monte Carlo (MC) estimation for tackling the above optimization problem, the reparameterization trick is employed.
Revised: To reduce the variance when using Monte Carlo (MC) estimation for tackling the above optimization problem, the reparameterization trick is employed.

Original: More speciﬁcally, assuming that q φ ( z | x ) =
Revised: More speciﬁcally, assuming that q φ ( z | x ) =

Original: N ( z | µ φ ( x ) , diag ( σ φ ( x ))) , we can do reparameterization as:
Revised: N ( z | µ φ ( x ) , diag ( σ φ ( x ))) , we can do reparameterization as:

Original: z = µ φ ( x ) + diag ( σ φ ( x )) 1 / 2 (cid:15) where the source of randomness (cid:15) ∼ N ( 0 , I ) and µ φ ( z ) , σ φ ( z ) are two neural networks representing the mean and covariance matrix of the approximate Gaussian posterior.
Revised: z = µ φ ( x ) + diag ( σ φ ( x )) 1 / 2 (cid:15) where the source of randomness (cid:15) ∼ N ( 0 , I ) and µ φ ( z ) , σ φ ( z ) are two neural networks representing the mean and covariance matrix of the approximate Gaussian posterior.

Original: The optimization problem in Eq.
Revised: The optimization problem in Eq.

Original: (1) can be equivalently rewritten as:
Revised: (1) can be equivalently rewritten as:

Original: [Equation]
Revised: [Equation]

Original: The ﬁrst term in Eq.
Revised: The ﬁrst term in Eq.

Original: (2) is regarded as the reconstruction term and the second term in this equation is regarded as the regularization term.
Revised: (2) is regarded as the reconstruction term and the second term in this equation is regarded as the regularization term.

Original: In this term, we minimize E x [ D KL
Revised: In this term, we minimize E x [ D KL

Original: q φ ( z | x ) (cid:107) p ( z ))] , hence trying to compress and squash the latent codes z for each true example x into those sampled from the prior distribution p ( z ) .
Revised: q φ ( z | x ) (cid:107) p ( z ))] , hence trying to compress and squash the latent codes z for each true example x into those sampled from the prior distribution p ( z ) .

Original: This observation is the key ingredient for us to develop our proposed model.
Revised: This observation is the key ingredient for us to develop our proposed model.

Original: 2. T HE K ULLBACK -L EIBLER D IVERGENCE AND L2 W ASSERSTEIN D ISTANCE
Revised: 2. T HE K ULLBACK -L EIBLER D IVERGENCE AND L2 W ASSERSTEIN D ISTANCE

Original: Given two distributions with the probability density functions p ( z ) and q ( z ) where z ∈ R d , the
Revised: Given two distributions with the probability density functions p ( z ) and q ( z ) where z ∈ R d , the

Original: Kullback-Leibler (KL) divergence between these two distributions is deﬁned as:
Revised: Kullback-Leibler (KL) divergence between these two distributions is deﬁned as:

Original: [Equation]
Revised: [Equation]

Original: Another divergence of interest is L2 Wasserstein (WS) distance with the cost function c ( z 1 , z 2 )
Revised: Another divergence of interest is L2 Wasserstein (WS) distance with the cost function c ( z 1 , z 2 )

Original: = (cid:107) z
Revised: = (cid:107) z

Original: 1 − z 2 (cid:107) 22 .
Revised: 1 − z 2 (cid:107) 22 .

Original: The L2 Wasserstein divergence between two distributions is deﬁned as:
Revised: The L2 Wasserstein divergence between two distributions is deﬁned as:

Original: [Equation]
Revised: [Equation]

Original: where Π ( q, p ) speciﬁes the set of all joint distributions over p, q which admits p, q as marginals.
Revised: where Π ( q, p ) speciﬁes the set of all joint distributions over p, q which admits p, q as marginals.

Original: If p, q are two Gaussians, i.e., p ( z ) =
Revised: If p, q are two Gaussians, i.e., p ( z ) =

Original: N ( z | µ 1 , Σ 1 ) and q ( z ) =
Revised: N ( z | µ 1 , Σ 1 ) and q ( z ) =

Original: N ( z | µ 2 , Σ 2 ) then both KL divergence and L2 WS distance can be computed in close forms as:
Revised: N ( z | µ 2 , Σ 2 ) then both KL divergence and L2 WS distance can be computed in close forms as:

Original: [Equation]
Revised: [Equation]

Original: where (cid:107)·(cid:107) F is the Frobenius norm and Σ 1 Σ 2 = Σ 2 Σ 1 .
Revised: where (cid:107)·(cid:107) F is the Frobenius norm and Σ 1 Σ 2 = Σ 2 Σ 1 .

Original:  T HE M AXIMAL D IVERGENCE S EQUENTIAL A UTO - ENCODER (MDSAE) FOR B INARY V ULNERABILITY D ETECTION 3. D ATA P ROCESSING AND E MBEDDING
Revised:  T HE M AXIMAL D IVERGENCE S EQUENTIAL A UTO - ENCODER (MDSAE) FOR B INARY V ULNERABILITY D ETECTION 3. D ATA P ROCESSING AND E MBEDDING

Original: For each machine instruction, we employ the
Revised: For each machine instruction, we employ the

Original: Capstone 2 binary disassembly framework to detect entire machine instructions.
Revised: Capstone 2 binary disassembly framework to detect entire machine instructions.

Original: We then eliminate redundant preﬁxes to obtain the core parts that contain the opcode and other signiﬁcant information.
Revised: We then eliminate redundant preﬁxes to obtain the core parts that contain the opcode and other signiﬁcant information.

Original: Each core part in a machine instruction consists of two parts: the opcode and instruction information (i.e., memory location, registers, etc.).
Revised: Each core part in a machine instruction consists of two parts: the opcode and instruction information (i.e., memory location, registers, etc.).

Original: We embed both the opcode and instruction information into vectors and then concatenate them.
Revised: We embed both the opcode and instruction information into vectors and then concatenate them.

Original: To embed the opcode, we build a vocabulary of opcodes and then multiply the one-hot vector of the opcode with the corresponding embedding matrix.
Revised: To embed the opcode, we build a vocabulary of opcodes and then multiply the one-hot vector of the opcode with the corresponding embedding matrix.

Original: To embed the instruction information, we build the vocabulary over 256 hex-bytes from 00 to FF , then view the instruction information as a sequence of hex-bytes to construct the frequency vector of a size 256 , and ﬁnally multiply this frequency vector with the corresponding embedding matrix.
Revised: To embed the instruction information, we build the vocabulary over 256 hex-bytes from 00 to FF , then view the instruction information as a sequence of hex-bytes to construct the frequency vector of a size 256 , and ﬁnally multiply this frequency vector with the corresponding embedding matrix.

Original: More speciﬁcally, the output embedding is e = e op (cid:107) e ii where e op = one-hot ( op )
Revised: More speciﬁcally, the output embedding is e = e op (cid:107) e ii where e op = one-hot ( op )

Original: × W op and e ii = freq ( ii ) × W ii with the opcode op , the instruction information ii and its frequency vector freq ( ii ) , and the embedding matrices W op and W ii .
Revised: × W op and e ii = freq ( ii ) × W ii with the opcode op , the instruction information ii and its frequency vector freq ( ii ) , and the embedding matrices W op and W ii .

Original: The process of embedding machine instructions is presented in Figure 1.
Revised: The process of embedding machine instructions is presented in Figure 1.

Original: 3. P ROPOSED M ODEL
Revised: 3. P ROPOSED M ODEL

Original: In this work, we view binary code x as a sequence of machine instructions, i.e., x = [ x i ] i =1 ,...,m where each x i is a machine instruction.
Revised: In this work, we view binary code x as a sequence of machine instructions, i.e., x = [ x i ] i =1 ,...,m where each x i is a machine instruction.

Original: Our idea is to encode x to the latent code z in such a way that the latent codes of data in different classes are encouraged to be maximally divergent.
Revised: Our idea is to encode x to the latent code z in such a way that the latent codes of data in different classes are encouraged to be maximally divergent.

Original: Let us denote the distributions of vulnerable and non-vulnerable sequences by p 1 ( x ) and p 0 ( x ) respectively.
Revised: Let us denote the distributions of vulnerable and non-vulnerable sequences by p 1 ( x ) and p 0 ( x ) respectively.

Original: Inspired by the Variational Auto-Encoder (Kingma & Welling, 2013), we propose to use a probabilistic decoder p θ ( x | z ) such that for z ∼ p 0 ( z ) , x drawn from p θ ( x | z ) can mimic those drawn from p 0 ( x ) and for z ∼ p 1 ( z ) , x drawn from p θ ( x | z ) can mimic those drawn from p 1 ( x ) .
Revised: Inspired by the Variational Auto-Encoder (Kingma & Welling, 2013), we propose to use a probabilistic decoder p θ ( x | z ) such that for z ∼ p 0 ( z ) , x drawn from p θ ( x | z ) can mimic those drawn from p 0 ( x ) and for z ∼ p 1 ( z ) , x drawn from p θ ( x | z ) can mimic those drawn from p 1 ( x ) .

Original: In other words, we aim to learn the probabilistic decoder p θ ( x | z ) satisfying:
Revised: In other words, we aim to learn the probabilistic decoder p θ ( x | z ) satisfying:

Original: [Equation]
Revised: [Equation]

Original: and
Revised: and

Original: [Equation]
Revised: [Equation]

Original: For any approximate posterior q φ ( z | x ) , we have the following lower bounds:
Revised: For any approximate posterior q φ ( z | x ) , we have the following lower bounds:

Original: [Figure]
Revised: [Figure]

Original: [Equation]
Revised: [Equation]

Original: [Figure]
Revised: [Figure]

Original: [Equation]
Revised: [Equation]

Original: [Figure]
Revised: [Figure]

Original: [Equation]
Revised: [Equation]

Original: [Figure]
Revised: [Figure]

Original: [Equation]
Revised: [Equation]

Original: [Figure]
Revised: [Figure]

Original: [Equation]
Revised: [Equation]

Original: [Figure]
Revised: [Figure]

Original: [Equation]
Revised: [Equation]

Original: Note that we use the same network for q φ ( z | x , y = 0) and q φ ( z | x , y = 1) and they are discriminated by the source of data used to ﬁt.
Revised: Note that we use the same network for q φ ( z | x , y = 0) and q φ ( z | x , y = 1) and they are discriminated by the source of data used to ﬁt.

Original: Using the architecture shown in Figure 2, we consider the probabilistic decoder p θ ( x | z ) of the following parametric form
Revised: Using the architecture shown in Figure 2, we consider the probabilistic decoder p θ ( x | z ) of the following parametric form

Original: [Equation]
Revised: [Equation]

Original: and hence we can further derive the lower bounds as:
Revised: and hence we can further derive the lower bounds as:

Original: [Equation]
Revised: [Equation]

Original: We arrive at the following optimization problem:
Revised: We arrive at the following optimization problem:

Original: [Equation]
Revised: [Equation]

Original: It is worth noting that since we are minimizing:
Revised: It is worth noting that since we are minimizing:

Original: [Equation]
Revised: [Equation]

Original: the encoding z ∼ q φ ( z | h L ) with y = 0 are absorbed (compressed) into the prior p 

Revised: the encoding z ∼ q φ ( z | h L ) with y = 0 are absorbed (compressed) into the prior p

Original: ( z ) .
Revised: ( z ) .

Original: Similarly, the encoding z ∼ q φ ( z | h L ) with y = 1 are compressed into the prior p 1 ( z ) .
Revised: Similarly, the encoding z ∼ q φ ( z | h L ) with y = 1 are compressed into the prior p 1 ( z ) .

Original: Therefore, to maximize the difference between the encodings of data in the two classes, we propose to maximize the divergence between p 0 ( z ) and p 1 ( z ) and arrive the following optimization problem:
Revised: Therefore, to maximize the difference between the encodings of data in the two classes, we propose to maximize the divergence between p 0 ( z ) and p 1 ( z ) and arrive the following optimization problem:

Original: [Equation]
Revised: [Equation]

Original: φ ) (cid:3) + E x : y =1 (cid:2) L 1 ( x ; θ, φ ) (cid:3) + αD (cid:0) p 0 ( z ) (cid:107) p 1 ( z ) (cid:1)(cid:9) where α > 0 is a non-negative trade-off parameter and D (cid:0) p 0 ( z ) (cid:107) p 1 ( z ) (cid:1) is the divergence between the two priors.
Revised: φ ) (cid:3) + E x : y =1 (cid:2) L 1 ( x ; θ, φ ) (cid:3) + αD (cid:0) p 0 ( z ) (cid:107) p 1 ( z ) (cid:1)(cid:9) where α > 0 is a non-negative trade-off parameter and D (cid:0) p 0 ( z ) (cid:107) p 1 ( z ) (cid:1) is the divergence between the two priors.

Original: To facilitate the evaluation, we endow these two priors with Gaussian distributions as follows: p k ( z ) =
Revised: To facilitate the evaluation, we endow these two priors with Gaussian distributions as follows: p k ( z ) =

Original: N ( z | µ k , Σ k ) , k = 0 , 1 .
Revised: N ( z | µ k , Σ k ) , k = 0 , 1 .

Original: We also propose using the Gaussian approximate posterior as: q φ ( z | h L ) =
Revised: We also propose using the Gaussian approximate posterior as: q φ ( z | h L ) =

Original: N ( z | µ φ ( h L ) , diag ( σ φ ( h L ))) which enables the reparameterization trick:
Revised: N ( z | µ φ ( h L ) , diag ( σ φ ( h L ))) which enables the reparameterization trick:

Original: z = µ φ ( h L ) + diag ( σ φ ( h L )) 1 / 2 (cid:15) , where the source of randomness (cid:15) ∼ N ( 0 , I ) and µ φ ( z ) , σ φ ( z ) are two neural networks representing the mean and covariance matrix of the approximate Gaussian posterior.
Revised: z = µ φ ( h L ) + diag ( σ φ ( h L )) 1 / 2 (cid:15) , where the source of randomness (cid:15) ∼ N ( 0 , I ) and µ φ ( z ) , σ φ ( z ) are two neural networks representing the mean and covariance matrix of the approximate Gaussian posterior.

Original: Hence we come to the following optimization problem:
Revised: Hence we come to the following optimization problem:

Original: [Equation]
Revised: [Equation]

Original: where we note that D (cid:0) p 0 ( z ) (cid:107) p 1 ( z ) (cid:1) is tractable for both the KL-divergence and L
Revised: where we note that D (cid:0) p 0 ( z ) (cid:107) p 1 ( z ) (cid:1) is tractable for both the KL-divergence and L

Original: Wasserstein distance (See Section 2.2) and L 0 ( x ; θ, φ ) , L 1 ( x ; θ, φ ) can be rewritten using the reparameterization trick as:
Revised: Wasserstein distance (See Section 2.2) and L 0 ( x ; θ, φ ) , L 1 ( x ; θ, φ ) can be rewritten using the reparameterization trick as:

Original: [Equation]
Revised: [Equation]

Original: with z = µ φ ( h L ) + diag ( σ φ ( h L )) 1 / 2 (cid:15) .
Revised: with z = µ φ ( h L ) + diag ( σ φ ( h L )) 1 / 2 (cid:15) .

Original: To classify data, we can train a classiﬁer C over the latent space either independently or simultaneously with the maximal divergence auto-encoder.
Revised: To classify data, we can train a classiﬁer C over the latent space either independently or simultaneously with the maximal divergence auto-encoder.

Original: If we train the classiﬁer simultaneously, the ﬁnal optimization problem is as follows:
Revised: If we train the classiﬁer simultaneously, the ﬁnal optimization problem is as follows:

Original: [Equation]
Revised: [Equation]

Original: where f
Revised: where f

Original: [Equation]
Revised: [Equation]

Original: , φ
Revised: , φ

Original: [Equation]
Revised: [Equation]

Original: αD (cid:0) p 0 ( z ) (cid:107) p 1 ( z ) (cid:1) + β ( E x : y =

Revised: αD (cid:0) p 0 ( z ) (cid:107) p 1 ( z ) (cid:1) + β ( E x : y =

Original: [log (1 − C ψ ( x ))]
Revised: [log (1 − C ψ ( x ))]

Original: + E x : y =1 [log C ψ ( x )]) where C ψ ( x ) stands for the probability to classify x as a vulnerable binary code ( y = 1 ), and α, β > 0 are two non-negative trade-off parameters.
Revised: + E x : y =1 [log C ψ ( x )]) where C ψ ( x ) stands for the probability to classify x as a vulnerable binary code ( y = 1 ), and α, β > 0 are two non-negative trade-off parameters.

Original: It is worth noting that to model the conditional distributions p θ ( x i
Revised: It is worth noting that to model the conditional distributions p θ ( x i

Original: | h
Revised: | h

Original: i − 1 , z ) , we only take into account the opcode of the machine instruction x i .
Revised: i − 1 , z ) , we only take into account the opcode of the machine instruction x i .

Original: Since this opcode lies in a ﬁxed vocabulary of the opcodes, we can use the softmax distribution to deﬁne the corresponding distribution p θ ( x i
Revised: Since this opcode lies in a ﬁxed vocabulary of the opcodes, we can use the softmax distribution to deﬁne the corresponding distribution p θ ( x i

Original: | h
Revised: | h

Original: i − 1 , z ) .
Revised: i − 1 , z ) .

Original: By this means, the reconstruction phase aims to reconstruct the opcodes of the machine instructions in a given binary rather than the whole machine instructions.
Revised: By this means, the reconstruction phase aims to reconstruct the opcodes of the machine instructions in a given binary rather than the whole machine instructions.

Original:  E XPERIMENTS 4. E XPERIMENTAL D ATASETS
Revised:  E XPERIMENTS 4. E XPERIMENTAL D ATASETS

Original: One of the most signiﬁcant contributions of our work is to create a labeled binary dataset for binary code vulnerability detection.
Revised: One of the most signiﬁcant contributions of our work is to create a labeled binary dataset for binary code vulnerability detection.

Original: We ﬁrst extracted the functions from the NDSS18 source code dataset.
Revised: We ﬁrst extracted the functions from the NDSS18 source code dataset.

Original: We then preprocessed and ﬁltered out any identical functions to obtain 13 , 000 functions, of which 9 , 000 could be ﬁxed to compile to binaries using our automatic tool.
Revised: We then preprocessed and ﬁltered out any identical functions to obtain 13 , 000 functions, of which 9 , 000 could be ﬁxed to compile to binaries using our automatic tool.

Original: In addition, we developed a tool based on Joern 3 to parse the semantic and syntactical relationships in a given piece of source code.
Revised: In addition, we developed a tool based on Joern 3 to parse the semantic and syntactical relationships in a given piece of source code.

Original: In particular, our tool ﬁrst used the compiler gcc/g++ (MinGW) to compile a given piece of source code, then captured the error messages, parsed these error messages, relied on Joern to be aware of the semantic and syntactical relationships of the error messages with respect to the source code, and ﬁnally ﬁxed the corresponding error message.
Revised: In particular, our tool ﬁrst used the compiler gcc/g++ (MinGW) to compile a given piece of source code, then captured the error messages, parsed these error messages, relied on Joern to be aware of the semantic and syntactical relationships of the error messages with respect to the source code, and ﬁnally ﬁxed the corresponding error message.

Original: This process was repeated until the given source is error-free and ready to compile to a binary.
Revised: This process was repeated until the given source is error-free and ready to compile to a binary.

Original: By compiling the compilable function source code under various platforms and architectures, we obtained 32 , 281 binary functions including 17 ,binaries for Windows and 14 , 304 binaries for Linux.
Revised: By compiling the compilable function source code under various platforms and architectures, we obtained 32 , 281 binary functions including 17 ,binaries for Windows and 14 , 304 binaries for Linux.

Original: The statistics of our binary dataset is given in
Revised: The statistics of our binary dataset is given in

Original: Table 1.
Revised: Table 1.

Original: Additionally, in order to obtain this binary dataset our tool ﬁxed tens of thousands of errors of which many are strongly associated with speciﬁc source code.
Revised: Additionally, in order to obtain this binary dataset our tool ﬁxed tens of thousands of errors of which many are strongly associated with speciﬁc source code.

Original: 4. B ASELINES
Revised: 4. B ASELINES

Original: [Table]
Revised: [Table]

Original: We compared our proposed methods MDSAE-R (for learning maximally divergent representations in conjunction with an independent linear classiﬁer to classify vulnerable and non-vulnerable functions) and MDSAE-C (for learning maximally divergent representations incorporating a linear classiﬁer) with the following baselines:
Revised: We compared our proposed methods MDSAE-R (for learning maximally divergent representations in conjunction with an independent linear classiﬁer to classify vulnerable and non-vulnerable functions) and MDSAE-C (for learning maximally divergent representations incorporating a linear classiﬁer) with the following baselines:

Original: • RNN-R : A Recurrent Neural Network (RNN) for learning representations and linear classiﬁer independently trained on the resulting representations for classifying vulnerable and non-vulnerable functions.
Revised: • RNN-R : A Recurrent Neural Network (RNN) for learning representations and linear classiﬁer independently trained on the resulting representations for classifying vulnerable and non-vulnerable functions.

Original: In addition, to learn representations in an unsupervised manner, we applied the method of language modeling whereby we trained the model to predict the opcode of the next machine instruction given the previous machine instructions.
Revised: In addition, to learn representations in an unsupervised manner, we applied the method of language modeling whereby we trained the model to predict the opcode of the next machine instruction given the previous machine instructions.

Original: • RNN-C :
Revised: • RNN-C :

Original: A RNN with a linear classiﬁer built on the top of the last hidden unit.
Revised: A RNN with a linear classiﬁer built on the top of the last hidden unit.

Original: • Para2Vec : The paragraph-to-vector distributional similarity model proposed in (Le & Mikolov, 2014).
Revised: • Para2Vec : The paragraph-to-vector distributional similarity model proposed in (Le & Mikolov, 2014).

Original: This work proposed to embed paragraphs including many words in a ﬁxed vocabulary into a vector space.
Revised: This work proposed to embed paragraphs including many words in a ﬁxed vocabulary into a vector space.

Original: To apply this work in our context, we view a binary as a sequence of opcodes residing in the ﬁxed vocabulary of the opcodes.
Revised: To apply this work in our context, we view a binary as a sequence of opcodes residing in the ﬁxed vocabulary of the opcodes.

Original: • SeqVAE-C : Sequential VAE as in Section 3.2, but we set two priors to N ( 0 , I ) and kept ﬁxed during training as in the original VAE.
Revised: • SeqVAE-C : Sequential VAE as in Section 3.2, but we set two priors to N ( 0 , I ) and kept ﬁxed during training as in the original VAE.

Original: A linear classiﬁer was built up on the top of the latent codes and trained simultaneously.
Revised: A linear classiﬁer was built up on the top of the latent codes and trained simultaneously.

Original: With this setting, we aim to show that learning the priors produces more separable representations, hence boosts the performance.
Revised: With this setting, we aim to show that learning the priors produces more separable representations, hence boosts the performance.

Original: • VulDeePecker : proposed in (Li et al., 2018) for source code vulnerability detection.
Revised: • VulDeePecker : proposed in (Li et al., 2018) for source code vulnerability detection.

Original: This model employed a Bidirectional RNN (BRNN) to take sequential inputs and then concatenated hidden units to input to a feedforward neural net classiﬁer.
Revised: This model employed a Bidirectional RNN (BRNN) to take sequential inputs and then concatenated hidden units to input to a feedforward neural net classiﬁer.

Original: This method can inherently be applied to binaries wherein sequences of machine instructions are inputted to the BRNN
Revised: This method can inherently be applied to binaries wherein sequences of machine instructions are inputted to the BRNN

Original: In addition, we also inspected two variants of divergence (i.e., KL divergence and L2 WS distance (See Section 2.2)) for formulating the divergence D (cid:0) p 0 ( z ) (cid:107) p 1 ( z ) (cid:1) in the optimization problem in Eq.
Revised: In addition, we also inspected two variants of divergence (i.e., KL divergence and L2 WS distance (See Section 2.2)) for formulating the divergence D (cid:0) p 0 ( z ) (cid:107) p 1 ( z ) (cid:1) in the optimization problem in Eq.

Original: Consequently, we have four variants of our proposed method, namely MDSAE-RKL,
Revised: Consequently, we have four variants of our proposed method, namely MDSAE-RKL,

Original: MDSAE-RWS, MDSAE-CKL, and MDSAE-CWS.
Revised: MDSAE-RWS, MDSAE-CKL, and MDSAE-CWS.

Original: 4. P ARAMETER S ETTING
Revised: 4. P ARAMETER S ETTING

Original: We split the data into 80% for training, 10% for validation, and the remaining 10% for testing.
Revised: We split the data into 80% for training, 10% for validation, and the remaining 10% for testing.

Original: We employed a dynamic RNN to tackle the variation in the number of machine instructions of the functions.
Revised: We employed a dynamic RNN to tackle the variation in the number of machine instructions of the functions.

Original: For the RNN baselines and our models, the size of hidden unit was set to 256.
Revised: For the RNN baselines and our models, the size of hidden unit was set to 256.

Original: For our model, the size of the latent space was set to 4,096, the trade-off parameters α , β were set to 2 × 10 −and 10 − 4 respectively.
Revised: For our model, the size of the latent space was set to 4,096, the trade-off parameters α , β were set to 2 × 10 −and 10 − 4 respectively.

Original: We used the Adam optimizer (Kingma & Ba, 2014) with an initial learning rate equal to 0.0001.
Revised: We used the Adam optimizer (Kingma & Ba, 2014) with an initial learning rate equal to 0.0001.

Original: The minibatch size was set to 64 and the number of epochs was set to 100.
Revised: The minibatch size was set to 64 and the number of epochs was set to 100.

Original: We implemented our proposed method in Python using Tensorﬂow (Abadi et al., 2016), an open-source software library for Machine Intelligence developed by the Google Brain Team.
Revised: We implemented our proposed method in Python using Tensorﬂow (Abadi et al., 2016), an open-source software library for Machine Intelligence developed by the Google Brain Team.

Original: The source code as well as the dataset are available in our github repository 4 .
Revised: The source code, as well as the dataset, is available in our GitHub repository 4 .

Original: We ran our experiments on a computer with an Intel Xeon Processor E5-1660 which had 8 cores at 3.0 GHz and 128 GB of RAM.
Revised: We ran our experiments on a computer with an Intel Xeon Processor E5-1660 which had 8 cores at 3.0 GHz and 128 GB of RAM.

Original: 4. E XPERIMENTAL R ESULTS 4.4. E XPERIMENTAL R ESULTS ON THE NDSS18 B INARY D ATASET
Revised: 4. E XPERIMENTAL R ESULTS 4.4. E XPERIMENTAL R ESULTS ON THE NDSS18 B INARY D ATASET

Original: We conducted the experiments on the subset of Windows binaries, the subset of Linux binaries, and the whole set of binaries to compare our methods with the baselines.
Revised: We conducted the experiments on the subset of Windows binaries, the subset of Linux binaries, and the whole set of binaries to compare our methods with the baselines.

Original: The experimental results are shown in Table 2.
Revised: The experimental results are shown in Table 2.

Original: It can be seen that our proposed method outperforms the baselines in all performance measures of interest.
Revised: It can be seen that our proposed method outperforms the baselines in all performance measures of interest.

Original: Speciﬁcally, in the ﬁeld of computer security, the recall is a very important measure of completeness since a higher recall value leads to fewer vulnerable functions being incorrectly classiﬁed as non-vulnerable, which can otherwise present an issue for code auditors when there can be a large imbalance in the number of non-vulnerable and vulnerable functions in real-world use.
Revised: Speciﬁcally, in the ﬁeld of computer security, the recall is a very important measure of completeness since a higher recall value leads to fewer vulnerable functions being incorrectly classiﬁed as non-vulnerable, which can otherwise present an issue for code auditors when there can be a large imbalance in the number of non-vulnerable and vulnerable functions in real-world use.

Original: In addition, the fact that the resulting data representations of MDSAE-RKL and
Revised: In addition, the fact that the resulting data representations of MDSAE-RKL and

Original: MDSAE-RWS work well with a linear classiﬁer conﬁrms our intuition and motivation of that the encouragement of data separation effectively supports the classiﬁers
Revised: MDSAE-RWS work well with a linear classiﬁer conﬁrms our intuition and motivation of that the encouragement of data separation effectively supports the classiﬁers

Original: [Table]
Revised: [Table]

Original: 78.4 MD
Revised: 78.4 MD

Original: [Table]
Revised: [Table]

Original: CKL 83.
Revised: CKL 83.

Original: 82.1 MD
Revised: 82.1 MD

Original: [Table]
Revised: [Table]

Original: CWS 84.
Revised: CWS 84.

Original: 4.4. I NSPECTIONS OF M ODEL B EHAVIORS
Revised: 4.4. I NSPECTIONS OF M ODEL B EHAVIORS

Original: Distances between Two Priors, Distributions of Vulnerable, Non-vulnerable Classes During
Revised: Distances between Two Priors, Distributions of Vulnerable, Non-vulnerable Classes During

Original: Training In this experiment, we study i) the L2 WS distance between the two priors, ii) the Euclidean distance of two means of priors (i.e., (cid:107) µ 0 − µ 1 (cid:107) ), iii) the KL divergence of q φ ( z | h L , y = 0) and p 0 ( z ) (i.e., D KL (cid:0) q φ ( z | h L , y = 0) (cid:107) p 0 ( z ) (cid:1) ), iv) the KL divergence of q φ ( z | h L , y = 1) and p 1 ( z ) (i.e., D KL (cid:0) q φ ( z | h L , y = 1) (cid:107) p 1 ( z ) (cid:1) ), v) the Maximum Mean
Revised: Training In this experiment, we study i) the L2 WS distance between the two priors, ii) the Euclidean distance of two means of priors (i.e., (cid:107) µ 0 − µ 1 (cid:107) ), iii) the KL divergence of q φ ( z | h L , y = 0) and p 0 ( z ) (i.e., D KL (cid:0) q φ ( z | h L , y = 0) (cid:107) p 0 ( z ) (cid:1) ), iv) the KL divergence of q φ ( z | h L , y = 1) and p 1 ( z ) (i.e., D KL (cid:0) q φ ( z | h L , y = 1) (cid:107) p 1 ( z ) (cid:1) ), v) the Maximum Mean

Original: Discrepancy (MMD) distance (Gretton et al., 2012) of q φ ( z | h L , y = 0) and q φ ( z | h L , y = 1) , and vi) the reconstruction loss across epochs of MDSAE-RWS– the variant of our proposed method for learning separable representations.
Revised: Discrepancy (MMD) distance (Gretton et al., 2012) of q φ ( z | h L , y = 0) and q φ ( z | h L , y = 1) , and vi) the reconstruction loss across epochs of MDSAE-RWS– the variant of our proposed method for learning separable representations.

Original: As shown in Figure 3, during the training process, two distributions p ( z | y = 0) and p ( z | y = 1) become consistently and gradually more distant with the increase in their MMD distance (Figure 3, second row, middle), hence implying the gradually increasing separation of the corresponding latent codes.
Revised: As shown in Figure 3, during the training process, two distributions p ( z | y = 0) and p ( z | y = 1) become consistently and gradually more distant with the increase in their MMD distance (Figure 3, second row, middle), hence implying the gradually increasing separation of the corresponding latent codes.

Original: In addition, as we expect, the two priors become consistently and gradually more distant (Figure 3, ﬁrst row, left-hand side and Figure 3, ﬁrst row, middle) and the latent codes of vulnerable ( y = 1 ) and non-vulnerable ( y = 0 ) classes become more compressed into its priors respectively (Figure 3, ﬁrst row, right-hand side and Figure 3, second row, left-hand side).
Revised: In addition, as we expect, the two priors become consistently and gradually more distant (Figure 3, ﬁrst row, left-hand side and Figure 3, ﬁrst row, middle) and the latent codes of vulnerable ( y = 1 ) and non-vulnerable ( y = 0 ) classes become more compressed into its priors respectively (Figure 3, ﬁrst row, right-hand side and Figure 3, second row, left-hand side).

Original: Furthermore, the reconstruction error consistently decreases which implies that the latent codes maintain crucial information of the original binaries (Figure 3, second row, right-hand side).
Revised: Furthermore, the reconstruction error consistently decreases which implies that the latent codes maintain crucial information of the original binaries (Figure 3, second row, right-hand side).

Original: middle) reconstruction
Revised: middle) reconstruction

Original: Visualization of Latent Codes of Two Classes in The Latent Space
Revised: Visualization of Latent Codes of Two Classes in The Latent Space

Original: In this experiment, we set the dimension of the latent space to 2 to visualize the latent codes of the two classes before and after training.
Revised: In this experiment, we set the dimension of the latent space to 2 to visualize the latent codes of the two classes before and after training.

Original: As shown in Figure 4, the latent codes of the two classes are intermingled before training, whereas, they become more separable and distinct after training.
Revised: As shown in Figure 4, the latent codes of the two classes are intermingled before training, whereas, they become more separable and distinct after training.

Original: This shows that our proposed methods discover data representations that support the classiﬁcation task.
Revised: This shows that our proposed methods discover data representations that support the classiﬁcation task.

Original:  C ONCLUSION
Revised:  C ONCLUSION

Original: The detection of vulnerabilities in binary code is an important problem in the software industry and in the ﬁeld of computer security.
Revised: The detection of vulnerabilities in binary code is an important problem in the software industry and in the ﬁeld of computer security.

Original: In this paper, we leverage recent advances in deep learning representation to propose the Maximal Divergence Sequential Auto-Encoder for binary vulnerability detection.
Revised: In this paper, we leverage recent advances in deep learning representation to propose the Maximal Divergence Sequential Auto-Encoder for binary vulnerability detection.

Original: Speciﬁcally, latent codes representing vulnerable and non-vulnerable binaries are encouraged to be maximally different while still being able to maintain crucial information from the original binaries.
Revised: Speciﬁcally, latent codes representing vulnerable and non-vulnerable binaries are encouraged to be maximally different while still being able to maintain crucial information from the original binaries.

Original: To address the issue of limited labelled public binary datasets for this problem and to facilitate research in the application of machine learning and deep learning to the domain of binary vulnerability detection, we have created a labelled binary software dataset.
Revised: To address the issue of limited labelled public binary datasets for this problem and to facilitate research in the application of machine learning and deep learning to the domain of binary vulnerability detection, we have created a labelled binary software dataset.

Original: Furthermore, our developed tool and approach can be reused to create other high-quality binary datasets.
Revised: Furthermore, our developed tool and approach can be reused to create other high-quality binary datasets.

Original: We conducted extensive experiments to compare our proposed methods with the baselines.
Revised: We conducted extensive experiments to compare our proposed methods with the baselines.

Original: The experimental results show that our proposed methods outperform the baselines in all performance measures of interest.
Revised: The experimental results show that our proposed methods outperform the baselines in all performance measures of interest.

Original: A A PPENDIX : T HE P ROCESS TO O BTAIN B INARIES FROM S OURCE C ODE
Revised: A A PPENDIX : T HE P ROCESS TO O BTAIN B INARIES FROM S OURCE C ODE

Original: The process of compiling the VulDeePecker dataset into binaries is divided into three main stages: collecting function source code, detecting and ﬁxing source code, and compiling source code to binary functions.
Revised: The process of compiling the VulDeePecker dataset into binaries is divided into three main stages: collecting function source code, detecting and ﬁxing source code, and compiling source code to binary functions.

Original: A. C OLLECTING F UNCTION SOURCE CODE
Revised: A. C OLLECTING F UNCTION SOURCE CODE

Original: The source code are collected from VulDeePecker GitHub 5 .
Revised: The source code is collected from VulDeePecker GitHub 5 .

Original: These source code involve two types of vulnerability in C/C++ programs: buffer error vulnerability CWE-119 (11,427 ﬁles) and resource management error vulnerability CWE-399 (2,088 ﬁles).
Revised: These source code involve two types of vulnerability in C/C++ programs: buffer error vulnerability CWE-119 (11,427 ﬁles) and resource management error vulnerability CWE-399 (2,088 ﬁles).

Original: Figure 8 provides an example of a source code ﬁle together with its highlighted buffer error vulnerability.
Revised: Figure 8 provides an example of a source code ﬁle together with its highlighted buffer error vulnerability.

Original: We then use Joern’s parser 6 to identify the start and end points of each function in order to recognize the function scope.
Revised: We then use Joern’s parser 6 to identify the start and end points of each function in order to recognize the function scope.

Original: After this step, 19,009 non-vulnerable and 12,946 vulnerable functions are detected and obtained.
Revised: After this step, 19,009 non-vulnerable and 12,946 vulnerable functions are detected and obtained.

Original: However, there are a considerable number of functions which are identical to each other.
Revised: However, there are a considerable number of functions which are identical to each other.

Original: They are either some common functions that are widely used or some unchanged functions in different versions of a particular source code ﬁle.
Revised: They are either some common functions that are widely used or some unchanged functions in different versions of a particular source code ﬁle.

Original: To address this issue, these identical functions are removed.
Revised: To address this issue, these identical functions are removed.

Original: Eventually, the numbers of distinctive non-vulnerable and vulnerable functions are 6,412 and 6,592 functions respectively (See Figure 5).
Revised: Eventually, the numbers of distinctive non-vulnerable and vulnerable functions are 6,412 and 6,592 functions respectively (See Figure 5).

Original: A. D ETECTING E RRORS AND F IXING SOURCE CODE
Revised: A. D ETECTING E RRORS AND F IXING SOURCE CODE

Original: At the second stage, as these functions are incomplete C/C++ code snippets and cannot be compiled successfully, the errors in source code are required being detected and ﬁxed to generate binaries.
Revised: At the second stage, as these functions are incomplete C/C++ code snippets and cannot be compiled successfully, the errors in source code are required being detected and ﬁxed to generate binaries.

Original: Therefore, we develop an automatic tool based on Joern to detect and ﬁx these functions.
Revised: Therefore, we develop an automatic tool based on Joern to detect and ﬁx these functions.

Original: The activity diagram of our tool is described in Figure 6.
Revised: The activity diagram of our tool is described in Figure 6.

Original: In the targeted directory, our automatic tool reads every ﬁle (each contains the source code of a function) sequentially.
Revised: In the targeted directory, our automatic tool reads every ﬁle (each contains the source code of a function) sequentially.

Original: The process of detecting and ﬁxing each function commences with the preprocessing of source code, which adds some necessary C/C++ libraries and the main function.
Revised: The process of detecting and ﬁxing each function commences with the preprocessing of source code, which adds some necessary C/C++ libraries and the main function.

Original: It is worth noting that at this step, our tool is able to reformat the source code using the clang-format 7 .
Revised: It is worth noting that at this step, our tool is able to reformat the source code using the clang-format 7 .

Original: Subsequently, the tool invokes the gcc/g++ (MinGW) compiler to compile the C/C++ source code respectively.
Revised: Subsequently, the tool invokes the gcc/g++ (MinGW) compiler to compile the C/C++ source code respectively.

Original: The compiler then captures the error messages and calls the corresponding solver for each speciﬁc error.
Revised: The compiler then captures the error messages and calls the corresponding solver for each speciﬁc error.

Original: The semantic and syntactic relationships of the error messages with respect to the source code are mainly analyzed by Joern’s parser.
Revised: The semantic and syntactic relationships of the error messages with respect to the source code are mainly analyzed by Joern’s parser.

Original: A function source code is ﬁxed successfully and ready to compile when the compiler cannot issue any error messages in the process of detecting it.
Revised: A function source code is ﬁxed successfully and ready to compile when the compiler cannot issue any error messages in the process of detecting it.

Original: The process of detecting errors and ﬁxing source code also has its own challenges.
Revised: The process of detecting errors and ﬁxing source code also has its own challenges.

Original: As mentioned before, we collect the code snippets of the functions which are always incomplete.
Revised: As mentioned before, we collect the code snippets of the functions which are always incomplete.

Original: This leads to the missing of the declarations for some objects due to the lack of certain libraries to which those objects belong.
Revised: This leads to the missing of the declarations for some objects due to the lack of certain libraries to which those objects belong.

Original: Figure 7 refers to a typical example of an uncompilable function.
Revised: Figure 7 refers to a typical example of an uncompilable function.

Original: For that code, when our automatic tool starts detecting the error, the gcc/g++ (MinGW) compiler informs the following error message which needs to be ﬁxed at line 23: ‘CWE761 Free Pointer Not at Start of Buffer wchar t environment 34 unionType’ has no member named ‘unionFirst’ .
Revised: For that code, when our automatic tool starts detecting the error, the gcc/g++ (MinGW) compiler informs the following error message which needs to be ﬁxed at line 23: ‘CWE761 Free Pointer Not at Start of Buffer wchar t environment 34 unionType’ has no member named ‘unionFirst’ .

Original: The error information is then sent to Joern’s parser in order to analyze and ﬁnd the appropriate solution to ﬁx this error.
Revised: The error information is then sent to Joern’s parser in order to analyze and ﬁnd the appropriate solution to ﬁx this error.

Original: Unfortunately, the parser is not conﬁdent enough to declare the ‘unionFirst’ variable as a member of the ‘CWE761 Free Pointer Not at Start of Buffer wchar t environment 34 unionType’ class.
Revised: Unfortunately, the parser is not conﬁdent enough to declare the ‘unionFirst’ variable as a member of the ‘CWE761 Free Pointer Not at Start of Buffer wchar t environment 34 unionType’ class.

Original: The reason arising from the fact that the ‘myUnion.unionFirst’ was assigned to the ‘data’ variable, but the parser cannot give any information about the data type of this variable.
Revised: The reason arising from the fact that the ‘myUnion.unionFirst’ was assigned to the ‘data’ variable, but the parser cannot give any information about the data type of this variable.

Original: In this situation, the error description is logged into a log ﬁle, the execution of the function including this error is skipped, and our tool proceeds to the next function.
Revised: In this situation, the error description is logged into a log ﬁle, the execution of the function including this error is skipped, and our tool proceeds to the next function.

Original: After detecting and ﬁxing errors process, we synthesize and do some statistics from the log ﬁle to know which errors account for the most popular quantities to upgrade promptly our tool.
Revised: After detecting and ﬁxing errors process, we synthesize and do some statistics from the log ﬁle to know which errors account for the most popular quantities to upgrade promptly our tool.

Original: We also consider the complexity of functions and the priority order of errors to ensure errors ﬁxed from easiest to hardest.
Revised: We also consider the complexity of functions and the priority order of errors to ensure errors ﬁxed from easiest to hardest.

Original: After each upgrade, our automatic tool is very likely to detect and ﬁx more complex errors to become more completed and stable.
Revised: After each upgrade, our automatic tool is very likely to detect and ﬁx more complex errors to become more completed and stable.

Original: The result we obtain from this stage is 8,991 ﬁxed source code.
Revised: The result we obtain from this stage is 8,991 ﬁxed source code.

Original: It is noticeable that while an original function source code has 40 errors on average, our automatic tool is able to detect and ﬁx up toand 28 general errors for each C and C++ source code respectively.
Revised: It is noticeable that while an original function source code has 40 errors on average, our automatic tool is able to detect and ﬁx up toand 28 general errors for each C and C++ source code respectively.

Original: A. C OMPILING SOURCE CODE TO B INARY C ODE
Revised: A. C OMPILING SOURCE CODE TO B INARY C ODE

Original: At the last stage, we compile 8,991 ﬁxed functions which contain 4,501 non-vulnerable and 4, vulnerable functions into binaries under two platforms (Windows and Ubuntu) and architectures (x86/x64).
Revised: At the last stage, we compile 8,991 ﬁxed functions which contain 4,501 non-vulnerable and 4, vulnerable functions into binaries under two platforms (Windows and Ubuntu) and architectures (x86/x64).

Original: The process of compilation raises a certain number of small errors due to the behavior inconsistency of gcc/g++ between two platforms which leads to the fact that some ﬁxed functions cannot be compiled.
Revised: The process of compilation raises a certain number of small errors due to the behavior inconsistency of gcc/g++ between two platforms which leads to the fact that some ﬁxed functions cannot be compiled.

Original: The total number of binary functions are 32,281 wherein 15,954 functions are non-vulnerable binaries and 16,327 functions are vulnerable binaries.
Revised: The total number of binary functions are 32,281 wherein 15,954 functions are non-vulnerable binaries and 16,327 functions are vulnerable binaries.

Original: As the consequence, we utilize the Capstone software to disassemble these binaries into assembly code.
Revised: As the consequence, we utilize the Capstone software to disassemble these binaries into assembly code.

Original: Figure 9 shows the assembly code together with its highlighted buffer error vulnerability of the source code ﬁle in Figure 8.
Revised: Figure 9 shows the assembly code together with its highlighted buffer error vulnerability of the source code ﬁle in Figure 8.

Original: Overall, Figure 5 shows the details of the stages in VulDeePecker dataset processing, together with the number of vulnerable and non-vulnerable functions obtained at the end of each stage.
Revised: Overall, Figure 5 shows the details of the stages in VulDeePecker dataset processing, together with the number of vulnerable and non-vulnerable functions obtained at the end of each stage.