Maximal Divergence Sequential Autoencoder for Binary Software Vulnerability Detection

1 INTRODUCTION . Software vulnerabilities are specific flaws or oversights in a piece of software that allow attackers to perform malicious acts including exposing or altering sensitive information , disrupting or destroying a system , or taking control of a computer system or program ( Dowd et al. , 2006 ) . Due to the ubiquity of computer software , the growth and the diversity in its development process , a great amount of computer software potentially includes software vulnerabilities . This fact makes the problem of software security vulnerability identification an important concern in the software industry and in the field of computer security . Although a great effort has been made by the security community , the severity of the threat from software vulnerabilities has gradually increased over the years . Numerous ∗Acknowledgement : This research was partially supported under the Defence Science and Technology Group ‘ s Next Generation Technologies Program . Dinh Phung and Trung Le further acknowledge the partial support from the Australian Research Council DP160109394 . †This work was done during the time when the first and second authors carried out internship at Monash University , Australia . exist of examples and incidents in the past two decades in which software vulnerabilities have imposed significant damages to companies and individuals ( Ghaffarian & Shahriari , 2017 ) . For example , vulnerabilities in popular browser plugins have threatened the security and privacy of millions of Internet users ( e.g. , Adobe Flash Player ( US-CERT 2015 ; Adobe Security Bulletin 2015 ) and Oracle Java ( US-CERT 2013 ) ) , vulnerabilities in popular and fundamental open-source software have also threatened the security of thousands of companies and their customers around the globe ( e.g. , Heartbleed ( Codenomicon 2014 ) and ShellShock ( Symantec Security Response 2014 ) . Software vulnerability detection ( SVD ) can be categorized into source code and binary code vulnerability detection . Source code vulnerability detection has been widely studied in a variety of works ( Shin et al. , 2011 ; Neuhaus et al. , 2007 ; Yamaguchi et al. , 2011 ; Li et al. , 2016 ; Kim et al. , 2017 ; Li et al. , 2018 ) . Most of the previous work in source code vulnerability detection ( Neuhaus et al. , 2007 ; Shin et al. , 2011 ; Yamaguchi et al. , 2011 ; Li et al. , 2016 ; Kim et al. , 2017 ) has been based on handcrafted features which are manually chosen by a limited number of domain experts . To mitigate the dependency on handcrafted features , the use of automatic features in SVD has been studied recently in ( Dam et al. , 2017 ; Li et al. , 2018 ; Lin et al. , 2018 ) . In particular , Dam et al . ( 2017 ) ; Lin et al . ( 2018 ) employed a Recurrent Neural Network ( RNN ) to transform sequences of code tokens to vectorial features , which are further fed to a separate classifier , while Li et al . ( 2018 ) combined learning the vector representation and the training of the classifier in a deep network . Compared with source code vulnerability detection , binary code vulnerability detection is significantly more difficult because much of the syntactic and semantic information provided by high-level programming languages is lost during the compilation process . The existence of such syntactic and semantic information makes it easier to reason how data and inputs drive the paths of execution . Unfortunately , a software binary , such as proprietary binary code ( with no access to source code ) or embedded systems code , is generally all that is made available for code analysis ( together perhaps with the processor architecture such as x86 etc. ) . The ability to detect the presence or absence of vulnerabilities in binary code , without getting access to source code , is therefore a major importance in the context of computer security . Some work has been proposed to detect vulnerabilities at the binary code level when source code is not available , notably work based on fuzzing , symbolic execution ( Cadar & Sen , 2013 ; Avancini & Ceccato , 2013 ; Meng et al. , 2016 ) , or techniques using handcrafted features extracted from dynamic analysis ( Grieco et al. , 2016 ; Cozzie et al. , 2008 ; White & Lüttgen , 2013 ) . To the best of our knowledge , there has been no work studying the use of automatically extracted features for binary code vulnerability detection , though there has been some work using automatic features in conjunction with deep learning methods for malware detection , notably ( Saxe & Berlin , 2015 ; Raff et al. , 2017 ) . It is worth noting that binary code vulnerability detection and malware detection are two different tasks . In particular , binary code vulnerability detection aims to detect specific flaws or oversights in binary code , while malware detection aims to detect if a given binary is malicious or not . The former is arguably harder in the sense that vulnerable and non-vulnerable binaries might be only slightly different , while there might be a clearer difference in general between malware and benign binaries . In addition , a significant constraint in research on binary code vulnerability detection is the lack of suitable binaries labeled as either vulnerable or non-vulnerable . Although we have some source code datasets for software vulnerability detection , to the best of our knowledge , there exists no large public binary dataset for the purpose of binary code vulnerability detection . The reason is that most source code in source code vulnerability detection datasets is not compilable due to incompleteness , and they have important pieces missing ( e.g. , variables , data types ) and relevant libraries – making the code compilable take a large effort in fixing a vast volume of source code . This arises from the nature of the process that involves collecting and labeling source code wherein we start from security reports in CVE1 and navigate through relevant websites to obtain code snippets of vulnerable and non-vulnerable source code . In this work , we leverage recent advances in deep learning to derive the automatic features of binary code for vulnerability detection . In particular , we view a given binary as a sequence of machine instructions and then use the theory of Variational Auto-Encoders ( VAE ) ( Kingma & Welling , 2013 ) to develop the Maximal Divergence Sequential Auto-Encoder ( MDSAE ) that can work out representations of binary code in such a way that representations of vulnerable and nonvulnerable binaries are encouraged to be maximally different for vulnerability detection purposes , 1https : //cve.mitre.org/ while still preserving crucial information inherent in the original binaries . In contrast to the original VAE wherein the data prior is kept fixed , we propose using two learnable Gaussian priors , one for each class . Based on the VAE principle , latent codes ( i.e. , data representations ) are absorbed ( or compressed ) into the data prior distribution , we further propose maximizing a divergence ( e.g. , Wasserstein ( WS ) distance or Kullback-Leibler ( KL ) divergence ) between these two priors to separate representations of vulnerable and non-vulnerable binaries . Our MDSAE can be used to produce data representations for another independent classifier ( e.g. , Support Vector Machine or Random Forest ) or incorporated with a shallow feedforward neural network built on top of the latent codes for simultaneously training both the mechanism to generate data representations and the classifier . The former is named MDSAE-R and the latter is named MDSAE-C. We summarize our contributions in this paper as follows : • We propose a novel method named Maximal Divergence Sequential Auto-Encoder ( MDSAE ) that leverages recent advances in deep learning representation ( namely , VAE ) for binary code vulnerability detection . • One of our most significant contributions is to create a labeled dataset for use in binary code vulnerability detection . In particular , we used the source code in the published NDSS18 dataset used in ( Li et al. , 2018 ) and then extracted vulnerable and non-vulnerable functions . We developed a tool that can automatically detect the syntactical errors in a given piece of source code , fix them , and finally compile the fixed source code into binaries for various platforms ( both Windows OS and Linux OS ) and architectures ( both x86 and x86-64 processors ) . Specifically , after preprocessing and filtering out identical functions from the NDSS18 source code dataset , we obtain 13 , 000 functions of which 9 , 000 are able to be fixed and compiled to binaries . By compiling the source code of these functions under the various platform and architecture options , we obtain 32 , 281 binary functions including 17 , 977 binaries for Windows and 14 , 304 binaries for Linux . • We conducted extensive experiments on the NDSS18 binary dataset . The experimental results show that the two variants MDSAE-R and MDSAE-C outperform the baselines in all performance measures of interest . It is not surprising that MDSAE-C achieves higher predictive performances compared with MDSAE-R , but the fact that MDSAE-R achieves good predictive performances confirms our hypothesis of encouraging the separation in representations of data in different classes so that a simple linear classifier subsequently trained on these data representations can obtain good predictive results . 2 RELATED BACKGROUND . 2.1 THE VARIATIONAL AUTO-ENCODER . The Variational Auto-Encoder ( VAE ) ( Kingma & Welling , 2013 ) is a probabilistic auto-encoder that takes into account both the reconstruction of true samples and generalization of samples generated from a latent space . The underlying idea is to learn a probabilistic decoder pθ ( x | z ) , z ∼ N ( 0 , I ) that can mimic the true data sample x1 , . . . , xN drawn from an existing but unknown data distribution pd ( x ) . VAE is developed based on the following lower bound : log pθ ( x ) ≥ L ( x ; θ , φ ) = Eqφ ( z|x ) [ log pθ ( x | z ) ] −DKL ( qφ ( z | x ) ‖p ( z ) ) where qφ ( z | x ) is the approximate posterior distribution . We need to maximize the log likelihood at each training example x . Therefore the objective function is of the following form : max θ , φ Ex [ Eqφ ( z|x ) [ log pθ ( x | z ) ] −DKL ( qφ ( z | x ) ‖p ( z ) ) ] ( 1 ) where x is drawn from the empirical data distribution . To reduce the variance when using Monte Carlo ( MC ) estimation for tackling the above optimization problem , the reparameterization trick is employed . More specifically , assuming that qφ ( z | x ) = N ( z | µφ ( x ) , diag ( σφ ( x ) ) ) , we can do reparameterization as : z = µφ ( x ) + diag ( σφ ( x ) ) 1/2 where the source of randomness ∼ N ( 0 , I ) and µφ ( z ) , σφ ( z ) are two neural networks representing the mean and covariance matrix of the approximate Gaussian posterior . The optimization problem in Eq . ( 1 ) can be equivalently rewritten as : max θ , φ Ex [ E [ log pθ ( x | µφ ( x ) + diag ( σφ ( x ) ) 1/2 ) ] −DKL ( qφ ( z | x ) ‖p ( z ) ) ] ( 2 ) The first term in Eq . ( 2 ) is regarded as the reconstruction term and the second term in this equation is regarded as the regularization term . In this term , we minimize Ex [ DKL ( qφ ( z | x ) ‖p ( z ) ) ] , hence trying to compress and squash the latent codes z for each true example x into those sampled from the prior distribution p ( z ) . This observation is the key ingredient for us to develop our proposed model .

Summary: This paper sets out to classify source code snippets are “vulnerable” or “not vulnerable” using sequential auto-encoders with two latent distributions (corresponding to the output classes), regularized to maximize divergence between theses two distributions (named Maximal Divergence Sequential Auto-Encoder).  The authors created a compiled subset of the NDSS18 vulnerable vs. non-vulnerable software dataset (which is listed as one of their primary contributions). The dataset construction required non-trivial effort since example code snippets are often incomplete and the authors needed to “fix” these code examples in order to compile them. The fixed code examples are then compiled against both Windows and Linux and both the x86 and x86-64 architectures. The inputs to all predictive models are the opcode sequence of the compiled programs.