Dynamic malware analysis executes the program in an isolated environment and monitors its run-time behaviour (e.g., system API calls) for malware detection. This technique has been proven to be effective against various code obfuscation techniques and newly released ("zero-day") malware. However, existing works typically only consider the API name while ignoring the arguments, or require complex feature engineering operations and expert knowledge to process the arguments. In this paper, we propose a novel and low-cost feature extraction approach, and an effective deep neural network architecture for accurate and fast malware detection. Specifically, the feature representation approach utilizes a feature hashing trick to encode the API call arguments associated with the API name. The deep neural network architecture applies multiple Gated-CNNs (convolutional neural networks) to transform the extracted features of each API call. The outputs are further processed through LSTM (long-short term memory networks) to learn the sequential correlation among API calls. Experiments show that our solution outperforms baselines significantly on a large real dataset. Valuable insights about feature engineering and architecture design are derived from ablation study.
@cite_46 introduce a feature representation called Malware Instruction Set (MIST). MIST uses several levels of features to represent a system call. The first level represents the category and name of API calls. The following levels are specified manually for each API call to represent their arguments. Therefore, the feature from the same level but for different APIs could have different semantics. The inconsistency imposes challenges to learn patterns using machine learning models. Qiao, @cite_5 extend the MIST and propose a representation called Byte-based Behavior Instruction Set (BBIS). They claim that only the first level of MIST is efficient and thus BBIS only uses the category and name of API calls. Besides, they propose an algorithm (CARL) to process consecutively repeated API calls.
@cite_5 @cite_46
1966917005 2121032650
We introduce a new representation for monitored behavior of malicious software called Malware Instruction Set (MIST). The representation is optimized for effective and efficient analysis of behavior using data mining and machine learning techniques. It can be obtained automatically during analysis of malware with a behavior monitoring tool or by converting existing behavior reports. The representation is not restricted to a particular monitoring tool and thus can also be used as a meta language to unify behavior reports of different sources. Analyzing the usage of Windows Application Program Interface (API) is a common way to understand behaviors of Malicious Software (malware) in either static analysis or dynamic analysis methods. In this work, we focus on the usage of frequent messages in API call sequences, and we hypothesize that frequent itemsets composed of API names and or API arguments could be valuable in the identification of the behavior of malware. For verification, we introduced clustering processes of malware binaries based on their frequent itemsets of API call sequences, and we evaluated the performance of malware clustering. Specific implementation processes for malware clustering, including API calls abstraction, frequent itemsets mining and similarity calculation, are illustrated. The experiment upon a big malware dataset demonstrated that merely using the frequent messages of API call sequences can achieve a high precision for malware clustering while significantly reducing the computation time. This also proves the importance of frequent itemsets in API call sequences for identifying the behavior of malware. Sequential models which analyze system API calls have shown promise for detecting unknown malware. Athiwaratkun and Stokes recently proposed a two-stage model which uses a long short-term memory (LSTM) model for learning a set of features which are then input to a second classifier. , first use a convolutional neural network followed by an LSTM to predict unknown malware. However, neither of these models consider the parameters which are input to the system API calls. These input parameters offer significant information regarding malicious intent. In this paper, we extend Athiwaratkun's model to include each system API call's two most input parameters. We then show that the proposed model dominates these previously proposed models in terms of the receiver operating characteristic (ROC) curve. Since signature based methods cannot identify sophisticated malware quickly and effectively, research is moving toward using samples' runtime behavior. But these methods are often slow and have lower detection rate and are not usually used in antivirus software. In this article we introduce a scalable method that relies on utilizing features other than traditional API calls to obtain higher accuracies. Two feature categories including API names and a combination of API names and their input arguments were extracted to investigate their effect in identifying and distinguishing malware and benign applications. Feature selection techniques are then applied to reduce the number of features and enhance the analysis time. Various classifiers were then utilized along with 10-fold cross validation approach to achieve an accuracy of 98.4 with a false positive rate less than two percent in best case. The small number of extracted features in the proposed technique and the high accuracy achieved makes it an appropriate approach to be used in industrial applications.