# Feed-Forward Deep Neural Network (FFDNN)-Based Deep Features for Static Malware Detection

Authors: Priyanka Singh; Samir Kumar Borgohain; Achintya Kumar Sarkar; Jayendra Kumar; Lakhan Dev Sharma
Journal: International Journal of Intelligent Systems
 (2023)

---

## Abstract

The portable executable header (PEH) information is commonly used as a feature for malware detection systems to train and validate machine learning (ML) or deep learning (DL) classifiers. We propose to extract the deep features from the PEH information through hidden layers of a feed-forward deep neural network (FFDNN). The extraction of deep features of hidden layers represents the dataset with a better generalization for malware detection. While feeding the deep feature of one hidden layer to the succeeding layer, the Gaussian error linear unit (GeLU) activation function is applied. The FFDNN is trained with the GeLU activation function using the deep features of individual layers as well as concatenated deep features of all hidden layers. Similarly, the ML classifiers are also trained and validated in with individual layer deep features and concatenated features. Three highly effective ML classifiers, random forest (RF), support vector machine (SVM), andk-nearest neighbour (k-NN) have been investigated. The performance of the proposed model is demonstrated using a statically significant large dataset. The obtained results are interesting and encouraging in terms of classification accuracy. The classification accuracy reaches 99.15% with the internal discriminative deep feature for the proposed FFDNN-ML classifier with the GeLU activation function.


---

## Body

## 1. Introduction

The statistical data till April 2022 show that the Windows operating system is being used by around 75% of computer users globally [1]. Being the widely used operating system for desktops and laptops in organizations and for personal computing, these systems are more prone to malware infections. Healthcare organizations were highly affected during the peak of the COVID-19 pandemic [2]. The computers of employees working from home (WFH) and educational institutions were attacked severely [3]. Cyber infection is disseminated through PE in various forms stored or installed in the system to perform a variety of malicious actions such as gaining access to its resources, stealing confidential data, and redirecting to malware host websites. Malware attacks have spread terror among the users of the computer and society. With the increase in the use of computer systems, malware has been replicating and proliferating exponentially since the beginning of the computer era. Despite the security concerns raised by the industry to safeguard computers and networks from cyber-attack, privacy and security are still of high concern. To halt this nuisance created by the malware creators, continuous observation and prevention are needed. Also, the systems need to be updated in real time. Nevertheless, as malware has evolved into the form of viruses, worms, Trojans, and its variants, different malware detection techniques have been developed by researchers.Traditional malware detection such as the signature-based approach has become outdated as it can only detect malware if the pattern or the signature is matched against the signature stored in the repositories [4]. These signatures are the unique patterns generated during the static execution of the malware executables [5]. Also, code obfuscation and shuffling of the code snippet are no more backbreaking jobs for the creator of the malware that results in the signature change of the same malware variant [6]. The behaviour-based approach evaluates malware dynamically executing the thief and analyzing it in an isolated environment. The behavioural log such as the execution sequence of the malware is recorded for further prosecution [7]. However, the direct execution of malware on a system for monitoring is always dangerous as it corrupts system files. On the other hand, the newly evolved malware had become smart enough to bypass the isolated environments [5]. The heuristic-based approach performed better than the signature and behaviour-based ones for detecting unknown malware and zero-day attack [8]. This approach has sufficient rules and patterns for the investigation of malware behaviour and controls the flow of the code. However, a higher false-positive rate is identified as a major limitation of the heuristic-based malware detection systems [9]. With the increasing number of computer users and the availability of advanced malware creation tools, malware detection became trivial. This led to the proliferation and speedy propagation of malware all over the globe leading to a massive amount of malware data. Researchers and industry have now adopted the data mining and machine learning (ML) approach to detect and classify malware [10, 11]. Rapid research contribution has been recognized in the last few decades in the field of malware detection using the ML approach.In ML techniques, data in terms of features are fed to train a model to learn and estimate a possible pattern of the data. Therefore, feature extraction is an important step in ML while preparing a dataset [12]. Definitely, a better representation of features makes the training and detection highly efficient. Nowadays, a drastic increase in malware has made the training phase time-consuming and tedious and ML classifiers inefficient [13]. The effectiveness of an ML classifier highly depends on the feature representation and the nature of the dataset used for training. The ML approach depends on feature extraction which requires proficient knowledge of the realm. It makes the false-positive rate (FPR) low which is an important consideration during the implementation of ML algorithms [14].Research in the realm of malware detection has embraced the diverse computational approach. In recent years, the nature-inspired computational approach has been geared up for malware detection, feature optimization, and classification [15–17]. That nature has been always very successful in solving complex objectives which inspired researchers to adopt bioinspired algorithms. The bioinspired algorithms have been successfully implemented for various applications such as image processing, robotics, fraud detection, and intrusion detection. Neurocomputing or artificial neural network is a subfield of nature or bioinspired computation that has been applied for malware detection and classification on windows, android systems, and IoT devices [18, 19]. However, the study and discussion about malware detection for android and IoT devices are out of the scope of this work. As with the availability of a massive amount of malware data to experiment with, deep neural networks or deep learning mimicking the human brain solves the problem of processing information from a huge volume of data. This deep bioinspired computation provides more and more generalization as an effect of the processing of neurons. As bioinspired evolutionary computing has gained momentum in the field of optimization, the deep learning techniques inspired by human brain functioning are the best learners in the context of generalization [20, 21]. Deep learning has been used in the field of cybersecurity in recent times for the prediction of cyberattacks. As the human brain learns to perceive, represent, and process the data, a deep neural network learns data representations to generate the model with the representations and utilizes this learning efficiently when new data come. This improves the generalization capability of the model as compared to other bioinspired methods. Also, scalability is no more an issue to performance as the huge volume of data can be handled without being more careful about feature engineering steps.Different malware detection models based on DNN or ML classifiers such as random forest (RF) [22], support vector machine (SVM) [23], and k-nearest neighbour (k-NN) [24] classifiers use PEH, signature, behavioural log such as the execution sequence as features for malware detection. Credit card fraud detection with DNN [25] and optimization of model hyperparameters using genetics and swarm algorithms can be found in [26–28]. In recent years, deep feature extraction of DNN has gained attention in many domains of pattern recognition [29–32]. In deep feature extraction, the DNN is trained using input data such as PEH with an activation function to minimize the classification error. Now, the output of the subsequent layer without the activation functions is considered deep features. Now the deep features of a hidden layer are used to train ML classifiers. As per our survey, the deep features have not been exploited much for malware detection. Therefore, we propose to extract deep features of an FFDNN with a multitask objective function that simultaneously classifies benign and malware. Thereafter, the deep features of hidden layers have been used to train the ML classifiers, RF, SVM, and k-NN. Besides, we also consider the concatenation of different deep features for malware detection. The network parameters are optimized using error backpropagation, and the output of DNN neurons and gradient contribution is controlled by the activation function. The nonlinear modelling capability of deep neural networks possesses the significant discriminative capability and is a potent back-end classifier.In this work, we propose a deep feature-based malware detection system where an FFDNN [33] is trained with multitask objective functions. During the network backpropagation for parameter optimization, the GeLU activation function is used to control the DNN neuron’s output and gradient contribution. The sigmoid [34, 35] and rectified linear unit (ReLU) [36] activation functions are commonly used for different applications such as speech recognition [37, 38] and image processing [39, 40] in the literature. The sigmoid function squashes the input space from 0 to 1. This results in a small derivative and gradient vanishing. The gradient vanishing poorly updates the initial layers yielding ineffective training; thus, the model lack generalization [41, 42]. The ReLU has a dynamic range of input and can be used for effective training, but it needs stochastic regularization as it lacks probabilistic interpretation [43]. The GeLU is introduced in [43] as a deterministic activation function that combines stochastic regularization. It is shown in [43] that the GeLU performs better than the rectified linear unit (ReLU) and exponential linear unit (eLU) for speech and language processing and computer vision applications. We have investigated the performance of different activation functions and found GeLU best performing for the proposed malware detection system too.


## 2. Contributions

The contributions of this work are in many folds as follows:(i)
We propose a DNN-based deep feature for a malware detection system, where PEH information are fed as input to the FFDNN with multitask objective functions at the output layer. It simultaneously classifies the malware and nonmalware sample and minimizes the prediction for the input at the output. Afterward, the output from a particular hidden layer is used as deep features. This deep feature representation is used to train the FFDNN and ML classifiers, RF, SVM, and KNN.(ii)
The deep features of one hidden layer are extensively analyzed and also concatenated with deep features of other hidden layers.(iii)
To the best of our knowledge, the GeLU activation function has been used for the first time in combination with DNN and ML classifiers for static malware detection. The performance is evaluated against the commonly used ReLU, scaled exponential linear units (SeLU), and eLU on a large dataset for statistical significance.(iv)
The proposed deep feature-based FFDNN-ML malware detection system shows better performance over the state of the art


## 3. Related Work

A 2-hidden layer DNN-based malware detection system is reported in [44], and results are presented against a binary dataset of 431,926 instances. Entropy histogram, PE Import, 2D strings, and PE metadata features have been used as extracted features. A better convergence was achieved with the parametric ReLU (PReLU) activation function and Adam optimizer. This system achieves a 95% detection rate (DR) with a 0.1% false-positive rate (FPR) with 1024 neurons in the input layer. An interesting time split validation performed over a traditional cross-validation approach is appreciable. However, the performance of time split validation has been depicted as degrading on test data. In [45], a staked autoencoder (SAE)-based DNN system for malware detection is reported. The system performance is demonstrated using the Windows API calls for a dataset consisting of 50,000 malware and benign executables. They achieved an accuracy of 96.85% with 3 hidden layers of 100 neurons per layer. However, the implication of the activation function may further improve the performance of this system. Yuxin and Siyi detected malware using a malware detection system based on the deep belief network (DBN) model with 2 hidden layers and a maximum of 200 neurons [46]. Performance evaluation is carried out by partitioning an OpCode-n-gram dataset into 4 sets of 850 malware and 850 benign samples. An accuracy of 96.5% is achieved using 200 distinct features. DBN is stacked up with restricted Boltzmann machine (RBM) layers that are related to global weight and tuning of the weights. However, DBN suffers a vanishing gradient problem [47]. Thus, the nonlinear activation function in DBN is critical in reducing the network complexity and improving the performance of pattern recognition.DeepMalNet architecture for malware detection has been reported in the literature [48]. This malware detection system is evaluated on the static representation extracted from portable executables. The Ember dataset [49] used in this work is a labelled benchmark dataset. The proposed work is trained using 10 hidden layers, an input layer with 2350 neurons and 1 output neuron. Experiments were performed on activation functions such as sigmoid, tanh, ReLU, eLU, and SeLU. The learning rate between 0.01 and 05 has been applied to the experimentation during 1000 iterations. An accuracy of 98.9% is achieved with the ReLU activation function with 10 hidden layers of DNN. However, the elaborated experimentation results with various parameter tuning are not mentioned in the work. Ye et al. proposed a heterogeneous deep learning framework for intelligent malware detection known as DeepAM [50]. This framework is composed of SAE multilayer restricted Boltzmann machines and a layer of associative memory. To train the model, a balanced dataset consists of 4500 benign files and 4500 malware files that have been used and tested using 1000 samples. WindowAPI call sequence was extracted for feature representation. For fine-tuning, a wake-sleep algorithm and a gradient descent algorithm are applied to modify the downward generation of weights between the layers. An accuracy of 98.82% is achieved with 3 hidden layers of 100 neurons in each layer. However, weight updating is not as smooth as backpropagation in FFDNN [51]. Rathore et al. proposed a malware detection system with an ML classifier and deep learning that used autoencoders for dimensionality reduction [52]. The dataset consists of 14,507 samples (2819 benign and 11308 malware executables). OpCode frequency was used as a discriminatory feature for evaluation. The mean square error (MSE) is used as a loss function, and the network is trained for 120 epochs. An accuracy of 98.9% is achieved using a 7 hidden layer DNN model.The malware visualization approach using a convolution neural network (CNN) for malware detection and classification has been trending in malware research as state of the art. Zhong and Gu proposed a multilevel deep learning system (MLDLS) where multiple deep learning models are organized in a tree structure [53]. Both static and dynamic features are extracted from malware and binary executables to develop a dataset with 2242,234 malware samples and 3425,176 benign samples. The major contribution is parallel computation by data distribution for each group of malware. MLDLS outperforms the other single deep learning models. A leaky ReLU activation function is used in the overall experiment. The model construction time is significantly less due to the parallel distribution of data and organization of the tree structure. As the number of samples is increased, the true positive rate (TPR) is significantly increased above 96% and the false positive rate (FPR) is decreased below 0.1%. However, the computational time is considerably high. Also, the conversion of tabular data to images using CNN has certain limitations [54].In [55], a DNN malware detection system based on visualization is presented. The malware detection system developed using deep learning architectures has been implemented on static, dynamic analysis, and image processing in the work. The authors have implemented a hybrid deep learning architecture incorporating CNN, recurrent neural network (RNN), and fully connected DNN for malware detection. In this work, various models both traditional and advanced ML models are trained for performance evaluation. The FCNN (DNN) for static analysis outperforms other models with an accuracy of 98.9%. However, the robustness of the proposed system framework is not discussed in this work. Authors in [56] proposed a malware detection system using convolution neural network-based VGG16 network functions for visualization of static and dynamic features of executables.The performance evaluation of the system is carried out for two models based on static and hybrid visualization. The hybrid model outperforms the static model with an accuracy of 94.70% on the test dataset. However, the experimentation is performed on a small dataset that is statistically insignificant. In [57], a similar visualization approach has been used. The PE executable is converted into a grayscale image, and then the image patterns are learned for classification. The authors proposed a deep forest method applying sliding windows and cascading layers wherein the sliding window approach is adapted from CNN. The highest accuracy obtained is 98.65% on the Malimg dataset. The malware visualization technology provides a vivid image of different malware types that aids in visualizing the difference between variants of malware. Although, the visualization method can tackle the code obfuscation problem but requires more time for the extraction of complex image patterns. Also, the transformation of tabular data to images suffers certain limitations.


## 4. Bioinspired Neural Network

Human brains possess superiority in executing tasks such as pattern recognition, flexible inference, control, perception, intuition, prediction, and decision-making. An artificial neural network is loosely inspired by the human nervous system and focuses on the learning and problem-solving skills of the system at hand. The neurons are the fundamental computation unit of the human brain connected to each other with a minute junction called the synapse. These junctions are responsible for cognitive abilities, such as perception, inference, and thought processing. Neurocomputation is brain-like computing and also referenced with artificial neural network (ANN) which is a subfield of bioinspired computation [20].Deep artificial neural networks or deep learning architectures imitate the pursuit in layers of neurons in the neocortex. It intelligently learns the stratified layers, representation levels, and abstractions. It comprehends the patterned data that it perceives from various sources such as numerical data, images, sound, signals, and text. Abstraction at a higher layer is the combination of abstractions of lower layers. These deep abstractions are generated from more than a single layer of nonlinear feature transformation. Deep learning automatically learns the feature representation at multiple layers of abstraction that aids in the automated mapping of the functions from the input to the output layer. This automated mapping does not require expert knowledge of feature engineering.


### 4.1. Neural Perspective

The study of the human nervous system is a vast domain. In this work, the organization and processing of the human nervous system are discussed in the context of a computational artificial neural network. Different levels of the nervous system can be organized as molecules, synapses, neurons, layers, maps, and systems. Out of the complex organization, the neurons are more evident and potential units for signal generation and capable of transmitting information to connected cells. Each neuronal unit performs its own specific function. One of such eminent units for comprehension of signal processing is synapses in the nervous system [58].Neurons consist of special extensions called neurites that can be divided into dendrites and axons. The function of the dendrites is to receive signals from other neurons. The axon performs propagation of the output signals to the connected neurons. The neurons are specialized in sending and receiving signals to and from other neurons. The neurons that send the signal are known aspresynaptic neurons, and the neurons that receive the signal are known to be postsynaptic. Thus, synapses are the junctions between the presynaptic and postsynaptic neurons. An equivalent nervous system representation and its components with directional signal processing are depicted in Figures 1(a) and 1(b), respectively. The propagation of the signals up to the cell body from presynaptic neurons is integrated after signal generation. The potential membrane is then responsible for making a decision whether the neurons are firing or sending the output signal to postsynaptic neurons depending on the neuron threshold. The connection between the neurons can be either forward or backward in the nervous system. These interconnected neurons are called neural networks. In a network of neurons, these neurons are organized into input, hidden, and output layers.Figure 1
Illustration of neurons and neural network equivalent representation in the nervous system. (a) Structural representation and signal flow. (b) Level-wise structural components in the nervous system.
(a)(b)ANN maps the architecture, features, and performance criteria similar to the human nervous system. The fundamental information processing occurs in neurons or nodes wherein neurons can exchange signals among other neurons. The neurons are connected to each other forming a neural network. The strength of the synapses corresponds to the weight value in the neurons. The learning process from the given data for adaptive knowledge is similar to the learning of the nervous system from the environment. However, the computational neural network does not fully resemble the complete structure and working of the nervous system.


### 4.2. Representation Perspective

The interconnected neurons demonstrate complex behaviours and information processing capabilities even in a small cluster. Single neurons cannot store and process complete or whole knowledge. The representation of information is distributed in a hierarchical manner, and the parallel processing of this information over various neurons is an important feature of the neural network [59]. In order to perform the more specific complex tasks with respect to information processing, specialized architectures are incorporated into the larger structure of networks. For most bioinspired artificial intelligence (AI) and ML algorithms, the performance heavily depends on the representation of data. So, feature representation plays a crucial role in the performance of the system designed [60, 61]. Traditionally, the handcrafted predefined feature as sectional header information shown in Figure 2 is fed to DNN/ML algorithms for classification.Figure 2
Illustration of some header/attribute information of the files in the dataset.Recently, DNN has gained the attention of researchers in different applications including speech and image processing. It is demonstrated by researchers that a system based on the deep representation feature yields better or provides complementary information. In other words, deep learning is found very useful for data learning, i.e., the mapping between the input and the output. In this concept, traditional (say, handcrafted) features are fed to train a DNN with an objective function to classify the target at the output layer (need label information, i.e., supervised) and predict the input at the output (unsupervised, i.e., no need label information) or combination of them. It is believed that different layers of the DNN capture different attributes for a particular task in high dimensional space. So, the output from the particular hidden layer of the DNN for a given input is extracted as adeep feature. The deep features are then used for classification using ML techniques. Another advantage of DNN is that it can handle a large amount of data by manipulating its model parameters. However, human-crafted data lacks flexibility and cannot be applied to diverse scenarios. Also, systems using such data cannot adapt to new data leading to poor generalization ability. This has been a powerful motivation for crafting flexible and automated feature representation methods.The evolution of deep learning can be considered as representation learning, wherein the process of feature extraction is automated when the deep architecture processes the data, learns, and understands the mapping between the input and the output. Deep learning is a data-driven process, as the model learns from the data presented. This aids in a significant boost in performance as human-developed feature extraction is deficient in accurate detection and generalization. Besides automation in feature extraction and feature learning, the learned feature representations occur in a distributed and hierarchical manner. The feature learning task is distributed over the layers each one computing its allotted task. In a distributed dense feature representation, multiple neurons concurrently represent a learned parameter.


## 4.1. Neural Perspective

The study of the human nervous system is a vast domain. In this work, the organization and processing of the human nervous system are discussed in the context of a computational artificial neural network. Different levels of the nervous system can be organized as molecules, synapses, neurons, layers, maps, and systems. Out of the complex organization, the neurons are more evident and potential units for signal generation and capable of transmitting information to connected cells. Each neuronal unit performs its own specific function. One of such eminent units for comprehension of signal processing is synapses in the nervous system [58].Neurons consist of special extensions called neurites that can be divided into dendrites and axons. The function of the dendrites is to receive signals from other neurons. The axon performs propagation of the output signals to the connected neurons. The neurons are specialized in sending and receiving signals to and from other neurons. The neurons that send the signal are known aspresynaptic neurons, and the neurons that receive the signal are known to be postsynaptic. Thus, synapses are the junctions between the presynaptic and postsynaptic neurons. An equivalent nervous system representation and its components with directional signal processing are depicted in Figures 1(a) and 1(b), respectively. The propagation of the signals up to the cell body from presynaptic neurons is integrated after signal generation. The potential membrane is then responsible for making a decision whether the neurons are firing or sending the output signal to postsynaptic neurons depending on the neuron threshold. The connection between the neurons can be either forward or backward in the nervous system. These interconnected neurons are called neural networks. In a network of neurons, these neurons are organized into input, hidden, and output layers.Figure 1
Illustration of neurons and neural network equivalent representation in the nervous system. (a) Structural representation and signal flow. (b) Level-wise structural components in the nervous system.
(a)(b)ANN maps the architecture, features, and performance criteria similar to the human nervous system. The fundamental information processing occurs in neurons or nodes wherein neurons can exchange signals among other neurons. The neurons are connected to each other forming a neural network. The strength of the synapses corresponds to the weight value in the neurons. The learning process from the given data for adaptive knowledge is similar to the learning of the nervous system from the environment. However, the computational neural network does not fully resemble the complete structure and working of the nervous system.


## 4.2. Representation Perspective

The interconnected neurons demonstrate complex behaviours and information processing capabilities even in a small cluster. Single neurons cannot store and process complete or whole knowledge. The representation of information is distributed in a hierarchical manner, and the parallel processing of this information over various neurons is an important feature of the neural network [59]. In order to perform the more specific complex tasks with respect to information processing, specialized architectures are incorporated into the larger structure of networks. For most bioinspired artificial intelligence (AI) and ML algorithms, the performance heavily depends on the representation of data. So, feature representation plays a crucial role in the performance of the system designed [60, 61]. Traditionally, the handcrafted predefined feature as sectional header information shown in Figure 2 is fed to DNN/ML algorithms for classification.Figure 2
Illustration of some header/attribute information of the files in the dataset.Recently, DNN has gained the attention of researchers in different applications including speech and image processing. It is demonstrated by researchers that a system based on the deep representation feature yields better or provides complementary information. In other words, deep learning is found very useful for data learning, i.e., the mapping between the input and the output. In this concept, traditional (say, handcrafted) features are fed to train a DNN with an objective function to classify the target at the output layer (need label information, i.e., supervised) and predict the input at the output (unsupervised, i.e., no need label information) or combination of them. It is believed that different layers of the DNN capture different attributes for a particular task in high dimensional space. So, the output from the particular hidden layer of the DNN for a given input is extracted as adeep feature. The deep features are then used for classification using ML techniques. Another advantage of DNN is that it can handle a large amount of data by manipulating its model parameters. However, human-crafted data lacks flexibility and cannot be applied to diverse scenarios. Also, systems using such data cannot adapt to new data leading to poor generalization ability. This has been a powerful motivation for crafting flexible and automated feature representation methods.The evolution of deep learning can be considered as representation learning, wherein the process of feature extraction is automated when the deep architecture processes the data, learns, and understands the mapping between the input and the output. Deep learning is a data-driven process, as the model learns from the data presented. This aids in a significant boost in performance as human-developed feature extraction is deficient in accurate detection and generalization. Besides automation in feature extraction and feature learning, the learned feature representations occur in a distributed and hierarchical manner. The feature learning task is distributed over the layers each one computing its allotted task. In a distributed dense feature representation, multiple neurons concurrently represent a learned parameter.


## 5. Methodology

In this section and the subsequent subsections, we elaborately describe the proposed system for malware detection. It consists of FFDNN, deep feature extraction, classification using DNN, and different ML techniques. The overall system is illustrated in Figure3. The components of the proposed DNN architecture are briefly discussed as follows. The experimentation of the proposed work starts with a traditional approach of training the ML classifiers and performing classification with the k-fold method. As our objective is to deal with a large volume of current and future generations of data, these ML classifiers alone are not sufficient. Thus, the DNN and ML classifiers are effectively integrated and discussed in the following sections.Figure 3
Depiction of the proposed bioinspired deep neural network static malware detection system.Initially, exclusive RF, SVM, and k-NN ML classifiers have been trained and tested for classification accuracy. The performance of these classifiers has been found to be poor due to a large number of instances, and this motivated authors to investigate the bioinspired DNN for efficient classification. The deep features of each layer are exploited in independent capabilities and concatenation. The proposed FFDNN-ML malware prediction and classification model is implemented in two-phase with the concept of feature concatenation and deep feature extraction. In both phases, feature concatenation is applied in the context of feature representation learning for classification. In the first phase, the features generated from each hidden deep layer are finally concatenated to train the FFDNN for prediction and classification. In the second phase, deep features of hidden layers are applied separately or in concatenation to the ML classifier, RF, SVM, and k-NN ML with parametric tuning for classification. This extracted deep feature provides better generalization for unrelated and untrained data. Feature concatenation effectively sums up the features with different criteria that aid in an efficient classification process.


### 5.1. FFDNN

The FFNNs have been used for training the model for multitasking objectives [62–64]. In a fully connected layer also called a dense layer, all the neurons in a layer are connected or mapped to each and every neuron in the previous layer. The input vector, x =  x1,x2,⋯xm, is multiplied with a m×n matrix of weights, and the output result from the layer is submitted to a nonlinear activation function. The weight matrix W is given by matrix 1.(1)W=w11w12…w1nw21w22…w2n⋮⋮…⋮wm1wm2…wmn.The neuron functions as a combinatory circuit perform the scalar product of the input vectorx and the weight vector w = w1,w2,⋯wm. The result generated is applied with the activation function f. The whole neural network generates the output vector y =  y1,y2,⋯yn. The activation function remains constant for each neuron. Thus, each NN is characterized by the weight matrix.In this method, handcrafted PEH information is fed to train an FFDNN with GeLU activation using multitask objective functions: we simultaneously (i) discriminate the malware and nonmalware classes and (ii) predict the input at the output layer of the DNN using the cross-entropyLce and L1 loss functions, respectively. At the last hidden layer of the DNN, the output from the previous different hidden layers is first concatenated and then projected to the output layer. The overall loss L function of the proposed FFDNN can be expressed as follows:(2)L=γLce+1−γL1=−γ1N∑i=1NzilogpXi+1−γ1N∑i=1NXi−X^i,where zi and X^i indicate the label and predicted value of Xi sample, respectively. γ decides the weight between the loss function. In our experiment, we provide equal weight to both loss functions. For analysis, we perform the experiment for different numbers of hidden layers (varies from 3 to 20), activation functions (ReLU, eLU, and SeLU), and the number of neurons per layer (varies from 32 to 256) in the proposed method.


#### 5.1.1. Perceptron

Perceptron is the simplest and basic ANN architecture. Rosenblatt [65] proposed the perceptron training algorithm that was inspired by Hebb’s rule [66]. According to Donald Hebb, when a human neuron activates another neuron, bonding between the neurons becomes more and more strong. This ANN accepts one data point at once to make a prediction and uses the perceptron learning rule (PLR) to minimize error for any incorrect prediction during training. The PLR is presented in equation (3). One training instance is fed at a time to perceptrons to make predictions. For a wrong prediction of an output neuron, the perceptron learning rule strengthens the connections for the reduction of the error generated. This reinforcement rule contributes toward the correct prediction. The PLR is presented in the following equation:(3)wi,jnext=wi,j+ηyj−y^jxi,where wi,j is weight connecting the ith neuron and jth neuron, xi is the ith instance for training, yj is the output of the jth neuron, y^j is the predicted output value of the jth neuron to generate output, and η is the learning rate used during PLR. This basic ANN model is unable to learn complex or nonlinear patterns for decision-making as the plane separating each class is linear. The limitation of the perceptron can be covered by using multilayer perceptron (MLP) ANN architecture. The MLP with various modifications is being used in deep learning research. Fundamentally, it is organised with an input layer, an output layer, and one or more hidden layers existing between the input and output layers. An MLP with all-to-one mapping between the neurons of subsequent layers is known as a fully connected neural network (FCNN) in deep learning as shown in Figure 4. An MLP with a deeper stack than one hidden layer is a deep neural network. The decision boundary for each output neuron or class is linear, so this basic perceptron model is unable to learn complex or nonlinear patterns for decision-making. The limitation of perceptrons can be covered by using multilayer perceptron (MLP) ANN architecture. The MLP with various modifications is being used in deep learning research. An MLP is organized with one input layer, one or more layers of hidden layers, and one final layer called the output layer. The MLP with all-to-one mapping between the neurons of subsequent layers is known as a fully connected FFNN in deep learning and is depicted in Figure 4. All layers excluding the output layer consisting of a bias neuron are fully connected to the next layer. An MLP with a deeper stack than one hidden layer is a deep neural network.Figure 4
Illustration of an FFDNN network architecture with input, hidden layer, neurons, and output.


#### 5.1.2. Backpropagation

During the network training, the output either for a single or multiobjective task is generated in the forward pass. This generated output value may be different from the original value. So, the backpropagation of an error signal is evaluated to minimize the cost function. For this proposed work, cross-entropy functionLce and L1 loss function are used to evaluate the classification probability and minimize the reconstruction error, respectively. The reason behind applying L1 loss for this work is that the data have been used without preprocessing which reduces feature preprocessing overhead in deep learning. So, the data may contain outliers, and the L1 loss function is not affected by the outliers present in the dataset. For multitask objectives, both the cost function for classification and prediction needs to be backpropagated to strengthen the connection. Cross-entropy function Lce and L1 loss function are expressed in the following equations:(4)Lce=−γ1N∑i=1NzilogpXi,(5)L1=1−γ1N∑i=1NXi−X^i.A schematic diagram of the backpropagation technique is depicted in Figure5. The working of the backpropagation algorithm is discussed henceforth. The full training is completed in batches in multiple iterations wherein each iteration is known as an epoch. The training instances are fed to the input layer that integrates the required parameters with each instance and fires it to the initial hidden layer as per activation. This forward passing of the feature vectors with required bias, weight, and activation function from one input layer to the next or a series of hidden layers and eventually to the output layer to generate the target output value is forward propagation. This phenomenon is also known as forwarding pass. The loss function is used to calculate the error between the predicted output value and the labelled output. For optimizing the value for better performance and generalization of the model, backward propagation or backward pass is applied during the training. The algorithm analyses and computes how much each connection weight and each bias parameter can be altered for error reduction by applying the chain rule which helps the backpropagation step fasten accurately. Backpropagation minimizes the cost function by fine-tuning the network’s weights and biases. For adjusting this parameter, the gradients of the cost function with respect to weights and bias need to be computed using the chain rule given in the following equation [61], where ni is the ith node in the network.(6)∂f∂x=∂f∂ni+∂ni∂x.Figure 5
Illustration of backpropagation and optimization processes in a neural network.


#### 5.1.3. GeLU Activation Function

The initial perceptron used binary threshold units [67]. The binary decision was further flattened using sigmoid activation functions and trained with backpropagation [68]. As the network grew deeper and deeper, the sigmoid function became less effective due to nonlinearity [61]. ReLU succeeded over the sigmoid which is preferred mostly due to its fast and better convergence property. Then, the eLU activation function which is a modification of ReLU was developed which improved the training speed on negative values [69]. The prediction of the deterministic decision during the evaluation of the network leads to new nonlinearity. The stochastic regularizer yields nonlinearity for an input x that randomly applies the identity or zero maps to a neuron’s input. The Gaussian error linear unit (GeLU) is a high-performing neural network activation function [70]. Unlike signs in ReLU, the inputs are weighted in magnitude by the GeLU nonlinearity. The GeLU activation function is xΦx, where Φx is the standard Gaussian cumulative distribution function. The GeLU function expressed using equation (7) is estimated using equation (8).(7)GeLUX=xPX≤x=xΦx,(8)GeLUX=0.5x1+tanh2πx+0.044715x3.


#### 5.1.4. Deep Feature

In this step, the handcrafted feature vector for a given inputX=x1,x2,…,xT is fed to the DNN obtained in Section 5.1. Then, the output from the particular hidden layer (without applying the activation function) is extracted as a deep feature (for the particular input). It can be expressed as(9)XDF−h=fFF−DNNX,where XDF−h denotes the deep feature from the hth hidden layer of the DNN for input X. This basically represents the mapping of the input data from one space to another. In this manner, deep features are extracted for all input data in the experiments. In the case of 512 neurons, per hidden layer in DNN yields the 512 dimension deep feature vector. Besides, we also consider the study of the concatenation of deep features extracted from the different hidden layers as a presentation. For example, DF1−5 indicates the concatenated deep feature of the hidden layers 1 to 5. If each hidden layer has 512 neurons, then DF1−5 gives 5×512 dimensional vector.The deep feature extracted from each hidden layer of the FFDNN has better representation and generalization. In the final step, deep features extracted from each hidden layer are then used layer-wise and in concatenation for the classification of malware using RF, SVM, and KNN classifiers. The deep feature extracted from each hidden layer of the FFDNN constitutes a malware classification system.


## 5.1. FFDNN

The FFNNs have been used for training the model for multitasking objectives [62–64]. In a fully connected layer also called a dense layer, all the neurons in a layer are connected or mapped to each and every neuron in the previous layer. The input vector, x =  x1,x2,⋯xm, is multiplied with a m×n matrix of weights, and the output result from the layer is submitted to a nonlinear activation function. The weight matrix W is given by matrix 1.(1)W=w11w12…w1nw21w22…w2n⋮⋮…⋮wm1wm2…wmn.The neuron functions as a combinatory circuit perform the scalar product of the input vectorx and the weight vector w = w1,w2,⋯wm. The result generated is applied with the activation function f. The whole neural network generates the output vector y =  y1,y2,⋯yn. The activation function remains constant for each neuron. Thus, each NN is characterized by the weight matrix.In this method, handcrafted PEH information is fed to train an FFDNN with GeLU activation using multitask objective functions: we simultaneously (i) discriminate the malware and nonmalware classes and (ii) predict the input at the output layer of the DNN using the cross-entropyLce and L1 loss functions, respectively. At the last hidden layer of the DNN, the output from the previous different hidden layers is first concatenated and then projected to the output layer. The overall loss L function of the proposed FFDNN can be expressed as follows:(2)L=γLce+1−γL1=−γ1N∑i=1NzilogpXi+1−γ1N∑i=1NXi−X^i,where zi and X^i indicate the label and predicted value of Xi sample, respectively. γ decides the weight between the loss function. In our experiment, we provide equal weight to both loss functions. For analysis, we perform the experiment for different numbers of hidden layers (varies from 3 to 20), activation functions (ReLU, eLU, and SeLU), and the number of neurons per layer (varies from 32 to 256) in the proposed method.


### 5.1.1. Perceptron

Perceptron is the simplest and basic ANN architecture. Rosenblatt [65] proposed the perceptron training algorithm that was inspired by Hebb’s rule [66]. According to Donald Hebb, when a human neuron activates another neuron, bonding between the neurons becomes more and more strong. This ANN accepts one data point at once to make a prediction and uses the perceptron learning rule (PLR) to minimize error for any incorrect prediction during training. The PLR is presented in equation (3). One training instance is fed at a time to perceptrons to make predictions. For a wrong prediction of an output neuron, the perceptron learning rule strengthens the connections for the reduction of the error generated. This reinforcement rule contributes toward the correct prediction. The PLR is presented in the following equation:(3)wi,jnext=wi,j+ηyj−y^jxi,where wi,j is weight connecting the ith neuron and jth neuron, xi is the ith instance for training, yj is the output of the jth neuron, y^j is the predicted output value of the jth neuron to generate output, and η is the learning rate used during PLR. This basic ANN model is unable to learn complex or nonlinear patterns for decision-making as the plane separating each class is linear. The limitation of the perceptron can be covered by using multilayer perceptron (MLP) ANN architecture. The MLP with various modifications is being used in deep learning research. Fundamentally, it is organised with an input layer, an output layer, and one or more hidden layers existing between the input and output layers. An MLP with all-to-one mapping between the neurons of subsequent layers is known as a fully connected neural network (FCNN) in deep learning as shown in Figure 4. An MLP with a deeper stack than one hidden layer is a deep neural network. The decision boundary for each output neuron or class is linear, so this basic perceptron model is unable to learn complex or nonlinear patterns for decision-making. The limitation of perceptrons can be covered by using multilayer perceptron (MLP) ANN architecture. The MLP with various modifications is being used in deep learning research. An MLP is organized with one input layer, one or more layers of hidden layers, and one final layer called the output layer. The MLP with all-to-one mapping between the neurons of subsequent layers is known as a fully connected FFNN in deep learning and is depicted in Figure 4. All layers excluding the output layer consisting of a bias neuron are fully connected to the next layer. An MLP with a deeper stack than one hidden layer is a deep neural network.Figure 4
Illustration of an FFDNN network architecture with input, hidden layer, neurons, and output.


### 5.1.2. Backpropagation

During the network training, the output either for a single or multiobjective task is generated in the forward pass. This generated output value may be different from the original value. So, the backpropagation of an error signal is evaluated to minimize the cost function. For this proposed work, cross-entropy functionLce and L1 loss function are used to evaluate the classification probability and minimize the reconstruction error, respectively. The reason behind applying L1 loss for this work is that the data have been used without preprocessing which reduces feature preprocessing overhead in deep learning. So, the data may contain outliers, and the L1 loss function is not affected by the outliers present in the dataset. For multitask objectives, both the cost function for classification and prediction needs to be backpropagated to strengthen the connection. Cross-entropy function Lce and L1 loss function are expressed in the following equations:(4)Lce=−γ1N∑i=1NzilogpXi,(5)L1=1−γ1N∑i=1NXi−X^i.A schematic diagram of the backpropagation technique is depicted in Figure5. The working of the backpropagation algorithm is discussed henceforth. The full training is completed in batches in multiple iterations wherein each iteration is known as an epoch. The training instances are fed to the input layer that integrates the required parameters with each instance and fires it to the initial hidden layer as per activation. This forward passing of the feature vectors with required bias, weight, and activation function from one input layer to the next or a series of hidden layers and eventually to the output layer to generate the target output value is forward propagation. This phenomenon is also known as forwarding pass. The loss function is used to calculate the error between the predicted output value and the labelled output. For optimizing the value for better performance and generalization of the model, backward propagation or backward pass is applied during the training. The algorithm analyses and computes how much each connection weight and each bias parameter can be altered for error reduction by applying the chain rule which helps the backpropagation step fasten accurately. Backpropagation minimizes the cost function by fine-tuning the network’s weights and biases. For adjusting this parameter, the gradients of the cost function with respect to weights and bias need to be computed using the chain rule given in the following equation [61], where ni is the ith node in the network.(6)∂f∂x=∂f∂ni+∂ni∂x.Figure 5
Illustration of backpropagation and optimization processes in a neural network.


### 5.1.3. GeLU Activation Function

The initial perceptron used binary threshold units [67]. The binary decision was further flattened using sigmoid activation functions and trained with backpropagation [68]. As the network grew deeper and deeper, the sigmoid function became less effective due to nonlinearity [61]. ReLU succeeded over the sigmoid which is preferred mostly due to its fast and better convergence property. Then, the eLU activation function which is a modification of ReLU was developed which improved the training speed on negative values [69]. The prediction of the deterministic decision during the evaluation of the network leads to new nonlinearity. The stochastic regularizer yields nonlinearity for an input x that randomly applies the identity or zero maps to a neuron’s input. The Gaussian error linear unit (GeLU) is a high-performing neural network activation function [70]. Unlike signs in ReLU, the inputs are weighted in magnitude by the GeLU nonlinearity. The GeLU activation function is xΦx, where Φx is the standard Gaussian cumulative distribution function. The GeLU function expressed using equation (7) is estimated using equation (8).(7)GeLUX=xPX≤x=xΦx,(8)GeLUX=0.5x1+tanh2πx+0.044715x3.


### 5.1.4. Deep Feature

In this step, the handcrafted feature vector for a given inputX=x1,x2,…,xT is fed to the DNN obtained in Section 5.1. Then, the output from the particular hidden layer (without applying the activation function) is extracted as a deep feature (for the particular input). It can be expressed as(9)XDF−h=fFF−DNNX,where XDF−h denotes the deep feature from the hth hidden layer of the DNN for input X. This basically represents the mapping of the input data from one space to another. In this manner, deep features are extracted for all input data in the experiments. In the case of 512 neurons, per hidden layer in DNN yields the 512 dimension deep feature vector. Besides, we also consider the study of the concatenation of deep features extracted from the different hidden layers as a presentation. For example, DF1−5 indicates the concatenated deep feature of the hidden layers 1 to 5. If each hidden layer has 512 neurons, then DF1−5 gives 5×512 dimensional vector.The deep feature extracted from each hidden layer of the FFDNN has better representation and generalization. In the final step, deep features extracted from each hidden layer are then used layer-wise and in concatenation for the classification of malware using RF, SVM, and KNN classifiers. The deep feature extracted from each hidden layer of the FFDNN constitutes a malware classification system.


## 5.1.1. Perceptron

Perceptron is the simplest and basic ANN architecture. Rosenblatt [65] proposed the perceptron training algorithm that was inspired by Hebb’s rule [66]. According to Donald Hebb, when a human neuron activates another neuron, bonding between the neurons becomes more and more strong. This ANN accepts one data point at once to make a prediction and uses the perceptron learning rule (PLR) to minimize error for any incorrect prediction during training. The PLR is presented in equation (3). One training instance is fed at a time to perceptrons to make predictions. For a wrong prediction of an output neuron, the perceptron learning rule strengthens the connections for the reduction of the error generated. This reinforcement rule contributes toward the correct prediction. The PLR is presented in the following equation:(3)wi,jnext=wi,j+ηyj−y^jxi,where wi,j is weight connecting the ith neuron and jth neuron, xi is the ith instance for training, yj is the output of the jth neuron, y^j is the predicted output value of the jth neuron to generate output, and η is the learning rate used during PLR. This basic ANN model is unable to learn complex or nonlinear patterns for decision-making as the plane separating each class is linear. The limitation of the perceptron can be covered by using multilayer perceptron (MLP) ANN architecture. The MLP with various modifications is being used in deep learning research. Fundamentally, it is organised with an input layer, an output layer, and one or more hidden layers existing between the input and output layers. An MLP with all-to-one mapping between the neurons of subsequent layers is known as a fully connected neural network (FCNN) in deep learning as shown in Figure 4. An MLP with a deeper stack than one hidden layer is a deep neural network. The decision boundary for each output neuron or class is linear, so this basic perceptron model is unable to learn complex or nonlinear patterns for decision-making. The limitation of perceptrons can be covered by using multilayer perceptron (MLP) ANN architecture. The MLP with various modifications is being used in deep learning research. An MLP is organized with one input layer, one or more layers of hidden layers, and one final layer called the output layer. The MLP with all-to-one mapping between the neurons of subsequent layers is known as a fully connected FFNN in deep learning and is depicted in Figure 4. All layers excluding the output layer consisting of a bias neuron are fully connected to the next layer. An MLP with a deeper stack than one hidden layer is a deep neural network.Figure 4
Illustration of an FFDNN network architecture with input, hidden layer, neurons, and output.


## 5.1.2. Backpropagation

During the network training, the output either for a single or multiobjective task is generated in the forward pass. This generated output value may be different from the original value. So, the backpropagation of an error signal is evaluated to minimize the cost function. For this proposed work, cross-entropy functionLce and L1 loss function are used to evaluate the classification probability and minimize the reconstruction error, respectively. The reason behind applying L1 loss for this work is that the data have been used without preprocessing which reduces feature preprocessing overhead in deep learning. So, the data may contain outliers, and the L1 loss function is not affected by the outliers present in the dataset. For multitask objectives, both the cost function for classification and prediction needs to be backpropagated to strengthen the connection. Cross-entropy function Lce and L1 loss function are expressed in the following equations:(4)Lce=−γ1N∑i=1NzilogpXi,(5)L1=1−γ1N∑i=1NXi−X^i.A schematic diagram of the backpropagation technique is depicted in Figure5. The working of the backpropagation algorithm is discussed henceforth. The full training is completed in batches in multiple iterations wherein each iteration is known as an epoch. The training instances are fed to the input layer that integrates the required parameters with each instance and fires it to the initial hidden layer as per activation. This forward passing of the feature vectors with required bias, weight, and activation function from one input layer to the next or a series of hidden layers and eventually to the output layer to generate the target output value is forward propagation. This phenomenon is also known as forwarding pass. The loss function is used to calculate the error between the predicted output value and the labelled output. For optimizing the value for better performance and generalization of the model, backward propagation or backward pass is applied during the training. The algorithm analyses and computes how much each connection weight and each bias parameter can be altered for error reduction by applying the chain rule which helps the backpropagation step fasten accurately. Backpropagation minimizes the cost function by fine-tuning the network’s weights and biases. For adjusting this parameter, the gradients of the cost function with respect to weights and bias need to be computed using the chain rule given in the following equation [61], where ni is the ith node in the network.(6)∂f∂x=∂f∂ni+∂ni∂x.Figure 5
Illustration of backpropagation and optimization processes in a neural network.


## 5.1.3. GeLU Activation Function

The initial perceptron used binary threshold units [67]. The binary decision was further flattened using sigmoid activation functions and trained with backpropagation [68]. As the network grew deeper and deeper, the sigmoid function became less effective due to nonlinearity [61]. ReLU succeeded over the sigmoid which is preferred mostly due to its fast and better convergence property. Then, the eLU activation function which is a modification of ReLU was developed which improved the training speed on negative values [69]. The prediction of the deterministic decision during the evaluation of the network leads to new nonlinearity. The stochastic regularizer yields nonlinearity for an input x that randomly applies the identity or zero maps to a neuron’s input. The Gaussian error linear unit (GeLU) is a high-performing neural network activation function [70]. Unlike signs in ReLU, the inputs are weighted in magnitude by the GeLU nonlinearity. The GeLU activation function is xΦx, where Φx is the standard Gaussian cumulative distribution function. The GeLU function expressed using equation (7) is estimated using equation (8).(7)GeLUX=xPX≤x=xΦx,(8)GeLUX=0.5x1+tanh2πx+0.044715x3.


## 5.1.4. Deep Feature

In this step, the handcrafted feature vector for a given inputX=x1,x2,…,xT is fed to the DNN obtained in Section 5.1. Then, the output from the particular hidden layer (without applying the activation function) is extracted as a deep feature (for the particular input). It can be expressed as(9)XDF−h=fFF−DNNX,where XDF−h denotes the deep feature from the hth hidden layer of the DNN for input X. This basically represents the mapping of the input data from one space to another. In this manner, deep features are extracted for all input data in the experiments. In the case of 512 neurons, per hidden layer in DNN yields the 512 dimension deep feature vector. Besides, we also consider the study of the concatenation of deep features extracted from the different hidden layers as a presentation. For example, DF1−5 indicates the concatenated deep feature of the hidden layers 1 to 5. If each hidden layer has 512 neurons, then DF1−5 gives 5×512 dimensional vector.The deep feature extracted from each hidden layer of the FFDNN has better representation and generalization. In the final step, deep features extracted from each hidden layer are then used layer-wise and in concatenation for the classification of malware using RF, SVM, and KNN classifiers. The deep feature extracted from each hidden layer of the FFDNN constitutes a malware classification system.


## 6. Experimental Setup

Experiments are conducted on the dataset [71], and details are presented in Section 6.1. The datasets are randomly split into n=10 folds. At a time, one set is considered for evaluation, and the rest are used for evaluation, i.e., one split is leave out at a time. The process is continued until all the data split evaluation is completed. For the FFDNN, the number of hidden layers (3,10,15, and 20), the number of neurons (32,64,128,256, and 512) per layer, and different activation functions (GeLU, ReLU, eLU, and SeLU) are varied to study for the analysis of the system performance. The value of the drop-out rate, regularization parameter, number of epochs, learning rate, and batch size are considered, respectively, 0.01, 0.0001, 30, 0.001, and 1024. The framework of the proposed DNN and ML classifier-integrated static malware detection model is shown in Figure 3. In phase-I, the multitask objective is implemented using a feed-forward neural network (FFNN) with backpropagation. The objectives to be obtained are the prediction and classification of malware and benign sample. The different training network is designed with 3 to 20 hidden layers excluding input and output layers for evaluation. The number of neurons or units in each hidden layer is set from n = 25 to 29 to verify the performance enhancement and robustness of the training network. During the implementation of each network training session, the number of neurons is increased to evaluate the classification error and reconstruction error. The output of the network is generated by the softmax layer implementing the softmax function. Feature dimension vectors generated from each hidden layer are concatenated after the last hidden layer output generation and used to train the DNN and ML classifiers. These feature vectors after being generated from the respective hidden layer are passed on for activation by applying the activation function. Popular DNN activation functions such as ReLU, SeLU, eLU, and GeLU have been investigated in combination with the traditional DNN approach with neurons n = 25 to 29 to choose the best performing one. GeLU is identified as a suitable and high-performance activation function for static malware detection. The concatenated features are then fed as input to the softmax layer for evaluation. The loss function, L1, is applied for the calculation of prediction loss and the difference between the predicted output value and the actual output value or label value. The reason for applying the L1 loss function is that no previous data preprocessing is required for the data. For classification, the cross-entropy function is applied. After obtaining the error, the network is trained using the backpropagation algorithm for optimizing the weight and bias to improve the reconstruction loss. The Adam optimizer is used as an optimizer for faster convergence.


### 6.1. Dataset

The dataset used in this work consists of the metadata from the portable executable header. The sources of the dataset are discussed in [71]. Each feature value has its own function regarding the type of file. The malware dataset consisting of 138048 instances and 54 features can be represented in a 138048×54 matrix, as X∈R138048×54 presented in matrix 11.(10)X=x11x21…x541x12x22…x542⋮⋮…⋮x1138048x2138048…x54138048.The dataset used in this work consists of 138048 data points and 54 dimensions. These features are extracted from the header of the PEH of malware and benign software for the Windows operating system. 41324 benign executables have been extracted from .exe and .dll files of the Windows operating system. 96,724 malware executables have been extracted from data available on Virus Share. The malware and benign samples are labelled as “0” and “1,” respectively, after extraction. The features can be extracted statically from the PEH using the pefile module in python. The values in the header can be viewed using tools such as CFF-Explorer.


### 6.2. DNN Classification Task

Different classification components such as the softmax function, regression classifier, and cost function have been expressed in [61]. For a given instance x, the softmax classification model initially computes the score skx for each class k given in equation (11). Then, softmax function is applied to the initial scores to estimate the probability for each class, where θk is the parameter vector of each class.(11)skx=XTθk.The probability of classk and p^k is computed by applying the scores to the softmax function given in equation (12). The exponential of each score is computed, and then normalization is applied to the function. The calculated scores are termed as logits.(12)p^k=σsxk,=expskx∑j=1Kexpsjx,where k is the total number of classes, sx is a vector with the scores of each class for the instance x, and σsxk is the estimated probability that x belongs to class k. The class with the high estimated probability that is the class with the highest score is predicted using the softmax regression classifier expressed in the following equation:(13)y^=argmaxσsxk.To generate a deep learning model that can calculate both high and low probability for each class, minimized cost function is required. Cross-entropy as a cost function given in equation (14) can be used to match the estimated class probabilities to the target class. Here, the objective is to generate a deep model that estimates a high and low probability for each target class, respectively. In order to achieve this objective, minimizing the cost function is required. Cross-entropy as a cost function given in equation (14) can be used to match the estimated class probabilities to the target class.(14)JΘ=−1m∑i=1m∑k=1Kykilogp^ki,where yki is the target probability and ith instance belongs to class k. The gradient vector of the cross-entropy cost function with respect to θk is expressed in the following equation:(15)∇θkJΘ=1m∑i=1mp^ki−y^kixi.Once the gradient vector is computed for each class, the Adam optimizer is used to find the parameterΘ to minimize the required cost function.


### 6.3. Performance Enhancement

The training of the DNN with a large number of hidden layers can be an exhaustive task. To speed up the training process that aids in performance enhancement, certain techniques have been used during the implementation of the proposed training method.


#### 6.3.1. Adam Optimizer

Optimization is a crucial part of deep network training and learning. Optimization in deep learning is basically applied for minimization of the cost function by modifying the model parameters such as weight and bias for better convergence. Initially, the model parameters are initialized randomly. To boost the speed of training the network, a faster optimizer such as Adam [72] which stands for adaptive moment estimation has been applied for the optimization in the proposed work. Adam is a stochastic, robust optimizer and adaptive learning rate algorithm with less memory requirement. It requires less tuning of the learning rate hyperparameter. Adam is often performing well in adaptive learning and outperforms other adaptive techniques such as the stochastic gradient descent (SGD) optimizer.


#### 6.3.2. Learning Rateη

The learning rateη is an important hyperparameter of optimization used for fast convergence. The learning rate determines how much each epoch model parameter should be updated according to the gradient of the loss function. The decision to choose a proper learning rate can be tedious. An important parameter in optimization is the step size, determined by the learning rate hyperparameter. If the learning rate is too small, then the algorithm requires many epochs for converging, thus taking a long time. For a higher learning rate, the steps will be bouncing across the curve with larger values making the algorithm diverge, thus failing to find an optimal solution, as presented in Figure 6.Figure 6
Illustration of the effect of the learning rate on the loss and number of epochs [64].


### 6.4. Preventing Overfitting

Overfitting is a common problem, where a model performs perfectly on training data but does not generalize well to untrained data or test data. To prevent overfitting during evaluation, methods such as batch normalization,l2 regularization, and dropout have been applied during DNN training.


#### 6.4.1. Batch Normalization

When the deep model is trained, the distribution of hidden layers gets altered as there is an update in the parameters of the previous layers. There is a regular adjustment in the distribution in the respective layers. This adjustment in distribution is known as an internal covariate shift. This problem is solved by a technique called batch normalization. In this technique, the input is normalized for every mini-batch and then the normalized weights are trained. The advantage of this technique is that even higher learning rates can be applied and weight initialization becomes less significant. During the training of DNN, the distribution of each input layer gets modified as the parameters of the previous layers change. This results in slowing down the training that demands lower learning rates and careful parameter initialization. Batch normalization solves this problem by normalizing the input for every mini-batch. Batch normalization is a popular method to prevent overfitting that normalizes the layers and trains the normalized weights. This allows us to use much higher learning rates and be less careful about initialization. It can be applied to any layer in the network.


#### 6.4.2. Regularization

A model that suffers from overfitting has a high variance, due to having many parameters, which results in the generation of a complex model. Similarly, the model can also suffer from underfitting or high bias, which results in generating a highly simple model that is incapable of learning the patterns well in the training data and performs poorly on unseen data. Regularization can be a better approach to tune to the model complexity for a good bias and variance [61]. Regularization is a very useful method that tackles high correlation among features, filters out noise from data, and finally prevents overfitting. The network is regularized by adding constraints to the parameters such as imposing a squared penalty on the weights such that the higher the weight so is the penalty called l2 regularization. It is discussed in [73] and expressed in the following equation, where λ is the regularization parameter.(16)λ2w2=λ2∑j=1mwj2.


### 6.5. Deep Feature Classification

The features extracted from the training of hidden layers of the deep model are called deep features. These deep features are then projected for classification to the classifiers such as random forest, support vector machine, andK-nearest neighbour with various parameters for performance evaluation. The features derived from each hidden layer or in the concatenation of features extracted from hidden layers can be fed to the classifiers. This feature can be more robust as it is generated during potent DNN training [74]. Also, it can generalize to large and untrained data.


## 6.1. Dataset

The dataset used in this work consists of the metadata from the portable executable header. The sources of the dataset are discussed in [71]. Each feature value has its own function regarding the type of file. The malware dataset consisting of 138048 instances and 54 features can be represented in a 138048×54 matrix, as X∈R138048×54 presented in matrix 11.(10)X=x11x21…x541x12x22…x542⋮⋮…⋮x1138048x2138048…x54138048.The dataset used in this work consists of 138048 data points and 54 dimensions. These features are extracted from the header of the PEH of malware and benign software for the Windows operating system. 41324 benign executables have been extracted from .exe and .dll files of the Windows operating system. 96,724 malware executables have been extracted from data available on Virus Share. The malware and benign samples are labelled as “0” and “1,” respectively, after extraction. The features can be extracted statically from the PEH using the pefile module in python. The values in the header can be viewed using tools such as CFF-Explorer.


## 6.2. DNN Classification Task

Different classification components such as the softmax function, regression classifier, and cost function have been expressed in [61]. For a given instance x, the softmax classification model initially computes the score skx for each class k given in equation (11). Then, softmax function is applied to the initial scores to estimate the probability for each class, where θk is the parameter vector of each class.(11)skx=XTθk.The probability of classk and p^k is computed by applying the scores to the softmax function given in equation (12). The exponential of each score is computed, and then normalization is applied to the function. The calculated scores are termed as logits.(12)p^k=σsxk,=expskx∑j=1Kexpsjx,where k is the total number of classes, sx is a vector with the scores of each class for the instance x, and σsxk is the estimated probability that x belongs to class k. The class with the high estimated probability that is the class with the highest score is predicted using the softmax regression classifier expressed in the following equation:(13)y^=argmaxσsxk.To generate a deep learning model that can calculate both high and low probability for each class, minimized cost function is required. Cross-entropy as a cost function given in equation (14) can be used to match the estimated class probabilities to the target class. Here, the objective is to generate a deep model that estimates a high and low probability for each target class, respectively. In order to achieve this objective, minimizing the cost function is required. Cross-entropy as a cost function given in equation (14) can be used to match the estimated class probabilities to the target class.(14)JΘ=−1m∑i=1m∑k=1Kykilogp^ki,where yki is the target probability and ith instance belongs to class k. The gradient vector of the cross-entropy cost function with respect to θk is expressed in the following equation:(15)∇θkJΘ=1m∑i=1mp^ki−y^kixi.Once the gradient vector is computed for each class, the Adam optimizer is used to find the parameterΘ to minimize the required cost function.


## 6.3. Performance Enhancement

The training of the DNN with a large number of hidden layers can be an exhaustive task. To speed up the training process that aids in performance enhancement, certain techniques have been used during the implementation of the proposed training method.


### 6.3.1. Adam Optimizer

Optimization is a crucial part of deep network training and learning. Optimization in deep learning is basically applied for minimization of the cost function by modifying the model parameters such as weight and bias for better convergence. Initially, the model parameters are initialized randomly. To boost the speed of training the network, a faster optimizer such as Adam [72] which stands for adaptive moment estimation has been applied for the optimization in the proposed work. Adam is a stochastic, robust optimizer and adaptive learning rate algorithm with less memory requirement. It requires less tuning of the learning rate hyperparameter. Adam is often performing well in adaptive learning and outperforms other adaptive techniques such as the stochastic gradient descent (SGD) optimizer.


### 6.3.2. Learning Rateη

The learning rateη is an important hyperparameter of optimization used for fast convergence. The learning rate determines how much each epoch model parameter should be updated according to the gradient of the loss function. The decision to choose a proper learning rate can be tedious. An important parameter in optimization is the step size, determined by the learning rate hyperparameter. If the learning rate is too small, then the algorithm requires many epochs for converging, thus taking a long time. For a higher learning rate, the steps will be bouncing across the curve with larger values making the algorithm diverge, thus failing to find an optimal solution, as presented in Figure 6.Figure 6
Illustration of the effect of the learning rate on the loss and number of epochs [64].


## 6.3.1. Adam Optimizer

Optimization is a crucial part of deep network training and learning. Optimization in deep learning is basically applied for minimization of the cost function by modifying the model parameters such as weight and bias for better convergence. Initially, the model parameters are initialized randomly. To boost the speed of training the network, a faster optimizer such as Adam [72] which stands for adaptive moment estimation has been applied for the optimization in the proposed work. Adam is a stochastic, robust optimizer and adaptive learning rate algorithm with less memory requirement. It requires less tuning of the learning rate hyperparameter. Adam is often performing well in adaptive learning and outperforms other adaptive techniques such as the stochastic gradient descent (SGD) optimizer.


## 6.3.2. Learning Rateη

The learning rateη is an important hyperparameter of optimization used for fast convergence. The learning rate determines how much each epoch model parameter should be updated according to the gradient of the loss function. The decision to choose a proper learning rate can be tedious. An important parameter in optimization is the step size, determined by the learning rate hyperparameter. If the learning rate is too small, then the algorithm requires many epochs for converging, thus taking a long time. For a higher learning rate, the steps will be bouncing across the curve with larger values making the algorithm diverge, thus failing to find an optimal solution, as presented in Figure 6.Figure 6
Illustration of the effect of the learning rate on the loss and number of epochs [64].


## 6.4. Preventing Overfitting

Overfitting is a common problem, where a model performs perfectly on training data but does not generalize well to untrained data or test data. To prevent overfitting during evaluation, methods such as batch normalization,l2 regularization, and dropout have been applied during DNN training.


### 6.4.1. Batch Normalization

When the deep model is trained, the distribution of hidden layers gets altered as there is an update in the parameters of the previous layers. There is a regular adjustment in the distribution in the respective layers. This adjustment in distribution is known as an internal covariate shift. This problem is solved by a technique called batch normalization. In this technique, the input is normalized for every mini-batch and then the normalized weights are trained. The advantage of this technique is that even higher learning rates can be applied and weight initialization becomes less significant. During the training of DNN, the distribution of each input layer gets modified as the parameters of the previous layers change. This results in slowing down the training that demands lower learning rates and careful parameter initialization. Batch normalization solves this problem by normalizing the input for every mini-batch. Batch normalization is a popular method to prevent overfitting that normalizes the layers and trains the normalized weights. This allows us to use much higher learning rates and be less careful about initialization. It can be applied to any layer in the network.


### 6.4.2. Regularization

A model that suffers from overfitting has a high variance, due to having many parameters, which results in the generation of a complex model. Similarly, the model can also suffer from underfitting or high bias, which results in generating a highly simple model that is incapable of learning the patterns well in the training data and performs poorly on unseen data. Regularization can be a better approach to tune to the model complexity for a good bias and variance [61]. Regularization is a very useful method that tackles high correlation among features, filters out noise from data, and finally prevents overfitting. The network is regularized by adding constraints to the parameters such as imposing a squared penalty on the weights such that the higher the weight so is the penalty called l2 regularization. It is discussed in [73] and expressed in the following equation, where λ is the regularization parameter.(16)λ2w2=λ2∑j=1mwj2.


## 6.4.1. Batch Normalization

When the deep model is trained, the distribution of hidden layers gets altered as there is an update in the parameters of the previous layers. There is a regular adjustment in the distribution in the respective layers. This adjustment in distribution is known as an internal covariate shift. This problem is solved by a technique called batch normalization. In this technique, the input is normalized for every mini-batch and then the normalized weights are trained. The advantage of this technique is that even higher learning rates can be applied and weight initialization becomes less significant. During the training of DNN, the distribution of each input layer gets modified as the parameters of the previous layers change. This results in slowing down the training that demands lower learning rates and careful parameter initialization. Batch normalization solves this problem by normalizing the input for every mini-batch. Batch normalization is a popular method to prevent overfitting that normalizes the layers and trains the normalized weights. This allows us to use much higher learning rates and be less careful about initialization. It can be applied to any layer in the network.


## 6.4.2. Regularization

A model that suffers from overfitting has a high variance, due to having many parameters, which results in the generation of a complex model. Similarly, the model can also suffer from underfitting or high bias, which results in generating a highly simple model that is incapable of learning the patterns well in the training data and performs poorly on unseen data. Regularization can be a better approach to tune to the model complexity for a good bias and variance [61]. Regularization is a very useful method that tackles high correlation among features, filters out noise from data, and finally prevents overfitting. The network is regularized by adding constraints to the parameters such as imposing a squared penalty on the weights such that the higher the weight so is the penalty called l2 regularization. It is discussed in [73] and expressed in the following equation, where λ is the regularization parameter.(16)λ2w2=λ2∑j=1mwj2.


## 6.5. Deep Feature Classification

The features extracted from the training of hidden layers of the deep model are called deep features. These deep features are then projected for classification to the classifiers such as random forest, support vector machine, andK-nearest neighbour with various parameters for performance evaluation. The features derived from each hidden layer or in the concatenation of features extracted from hidden layers can be fed to the classifiers. This feature can be more robust as it is generated during potent DNN training [74]. Also, it can generalize to large and untrained data.


## 7. Experimentation and Result Analysis

In this work, we have used the publicly available database for the system evaluation. The link to the dataset is as follows [71]. For unbiased experimentation, we consider the N-fold leave-one-out criterion for the system evaluation. It is well known that leave-one-out is an unbiased approach for system evaluation. In addition, it consists of ≈139 k trials which are statistically significant. Therefore, the study of the technique on the other databases is kept for the future direction. The primary investigation of classification starts with the training of three popular ML classifiers, RF, SVM, and k-NN. Also, different packages of SVM and k-NN named in this paper as SVM-1, SVM-2, and SVM-3 and k-NN1 with different numbers of nearest neighbours have been trained and validated.SVM-1 uses the linear kernel, along with regularization parameter (RP) 1, a maximum number of iterations as 10000, and automatic selection of gamma. SVM-2 is similar to SVM-1 but has higher tolerance. SVM-3 uses radial basis function (RBF), with a similar RP and a maximum number of iterations of 10000 but with gamma fixed to 0.7. The accuracy of these classifiers extracted exclusively is depicted in Figure7. It is clear in Figure 7 that the ML classifiers alone are not sufficient to deal with a large volume of data. The highest accuracy of 91.16% is exhibited by the RF classifier. The performance of SVM-1 and SVM-2 is very poor as they use a linear kernel. The dataset is large and definitely nonlinear in nature. SVM-3, which uses the RBF kernel, has a better accuracy of 85.88%. The k-NN has better performance for a lower number of nearest neighbours, but it is still not up to the mark. Also, a lower number of nearest neighbours make the classifier biased and may have poor variance.Figure 7
Comparison of the performance of malware detection in terms of accuracy with RF, SVM, and k-NN ML classifiers for different parameters.In the later stage, a 20-layer FFDNN with different activation functions is trained and its deep features are extracted to train the FFDNN as well as ML classifiers. The investigation is performed using different activation functions, SeLU, eLU, ReLU, and GeLU, of the DNN. The objective of this investigation is to identify the best performing activation function for the malware detection application. The obtained accuracy of the DNN with different activation functions is presented in Table1. Here, the DNN is used in a traditional way and functions as a black box. The hidden layer features are unexplored and have no direct intervention in the final results. As discussed in the earlier sections, GeLU is a more high-performance activation function than the other one, thus achieving the best accuracy of 98.17% among all. It is understood that GeLU performs well with a small number of neurons. The performance of other activation functions is reasonably good for a larger number of neurons. ReLU exhibits its best accuracy for 128 neurons and eLU and SeLU for 64 neurons. In the next step, the effectiveness of GeLU is examined against the increasing number of the DNN hidden layers and the number of neurons, as presented in Table 2.Table 1
Accuracy of DNN with 3 hidden layers HL for different numbers of neurons and activation functions.
No. of neuronsGeLUReLUeLUSeLU3298.1797.8597.8397.26498.1597.8197.8897.2212898.1397.8997.7897.0325697.9597.7297.6596.7851297.4297.5197.3797.26Bold values indicate the peak accuracy achieved using the corresponding activation function.Table 2
Accuracy of DNN with GeLU activation function for different numbers of HL and neurons.
HLN=32N=64N=128N=256N=512398.1798.1598.1397.9597.421098.1598.15698.1798.2398.211598.1998.1698.1898.2198.142098.2098.2298.1598.2098.14Bold values indicate the peak accuracy achieved for the corresponding numbers of neurons.It is observed that GeLU exhibits better accuracy with a less number of neurons but a larger number of hidden layers or a small number of hidden layers but a large number of neurons in an FFDNN. In both cases, the obtained accuracy has a marginal difference. For the 32 neurons and 20 hidden layers, the accuracy is 98.20%, whereas for 256 neurons and 10 hidden layers, the accuracy is 98.21%. So, either case may be used for a high-performance classification. From the previous observations, the GeLU activation function has been used and investigated for either case of the proposed model.The training and validation loss of the DNN for 256 neurons, 10 deep hidden layers, and 11 splits is shown in Figure8. The training loss and validation loss in the graph represent how perfectly the model fits into the trained and untrained data, respectively. As the dataset is larger, the data are separated into the train, validate, and test sets. From the graph, it can be observed that there is a minimal gap between training loss and validation loss in each split. Also, the loss is getting smaller with the number of epochs and converging fast.Figure 8
Illustration of the training and validation losses over a number of epochs on different folds of data for an FFDNN consisting of 10 hidden layers and 256 neurons/layers: (a) splits 1 and 2; (b) splits 4 and 5; (c) splits 7 and 8; (d) splits 9 and 10.
(a)(b)(c)(d)Proceeding to the experimentation, the proposed model with a different number of hidden layers, a number of neurons, and a number of splits has been created and investigated. At one time, each hidden layer in the network is trained using neuronsn = 25 to 29 and increasing the number of deep layers from 3 to 20 subsequently. In the first deep model training, 3 hidden layers with 11 splits and 32 neurons in each layer have been examined. Due to the huge volume of data and with an objective to improve performance, the training data are randomly distributed and split into batches of 1024 data points. Each batch is executed in 30 epochs during the training to generate scores later used for the calculation of the reconstruction error and classification probability. 10-fold cross-validation is applied to the data points to validate the training. L1 loss and cross-entropy loss functions are used to calculate reconstruction loss and classification loss. The Adam optimizer is used for weight optimization during backpropagation for faster convergence with a learning rate η = 0.001. To prevent overfitting, the encoding parameters such as the L2 regularizer and he-initializer have been used for the network training. However, a few conclusive results are presented in Tables 3–6. In Table 3, the results are presented for the 3-layer 256 neurons with 11 splits. The accuracy of this model is reasonable with the highest accuracy of 98.78% for the k-NN with 3 nearest neighbours. As discussed earlier, k-NN with a lower neighbour count may be biased, and a further investigation is carried out. Although, it is observed that the RF and SVM classifiers performed well with a higher number of deep hidden layer features or concatenated features of all layers. The k-NN exhibits better performance for a lower number of hidden layer features. The maximum achieved accuracy of all models is appreciable and above 98%. The results of the second model presented in Table 4 have the same number of neurons and splits but 10 hidden layers. Among all models, the best accuracy of 99.15% is achieved by SVM-3 in concatenation form with 256 neurons and 11 splits. The obtained result is definitely remarkable for such a large dataset. The performance of k-NN is also considerable for a lower number of deep layers. The k-NN achieves the highest accuracy of 98.98% for the independent third hidden layer. Similarly, the results of the proposed model with 15 and 20 layers have similar observations. The performance of some models is inconclusive which is natural for artificial inelegance algorithms. For example, SVM-1 exhibits an accuracy of 98.16% using the independent 14th layer feature of the 20-layer model. From the previously mentioned observations, it is sure that the RF and SVM ML classifiers with bioinspired DNN GeLU activation functions are a potential solution to the current regime for malware detection and prevention.Table 3
Accuracy of ML classifiers against individual and concatenated 3-HLs deep features for 256 neurons and 11 splits.
FeaturesRFSVMk-NNSVM1SVM2SVM3K-NN1K-NN2K-NN3K-NN4K-NN5K-NN6DF196.3856.3496.7798.3498.7798.5498.7898.7298.7598.68DF296.3596.9997.398.398.7498.5298.7698.7198.7698.69DF397.3598.0998.0298.2298.6198.4198.6898.6198.6798.62DF (1–3)98.1696.7298.0998.3998.7398.5298.7798.7198.7598.67The bold values indicate the peak accuracy achieved using the corresponding Machine Learning classifier.Table 4
Accuracy of ML classifiers against individual and concatenated 10-HLs deep features for 256 neurons and 11 splits.
FeaturesRFSVMk-NNSVM1SVM2SVM3K-NN1K-NN2K-NN3K-NN4K-NN5K-NN6DF196.3256.2996.7798.3398.7898.6498.2498.6198.6198.53DF296.3697.0397.3198.1998.2998.8398.8298.6698.6798.47DF397.6397.1797.0298.198.3898.5098.7498.7298.8198.98DF497.0697.5597.0998.3898.1798.1398.6998.5198.8798.24DF597.6097.6797.2398.1598.4198.1798.4498.8798.5598.65DF697.7197.6997.3098.9297.9498.2698.2898.1898.1598.24DF797.6197.8197.4298.1597.6298.0798.5298.2198.2598.16DF897.5098.8798.3198.7797.3497.3698.0398.3598.0398.11DF997.7298.1897.8398.9497.2597.4898.4398.0998.1498.06DF1097.6498.3297.2598.8897.5197.0698.4198.1997.9198.23DF (1–10)97.9598.5698.3799.1597.8397.5098.1098.0598.0898.20The bold values indicate the peak accuracy achieved using the corresponding machine learning classifier.Table 5
Accuracy of ML classifiers against individual and concatenated 15-HLs deep features for 256 neurons and 11 splits.
FeaturesRFSVMk-NNSVM1SVM2SVM3K-NN1K-NN2K-NN3K-NN4K-NN5K-NN6DF197.6348.6296.1196.1498.1297.9698.1698.1798.0197.95DF296.2696.1795.4696.4998.4597.2198.0398.1098.3798.34DF396.1296.2696.3396.7698.6297.9098.2198.5798.4798.71DF496.2196.1597.4396.7398.5598.0898.2198.0298.4598.37DF597.0897.0196.3596.2998.0498.4198.3398.3398.5798.18DF697.4596.5396.5497.0197.5097.0397.9698.2598.0298.25DF795.2496.3996.9796.1397.4997.4498.1798.2498.5397.67DF897.2696.9496.6797.3897.9397.0998.0898.1998.1598.80DF996.9896.8096.3596.8897.5997.1997.5097.5797.5797.38DF1096.9597.1796.9797.1897.6597.3897.3597.9997.5498.46DF1197.8697.8296.4096.6897.9097.1696.4497.3897.5098.36DF1296.8097.9996.3497.3397.2097.1497.9197.5198.3898.34DF1396.2996.7096.9297.3697.5397.9796.7098.1397.7297.38DF1497.6897.2097.5297.6096.9497.7797.0198.5896.2997.68DF1597.7397.6797.8398.1896.8897.0496.9497.6896.4897.99DF (1–15)97.4197.5598.1398.4096.5796.2496.8796.2197.6596.95The bold values indicate the peak accuracy achieved using the corresponding machine learning classifier.Table 6
Accuracy of ML classifiers against individual and concatenated 20-HLs deep features for 256 neurons and 11 splits.
FeaturesRFSVMk-NNSVM1SVM2SVM3K-NN1K-NN2K-NN3K-NN4K-NN5K-NN6DF196.1448.6396.1196.1498.1297.9298.1498.1698.0097.96DF296.2796.1595.4896.5298.4598.2198.4398.1398.3598.32DF396.1296.2496.3196.7698.5497.9398.1997.9798.4798.73DF496.2396.1797.4696.6998.7598.0898.2497.9798.4598.37DF597.0497.0196.3296.2598.0398.3998.3298.2998.5898.18DF697.4996.5196.5797.0397.5297.0197.9598.2398.0398.22DF795.2196.4396.9696.1697.4997.4798.1798.2098.5497.65DF897.2896.9896.6297.4097.9297.1198.0798.1698.1698.83DF997.0096.7996.3596.8997.6397.1997.5097.5997.6197.35DF1096.9397.1996.9997.1797.6197.3797.3498.0397.5698.44DF1197.8697.8196.4296.6797.8697.1696.4398.1197.5398.38DF1296.8096.9996.3797.3197.2398.1197.9297.5298.3998.32DF1396.3196.7196.9497.3497.5197.9896.7298.1197.7197.40DF1497.6798.1697.5197.5896.9497.7797.0198.5796.3097.67DF1597.7297.6897.5398.1596.9197.0196.9097.6696.4698.03DF1697.4497.5597.1498.4196.5796.2496.8696.2397.6696.96DF1797.2497.8397.0898.8397.4196.6096.8396.2297.3597.51DF1896.8397.6297.3198.0297.1196.6897.0296.8397.4097.79DF1997.6597.9897.8398.2396.3897.0296.8697.0197.0697.17DF2097.6097.7797.8898.6197.0796.1596.6397.2197.3897.36DF (1–20)97.4397.6497.9598.4797.2097.6996.5297.6897.1897.57The bold values indicate the peak accuracy achieved using the corresponding machine learning classifier.The proposed work is compared with some existing work and presented in Table7. While designing the system model, the core network, the type of feature used, activation functions, and most importantly, the feature representation have been considered, as presented in Table 7. In the literature, numerous works have been reported for malware detection using neural networks. However, different works use different types of core networks such as FFDNN [44], SAE-DBN [45, 50], DBN [46], and CNN [55–57]. Similarly, different works used different types of features, activation functions, and feature representations. In most of the work, the core learning model is being trained by a traditional feature representation such as PEH, API call, and image. Different evaluation parameters such as accuracy (Ac), detection rate (Dr), and true positive rate (TPR) have been presented. In [46], a DBN is trained using the deep feature extracted using an unsupervised dataset and a limited investigation is performed using deep features. It is clear in Table 7 that an extensive investigation is carried out in this work, and the performance achieved using deep feature representation is superior.Table 7
Comparison of the proposed work with some existing literature.
Ref. no.Core networkFeature typeActivation functionFeature representationPerformance (%)[44]FFDNNPE import, 2D string, and entropy histogramReLU and PReLUTraditional95 Dr[45]SAE-DNNWindows API callsNATraditional96.85 Ac[46]DBNOpCodeNADeep featuresApprox 98 Ac[48]DNNPEHReLUTraditional98.9 Ac[50]SAE-DBNSystem callNATraditional98.8 Ac[53]Multilevel-DNNAPI system callLeaky ReLUTraditional96 TPR[55]CNN, RNN, LSTMImage and API callReLUTraditional96.3 Ac[56]CNN-VGG16ImageNATraditional94.7 Ac[57]CNN, RFImageNA98.65 DrThis workFFDNN-MLPEHeLU, ReLU, SeLU, and GeLUConcatenated deep features99.15 Ac


### 7.1. Network Model Construction Time

This proposed work is implemented on Linux on dual boot with Windows operating System configured with Intel(R) Core(TM) i5-8250U CPU 1.60 GHz and 8.00 GB of RAM. Training of deep learning models is computationally expensive. The construction time or the training time of the FFDNN network for different layers (in hours) is depicted in Figure9. The time consumed in each scenario is observed and noted in approximation. The DNN training for classification and prediction was conducted in comparatively lesser time than the training of ML-integrated DNN classifiers. The SVM model for the deep features was trained for more than 32 hours for 3 hidden layers with neuron size 32. It can be observed that the time of model construction increases as the number of hidden layers is increased with the increasing size of neurons. However, the training of the proposed model is a one-time process and the training time can be significantly reduced due to the availability of online cloud-based graphical processing units (GPUs). A consumer may get access to these GPUs for a short period of time to train the proposed model at a minimal cost.Figure 9
Comparison of the computation (in terms of times) required for the construction of the proposed model for different system configurations (neurons and no. of hidden layers). (a) Exclusive DNN. (b) DNN-ML with 3 and 7 layers. (c) DNN-ML with 15 and 20 layers.
(a)(b)(c)


## 7.1. Network Model Construction Time

This proposed work is implemented on Linux on dual boot with Windows operating System configured with Intel(R) Core(TM) i5-8250U CPU 1.60 GHz and 8.00 GB of RAM. Training of deep learning models is computationally expensive. The construction time or the training time of the FFDNN network for different layers (in hours) is depicted in Figure9. The time consumed in each scenario is observed and noted in approximation. The DNN training for classification and prediction was conducted in comparatively lesser time than the training of ML-integrated DNN classifiers. The SVM model for the deep features was trained for more than 32 hours for 3 hidden layers with neuron size 32. It can be observed that the time of model construction increases as the number of hidden layers is increased with the increasing size of neurons. However, the training of the proposed model is a one-time process and the training time can be significantly reduced due to the availability of online cloud-based graphical processing units (GPUs). A consumer may get access to these GPUs for a short period of time to train the proposed model at a minimal cost.Figure 9
Comparison of the computation (in terms of times) required for the construction of the proposed model for different system configurations (neurons and no. of hidden layers). (a) Exclusive DNN. (b) DNN-ML with 3 and 7 layers. (c) DNN-ML with 15 and 20 layers.
(a)(b)(c)


## 8. Conclusion

We propose deep feature extraction of PEH for an FFDNN-ML malware detection system. The examinations have been carried out from scratch including training and validation of exclusive ML classifiers and FFNN. The observations of the primary investigations are highly informative, and they helped in developing the proposed bioinspired DNN-based malware detection system. The conclusive observations of the proposed work are as follows:(i)
The classification accuracy obtained using an individual hidden layer indicates that the extracted deep features of hidden layers have better representation than a raw dataset. Also, the deep features of different layers have different presentations, and by making the network deeper, a dataset can be represented in n-dimensions with better generalization.(ii)
The RF, SVM, and k-NN ML classifiers are individually insufficient to efficiently detect malware when trained using a large dataset. The RF, k-NN, and SVM-RBF have some degree of accuracy, but SVM-linear is not at all recommended.(iii)
The GeLU activation function is highly recommended even when using the DNN in a traditional way. That is, the DNN will just act as a black box and it will produce a classified value from the output layer.(iv)
The FFDNN with the GeLU activation function performs well for a large number of deep hidden layers with a less number of neurons per layer or a small number of hidden layers with a large number of neurons per layer. Either of these two cases may be used for effective classification.(v)
The proposed bioinspired DNN-based malware detection system exploits the deep hidden features and makes use of it for effective classification when trained using the dataset used in this work. In addition, the concatenation of these deep features in the proposed way has shown highly interesting results in combination with RF and SVM ML classifiers.(vi)
Deep feature extraction with complex DNN/architecture such as transformer and ResNet could be interesting to capture more attributes on the data.



---

*Source: 9544481-2023-02-20.xml*