# Deep Learning Model with Sequential Features for Malware Classification
## 3 Malware Classification Method Based on Sequence Features and Deep Learning
### Combine TCN and BiGRU for Feature Extraction  
The advantages of the TCN model are extraction of past data by one-dimensional causal convolution to guarantee temporality, time savings via the skip connect block, extraction of temporal features by dilated convolution, and the fusion of the average pooling layer with the maximum pooling layer. The advantage of using the GRU model is its nonlinear fitting ability to efficiently extract the data features and its faster convergence speed than the LSTM model [24]. The two-way GRU model better captures the sequence features of the opcode by collecting information forward and backward, thus improving the accuracy of model classification. These two models are integrated into the TCN-BiGRU model to obtain better accuracy as well as lower loss values. The structure of the integrated model is shown in Figure 8.  
In Figure 8, the TCN-BiGRU model structure includes:  
1. Input layer: processed malicious code opcode feature data and shape (total number of samples, time step, and feature dimension).
2. Time series convolutional network layer: the feature vectors \(T_{j}\) were extracted via TCN, and the residual units were set up in two layers. A residual unit consisted of two convolutional units and one nonlinear mapping, and the convolutional kernel weights were normalized. The residual unit in Figure 8 was used only as the input layer to the hidden layer; the same was true for the hidden layer to the output layer. The convolution kernel size value was 4, and the dilation coefficient was (1, 2). Dropout was added to prevent overfitting in training.
3. The different features extracted from the average pooling layer, as well as the maximum pooling layer, were fused as pooling outputs. We merged the average with the maximum pooling layer.
4. The combined pooling layer consisted of a maximum pooling and an average pooling layer, each of which was calculated as shown in Equation (6). Maximum pooling and average pooling were obtained by traversing the pooling window with the input  
Figure 8: TCN-BiGRU model.  
from the previous layer of the network. The pooled maximum and average values were then summed and passed to the next layer of the model structure.  
\[\left\{\begin{array}{l}h_{\max}=\maxpool(h)\\ h_{avg}=avgpool(h)\\ h_{fuse}=h_{\max}\oplus h_{avg}\end{array}\right. \tag{6}\]  
where \(h\) is the input from the upper layer network into the fused pooling layer; \(h_{\max}\) is the maximum pooling output; \(h_{avg}\) is the average pooling output; and \(h_{fuse}\) is the output obtained by combining maximum pooling and average pooling in parallel.  
Bidirectional gated recurrent unit layer: The figure shows the structure of the GRU unit when it had two layers. The output vector of the TCN model was first used as the input of the GRU to extract the long-term correlation in the time series. Then the data were output with the results obtained from two layers of BiGRU.  
Output layer: Output the result of the last moment of the BiGRU to the classification layer.