# Deep Feature Fusion via Transfer Learning for Multi-Class Network Intrusion Detection
## 4 Proposed Method
### Detailed Description of the Proposed Model Structure  
The feature fusion process integrates the complementary advantages of the binary classification model (Source Model #1) and the multi-class classification model (Source Model #2), both trained using transfer learning techniques in the source and target domains. Through this process, binary and multi-class classification features extracted from both models are fused, ultimately aiming for multi-class classification, including newly emerging attack types within the CIC-IDS2018 dataset. Figure 3 illustrates the detailed structure of the binary and multi-class classification models in the source domain. Source Model #1 is designed as a binary classification model for network intrusion detection, while Source Model #2 is a multi-class classification model for identifying various attack types. Both models share an identical network structure except for the classifier. Each model combines a one-dimensional convolutional neural network (1D-CNN)-based feature extraction module with a multi-head self-attention mechanism to learn patterns in network traffic data effectively. First, the 1D-CNN block sequentially extracts key features from traffic data. Then, the multi-head self-attention block is applied to assign weights to important features, enhancing learning performance. Figure 3: Detailed structure of source domain binary and multi-class classification models. Both models are trained using the CIC-IDS2017 dataset, with input data undergoing the preprocessing steps shown in Figure 1, where it is transformed into a two-dimensional array of shapes (76,1). These input data are defined as shown in Equation (2). \[X=[x_{1},x_{2},\ldots,x_{76}],x_{t}\in R^{1}, \tag{2}\]  
where \(x_{t}\) represents the t-th feature value. At the input stage of the model, the data passes through two 1D-CNN blocks, each consisting of a Conv1D layer and a MaxPooling layer. This process extracts key data features while reducing dimensionality and improving computational efficiency. The convolution and max-pooling operations performed in each 11D-CNN block \(l(l=1,2)\) are defined by Equations (3) and (4), respectively. \[h_{i}^{(l)}=ReLU(\sum_{j=0}^{k-1}W_{j}^{(l)}h_{i+j}^{(l-1)}+b^{(l)}), \tag{3}\]  
where \(ReLU\) is a function that outputs negative values as zero and positive values as they are for input values, defined as \(ReLU(x)=\max(0,x)\). \(W\) represents the weight of the convolution filter, \(h\) represents the feature value of the previous layer, and \(b\) represents the bias. \[h_{j}^{(l)}(pool)=\max(h_{2j-1}^{(l)},h_{2j}^{(l)}). \tag{4}\]  
The features extracted through the additional Conv 1D layer are then input to the multi-head attention (MA) block. This process assigns higher weights to important patterns in the input data, facilitating more effective learning. The Attention Score applied to multi-head self-attention is calculated from Equations (5)-(7) [36]. \[Q=Z\cdot W_{Q},\ K=Z\cdot W_{K},\ V=Z\cdot W_{V}. \tag{5}\]  
\[Attention(Q,\ K,\ V)=softmax\Big{(}\frac{QK^{\intercal}}{\sqrt{d_{k}}}\Big{)}V. \tag{6}\]  
\[Attention\ Score=\frac{Q\cdot K^{T}}{\sqrt{d_{k}}}. \tag{7}\]  
Here, Equation (5) represents the process of generating Query (Q), Key (K), and Value (V) by multiplying the weight matrices \(W_{Q},\ W_{K},\) and \(\ W_{V}\) for the input vector Z. That is, Z represents each input vector, Q, K, and V represent Query, Key, and Value, respectively, and \(W_{Q},\ W_{K},\ W_{V}\) are the corresponding weight matrices. Equation (6) defines the Attention function, where the dot product of Q and K is scaled by dividing it by the square root of the Key vector dimension \(d_{k}\). The result is then passed through the Softmax function, generating a probability distribution between 0 and 1, ensuring that important features receive higher attention through this distribution. Equation (7) represents the computed Attention Score, which is defined as the value obtained by dividing the dot product of Q and K by the square root of \(d_{k}\).