# Unknown Traffic Recognition Based on Multi-Feature Fusion and Incremental Learning
## 3 Design of Framework
### Feature Extraction and Fusion
#### 3.2.2 Feature Selection and Fusion  
Many features are extracted from deep neural networks and may contain many redundant elements. Given that our research application is mainly focused on encrypted traffic, where there is more redundant information than normal traffic, it affects the accuracy of subsequent classifiers, so it is necessary to select the extracted deep features.  
We adopted the minimum redundancy maximum relevance (mRMR) algorithm [48] to select the features that have the least redundancy and the most impact on the target from a large number of features. It evaluates the importance of each feature by calculating the correlation of each feature with the target feature and the independence of other features. Finally, based on the evaluation results, the features are selected from high to low until the required number of features is reached. In other words, this method can effectively balance relevance and redundancy to sort the feature set. The specific calculation procedure for feature selection using the mRMR algorithm is as follows:  
In this study, we used mutual information as a criterion to measure the redundancy between features and the correlation between features and class variables, which was calculated using Equation (5), where \(X\) and \(Y\) are two different types of variables; \(p_{1}(x)\) and \(p_{2}(y)\) are the respective marginal probability distribution functions; and \(p(x,y)\) is their joint probability distribution function.  
\[I(x,y)=\sum_{x\in X}\sum_{y\in Y}p(x,y)log(\frac{p(x,y)}{p_{1}(x)\,p_{2}(y)}) \tag{5}\]Our goal is to find a feature subset \(S\) of the deep feature set \(F\). First, we initialize a set \(S\). For each feature \(F_{i}\) in \(F\), we calculate its correlation \(W\) with the target category \(c\) using mutual information \(I(F_{i},c)\), as shown in Equation (6). The redundancy is calculated using the average of the mutual information between features, as shown in Equation (7).  
\[W=\frac{1}{|F|}\sum_{F_{i}\in F}I(F_{i},c) \tag{6}\]  
\[R=\frac{1}{|F|^{2}}\sum_{F_{i},F_{j}\in F}I(F_{i},F_{j}) \tag{7}\]  
We want the correlation \(W\) between features and classes to be as large as possible and the redundancy \(R\) between features to be as small as possible. Therefore, the features can be sorted by the simple combinations of the two conditions, as indicated in Equation (8). The features that satisfy the following formula are selected and stored in the set \(S\).  
\[max\{W-R\} \tag{8}\]  
We keep repeating the above steps until the set \(S\) has the required number of elements or ends the loop when it is identical to the elements in \(F\).  
In our proposed architecture, 1000 features of each deep neural network channel are extracted as the input to the mRMR algorithm, and the number is reduced to 100 after filtering. The selected feature representations are shown in Equations (9)-(11).  
\[F^{{}^{\prime}}_{AlexNet}=\{F^{{}^{\prime}}_{1},F^{{}^{\prime}}_{2},...,F^{{} ^{\prime}}_{100}\} \tag{9}\]  
\[F^{{}^{\prime}}_{VGG16}=\{F^{{}^{\prime}}_{1},F^{{}^{\prime}}_{2},...,F^{{} ^{\prime}}_{100}\} \tag{10}\]  
\[F^{{}^{\prime}}_{LSTM}=\{F^{{}^{\prime}}_{1},F^{{}^{\prime}}_{2},...,F^{{} ^{\prime}}_{100}\} \tag{11}\]  
These selected features are next merged and stored in the feature database. The fused feature vector \(F_{Fusion}\) is a simple concatenation of the selection vectors of the three parallel channels AlexNet, VGG16 and LSTM and is represented as follows:  
\[F_{Fusion}=F^{{}^{\prime}}_{AlexNet}\oplus F^{{}^{\prime}}_{VGG16}\oplus F^{{} ^{\prime}}_{LSTM} \tag{12}\]