# A Network Device Identification Method Based on Packet Temporal Features and Machine Learning
## 3 Inception-TCN-Attention Model Design
### Packet Temporal Feature Extraction  
In the temporal feature extraction stage, the input data were a multidimensional time series, and we designed an independent Inception-TCN model for each variable. The features extracted by each sub-model were multi-channel, and the number of channels was correlated with the number of channels computed by the convolution of the TCN model. After obtaining the channel features, to ensure that the features of different abstraction levels were not lost, the algorithm extracted features from each channel, which were spliced to generate a new feature matrix. This not only retained the features of the original data but also increased the dimensionality and depth of the features, thus portraying the nature of the data more comprehensively. This new feature matrix was finally used as the output of the feature extraction stage. Figure 4 shows the process of the multidimensional packet temporal series data from the original input to the formation of the feature matrix. In Figure 4, the left matrix represents the input data. In this matrix, each row corresponded to a different feature of the packet; here, it was the 27-dimensional feature after feature engineering, where each column represented the time step of the packet. The timestep corresponded to the length of the sequence, and we took the length of the packet to be 100. The Inception-TCN model was the sub-model corresponding to each variable, whose main task was to extract the features from the respective variables. The output of each sub-model was multi-channel, i.e., the output features consisted of multiple sequences that did not exist independently but were related to each other. The features extracted from multiple sequences by each sub-model were spliced together to obtain the feature matrix \(G\)  
Figure 3: The Inception-TCN-Attention model. This integrated the feature information of all the variables at different time steps, providing rich fingerprint features for the subsequent data analysis and identification of the network devices. Through this feature extraction process, not only was the key information of the data retained, but the expressiveness and interpretability of the data were enhanced. The structure of the sub-model for extracting features for each variable is given in Figure 5. The input data for each sub-model were a single sequence of features from a sequence of multidimensional packets. These sequences were first passed through the Inception structure, where convolutional results of different convolutional kernel sizes were computed to be spliced to extract information at different time scales, and then processed through a series of TCN residual blocks to progressively extract features at different levels of abstraction. The Inception structure contains four branches: the convolutional layers with convolutional kernels of 1, 3, and 5, and the branch with the largest pooling layer. The use of a 1 \(\times\) 1 convolution reduces the number of input channels, which, in turn, reduces the computation demand of the subsequent convolutional layers. This design can effectively reduce the number of parameters and computational complexity of the model, making the model more efficient. The Inception structure uses different sizes of convolution kernels and pooling layers to capture features at different scales, which helps to improve the recognition ability of the model, and multiple parallel convolutional and pooling layers can increase the nonlinearity of the model, which helps to avoid overfitting and improves  
Figure 4: The feature extraction process. Multidimensional packets from the raw input were used to generate feature matrices. Figure 5: Inception-TCN model structure. the generalizability of the model. In addition, the Inception structure can easily extend the depth and width of the model by increasing or decreasing the number and type of Inception modules, which provides flexibility for model optimization.