# Development of Fingerprint Identification Based on Device Flow in Industrial Control System
## 4 Simulation Experiments
### Experimental Results and Performance Analysis
#### 4.3.2 Performance Analysis of Device Traffic Fingerprint Model  
The data set used in the experiment in this section is the traffic data collected from the actual industrial control system. There are thousands of industrial control devices. However, during the experiment, it is found that many devices are clearly differentiated, which is of no help in evaluating the recognition ability of the model. Therefore, this article selects ten devices on three common industrial control protocols, including Modbus protocol, s7 comm protocol and enip protocol. The traffic data collected from these devices are similar and indistinguishable, and experiments based on these devices can well evaluate the recognition ability of the model, and all experiments are conducted under the error threshold \(\tau=3\). Figure 12 describes the relationship between the recognition accuracy of each model and the number of devices in different industrial control protocols. Accuracy refers to the ratio of the number of correctly classified samples to the total number of samples. Figure 12a shows that the device traffic fingerprint model has the highest recognition accuracy. The highest accuracy rate is about 0.99, and the lowest accuracy rate is about 0.97. As the number of device fingerprints increases, the accuracy rate tends to decrease. Figure 12b,c are graphs of random forest and SVM models, respectively. Overall, the accuracy of random forest is slightly higher than that of SVM, but the accuracy of both  
Figure 11: Fragmentation prediction model boxplot. \begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Function** & **Parameter** & **Coef** & **SE Coef** & **T** & **P** \\ \hline \multirow{3}{*}{Edit Distance} & Constant & 2328.1 & 22.8 & 102.0 & 0 \\  & Length1 & 34.0 & 1.7 & 20.40 & 0 \\  & Length2 & 316.2 & 1.7 & 189.9 & 0 \\ \hline \multirow{3}{*}{Shape} & Constant & 96.8 & 29.9 & 3.23 & 0 \\  & Length1 & 64.9 & 2.2 & 29.17 & \(3^{-2}\) \\  & Length2 & 803.4 & 2.2 & 366.9 & 0 \\ \hline \multirow{3}{*}{Key Board} & Constant & \(-\)52,085 & 329.1 & \(-\)158.2 & 0 \\  & Length1 & 1982.5 & 24.1 & 82.37 & 0 \\ \cline{1-1}  & Length2 & 15,981 & 24.0 & 664.9 & 0 \\ \hline \hline \end{tabular}
\end{table} 
Table 5: Regression statistical analysis table. models drops off a cliff when the number of devices is 7. Because the device configurations in the same industrial system are similar, the newly added device has almost the same characteristics as a previous device, so the trained model recognizes both devices as one, resulting in a drop in accuracy. When the number of device fingerprints is greater than 7, the accuracy rate is slightly improved, because the recognition accuracy of the newly added device is relatively high, which reduces the impact on the identification confusion of the two devices. Figure 12d is a graph of the LSTM model. The recognition effect is relatively stable, but the accuracy is slightly worse than that of the device traffic fingerprint model. There is a large amount of noise in the actual industrial system, which will affect the recognition of the LSTM model. In Figure 12, the recognition accuracy of devices using different industrial control protocols is also different. For LSTM and device traffic fingerprinting models, the problem is pattern abstraction. Different industrial control protocols have different formats and abstract functions. The s7 comm protocol has a complex format and has obvious characteristic fields, which can accurately abstract data packets. The Modbus protocol has few feature fields, and the pattern sequences abstracted by different devices are highly similar, which will affect the recognition performance. There are obvious characteristic fields in the enip protocol, but the change rate of these fields is high, and the similarity of the pattern sequence abstracted by the same device in different time periods will be low, which will affect the recognition result.