# Anomaly Detection Method for Unknown Protocols in a Power Plant ICS Network with Decision Tree
## 3 Results
### Test Verification
#### 3.2.1 Generator Engine Performance Verification  
The test environment was verified through several performance tests, including fingerprinting confirmation, metadata generation, metadata storage speed measurement, and metadata search speed measurement, as shown in Table 9 [40].  
Test verification was confirmed with a four-step procedure as follows.  
The verification process involved several steps, such as comparing the results of Wireshark by fingerprinting the traffic generated by Tcreplay, measuring the rate of traffic  
Figure 10: Test environment configuration.  
\begin{table}
\begin{tabular}{c c c} \hline \hline
**Seq** & **Test Item** & **Test Goal** \\ \hline
1 & Fingerprinting success rate & Measurement of OS information extraction success rate (\%) from the fingerprinting of the packets \\ \hline
2 & Metadata creation success rate & Measurement of the metadata creation success rate (\%) \\ \hline
3 & Metadata storage speed & Check the speed of saving the PCAP (Packet Capture) file with the metadata stored as a JSON (JavaScript Object Notation) file \\ \hline
4 & Metadata retrieval speed & Check the metadata retrieval speed \\ \hline \hline \end{tabular}
\end{table}
Table 9: Test items.  
and metadata generation, and checking the communication metadata fast storage test command and search query execution.  
The Telecommunications Technology Association (TTA) of the Republic of Korea certified the results of each of the five tests performed, achieving a 100% extraction success rate and 100% metadata creation success rate. For the PCAP (Packet Capture) file, the metadata creation speed was 21596.603 flows per second, and the metadata search speed was 0.0005492 ms.  
In the first experimental result, the accuracy of fingerprinting extraction was measured by comparing the number of SYN packets and SYN + ACK packets and confirming their consistency.  
As shown in Table 10, it was confirmed that the sum of TCP packets and SYN + ACK TCP packets was the same as the result of the fingerprinted packets, validating the accuracy of the fingerprinting extraction.  
In the second experimental result, metadata generation accuracy was verified by comparing and confirming the number of metadata flows, as shown in Table 11. The simulator was also verified by checking that the 636 flows generated by the source terminal were correct.  
The third experimental result measured the speed of metadata storage, and the result is shown in Table 12. The generated flows per second were divided by the evaluation time to verify the metadata storage speed.  
\begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Item** & **1** & **2** & **3** & **4** & **5** \\ \hline Processed & 1,004,099 & 1,004,105 & 1,004,097 & 1,004,099 & 1,004,100 \\ Packets & 964,819 & 964,819 & 964,819 & 964,819 & 964,819 \\ \hline Captured SYN & 27,761 & 27,761 & 27,761 & 27,761 & 27,761 \\ TCP Packets & 2030 & 2030 & 2030 & 2030 & 2030 \\ \hline Normal & 27,761 & 27,761 & 27,761 & 27,761 & 27,761 \\ Packets & 27,761 & 27,761 & 27,761 & 27,761 & 27,761 \\ \hline Processed SYN\_ACK & 25,844 & 25,844 & 25,844 & 25,844 & 25,844 & 25,844 \\ TCP Packets & 53,605 & 53,605 & 53,605 & 53,605 & 53,605 \\ \hline Processed UDP & 35,571 & 35,571 & 35,571 & 35,571 & 35,571 \\ Packets & & & & & \\ \hline Elapsed & 15.641558 & 22.225767 & 12.28054 & 15.896778 & 16.937732 \\ Time & & & & & \\ \hline Packet Processed & 64,194.30698 & 45,177.51839 & 81,763.26262 & 63,163.67994 & 59,281.84575 \\ (packets/s) & 3427.088191 & 2411.840269 & 4365.03641 & 3372.066961 & 3164.827548 \\ \hline \hline \end{tabular}
\end{table}
Table 10: Test results for the fingerprinting extraction success rate.  
The fourth experimental result measured the speed of metadata search, and the results are shown in Table 13.  
As a result of the operational test for the test environment, the extraction check of the experimental data showed that it was normally extracted at the time point.