# A Novel Network Protocol Syntax Extracting Method for Grammar-Based Fuzzing
## 5 Experiment
### Results  
Table 3 shows the actual number of data packet types owned by Modbus TCP, Modbus RTU, DCCP, DNP3.0, and S7 COMM protocols, the number of data packet types extracted using our method, the correct extracted data packet types number, and the calculated accuracy and coverage information. Although the accuracy of the DCCP protocol and S7 COMM protocol is not high, the coverage is relatively high, which means that we can effectively extract most protocol packet types. Table 4 shows the total number of fields owned by the protocols, the number of fields extracted using our method, the number of correctly extracted fields, and the corresponding accuracy and coverage information. In most cases, our method is able to extract most fields and maintain high accuracy. Figure 8 is a part of syntax information output results which is acquired by using our method over the Modbus TCP protocol. The structure in line \"2\" represents a data packet type, which contains all the component fields of the data packet type \"READ_COILS\" and detailed attribute information of each field. Figure 8: Partial output results of Modbus TCP protocol. \begin{table}
\begin{tabular}{l c c c c c} \hline \hline
**Protocol** & **Actual** & **Extracted** & **Correct** & **Accuracy** & **Coverage** \\ \hline Modbus TCP & 28 & 26 & 25 & 0.962 & 0.893 \\ Modbus RTU & 26 & 24 & 23 & 0.958 & 0.885 \\ DCCP & 20 & 21 & 20 & 0.952 & 1.0 \\ DNP3.0 & 17 & 16 & 15 & 0.938 & 0.882 \\ S/COMM & 19 & 18 & 16 & 0.889 & 0.842 \\ COTP & 16 & 7 & 16 & 0.941 & 1.0 \\ TPKT & 3 & 3 & 3 & 1.0 & 1.0 \\ \hline Average & & & & 0.944 & 0.922 \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Total fields extraction results. \begin{table}
\begin{tabular}{l c c c c c} \hline \hline
**Protocol** & **Actual** & **Extracted** & **Correct** & **Accuracy** & **Coverage** \\ \hline Modbus TCP & 35 & 40 & 34 & 0.850 & 0.971 \\ Modbus RTU & 35 & 40 & 34 & 0.850 & 0.971 \\ DCCP & 12 & 22 & 11 & 0.500 & 0.917 \\ DNP3.0 & 27 & 28 & 22 & 0.786 & 0.815 \\ S/COMM & 13 & 21 & 11 & 0.514 & 0.846 \\ COTP & 11 & 14 & 11 & 0.786 & 1.0 \\ TPKT & 1 & 1 & 1 & 1.0 & 1.0 \\ \hline Average & & & & 0.680 & 0.843 \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Packet type extraction results. Comparison and Discussion  
Table 5 is the entire comparison results of our method, the manual method, the traffic-based method [15], and the NLP-based artificial intelligence method [18]. It can be seen from the results that our method not only maintains both the high-accuracy feature of the manual method and the high-automation feature of the artificial intelligence method, but also has better versatility and can extract more comprehensive protocol syntax information, wherein, in terms of accuracy, our method can extract the field information of the protocol with an average accuracy higher than 0.9, which is much higher than the 0.82 of the NLP-based method. In terms of processing time per protocol, our method can process in a shorter time, usually a few seconds, with very low resource consumption. In terms of applicability, our method can cover a wider range of protocols, including RFC protocols, IoT protocols, and industrial control protocols, and does not rely on real traffic. In terms of syntax information comprehensiveness and accuracy, we compared our method with the most similar NLP-based method on the DCCP protocol, the results are shown in Table 6 and Figure 9. The Table 6 results show that our method can acquire more protocol syntax information, including packet types of the protocol and the component field information of each packet type. The information of each field includes the field name, offset, length, bit-level length information, data type, optional values, endian, and dependencies between fields. Moreover, our approach also provides support for multi-layer protocols.