# Fine-Grained High-Utility Dynamic Fingerprinting Extraction for Network Traffic Analysis
## 4 Fine-Grained High-Utility Dynamic Fingerprinting Extraction
### The Utility of Fingerprinting  
The process of obtaining the most typical key exchange in the communication process can more accurately describe the communication process. The TCP Triple Handshake Protocol accounts for more than 80% of the total number of packets. It is difficult to analyze except for differences in length. The number of TLS-related messages is about 10%. Every time an encrypted communication process is established, the communication between the client and the server requires a complete key and certificate exchange process, which has a great impact on the analysis of communication behavior. That is, the smaller number of message types have strong behavior analysis capabilities. Therefore, the differentiation ability of such messages to data flows is defined as utility, which can be understood as the value of the message itself. Similar to natural language processing, there are some frequently used words in natural languages, such as \"de\", \"le\", and \"me\", which are not helpful for the next algorithm. Here, combined with the evaluation of the importance of vocabulary in natural language processing algorithms, the Term Frequency-Inverse Document Frequency (tf-idf) algorithm defines the utility. In the sequence representation \(s=\langle p_{l},p_{m},\cdots,p_{n}\rangle\) of an \(N\)-gram model, the utility of a term \(p_{m}\) is defined as:  
\[u(p_{m})=\varphi(n_{p_{m}},l)\times\phi(n_{d},n_{d}^{p_{m}}) \tag{15}\]  
The number of messages \(p_{m}\) in sequence \(s\) is \(n_{p_{m}}\), and the length of \(s\) is \(l\). The proportion of \(p_{m}\) in sequence \(s\) is \(\varphi(n_{p_{m}},l)\), which is defined as follows:  
\[\varphi(n_{p_{m}},n)=\frac{n_{p_{m}}}{n} \tag{16}\]  
The total number of fingerprints is \(n_{d}\), \(n_{d}^{p_{m}}\) is the fingerprint containing the message \(p_{m}\), and \(\phi\Big{(}n_{d},n_{d}^{p_{m}}\Big{)}\) is defined as the following:  
\[\phi(n_{d},n_{d}^{p_{m}})=\ln\frac{n_{d}}{n_{d}^{p_{m}}+1} \tag{17}\]A large number of data flow fragments in Table 3 are useless, for example, a data flow fragment with only one packet. Table 3 shows only a small part of the data flow. To illustrate the utility's calculation, assume that the Table 3 below lists all the fragments of the data flow. Table 4 describes the utility calculation process. When network traffic is collected for a long enough time, a large-scale utility using a fingerprint is calculated by a statistical method, which is similar to natural language. A large enough corpus ensures the accuracy of the tf-idf algorithm. Data stream fragments are directly associated with transactional databases in the frequent item set mining. Calculate the utility of message 23:2 in Table 4 for a fingerprint TID = 4:  
\[u(23:2)=\varphi(n_{232,n})\times\varphi(n_{d},n_{d}^{232})=\frac{1}{8}\times \ln\frac{8}{5}=0.575\]  
The average utility of sequence \(s=\langle p_{l},p_{m},\cdots,p_{n}\rangle\) calculates the average utility of the whole fingerprint according to the utility of the message, which is defined as follows:  
\[au(s)=\frac{\sum_{X\in(l,m,\cdots,m)}u(P_{X})}{l(p_{X})} \tag{18}\]  
The utility still calculates the entire fingerprint with the application fingerprint of TID = 4:  
\[au(s_{TID=4}) =\frac{u(11:01)+u(11:03)+u(11:02)+u(232)+u(11:02)+u(233)+u(23 5)+u(23:8)}{l(\varepsilon_{TID=4})}\] \[=\frac{0+0+0+0.575+0.087+0.087+0.087}{8}=0.105\]  
Tables 5 and 6 show the calculation results of each fingerprint. \begin{table}
\begin{tabular}{c c c c c c c c c c} \hline \hline
**Item** & **11:01** & **11:02** & **11:03** & **21:** & **23:2** & **23:3** & **23:5** & **23:8** & **23:10** & **23:12** & **24:** \\ \hline Utility & 0 & 0 & 0 & 0.212 & 0.254 & 0.236 & 0.236 & 0.236 & 0.236 & 0.288 & 0.821 \\ \hline \hline \end{tabular}
\end{table} 
Table 6: The utility of items.