# A Network Traffic Abnormal Detection Method: Sketch-Based Profile Evolution
## 3 Sketch-Based Profile Evolution for Network Traffic Abnormal Detection
### Sketch-Based Network Traffic Structuring Process  
Abnormal network traffic detection requires a Sketch-based network traffic structuring process to generate a TG. The network traffic structuring process first takes the original network traffic as the analysis target, extracts the features of the original network traffic by analyzing the communication sessions (i.e., applies the fingerprint), classifies the obtained traffic, and finally generates the TG according to the obtained application fingerprint and classification results. In data stream mining, the data stream is generated continuously, which consumes a lot of memory, and it is also very resource intensive when searching for historical data, so the Sketch method is introduced. The Sketch data structure [44] can quickly query the data that passes through a large amount of traffic in sequence, i.e., it can provide real-time up-to-date summary data about the data stream and answer the user's query quickly with a certain accuracy guarantee. It can also store the characteristic information of the traffic in the network in real time in a high-speed and complicated network environment, and it takes up little space. In addition, it is possible to compress a large amount of information into a smaller data structure using hash functions, i.e., mapping the data stream into a two-dimensional space [45] to create an abbreviated version of the dataset. The specific method is to first extract keywords from the captured data stream to form a vector, and then to use two sets of hash functions to map the keyword vector into a two-dimensional array to complete the compression of the data stream. In doing so, the two groups of hash functions can choose different mapping methods. For the hash function [46] given two datasets, \(X\) and \(Y\), the mapping relation between datasets \(X\) and \(Y\) is \(R\), which is usually represented by a function: \(f_{R}:x\overset{R}{\rightarrow}y(\forall x\in X,y\in Y)\). When using the network measurement method based on Sketch, the conflict problem to be addressed is the stable state of network traffic behavior, that is, the era of identical data flow shows that the behavior has not changed at this time, and due to the fact that the generated TG map is continuously stored, it can search for information and locate anomalies in historical data. Figure 2 is the Sketch data flow summary method. When anomaly analysis is carried out on a single user in a network environment, network traffic is primarily affected by humans. However, if we extend the consideration to a macro perspective, such as the backbone network or the network of an ISP Internet carrier provider, the human factor is almost erased. In large networks, the characteristics of the overall network are not affected by changes in individual users but change over time. Therefore, to eliminate the influence of the human factor on the behavior modeling and generate a TG for the behavior over this period, the concept of \"structured\" is proposed. The concrete embodiment of network traffic is a continuously generated packet stream. The original attributes of each packet are few, including IP address, format, size, time, etc. According to these attributes, the features that can be analyzed in a fine granularity can be extracted, and the data stream can be processed quickly to prevent being annihilated by a large number of packets. In this paper, the original network traffic is directly taken as the analysis object, and the behavior of the mobile terminal is obtained without analyzing the content of the packet. Moreover, due to the huge amount of data, it is impossible to adopt the manual analysis method in practical application, and the fine-grained application behavior characteristics of the mobile terminal [47]--the application fingerprint--can be obtained online in real-time through the principle of the communication session.