# A Network Traffic Abnormal Detection Method: Sketch-Based Profile Evolution
## 3 Sketch-Based Profile Evolution for Network Traffic Abnormal Detection
### Behavior Pattern Detection Method Based on Profile Evolution  
During a period, the user's behavior is relatively stable. When performing anomaly detection, the user's Profile needs to be established first. Suppose the vector extracted by the CNN-LSTM network feature is \(T=(t_{1},t_{2},\cdots,t_{i})\), and the Base of the user behavior Profile \(B=[b_{1},b_{2},\cdots,b_{i}]\). Equation (12) linearly expresses the relationship between \(T\) and \(B\):  
\[L_{w,b}=\left\|T-B\right\|=\left|\left|T-\sum_{l=1}^{i}w_{l}\cdot b_{l}\right|\right| \tag{12}\]  
The basis generation problem is expressed according to the calculation of the basis set with the smallest reconstruction error, as shown in Equation (13), where \(T\) is the input signal and \(w_{l}\) is the weight vector \(B=[b_{1},b_{2},\cdots,b_{i}]\). Equation (13) is to obtain the \(L_{w,b}\) minimized parameters \(w,b\):  
\[\text{arg min}L_{w,b}=\text{arg min}\left|\left|T-\sum_{l=1}^{i}w_{l}\cdot b_{ l}\right|\right| \tag{13}\]  
The vector of the newly generated network data stream is \(T^{(1)},T^{(2)},\cdots,T^{(n)}\). Equation (14) is obtained according to Equation (13):  
\[L_{w,b}^{(n)}=\left|\left|T^{(n)}-\sum_{l=1}^{n}w_{l}^{(n-1)}\cdot b_{l}^{(n-1 )}\right|\right| \tag{14}\]  
Equation (14) can be refactored into a new objective function to solve the minimization problem:  
\[\underset{w,b}{\text{arg min}}\left|\left|L_{w,b}^{(n)}-\sum_{k=1}^{n}\sum_{l =1}^{i}w_{l}^{(k)}\cdot b_{l}^{(k)}\right|\right| \tag{15}\]  
Singular value decomposition [51] (SVD) is performed on Equation (15), and the feature continuously generated by the CNN-LSTM network is \(L^{(n)}\), as shown in Equation (16):  
\[L^{(n)}= \begin{bmatrix}t_{1}^{(1)}&t_{2}^{(1)}&\cdots&t_{i}^{(1)}\\ t_{1}^{(2)}&t_{2}^{(2)}&\cdots&t_{i}^{(2)}\\ \vdots&\vdots&\ddots&\vdots\\ t_{1}^{(n)}&t_{2}^{(n)}&\cdots&t_{i}^{(n)}\end{bmatrix}  \tag{16}\]  
Perform SVD on \(L^{(n)}\), as shown in Equation (17):  
\[L^{(n)}=U\Sigma V^{T} \tag{17}\]  
Among them, \(U,V^{T}\) is a unitary matrix, \(\Sigma\) is a diagonal matrix, as shown in Equation (18):  
\[\Sigma= \begin{bmatrix}\tau_{1}&0&\cdots&0\\ 0&\tau_{2}&\cdots&0\\ \vdots&\vdots&\ddots&\vdots\\ 0&0&\cdots&\tau_{n}\end{bmatrix}  \tag{18}\]  
Because \(\Sigma\) is a diagonal matrix, the SVD of \(L^{(n)}\) can also be written as shown in Equation (19):  
\[L(n)=\sum_{l=1}^{n}u_{l}\tau_{l}v_{l}^{T} \tag{19}\]Convert Equation (17) to \(U\Sigma=V^{T}L^{(n)}\), and associate \(U=B\) (the energy of \(U\) is concentrated on the diagonal) to obtain the corresponding weight \(w_{l}\), as shown in Equation (20):  
\[\widehat{w_{l}}=\frac{L^{(n)}U}{\left|L^{(n)}\right|} \tag{20}\]  
Therefore, the user's Profile is shown in Equation (21):  
\[B=[b_{1},b_{2},\cdots,b_{i}]=\sum_{l=1}^{i}\widehat{w_{l}}\cdot b_{l} \tag{21}\]  
Assuming that the newly extracted feature vector is \(T^{\prime}=x_{1}^{\prime},x_{2}^{\prime},\cdots,x_{i}^{\prime}\), set a threshold \(\delta\), and the distance between the new feature and the Profile is calculated by Power Distance [52], as shown in Equation (22):  
\[D_{B,Xt}=(\sum_{l=1}^{i}\left|x_{l}^{\prime}-\widehat{w_{l}}\right|^{p})^{ \frac{1}{p}} \tag{22}\]  
Power \((p,\ r)\) is a distance measure of the distance between two vectors of length \(i\). It is usually expressed as the physical distance of two locations in three-dimensional space, but can be applied to vectors of any dimension when the data type is numeric. The specific distance metric is determined according to the values of \(p\) and \(r\). For example, if \(p=r=2\), Equation (22) becomes the Euclidean distance. If the calculated distance is greater than \(\delta\), it will become an abnormal point. The newly acquired sample vector not only needs to calculate the distance from the Profile for anomaly detection but also has an impact on the Profile. The new data flow will evolve the old Basis into a new Basis, so the detection model will follow the data flow.