# Computer User Behavior Anomaly Detection Based on K-Means Algorithm

Authors: Yang Yang; Juan Hao; Jianguang Zhao; Cihang Chen; Haoyue Sun
Journal: Security and Communication Networks
 (2022)

---

## Abstract

As an effective network protection method, computer user behavior anomaly detection can detect unknown attack behaviors. In order to detect user behavior anomalies more efficiently, the authors propose a computer user behavior anomaly detection model based on the K-means algorithm. According to the actual characteristics of single-user behavior, the algorithm uses sliding time window to define transactions and uses the first location strategy to mine behavior patterns. On this basis, the fault-tolerant mode is adopted to compare the current behavior mode with the normal behavior mode, and the anomaly detection results are obtained. Experiments show that using data mining technology, the association rules of user commands, and the mining of sequence patterns, it can effectively discover the user’s behavior pattern, and using sequence matching algorithms such as recursive correlation functions and calculating the similarity between the user’s historical pattern and the current pattern, it provides the possibility to accurately judge user behavior. The following conclusions are obtained through experiments: the model training time is short, the accuracy is high, and it has certain robustness.


---

## Body

## 1. Introduction

With the rapid development of the Internet and the continuous expansion of network data, the problem of network threats has become more and more prominent. Various abnormal events are mixed with normal business traffic, especially some botnets, Trojans, denial of service, cross-site scripting/cross-site request forgery (XSS/CSRF), and other attacks. In order to effectively identify and locate abnormal computer user behaviors, discover potential attack behaviors, and ensure the safe operation of network applications, abnormal traffic detection technology is facing severe challenges [1]. Traditional user behavior classification methods are mainly divided into four types: a port-based method, a deep packet inspection (DPI) based method, a statistics-based method, and a behavior-based method. Among them, the port-based method has low accuracy, and the DPI-based method cannot handle encrypted traffic and has high complexity; therefore, most of the current research is based on statistics and behavior-based methods, and some researches try to use some big data analysis methods. With the gradual increase of network applications, the overall complexity of user behavior characteristics continues to increase, and the subjective judgment difference between data collection and data analysis, the accuracy of network abnormal traffic detection still needs to be improved [2]. The authors propose a user behavior anomaly detection model based on the K-means algorithm, as shown in Figure 1. It quantifies the attributes of user behavior characteristics into direct values for classification and then applies the K-means clustering analysis algorithm to the abnormal user behavior detection to cluster the abnormal user behavior, thereby improving the detection accuracy.Figure 1
Behavior anomaly detection framework.


## 2. Literature Review

Navia et al. stated that anomaly detection (anomaly detection) is the main research direction of intrusion detection systems at present; its characteristic is that through the detection of abnormal behavior of the system, unknown attack modes can be found. The key problem of anomaly detection lies in the establishment of normal usage profiles and how to use this pattern to compare and judge the current user behavior [3]. El-Khatib et al. stated that a behavioral pattern refers to a certain regularity in the process of program execution or user operation [4]. Wu et al. believe that a lot of practical experience tells us, whether it is the execution of the program or the behavior of the user; there is a close correlation in the system characteristics. For example, some privileged programs always access system files in specific directories, and programmers often edit and compile C programs [5]. Obert and Chavez stated that these behavioral features with strong consistency are exactly the components of the behavioral patterns we hope to mine [6]. Wu and others believed that to improve science, the problem of information security became more and more prominent. End users with legal rights or Trojan horse virus that illegally obtain user rights and so on maliciously violate the rules and affect the normal operation of the system and cause serious consequences such as loss of secrets. In the early stage of user behavior control, access control policies were usually used to strictly control users’ behavior rights, but they could not prevent the active destruction of legitimate users [7]. Zhou et al. stated that user behavior anomaly detection technology is an effective way to prevent legitimate users from carrying out active sabotage. User behaviors often show certain patterns in history. Therefore, by establishing behavioral patterns, discover inconsistencies between patterns and implement anomaly detection of user behavior [8]. Veeraiyan et al. believe that currently, the shortcomings of user behavior anomaly detection methods are: insufficient ability to automatically process large amounts of data [9]. Seta and Hartomo indicated that a user behavior anomaly detection algorithm based on pattern mining can automatically process massive user behavior audit data; in order to a certain extent, the detection accuracy can be improved [10]. Kachaoui and Belangour believe that some experts have used sequential pattern mining to establish user behavior patterns for anomaly detection and have achieved certain results; however, in order to facilitate data processing, the user behavior patterns established by them are not objective enough (not in line with the contingency and multirepetition characteristics of user behavior), resulting in inaccurate detection results [11]. The pattern mining-based anomaly detection algorithm proposed by Umamaheswari et al. applies sequential patterns to mine the behavior patterns of a single user and improves the rationality and accuracy of anomaly detection by improving the pattern comparison method; however, the obtained frequent sequence is the behavior that is connected before and after in the behavior sequence, which is inconsistent with the contingency of user behavior and does not take into account the multirepetitive characteristics of user behavior, which makes it have certain limitations in application [12].


## 3. Methods

### 3.1. The Principle and Basic Algorithm of User Behavior Pattern Mining

#### 3.1.1. Characteristics of User Behavior

User behavior includes multiple operations, which can be represented by behavior sequences. Single-user behavior has obvious characteristics in history: regularity, contingency, and multiple repetitions.(1)
Regularity. A lot of practical experience shows that different users have different behavioral habits due to factors such as environment, occupation, and interests, and users’ habitual behaviors are closely related and show regularity. The regularity is reflected in the fact that users frequently perform certain operations, which are also called user behavior habits by many scholars. For example, which software is opened first by the user after booting, which file is accessed most frequently, which operations are performed most frequently, and the sequence of operations that are performed most frequently, and so on; all show regularity in history [13]. This regularity lays the foundation for sequential pattern mining of user behavior. However, the regularity of user behavior has a certain time constraint. When the time interval between two behaviors is too large, the two behaviors are irrelevant or weakly related, and regularity cannot be discussed. By mining these frequent and strongly correlated habitual behaviors of the same user, a normal user behavior pattern can be established.(2)
Chance. The actual user behavior shows a certain regularity as a whole but also has a local contingency. The user’s behavior will be affected by factors such as work and study tasks, environment, and so on, resulting in the user’s habitual behavior adding other operations as a last resort, and the number of added operations and operation content is not fixed. The contingency of user behavior is weakened, and even the succession relationship between users’ habitual behavior is eliminated. For example, the user hasI1,I2,I4,I3,I5, I1,I6,I3, I1,I5,I3, and other operation sequences in history; it can be clearly seen that I1,I3 is the user’s habitual operation sequence, but it does not appear in the form of a short sequence in the actual operation sequence.(3)
Multiple repeatability. Due to the contingency of user behavior, a certain behavior will appear multiple times within a certain period of time, and there is still a correlation and temporal relationship between these same behaviors, which increases the difficulty of sequential pattern mining [14]. For example, a user’s behavior sequence is I1,I2,I1,I3,I2,I1, and these behaviors belong to the same transaction, and behavior I1,I2 occurs multiple times, but due to the time sequence relationship between the behaviors, they cannot be merged; this is exactly the problem that traditional sequential pattern mining algorithms cannot solve.


### 3.2. Mining Principle and Basic Algorithm

Figure2 is a block diagram of the system.Figure 2
Schematic diagram of the module.The application system is based on the Telnet protocol; by intercepting the data packets transmitted in the network and performing corresponding protocol analysis, the commands submitted by the user to the Telnet host and the results returned by the host can be completely restored [15]. We combine each command submitted by the user with other attributes associated with it to form an audit record. The purpose is to extract from the restored user command audit record set, apply association analysis and sequence mining in data mining to extract the correlations existing in the commands executed by users under normal circumstances, and establish the historical behavior pattern of each user; it provides a basis for comparison of user behaviors in the actual detection process.Suppose data setD contains n objects in Euclidean space. The partitioning method assigns objects in D to K clusters C1,…,CK such that for 1≤i,j≤K,Ci⊂D and Ci∩Cj=∅. Use an objective function to evaluate the quality of the partition and make the objects in the cluster similar to each other; it is different from objects in other clusters [16]. That is, the objective function targets high intracluster similarity and intercluster low similarity. The centroid-based partitioning technique uses the centroid of cluster ci to represent the cluster. Conceptually, the centroid of a cluster is its center point. The difference between the object p∈Ci and the representative ci of the cluster can be measured by distp,ci, where distx,y is the Euclidean distance between two points x and y. The objective function is as follows:(1)PY=ckk=1,2,…,K,where E is the sum of squared errors of all objects in the dataset, p is a point in space representing a given data object, and ci is the centroid of cluster Ci. The limitation of the K-means clustering algorithm lies in the data initialization, and the choice of the initial value of the cluster center will affect the final clustering result [17]. In this regard, the authors have carried out several experiments to obtain clustering results under different initial values, compare the mean variance of each result, and take the clustering model with the smallest variance as the user grouping result.


### 3.3. Computer User Behavior Anomaly Detection Model Based on K-Means Algorithm

This model can be divided into three modules: data preprocessing, training, and detection according to the function, as shown in Figure3. The data preprocessing module first obtains the network data packets, then analyzes the protocol and extracts the information, and obtains the basic characteristic information of the original network traffic data. At the same time, the raw network traffic data is preprocessed; the training data that can be input into the model and the real-time data that can be used for intrusion detection are obtained; and the main content is the quantification of the flow information. The training module mainly uses it to train the training data; a model that can perform network traffic anomaly detection is obtained [18]. Finally, in the detection module, use the trained intrusion detection model to receive real-time data from the acquisition and processing module as input, detect abnormal traffic in the real data, and compare it with the actual abnormal traffic data.Figure 3
User behavior anomaly detection model.After the flow information is quantified, K-means is used to perform cluster analysis on the quantified data; the specific steps are as follows.Step 1: In the few labeled samples, all the labeled samples are divided intok classes according to the class labeled types; in this way, the value of k is determined, and at the same time, the mean of all samples in each class is used as the initial cluster center of the class [19].Step 2: After dividing the overall sample according to the principle of proximity, calculate its clustering quality; the clustering quality is evaluated by the sum of squares of errors from all objects to the cluster centers, where the smaller theE, the higher the clustering quality of the sample. The calculation formula of E is as follows:(2)E=∑i=1k∑p∈Cidisp,Ci2.Step 3:k=k+1.Step 4: Select pointa from the unlabeled samples as the center of the new cluster, where a should divide point b away from the center of its current cluster into clusters with a center point, and the distance to the center point decreases, that is, make the following equation take the maximum value:(3)dsum=∑b=1nmax0,b−Cb2−b−a.Step 5: Perform a nearest allocation to the overall sample again and calculate the clustering quality of each class.Step 6: If the clustering quality ofk clusters is higher than that of k–1 clusters, go to step 3; otherwise, if k=k−1, go to step 7.Step 7: Check whether all clusters contain isolated clusters that only contain the center points of the newly added clusters; if so, delete the center point set and unlabeled samples of the cluster,k=k−1, and repeat steps 2–6. If not, use step 6 to use the determined new k value and k initial cluster center points as initial input, and redistribution is performed continuously [20]. Update the mean of the objects in the group to update the cluster centers until the cluster centers converge and obtain k partitions for the entire sample set.


## 3.1. The Principle and Basic Algorithm of User Behavior Pattern Mining

### 3.1.1. Characteristics of User Behavior

User behavior includes multiple operations, which can be represented by behavior sequences. Single-user behavior has obvious characteristics in history: regularity, contingency, and multiple repetitions.(1)
Regularity. A lot of practical experience shows that different users have different behavioral habits due to factors such as environment, occupation, and interests, and users’ habitual behaviors are closely related and show regularity. The regularity is reflected in the fact that users frequently perform certain operations, which are also called user behavior habits by many scholars. For example, which software is opened first by the user after booting, which file is accessed most frequently, which operations are performed most frequently, and the sequence of operations that are performed most frequently, and so on; all show regularity in history [13]. This regularity lays the foundation for sequential pattern mining of user behavior. However, the regularity of user behavior has a certain time constraint. When the time interval between two behaviors is too large, the two behaviors are irrelevant or weakly related, and regularity cannot be discussed. By mining these frequent and strongly correlated habitual behaviors of the same user, a normal user behavior pattern can be established.(2)
Chance. The actual user behavior shows a certain regularity as a whole but also has a local contingency. The user’s behavior will be affected by factors such as work and study tasks, environment, and so on, resulting in the user’s habitual behavior adding other operations as a last resort, and the number of added operations and operation content is not fixed. The contingency of user behavior is weakened, and even the succession relationship between users’ habitual behavior is eliminated. For example, the user hasI1,I2,I4,I3,I5, I1,I6,I3, I1,I5,I3, and other operation sequences in history; it can be clearly seen that I1,I3 is the user’s habitual operation sequence, but it does not appear in the form of a short sequence in the actual operation sequence.(3)
Multiple repeatability. Due to the contingency of user behavior, a certain behavior will appear multiple times within a certain period of time, and there is still a correlation and temporal relationship between these same behaviors, which increases the difficulty of sequential pattern mining [14]. For example, a user’s behavior sequence is I1,I2,I1,I3,I2,I1, and these behaviors belong to the same transaction, and behavior I1,I2 occurs multiple times, but due to the time sequence relationship between the behaviors, they cannot be merged; this is exactly the problem that traditional sequential pattern mining algorithms cannot solve.


## 3.1.1. Characteristics of User Behavior

User behavior includes multiple operations, which can be represented by behavior sequences. Single-user behavior has obvious characteristics in history: regularity, contingency, and multiple repetitions.(1)
Regularity. A lot of practical experience shows that different users have different behavioral habits due to factors such as environment, occupation, and interests, and users’ habitual behaviors are closely related and show regularity. The regularity is reflected in the fact that users frequently perform certain operations, which are also called user behavior habits by many scholars. For example, which software is opened first by the user after booting, which file is accessed most frequently, which operations are performed most frequently, and the sequence of operations that are performed most frequently, and so on; all show regularity in history [13]. This regularity lays the foundation for sequential pattern mining of user behavior. However, the regularity of user behavior has a certain time constraint. When the time interval between two behaviors is too large, the two behaviors are irrelevant or weakly related, and regularity cannot be discussed. By mining these frequent and strongly correlated habitual behaviors of the same user, a normal user behavior pattern can be established.(2)
Chance. The actual user behavior shows a certain regularity as a whole but also has a local contingency. The user’s behavior will be affected by factors such as work and study tasks, environment, and so on, resulting in the user’s habitual behavior adding other operations as a last resort, and the number of added operations and operation content is not fixed. The contingency of user behavior is weakened, and even the succession relationship between users’ habitual behavior is eliminated. For example, the user hasI1,I2,I4,I3,I5, I1,I6,I3, I1,I5,I3, and other operation sequences in history; it can be clearly seen that I1,I3 is the user’s habitual operation sequence, but it does not appear in the form of a short sequence in the actual operation sequence.(3)
Multiple repeatability. Due to the contingency of user behavior, a certain behavior will appear multiple times within a certain period of time, and there is still a correlation and temporal relationship between these same behaviors, which increases the difficulty of sequential pattern mining [14]. For example, a user’s behavior sequence is I1,I2,I1,I3,I2,I1, and these behaviors belong to the same transaction, and behavior I1,I2 occurs multiple times, but due to the time sequence relationship between the behaviors, they cannot be merged; this is exactly the problem that traditional sequential pattern mining algorithms cannot solve.


## 3.2. Mining Principle and Basic Algorithm

Figure2 is a block diagram of the system.Figure 2
Schematic diagram of the module.The application system is based on the Telnet protocol; by intercepting the data packets transmitted in the network and performing corresponding protocol analysis, the commands submitted by the user to the Telnet host and the results returned by the host can be completely restored [15]. We combine each command submitted by the user with other attributes associated with it to form an audit record. The purpose is to extract from the restored user command audit record set, apply association analysis and sequence mining in data mining to extract the correlations existing in the commands executed by users under normal circumstances, and establish the historical behavior pattern of each user; it provides a basis for comparison of user behaviors in the actual detection process.Suppose data setD contains n objects in Euclidean space. The partitioning method assigns objects in D to K clusters C1,…,CK such that for 1≤i,j≤K,Ci⊂D and Ci∩Cj=∅. Use an objective function to evaluate the quality of the partition and make the objects in the cluster similar to each other; it is different from objects in other clusters [16]. That is, the objective function targets high intracluster similarity and intercluster low similarity. The centroid-based partitioning technique uses the centroid of cluster ci to represent the cluster. Conceptually, the centroid of a cluster is its center point. The difference between the object p∈Ci and the representative ci of the cluster can be measured by distp,ci, where distx,y is the Euclidean distance between two points x and y. The objective function is as follows:(1)PY=ckk=1,2,…,K,where E is the sum of squared errors of all objects in the dataset, p is a point in space representing a given data object, and ci is the centroid of cluster Ci. The limitation of the K-means clustering algorithm lies in the data initialization, and the choice of the initial value of the cluster center will affect the final clustering result [17]. In this regard, the authors have carried out several experiments to obtain clustering results under different initial values, compare the mean variance of each result, and take the clustering model with the smallest variance as the user grouping result.


## 3.3. Computer User Behavior Anomaly Detection Model Based on K-Means Algorithm

This model can be divided into three modules: data preprocessing, training, and detection according to the function, as shown in Figure3. The data preprocessing module first obtains the network data packets, then analyzes the protocol and extracts the information, and obtains the basic characteristic information of the original network traffic data. At the same time, the raw network traffic data is preprocessed; the training data that can be input into the model and the real-time data that can be used for intrusion detection are obtained; and the main content is the quantification of the flow information. The training module mainly uses it to train the training data; a model that can perform network traffic anomaly detection is obtained [18]. Finally, in the detection module, use the trained intrusion detection model to receive real-time data from the acquisition and processing module as input, detect abnormal traffic in the real data, and compare it with the actual abnormal traffic data.Figure 3
User behavior anomaly detection model.After the flow information is quantified, K-means is used to perform cluster analysis on the quantified data; the specific steps are as follows.Step 1: In the few labeled samples, all the labeled samples are divided intok classes according to the class labeled types; in this way, the value of k is determined, and at the same time, the mean of all samples in each class is used as the initial cluster center of the class [19].Step 2: After dividing the overall sample according to the principle of proximity, calculate its clustering quality; the clustering quality is evaluated by the sum of squares of errors from all objects to the cluster centers, where the smaller theE, the higher the clustering quality of the sample. The calculation formula of E is as follows:(2)E=∑i=1k∑p∈Cidisp,Ci2.Step 3:k=k+1.Step 4: Select pointa from the unlabeled samples as the center of the new cluster, where a should divide point b away from the center of its current cluster into clusters with a center point, and the distance to the center point decreases, that is, make the following equation take the maximum value:(3)dsum=∑b=1nmax0,b−Cb2−b−a.Step 5: Perform a nearest allocation to the overall sample again and calculate the clustering quality of each class.Step 6: If the clustering quality ofk clusters is higher than that of k–1 clusters, go to step 3; otherwise, if k=k−1, go to step 7.Step 7: Check whether all clusters contain isolated clusters that only contain the center points of the newly added clusters; if so, delete the center point set and unlabeled samples of the cluster,k=k−1, and repeat steps 2–6. If not, use step 6 to use the determined new k value and k initial cluster center points as initial input, and redistribution is performed continuously [20]. Update the mean of the objects in the group to update the cluster centers until the cluster centers converge and obtain k partitions for the entire sample set.


## 4. Results and Analysis

### 4.1. User Data Preparation

#### 4.1.1. Data Dimensionality Reduction

The data mainly comes from the background database of the system; the user login time and usage traffic in the database are mainly stored in the User_log table of the database; the table has a total of 23 fields: login_id, user_login_name, user_ip, user_Jogin_time time_long, user_in_bytes, free_in_bytes, user_mac, and so on. In order to analyze the online behavior of campus network users, establish a multidimensional data feature item that can reflect the user’s characteristics, and extract useful data items from the 23 fields of the user_log database table, after careful inspection of various parameters, it is determined that the user behavior characteristic items are user_login_name, user_ip, user_login_time, time_long, user_in_bytes, and free_in_bytes; they are user name, user login IP, user login time, user online time, download traffic, free download traffic (free traffic for teachers), and other 6 items of data. Taking the log data of a single month as an example because the user login log mainly records the user behavior data when the user logs in once, that is to say, the same user corresponds to multiple login records. Therefore, in order to realize the overall analysis of the users of the entire campus network, aggregate statistics must be performed according to the logged-in user names. At the same time, to analyze the monthly usage traffic of a user, the user_in_bytes (download traffic) field must be extracted and accumulated. The analysis of online duration mainly extracts the time_long (online duration each time) field and accumulates all the user’s usage duration within a month and divides it by the number of days in the month [21]. In addition to statistics based on user names, it is also necessary to extract and analyze based on IP addresses, as a basis for dividing egress traffic. Finally, we mainly use the usage time and average monthly traffic to perform cluster analysis on user behavior to see if a representative user behavior pattern can be generated.


#### 4.1.2. Data Packet Processing

At present, the school has more than a dozen billing groups, among which the flow billing groups need to be adjusted: student monthly subscription group (3 GB traffic for 10 yuan per month, 8 GB traffic for 20 yuan per month, and 15 GB traffic for 30 yuan per month), free group for teachers (unlimited traffic for resources in the education network, 100 MB extra network traffic for free), and postgraduate account in the teaching area (each account allows multiple people log in at the same time, and the education network traffic is not limited). The authors select student users as experimental data; students are mainly divided into 3 billing groups, and 3 classes can be obtained by clustering; the average traffic of each class is counted, which can be used as a reference for the monthly traffic of the user group. The number of student logins is 800,000 times, and through sorting, there are a total of 10,000 common student users. The school’s egress bandwidth has a total of 1,000 Mb traffic, which are 600 Mb education network bandwidth, 300 Mb Unicom bandwidth, and 100 Mb telecom bandwidth. Among them, the international network traffic of the education network bandwidth is relatively small, and the delay to other networks is relatively large, which is more suitable for students in scientific research areas. The Unicom line is relatively smooth and can be used to access the external network more, areas with high network requirements, including office areas, family areas, and so on. Telecom lines are relatively smooth and stable and are mainly used for server bandwidth [22]. Statistics are based on IP addresses, statistics of monthly traffic, and the online duration of each IP address; then, the IP addresses are corresponding to the buildings, and several types are obtained through clustering, and the average bandwidth of each type is calculated as the basis for allocating the egress bandwidth.


### 4.2. Clustering Results and Analysis

Through the improved K-means algorithm, 3 user behavior models are obtained by clustering: the first user behavior model: a total of 5,500 users, accounting for 55% of the total, the average daily Internet access time is 200 minutes, the monthly average traffic usage of such users is within 0∼8G, and the per capita monthly traffic is 5G; the second type of user behavior model: a total of 3,300 users, accounting for 33% of the total, with an average daily Internet access time of 8 to 15 hours, the monthly average traffic usage of this type of users is within 8G to 15G, and the per capita monthly traffic is 12G; and the third type of user behavior model: a total of 1,200 users, accounting for 12% of the total, the average daily Internet access time is 15 hours, the average monthly traffic of this type of users is within 15G∼25G, and the per capita monthly traffic is 20G. After analysis, it can be concluded that the similarity and correlation within each model are very strong, and the similarity and correlation between different models are very low. The network administrator can formulate corresponding traffic charging policies according to the result, so as to make the most efficient use of the network bandwidth. In order to verify the validity of the algorithm, the algorithm program was deployed on a university campus network. The IPFIX raw data obtained from the egress router of a university campus network is about 300 GB per day; after filtering the raw data, after only the records of DNS communication are left, the daily data is about 6 GB; it takes about 100 minutes to filter the raw data of a day using a common PC server, usually with 5 min as the granularity; and the time required for the screening program to process 5 min of IPFIX data is about 20 s. Therefore, it can be considered that the current analysis and processing is the traffic that occurred in the previous 5 minutes, from the characteristics of the attack duration, it can be regarded as the real-time detection of the attack. After the authors’ detection and analysis program was deployed on a university campus network, according to the automatic alarm e-mail of the alarm platform, it was found that at 14:00 on a certain day in July 2021; a certain DNS within the university campus network of 166.111.77.2 was abnormal; at the moment when this abnormality occurs, the number of DNS connections Hf, the traffic Hb, and the data packet Hp are shown in Figures4–6.Figure 4
Alarm time Hf.Figure 5
Alarm time Hb.Figure 6
Alarm time Hp.It can be found from Figure5 that one abnormal DNS host is marked at the alarm time, and two abnormal DNS hosts are marked in Figures 5 and 6, one of which is 166.111.77.2. At this time, the number of connections Hf of 166.111.77.2 is 7,703,520; the average number of connections per second of this host is 25,679; the average number of connections per second of the entire DNS within the 5 minutes is 223,357; and the average number of connections per second is 744, so 166.111.77.2 is marked as a suspicious host. At the same time, its traffic Hb and the number of data packets Hp also have a very obvious peak value; according to the judgment of these three traffic characteristics, this DNS behavior can be marked as suspicious. For the two DNS hosts shown in Figures 5 and 6, because their connection numbers are not outstanding, the behavior of the host is not marked as suspicious. By checking the original data, it is found that the two hosts are behaving normally at this time.For further confirmation, it was found that the exceeded value of 166.111.77.2 reached 12 553 at this time, that is, the data packets in the inbound and outbound directions of the DNS were skewed by more than 100 times. At this time, it can be basically determined that the DNS behavior is abnormal, so the alarm program sends an alarm e-mail to the administrator’s mailbox. Next, observe the continuous exceed change of 166.111.77.2, as shown in Figure7.Figure 7
Exceed data changes.As can be seen from Figure5, the behavior of the DNS host was normal at 13:55, suddenly jumped at 14:00, and continued to drop suddenly at 14:25, by checking the original data of IPFIX; it is found that the host is indeed abnormal, and through the comparison of the oblique directions of incoming and outgoing packets, it is found that outbound data packets exceed inbound data packets by 100 times, the DNS host is indeed used as a traffic amplifier to send traffic to certain destination addresses outside the school. In summary, this detection method can effectively detect DNS abnormal behavior in real time. In the subsequent running experiments, it was found that when a hot page appeared in the school, the number of connections and traffic of a certain DNS address also increased greatly, but it is usually slow, and the skew values of incoming and outgoing data packets usually do not deviate much, so this situation usually does not appear in the alarm message. In addition, statistics on the accumulated alarm information show that in July 2021, there are DNS hosts with suspicious behavior almost every day on the campus network, and the suspicious behavior usually lasts no more than 60 minutes, and most of them are concentrated on some fixed DNS hosts; it should be that these DNS hosts have been lacking detailed security policy implementation. From the analysis of the IP addresses used by attackers, it usually comes from two aspects: one is the IP addresses of other universities in China’s education and scientific research computer network, and the other is the foreign IP addresses; the main reason is that these hosts usually use static IP addresses, which are easier to control after being infected with Trojans. Judging from the victim’s IP address, there are some hotspot applications on the public network, such as the website of a game company. In terms of attack traffic, a DNS host used as a traffic amplifier sent a total of 10.9 GB of data within 5 minutes; according to the detection, the attack traffic is concentrated on the UPD data, with 9.2 GB, that is, the UDP attack traffic that lasts 30 MB per second.


## 4.1. User Data Preparation

### 4.1.1. Data Dimensionality Reduction

The data mainly comes from the background database of the system; the user login time and usage traffic in the database are mainly stored in the User_log table of the database; the table has a total of 23 fields: login_id, user_login_name, user_ip, user_Jogin_time time_long, user_in_bytes, free_in_bytes, user_mac, and so on. In order to analyze the online behavior of campus network users, establish a multidimensional data feature item that can reflect the user’s characteristics, and extract useful data items from the 23 fields of the user_log database table, after careful inspection of various parameters, it is determined that the user behavior characteristic items are user_login_name, user_ip, user_login_time, time_long, user_in_bytes, and free_in_bytes; they are user name, user login IP, user login time, user online time, download traffic, free download traffic (free traffic for teachers), and other 6 items of data. Taking the log data of a single month as an example because the user login log mainly records the user behavior data when the user logs in once, that is to say, the same user corresponds to multiple login records. Therefore, in order to realize the overall analysis of the users of the entire campus network, aggregate statistics must be performed according to the logged-in user names. At the same time, to analyze the monthly usage traffic of a user, the user_in_bytes (download traffic) field must be extracted and accumulated. The analysis of online duration mainly extracts the time_long (online duration each time) field and accumulates all the user’s usage duration within a month and divides it by the number of days in the month [21]. In addition to statistics based on user names, it is also necessary to extract and analyze based on IP addresses, as a basis for dividing egress traffic. Finally, we mainly use the usage time and average monthly traffic to perform cluster analysis on user behavior to see if a representative user behavior pattern can be generated.


### 4.1.2. Data Packet Processing

At present, the school has more than a dozen billing groups, among which the flow billing groups need to be adjusted: student monthly subscription group (3 GB traffic for 10 yuan per month, 8 GB traffic for 20 yuan per month, and 15 GB traffic for 30 yuan per month), free group for teachers (unlimited traffic for resources in the education network, 100 MB extra network traffic for free), and postgraduate account in the teaching area (each account allows multiple people log in at the same time, and the education network traffic is not limited). The authors select student users as experimental data; students are mainly divided into 3 billing groups, and 3 classes can be obtained by clustering; the average traffic of each class is counted, which can be used as a reference for the monthly traffic of the user group. The number of student logins is 800,000 times, and through sorting, there are a total of 10,000 common student users. The school’s egress bandwidth has a total of 1,000 Mb traffic, which are 600 Mb education network bandwidth, 300 Mb Unicom bandwidth, and 100 Mb telecom bandwidth. Among them, the international network traffic of the education network bandwidth is relatively small, and the delay to other networks is relatively large, which is more suitable for students in scientific research areas. The Unicom line is relatively smooth and can be used to access the external network more, areas with high network requirements, including office areas, family areas, and so on. Telecom lines are relatively smooth and stable and are mainly used for server bandwidth [22]. Statistics are based on IP addresses, statistics of monthly traffic, and the online duration of each IP address; then, the IP addresses are corresponding to the buildings, and several types are obtained through clustering, and the average bandwidth of each type is calculated as the basis for allocating the egress bandwidth.


## 4.1.1. Data Dimensionality Reduction

The data mainly comes from the background database of the system; the user login time and usage traffic in the database are mainly stored in the User_log table of the database; the table has a total of 23 fields: login_id, user_login_name, user_ip, user_Jogin_time time_long, user_in_bytes, free_in_bytes, user_mac, and so on. In order to analyze the online behavior of campus network users, establish a multidimensional data feature item that can reflect the user’s characteristics, and extract useful data items from the 23 fields of the user_log database table, after careful inspection of various parameters, it is determined that the user behavior characteristic items are user_login_name, user_ip, user_login_time, time_long, user_in_bytes, and free_in_bytes; they are user name, user login IP, user login time, user online time, download traffic, free download traffic (free traffic for teachers), and other 6 items of data. Taking the log data of a single month as an example because the user login log mainly records the user behavior data when the user logs in once, that is to say, the same user corresponds to multiple login records. Therefore, in order to realize the overall analysis of the users of the entire campus network, aggregate statistics must be performed according to the logged-in user names. At the same time, to analyze the monthly usage traffic of a user, the user_in_bytes (download traffic) field must be extracted and accumulated. The analysis of online duration mainly extracts the time_long (online duration each time) field and accumulates all the user’s usage duration within a month and divides it by the number of days in the month [21]. In addition to statistics based on user names, it is also necessary to extract and analyze based on IP addresses, as a basis for dividing egress traffic. Finally, we mainly use the usage time and average monthly traffic to perform cluster analysis on user behavior to see if a representative user behavior pattern can be generated.


## 4.1.2. Data Packet Processing

At present, the school has more than a dozen billing groups, among which the flow billing groups need to be adjusted: student monthly subscription group (3 GB traffic for 10 yuan per month, 8 GB traffic for 20 yuan per month, and 15 GB traffic for 30 yuan per month), free group for teachers (unlimited traffic for resources in the education network, 100 MB extra network traffic for free), and postgraduate account in the teaching area (each account allows multiple people log in at the same time, and the education network traffic is not limited). The authors select student users as experimental data; students are mainly divided into 3 billing groups, and 3 classes can be obtained by clustering; the average traffic of each class is counted, which can be used as a reference for the monthly traffic of the user group. The number of student logins is 800,000 times, and through sorting, there are a total of 10,000 common student users. The school’s egress bandwidth has a total of 1,000 Mb traffic, which are 600 Mb education network bandwidth, 300 Mb Unicom bandwidth, and 100 Mb telecom bandwidth. Among them, the international network traffic of the education network bandwidth is relatively small, and the delay to other networks is relatively large, which is more suitable for students in scientific research areas. The Unicom line is relatively smooth and can be used to access the external network more, areas with high network requirements, including office areas, family areas, and so on. Telecom lines are relatively smooth and stable and are mainly used for server bandwidth [22]. Statistics are based on IP addresses, statistics of monthly traffic, and the online duration of each IP address; then, the IP addresses are corresponding to the buildings, and several types are obtained through clustering, and the average bandwidth of each type is calculated as the basis for allocating the egress bandwidth.


## 4.2. Clustering Results and Analysis

Through the improved K-means algorithm, 3 user behavior models are obtained by clustering: the first user behavior model: a total of 5,500 users, accounting for 55% of the total, the average daily Internet access time is 200 minutes, the monthly average traffic usage of such users is within 0∼8G, and the per capita monthly traffic is 5G; the second type of user behavior model: a total of 3,300 users, accounting for 33% of the total, with an average daily Internet access time of 8 to 15 hours, the monthly average traffic usage of this type of users is within 8G to 15G, and the per capita monthly traffic is 12G; and the third type of user behavior model: a total of 1,200 users, accounting for 12% of the total, the average daily Internet access time is 15 hours, the average monthly traffic of this type of users is within 15G∼25G, and the per capita monthly traffic is 20G. After analysis, it can be concluded that the similarity and correlation within each model are very strong, and the similarity and correlation between different models are very low. The network administrator can formulate corresponding traffic charging policies according to the result, so as to make the most efficient use of the network bandwidth. In order to verify the validity of the algorithm, the algorithm program was deployed on a university campus network. The IPFIX raw data obtained from the egress router of a university campus network is about 300 GB per day; after filtering the raw data, after only the records of DNS communication are left, the daily data is about 6 GB; it takes about 100 minutes to filter the raw data of a day using a common PC server, usually with 5 min as the granularity; and the time required for the screening program to process 5 min of IPFIX data is about 20 s. Therefore, it can be considered that the current analysis and processing is the traffic that occurred in the previous 5 minutes, from the characteristics of the attack duration, it can be regarded as the real-time detection of the attack. After the authors’ detection and analysis program was deployed on a university campus network, according to the automatic alarm e-mail of the alarm platform, it was found that at 14:00 on a certain day in July 2021; a certain DNS within the university campus network of 166.111.77.2 was abnormal; at the moment when this abnormality occurs, the number of DNS connections Hf, the traffic Hb, and the data packet Hp are shown in Figures4–6.Figure 4
Alarm time Hf.Figure 5
Alarm time Hb.Figure 6
Alarm time Hp.It can be found from Figure5 that one abnormal DNS host is marked at the alarm time, and two abnormal DNS hosts are marked in Figures 5 and 6, one of which is 166.111.77.2. At this time, the number of connections Hf of 166.111.77.2 is 7,703,520; the average number of connections per second of this host is 25,679; the average number of connections per second of the entire DNS within the 5 minutes is 223,357; and the average number of connections per second is 744, so 166.111.77.2 is marked as a suspicious host. At the same time, its traffic Hb and the number of data packets Hp also have a very obvious peak value; according to the judgment of these three traffic characteristics, this DNS behavior can be marked as suspicious. For the two DNS hosts shown in Figures 5 and 6, because their connection numbers are not outstanding, the behavior of the host is not marked as suspicious. By checking the original data, it is found that the two hosts are behaving normally at this time.For further confirmation, it was found that the exceeded value of 166.111.77.2 reached 12 553 at this time, that is, the data packets in the inbound and outbound directions of the DNS were skewed by more than 100 times. At this time, it can be basically determined that the DNS behavior is abnormal, so the alarm program sends an alarm e-mail to the administrator’s mailbox. Next, observe the continuous exceed change of 166.111.77.2, as shown in Figure7.Figure 7
Exceed data changes.As can be seen from Figure5, the behavior of the DNS host was normal at 13:55, suddenly jumped at 14:00, and continued to drop suddenly at 14:25, by checking the original data of IPFIX; it is found that the host is indeed abnormal, and through the comparison of the oblique directions of incoming and outgoing packets, it is found that outbound data packets exceed inbound data packets by 100 times, the DNS host is indeed used as a traffic amplifier to send traffic to certain destination addresses outside the school. In summary, this detection method can effectively detect DNS abnormal behavior in real time. In the subsequent running experiments, it was found that when a hot page appeared in the school, the number of connections and traffic of a certain DNS address also increased greatly, but it is usually slow, and the skew values of incoming and outgoing data packets usually do not deviate much, so this situation usually does not appear in the alarm message. In addition, statistics on the accumulated alarm information show that in July 2021, there are DNS hosts with suspicious behavior almost every day on the campus network, and the suspicious behavior usually lasts no more than 60 minutes, and most of them are concentrated on some fixed DNS hosts; it should be that these DNS hosts have been lacking detailed security policy implementation. From the analysis of the IP addresses used by attackers, it usually comes from two aspects: one is the IP addresses of other universities in China’s education and scientific research computer network, and the other is the foreign IP addresses; the main reason is that these hosts usually use static IP addresses, which are easier to control after being infected with Trojans. Judging from the victim’s IP address, there are some hotspot applications on the public network, such as the website of a game company. In terms of attack traffic, a DNS host used as a traffic amplifier sent a total of 10.9 GB of data within 5 minutes; according to the detection, the attack traffic is concentrated on the UPD data, with 9.2 GB, that is, the UDP attack traffic that lasts 30 MB per second.


## 5. Conclusion

The authors’ proposed network traffic anomaly detection model is based on K-means algorithm; first, the collected network traffic characteristic attributes are quantified into direct values, which is convenient for subsequent classification, and then the K-means algorithm is used to perform cluster analysis on abnormal traffic, a detection model is trained, which is superior to the traditional traffic anomaly detection model in terms of detection rate, false positive rate, and location attack detection rate. The authors present a user behavior anomaly detection algorithm based on pattern mining, which is aimed at the actual characteristics of single-user behavior; sequential pattern mining is performed by applying a sliding time window to delimit transactions, and a first fixed policy, on the basis of a tolerant pattern comparison process, is applied to compare the current behavior sequence with the normal behavior pattern to obtain anomaly detection results. The authors collect the behavior audit data of a single user to conduct experiments; the experimental results show that the correlation can objectively describe the mode comparison results, and the algorithm can effectively determine normal behavior and abnormal behavior. The normal historical behavior data used by the algorithm can be continuously updated so that the obtained normal behavior pattern can reflect the habitual operation of the user behavior in the recent time, which is more in line with the actual situation of the user behavior. This makes the anomaly detection result more reasonable. However, the algorithm needs to scan the database multiple times, and the mining efficiency is low. How to further improve the mining efficiency of the algorithm is the content that needs to be studied in the next step.



---

*Source: 3084420-2022-05-24.xml*