# Peertrap: An Unstructured P2P Botnet Detection Framework Based on SAW Community Discovery

Authors: Ying Xing; Hui Shu; Fei Kang; Hao Zhao
Journal: Wireless Communications and Mobile Computing
 (2022)

---

## Abstract

Botnet has become one of the serious threats to the Internet ecosystem, and botnet detection is crucial for tracking and mitigating network threats on the Internet. In the evolution of emerging botnets, peer-to-peer (P2P) botnets are more dangerous and resistant because of their distributed characteristics. Among them, unstructured P2P botnets use custom protocols for communication, which can be integrated with legitimate P2P traffic. Moreover, their topological structure is more complex, and a complete topology cannot be obtained easily, making them more concealed and difficult to detect. The bot itself is a kind of overlay network, and research shows that the nodes with shared neighbors usually belong to a certain community. Aiming at unstructured P2P botnets and exploiting complex network theory, from the perspective of shared neighbor nodes, this article proposes a botnet detection framework called Peertrap based on self-avoiding random walks (SAW) community detection under the condition of incomplete topological information. Firstly, network traffic is converted into Netflow, by utilizing Apache Flink big data platform. Also, a P2P traffic cluster feature extraction rule is proposed for distinguishing P2P traffic from non-P2P traffic, and it is formulated by using the upstream and downstream traffic and address distribution threshold features. Then, the confidence between P2P clusters is calculated by the Jaccard coefficient to construct a shared neighbor graph, and the same type of P2P communities are mined by hierarchical clustering using SAW algorithm combined with PCA. Finally, two community attributes, mean address distribution degree and mean closeness degree, are used to distinguish botnets. Experiments are conducted on three unstructured P2P botnets datasets, Sality, Kelihos, and ZeroAccess, and the CTU classic datasets, and then good detection results can be achieved. The framework overcomes one of the most critical P2P botnet detection challenges. It can detect P2P bots with high accuracy in the presence of legitimate P2P traffic, incomplete information network topology, and C&C channel encryption. Our method embodies the typical application of complex network theory in botnet detection field, and it can detect botnets from different families in the network, with good parallelism and scalability.


---

## Body

## 1. Introduction

Botnet is an overlay network for malicious activities, which is formed by many hosts (bot or zombies) infected by bot’s programs and controlled by Botmasters [1, 2]. Botnet is usually composed of Botmaster, command and control (C&C) channel, and bot. Botnet can launch network attacks such as distributed denial of service (DDoS), spam, phishing, click fraud, and stealing of sensitive information, which has been regarded as one of the most serious threats to many industries such as finance, education, government, medical care, critical infrastructure, and the Internet of Things (IoT) currently [3]. Unlike traditional viruses and worms, Botnet is a malicious code in advanced form, and its core is the C&C channel. Traditional centralized C&C architecture and the HTTP protocol–based architecture are easy to be traced and dismantled by security guards. The evolution of botnet is featured with platform diversification, communication concealment, intelligent control, etc. The distributed botnet architecture based on P2P protocol can very well solve the problem of single-point failure, and its function, structure, and hiding techniques has been greatly improved. P2P technology has been widely used in many fields, such as BitTorrent and eMule for file sharing, PPTV for live video broadcast, QQ and Skype for instant messaging, etc. P2P botnets are the same as legitimate P2P networks, having characteristics of distributed topology, mutual communication between nodes, and frequent activities of various messages in the network. In the future, more P2P botnet variants and more complex botnet malware will appear.According to the communication protocol, P2P botnets can be divided into structured botnets and unstructured botnets [4]. Structured botnets commonly use P2P protocols based on Distributed Hash Table (DHT), such as storm [5], hajime [6], linux.rex.1 [7], and mozi [8], and bots locate and find resources through < key,value > index structure. Structured P2P botnet is simple and reliable to construct. However, they are vulnerable to Index Poisoning attacks [9] and Sybil attacks [10], and they are easy to measure through Crawler and Sybil nodes [11]. Unstructured P2P botnets are implemented by custom P2P protocols, which usually communicate by random scanning or Peer-list. They are flexible in selecting neighboring peers and routing mechanisms. Botnets based on random scanning have inherent vulnerabilities of abnormal traffic. Peer-list exchange is adopted for communication, and each bot needs to maintain a list containing node information, namely, peer-list. Also, each bot, regularly and randomly selects some nodes to communicate, obtain attacker instructions, and update node information, which is used in many new P2P botnets, such as Kelihos [12], Waledac [13], Zero Access [14], Sality, Nugache, Miner, and Hide and Seek. Peer-list exchange-based botnets are vulnerable to Peer-list pollution. Unstructured P2P botnet has good flexibility, scalability, robustness, better concealment and intelligence, and more complex topology. Besides, it can be integrated with legitimate P2P traffic and is not easy to be tracked and measured. However, there is no specific structure available, so it is difficult to reconstruct and detect.There is a plethora of studies on botnet detection. Most of them focus on the structured P2P architecture [15–17], but few research groups pay attention to the unstructured P2P botnet detection. Methods such as [18–20] assume that botnets form structured P2P coverage. Based on Kademlia [21] or Chord [22], this coverage can be inferred from the stream data by using the topological structure hidden in the communication relationship between bots. Such methods usually assume that the complete communication in botnets is known, so they are not suitable for detecting unstructured P2P botnets. In addition, malicious traffic usually overlaps with legitimate traffic, which also is one of the main challenges for detection.Complex networks are highly complex networks, which are abstractions of complex systems. The networks with some or all the properties of self-organization, self-similarity, attractor, small world, and scale-free are called complex networks, such as Internet, social network, power network, and biological network. The communication between bots is an overlay network. Complex network community detection is a multi-disciplinary frontier research hotspot in recent years [23–25]. In P2P networks, the network flows of different hosts in the same community should have similar group behavior patterns. Thus, P2P botnet detection can be regarded as a community detection problem, and each host may be running P2P bots or legitimate P2P applications. For the unstructured P2P scenario, it is assumed that under the incomplete information topology, although hosts in botnets are not directly connected with each other, they should at least share the same communication partners [26]. The research shows that the nodes with shared neighbors usually belong to a certain community. Therefore, based on the theory of complex networks community detection, this paper proposes a botnet detection framework called Peertrap, based on SAW algorithm. First, the network traffic is transformed into Netflow, the P2P traffic is extracted by using the Apache Flink big data framework, and quintuple P2P clusters are generated by using features, such as upstream and downstream traffic, and address distribution threshold. Then, the confidence between clusters is calculated to construct a shared neighbor association graph, and the SAW algorithm is used to mine communities. Finally, two community attributes, mean address distribution degree, and mean closeness degree are used to distinguish botnets. Experiments on three datasets and unstructured P2P botnets, Sality, Kelihos, and ZeroAccess, show that good detection results can be achieved.The main contributions of this article are as follows:(1)
For unstructured P2P botnets, a scalable P2P botnet detection framework is proposed.(2)
A P2P traffic feature extraction rule is proposed, which can quickly distinguish P2P traffic from non-P2P traffic by utilizing the Apache Flink big data platform.(3)
The proposed method can perform detection in the network with incomplete topology information. Based on shared neighbor nodes, a shared neighbor graph is constructed by calculating the confidence level, and the SAW algorithm is combined with principal component analysis (PCA) to define the dissimilarity for agglomerative hierarchical clustering to distinguish communities. This is a typical application of complex network theory in the field of botnet detection.(4)
The proposed method can perform detection under C&C channel encryption with overlapping benign P2P and malignant P2P traffic on the host.The organizational structure of this paper is as follows. The second part introduces the typical unstructured P2P botnets and the current situation of detection technology. The third part describes the basic principles and specific processes of the proposed framework. The fourth part expounds the experimental analysis and verification. The fifth part summarizes the full text.


## 2. Related Work

### 2.1. Typical Unstructured P2P Botnet

In this section, the evolution mechanism of typical unstructured P2P botnet, such as Nugache, Sality, Kelihos, and ZeroAccess, is briefly introduced.(1)
Nugache: As is a P2P botnet based on custom protocol, Nugache first appeared in early 2006 [27], and its main purpose is DDoS attacks. Nugache botnet is one of the earliest botnets that use strong cryptography to protect their communication. Some messages are signed with 4096 bit RSA key to prevent unauthorized control, and part of peer-to-peer communication is encrypted with RSA/Rijndael hybrid scheme.(2)
Sality: The P2P version of Sality first appeared in early 2008. It uses pull-based unstructured P2P network to spread URLs, and exchange new URLs with the neighbors it frequently contacts. Sality uses a special “announcement” message to announce itself to routable peers in a P2P network. Also, it adopts a reputation mechanism, and each roamer tracks the reputation of all its neighboring peers [28].(3)
Kelihos (Also known as Hlux): Kelihos is an unstructured P2P botnet that first appeared at the end of 2010. It is mainly used for spam and ID theft [12], and it implements many backup channels. If all its peers cannot be accessed, these backup channels can be activated by bot. The DNS fast flux network is the most important module in Kelihos. Each bot contains a hard-coded domain name that points to another infected bot. The bot can be used as a seed node to restore the connection in a P2P network.(4)
ZeroAccess (also known as Sirefef): ZeroAccess is a malware download program using unstructured P2P architecture. The ZeroAccess bots periodically query neighbors for new malware payloads and use a broadcast mechanism to push the list of new peers to all bots [29]. Zombies can be quarantined by sending peer lists to the bot, which contains invalid entries with the most recent timestamp. ZeroAccess v1/v2 are two variants of ZeroAccess, which appeared in mid-2009 and early 2012, respectively. The variants are organized into seven disjoint networks.(5)
Hide & seek (also known as HNS): As a botnet of IoT, it first appeared in January 2018. HNS uses vulnerabilities for propagation (two of the vulnerabilities are specific to some IP cameras), and it aims to infect home routers [30]. HNS agent has a modular plug-in architecture that contains two payload modules: an information thief and a Monero miner module. HNS supports custom P2P protocols. The nodes are allowed to receive remote instructions from the network, and each node has a small peer list, which is maintained and updated by frequent requests for new information in P2P networksTable1 gives the communication protocol, message propagation method, communication direction, C&C architecture, and purpose of P2P botnet families [31].Table 1
Overview of P2P botnet families.
FamilyProtocolProp.Dir.C&CPurposeNugache [27]CustomGossipPullP2PD, TSality [28]CustomGossipPullP2PD, N, PKelihos [12]CustomGossipPullHybridC, D, M, N, SZeroAccess [29]CustomGossipPullP2PPHide & seek [30]CustomGossipPullP2PTThe P2P botnets listed in Table 1 have many purposes, C = Click Fraud, D = DDoS, M = Bitcoin Mining, N = Network Services, P = Pay-Per-Install, S = Spam, T = Credential Theft.


### 2.2. Related Research

Most existing botnet detection methods are based on the statistical characteristics of packets and flows [32, 33]. Packet-based detection methods perform deep packet detection on payloads, and they detect bot activities according to predefined patterns and feature codes retrieved from well-known bots. For example, BotHunter relies on Snort and has a rich signature database that can quickly and accurately discover botnet activities [25]. However, robots can avoid signature-based detection by using advanced encryption mechanisms or code obfuscation tools. Flow-based statistical features are divided into attribute features and behavior features, which are independent of payload information. The attribute features are extracted from network traffic and classified through machine learning methods, including duration, message size, port number, packet sending rate, reconnection times, etc. The behavior feature assumes that the botnets in the same botnet have similar malicious activities and communication patterns, and periodic laws of frequent bot communication are extracted according to the network traffic. Abnormal points are detected by the establishment of communication time series, and abnormal points are detected. Intelligent botnets can easily randomize these features [34], so flow-based statistical feature detection methods are often invalidated by advanced botnets. Considering the network communication in the P2P network, the network behaviors of benign and infected P2P hosts contain some similarities at the network flow (or session) level. Therefore, these methods cannot resist the coexistence of P2P malicious traffic and benign traffic.Entelechea [35] proposed to use collaborative graph mining method to identify a group of P2P bots on the hyper-traffic graph that is constructed based on network traffic function (i.e., hourly traffic and duration of each traffic). They thought that P2P botnet flows often have small sizes and long duration.Peerhunter [36] suggested using host-level community behavior analysis to detect P2P botnets, but this study did not consider that P2P botnets and legitimate P2P applications could run on the same set of hosts.Botgrep [37] illustrated a method to detect structured P2P botnets by NetFlow communication graph, and this method converges fast from random walk to a stable distribution. Also, the accuracy of this method was further improved by recursively applying Metropolis–Hastings sampling and using a modified version of the SybilInfer clustering algorithm. The algorithm treated each subgraph as a social media graph, and it treated suspicious bots as Sybil nodes according to the probability model of P2P nodes. Tests of the method were performed on many synthetic structured P2P topologies, including Kademlia and Chord, and high-precision results were achieved. However, this method could only identify structured communication subgraphs, but it failed to determine which subgraphs contain P2P botnets.BotTrack [20] constructed a communication diagram to the host from NetFlow records. Then, a modified version of PageRank algorithm was used to determine the hub and permission level for each node. It was believed that the infected machines in structured P2P botnets showed distinguishable connection patterns, and well-connected host groups had similar hub and privilege-level values. These data points form a compact group that could be extracted with DBSCAN. Also, by adjusting PageRank weight, the previous knowledge about infection could be used to improve the accuracy of the algorithm. The method was also tested on Kademlia and Chord.Zhang et al. [38] established a scalable system for stealth P2P-botnet detection. In the waiting stage, the system could detect stealth P2P botnets without knowing the existing malicious behaviors in advance. By extracting P2P traffic fingerprint for analysis, it could work when botnet traffic overlaps with legitimate P2P traffic on the same host. However, the experimental dataset in the study was slightly biased and less challenging. For example, in the dataset, the number of bots was twice the number of legitimate P2P hosts, making it easier for bots to form clusters than legitimate P2P hosts.Daneshgar and Abbaspour [39] expounded a new P2P botnet detection framework based on two-stage sequential pattern mining (SPM), which was resilient to existing legitimate P2P traffic. First, the P2P management flow of hosts was filtered and transformed into a sequence database. Then, the frequent behaviors of hosts were extracted by using the SPM method. The framework could detect P2P bots in the presence of legitimate P2P traffic, and the detection accuracy reaches 99.2%.Peerclean [40] presented a method to detect P2P bots in network traffic by using dynamic group behavior analysis (DGBA) and machine learning. In the training stage, network traffic is transformed into Netflow, and some connection functions were extracted by DGBA from each traffic to construct an SVM classifier. In the detection stage, abnormal behaviors were searched from each unmarked cluster to capture P2P robots. This method could produce high-precision results on unstructured botnets (including Sality and ZeroAccess). However, since the method depends on the kernel function used for SVM, the constructed classifier might be overfitted.Coskun et al. [26] put forward a detection method especially for unstructured P2P botnets. The method first establishes the interconnection graph of the entire network and then tries to use “seeds” (known bots) to identify other botnets. The authors believed that although the hosts in botnets were not directly connected with each other, they should at least share the same communication partners. Also, these interconnected hosts communicate with fewer than k hosts between them. Based on this, an interconnection graph should be obtained, in which the nodes stand for each host, and an edge between two nodes indicates that these hosts were interconnected. Then, the “dye pumping” algorithm was introduced. However, this method was impractical because “seeds” cannot be obtained in advance.Muhs [41] transformed NetFlow recorded data into a weighted adjacency matrix that was suitable for random walk. Based on the BotGrep algorithm, random walk was used to detect unstructured P2P botnets on large communication graphs, and DBSCAN clustering was used for classification. It should be noted that this method was proposed under the assumption of a complete communication graph.Comparative analysis of typical P2P botnet detection methods is shown in Table2.Table 2
Comparative analysis of typical P2P botnet detection methods.
Communication protocolDetection contentflexibilityAdopt technologyData setAccuracy of detectionGeneral p2pStructured P2PUnstructured P2PFlow-basedBinary-basedEncrypted channelOverlapping flowCommunity discoverygraph-based methodMachine learningstatistical analysisStormWaledacSalityKelihosZeroAccessZeusKademliaChordKoorde95%–100%90%–94%Peerhunter [36]✓✓✓✓✓✓✓✓✓✓[38]✓✓✓✓✓✓✓✓✓[39]✓✓✓✓✓✓✓✓Botgrep [37]✓✓✓✓✓✓✓BotTrack [20]✓✓✓✓✓✓✓✓Peerclean [40]✓✓✓✓✓✓✓✓✓[41]✓✓✓✓✓✓✓✓Our method✓✓✓✓✓✓✓✓✓The above methods based on graphics or groups are affected by the complete visibility of bot communication relations, so they are not suitable for unstructured P2P botnet detection. Therefore, in this research, based on the complex network theory, a P2P cluster shared neighbor graph is constructed, and the SAW-based algorithm is used to distinguish communities and identify malicious communities.


## 2.1. Typical Unstructured P2P Botnet

In this section, the evolution mechanism of typical unstructured P2P botnet, such as Nugache, Sality, Kelihos, and ZeroAccess, is briefly introduced.(1)
Nugache: As is a P2P botnet based on custom protocol, Nugache first appeared in early 2006 [27], and its main purpose is DDoS attacks. Nugache botnet is one of the earliest botnets that use strong cryptography to protect their communication. Some messages are signed with 4096 bit RSA key to prevent unauthorized control, and part of peer-to-peer communication is encrypted with RSA/Rijndael hybrid scheme.(2)
Sality: The P2P version of Sality first appeared in early 2008. It uses pull-based unstructured P2P network to spread URLs, and exchange new URLs with the neighbors it frequently contacts. Sality uses a special “announcement” message to announce itself to routable peers in a P2P network. Also, it adopts a reputation mechanism, and each roamer tracks the reputation of all its neighboring peers [28].(3)
Kelihos (Also known as Hlux): Kelihos is an unstructured P2P botnet that first appeared at the end of 2010. It is mainly used for spam and ID theft [12], and it implements many backup channels. If all its peers cannot be accessed, these backup channels can be activated by bot. The DNS fast flux network is the most important module in Kelihos. Each bot contains a hard-coded domain name that points to another infected bot. The bot can be used as a seed node to restore the connection in a P2P network.(4)
ZeroAccess (also known as Sirefef): ZeroAccess is a malware download program using unstructured P2P architecture. The ZeroAccess bots periodically query neighbors for new malware payloads and use a broadcast mechanism to push the list of new peers to all bots [29]. Zombies can be quarantined by sending peer lists to the bot, which contains invalid entries with the most recent timestamp. ZeroAccess v1/v2 are two variants of ZeroAccess, which appeared in mid-2009 and early 2012, respectively. The variants are organized into seven disjoint networks.(5)
Hide & seek (also known as HNS): As a botnet of IoT, it first appeared in January 2018. HNS uses vulnerabilities for propagation (two of the vulnerabilities are specific to some IP cameras), and it aims to infect home routers [30]. HNS agent has a modular plug-in architecture that contains two payload modules: an information thief and a Monero miner module. HNS supports custom P2P protocols. The nodes are allowed to receive remote instructions from the network, and each node has a small peer list, which is maintained and updated by frequent requests for new information in P2P networksTable1 gives the communication protocol, message propagation method, communication direction, C&C architecture, and purpose of P2P botnet families [31].Table 1
Overview of P2P botnet families.
FamilyProtocolProp.Dir.C&CPurposeNugache [27]CustomGossipPullP2PD, TSality [28]CustomGossipPullP2PD, N, PKelihos [12]CustomGossipPullHybridC, D, M, N, SZeroAccess [29]CustomGossipPullP2PPHide & seek [30]CustomGossipPullP2PTThe P2P botnets listed in Table 1 have many purposes, C = Click Fraud, D = DDoS, M = Bitcoin Mining, N = Network Services, P = Pay-Per-Install, S = Spam, T = Credential Theft.


## 2.2. Related Research

Most existing botnet detection methods are based on the statistical characteristics of packets and flows [32, 33]. Packet-based detection methods perform deep packet detection on payloads, and they detect bot activities according to predefined patterns and feature codes retrieved from well-known bots. For example, BotHunter relies on Snort and has a rich signature database that can quickly and accurately discover botnet activities [25]. However, robots can avoid signature-based detection by using advanced encryption mechanisms or code obfuscation tools. Flow-based statistical features are divided into attribute features and behavior features, which are independent of payload information. The attribute features are extracted from network traffic and classified through machine learning methods, including duration, message size, port number, packet sending rate, reconnection times, etc. The behavior feature assumes that the botnets in the same botnet have similar malicious activities and communication patterns, and periodic laws of frequent bot communication are extracted according to the network traffic. Abnormal points are detected by the establishment of communication time series, and abnormal points are detected. Intelligent botnets can easily randomize these features [34], so flow-based statistical feature detection methods are often invalidated by advanced botnets. Considering the network communication in the P2P network, the network behaviors of benign and infected P2P hosts contain some similarities at the network flow (or session) level. Therefore, these methods cannot resist the coexistence of P2P malicious traffic and benign traffic.Entelechea [35] proposed to use collaborative graph mining method to identify a group of P2P bots on the hyper-traffic graph that is constructed based on network traffic function (i.e., hourly traffic and duration of each traffic). They thought that P2P botnet flows often have small sizes and long duration.Peerhunter [36] suggested using host-level community behavior analysis to detect P2P botnets, but this study did not consider that P2P botnets and legitimate P2P applications could run on the same set of hosts.Botgrep [37] illustrated a method to detect structured P2P botnets by NetFlow communication graph, and this method converges fast from random walk to a stable distribution. Also, the accuracy of this method was further improved by recursively applying Metropolis–Hastings sampling and using a modified version of the SybilInfer clustering algorithm. The algorithm treated each subgraph as a social media graph, and it treated suspicious bots as Sybil nodes according to the probability model of P2P nodes. Tests of the method were performed on many synthetic structured P2P topologies, including Kademlia and Chord, and high-precision results were achieved. However, this method could only identify structured communication subgraphs, but it failed to determine which subgraphs contain P2P botnets.BotTrack [20] constructed a communication diagram to the host from NetFlow records. Then, a modified version of PageRank algorithm was used to determine the hub and permission level for each node. It was believed that the infected machines in structured P2P botnets showed distinguishable connection patterns, and well-connected host groups had similar hub and privilege-level values. These data points form a compact group that could be extracted with DBSCAN. Also, by adjusting PageRank weight, the previous knowledge about infection could be used to improve the accuracy of the algorithm. The method was also tested on Kademlia and Chord.Zhang et al. [38] established a scalable system for stealth P2P-botnet detection. In the waiting stage, the system could detect stealth P2P botnets without knowing the existing malicious behaviors in advance. By extracting P2P traffic fingerprint for analysis, it could work when botnet traffic overlaps with legitimate P2P traffic on the same host. However, the experimental dataset in the study was slightly biased and less challenging. For example, in the dataset, the number of bots was twice the number of legitimate P2P hosts, making it easier for bots to form clusters than legitimate P2P hosts.Daneshgar and Abbaspour [39] expounded a new P2P botnet detection framework based on two-stage sequential pattern mining (SPM), which was resilient to existing legitimate P2P traffic. First, the P2P management flow of hosts was filtered and transformed into a sequence database. Then, the frequent behaviors of hosts were extracted by using the SPM method. The framework could detect P2P bots in the presence of legitimate P2P traffic, and the detection accuracy reaches 99.2%.Peerclean [40] presented a method to detect P2P bots in network traffic by using dynamic group behavior analysis (DGBA) and machine learning. In the training stage, network traffic is transformed into Netflow, and some connection functions were extracted by DGBA from each traffic to construct an SVM classifier. In the detection stage, abnormal behaviors were searched from each unmarked cluster to capture P2P robots. This method could produce high-precision results on unstructured botnets (including Sality and ZeroAccess). However, since the method depends on the kernel function used for SVM, the constructed classifier might be overfitted.Coskun et al. [26] put forward a detection method especially for unstructured P2P botnets. The method first establishes the interconnection graph of the entire network and then tries to use “seeds” (known bots) to identify other botnets. The authors believed that although the hosts in botnets were not directly connected with each other, they should at least share the same communication partners. Also, these interconnected hosts communicate with fewer than k hosts between them. Based on this, an interconnection graph should be obtained, in which the nodes stand for each host, and an edge between two nodes indicates that these hosts were interconnected. Then, the “dye pumping” algorithm was introduced. However, this method was impractical because “seeds” cannot be obtained in advance.Muhs [41] transformed NetFlow recorded data into a weighted adjacency matrix that was suitable for random walk. Based on the BotGrep algorithm, random walk was used to detect unstructured P2P botnets on large communication graphs, and DBSCAN clustering was used for classification. It should be noted that this method was proposed under the assumption of a complete communication graph.Comparative analysis of typical P2P botnet detection methods is shown in Table2.Table 2
Comparative analysis of typical P2P botnet detection methods.
Communication protocolDetection contentflexibilityAdopt technologyData setAccuracy of detectionGeneral p2pStructured P2PUnstructured P2PFlow-basedBinary-basedEncrypted channelOverlapping flowCommunity discoverygraph-based methodMachine learningstatistical analysisStormWaledacSalityKelihosZeroAccessZeusKademliaChordKoorde95%–100%90%–94%Peerhunter [36]✓✓✓✓✓✓✓✓✓✓[38]✓✓✓✓✓✓✓✓✓[39]✓✓✓✓✓✓✓✓Botgrep [37]✓✓✓✓✓✓✓BotTrack [20]✓✓✓✓✓✓✓✓Peerclean [40]✓✓✓✓✓✓✓✓✓[41]✓✓✓✓✓✓✓✓Our method✓✓✓✓✓✓✓✓✓The above methods based on graphics or groups are affected by the complete visibility of bot communication relations, so they are not suitable for unstructured P2P botnet detection. Therefore, in this research, based on the complex network theory, a P2P cluster shared neighbor graph is constructed, and the SAW-based algorithm is used to distinguish communities and identify malicious communities.


## 3. Proposed Methodology

### 3.1. Framework

P2P botnets, like other legitimate P2P networks, communicate with each other under a distributed topology structure. As shown in Figure1, botnets can be mixed in normal P2P communication with strong concealment. Besides, unstructured P2P botnets usually communicate in a random way, which results in a more complex structure and makes it difficult to obtain a complete topology.Figure 1
Botnet construction mechanism.The existing studies show that the nodes with shared neighbors usually belong to a certain community. The basic idea of Peertrap, a detection framework proposed in this paper, is to build a shared neighbor graph by extracting P2P network traffic with Apache Flink big data platform [42], and distinguish communities by using community detection algorithm in complex network theory to distinguish benign communities and malicious communities. The richer the network traffic data, the wider the range of botnets detected, and the more botnet nodes detected, but the topology of the botnet cannot be obtained.The Peertrap framework consists of five modules: data preprocessing, feature extraction, shared neighbor graph construction, community detection and classification. The workflow diagram is shown in Figure2 and briefly described as follows:Step 1: Data Preprocessing. By using the ARGUS tool, the original network traffic in pcap format is converted into that in Netflow format.Step 2: P2P Feature Extraction. Based on the Apache Flink big data platform, the Netflow format data are converted into quintuple cluster flow according to the upstream and downstream traffic, and destination IP address. Then, address dispersion characteristics are selected to filter out P2P clusters.Step 3: Shared neighbor graph Construction. The Jaccard coefficient is used to calculate the weight of shared neighbor nodes among P2P clusters, and the shared neighbor graph is constructed to reveal the properties between clusters.Step 4: Community Detection. The SAW algorithm is adopted to process the shared neighbor graph. Every vertex in the network will not be revisited during the same walk. The matrix between vertices is generated, and PCA is used to calculate the Bray–Curtis dissimilarity. Meanwhile, hierarchical clustering is used to calculate the similarity of nodes, and community mining is conducted to cluster similar P2P nodes.Step 5: Classification. By using the community attributes, mean address distribution degree, and mean closeness degree, the community detection results are classified to filter out botnets.Figure 2
Schematic diagram of Peertrap framework.


### 3.2. Data Preprocessing

This paper aims to analyze the P2P zombie behavior in network traffic. The captured original network traffic is in pcap format, which consists of packets. The data volume is huge, and the statistical characteristics are not obvious. Therefore, the ARGUS traffic processing tool is used to convert the network traffic into a bidirectional Netflow format. The related concepts are introduced as follows [43]:Definition 1.
Data packet. A data packet is defined asp=xp,sp,tp, where xp represents the five-tuple src_IP,src_port,dst_IP,dst_port,protocol, i.e., source IP, source port, destination IP, destination port, and transmission protocol, sp represents the size of the data packet, and tp represents the start time of the data packet. The original network traffic can be expressed as a set of data packets p=p1,p2,…,pn, where n is the number of data packets contained in the dataset.Definition 2.
Netflow. The data flow is composed of data packets with the same five-tuple data packets in network traffic, and the data packets in the flow are arranged in time sequence, i.e.,p1=x1,s1,t1,p2=x2,s2,t2,…,pi=xi,si,ti, x1=x2=…=xi,t1<t2<…ti. A single data stream is defined as f=xf,sf,df,tf,, where xf represents the same 5-tuple of all data packets in the stream, and sf represents the sum of the sizes of all data packets in the stream; df indicates the duration of the stream; tf indicates the start time of the first data packet in the stream. Data flow can be divided into unidirectional flow and bidirectional flow. This paper adopts Bi-Netflow, which contains bidirectional statistical characteristics from source IP to destination IP address and from destination IP to source IP address.


### 3.3. P2P Feature Extraction

Compared with traditional Internet traffic, P2P traffic exhibits the following characteristics.(1)
Longer duration. P2P applications usually open long-term links through which they exchange management data packets for high-speed transmission and spend a long time online.(2)
Consistency of upstream and downstream traffic. P2P hosts usually communicate with peers directly through IP addresses without a query from DNS servers [36]. In P2P traffic, upstream (outgoing) and downstream (incoming) traffic usually has the same characteristics.(3)
The destination addresses are widely distributed. P2P network is decentralized. To maintain their connectivity and maximize resource access from other peer nodes, P2P hosts usually communicate with peers distributed in a large range of physical networks, which leads to these peers being widely distributed in many different subnets. The subnet number can be identified by BGP prefix of the IP addresses [44].Currently, the commonly used P2P traffic identification and classification methods can be divided into three categories: port-based method, signature-based method, and traffic feature-based method. Because the message load is usually encrypted, botnets usually use port changes to avoid tracking, and they even use common ports to disguise and avoid firewall restrictions. So, they are not suitable for new P2P traffic extraction.Based on the characteristics of network traffic, this paper proposes a set of P2P traffic filtering rules and relies on Apache Flink big data framework for data processing. By implementing the Google Dataflow streaming computing model, Flink achieves high throughput, low latency, and high performance. Also, Flink can process network data quickly based on the fixed cache and supports lightweight distributed Snapshot for fault-tolerant management. Besides, Flink periodically maintains the state persistently through distributed snapshots and checkpoints, so that the correct result can be calculated even when the system is shut down or abnormal. The Flink big data platform is a scalable data processing framework. With the rapid increase of network traffic data, the platform can quickly and dynamically expand the cluster, adapt to traffic changes, and has good scalability.Definition 3.
Flow Filter. Botnet detection methods using traffic statistics have been widely investigated. This paper performs filtering based on the average number of bytes-per-packets (BPP) [44] in each P2P management flow, which can be specifically divided into upstream outgoing flows and downstream incoming flows. For the Netflow of each host, extraction is performed on the same four-tuple, and the five-tuple cluster are obtained, i.e., FFH=srcip,protocol,bppout,bppin,dstip1,dstip2,dstip3,…,dstipn. srcip refers to the source IP address, protocol refers to TCP or UDP protocol; bppout refers to BPP from source IP to destination IP; bppin refers to BPP from destination IP to source IP, and dd refers to the set of destination IP addresses in arrays dstip1,dstip2,dstip3,…,dstipn. In this step, damaged data packets and data packets related to IP (Internet protocol) broadcast, ICMP (Internet control message protocol) and ARP (address resolution protocol) requests will be filtered out and discarded from the network traffic. Traffic filtering is only performed for the transport layer protocol of TCP/UDP.Definition 4.
Address distribution. For each flow filterFF, the set of destination IP addresses is dstip1,dstip2,dstip3,…,dstipn. Literature [39] pointed out that the destination IP addresses of P2P traffic are distributed in extensive networks, and the number of BGP prefixes of P2P traffic is higher than that of non-P2P traffic. Besides, two IP addresses with different /16IP prefixes may reside in different networks owned by different organizations, providing a good approximation of the network boundary [44]. Therefore, for each dstip set, the δad threshold is defined to calculate the number of different /16IP affixes, and the P2P traffic can be filtered out.
According to the above definitions and feature selection process, the P2P flows of H denoted asFFp2pH are defined in formula (1):(1)FFp2pH=flow∈FFH|srcip,protocol,bppout,bppin,ddH≥δad.


### 3.4. Shared Neighbor Graph Construction

Through the P2P filtering module, the five-tuples of P2P traffic are obtained. However, botnet traffic is often mixed with legitimate P2P traffic, which is more concealed. In fact, for P2P botnets with unstructured topology, bots will randomly select peers to communicate with; thus, its own network topology is not easy to detect. However, within any given time range, the P2P bot program pair will communicate with at least one public external bot [26], indicating that a pair of bots in the network are likely to contact each other. By contrast, a pair of legitimate hosts may have little contact with each other, because almost all hosts communicate with some popular servers (such as google.com and facebook.com).With incomplete information, that is, without knowing the internal structure of the network, the system is deployed at the network boundary to capture the network flow between internal hosts and external hosts. The definition of robots sharing similar neighbors in unstructured P2P botnets is theoretically proved in detail in reference [26]. The above analysis shows that the nodes with shared neighbors usually belong to a certain community. Therefore, a shared neighbor graph can be constructed to mine the P2P clusters in the same community and further separate botnets from legitimate applications.In this paper, an iterative algorithm using Jaccard coefficients is proposed to calculate the confidence between FFs as weights. Then, a shared neighbor graph is constructed for community discovery.Definition 5.
Degree of Confidence. As shown in formula (3), for every pair of FFp2pHi, and FFp2pHj, the Jaccard coefficient is used to calculate the confidence Simp2pHi,Hj between ddi,j, that is, the proportion of shared neighbor nodes. If the confidence is greater than 0, there are trusted neighbor nodes between two clusters. Also, the confidence threshold is denoted as δsim, which represents the degree of closeness between two clusters. The larger the value, the higher the degree of association, and it is possible for Hi and Hj to be in the same community.
The degree of confidence calculation formula is shown in formula (2):(2)Simp2pHi,Hj=ddHi∩ddHjddHi∪ddHj0,ifddHi∩ddHj≠0,ddHi∩ddHj≠0,0,otherwise.Definition 6.
Shared Neighbor Graph (SNG). SNG is defined as an undirected graphG=V,E, where V is a set of vertices that refers to the numbers of FFp2pH, and E is a set of edges that refers to two P2P clusters with a confidence greater than 0, as edges, i.e., < FFp2pHi, FFp2pHj >. wij refers to the confidence of two P2P clusters, i.e., Simp2pHi,Hj. The entire topological structure of the SNG graph is represented by two-dimensional adjacency matrix. At this stage, the SNG contains benign and malignant communities.
The SNG is shown in Figure3. Hosts A, B, and C are vertices in the interconnected graph, in which Host A and Host B share a neighbor node X with a weight of 1, and Host B and Host C share two neighbor nodes Y and Z with a weight of 2.
Detailed steps of SNG construction algorithm are shown in Algorithm 1:
The schematic diagram of SNG of P2P network with 500 nodes produced by Gephi tool [45] is shown in Figure 4.Algorithm 1: Shared Neighbor Graph Construction.
Input:FFp2pHi, FFp2pHjOutput:G = {V, E}, wijforFFp2pHi, FFp2pHj∈FFp2p docalculatewij⟵Simp2pHi,Hjend forforFFp2pHi∈FFp2p dovi ⟵FFp2pHiV⟵V∪viend forforFFp2pHi, FFp2pHj∈FFp2p doifSimp2pHi,Hj>0 theneij⟵vi,vjE ⟵E∪eijend ifend forreturnG = {V, E}, wijFigure 3
Schematic diagram of shared neighbor graph.Figure 4
Schematic diagram of SNG generation in P2P network with 500 nodes.


### 3.5. Community Detection Models

P2P botnets do not have a centralized C&C server infrastructure. The attacker can submit the command to any zombie, which then spreads the command to the infected companion. As the number of connections between bots increases, the command propagation delay will decrease. Meanwhile, infected hosts can be shut down, and the malware can be removed, leading to node and edge failures that botnets must endure. To achieve high resilience against node failures and targeted attacks while allowing rapid command propagation, P2P botnet nodes must be well connected with each other. Due to this rapid mixing characteristic, the state probability quality of a botnet node is closer to the stationary probability distribution than the rest of the infected slow mixture in the network [46]. This is more obvious when many random walks are performed on the graph.Random walk algorithm is a flexible tool for network characterization and exploration. In every step, the walkers visit a vertex randomly selected from all neighbors of the current vertex. So, this algorithm is very suitable for extracting fast mixed P2P subgraphs from large communication graphs. SAW is a highly efficient random walk algorithm [47] because every vertex in the network will not be revisited during the same walk, and the walker retains the memory of the path taken. After the same type of P2P small groups are obtained from the SNG, the main purpose of this module is to generate the matrix between vertices based on SAW and reduce the dimension of the generated matrix by PCA. Meanwhile, the Bray–Curtis dissimilarity is calculated, and nodes’ similarity is calculated through hierarchical clustering. Then, communities’ mining is performed, and the largest communities of the same type of P2P network generated by different P2P applications are clustered.Definition 7.
Transition probabilitypij. Define Mi as the number of steps starts from vertex i, and mij is the number of visits to vertex j before stopping. The transition probability is defined as the proportion of walkers who start SAW at vertex i and reach vertex j before stopping, as shown in formula (3). It is a measure of the difficulty for a walker from vertex i to vertex j, and it has nothing to do with topological distance.(3)pij=mijMi.Definition 8.
Average lengthlij. First, the number of steps for each walker to reach j is calculated, i.e., lijwik (where wik is the k−th walk from i). Then, all wik walkers passing through j are summed up. The definition of average length lij is as shown in formula (4). The average number of steps of i,j pairs reveals the topological distance of the network.(4)<lij>=1mij∑lijwik.Definition 9.
Vertex matrixfij. fij can be defined by combing the above two definitions in a metric, and the definition is shown in formula (5). The reciprocal of the average length quantifies the “closeness” of the two vertices.(5)fij=pijlij.
Note that wheni≠j, 0≤fij≤1 generally, fij≠fji. If no pedestrians pass j from i, fij=0.
In fact, a group of vertices in the same community tends to have relatively high values ofpij, and low values of lij, because their density is higher at the inner edges of the community. Also, the walkers starting from different vertices in the same community tend to have similar views on the network.
To obtain the best community distribution, we use PCA [48] to extract the relevant information from the matrix and reduce the dimension. The linear transformation adopted is F˜=P⋅F, where P is the new basis composed of the principal components of F and F˜ is the data projected into P. Based on this, the most relevant components (feature vectors related to higher eigenvalues) are selected to classify vertices.
To compare the two verticesi and j, the Bray–Curtis dissimilarity is calculated [49] in formula (6).(6)di,j=∑k=1nf˜i,k−f˜j,k∑k=1nf˜i,k+f˜j,k.
If two verticesi and j belong to the same community, they are similar from the point of view of their network, so di,j tends to be small.
The modularityQ is taken as community metric [50, 51]. Denoting the score of edges in group i as eii and the score of edges connected to the vertices in community i as ai, the modularity can be written as (7)Q=∑i=1nceii−ai2.


### 3.6. Classification

This paper defines two community attributes to distinguish between benign and malignant communities.Definition 10.
Mean Address Distribution degree (MAD)αmad. It reflects P2P behavior. To maintain its connectivity and maximize resource access from other peer nodes, bots are usually distributed in a large range of physical networks, so malicious communities usually have a high MAD value [44]. The average dispersion of the community is defined as the number of prefixes of any numbered cluster /16 divided by /32, and then the mean value is calculated. The calculation formula is described as (8)αmad=dd/16Hi/dd/32Hin.Definition 11.
Mean closeness degree (MCD)βmcd. It reflects the trust degree of shared neighbor nodes in the internal communication of the community. Also, it reflects the communication behavior between nodes. The average compactness within a community is the sum of the weights between the edges of any node pair j,k divided by the total number of edges between them. The MCD of a pair of bots in the same botnet is much higher than that of a P2P botnet. The average MCD between all host pairs in a network community is defined as βmcd, and the calculation formula of βmcd is shown as in formula (9).(9)βmcd=∑Simp2pHj,Hknn−1/2.
After community detection, larger communities of similar P2P applications are obtained, but a mixture of benign and malignant communities is involved. This module uses two community attribute values to filter out zombie communities. The introduction of the parameter values can refer to experiment 4.4.
Common variables and settings in our system are shown in Table3.Table 3
The variables and sets used in the system model.
Variables or setsDescriptionsrcipSource IP addressdstipDestination IP addressprotocolConnection protocol (udp, tcp, icmp, igmp, etc.)xpRepresents the five-tuplesrc_IP,src_port,dst_IP,dst_port,protocolsp,The size of data packettp,The start time of data packetBPPThe average number of bytes-per-packets in each P2P management flowbppoutBPP from source IP to destination IPbppinBPP from destination IP to source IPddThe set of destination IP addresses in arraysdstip1,dstip2,dstip3,…,dstipnFFHThe five-tuple cluster extracted according to the same four-tuple for netflow of each hostδadThe number of different/16IP affixeswijThe confidence of two P2P clustersMiThe number of steps starts from verteximijThe number of visits to vertexj before stoppingpijTransition probabilitywikThek−th walk from ilijAverage length for allwik walkers passing through j are summed upfijVertex matrix, transition probability times, the reciprocal of the average lengthdi,jBray–Curtis dissimilarity of two verticesi and jeiiThe score of edges in groupiaiThe score of edges connected to the vertices in communityiQModularityαmadThe average dispersion of the community is defined as, the number of prefixes of any numbered cluster/16 divided by/32, and then the mean value is calculatedβmcdThe mean closeness degree reflects the trust degree of shared neighbor nodes in the internal communication of the community


### 3.7. Time Complexity

In this section, we calculate the time complexity of the system. In the second step, the time complexity of the p2p cluster extraction isOs, and s is the number of the Netflow network traffic. In the third step, the SNG diagram is generated, and the weight of the two clusters is required, and the time complexity is Os2. In the fourth step, the time complexity of the saw community is calculated as Omn2+n3logn, where the number of random walks is Omn2, where m is the number of walker per vertex with at most n hops each. Each time the hierarchical clustering and the module of time complexity are On3logn, which are required to be repeated n times.Thus, the overall time complexity of the system isOn3logn+s2.


## 3.1. Framework

P2P botnets, like other legitimate P2P networks, communicate with each other under a distributed topology structure. As shown in Figure1, botnets can be mixed in normal P2P communication with strong concealment. Besides, unstructured P2P botnets usually communicate in a random way, which results in a more complex structure and makes it difficult to obtain a complete topology.Figure 1
Botnet construction mechanism.The existing studies show that the nodes with shared neighbors usually belong to a certain community. The basic idea of Peertrap, a detection framework proposed in this paper, is to build a shared neighbor graph by extracting P2P network traffic with Apache Flink big data platform [42], and distinguish communities by using community detection algorithm in complex network theory to distinguish benign communities and malicious communities. The richer the network traffic data, the wider the range of botnets detected, and the more botnet nodes detected, but the topology of the botnet cannot be obtained.The Peertrap framework consists of five modules: data preprocessing, feature extraction, shared neighbor graph construction, community detection and classification. The workflow diagram is shown in Figure2 and briefly described as follows:Step 1: Data Preprocessing. By using the ARGUS tool, the original network traffic in pcap format is converted into that in Netflow format.Step 2: P2P Feature Extraction. Based on the Apache Flink big data platform, the Netflow format data are converted into quintuple cluster flow according to the upstream and downstream traffic, and destination IP address. Then, address dispersion characteristics are selected to filter out P2P clusters.Step 3: Shared neighbor graph Construction. The Jaccard coefficient is used to calculate the weight of shared neighbor nodes among P2P clusters, and the shared neighbor graph is constructed to reveal the properties between clusters.Step 4: Community Detection. The SAW algorithm is adopted to process the shared neighbor graph. Every vertex in the network will not be revisited during the same walk. The matrix between vertices is generated, and PCA is used to calculate the Bray–Curtis dissimilarity. Meanwhile, hierarchical clustering is used to calculate the similarity of nodes, and community mining is conducted to cluster similar P2P nodes.Step 5: Classification. By using the community attributes, mean address distribution degree, and mean closeness degree, the community detection results are classified to filter out botnets.Figure 2
Schematic diagram of Peertrap framework.


## 3.2. Data Preprocessing

This paper aims to analyze the P2P zombie behavior in network traffic. The captured original network traffic is in pcap format, which consists of packets. The data volume is huge, and the statistical characteristics are not obvious. Therefore, the ARGUS traffic processing tool is used to convert the network traffic into a bidirectional Netflow format. The related concepts are introduced as follows [43]:Definition 1.
Data packet. A data packet is defined asp=xp,sp,tp, where xp represents the five-tuple src_IP,src_port,dst_IP,dst_port,protocol, i.e., source IP, source port, destination IP, destination port, and transmission protocol, sp represents the size of the data packet, and tp represents the start time of the data packet. The original network traffic can be expressed as a set of data packets p=p1,p2,…,pn, where n is the number of data packets contained in the dataset.Definition 2.
Netflow. The data flow is composed of data packets with the same five-tuple data packets in network traffic, and the data packets in the flow are arranged in time sequence, i.e.,p1=x1,s1,t1,p2=x2,s2,t2,…,pi=xi,si,ti, x1=x2=…=xi,t1<t2<…ti. A single data stream is defined as f=xf,sf,df,tf,, where xf represents the same 5-tuple of all data packets in the stream, and sf represents the sum of the sizes of all data packets in the stream; df indicates the duration of the stream; tf indicates the start time of the first data packet in the stream. Data flow can be divided into unidirectional flow and bidirectional flow. This paper adopts Bi-Netflow, which contains bidirectional statistical characteristics from source IP to destination IP address and from destination IP to source IP address.


## 3.3. P2P Feature Extraction

Compared with traditional Internet traffic, P2P traffic exhibits the following characteristics.(1)
Longer duration. P2P applications usually open long-term links through which they exchange management data packets for high-speed transmission and spend a long time online.(2)
Consistency of upstream and downstream traffic. P2P hosts usually communicate with peers directly through IP addresses without a query from DNS servers [36]. In P2P traffic, upstream (outgoing) and downstream (incoming) traffic usually has the same characteristics.(3)
The destination addresses are widely distributed. P2P network is decentralized. To maintain their connectivity and maximize resource access from other peer nodes, P2P hosts usually communicate with peers distributed in a large range of physical networks, which leads to these peers being widely distributed in many different subnets. The subnet number can be identified by BGP prefix of the IP addresses [44].Currently, the commonly used P2P traffic identification and classification methods can be divided into three categories: port-based method, signature-based method, and traffic feature-based method. Because the message load is usually encrypted, botnets usually use port changes to avoid tracking, and they even use common ports to disguise and avoid firewall restrictions. So, they are not suitable for new P2P traffic extraction.Based on the characteristics of network traffic, this paper proposes a set of P2P traffic filtering rules and relies on Apache Flink big data framework for data processing. By implementing the Google Dataflow streaming computing model, Flink achieves high throughput, low latency, and high performance. Also, Flink can process network data quickly based on the fixed cache and supports lightweight distributed Snapshot for fault-tolerant management. Besides, Flink periodically maintains the state persistently through distributed snapshots and checkpoints, so that the correct result can be calculated even when the system is shut down or abnormal. The Flink big data platform is a scalable data processing framework. With the rapid increase of network traffic data, the platform can quickly and dynamically expand the cluster, adapt to traffic changes, and has good scalability.Definition 3.
Flow Filter. Botnet detection methods using traffic statistics have been widely investigated. This paper performs filtering based on the average number of bytes-per-packets (BPP) [44] in each P2P management flow, which can be specifically divided into upstream outgoing flows and downstream incoming flows. For the Netflow of each host, extraction is performed on the same four-tuple, and the five-tuple cluster are obtained, i.e., FFH=srcip,protocol,bppout,bppin,dstip1,dstip2,dstip3,…,dstipn. srcip refers to the source IP address, protocol refers to TCP or UDP protocol; bppout refers to BPP from source IP to destination IP; bppin refers to BPP from destination IP to source IP, and dd refers to the set of destination IP addresses in arrays dstip1,dstip2,dstip3,…,dstipn. In this step, damaged data packets and data packets related to IP (Internet protocol) broadcast, ICMP (Internet control message protocol) and ARP (address resolution protocol) requests will be filtered out and discarded from the network traffic. Traffic filtering is only performed for the transport layer protocol of TCP/UDP.Definition 4.
Address distribution. For each flow filterFF, the set of destination IP addresses is dstip1,dstip2,dstip3,…,dstipn. Literature [39] pointed out that the destination IP addresses of P2P traffic are distributed in extensive networks, and the number of BGP prefixes of P2P traffic is higher than that of non-P2P traffic. Besides, two IP addresses with different /16IP prefixes may reside in different networks owned by different organizations, providing a good approximation of the network boundary [44]. Therefore, for each dstip set, the δad threshold is defined to calculate the number of different /16IP affixes, and the P2P traffic can be filtered out.
According to the above definitions and feature selection process, the P2P flows of H denoted asFFp2pH are defined in formula (1):(1)FFp2pH=flow∈FFH|srcip,protocol,bppout,bppin,ddH≥δad.


## 3.4. Shared Neighbor Graph Construction

Through the P2P filtering module, the five-tuples of P2P traffic are obtained. However, botnet traffic is often mixed with legitimate P2P traffic, which is more concealed. In fact, for P2P botnets with unstructured topology, bots will randomly select peers to communicate with; thus, its own network topology is not easy to detect. However, within any given time range, the P2P bot program pair will communicate with at least one public external bot [26], indicating that a pair of bots in the network are likely to contact each other. By contrast, a pair of legitimate hosts may have little contact with each other, because almost all hosts communicate with some popular servers (such as google.com and facebook.com).With incomplete information, that is, without knowing the internal structure of the network, the system is deployed at the network boundary to capture the network flow between internal hosts and external hosts. The definition of robots sharing similar neighbors in unstructured P2P botnets is theoretically proved in detail in reference [26]. The above analysis shows that the nodes with shared neighbors usually belong to a certain community. Therefore, a shared neighbor graph can be constructed to mine the P2P clusters in the same community and further separate botnets from legitimate applications.In this paper, an iterative algorithm using Jaccard coefficients is proposed to calculate the confidence between FFs as weights. Then, a shared neighbor graph is constructed for community discovery.Definition 5.
Degree of Confidence. As shown in formula (3), for every pair of FFp2pHi, and FFp2pHj, the Jaccard coefficient is used to calculate the confidence Simp2pHi,Hj between ddi,j, that is, the proportion of shared neighbor nodes. If the confidence is greater than 0, there are trusted neighbor nodes between two clusters. Also, the confidence threshold is denoted as δsim, which represents the degree of closeness between two clusters. The larger the value, the higher the degree of association, and it is possible for Hi and Hj to be in the same community.
The degree of confidence calculation formula is shown in formula (2):(2)Simp2pHi,Hj=ddHi∩ddHjddHi∪ddHj0,ifddHi∩ddHj≠0,ddHi∩ddHj≠0,0,otherwise.Definition 6.
Shared Neighbor Graph (SNG). SNG is defined as an undirected graphG=V,E, where V is a set of vertices that refers to the numbers of FFp2pH, and E is a set of edges that refers to two P2P clusters with a confidence greater than 0, as edges, i.e., < FFp2pHi, FFp2pHj >. wij refers to the confidence of two P2P clusters, i.e., Simp2pHi,Hj. The entire topological structure of the SNG graph is represented by two-dimensional adjacency matrix. At this stage, the SNG contains benign and malignant communities.
The SNG is shown in Figure3. Hosts A, B, and C are vertices in the interconnected graph, in which Host A and Host B share a neighbor node X with a weight of 1, and Host B and Host C share two neighbor nodes Y and Z with a weight of 2.
Detailed steps of SNG construction algorithm are shown in Algorithm 1:
The schematic diagram of SNG of P2P network with 500 nodes produced by Gephi tool [45] is shown in Figure 4.Algorithm 1: Shared Neighbor Graph Construction.
Input:FFp2pHi, FFp2pHjOutput:G = {V, E}, wijforFFp2pHi, FFp2pHj∈FFp2p docalculatewij⟵Simp2pHi,Hjend forforFFp2pHi∈FFp2p dovi ⟵FFp2pHiV⟵V∪viend forforFFp2pHi, FFp2pHj∈FFp2p doifSimp2pHi,Hj>0 theneij⟵vi,vjE ⟵E∪eijend ifend forreturnG = {V, E}, wijFigure 3
Schematic diagram of shared neighbor graph.Figure 4
Schematic diagram of SNG generation in P2P network with 500 nodes.


## 3.5. Community Detection Models

P2P botnets do not have a centralized C&C server infrastructure. The attacker can submit the command to any zombie, which then spreads the command to the infected companion. As the number of connections between bots increases, the command propagation delay will decrease. Meanwhile, infected hosts can be shut down, and the malware can be removed, leading to node and edge failures that botnets must endure. To achieve high resilience against node failures and targeted attacks while allowing rapid command propagation, P2P botnet nodes must be well connected with each other. Due to this rapid mixing characteristic, the state probability quality of a botnet node is closer to the stationary probability distribution than the rest of the infected slow mixture in the network [46]. This is more obvious when many random walks are performed on the graph.Random walk algorithm is a flexible tool for network characterization and exploration. In every step, the walkers visit a vertex randomly selected from all neighbors of the current vertex. So, this algorithm is very suitable for extracting fast mixed P2P subgraphs from large communication graphs. SAW is a highly efficient random walk algorithm [47] because every vertex in the network will not be revisited during the same walk, and the walker retains the memory of the path taken. After the same type of P2P small groups are obtained from the SNG, the main purpose of this module is to generate the matrix between vertices based on SAW and reduce the dimension of the generated matrix by PCA. Meanwhile, the Bray–Curtis dissimilarity is calculated, and nodes’ similarity is calculated through hierarchical clustering. Then, communities’ mining is performed, and the largest communities of the same type of P2P network generated by different P2P applications are clustered.Definition 7.
Transition probabilitypij. Define Mi as the number of steps starts from vertex i, and mij is the number of visits to vertex j before stopping. The transition probability is defined as the proportion of walkers who start SAW at vertex i and reach vertex j before stopping, as shown in formula (3). It is a measure of the difficulty for a walker from vertex i to vertex j, and it has nothing to do with topological distance.(3)pij=mijMi.Definition 8.
Average lengthlij. First, the number of steps for each walker to reach j is calculated, i.e., lijwik (where wik is the k−th walk from i). Then, all wik walkers passing through j are summed up. The definition of average length lij is as shown in formula (4). The average number of steps of i,j pairs reveals the topological distance of the network.(4)<lij>=1mij∑lijwik.Definition 9.
Vertex matrixfij. fij can be defined by combing the above two definitions in a metric, and the definition is shown in formula (5). The reciprocal of the average length quantifies the “closeness” of the two vertices.(5)fij=pijlij.
Note that wheni≠j, 0≤fij≤1 generally, fij≠fji. If no pedestrians pass j from i, fij=0.
In fact, a group of vertices in the same community tends to have relatively high values ofpij, and low values of lij, because their density is higher at the inner edges of the community. Also, the walkers starting from different vertices in the same community tend to have similar views on the network.
To obtain the best community distribution, we use PCA [48] to extract the relevant information from the matrix and reduce the dimension. The linear transformation adopted is F˜=P⋅F, where P is the new basis composed of the principal components of F and F˜ is the data projected into P. Based on this, the most relevant components (feature vectors related to higher eigenvalues) are selected to classify vertices.
To compare the two verticesi and j, the Bray–Curtis dissimilarity is calculated [49] in formula (6).(6)di,j=∑k=1nf˜i,k−f˜j,k∑k=1nf˜i,k+f˜j,k.
If two verticesi and j belong to the same community, they are similar from the point of view of their network, so di,j tends to be small.
The modularityQ is taken as community metric [50, 51]. Denoting the score of edges in group i as eii and the score of edges connected to the vertices in community i as ai, the modularity can be written as (7)Q=∑i=1nceii−ai2.


## 3.6. Classification

This paper defines two community attributes to distinguish between benign and malignant communities.Definition 10.
Mean Address Distribution degree (MAD)αmad. It reflects P2P behavior. To maintain its connectivity and maximize resource access from other peer nodes, bots are usually distributed in a large range of physical networks, so malicious communities usually have a high MAD value [44]. The average dispersion of the community is defined as the number of prefixes of any numbered cluster /16 divided by /32, and then the mean value is calculated. The calculation formula is described as (8)αmad=dd/16Hi/dd/32Hin.Definition 11.
Mean closeness degree (MCD)βmcd. It reflects the trust degree of shared neighbor nodes in the internal communication of the community. Also, it reflects the communication behavior between nodes. The average compactness within a community is the sum of the weights between the edges of any node pair j,k divided by the total number of edges between them. The MCD of a pair of bots in the same botnet is much higher than that of a P2P botnet. The average MCD between all host pairs in a network community is defined as βmcd, and the calculation formula of βmcd is shown as in formula (9).(9)βmcd=∑Simp2pHj,Hknn−1/2.
After community detection, larger communities of similar P2P applications are obtained, but a mixture of benign and malignant communities is involved. This module uses two community attribute values to filter out zombie communities. The introduction of the parameter values can refer to experiment 4.4.
Common variables and settings in our system are shown in Table3.Table 3
The variables and sets used in the system model.
Variables or setsDescriptionsrcipSource IP addressdstipDestination IP addressprotocolConnection protocol (udp, tcp, icmp, igmp, etc.)xpRepresents the five-tuplesrc_IP,src_port,dst_IP,dst_port,protocolsp,The size of data packettp,The start time of data packetBPPThe average number of bytes-per-packets in each P2P management flowbppoutBPP from source IP to destination IPbppinBPP from destination IP to source IPddThe set of destination IP addresses in arraysdstip1,dstip2,dstip3,…,dstipnFFHThe five-tuple cluster extracted according to the same four-tuple for netflow of each hostδadThe number of different/16IP affixeswijThe confidence of two P2P clustersMiThe number of steps starts from verteximijThe number of visits to vertexj before stoppingpijTransition probabilitywikThek−th walk from ilijAverage length for allwik walkers passing through j are summed upfijVertex matrix, transition probability times, the reciprocal of the average lengthdi,jBray–Curtis dissimilarity of two verticesi and jeiiThe score of edges in groupiaiThe score of edges connected to the vertices in communityiQModularityαmadThe average dispersion of the community is defined as, the number of prefixes of any numbered cluster/16 divided by/32, and then the mean value is calculatedβmcdThe mean closeness degree reflects the trust degree of shared neighbor nodes in the internal communication of the community


## 3.7. Time Complexity

In this section, we calculate the time complexity of the system. In the second step, the time complexity of the p2p cluster extraction isOs, and s is the number of the Netflow network traffic. In the third step, the SNG diagram is generated, and the weight of the two clusters is required, and the time complexity is Os2. In the fourth step, the time complexity of the saw community is calculated as Omn2+n3logn, where the number of random walks is Omn2, where m is the number of walker per vertex with at most n hops each. Each time the hierarchical clustering and the module of time complexity are On3logn, which are required to be repeated n times.Thus, the overall time complexity of the system isOn3logn+s2.


## 4. Experimental Evaluation

### 4.1. Experimental Environment

Hardware environment: The experiment is conducted on a computer equipped with Intel (r) core (TM) i7-8250u CPU @ 1.80ghz, 16 GB ram, and Geforce gtx650ti. The computer runs Windows 10 enterprise operating system. Apache Flink big data platform.Software environment: The software used in the experiment includes ARGUS [52], python3.6, Apache Flink, and igarph. Igraph [53] is a powerful tool for complex network analysis. It is an open source tool library written in pure C language and provides some interfaces to R and python.


### 4.2. Dataset Description

To verify the effectiveness of the proposed Peertrap botnet detection system, a large amount of real network traffic is required. This article uses data collection, which are mainly three types of network tracking data, including P2P bot tracking, P2P legitimate application tracking, and network background traffic. The hourly trace information for 24 hours is listed in Table4.(1)
P2P botnet traffic. It contains 24-hour activity tracks of three unstructured P2P botnets, i.e., Sality, Kelihos, and ZeroAccess. The P2P botnets data are derived from the third-party cooperative organization in cooperation project of the research group. Literature [54] provides binary sample files of Sality, Kelihos, and ZeroAccess in the dataset in this article. The initial preset nodes were 21, which they had carried out harmlessly by processing of the Sality, Kelihos, and ZeroAccess botnet virus, and only run the infection spread and basic communication interactions of bots programs, preventing all their malicious functions and attacks. After the traffic was acquired, preprocessing, such as filtering and de-duplication, was performed, and a 24-hour P2P botnet traffic trace was obtained, as shown in Table 4.(2)
P2P legitimate applications. The legal P2P network track was obtained from the University of Georgia [55], which included 16 eMule hosts, 16 FrostWire hosts, 14 uTorrent hosts, and 14 Vuze hosts. A group of continuous 24-hour network tracks for each host was randomly selected, as shown in Table 5.(3)
Network background traffic. The dataset [56] in the traffic archive of the MAWI working group was used as our background network tracking. The dataset contains approximately 407,523,221 flows and 48,607,304 IP addresses.Table 4
Trace of P2P Botnetst.
TraceDur (hour)# of Bots# of Flows# of dstIPsP2P bots24216430921178815Table 5
Trace of P2P legitimate applications.
TraceDur (hour)# of Hosts# of Flows# of dstIPsP2P legitimate applications2460270137775182365


### 4.3. Performance Metrics

We use the following terms for determining the quality of the classification models:The ground-truth obtained from the mapping process enables us to gather a confusion matrix from the dataset withTrue Positive (TP): It is judged as botnet traffic, but it is also botnet trafficFalse Positive (FP): It is judged as botnet traffic, but it is actually normal trafficFalse Negative (FN): It is judged as normal traffic, but it is actually botnet trafficTrue Negative (TN): It is judged as normal flow, but in fact it is also normal flowThis data are presented as precision, recall, F1 score, and MCC as given in formulas10–13.(10)Precision=TPTP+FP,(11)Recall=TPTP+FN,(12)F1−score=2∗Precision∗RecallPrecision+Recall,(13)MCC=TP×TN−FP×FNTP+FPTP+FNTN+FPTN+FN.


### 4.4. Evaluation Results

The core problem solved by this system is for unstructured P2P botnets. Under the condition of incomplete information, normal traffic and malicious traffic are mixed, by constructing a shared neighbor graph and using community mining algorithms in complex networks to gather similar communities. Finally, two thresholds are used to filter out the botnet communities.First, the ARGUS tool is used to process and cluster the network tracking into a TCP/UDP stream in the 5-tuple format. Then, statistical analysis is conducted on the upstream and downstream traffic in the Netflow, and the Apache Flink big data platform is used to generate P2P clusters. The P2P cluster feature extraction is extracted according to the quadruple, and the formula isFFH=srcip,protocol,bppout,bppin,dstip1,dstip2,dstip3,…,dstipn, the specific data example is shown in Table 6. Next, P2P network traffic detection is evaluated with different δad. The threshold of P2P traffic address dispersion is much larger than that of non-P2P traffic.Table 6
P2P cluster data example.
srcipprotocolbppoutbppindd18.36.210.99udp12790[218.152.128.184,173.98.117.196,80.191.78.123, 187.138.130.187, …,211.89.118.208,]106.200.51.115udp1065[215.133.155.64, 186.235.143.64, 41.151.159.67, 80.191.78.123,18.223.140.54, …,175.224.146.65]203.114.222.8tcp6956[186.160.46.205, 24.81.204.107, 179.148.55.33, 182.20.49.204, 160.251.108.44, …,40.240.72.44]Subsequently, the SNG is constructed, and the SAW algorithm is used for community mining. The community attributes are evaluated, and due to more frequent communication, the mean address distribution degreeαmad and mean closeness degree βmcd of the bot behavior are higher than the values of benign P2P communities. The comparison results of detection for different thresholds are listed in Table 7. The experimental results show that the precision reaches 100% under the best parameter setting of δad = 20, αmad = 0.5, and βmcd = 0.2. In the dataset, botnets account for less traffic information, and background traffic accounts for the majority. To better measure the data, we also calculated the MCC indicators under different threshold adjustments.Table 7
The botnet detection and comparison results of different thresholds.
δadαmadβmcdPrecisionRecallF1MCC200.30.050.15909110.274510.3988590.10.24418610.3925230.4941490.150.87510.9333330.9354140.211110.2511110.311110.3511110.411110.4511110.511110.50.050.21428610.3529410.4629070.10.58333310.7368420.7637620.150.87510.9333330.9354140.211110.2511110.311110.3511110.411110.4511110.51111300.30.050.22826110.3716810.4777640.10.42857110.60.6546520.1511110.211110.2511110.311110.3511110.411110.4511110.511110.50.050.3387110.5060240.5819860.111110.1511110.211110.2511110.311110.3511110.411110.4511110.51111400.30.050.29577510.4565220.543850.10.56756810.7241380.753370.1511110.211110.2511110.311110.3511110.411110.4511110.511110.50.050.38888910.560.6236080.111110.1511110.211110.2511110.311110.3511110.411110.4511110.51111The comparative experimental results are shown in Figures5–7.Figure 5
Shows the changes of accuracy, recall rate, F1 score, and MCC withβmcd when δad = 20 αmad = 0.3 and 0.5, respectively.
(a)(b)Figure 6
Changes of accuracy, recall rate, F1 score, and MCC withβmcd when δad = 30 αmad = 0.3 and 0.5, respectively.
(a)(b)Figure 7
Changes of accuracy, recall rate, F1 score, and MCC withβmcd when δad = 40 αmad = 0.3 and 0.5, respectively.
(a)(b)


### 4.5. Comparison with respect to Existing Studies

In Ref. [40], Yan et al. proposed a Peerclean method to detect P2P bots in network traffic by using dynamic group behavior analysis (DGBA) and machine learning. The method consists of two phases: training and detection. In the training phase, the network traffic is aggregated into Netflow. Also, some connection functions are extracted from each flow, such as cluster connection, shared neighbor feature, important connection feature, time feature, etc. Then, the supervised machine learning (SVM) method is used to train these feature vector pairs for classification. The real situation is obtained by simulating benign P2P applications in a laboratory environment and running P2P tracking of Sality, Kelihos, and ZeroAccess botnets in a controlled environment. In the detection phase, to extract benign and malicious hosts participating in P2P networks, the similarity propagation algorithm specified in Ref. [57] is used to calculate the group characteristics of aggregate flows, and then these characteristics are clustered. Finally, the bots are extracted under many function-specific thresholds. Machine learning and flow feature analysis should be combined in Peerclean.In this paper, the experimental results of the shared neighbor feature, important connection feature, and all features are selected and compared with those of our method. The comparison results of various indicators with Peerclean are listed in Table8.Table 8
Comparative analysis of experimental results.
MethodPrecision (%)Recall (%)F1 (%)Shared neighbor feature92.768.891.7Significant connection feature91.866.790All the features98.810097.2Our method100100100Analysis of experimental results: users of the peerclean method need to set many variables. Also, the classifier may overfit because it depends on the kernel function used for SVM. The method proposed in this paper achieves better results based on behavior analysis and avoids the overfitting problem in machine learning.


### 4.6. P2P Botnet Detection on Classic Datasets

In this section, experiments are conducted on P2P botnets in classic datasets.The traffic data of the structured P2P botnet is taken from the famous CTU-13 dataset [58]. CTU-13 was established by the Malware Capture project organized by CTU University (Czech Institute of Technology Prague), which is especially responsible for collecting and capturing various types of malicious traffic and normal traffic. The dataset consists of 13 scenarios, as listed in Table 9. This study considers the 12th scenario, i.e., the P2P botnet NSIS scenario, which contains botnet traffic as well as normal traffic and background traffic. The MD5 of NSIS.ay is eaf85db9898d3c9101fd5fcfa4ac80e4, and the duration is 1 hour, 13 minutes, and 21 seconds. In the NetFlows of this scenario, botnet flows is 2143 (0.65%), normal flows is 7628 (2.34%), c&c flows is 25 (0.007%), and the background flows is 315675 (96.99%).Table 9
Amount of data on each botnet scenario.
IDDuration#Packets#NetFlowsSize (GB)Bot#Bots16.1571,971,4822,824,63752Neris124.2171,851,3001,808,12360Neris1366.85167,730,3954,710,639121Rbot144.2162,089,1351,121,07753Rbot1511.634,481,167129,83337.6Virut162.1838,764,357558,92030Menti170.387,467,139114,0785.8Sogou1819.5155.207,7992,954,231123Murlo195.18115,415,3212,753,88594Neris10104.7590,389.7821,309,79273Rbot10110.266,337,202107,2525.2Rbot3121.2113,212,268325,4728.3NSIS.ay31316.3650,888,2561,925,15034Virut1Our parameter settings and experimental results on this dataset are shown in Table10:Table 10
Parameter setting and experimental results.
P2P botParameter settingIndicatorNSISδadαmadβmcdPrecisionRecallF1200.80.05199.7%99.8%Experimental results show that this method is also suitable for NSIS P2P botnet detection.To verify the effectiveness of Peertrap, three baseline methods are evaluated, including Bot-DL [59], Graph-ML [60], and Bot-AHGCN [61]. For the baseline method, the source code published by the authors in reference [61] is used with the same parameter setting. The analysis of experimental results is listed in Table 11.Table 11
Comparative analysis of experimental results.
MethodPrecision (%)Recall (%)F1 (%)Bot-DL94.2191.3492.75Graph-ML92.3187.5088.48Bot-AHGCN98.2498.3198.27Our method10099.799.8Bot-DL is the most advanced botnet detection method based on deep learning. It applies deep neural network to feature modeling of bots by analyzing a single network flow.Graph-ML combines graph theory and machine learning to solve the problem of botnet detection. It uses supervised and unsupervised machine learning to build a two-stage graph-based robot detection system.Bot-AHGCN models fine-grained network flow objects (such as IP, response) as multi-attribute heterogeneous graphs. Also, by introducing meaningful meta-paths and meta-graphs to provide better interpretability, Bot-AHGCN converts Bot detection problems into classification tasks on semi-supervised node graphs.


### 4.7. Possible Evasion Techniques

Botnet detection is essentially a confrontational problem. Botnet guards always attempt to enhance their botnets to evade detection systems. Although our proposed method can resist the existence of legitimate P2P traffic, there are still some evasion techniques that may be abused by opponents to evade detection systems.(1)
Traffic confusion is performed by an adversarial sample generation technique to avoid detection. P2P bots can imitate the behavior of legitimate P2P traffic in terms of packet size and timing behavior.(2)
Avoidance by restricting the activities of bots. Many P2P botnet detection schemes rely on the community activities of bots revealed through different community detection and clustering techniques. However, opponents can attempt to recruit P2P bots in different networks to limit the number of bots in each network.


### 4.8. Possible Limitations

Botnets are diverse and constantly evolving. Although many technologies or algorithms have been used to solve botnet detection problems and have achieved good results, no algorithm can solve all botnet problems. Although this method has some advantages, it still has the following limitations:(1)
The IOT device has the characteristics of small size and small memory capacity. They are resource-constrained and have relatively small connectivity. The concept of mutual/common neighbors may not be applicable to parasitic P2P botnets such as Hajime, Mozi, and HNS on the Internet of Things environment.(2)
The life cycle of a botnet mainly includes the stages of Propagation, Rally, Interaction, and Malicious Activities. Our data collection is carried out on the infected host. The botnet and the detection process are at the stage of interaction and attack between the botnet and other external networks. Therefore, early detection during the propagation and internal addressing phase of botnets is also a challenging task.


## 4.1. Experimental Environment

Hardware environment: The experiment is conducted on a computer equipped with Intel (r) core (TM) i7-8250u CPU @ 1.80ghz, 16 GB ram, and Geforce gtx650ti. The computer runs Windows 10 enterprise operating system. Apache Flink big data platform.Software environment: The software used in the experiment includes ARGUS [52], python3.6, Apache Flink, and igarph. Igraph [53] is a powerful tool for complex network analysis. It is an open source tool library written in pure C language and provides some interfaces to R and python.


## 4.2. Dataset Description

To verify the effectiveness of the proposed Peertrap botnet detection system, a large amount of real network traffic is required. This article uses data collection, which are mainly three types of network tracking data, including P2P bot tracking, P2P legitimate application tracking, and network background traffic. The hourly trace information for 24 hours is listed in Table4.(1)
P2P botnet traffic. It contains 24-hour activity tracks of three unstructured P2P botnets, i.e., Sality, Kelihos, and ZeroAccess. The P2P botnets data are derived from the third-party cooperative organization in cooperation project of the research group. Literature [54] provides binary sample files of Sality, Kelihos, and ZeroAccess in the dataset in this article. The initial preset nodes were 21, which they had carried out harmlessly by processing of the Sality, Kelihos, and ZeroAccess botnet virus, and only run the infection spread and basic communication interactions of bots programs, preventing all their malicious functions and attacks. After the traffic was acquired, preprocessing, such as filtering and de-duplication, was performed, and a 24-hour P2P botnet traffic trace was obtained, as shown in Table 4.(2)
P2P legitimate applications. The legal P2P network track was obtained from the University of Georgia [55], which included 16 eMule hosts, 16 FrostWire hosts, 14 uTorrent hosts, and 14 Vuze hosts. A group of continuous 24-hour network tracks for each host was randomly selected, as shown in Table 5.(3)
Network background traffic. The dataset [56] in the traffic archive of the MAWI working group was used as our background network tracking. The dataset contains approximately 407,523,221 flows and 48,607,304 IP addresses.Table 4
Trace of P2P Botnetst.
TraceDur (hour)# of Bots# of Flows# of dstIPsP2P bots24216430921178815Table 5
Trace of P2P legitimate applications.
TraceDur (hour)# of Hosts# of Flows# of dstIPsP2P legitimate applications2460270137775182365


## 4.3. Performance Metrics

We use the following terms for determining the quality of the classification models:The ground-truth obtained from the mapping process enables us to gather a confusion matrix from the dataset withTrue Positive (TP): It is judged as botnet traffic, but it is also botnet trafficFalse Positive (FP): It is judged as botnet traffic, but it is actually normal trafficFalse Negative (FN): It is judged as normal traffic, but it is actually botnet trafficTrue Negative (TN): It is judged as normal flow, but in fact it is also normal flowThis data are presented as precision, recall, F1 score, and MCC as given in formulas10–13.(10)Precision=TPTP+FP,(11)Recall=TPTP+FN,(12)F1−score=2∗Precision∗RecallPrecision+Recall,(13)MCC=TP×TN−FP×FNTP+FPTP+FNTN+FPTN+FN.


## 4.4. Evaluation Results

The core problem solved by this system is for unstructured P2P botnets. Under the condition of incomplete information, normal traffic and malicious traffic are mixed, by constructing a shared neighbor graph and using community mining algorithms in complex networks to gather similar communities. Finally, two thresholds are used to filter out the botnet communities.First, the ARGUS tool is used to process and cluster the network tracking into a TCP/UDP stream in the 5-tuple format. Then, statistical analysis is conducted on the upstream and downstream traffic in the Netflow, and the Apache Flink big data platform is used to generate P2P clusters. The P2P cluster feature extraction is extracted according to the quadruple, and the formula isFFH=srcip,protocol,bppout,bppin,dstip1,dstip2,dstip3,…,dstipn, the specific data example is shown in Table 6. Next, P2P network traffic detection is evaluated with different δad. The threshold of P2P traffic address dispersion is much larger than that of non-P2P traffic.Table 6
P2P cluster data example.
srcipprotocolbppoutbppindd18.36.210.99udp12790[218.152.128.184,173.98.117.196,80.191.78.123, 187.138.130.187, …,211.89.118.208,]106.200.51.115udp1065[215.133.155.64, 186.235.143.64, 41.151.159.67, 80.191.78.123,18.223.140.54, …,175.224.146.65]203.114.222.8tcp6956[186.160.46.205, 24.81.204.107, 179.148.55.33, 182.20.49.204, 160.251.108.44, …,40.240.72.44]Subsequently, the SNG is constructed, and the SAW algorithm is used for community mining. The community attributes are evaluated, and due to more frequent communication, the mean address distribution degreeαmad and mean closeness degree βmcd of the bot behavior are higher than the values of benign P2P communities. The comparison results of detection for different thresholds are listed in Table 7. The experimental results show that the precision reaches 100% under the best parameter setting of δad = 20, αmad = 0.5, and βmcd = 0.2. In the dataset, botnets account for less traffic information, and background traffic accounts for the majority. To better measure the data, we also calculated the MCC indicators under different threshold adjustments.Table 7
The botnet detection and comparison results of different thresholds.
δadαmadβmcdPrecisionRecallF1MCC200.30.050.15909110.274510.3988590.10.24418610.3925230.4941490.150.87510.9333330.9354140.211110.2511110.311110.3511110.411110.4511110.511110.50.050.21428610.3529410.4629070.10.58333310.7368420.7637620.150.87510.9333330.9354140.211110.2511110.311110.3511110.411110.4511110.51111300.30.050.22826110.3716810.4777640.10.42857110.60.6546520.1511110.211110.2511110.311110.3511110.411110.4511110.511110.50.050.3387110.5060240.5819860.111110.1511110.211110.2511110.311110.3511110.411110.4511110.51111400.30.050.29577510.4565220.543850.10.56756810.7241380.753370.1511110.211110.2511110.311110.3511110.411110.4511110.511110.50.050.38888910.560.6236080.111110.1511110.211110.2511110.311110.3511110.411110.4511110.51111The comparative experimental results are shown in Figures5–7.Figure 5
Shows the changes of accuracy, recall rate, F1 score, and MCC withβmcd when δad = 20 αmad = 0.3 and 0.5, respectively.
(a)(b)Figure 6
Changes of accuracy, recall rate, F1 score, and MCC withβmcd when δad = 30 αmad = 0.3 and 0.5, respectively.
(a)(b)Figure 7
Changes of accuracy, recall rate, F1 score, and MCC withβmcd when δad = 40 αmad = 0.3 and 0.5, respectively.
(a)(b)


## 4.5. Comparison with respect to Existing Studies

In Ref. [40], Yan et al. proposed a Peerclean method to detect P2P bots in network traffic by using dynamic group behavior analysis (DGBA) and machine learning. The method consists of two phases: training and detection. In the training phase, the network traffic is aggregated into Netflow. Also, some connection functions are extracted from each flow, such as cluster connection, shared neighbor feature, important connection feature, time feature, etc. Then, the supervised machine learning (SVM) method is used to train these feature vector pairs for classification. The real situation is obtained by simulating benign P2P applications in a laboratory environment and running P2P tracking of Sality, Kelihos, and ZeroAccess botnets in a controlled environment. In the detection phase, to extract benign and malicious hosts participating in P2P networks, the similarity propagation algorithm specified in Ref. [57] is used to calculate the group characteristics of aggregate flows, and then these characteristics are clustered. Finally, the bots are extracted under many function-specific thresholds. Machine learning and flow feature analysis should be combined in Peerclean.In this paper, the experimental results of the shared neighbor feature, important connection feature, and all features are selected and compared with those of our method. The comparison results of various indicators with Peerclean are listed in Table8.Table 8
Comparative analysis of experimental results.
MethodPrecision (%)Recall (%)F1 (%)Shared neighbor feature92.768.891.7Significant connection feature91.866.790All the features98.810097.2Our method100100100Analysis of experimental results: users of the peerclean method need to set many variables. Also, the classifier may overfit because it depends on the kernel function used for SVM. The method proposed in this paper achieves better results based on behavior analysis and avoids the overfitting problem in machine learning.


## 4.6. P2P Botnet Detection on Classic Datasets

In this section, experiments are conducted on P2P botnets in classic datasets.The traffic data of the structured P2P botnet is taken from the famous CTU-13 dataset [58]. CTU-13 was established by the Malware Capture project organized by CTU University (Czech Institute of Technology Prague), which is especially responsible for collecting and capturing various types of malicious traffic and normal traffic. The dataset consists of 13 scenarios, as listed in Table 9. This study considers the 12th scenario, i.e., the P2P botnet NSIS scenario, which contains botnet traffic as well as normal traffic and background traffic. The MD5 of NSIS.ay is eaf85db9898d3c9101fd5fcfa4ac80e4, and the duration is 1 hour, 13 minutes, and 21 seconds. In the NetFlows of this scenario, botnet flows is 2143 (0.65%), normal flows is 7628 (2.34%), c&c flows is 25 (0.007%), and the background flows is 315675 (96.99%).Table 9
Amount of data on each botnet scenario.
IDDuration#Packets#NetFlowsSize (GB)Bot#Bots16.1571,971,4822,824,63752Neris124.2171,851,3001,808,12360Neris1366.85167,730,3954,710,639121Rbot144.2162,089,1351,121,07753Rbot1511.634,481,167129,83337.6Virut162.1838,764,357558,92030Menti170.387,467,139114,0785.8Sogou1819.5155.207,7992,954,231123Murlo195.18115,415,3212,753,88594Neris10104.7590,389.7821,309,79273Rbot10110.266,337,202107,2525.2Rbot3121.2113,212,268325,4728.3NSIS.ay31316.3650,888,2561,925,15034Virut1Our parameter settings and experimental results on this dataset are shown in Table10:Table 10
Parameter setting and experimental results.
P2P botParameter settingIndicatorNSISδadαmadβmcdPrecisionRecallF1200.80.05199.7%99.8%Experimental results show that this method is also suitable for NSIS P2P botnet detection.To verify the effectiveness of Peertrap, three baseline methods are evaluated, including Bot-DL [59], Graph-ML [60], and Bot-AHGCN [61]. For the baseline method, the source code published by the authors in reference [61] is used with the same parameter setting. The analysis of experimental results is listed in Table 11.Table 11
Comparative analysis of experimental results.
MethodPrecision (%)Recall (%)F1 (%)Bot-DL94.2191.3492.75Graph-ML92.3187.5088.48Bot-AHGCN98.2498.3198.27Our method10099.799.8Bot-DL is the most advanced botnet detection method based on deep learning. It applies deep neural network to feature modeling of bots by analyzing a single network flow.Graph-ML combines graph theory and machine learning to solve the problem of botnet detection. It uses supervised and unsupervised machine learning to build a two-stage graph-based robot detection system.Bot-AHGCN models fine-grained network flow objects (such as IP, response) as multi-attribute heterogeneous graphs. Also, by introducing meaningful meta-paths and meta-graphs to provide better interpretability, Bot-AHGCN converts Bot detection problems into classification tasks on semi-supervised node graphs.


## 4.7. Possible Evasion Techniques

Botnet detection is essentially a confrontational problem. Botnet guards always attempt to enhance their botnets to evade detection systems. Although our proposed method can resist the existence of legitimate P2P traffic, there are still some evasion techniques that may be abused by opponents to evade detection systems.(1)
Traffic confusion is performed by an adversarial sample generation technique to avoid detection. P2P bots can imitate the behavior of legitimate P2P traffic in terms of packet size and timing behavior.(2)
Avoidance by restricting the activities of bots. Many P2P botnet detection schemes rely on the community activities of bots revealed through different community detection and clustering techniques. However, opponents can attempt to recruit P2P bots in different networks to limit the number of bots in each network.


## 4.8. Possible Limitations

Botnets are diverse and constantly evolving. Although many technologies or algorithms have been used to solve botnet detection problems and have achieved good results, no algorithm can solve all botnet problems. Although this method has some advantages, it still has the following limitations:(1)
The IOT device has the characteristics of small size and small memory capacity. They are resource-constrained and have relatively small connectivity. The concept of mutual/common neighbors may not be applicable to parasitic P2P botnets such as Hajime, Mozi, and HNS on the Internet of Things environment.(2)
The life cycle of a botnet mainly includes the stages of Propagation, Rally, Interaction, and Malicious Activities. Our data collection is carried out on the infected host. The botnet and the detection process are at the stage of interaction and attack between the botnet and other external networks. Therefore, early detection during the propagation and internal addressing phase of botnets is also a challenging task.


## 5. Conclusion and Future Work

P2P botnets mix malicious traffic with legitimate traffic, which is one of the most significant challenges in detecting botnets. The topology of unstructured P2P botnets is complex, and the community structure is not obvious enough, thus increasing the difficulty of botnet detection. Exploiting complex network theory under the condition of incomplete topological information, this paper proposes a botnet detection framework based on random walks and designs a rule using big data platforms to distinguish P2P and non-P2P traffic. Besides, the Jaccard coefficient is used to calculate the P2P cluster confidence to construct a shared neighbor graph, and SAW is used to distinguish between benign and malignant communities. An evaluation experiment was conducted on the dataset, and the experimental results were compared with those of the existing method on an unstructured dataset. The results show that P2P bots can be detected with high accuracy in the presence of legitimate P2P traffic. In future research, the advanced hybrid P2P type botnet detection will be investigated. Since this type of botnet involves central servers and key nodes, the network behavior and topology are more complex. Meanwhile, botnet detection in the case of traffic confusion using adversarial sample generation technology will be explored.



---

*Source: 9900396-2022-02-08.xml*