# SybilHP: Sybil Detection in Directed Social Networks with Adaptive Homophily Prediction
## 5 Experiment  
### Experiment Setup  
**Dataset description:** In this section, we first evaluate the influence of various factors including Sybil attack strength, Sybil group scale, and label noise in SybilHP. Since our experiments require social networks with various forms and patterns, we synthesize benign and Sybil regions based on a real-world social graph. (1) Synthetic-directed Pokec  
For the sake of fairness and comparison, we adopted this directed Pokec network [43] from the official repository ([[https://github.com/binghuiwang/sybildetection](https://github.com/binghuiwang/sybildetection)]([https://github.com/binghuiwang/sybildetection](https://github.com/binghuiwang/sybildetection)) (accessed on 2 July 2022)) of for GANG [20], SybilSCAR [17] and SybilBelief [16] for robustness and performance illustration. In particular, we extract a connected component that contains 10,000 nodes and 90,065 edges from Pokec as the benign region, and then we make the Sybil region a replicate of the benign region and add (bidirectional, unidirectional) attack edges between the two regions uniformly at random. If not specified, we add 500 bidirectional edges, 1000 unidirectional Sybil-to-benign attack edges, and 100 unidirectional benign-to-Sybil compromised edges as illustrated in Figure 6. We keep 100 Sybil and 100 benign users as the training set and test on the overall social graph. Furthermore, we compare the detection performance of the proposed method and some state-of-the-art benchmark methods on a large real Twitter dataset. (2) Real-world Twitter follower-followee graph  
By the means of breadth-first search (BFS) graph traversal, we sampled a Twitter follower-followee graph with 269,640 nodes and 6,818,501 edges from [44]. The original data was crawled by [PERSON] in 2009 [44]. The graph is directed and includes 41,652,230 users and 1,468,364,884 edges. However, only 10,000 Sybils and 10,000,000 benign users are labeled, which can be used as ground truth for training and testing. The remaining nodes are treated as unknown. In other words, there are less than a quarter of labeled nodes in the original dataset. The ratio of Sybil to benign nodes is even more severely imbalanced at 1:100, which makes the discrimination of benign nodes dominate the performance evaluation results. To address the under-labeling and imbalance of the original dataset, we sampled the original dataset. The BFS starts from these labeled nodes, and we only keep those labeled neighbors until all nodes are reached. Finally, we delete those isolated nodes, and we obtain 91,263 Sybils and 178,377 benign users to form our connected and labeled social graph. We divide 9000 Sybil and 17,000 benign users (about 10%) from them as the labeled training set and test on the overall social graph. An overview of the datasets is presented in Table 1.  
Figure 6: An abridged illustration of the synthesized Pokec dataset. Compared methods: We compared SybilHP with directed graph-based method GANG [20] (including a matrix version and a basic version) and other LBP-based methods SybilSCAR [17] (SybilSCAR-D) and SybilBelief [16]. For undirected graph-based methods, we transformed the directed graph to be undirected by only keeping those bidirectional edges (which is recommended by the original paper). Note that this can cause many nodes isolated and fail to involve in the LBP process. Parameter setting: For SybilHP, we set the prior probability for labeled Sybil nodes \(p=0.9\), which was also suggested by authors of GANG, SybilSCAR, and SybilBelief; We assigned initial homophily strength parameters \(\overrightarrow{w}=0.99\), \(\overrightarrow{w_{s}}=0.77\), \(\overrightarrow{w_{b}}=0.97\), \(\overleftarrow{w_{b}}=0.73\), \(\overleftarrow{w_{s}}=0.95\), and set the number of LBP iterations \(iter=5\). For GANG, we set \(w=0.51\) adapting to Twitter as suggested by the author. Note that we also adopt the basic version of GANG with an optimized parameter (\(w=0.63\)) for our Twitter dataset.