# An Attention-Based Graph Neural Network for Spam Bot Detection in Social Networks
## 3 The Proposed Approach
### GAT-Based Spam Bot Detection Model  
In this subsection, we describe the proposed method for spam bot detection in social networks. First, since our goal is to identify spam bots in social networks, it can be formulated as a node classification problem on a directed bipartite graph with attribute nodes and edges. Therefore, we improve the GAT [22] model and propose a spam bot detection model that aggregates different neighborhood relations. The proposed method considers different types of neighborhood information. The framework of the whole model can be shown in Figure 3. By explicitly considering the neighbor relationship in the two relational subgraphs described in Section 3.2, the attention mechanism is used to aggregate neighbor nodes' features onto the central node, and the importance of different nodes is learned. Finally, the whole model is trained end-to-end. Next, we introduce how to model and describe the algorithm of the proposed method in detail. To model the graph structure, we obtain the user embedding by collecting the embedding of neighbors. However, before aggregating each node's neighbor information, we should know that in the social network structure, any two user nodes can be connected through different relationships (such as followers). In a given relationship, each user has some neighbors based on this relationship. Since each neighbor has different degrees of influence on the user, the purpose of node-level attention is to learn the importance of neighbors to this node in the social graph, and assign different attention values to all neighbors. We utilize node-level attention to express the importance of neighbor nodes relative to a node in the social graph, and finally form node embedding. Specifically, we use self-attention [29] on the nodes to learn the weights of various nodes. First, consider the user-follow subgraph. For a node \(i\in V\) in the user-follow subgraph, the feature vector corresponding to the \(l\) level is \(h_{i}^{f}=\left\{h_{1}^{f},h_{2}^{f},\ldots,h_{N}^{f}\right\}\), \(h_{i}^{f}\in R^{d(l)}\), where \(N\) is the number of nodes and \(d(l)\) is the number of features in each node. After using the attention mechanism to implement the aggregation operation, the new feature vector is represented as \({{{I_{i}^{f}}^{\prime}}^{\prime}}=\left\{{{{I_{1}^{f}}^{\prime}}^{\prime}},{{ {I_{2}^{f}}^{\prime}}^{\prime}},\ldots,{{{I_{N}^{f}}^{\prime}}^{\prime}}} \right\},{{{I_{i}^{f}}^{\prime}}}\in R^{d(l)}\). In order to get a good enough representation ability, the input features need to be transformed into higher-level features. The input feature needs at least one linear transformation to get the output feature, so we need to train a weight matrix for all nodes: \(W^{f}\in R^{d(l+1)d(l)}\), which is the relationship between the \(d(l)\) features of input and the \(d(l+1)\) features of output. Figure 2: The retweet relationship between users in social networks. Given a pair of nodes, \(i\in V\) and \(j\in V\), which are connected by a relationship \((i,j)\), we can learn the weight coefficient of \(e_{ij}^{f}\) that indicates the importance of node \(j\) to node \(i\). The importance of nodes to \((i,j)\) can be formulated as:  
\[e_{ij}^{f}=att(W^{f}h_{i}^{f},W^{f}h_{j}^{f}) \tag{1}\]  
where \(W^{f}h_{i}^{f}\) means the line transformation for the feature \(h_{i}^{f}\) of node \(i\) in the user-follow subgraph, and \(att\) is a self-attention mechanism for each node, in which a single layer feedforward neural network is used. We use LeakyReLU as the activation function:  
\[\text{LeakyReLU}(x)= \begin{cases}x,&if\ x\geq 0\\ \text{negative\_slope}*x,&\text{otherwise}\end{cases}  \tag{2}\]  
where \(x\) is the input, negative slope is a negative gradient, and the default value is \(1\times 10^{-2}\).