# An Attention-Based Graph Neural Network for Spam Bot Detection in Social Networks
## 3 The Proposed Approach
### GAT-Based Spam Bot Detection Model
The nonlinear activation of LeakyReLU is added:  
\[e_{ij}^{f}=\text{LeakyReLU}(a^{T}[W^{f}h_{i}^{f}\left\|W^{f}h_{j}^{f})\right.) \tag{3}\]  
where \(\parallel\) represents the splicing operation, and \(a^{T}\) is the weight vector after transposition, which is used to parameterize the attention mechanism \(att\). Equation (3) indicates the importance of node \(j\) to node \(i\) without considering the information of graph structure. Therefore, this attention mechanism is introduced into the graph structure through masked attention. Thus, we only need to calculate the \(e_{ij}^{f}\) of node \(j\in N_{i}^{f}\), where \(N_{i}^{f}\) is the neighbor of node \(i\) (including itself) in the user-follow subgraph. In order to allocate the weight better, we need to  
Figure 3: The framework of the proposed model. normalize the correlation between the current central node and all its neighbors, and get the weight coefficient \(\alpha_{ij}^{f}\) by the SoftMax function:  
\[\alpha_{ij}^{f}=\text{softmax}_{j}(e_{ij}^{f})=\frac{\exp(e_{ij}^{f})}{\sum \limits_{k\in N_{i}^{f}}\exp(e_{ik}^{f})} \tag{4}\]  
By processing the above formula, the sum of the weight coefficients of all the neighbors of the current central node is 1. After full expansion, the coefficients calculated by the attention mechanism can be expressed as follows:  
\[\alpha_{ij}^{f}=\frac{\exp(\text{LeakyReLU}(a^{T}[W^{f}h_{i}^{f}\Big{\|}W^{ f}h_{j}^{f}]))}{\sum\limits_{k\in N_{i}^{f}}\exp(\text{LeakyReLU}(a^{T}[W^{f}h_{i}^{f} \Big{\|}W^{f}h_{k}^{f}]))} \tag{5}\]  
It can be seen that the weight coefficients of \((i,j)\) depend on their features. In addition, the weight coefficient calculated by Equation (5) is asymmetric. Therefore, their contributions are different from each other. Note also that the weight coefficients \(\alpha_{ij}^{f}\) are asymmetric, which means that they contribute differently to each other. The normalized attention coefficient is used to calculate the linear combination of corresponding features as the final output feature of each node:  
\[h_{i}^{f^{\prime}}=\sigma(\sum\limits_{j\in N_{i}^{f}}\alpha_{ij}^{f}W^{f}h_{ j}^{f}) \tag{6}\]  
where \(\sigma\) denotes ELU nonlinear activation. The user-retweet subgraph is treated in the same way. However, when we deal with the connection between the user-follow and user-retweet subgraphs simultaneously, we usually do not know the importance of transformation information, which comes from different subgraphs. We cannot simply average the information together, but adaptively estimate the learning process of different subgraphs. Then, the embedding of the node \(i\) can be aggregated by the projection features of neighbors with corresponding coefficients, as follows:  
\[h_{i}^{\prime}=\sigma(\frac{1}{2}(\beta\sum\limits_{j\in N_{i}^{f}}\alpha_{ ij}^{f}W^{f}h_{j}+(1-\beta)\sum\limits_{j\in N_{i}^{r}}\alpha_{ij}^{r}W^{r}h_{ j})) \tag{7}\]  
where \(\beta\) is a free parameter to be estimated, the superscripts \(f\) and \(r\) represent the user-follow subgraph and user-retweet subgraphs, respectively. In order to stabilize the learning process of self-attention, we use the multi-head attention mechanism which is used in the GAT [22] paper. We use \(K\) independent attention heads to perform the transformation of Equation (7), then use the average value and delay the application of the final nonlinearity to obtain the following output feature representation:  
\[\hat{h}_{i}^{\prime}=\sigma(\frac{1}{K}\sum\limits_{k=1}^{K}(\frac{1}{2}( \beta\sum\limits_{j\in N_{i}^{f}}\alpha_{ij}^{f}W^{f}h_{j}+(1-\beta)\sum \limits_{j\in N_{i}^{r}}\alpha_{ij}^{r}W^{r}h_{j}))) \tag{8}\]  
where the \(K\) is the total number of times the attention mechanism is executed. In our experiment, to optimize the task of node classification, we integrate the representation of nodes into the full connection layer, including SoftMax for prediction, to infer the classification label.