# JARUA: Joint Embedding of Attributes and Relations for User Alignment across Social Networks
## 4 Methodology
### Attention-Based Structure Embedding
#### 4.3.2 Graph Attention Network  
GAT has been proven to be successful in many tasks in natural language processing and image processing. This study introduces the mechanism of GAT to learn alignment-oriented embeddings for linking social networks. The aim of GAT is to calculate each user's hidden representations in two networks by their user neighbors. GAT follows a self-attention strategy in its learning process.  
First, the embedding \(h_{u_{i}}\in\mathbb{R}^{1\times d_{i}}\) of each user \(u_{i}\) with dimension \(d_{s}\) is randomly initialized. Second, we average the user \(u_{i}\) with its neighbors as:  
\[h_{u_{i}}^{in}=\frac{1}{|\mathbb{N}_{i}|+1}\sum_{u_{j}\in\mathbb{N}_{i}\cup u_ {i}}h_{u_{j}} \tag{8}\]  
where \(\mathbb{N}_{i}\) is the set of \(u_{i}\)'s neighboring users. The attention coefficients can be calculated by:  
\[e_{ij}=\overrightarrow{v}^{T}\Big{[}h_{u_{i}}^{in}\Big{|}h_{u_{j}}^{in}\Big{]} \tag{9}\]  
where \(e_{ij}\) represents the weighted importance of neighboring \(u_{j}\) to \(u_{i}\), notation \(\|\) indicates the concatenate operation, and \(\overrightarrow{v}\in\mathbb{R}^{1\times 2d_{s}}\) represents the shared attention weight vector. Following [48], we do not adopt the transformation matrix \(W\) for the input feature in Equation (9), which is different from the original GAT. Third, the adjacent users of \(u_{i}\) are normalized using a softmax function and a LeakyReLU nonlinearity with a negative input slope of 0.2. Such normalization makes the coefficients between different users easy to compare, which is calculated as:  
\[\alpha_{ij}=Softmax(e_{ij})=\frac{\exp(LeakyReLU(e_{ij}))}{\sum_{k\in\mathbb{ N}_{i}}\exp(LeakyReLU(e_{ik}))} \tag{10}\]  
A nonlinear function \(ReLU(\cdot)=\max(\cdot,0)\) is applied to the combination of participating neighbors. The operation generates the output features \(h_{u_{i}}^{out}\) of each user as:  
\[h_{u_{i}}^{out}=ReLU\left(\sum_{u_{j}\in\mathbb{N}_{i}}\alpha_{ij}h_{u_{j}}^{ in}\right) \tag{11}\]  
This work also adopts the multi-head mechanism to realize a stable training process. Specifically, a total of \(Z\) independent attention heads execute the transformation of Equation (11). Consequently, the averaged features yield the following output feature:  
\[h_{u_{i}}^{out}=ReLU\left(\frac{1}{Z}\sum_{z=1}^{Z}\left[\sum_{u_{j}\in\mathbb{ N}_{i}}\alpha_{ij}^{z}h_{u_{j}}^{in}\right]\right) \tag{12}\]  
where \(z\) is the indicator of an attention head and \(\alpha_{ij}^{z}\) represents the attention coefficient of the \(z\)-th attention head. We further expand the attention mechanism to multi-hop neighboring level information by attaching more layers, forming a more global-aware representation of social networks. Let \(h_{u_{i}}^{out(0)},\ldots,h_{u_{i}}^{out(I)}\) be the output features of user from 0-th (input features) to \(l\)-th layer. The final output features \(h^{struc}_{u_{i}}\) of user \(u_{i}\) can be obtained by concatenating all the layers together as:  
\[h^{struc}_{u_{i}}=\left[h^{out(0)}_{u_{i}}\|,\ldots,\|h^{out(l)}_{u_{i}}\right] \tag{13}\]