# Entity-Centric Fully Connected GCN for Relation Classification
## 2 Materials and Methods
### Attention Module  
The attention mechanism can automatically calculate the importance of different words in a sentence. Transformer model [24] shows that \"self-attention\" can achieve better results in sequence coding, it can compute the correlation between words in the sentence. In a text sequence, the importance of words is often different. Especially entity nodes are important for relation classification. \"Self-attention\" can capture the internal structure of a sentence by learning the interaction between words in the sentence, and  
Figure 1: FCGCN model architecture diagram. The left part shows the overall architecture of FCGCN, and the right part shows FCGCN’s entity-centric fully connected logical adjacency matrix, where \(x_{o}\), \(x_{s}\) represents the physical node and \(x_{r}\) represents the root node. obtain the attention score between each word in the sentence. Therefore, more semantic information between words can be obtained through the \"self-attention\" mechanism. In this paper, we use multi-head attention to distinguish the importance of words in a sentence. The input given in this paper is the vector \(\mathbf{s}=\left[e_{1}^{w},e_{2}^{w},...,e_{n}^{w}\right]\) after word embedding. In the self-attention process, we set query, key and value are equal to \(s\), which is \(\mathbf{Q}=\mathbf{K}=\mathbf{V}=\mathbf{s}\in\mathbb{R}^{n\times d_{w}}\). We get the hidden representation of the sentence as follows. \[\mathbf{H}=Attention(\mathbf{Q},\mathbf{K},\mathbf{V})=soft\max\left(\frac{\mathbf{Q}\mathbf{K}^{T}}{ \sqrt{d_{w}}}\right)\mathbf{V} \tag{5}\]  
In the multi-head self-attention module, we pass Q, K, and V through a linear transformation, and input them into the scaled dot-roduct attention, and perform the above steps \(h\) times. Therefore, the calculation of the i-th head is as follows  
\[\mathbf{H}_{i}=Attention\Big{(}\mathbf{Q}\mathbf{W}_{i}^{Q},\mathbf{K}\mathbf{W}_{i}^{K},\mathbf{V} \mathbf{W}_{i}^{V}\Big{)}(i=1,....,h) \tag{6}\]  
where \(\mathbf{W}_{i}^{Q},\mathbf{W}_{i}^{K},\mathbf{W}_{i}^{V}\in\mathbb{R}^{m\times d_{m}}\), \(d_{m}=m/h\). The output of the multi-head attention layer is obtained by concatenating the outputs of \(h\) heads, which is then input into a linear transformation to generate the final attention values. \[\mathbf{M}=MultiHead(\mathbf{Q},\mathbf{K},\mathbf{V})=\mathbf{W}^{o}Concat(\mathbf{H}_{i},....,\mathbf{H }_{h}) \tag{7}\]  
where \(\mathbf{M}=[\mathbf{m}_{1},\mathbf{m}_{2},...,\mathbf{m}_{n}]\) is the hidden representations sentences generated by multi-head self-attention, \(\mathbf{W}^{o}\in\mathbb{R}^{d_{t}\times d_{t}}\) represents the weight matrix. To make better use of the information captured by multi-head self-attention, the same feedforward network is employed to the \(\mathbf{m}_{i}\) in the multi-head self-attention output \(\mathbf{M}\). The calculation method is as follows  
\[FFNN(\mathbf{m}_{i})=p(\mathbf{m}_{i}\mathbf{M}_{1}+\mathbf{\beta}_{1})\mathbf{M}_{2}+\mathbf{\beta}_ {2} \tag{8}\]  
where \(\mathbf{M}_{1}\in\mathbb{R}^{d\times d_{t}}\), \(\mathbf{M}_{2}\in\mathbb{R}^{d_{q}\times d}\) represents the conversion matrix. \(\mathbf{\beta}_{1}\) and \(\mathbf{\beta}_{2}\) represents the bias, and \(d_{q}\) represents the size of the hidden layer. Inspired by [PERSON] et al. [24], we integrate the multi-head self-attention and the output of the feedforward layer through the residual connection [25], and then perform layer normalization [26]. The calculation method is as follows  
\[\mathbf{C}=LayerNorm\big{(}\mathbf{\hat{M}}+FFNN\big{(}\mathbf{\hat{M}}\big{)}\big{)} \tag{9}\]  
where \(\mathbf{C}=\{\mathbf{c}_{1},\mathbf{c}_{2},...,\mathbf{c}_{n}\}\in\mathbb{R}^{n\times d}\), \(\mathbf{\hat{M}}=LayerNorm(\mathbf{M}+\mathbf{s})\) represents the residual connection between sentence embedding and multi-head self-attention layer output.