# Targeted Sentiment Classification Based on Attentional Encoding and Graph Convolutional Networks
## 3 The Attention-Encoding-Based Graph Convolutional Network Model (AEGCN)
### Syntactic Information Encoding
#### 3.2.1 Graph Convolution Network  
Graph convolution networks [24] are particularly skilled at dealing with graph data with rich relational information. Given a graph with \(k\) nodes, an adjacency matrix \(A\in R^{k\times k}\) can be obtained by listing the graphs. For convenience, A GCN has \(L\) layers \(l\in[1,2,\cdots,L]\), where \(h_{i}^{L}\) is the final state of node \(i\). The graph convolution of a node can be described as:  
\[h_{i}^{l}=\sigma\left(\sum_{j=1}^{k}A_{ij}W^{l}h_{j}^{l-1}+b^{l}\right) \tag{7}\]  
where \(W^{l}\) is the linear transformation weight matrix, \(b^{l}\) is the offset vector, and \(\sigma\) is a nonlinear function, such as Re\(LU\). An example of a GCN layer is shown in Figure 3.  
Figure 3: An example of a GCN layer.  
First, the hidden state of the previous layer \(h_{i}^{l-1}\) undergoes a linear transformation. Then, each hidden state is shaped by the node information directly related to it to obtain the hidden state of the current layer \(h_{i}^{l}\).  
Graph convolution over dependency trees [15] was proposed by [PERSON] et al. In the process of graph convolution, each graph convolution can only encode the information of the immediate neighbor nodes. So, if a GCN has \(L\) layers, the information of a node in the graph will only be affected \(L\) times by its adjacent nodes. In view of this, syntactic constraints are added to the targets of the sentence by convoluting the syntactic dependency tree of the sentence. Thus, the syntactic distance of the corresponding descriptors can be determined. In addition, when a specific target is described by a non-consecutive word, the method can aggregate the features of non-consecutive words without missing their information. Therefore, this paper puts forward a method to include the syntactic information by combining the graph convolution over dependency trees with point-wise convolution, then encoding the syntactic information obtained by multi-head self-attention to get the final syntactic information coding. The position-aware transformation is made for \(h_{i}^{l}\) before feeding \(H^{c}\) into continuous GCN layers:  
\[s_{i}^{l}=F(h_{i}^{l}) \tag{8}\]  
where \(F\left(\cdot\right)\) is a position weight distribution function used in [11; 12] that is used to enhance the importance of the words close to a specific target. The \(F\left(\cdot\right)\) function is as follows:  
\[q_{i}=\left\{\begin{array}{l}1-\frac{\tau+1-i}{n},1\leq i\leq\tau+1\\ 0,\tau+1\leq i\leq\tau+m\\ 1-\frac{i-\tau-m}{n},\tau+m\leq i\leq n\end{array}\right. \tag{9}\]  
\[F(h_{i}^{l})=q_{i}h_{i}^{l} \tag{10}\]  
where \(q_{i}\in R\) is the position weight of the \(i\)-th token.  
Next, after constructing the dependency tree of a given sentence, AEGCN first obtains the adjacency matrix \(A\in R^{n\times n}\) according to the words in the sentence. Then, following the idea of self looping [17], each word is manually set to be adjacent to itself--that is, the diagonal value of \(A\) is \(1\). The opposite direction of a dependency architecture is also included, which means \(A_{ij}=1\) and \(A_{ji}=1\) if there is an edge going from node \(i\) to node \(j\); otherwise, \(A_{ij}=0\) and \(A_{ji}=0\) (see Figures 4 and 5). Finally, the representation of each node is updated with a graph convolution [17] with a normalization factor, as follows:  
\[\bar{h}_{i}^{l}=\sum_{j=1}^{n}A_{ij}W^{l}s_{j}^{l-1} \tag{11}\]  
\[h_{i}^{l}=\text{Re}LU(\frac{\bar{h}_{i}^{l}}{(d_{i}+1)}+b^{l}) \tag{12}\]  
where \(s_{j}^{l-1}\in R^{2d_{h}}\) is the representation of the \(j\)-th symbol, which is the output of the previous GCN layer. \(h_{i}^{l}\in R^{2d_{h}}\) is the output of the current GCN layer, and \(d_{i}=\sum_{j=1}^{n}A_{ij}\) is the degree of the \(i\)-th symbol in the tree. \(W^{l}\) and bias \(b^{l}\) are both learnable parameters.  
Figure 4: An example of syntactic dependency tree of a given sentence. PRP: pronoun, personal; RB: adverb; cop: copula; advmod: adverbial modifier.