# KGGCN: Knowledge-Guided Graph Convolutional Networks for Distantly Supervised Relation Extraction
## 3 Methodology
### Sentence Representation
#### 3.1.3 Knowledge Attention Based GCN  
In this paper, we employ a knowledge-guided GCN model to build the context encoder and transform the sentences into low-dimension vectors, as shown in Figure 2. The GCN model is an adaptation of the convolutional neural network for encoding graphs, which encodes the dependency structure over the input sentence with efficient graph convolution operation to generate the vector representation of the sentence. Given a graph (dependency  
Figure 2: The framework of the sentence embedding module. The upper right part illustrates the generation of the word-level knowledge attention. The lower right part is the knowledge attention based graph convolutional network, which calculates the vector representation of input sentences. tree of a input sentence) with \(k\) nodes, we can represent the graph with an \(k\times k\) adjacency matrix \(\mathbf{A}\). \(A_{i,j}=1\) if there is an edge between nodes \(i\) and \(j\), otherwise \(A_{i,j}=0\). We denote the input vector as \(\mathbf{h}_{i}^{(l-1)}\) and the output vector of node \(i\) at the \(l\)-th layer of the GCN model as \(\mathbf{h}_{i}^{(l)}\). The graph convolution operation is expressed as:  
\[\mathbf{h}_{i}^{(l)}=ReLU(\sum_{j=1}^{k}\mathbf{A}_{i,j}\mathbf{W}^{l}\mathbf{h}_{j}^{(l-1)}+ \mathbf{b}^{l}), \tag{5}\]  
where \(ReLU\) is an activation function, \(\mathbf{W}^{l}\) is the weight matrix at \(l\)-th layer, and \(\mathbf{b}^{l}\) is the bias vector. The new hidden representation \(\mathbf{h}_{i}^{(l)}\) of node \(i\) is obtained by considering only its immediate neighbors. Repeating this graph convolution operation \(L\) times forms a \(L\) layers GCN model. All these operations are conducted with matrix multiplications, which is suitable for batch computation over instances and running on GPUs. Since the information propagation between nodes is performed in parallel, the efficiency of the model is not affected by the depth of the dependency tree. In order to adapt the GCN model to encode the sentences, we convert each dependency tree of the sentences into its corresponding adjacency matrix \(\mathbf{A}\). If there is a dependency edge between words \(w_{i}\) and \(w_{j}\), \(A_{i,j}=1\). Motivated by [PERSON] et al. [7], the position features of words are also considered in our work, which can express the structural features of a sentence. Each word in a sentence has two relative distances \(PF_{1}\) and \(PF_{2}\) with entities \(e_{1}\) and \(e_{2}\). Take the sentence mentioned in the previous section as an example, the relative distances of word \"co-founded\" to \"_[PERSON]_\" and \"_Microsoft_\" are \(-1\) and \(1\). The position embedding of each word in the sentence \(S\) is denoted by \(\mathbf{e}_{i,1}^{p}\in\mathbb{R}^{d_{P}}\) and \(\mathbf{e}_{i,2}^{p}\in\mathbb{R}^{d_{P}}\), where \(d_{P}\) is the size of the position embeddings. The \(i\)-th word in sentence \(S\) can then be projected to a low-dimensional vector \(\mathbf{w}_{i}=[\mathbf{e}_{i}^{w};\mathbf{e}_{i,1}^{p};\mathbf{e}_{i,2}^{p}]\in\mathbb{R}^{d}\) by concatenating the word embedding with two position embeddings, where \(d=d_{w}+2d_{p}\). The initial representation of the sentence \(S\) can be expressed as \(\mathbf{S}=\{\mathbf{w}_{1},\mathbf{w}_{2},...,\mathbf{w}_{m}\}\in\mathbb{R}^{m\times d}\). Then, each node of the dependency tree can be represented by it's corresponding word embedding, and the inputs of the GCN \(\mathbf{h}^{(0)}=\{\mathbf{h}_{1}^{(0)},\mathbf{h}_{2}^{(0)},...,\mathbf{h}_{m}^{(0)}\}=\{\mathbf{ w}_{1},\mathbf{w}_{2},...,\mathbf{w}_{m}\}\) are obtained. Since words never connect to themselves in the original dependency tree, the information of \(\mathbf{h}_{i}^{l-1}\) can never be propagated to \(\mathbf{h}_{i}^{l}\). To address this issue, we update the dependency structure by adding a self-loop for each word. Thus, the updated adjacency matrix is expressed as \(\mathbf{\bar{A}}=\mathbf{A}+\mathbf{I}\), where \(\mathbf{I}\in\mathbb{R}^{m\times m}\) is an identity matrix.