# Improving Entity Linking by Introducing Knowledge Graph Structure Information
## 4 Model
### Global Model  
[PERSON] and [PERSON] [6] mainly considered the consistency between entities. However, [PERSON] and [PERSON] [7] proposed that there is not only consistency between entities, but there are also some latent relations that can support the constraints on entities. Assuming that there are \(K\) latent relations, each relation \(k\) corresponds to a pair \((m_{i},m_{j})\), so the second term of Equation (3) can be written as:  
\[\Phi\big{(}e_{i},e_{j},D\big{)}=\sum\limits_{k=1}^{K}\alpha_{ijk}\Phi_{k} \big{(}e_{i},e_{j},D\big{)} \tag{8}\]  
That is, the paired score \((m_{i},m_{j})\) is the weighted sum of the corresponding scores of each relation, and \(\alpha_{ijk}\) is the weight corresponding to the relation \(k\). Here, each relation \(k\) is a diagonal matrix \(\mathbf{R}_{k}\in\mathbb{R}^{d\times d}\), and  
\[\Phi_{k}\big{(}e_{i},e_{j},D\big{)}=\mathbf{e}_{i}^{T}\mathbf{R}_{k}\mathbf{e }_{j} \tag{9}\]  
The weight \(\alpha_{ijk}\) is the normalized score:  
\[\alpha_{ijk}=\frac{1}{Z_{ijk}}\text{exp}\left\{\frac{f^{T}(m_{i},c_{i})\mathbf{ D}_{k}f(m_{j},c_{j})}{\sqrt{d}}\right\} \tag{10}\]  
where \(Z_{ijk}\) is the normalization factor, \(\mathbf{D}_{k}\in\mathbb{R}^{d\times d}\) is a diagonal matrix, and \(f(m_{i},c_{i})\) is a single-layer neural network, which is used to obtain the local context representation of the mention \(m_{i}\). For \(c_{i}\), we first obtain the average \(c_{l}\) of the word embeddings of the context words on the left of the mention \(m_{i}\), then obtain the average \(c_{r}\) of the word embeddings of the context words on the right, and finally take the concatenation of \(c_{l}\) and \(c_{r}\). In addition, [PERSON] and [PERSON] [7] proposed two normalization methods of \(Z_{ijk}\): normalization over relations and normalization over mentions. We adopt the method of normalization over mentions, then  
\[Z_{ijk}=\sum\limits_{ \begin{subarray}{c}j=1\\ j\
eq i\end{subarray} }^{n}\exp\left\{\frac{f^{T}(m_{i},c_{i})\mathbf{D}_{k}f(m _{j},c_{j})}{\sqrt{d}}\right\} \tag{11}\]  
Now \(\sum_{j=1,j\
eq i}^{n}\alpha_{ijk}=1\), which means that for each relation \(k\) and mention \(m_{i}\), we want to find another mention that has a relation \(k\) with the mention \(m_{i}\). The entity embeddings \(\mathbf{e}_{i},\mathbf{e}_{j}\) here are obtained by training using word-entity co-occurrence counts in Wikipedia, so the global model is called the WikiEmbs model, and there is \(\Phi_{wiki}\big{(}e_{i},e_{j},D\big{)}=\Phi\big{(}e_{i},e_{j},D\big{)}\). The WikiEmbs model essentially only uses the semantic information of the entities; that is, the more semantically related entities have a greater probability of appearing in the same document. However, the structural information in the knowledge graph is ignored, so we propose the KGEmbs model, which explicitly uses the knowledge graph structure information in the global model. Our motivation is that the knowledge graph structure should be maintained when the entity mentions in a document are mapped to the knowledge base. Assuming that there are \(R_{n}\) relations (Section 3.2), the second term in Equation (3) can be written as:  
\[\Phi_{KG}\big{(}e_{i},e_{j},D\big{)}=\max_{r\in R_{n}}f_{KG}\big{(}e_{i},e_{j},r \big{)} \tag{12}\]  
where \(f_{KG}\big{(}e_{i},e_{j},r\big{)}\) is the scoring function of the knowledge graph embedding method; that is, for all relations \(R\), the score of \((e_{i},e_{j})\) must be calculated, and then the maximum value is taken. The TransE [39] model is used here, and because the head entity and tail entity in \((e_{i},e_{j})\) cannot be distinguished, there is:  
\[f_{KG}\big{(}e_{i},e_{j},r\big{)}=\max\big{(}\gamma_{1}-d\big{(}e_{i}+r,e_{j} \big{)},\gamma_{1}-d\big{(}e_{j}+r,e_{i}\big{)}\big{)} \tag{13}\]  
where \(\gamma_{1}\) is consistent with \(\gamma_{1}\) in Equation (1), and  
\[d\big{(}h+r,t\big{)}=\left\|\mathbf{h}+\mathbf{r}-\mathbf{t}\right\|_{1} \tag{14}\]  
Among them, the smaller \(d(h+r,t)\), the greater the probability that the entities \(h\) and \(t\) have the relation \(r\).