## 3 Materials and Methods
### Relation Extraction  
Relation extraction is to predict the relations between the entities in addition to those assigned to the _none_ class in the candidate entities. Therefore, the essential basis for relation extraction is entity pair representation. In addition, relation extraction also depends on the context information where entity pairs are located, especially the local context.  
For local context information, we adopt Bi-LSTM to enhance its implicit semantic information instead of sharing input with name entity recognition. The Bi-LSTM responsible for obtaining the implicit semantic information of local contexts is denoted as Bi - LSTM\({}_{r}\). In addition, the local context length representation is added. The length of the local context reflects the spacing of entities, which affects the judgment of the relationship between entities. The smaller the entity interval is, the greater the degree of association between entity pairs.  
For the global context information, we use the last hidden state obtained by Bi - LSTM\({}_{r}\) as the global semantics representation instead of CLS in Equation (10). This is because CLS information obtained by BERT has been added in the entity pair representation, and global semantic information is obtained through different methods, which makes the representation of global semantics more abundant.  
First, we enhance implicit semantics of the contextual representation by Bi - LSTM\({}_{r}\).  
\[\mathbf{X}^{r}=(x_{1}^{r},x_{2}^{r},\dots,x_{n}^{r})=\text{Bi}-\text{LSTM}_{r}( \mathbf{X}^{w}) \tag{11}\]  
The local context is the text sequence from the end of the first entity to the beginning of the second entity. We aggregate local context representation by Maxpooling:  
\[\mathbf{c}_{a-b}=\text{Maxpooling}(\mathbf{x}_{a_{end}}^{r},\mathbf{x}_{a_{end}+1}^{r}, \dots,\mathbf{x}_{b_{start}}^{r}) \tag{12}\]  
where \(a_{end}\) denotes the subscript of the end of the first entity, and \(b_{start}\) denotes the subscript of the beginning of the second entity.  
The method of obtaining the local context length representation and the entity length representation is similar. Local context length representation is also obtained from the local context length representation matrix, and each local context length representation has its corresponding representation vector. Finally, the entity pair representation vector \(\mathbf{e}_{a},\mathbf{e}_{b}\), the local context length representation vector \(\mathbf{w}_{g}^{c}\), and the last hidden state \(\mathbf{h}\) from Bi - LSTM\({}_{r}\) are concatenated to form the final relation representation vector. Then, we adopt Softmax classification function to classify the relation representation.  
\[\mathbf{y}_{a}^{r}=\text{Softmax}(\mathbf{W}_{4}[\mathbf{e}_{a};\mathbf{c}_{a-b};\mathbf{e}_{b}; \mathbf{w}_{g}^{c};\mathbf{h}]+\mathbf{b}_{4}) \tag{13}\]  
\[\mathbf{y}_{b}^{r}=\text{Softmax}(\mathbf{W}_{5}[\mathbf{e}_{b};\mathbf{c}_{b-a};\mathbf{e}_{a}; \mathbf{w}_{g}^{c};\mathbf{h}]+\mathbf{b}_{5}) \tag{14}\]  
where \(\mathbf{w}_{g}^{c}\) is the representation vector of the local context length \(g\), \(\mathbf{W}_{4}\), \(\mathbf{W}_{5}\), \(\mathbf{b}_{4}\) and \(\mathbf{b}_{5}\) are the trainable parameter vectors.  
Due to the asymmetry of the relationship, the possibility of bidirectional relationship is considered in relation extraction. If any one value of \(\mathbf{y}_{a}^{r}\) and \(\mathbf{y}_{b}^{r}\) does not reach the threshold, it is considered that there is no relationship between entity \(a\) and entity \(b\).