## 3 Methods
### Relation Classification Module  
After the named entity recognition, span pairs were selected from the set, S, of spans predicted to be entities for relation classification. Since relations have asymmetry, the input span pair \(\mathrm{s_{i}}\), \(\mathrm{s_{i}}\) is different from span pair \(\mathrm{s_{i}}\), \(\mathrm{s_{i}}\) for the relation classification. In the relation classification, a context embedding vector is used for extracting the local context information between span \(\mathrm{s_{i}}\) and span \(\mathrm{s_{i}}\), which is calculated by Equation (11):  
\[\mathrm{c_{i,j}}=\mathrm{maxpool}(\mathrm{e_{k}},\mathrm{e_{k+1}},\dots, \mathrm{e_{l}}),1\geq\mathrm{k} \tag{11}\]  
Figure 3: The span for entity classification mainly consists of five parts: first, the basic span embedding obtained through maximum pooling (blue), then the width feature of the span (deep purple), sentence vector \(\mathrm{C}\) as the classification vector for entity classification (light purple), fine-grained part of speech embedding obtained by calculating the part of speech feature of the word vector within the span (orange), and coarse-grained part of speech embedding (brown) obtained by calculating the part of speech features of the span. \(\mathrm{c_{ij}}\) is the context embedding vector between span \(\mathrm{s_{i}}\) and \(\mathrm{s_{j}}\). When span \(\mathrm{s_{i}}\) and \(\mathrm{s_{j}}\) overlap, we define \(\mathrm{c_{ij}}=0\). \(\mathrm{e_{k}},\mathrm{e_{k+1}},\ldots,\mathrm{e_{i}}\) are the embeddings of word tokens between the span pair. The span embedding used in the relation extraction is calculated by Equation (12), where \(\mathrm{N_{1}}=\mathrm{N_{f}}*2+\mathrm{N_{d}}+\mathrm{N_{m}}\):  
\[\mathrm{a_{i}^{r}}=\mathrm{concat}\Big{(}\mathrm{e_{i}^{s}},\mathrm{e_{i}^{m} },\mathrm{e_{i}^{c}},\mathrm{e_{i}^{f}}\Big{)},\mathrm{e_{i}^{r}}\in\mathbb{R }^{\mathrm{N_{1}}} \tag{12}\]  
The span embeddings for relation extraction of the two spans \(\mathrm{a_{i}^{r}}\), \(\mathrm{a_{j}^{r}}\) and the context embedding vector are concatenated and used in relation classification. The bidirectional relation embeddings \(\mathrm{a_{ij}^{r}}\) and \(\mathrm{a_{j}^{r}}\) are composed of two span embeddings and the context embedding vector, where \(\mathrm{N_{a}}=\mathrm{N_{1}}*2+\mathrm{N_{d}}\):  
\[\mathrm{a_{i,j}^{r}}=\mathrm{concat}\Big{(}\mathrm{a_{i}^{r}}, \mathrm{c_{ij}},\mathrm{a_{i}^{r}}\Big{)},\mathrm{a_{i,j}^{r}}\in\mathbb{R}^{ \mathrm{N_{a}}} \tag{13}\] \[\mathrm{a_{j,i}^{r}}=\mathrm{concat}\Big{(}\mathrm{a_{j}^{r}}, \mathrm{c_{ij}},\mathrm{a_{i}^{r}}\Big{)},\mathrm{a_{j,i}^{r}}\in\mathbb{R}^{ \mathrm{N_{a}}}\]  
Then the transformer encoder is used for classifying the relation embeddings, shown in Figure 4. Relation classifiers use a transformer encoder with the sigmoid function to calculate the probability of each relation type. The intrinsic connection between span-pair embeddings is captured and learned using a multi-head self-attention layer, which contains multi-head self-attention encoder, and in the experiments, the number of self-attention encoder is set to \(\mathrm{H}=4\). Each self-attention encoder contains three learnable parameters WQ, WK, WV, used to obtain Q, K and V. Each of the three fully connected layers are defined as learnable parameters. \[\mathrm{Q_{i,j,h}^{r}}=\mathrm{WQ_{h}}\Big{(}\mathrm{a_{ij}^{r}}\Big{)}, \mathrm{K_{ij,h}^{r}}=\mathrm{WK_{h}}\Big{(}\mathrm{a_{ij}^{r}}\Big{)}, \mathrm{V_{ij,h}^{r}}=\mathrm{WV_{h}}\Big{(}\mathrm{a_{ij}^{r}}\Big{)} \tag{14}\]  
h denotes the h-th self-attentive encoder. The result of multiplying the query matrix Q with K is divided by the dimension of the input \(\mathrm{a_{ij}^{r}}\) to obtain the scaled dot product attention. Afterwards, softmax transforms probability scores, which are multiplied with the matrix V to obtain the output weight \(\mathrm{Z_{i,j,h}^{r}}\) for each self-attention layer.