## 4 Our Approach
### Encoder Framework  
Given a sentence \(S=(w_{1},w_{2},\ldots,w_{n})\), we use BERT (bert-base-Chinese) [4] to encode the input sentence. BERT was adopted to encode the text since it had a great impact and achieved comparable results to the more advanced models in TPLInker [6], PURE [23], Cas-Rel [31] and PRGC [32]. After obtaining the sentence representations \(H=(h_{1},h_{2},\ldots,h_{n})\), then we created the span matrix, which is an \(n\times n\times d\) tensor, where n is the length of the sentence and d is the number of labels. We defined the representation of the \((i,j)\) position in this matrix as \(s_{ij}\):  
\[s_{ij}=h_{i}\oplus h_{j}\oplus\left(h_{i}\cdot h_{j}\right)\oplus\left(h_{i}- h_{j}\right) \tag{2}\]  
where \(\oplus\) denotes concatenation. For each \(s_{ij}\), we applied a Feed-Forward Neural Network and Sigmoid activation function to create scores for each label at each position:  
\[FFN\big{(}s_{ij}\big{)}=W_{2}\cdot ReLU\big{(}W_{1}\cdot s_{ij}+b_{1}\big{)}+ b_{2} \tag{3}\]  
\[Score_{d}=Sigmoid\big{(}FFN\big{(}s_{ij}\big{)}\big{)} \tag{4}\]  
Figure 1: The overall structure of the model. To make it easier to understand, the FFN of ORE and the FFN of NER are drawn separately in the figure, but they actually share the same FFN. In order to make each character in Chinese and English correspond, here we translated “Shanghai” to “Shang Hai”.  
where \(W_{1},W_{2}\) are parameter matrices and \(b_{1}\) and \(b_{2}\) are bias vectors. \(Score_{d}\) is a d-dimensional vector, corresponding to the score of d labels, respectively.  
By assuming that the number of NER categories, FRE relation categories and ORE object types are \(x,y\) and z, respectively, the total number of labels is d = x + 2y + 2 + z + 4(6). Here, x means that for each NER category, generate one label for it; 2y means for one FRE relation category, it can be determined when the head and tail of two entities successfully shake hands at the same time, so we generated two labels for each FRE relation category to indicate the success of the head and tail handshake, respectively. The 2 + z represents the number of ORE elements, which are subject, predicate and z object types. The 4(6) depends on whether the handshaking times performed by the ORE task are two or three. The specific details of handshaking decoding are introduced in Section 4.2. We defined the training loss as below:  
\[loss=log(1+\sum_{m\in\Omega{neg}}e^{score(m)})+log(1+\sum_{n\in\Omega{pos}}e^ {score(n)}) \tag{5}\]  
where \(score(x)\) means the score of the x-th label. \(\Omega{pos}\) is the set of positive label indices, and \(\Omega{neg}\) is the set of negative label indices.  
The naive multi-label classification method uses Sigmoid as the activation function and then turns it into d two-classification problems, using the sum of binary cross-entropy as the loss. Obviously, when d \(\gg\) k (k is the number of multi-label classification labels at a certain position with 1), this approach faces a serious class imbalance problem. In our method, we compared the target class scores with the non-target class scores pairwise [34]; since logsumexp is a smooth approximation function of max, it can automatically highlight items with larger errors to alleviate the class imbalance problem.