# Joint Extraction of Entities and Relations Based on Enhanced Span and Gate Mechanism
## 3 ESGM Model
### Span Extraction Module  
Entity and relation extraction using spans can effectively address the issue of nested entities. Such methods can generally be categorized into two approaches, namely, the combination of span enumeration with classification, and pointer network methods. In the span enumeration with classification approach, all possible spans of different lengths in the sentence are enumerated and entity classification is performed using a softmax function. This method results in a large number of redundant spans, as the number of enumerated spans in a sentence with N tokens is N(N + 1)/2. This increases the computational complexity of entity recognition. Pointer network methods are commonly used in MRC tasks. They use two n-class markers to predict the begin and end positions of spans, where n represents the length of the sentence. This method employs a sigmoid function to forecast whether each token within the sentence signifies a starting or concluding position, thereby allowing the output of a single span only when a specific query is provided. Building upon these approaches, in this paper we utilize two binary classifiers; one predictswhether each token is a start position of a span, and the other predicts whether each token is an end position of a span. This approach allows for multiple start and end positions to be found, enabling the extraction of multiple relevant entities. To address the issue of nested entities, a one-to-many matching method is proposed which matches the begin position token with all the subsequent final position tokens, resulting in the final span set. After passing the sequence of sentences through BERT, the final Transformer layer output, which represents the representation of each token in the sentence, is obtained. These representations are then input into the span extraction module. In this module, the model predicts the probability of each token being the starting position of a span, then predicts the probability of each token being the end position of a span using the same approach. The probability predictions are calculated according to Equations (4) and (5). \[p_{i}^{start\_s}=\sigma(W_{start\_s}\cdot t_{i}+b_{start\_s}) \tag{4}\]  
\[p_{i}^{end\_s}=\sigma(W_{end\_s}\cdot t_{i}+b_{end\_s}) \tag{5}\]  
In the above equations, \(p_{i}^{start\_s}\) and \(p_{i}^{end\_s}\) respectively denote the probabilities that the \(i\)th token within a given sentence is the start or end position of the subject. Here, \(t_{i}\) represents the final Transformer layer encoding representation of the \(i\)th token in the input sentence \(t_{i}=H_{N}[i]\), the weights are denoted by \(W_{start\_s}\) and \(W_{end\_s}\), the offset is denoted by \(b_{start\_s}\) and \(b_{end\_s}\), and \(\sigma\) represents the sigmoid activation function. During the probability prediction stage, we set a threshold to compare it with the predicted probabilities; the results are labeled using two binary markers. Following references [7; 9; 26], we conducted several experiments, finding that setting the threshold to 0.5 has a good experimental effect; thus, we set the threshold as 0.5. Tokens with probabilities that exceed the threshold are marked as 1 at their corresponding positions, while the remaining positions are marked as 0. After labeling the tokens, the next step is to form spans by combining those tokens marked as 1 in the start and end markers. In this paper, we adopt a one-to-many matching principle where a start position can be matched with multiple end positions. As shown in Figure 2, the start marker labels \"New\" and \"United\" as 1, while the end marker labels \"York\", \"University\", and \"States\" as 1. By applying the one-to-many matching approach, the tokens marked as 1 in the start positions are combined with the tokens marked as 1 in the subsequent end positions. For example, \"New\" is combined with \"York\" to form \"New York\", with \"University\" to form \"New York University\", and with \"States\" to form \"New York University is located in the United States\".