# Syntax-Informed Self-Attention Network for Span-Based Joint Entity and Relation Extraction
## 1 Introduction  
Relation Extraction (RE) is the task of aiming to find semantic relationships between pairs of the entity mentions from unstructured texts based on entity recognition. As an essential task in information extraction, which is widely applied in question answering [1], populate knowledge base [2], and other knowledge-based tasks. The relational facts obtained from RE are often expressed in form of knowledge triplet {\(h\), \(r\), \(t\)}, where \(h\) and \(t\) represent head and tail entity, respectively, and \(r\) represents the relation between the entities. One annotated example is illustrated in Figure 1.  
Previous works can be divided into two categories: one is in pipelined setting, the other is in joint setting. The pipelined method [3; 4; 5] divides the task into two steps: named entity recognition (NER) and relation extraction (RE). Although this kind of method seems reasonable, it ignores relevance between the two tasks. For example, by identifying entity  
Figure 1: Annotated result in a sample sentence (ORG: organization; PER: person). mentions, we can classify \"[PERSON]\" and \"Chicago\" as _person_ and _locate_ type, respectively. It is easy to find that the type of above two entities is helpful for identifying relation \"Live-in\". To tackle with this problem, joint method is proposed. Different from pipelined models, joint models can extract both entities and relations with shared parameters in an end-to-end architecture. It can effectively integrate the feature of entities and relations and achieve better results in the task. For the problem of extracting relations, the relational facts in sentences are often complicated: a sentence may contain multiple entities, an entity may participate in multiple relational triplets, and triplets may have overlapped entity pairs. In addition to this, the nesting issue may occur in these entity mentions. Refs. [6; 7; 8] only can cover the situation where none of triplets have overlapped entities in a sentence. After that, many works like [9; 10; 11] can cover the situation where triplets have overlapped entity pairs and an entity participates in different triplets. Among these models, each token corresponds to a single fixed representation and we usually refer to them as token-based models. However, some tokens may participate in different spans, such as \"the minister of the department of education\" and \"the department of education\". It's hard for token-level models to model overlapping entities due to their natural limitation that one token has a single fixed representation. To tackle this issue, span-based models are proposed. Refs. [12; 13] solve the problem of entity nesting by searching all possible spans exhaustively to complete RE task. At present, span-based joint entity and relation extraction models can cover the most complex situations. However, despite span-based model can cover more cases, it still has many shortcomings that affect the quality of extraction. First, while maintaining an efficient exhaustive search on spans, the importance of syntactic features is not taken into consideration. It will lead to a problem that the prediction of a relation between two entities is related based on corresponding entity types, but in fact they are not related in the sentence. Employing additional syntactic features is helpful for this issue. Second, although previous state-of-the-art work like [13] has proven that extracting local context is superior to global context for RE task, but simply conducting a pooling method over the context directly to align feature dimensions can result in the loss of some critical information. For the processing of local context, in-depth learning of contextual features is needed. Recently, Transformer [14] is a popular model applied on a wide range of Natural Language Processing (NLP) tasks.