# Syntax-Informed Self-Attention Network for Span-Based Joint Entity and Relation Extraction
## 2 Related Works
### Joint Entity and Relation Extraction Model  
Early joint models were mainly feature-based [36; 37; 38]. They often rely on the quality of feature construction heavily, which is time-consuming and lacks sufficient generalizability. With success of deep learning on many NLP tasks, neural-based models make up for the shortcomings of feature-based models, and have many applications in the field of RE tasks. Ref. [6] proposed to employ RNN-LSTM with Tree-LSTM for joint entity and relation extraction, and modeled syntactic and sequence information simultaneously. Ref. [39] proposed a hybrid neural network that employed BiLSTM for entity recognition and CNN for relationship extraction without any handcrafted features. Refs. [40; 41] proposed to model the task as a table filling problem, but it would cause redundant information and increase the possibility of errors. Ref. [8] first proposed to model the task as a sequence labeling problem on BIOES scheme. They synthesized entity types and relations as labels, and artificially features were not adopted. To some extent, although the problem of error propagation is avoided, it has shortcomings on identification of the overlapping relations. After that, Ref. [9] proposed to divide situations into three categories: Normal (none of its triplets have overlapped entities), Single Entity Overlap (SEO), and Entity Pair Overlap (EPO). They viewed the task as a triple generation process based on seq2 seq with a copy mechanism to cover all above situations. Refs. [6; 7; 8] only solved the problem of the Normal situation. After that, some works began to cover SEO and EPO situations. Ref. [10] employed BiLSTM to encode sequence information and modeled the RE task as a multi-head selection problem. In addition, Conditional Random Field (CRF) was used to decode so that the model can identify entities and all possible relations between them to solve overlapping relations issue. Based on the architecture of [10], Refs. [42; 43] incorporated different types of attention, but still based on a sequence labeling scheme. Ref. [11] adopted GCN to encode dependency trees and BiLSTM as their sequence model, and integrated them into an end-to-end architecture. Ref. [44] formulated the joint extraction as a token pair linking problem and introduced a novel handshaking tagging scheme to align the boundary tokens of entity pairs under each relation type. In addition, the results showed that their method had good performance on relation extraction task. Due to the entity nesting issue, span-based method becomes popular. Refs. [12; 13] completed the NER task by exhaustively searching spans and made these spans into candidate entity pairs, and let the model cover situations not considered in [9]. [PERSON]. [45] dynamically constructed span graphs, by selecting the most confident entity spans and linking these nodes with confidence-weight. Through propagation of the graph, it can refine the span representation iteratively. On the basis of [45], [PERSON]. [46] replaced the RNN encoder with BERT, and get better extraction results. [PERSON]. [47] was also based on the Transformer and modeled the task as a multi-turn QA problem in an innovative manner, but it required constructing question templates manually, which was time-consuming. Overall, span-based method has achieved the best performance on multiple benchmarks of joint models and using the Transformer for feature extraction usually has better performance.