# CRSAtt: By Capturing Relational Span and Using Attention for Relation Classification
###### Abstract  
Relation classification is an important fundamental task in information extraction, and convolutional neural networks have been commonly applied to relation classification with good results. In recent years, due to the proposed pre-training model BERT, the use of which as a feature extraction architecture has become more and more popular, convolutional neural networks have gradually withdrawn from the stage of NLP, and the relation classification/extraction model based on pre-training BERT has achieved state-of-the-art results. However, none of these methods consider how to accurately capture the semantic features of the relationships between entities to reduce the number of noisy words in a sentence that are not helpful for relation classification. Moreover, these methods do not have a systematic prediction structure to fully utilize the extracted features for the relational classification task. To address these problems, a SpanBert-based relation classification model is proposed in this paper. Compared with existing Bert-based architectures, the model is able to understand the semantic information of the relationships between entities more accurately, and it can fully utilize the extracted features to represent the degree of dependency of a pair of entities with each type of relationship. In this paper, we design a feature fusion method called \"SRS\" (Strengthen Relational Semantics) and an attention-based prediction structure. Compared with existing methods, the feature fusion method proposed in this paper can reduce the noise interference of irrelevant words when extracting relational semantics, and the prediction structure proposed in this paper can make full use of semantic features for relational classification. We achieved advanced results on the SemEval-2010 Task 8 and the KBP37 relational dataset. relation classification; pre-training model; attention; feature fusion +
Footnote †: journal:  
networks. Ref. [7] used convolutional neural networks to extract lexical-level features and sentence-level features and map them into high-level features for relation classification. Ref. [8] applied recurrent neural networks to relation classification. Recurrent neural networks accumulate the semantics of sentences word by word, whereas convolutional neural networks must learn two local patterns and merge them. Obviously, the recurrent neural network model is more reasonable. Further, the semantic distribution formed by recurrent neural networks is smoother than that of convolutional neural networks. Although recurrent neural networks have excellent performance in extracting local features, they are somewhat weak in processing sequential data. Ref. [9] used a Bidirectional Long Short-Term Memory network, which aims to address the fact that Bidirectional Recurrent Neural networks do not extract information with long-term dependencies well. Recently, pre-trained language models have had a significant impact in the field of natural language processing. [PERSON]. [10] used a pre-trained Bert model as a feature extractor and obtained quite good results by stitching the extracted features together and feeding them into the classifier. [PERSON]. [11] outlined an innovative approach to perform textual information extraction by using domain ontologies and language rules, which is experimentally proven to be a feasible approach. Most of the above methods connect lexical-level features and sentence-level features to form the final feature vector for classification. However, sentence-level features cannot accurately represent the relation between two entities, and inevitably carry interference information. This will affect the final relation classification results. Moreover, existing methods put entity features and sentence features through simple splicing and then feed the features into Linear() function to obtain prediction results. Such a straightforward approach not only fails to make full use of feature information but also loses some semantic information.