# CRSAtt: By Capturing Relational Span and Using Attention for Relation Classification
## 2 Related Work  
Traditionally, the two main approaches to relation extraction are supervised and semi-supervised. Supervised relation extraction, also known as relation classification, is the most effective and well-researched method that uses fully labeled manual data for training. The supervised relation extraction task does not have a subtask of entity recognition; so, the main structure of the model is a feature extractor + relational classifier. Remote supervision is a form of semi-supervision and is currently a solution to the lack of supervision data. However, there are several problems with remotely supervised data: Firstly, remote supervision of labeled data has a large number of errors. Secondly, it cannot solve the situation where a pair of entities contains multiple relations. Thirdly, the False-negative problem, where instances labeled as negative samples actually have relations, but such knowledge does not exist in the knowledge graph, leading to labeling errors. Most of the current research on remote supervision focuses on the first problem. The research in this paper focuses on supervised relation classification, and the main feature extraction networks for relation classification are the convolutional neural network, recurrent neural network, and attention-based mechanism of transformer [13; 14], and [PERSON] et al. [15]. Convolutional neural networks: [PERSON]. [16] applied convolutional neural networks to achieve relation classification and proposed a convolutional DNN algorithm for extracting lexical-level features and sentence-level features. Positional features are also proposed to encode the relative distance between the current word and the target word pair. The model architecture proposed by [17] is basically the same as the previous work; the biggest change is the replacement of the loss function. The innovation lies in the Ranking loss, which enables the model to consider not only the positive category score as high as possible but also the category score that is prone to misclassification as low as possible compared with the Softmax function. The disadvantage is still the defect of the model structure. [PERSON]. [18] proposed a new structure, the Augmented Dependency Path, which combines the shortest dependency path between two entities and a subtree connected to the shortest dependency path. By modeling the subtree using a recurrent neural network, a representation of the generated dependency subtree is appended to the words on the shortest dependency path so that the words on the shortest dependency path receive new word embeddings, and then a convolutional neural network is used to capture the key features on the shortest dependency path. The above methods all use convolutional neural networks as feature extraction frameworks, and all use the Linear() function to map the probability distribution of relational categories. Although convolutional neural networks are widely used in computer vision because of their excellent local information extraction ability, global-dependent information is very important in the field of NLP, especially in the field of relation classification. Convolutional neural networks have been phased out because of their shortcomings in extracting global information. Table 1 shows a summary of our model and the convolutional-neural-network-based models. \begin{table}
\begin{tabular}{c c c c} \hline \hline \multirow{2}{*}{**Model**} & **Feature Extraction Framework** & \multirow{2}{*}{**Classification Method**} & \multirow{2}{*}{**Year**} \\ \cline{1-1}  & & & \\ \hline DNN [16] & CNN & Linear + SoftMax & 2014 \\ CR-CNN [17] & CNN & Linear + SoftMax & 2015 \\ DepNN [18] & CNN + RNN & Linear + SoftMax & 2015 \\ CRSAtt(Our) & SpanBert & Multi-Class Attention + & \multirow{2}{*}{-} \\  & & SoftMax & \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Summary of our model and the convolutional-neural-network-based models. Recurrent neural networks: Ref.