# CRSAtt: By Capturing Relational Span and Using Attention for Relation Classification
## 3 Methodology
### Attention-Based Prediction Structure  
We designed an attention-based prediction structure for fusing global and local semantic features. The previous prediction layer simply passes the feature vector through a fully connected layer, maps the probability distribution of each category in the multi-classification by a linear function, and then uses a softmax classifier for classification. Such a direct reduction of the vector dimension using the linear function not only reduces the ability of the model to capture features but also loses some semantic information in the feature dimension transformation. To solve this problem, we propose to use attention instead of a fully connected layer. We use our own design of Multi-Class Attention (see Figure 4 for details) to map each relation vector space, and finally, calculate the probability distribution of each relation. The prediction structure is shown in Figure 3. When the extracted feature vectors \(\mathbf{w}\), \(\mathbf{s}\), \(\mathbf{e1}\), and \(\mathbf{e2}\) are input to the prediction structure, we let these features construct the \(\mathbf{q}\) and \(\mathbf{k}\) needed for attention, respectively. We fuse the \(\mathbf{w}\) and \(\mathbf{s}\) using \"SRS\" to obtain the reinforcement vector \(\mathbf{sr}\) and then stitch \(\mathbf{sr}\) and \(\mathbf{e1}\) together to form \(\mathbf{q}\); \(\mathbf{k}\) is composed of \(\mathbf{e2}\). The reason we use this design is to form a paradigm that allows an entity and a relation to query and verify another appropriate entity. We believe that such  
Figure 3: This is the internal detail diagram of the prediction structure. When the \(entity_{feature}\), \(w\), and \(s\) are fed into the prediction structure, they will be divided into two ways, one way to construct the keys needed for Multi-Class Attention and one way to construct the queries needed for Multi-Class Attention. SRS is the process of averaging the \(w\) representing the global semantics and the \(s\) representing the local semantics, and then feeding them into the fully connected layer.  
a paradigm is also consistent with our human intuition (e.g., given an entity \"[PERSON]\" and a relation \"author,\" our intuition is that the other entity must be a person's name). Therefore, this paradigm can help the model to perform the relation classification task better. After feeding \(\mathbf{q}\) and \(\mathbf{k}\) into Multi-Class Attention, we obtain a vector of probability distributions for each relation, which is finally fed into a softmax classifier for classification.  
\[\mathbf{q}=\text{concat}(W_{3}[\text{GELU}(e1)]+b_{3},\text{sr}_{\text{ feature}}\,) \tag{9}\]  
\[\mathbf{k}=W_{4}[\text{GELU}(e2)]+b_{4} \tag{10}\]  
\[\mathbf{p}=\text{Multi-Class Attention }(q,k) \tag{11}\]  
\[e1,e2\in\text{ entity }_{\text{ feature}} \tag{12}\]  
where \(q\) represents the queries of the attention mechanism; \(k\) represents the keys of the attention mechanism; \(e1,e2\) represent the entity vectors; \(sr_{feature}\) represents the SRS fusion feature; and \(p\) represents the probability distribution of the relation.