# CRSAtt: By Capturing Relational Span and Using Attention for Relation Classification
## 4 Experiment
### Experimental Results
We set the paradigm of \(\mathbf{q}\) and \(\mathbf{k}\) to make full use of various features, which, in turn, facilitates the model to better identify the correct class of relationships. We also find that the score of f1 exceeds 90% for five relations, namely, Cause-Effect, Content-Container, Entity-Destination, Member-Collection, and Message-Topic. Our guess is that these five relations may be more in line with our SRS feature vector extraction paradigm, where the relationship between entities is hidden between two entities. In the above experiments, it has been demonstrated that our model has a good ability to capture the semantics of relationships between entities and to make full use of various features. To further validate the generalizability of our model, we conducted experiments on the KBP37 dataset. The experimental results are shown in Table 9. We selected some of the models from the above experiments for comparison. These include GLFN, R-BERT, Att-RCNN, LGCNN, MALNET, Bi-SDP-Att, and D-BERT models. The experimental results show that our model has good generalization. Both the basic and advanced models outperform existing methods. The base model with SpanBert as the feature extraction architecture achieves an f1 value of 69.33% on the KBP37 dataset, and the advanced model with spanbert_large as the feature extraction architecture achieves an f1 value of 69.55% on the KBP37 dataset. \begin{table}
\begin{tabular}{c c c c c c c c c} \hline \hline \multirow{2}{*}{**Relation**} & \multicolumn{3}{c}{**Precision**} & \multicolumn{3}{c}{**Recall**} & \multicolumn{3}{c}{**f1**} \\ \cline{2-10}  & **BERT** & **BERT-ECM** & **Our** & **BERT** & **BERT-ECM** & **Our** & **BERT** & **BERT-ECM** & **Our** \\ \hline Cause-Effect & 91.72 & 93.33 & **94.51** & 94.51 & 93.91 & **94.51** & 93.09 & 93.62 & **94.51** \\ \hline Component-Whole & 85.29 & **87.23** & 87.11 & 83.65 & **89.74** & 85.78 & 84.47 & **88.47** & 87.94 \\ \hline Content-Container & 85.10 & 88.24 & **89.16** & 92.19 & 93.75 & **94.27** & 88.50 & 90.19 & **91.65** \\ \hline Entity-Destination & 91.61 & **93.86** & 92.33 & 93.49 & 94.18 & **94.66** & 92.54 & **94.02** & 93.58 \\ \hline Entity-Origin & 86.97 & **92.43** & 86.30 & 87.98 & 89.92 & **90.31** & 87.48 & **91.16** & 88.26 \\ \hline Instrument-Agency & 80.12 & **86.43** & 85.71 & **82.69** & 77.56 & 80.77 & 81.39 & 81.76 & **83.17** \\ \hline Member-Collection & 85.43 & 88.33 & **88.75** & 90.56 & 90.99 & **91.42** & 87.92 & 89.64 & **90.06** \\ \hline Message-Topic & 87.11 & **87.46** & 86.67 & **95.79** & 93.49 & 94.64 & **91.24** & 90.37 & 90.48 \\ \hline Product-Producer & 84.81 & **87.61** & 85.66 & 87.01 & **91.77** & 90.48 & 85.90 & **89.64** & 88.00 \\ \hline Average & 87.03 & **89.44** & 88.78 & 90.10 & 90.59 & **91.60** & 88.54 & 89.95 & **90.17** \\ \hline \hline \end{tabular}  
* The value in bold denotes the highest value of this evaluation metric.  
\end{table} 
Table 8: Comparison of precision, recall, and f1 score of our model with RBERT model and BERT-ECM on the SemEval-2010 Task 8 dataset. \begin{table}
\begin{tabular}{c c} \hline \hline
**Method** & **f1** \\ \hline GLFN [34] & 65.2 \\ R-BERT [10] & 68.65 \\ Att-RCNN [35] & 61.83 \\ LGCNN [36] & 63.2 \\ MALNet [22] & 61.4 \\ Bi-SDP-Att [37] & 64.39 \\ D-BERT [27] & 69.2 \\ CRSAtt (spanbert) & 69.33 \\ CRSAtt (spanbert_large) & **69.55** \\ \hline \hline \end{tabular}  
* The value in bold denotes the highest value of this evaluation metric.  
\end{table} 
Table 9: Comparison of our model with existing methods on the KBP37 dataset.