# Few-Shot Relation Extraction on Ancient Chinese Documents
## 5 Experiments
### Ablation Study  
To demonstrate the effect and importance of each component in MASCOT-PA, we performed the ablation study. Table 7 displays three ablation modules' performance, which includes our proposed MASCOT encoder, the paired function, and the input instance attention. We also used the standard FewRel 1.0 dataset to evaluate each module's effectiveness on this few-shot task. Given that the pair function depends on the enhancement and transformation that instance attention implements, it cannot be set apart from instance attention as a single module. The paired attention mentioned in Section 4.2 has achieved a remarkable performance, which means that this operation for query and support instances is suitable. The attention matrix collects similarities between query and support instances and magnifies the differences and noise in sentences and words will have less influence on class prediction by sharing this background information. The paired function also enhanced the model relation learning ability compared with the single instance attention module, and it increases the model performance by 0.67%. MASCO has a stronger expression ability than CNN in the base model. However, in Nos. 2 and 5 in Table 7, MASCOT improves the model by 2.58% and 3.9% without and with pair attention, respectively. Therefore, only paired attention can learn and utilize the external features even though MASCOT generates richer expression features. We also deeply analyzed the sentence encoders in Table 8. The MASCOT encoder adds a three-fold convolution layer to enhance the local relation between words compared with the standard transformer encoder. The convolutional DNN [31] shows that a proper \begin{table}
\begin{tabular}{l c c c c} \hline \hline  & **5-w-1-s** & **5-w-5-s** & **10-w-1-s** & **10-w-5-s** \\ \hline Proto (CNN) [6] & 69.20 & 84.79 & 56.44 & 75.55 \\ \hline Proto-HATT [58] & 75.01 & 87.09 & 62.48 & 77.50 \\ \hline MLMAN [13] & 79.01 & 88.86 & 67.37 & 80.07 \\ \hline TPN (BERT) [59] & 80.14 & 93.60 & 72.67 & 89.83 \\ \hline Proto (BERT) [36] & 80.68 & 89.60 & 71.48 & 82.89 \\ \hline ConceptFERE(s) [60] & 84.28 & 90.34 & 74.00 & 81.82 \\ \hline CTEG [58] & 84.72 & 92.52 & 76.01 & 84.89 \\ \hline BERT-PAIR [58] & 85.66 & 89.48 & 76.84 & 81.76 \\ \hline BERTEM+MTB [8] & 88.9 & - & - & - \\ \hline baseline & 73.64 & 86.93 & 60.91 & 76.88 \\ \hline local GloVe MASCOT-PA & 76.43 & 86.01 & 62.31 & 74.84 \\ \hline MASCOT-PA & 81.28 & 89.25 & 69.03 & 80.65 \\ \hline \hline \end{tabular}
\end{table} 
Table 6: Overall result for the FewRel 1.0 task, where \(N\)-w-\(K\)-s represent the \(N\)-way-\(K\)-shot task performance. \begin{table}
\begin{tabular}{c c c c} \hline \hline
**MASCOT** & **Pair** & **Attention** & **No.** & **FewRel 5-w-1-s** \\ \hline \(\bigtimes\) & \(\bigtimes\) & 1 & 73.64 \\ \(\bigvee\) & \(\bigtimes\) & 2 & 75.49 \\ \(\bigtimes\) & \(\bigtimes\) & 3 & 77.67 \\ \(\bigtimes\) & \(\bigvee\) & 4 & 78.34 \\ \(\bigvee\) & \(\bigvee\) & 5 & 81.28 \\ \hline \hline \end{tabular}
\end{table} 
Table 7: Ablation study for our model, MASCOT is the MASCOT encoder, pair is the pair function, attention is the instance attention. convolution network should have the ability to represent sentence-level features. In addition, MASCOT applies a parallel structure to avoid the information loss problem, which BRAN is mainly suffered from. MASCOT will be more expressive than BRAN and alleviate BRAN's overfitting problem. As a result, MASCOT increases the model's performance to \(75.49\%\). By comparing the above three encoders, we can learn that transformer has a stable performance as the layer gets deeper by simultaneously computing the global and local features through multi-head attention and convolution layer, while it does not surpass MASCOT. From Figure 5, we can see that the residual connection structure for BRAN does not properly function after stacking six layers, which is probably due to the inner structure between the multi-head attention and convolution layer.