# FREDC: A Few-Shot Relation Extraction Dataset for Chinese
## 4 Experiments
### Analysis of Results  
Our research employed a diverse array of experimental configurations to conduct a comprehensive evaluation of existing models' performance on the FREDC dataset. In the baseline model testing phase, all models produced valid experimental results that were in line with their anticipated performance levels, thereby underscoring the utility and effectiveness of the FREDC dataset. However, it is evident that FREDC presents significant challenges to these baseline models, particularly in cross-domain experiments that involve data from two specialized fields, where the majority of models exhibited performance levels below 30%. In the cross-domain tasks, we conducted a comparative analysis of the experimental outcomes of the Proto and BERT-Pair models across two distinct datasets. The experimental findings indicate a decrease in accuracy of approximately 15-30% when processing Chinese text as opposed to English. This discrepancy can be attributed, in part, to our Chinese cross-domain test subset, which provides a comprehensive assessment of the RE capabilities of models across two domains, presenting a task of greater complexity compared to FewRel2.0. In our experimental investigations of the NOTA task, we evaluated the performance of the BERT-Pair model across various datasets. The results suggest that the FREDC dataset remains effective in the context of NOTA experiments. However, the introduction of NOTA in the testing phase revealed that the Chinese dataset had a more pronounced impact on model accuracy than the English dataset, with a performance discrepancy of approximately 20%. It is noteworthy that under the 5-way-1-shot and 5-way-5-shot configurations with NOTA of 0%, the performance differential between the Chinese and English datasets was significantly distinct, implying that with adequate data support, the performance gap between these datasets can be substantially reduced. The FREDC dataset, specifically curated for Chinese few-shot RE tasks, exhibits unique attributes; within the medical domain, a significant proportion of relations are disease-related, leading to multiple relation types that are prone to confusion. For example, certain relations may involve recurring entity types or identical head and tail entity types, as illustrated in Figure 3a. In the financial domain, majority relations predominantly involve two companies, thereby limiting the range of head and tail entity types. Additionally, the same entity may assume different meanings in various contexts, and altering the positions of the head and tail entities can result in new relations, as depicted in Figure 3b. Given these specialized characteristics of the FREDC dataset, reliance on entity differentiation alone is insufficient for predicting relations. The model must integrate semantic information to infer relations between entities, which substantially complicates the task of few-shot RE. \begin{table}
\begin{tabular}{c c c c c} \hline \multirow{2}{*}{**NOTA (\%)**} & \multicolumn{2}{c}{**5-1**} & \multicolumn{2}{c}{**5-5**} \\ \cline{2-5}  & **FewRel2.0** & **FREDC** & **FewRel2.0** & **FREDC** \\ \hline
0\% & 76.73 & 52.62 & 83.32 & 72.00 \\
50\% & 80.31 & 46.28 & 86.06 & 49.13 \\ \hline \end{tabular}
\end{table} 
Table 5: The accuracy (%) of BERT-Pair in NOTA task on different datasets. In addition, some performance loss may stem from the limitations of our work. When different baseline models make predictions, we observe the following common issues, with specific error output details shown in Table 6.  
The main demonstration is that our modified baseline model is deficient in two main areas. Firstly, the encoding methodology employed by BERT-Chinese operates at the character level for Chinese, in contrast to the word-level encoding utilized for English, which may impede the ability of models to grasp the nuances of word meanings.