# An Open Relation Extraction System for Web Text Information
## 4 Experimental Results and Experiment Analysis
### Experiments  
The steps of few-shot relation learning are: first, pre-train the encoder, then pre-train the sample selector K-tBERT; then, iteratively train the relation classifier in Graph Snowball and output the model performance evaluation value. We conducted three sets of experiments on the settings of each baseline model. Each set of experiments sampled 5 seed relation instances, 10 seed relation instances, and 15 seed relation instances of relation r, respectively. The first experiment followed the one-way five-shot setup. The corpus used for multiple iterations contained 899,997 relation instances and 464,216 entity pairs during each set of experiments. BREDS [35] is an advanced version of the original Snowball [20]. Relation Siamese network (CNN) is a network structure that uses CNN as the encoder. It adopts CNN [36] to learn an embedding function, which can map sentence pairs in the input space to a new embedding space and then calculate the similarity in the embedding space. Tables 1-3 show ORES's results for few-shot relation learning. We also list the experimental data for several previous baseline models. Since these models do not have settings for extracting new relations, this paper does not make specific comparisons and evaluations. It can be found that the performance of the ORES model is better. Our experimental results are mainly compared with Neural Snowball with the setting of extracting new relations. Furthermore, we set up the compatibility experiments, comparing the experimental results of sample selector 2 with t-BERT and K-BERT encoding settings, respectively. Sample selector 2 adopts the encoding setting of t-BERT from our previous work, Topic Snowball. From the five-shot experimental setup in Table 1, it can be seen that the recall and F1 value of Topic Snowball were significantly improved compared to Neural Snowball. After introducing the topic model, the diversity of the filtered sentences was significantly improved. However, the precision was unsatisfactory. This indicates that adding the topic model sacrificed some accuracy but increased the diversity of instances. Comparing Topic Snowball and ORES, it can be seen that the model's accuracy is improved after injecting external knowledge. This indicates that injecting external knowledge is beneficial to improving the accuracy of word embedding vector representation. In addition, the accuracy of ORES was lower than that of Neural Snowball, which was related to the selection of external knowledge sources. Table 2 shows the experimental results for 10 seed instances. Compared with Neural Snowball, the recall rate of Topic Snowball was improved. This also indicates that the topic model is effective. The decrease in precision indicates that the accuracy of Topic \begin{table}
\begin{tabular}{c c c c} \hline \hline
**Model** & **Precision (\%)** & **Recall (\%)** & **F1 (\%)** \\ \hline BREDS & 33.71 & 11.89 & 17.58 \\ Fine-tuning (CNN) & 46.90 & 9.08 & 15.22 \\ Distant Supervision (CNN) & 44.99 & 31.06 & 36.75 \\ Fine-tuning (BERT) & 50.85 & 16.66 & 25.10 \\ Distant Supervision (BERT) & 38.06 & 51.18 & 43.66 \\ Neural Snowball (BERT) & 56.87 & 40.43 & 47.26 \\ \hline Our Topic Snowball (tBERT) & 54.84 & 69.25 & 53.13 \\ Our ORES (t-KBERT) & 55.69 & 69.78 & 61.94 \\ \hline \hline \end{tabular}
\end{table} 
Table 1: The 5-shot experimental results on various previous baseline models. Snowball is related to the setting of topic weights. A more accurate algorithm is needed for the reasonable allocation of topic and semantic weights in similarity detection. From the accuracy rates of ORES and Topic Snowball, it can be seen that the word vector representation of entities was more accurate after injecting external knowledge. Using K-BERT encoding will reduce the influence of topic weights on similarity detection. Compared with Topic Snowball, the overall performance of ORES was improved. Compared with Neural Snowball, ORES had a higher recall, lower precision, and lower F1 value.