SlowGuess/NAACL_2022_OCR huggingface dataset:
There are two more further concerns when we use the NYT+FB dataset, which are also raised by Tran et al. (2020). Firstly, the development and test sets contain lots of wrong/noisy labeled instances, where we found that more than 40 out of 100 randomly selected sentences were given the wrong relations. Secondly, the development and test sets are part of the training set. Even under the setting of unsupervised relation extraction, this is still not conducive to reflecting the performance of models on unseen data. Therefore, we follow Tran et al. (2020) and additionally evaluate all models on the test set of TACRED (Zhang et al., 2017), a large-scale crowd-sourced relation extraction dataset with 42 relation types (including no Relation) and 18,659 relation mentions in the test set.
We use standard unsupervised evaluation metrics for comparisons with other eight baseline algorithms: 1) rel-LDA (Yao et al., 2011), generative model that considers the unsupervised relation extraction as a topic model. We choose the full relLDA with a total number of 8 features for comparison. 2) MARCH(Marcheggiani and Titov, 2016) proposed a discrete-state variational autoencoder (VAE) to tackle URE. 3) UIE (Simon et al., 2019) trains a discriminative RE model on unlabeled instances by forcing the model to predict each relation with confidence and encourages the number of each relation to be predicted on average, where two base models (UIE-PCNN and UIE-BERT) are considered. 4) SelfORE (Hu et al., 2020) is a self
supervised framework that clusters self-supervised signals generated by BERT adaptively and bootstraps these signals iteratively by relation classification. 5) EType (Tran et al., 2020) uses one-hot vector of the entity type pair to ascertain the important features in URE. 6) MORE (Wang et al., 2021) utilizes deep metric learning to obtain rich supervision signals from labeled data and drive the neural model to learn semantic relational representation directly. 7) OHRE (Zhang et al., 2021b) proposed a dynamic hierarchical triplet objective and hierarchical curriculum training paradigm for open relation extraction. 8) EIURE (Liu et al., 2021) is the state-of-the-art method that intervenes in the context and entities respectively to obtain the underlying causal effects of them. Since most of the baseline methods do not exactly match the dataset and experimental setup of our method, the baselines are reproduced and adjusted to the same setting to ensure a fair comparison.
Since most baseline methods adopted the setting by clustering all samples into 10 relation classes (Simon et al., 2019; Hu et al., 2020; Tran et al., 2020; Liu et al., 2021), we adjust the  $p_{\perp}$  in Algorithm 1 to get the same results for fair comparison, and name this setting HiURE w. 10 clusters. Although 10 relation classes are lower than the number of true relation types in the dataset, it still reveals important insights about models' ability to tackle skewed distribution.
Table 1 demonstrates the average performance and standard deviation of the three runs of our model in comparison with the baselines on NYT+FB and TACRED. We can observe that EIURE achieves the best performance among all the baselines, which is considered the previous state-of-the-art method. The proposed HiURE outperforms all baseline models consistently on  $\mathbf{B}^{3}$  F1, V-measure F1, and ARI. HiURE on average achieves  $3.4\%$  higher in  $\mathbf{B}^{3}$  F1,  $2.9\%$  higher in V-measure F1, and  $3.9\%$  higher in ARI on two datasets when comparing with EIURE. The standard deviation of HiURE is particularly lower than other baseline methods, which validates its robustness. Furthermore, the performance of HiURE on TACRED exceeds all the baseline methods by at least  $2.1\%$ . These performance gains are likely from both 1) higher-quality manually labeled samples in TACRED and 2) an improved discriminative
Dataset	Model	B3	V-measure	ARI
F1	Prec.	Rec.	F1	Hom.	Comp.
NYT+FB	rel-LDA(Yao et al., 2011)	29.1±2.5	24.8±3.2	35.2±2.1	30.0±2.3	26.1±3.3	35.1±3.5	13.3±2.7
March(Marcheggiani and Titov, 2016)	35.2±3.5	23.8±3.2	67.1±4.1	27.0±3.0	18.6±1.8	49.6±3.1	18.7±2.6
UIE-PCNN(Simon et al., 2019)	37.5±2.9	31.1±3.0	47.4±2.8	38.7±3.2	32.6±3.3	47.8±2.9	27.6±2.5
UIE-BERT(Simon et al., 2019)	38.7±2.8	32.2±2.4	48.5±2.9	37.8±2.1	32.3±2.9	45.7±3.1	29.4±2.3
SelfORE(Hu et al., 2020)	41.4±1.9	38.5±2.2	44.7±1.8	40.4±1.7	37.8±2.4	43.3±1.9	35.0±2.0
ETYPE(Tran et al., 2020)	41.9±2.0	31.3±2.1	63.7±2.0	40.6±2.2	31.8±2.5	56.2±1.8	32.7±1.9
MORE(Wang et al., 2021)	42.0±2.2	43.8±1.9	40.3±2.0	41.9±2.1	40.8±2.2	43.1±2.4	35.6±2.1
OHRE(Zhang et al., 2021b)	42.5±1.9	32.7±1.8	60.7±2.3	42.3±1.8	34.8±2.1	53.9±2.5	33.6±1.8
EIURE(Liu et al., 2021)	43.1±1.8	48.4±1.9	38.8±1.8	42.7±1.6	37.7±1.5	49.2±1.6	34.5±1.4
HiURE w/o ExemNCE	40.2±1.4	37.4±1.6	43.5±1.5	39.5±1.6	34.2±1.7	46.7±1.6	32.9±1.1
HiURE w/o HPC	41.4±1.2	38.7±1.0	44.3±0.9	41.5±1.3	37.2±1.1	47.0±0.8	34.3±0.9
HiURE w. 10 clusters	44.3±0.5	39.9±0.6	49.8±0.5	44.9±0.4	40.0±0.5	51.2±0.4	38.3±0.6
HiURE	45.3±0.6	40.2±0.7	51.8±0.6	45.9±0.5	40.0±0.6	53.8±0.5	38.6±0.7
TACRED	rel-LDA(Yao et al., 2011)	35.6±2.6	32.9±2.5	38.8±3.1	38.0±3.5	33.7±2.6	43.6±3.7	21.9±2.6
March(Marcheggiani and Titov, 2016)	38.8±2.9	35.5±2.8	42.7±3.2	40.6±3.1	36.1±2.7	46.5±3.2	25.3±2.7
UIE-PCNN(Simon et al., 2019)	41.4±2.4	44.0±2.7	39.1±2.1	41.3±2.3	40.6±2.2	42.1±2.6	30.6±2.5
UIE-BERT(Simon et al., 2019)	43.1±2.0	43.1±1.9	43.2±2.3	49.4±2.1	48.8±2.1	50.1±2.5	32.5±2.4
SelfORE(Hu et al., 2020)	47.6±1.7	51.6±2.0	44.2±1.9	52.1±2.2	51.3±2.0	52.9±2.3	36.1±2.0
ETYPE(Tran et al., 2020)	49.3±1.9	51.9±2.1	47.0±1.8	53.6±2.2	52.5±2.1	54.8±1.9	35.7±2.1
MORE(Wang et al., 2021)	50.2±1.8	56.9±2.2	44.9±1.8	57.4±2.1	56.7±1.8	58.1±2.3	37.3±1.9
OHRE(Zhang et al., 2021b)	51.8±1.6	55.2±2.1	48.7±1.7	56.4±1.8	55.5±1.9	57.3±2.1	38.0±1.7
EIURE(Liu et al., 2021)	52.2±1.4	57.4±1.3	47.8±1.5	58.7±1.2	57.7±1.4	59.7±1.7	38.6±1.1
HiURE w/o ExemNCE	47.3±1.1	51.2±1.2	43.9±0.9	56.4±1.0	50.3±1.2	64.2±1.4	36.9±1.0
HiURE w/o HPC	48.4±0.9	50.3±0.8	46.7±1.2	58.1±1.1	51.8±1.4	66.2±1.5	37.8±0.8
HiURE w. 10 clusters	55.8±0.4	57.8±0.3	54.0±0.5	59.7±0.6	57.6±0.5	61.9±0.6	40.5±0.4
HiURE	56.7±0.4	58.4±0.5	55.0±0.3	61.3±0.5	59.5±0.6	63.1±0.4	42.2±0.5
Table 1: Quantitative performance evaluation on two datasets.
(a) Effect of CHA
(b) Effect of HiNCE
Figure 3: Effect of Cross Hierarchy Attention on NLL loss on NYT+FB dataset (left) and HiNCE on average performance of two datasets (right) while training.
Figure 4: Relation hierarchy derived from the feature space on the NYT+FB dataset.
power of HiURE considering the variation and semantic shift from NYT+FB to TACRED.
Effectiveness of HPC. HPC considers all data points and uses their mutual similarity to find the most suitable points as exemplars for each cluster, these exemplars could update the instances in their own clusters and transfer the relational features from high-level relations to base-level through the cross hierarchy attention. From Table 1, HiURE w/o HPC, which uses  $k$ -means instead of the proposed hierarchical clustering, gives  $4.7\%$  less performance in average over all metrics when comparing with HiURE.
Effectiveness of Cross Hierarchy Attention. In order to explore how CHA helps data points to obtain the semantics of exemplars as training signals
in HPC, Figure 3(a) illustrates the log loss values of HiNCE during the training epochs. Based on the loss curve, using Cross Hierarchy Attention leads to a consistently lowered loss value, which implies that it provides high-quality signals to help train a better relational clustering model.
Considering that our exemplars correspond to specific data points and relations, we further show the hierarchical relations the model derived from the dataset. From Figure 4, we can observe a three-layer exemplars structure the model derives from the NYT+FB dataset without any prior knowledge. The high-level relations and base-level relations belonging to an original cluster convey similar relation categories, which demonstrates the rationality of exemplars in relational feature clustering. As
(a) HiURE (higher-level relations)
(b) HiURE
(c) HiURE w/o ExemNCE
(d) HiURE w/o HPC
Figure 5: Visualizing contextualized entity-level features after t-SNE dimension reduction on TACRED dataset.
the number of exemplars between different layers increases, some exemplars are adaptively replaced with more fine-grained ones in the base-level layer.
Note that the approach in this paper works best only when the relational structure in the dataset is hierarchical. Other cases, such as graph structures or binary structures, are untested and may not perform optimally.
Effectiveness of HiNCE. The main purpose of HiNCE is to leverage exemplar-wise contrastive learning in addition to instance-wise. HiNCE avoids the pitfall where many instance-wise negative pairs share similar semantics but are undesirably pushed apart in the feature space. We first conduct an ablation study to demonstrate the effectiveness of this module. From Table 1, HiURE w/o HiNCE gives us  $6.3\%$  less performance averaged over all metrics. Then we report the average performance of  $\mathrm{B}^3$  F1, V-measure F1, and ARI on the two datasets changing with epochs, which reflects the quality and purity of the clusters generated by HiURE. From Figure 3(b), compared to InfoNCE alone, training on HiNCE can improve the performance as training epochs increase, indicating that better representations are obtained to form more semantically meaningful clusters.
Visualize Hierarchical Contextualized Features. To further intuitively show how tree-structured hierarchical exemplars help learn better contextualized relational features on entity pairs for URE, we visualize the contextual representation space  $\mathbb{R}^{(2 + P)\cdot b_R}$
after dimension reduction using t-SNE (Maaten and Hinton, 2008). We randomly choose 400 relations from TACRED dataset and the visualization results are shown in Figure 5.
From Figure 5 (a), we can see that HiURE can give proper clustering results to the higher-level relational features generated by propagation clustering, where features are colored according to their clustering labels. In order to explore how our modules utilize high-level relation features to guide the clustering of base-level relations, we preserve the color series of the corresponding high-level clustering relation labels, while base-level clustering relation labels with different shapes to get Figure 5 (b) (c) (d). HiURE in (b) learns denser clusters and discriminative features. However, HiURE without ExemNCE in (c) is difficult to obtain the semantics of the sentences without exemplar-wise information, which makes the clustering results loose and error-prone. When Hierarchical Propagation Clustering is not applied as (d),  $k$ -means is adopted to perform clustering on the high-level relational features, which could not use exemplars to update relational features or mutual similarity between feature points. On that occasion, HiURE w/o HPC gives the results where the points between clusters are more likely to be mixed. The outcomes revealed above prove the effectiveness of HiURE to obtain the semantics of sentences while distinguishing between similar and dissimilar sentences.
Hyperparameter Analysis. We have explicitly introduced two hyperparameters  $P$  in the encoder and  $L$  in the HPC algorithm. We first study the number of Random Span words  $P$  which affects the fixed-length of relation representation in Eq. 2 by changing  $P$  from 1 to 4 and report the average performance of  $\mathrm{B}^3$  F1, V-measure F1, and ARI on NYT+FB and TACRED. From Table 2, the fluctuation results indicate that both information deficiency and redundancy of relation representations will affect the model's performance. Using short span words will introduce less-information relational features so that is hard to transfer representations from a large scale of sentences, while long span words will cause high computational complexity and lead to information redundancy.
Then, we study the level of  $L$  hierarchical layers as well as the way of building preference sequences to form them, so as to discover the most suitable tree-structured hierarchical relations for the data distribution. We change  $L$  from 2 to 5 with fixed
Dataset / P	1	2	3	4	5
NYT+FB	40.9	42.5	41.3	40.6	39.2
TACRED	51.2	52.4	51.4	50.6	49.8
Dataset / L	2	3	4	5	3+M
NYT+FB	38.6	42.5	40.9	39.2	42.5
TACRED	48.8	52.4	50.4	49.6	52.1
Table 2: Average performance with different number of  $P$  and  $L$  on NYT+FB and TACRED.
top preference  $p_{\top}$  and bottom preference  $p_{\bot}$  to get the effect of  $L$  and report the average performance in Table 2. The fluctuation here implies that fewer layers fail to transfer more information while more layers may cause exemplar-level information conflicts between different coarse-grained layers. (Moiane and Machado, 2018) has shown that the minimum and median value of the similarity matrix are the best preferences for propagation clustering, so we manually adjust the preference sequence between them multiple times with  $L = 3$  and get the average results as  $3 + \mathbb{M}$  to compare with the automatically generated ones by Line 3-4 in HPC. The results show that the bottom layer is not so sensitive to the preference sequence as long as it is reasonable, which proves the practicability and effectiveness of the equation in Line 4.
Unsupervised relation extraction has received attention recently (Simon et al., 2019; Tran et al., 2020; Hu et al., 2020), due to the ability to discover relational knowledge without access to annotations or external resources. Unsupervised models either 1) cluster the relation features extracted from the sentence, or 2) make more assumptions as learning signals to discover better relational representations.
Among clustering models, an important milestone is the self-supervised learning approach (Wiles et al., 2018; Caron et al., 2018; Hu et al., 2020), assuming the cluster assignments as pseudolabels and a classification objective is optimized. However, these works heavily rely on a frequently re-initialized linear classification layer which interferes with representation learning. Zhan et al. (2020) proposes Online Deep Clustering that performs clustering and network update simultaneously rather than alternatingly to tackle this concern, however, the noisy pseudo labels still affect feature clustering when updating the network (Hu et al., 2021a; Li et al., 2022b; Lin et al., 2022).
Inspired by the success of contrastive learning in computer vision tasks (He et al., 2020; Li et al.,
2021; Caron et al., 2020), instance-wise contrastive learning in information extraction tasks (Peng et al., 2020; Li et al., 2022a), and large pre-trained language models that show great potential to encode meaningful semantics for various downstream tasks (Devlin et al., 2019; Soares et al., 2019; Hu et al., 2021b), we proposed a hierarchical exemplar contrastive learning schema for unsupervised relation extraction. It has the advantages of supervised learning to capture high-level semantics in the relational features instead of exploiting base-level sentence differences to strengthen discriminative power and also keeps the advantage of unsupervised learning to handle the cases where the number of relations is unknown in advance.
In this paper, we propose a contrastive learning framework model HiURE for unsupervised relation extraction. Different from conventional self-supervised models which either endure gradual drift or perform instance-wise contrastive learning without considering hierarchical relation structure, our model leverages HPC to obtain hierarchical exemplars from relational feature space and further utilizes exemplars to hierarchically update relational features of sentences and is optimized by performing both instance and exemplar-wise contrastive learning through HiNCE and propagation clustering iteratively. Experiments on two public datasets show the effectiveness of HiURE over competitive baselines.
We thank the reviewers for their valuable comments. The work was supported by the National Key Research and Development Program of China (No. 2019YFB1704003), the National Nature Science Foundation of China (No. 62021002 and No. 71690231), NSF under grants III-1763325, III-1909323, III-2106758, SaTC-1930941, Tsinghua BNRist and Beijing Key Laboratory of Industrial Bigdata System and Application.
Amit Bagga and Breck Baldwin. 1998. Entity-based cross-document core erencing using the vector space model. In Proc. of ACL-IJCNLP, pages 79-85.
Kurt Bollacker, Colin Evans, Praveen Paritosh, Tim Sturge, and Jamie Taylor. 2008. Freebase: a collabor-
ratively created graph database for structuring human knowledge. In Proc. of SIGMOD, pages 1247-1250. AcM.
Antoine Bordes, Sumit Chopra, and Jason Weston. 2014. Question answering with subgraph embeddings. In Proc. of EMNLP, pages 615-620.
Mathilde Caron, Piotr Bojanowski, Armand Joulin, and Matthijs Douze. 2018. Deep clustering for unsupervised learning of visual features. In Proc. of ECCV, pages 132-149.
Mathilde Caron, Ishan Misra, Julien Mairal, Priya Goyal, Piotr Bojanowski, and Armand Joulin. 2020. Unsupervised learning of visual features by contrasting cluster assignments. NeurIPS, 33.
Francesco Corcoglioni, Mauro Dragoni, Marco Rospocher, and Alessio Palmero Aprosio. 2016. Knowledge extraction for information retrieval. In European Semantic Web Conference, pages 317-333. Springer.
James R Curran, Tara Murphy, and Bernhard Scholz. 2007. Minimising semantic drift with mutual exclusion bootstrapping. In Proc. of PACL, volume 6, pages 172-180. Bali.
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proc. of NAACL-HLT, pages 4171-4186.
Brendan J Frey and Delbert Dueck. 2007. Clustering by passing messages between data points. science, 315(5814):972-976.
Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick. 2020. Momentum contrast for unsupervised visual representation learning. In Proc. of CVPR, pages 9729-9738.
Xuming Hu, Fukun Ma, Chenyao Liu, Chenwei Zhang, Lijie Wen, and Philip S Yu. 2021a. Semi-supervised relation extraction via incremental meta self-training. In Proc. of EMNLP: Findings.
Xuming Hu, Lijie Wen, Yusong Xu, Chenwei Zhang, and Philip Yu. 2020. SelfORE: Self-supervised relational feature learning for open relation extraction. In Proc. of EMNLP, pages 3673-3682, Online. Association for Computational Linguistics.
Xuming Hu, Chenwei Zhang, Yawen Yang, Xiaohe Li, Li Lin, Lijie Wen, and S Yu Philip. 2021b. Gradient imitation reinforcement learning for low resource relation extraction. In Proc. of EMNLP, pages 2737-2746.
Lawrence Hubert and Phipps Arabie. 1985. Comparing partitions. Journal of classification, 2(1):193-218.
Junnan Li, Pan Zhou, Caiming Xiong, Richard Socher, and Steven CH Hoi. 2021. Prototypical contrastive learning of unsupervised representations. ICLR.
Shu'ang Li, Xuming Hu, Li Lin, and Lijie Wen. 2022a. Pair-level supervised contrastive learning for natural language inference. In Proc. of ICASSP.
Xiaohe Li, Lijie Wen, Yawen Deng, Fuli Feng, Xuming Hu, Lei Wang, and Zide Fan. 2022b. Graph neural network with curriculum learning for imbalanced node classification. arXiv preprint arXiv:2202.02529.
Li Lin, Yixin Cao, Lifu Huang, Shuang Li, Xuming Hu, Lijie Wen, and Jianmin Wang. 2022. Inferring commonsense explanations as prompts for future event generation. In Proc. of SIGIR.
Fangchao Liu, Lingyong Yan, Hongyu Lin, Xianpei Han, and Le Sun. 2021. Element intervention for open relation extraction. In Proc. of ACL, pages 4683-4693, Online. Association for Computational Linguistics.
Ilya Loshchilov and Frank Hutter. 2017. Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101.
Minh-Thang Luong, Hieu Pham, and Christopher D Manning. 2015. Effective approaches to attention-based neural machine translation. In Proc. of EMNLP, pages 1412-1421.
Laurens van der Maaten and Geoffrey Hinton. 2008. Visualizing data using t-sne. JLMR, 9(Nov):2579-2605.
Andrea Madotto, Chien-Sheng Wu, and Pascale Fung. 2018. Mem2seq: Effectively incorporating knowledge bases into end-to-end task-oriented dialog systems. In Proc. of ACL, pages 1468-1478.
Diego Marcheggiani and Ivan Titov. 2016. Discrete-state variational autoencoders for joint discovery and factorization of relations. TACL, 4:231-244.
Andre Fenias Moiane and Álvaro Muriel Lima Machado. 2018. Evaluation of the clustering performance of affinity propagation algorithm considering the influence of preference parameter and damping factor. Boletim de Ciências Geólicas, 24:426-441.
Aaron van den Oord, Yazhe Li, and Oriol Vinyals. 2018. Representation learning with contrastive predictive coding. arXiv preprint arXiv:1807.03748.
Hao Peng, Tianyu Gao, Xu Han, Yankai Lin, Peng Li, Zhiyuan Liu, Maosong Sun, and Jie Zhou. 2020. Learning from context or names? an empirical study on neural relation extraction. In Proc. of EMNLP, pages 3661-3672.
Andrew Rosenberg and Julia Hirschberg. 2007. Vmeasure: A conditional entropy-based external cluster evaluation measure. In Proc. of EMNLP-CoNLL, pages 410-420.
Evan Sandhaus. 2008. The new york times annotated corpus. Linguistic Data Consortium, Philadelphia, 6(12):e26752.
Étienne Simon, Vincent Guigue, and Benjamin Piwowarski. 2019. Unsupervised information extraction: Regularizing discriminative approaches with relation distribution losses. In Proc. of ACL, pages 1378-1387.
Livio Baldini Soares, Nicholas FitzGerald, Jeffrey Ling, and Tom Kwiatkowski. 2019. Matching the blanks: Distributional similarity for relation learning. In Proc. of ACL, pages 2895-2905.
Thy Thy Tran, Phong Le, and Sophia Ananiadou. 2020. Revisiting unsupervised relation extraction. In Proc. of ACL, pages 7498-7505, Online. Association for Computational Linguistics.
Yutong Wang, Renze Lou, Kai Zhang, Mao Yan Chen, and Yujiu Yang. 2021. More: A metric learning based framework for open-domain relation extraction. In Proc. of ICASSP, pages 7698-7702. IEEE.
Olivia Wiles, A Koepke, and Andrew Zisserman. 2018. Self-supervised learning of a facial attribute embedding from video. arXiv preprint arXiv:1808.06882.
Zhirong Wu, Yuanjun Xiong, Stella X Yu, and Dahua Lin. 2018. Unsupervised feature learning via non-parametric instance discrimination. In Proc. of CVPR, pages 3733-3742.
Limin Yao, Aria Haghighi, Sebastian Riedel, and Andrew McCallum. 2011. Structured relation discovery using generative models. In Proc. of EMNLP, pages 1456-1466. Association for Computational Linguistics.
Xiaohang Zhan, Jiahao Xie, Ziwei Liu, Yew-Soon Ong, and Chen Change Loy. 2020. Online deep clustering for unsupervised representation learning. In Proc. of CVPR, pages 6688-6697.
Chiyuan Zhang, Samy Bengio, Moritz Hardt, Benjamin Recht, and Oriol Vinyals. 2021a. Understanding deep learning (still) requires rethinking generalization. Communications of the ACM, 64(3):107-115.
Kai Zhang, Yuan Yao, Ruobing Xie, Xu Han, Zhiyuan Liu, Fen Lin, Leyu Lin, and Maosong Sun. 2021b. Open hierarchical relation extraction. In Proc. of NAACL-HLT, pages 5682-5693.
Yuhao Zhang, Victor Zhong, Danqi Chen, Gabor Angeli, and Christopher D Manning. 2017. Position-aware attention and supervised data improve slot filling. In Proc. of EMNLP, pages 35-45.