SlowGuess/NAACL_2022_OCR huggingface dataset:
Shuliang Liu $^{1*}$ , Xuming Hu $^{1*}$ , Chenwei Zhang $^{2\dagger}$ , Shu'ang Li $^{1}$ , Lijie Wen $^{1\dagger}$ , Philip S. Yu $^{1,3}$
{lius119,hxm19,lisa18}@mails.tsinghua.edu.cn
cwzhang@amazon.com; wenlj@tsinghua.edu.cn; psyu@uic.edu
Unsupervised relation extraction aims to extract the relationship between entities from natural language sentences without prior information on relational scope or distribution. Existing works either utilize self-supervised schemes to refine relational feature signals by iteratively leveraging adaptive clustering and classification that provoke gradual drift problems, or adopt instance-wise contrastive learning which unreasonably pushes apart those sentence pairs that are semantically similar. To overcome these defects, we propose a novel contrastive learning framework named HiURE, which has the capability to derive hierarchical signals from relational feature space using cross hierarchy attention and effectively optimize relation representation of sentences under exemplar-wise contrastive learning. Experimental results on two public datasets demonstrate the advanced effectiveness and robustness of HiURE on unsupervised relation extraction when compared with state-of-the-art models. Source code is available here<sup>1</sup>.
Relation Extraction (RE) aims to discover the semantic (binary) relation that holds between two entities from plain text. For instance, "Kisselhead was born in Adriantail ..., we can extract a relation /people/person/place_of_birth between the two head-tail entities. The extracted relations could be used in various downstream applications such as information retrieval (Corcoglioni et al., 2016), question answering (Bordes et al., 2014), and dialog systems (Madotto et al., 2018).
Existing RE methods can achieve decent results with manually annotated data or human-curated knowledge bases. While in practice, human annotation can be labor-intensive to obtain and hard
to scale up to newly created relations. Lots of efforts are devoted to alleviating the impact of human annotations in relation extraction. Unsupervised Relation Extraction (URE) is especially promising since it does not require any prior information on relation scope and distribution. The main challenge in URE is how to cluster semantic information of sentences in the relational feature space.
Simon et al. (2019) adopted skewness and dispersion losses to enforce the relation classifier to be confident in the relational feature prediction and ensure all relation types can be predicted averagely in a minibatch. But it still requires the exact number of relation types in advance, and the relation classifier could not be improved by obtained clustering results. Hu et al. (2020) encoded relational feature space in a self-supervised method that bootstraps relational feature signals by leveraging adaptive clustering and classification iteratively. Nonetheless, like other self-training methods, the noisy clustering results will iteratively result in the model deviating from the global minima, which is also known as the gradual drift problem (Curran et al., 2007; Zhang et al., 2021a).
Peng et al. (2020) leveraged contrastive learning to obtain a flat metric for sentence similarity in a relational feature space. However, it only considers the relational semantics in the feature space from an instance perspective, which will treat each sentence as an independent data point. As scaling up to a larger corpus with potentially more relations in a contrastive learning framework, it becomes more frequent that sentence pairs sharing similar semantics are undesirably pushed apart in a flat relational feature space. Meanwhile, we observe that many relation types can be organized in a hierarchical structure. For example, the relations /people/person/place_of_birth and /people/family/country share the same parent semantic on /people, which means that they belong to the same semantic cluster from
<sup>1</sup>https://github.com/THU-BPM/HiURE
Human Language Technologies, pages 5970 - 5980
Figure 1: Framework of HiURE. Sentence representations will be augmented through Random Spans with fixed Entities, then transmitted into Propulsion and Momentum Encoder respectively. The HPC algorithm takes Momentum feature  $H$  as input and generates  $L$  layers of clustering results together with  $L$  exemplar sets  $\mathcal{C}$ . HiNCE takes  $H$  and  $H'$  for instance-wise while  $H$  and  $\mathcal{C}$  for exemplar-wise contrastive learning.
a hierarchical perspective. Unfortunately, these two relations will be pushed away from each other in an instance-wise contrastive learning framework.
Therefore, our intuitive approach is to alleviate the dilemma of similar sentences being pushed apart in contrastive learning by leveraging the hierarchical cluster semantic structure of sentences. Nevertheless, traditional hierarchical clustering methods all suffer from the gradual drift problem. Thereby, we try to exploit a new approach of hierarchical clustering by combining propagation clustering and attention mechanism. We first define exemplar as a representative instance for a group of semantically similar sentences in certain clustering results. Exemplars can be in different granularities and organized in a hierarchical structure. In order to enforce relational features to be more similar to their corresponding exemplars in all parent granularities than others, we propose HiURE, a novel contrastive learning framework for URE that combines both the instance-wise and exemplar-wise learning strategies, to gather more reasonable relation representations and better classification results.
The proposed HiURE model is composed of two modules: Contextualized Relation Encoder and Hierarchical Exemplar Contrastive Learning. As shown in Figure 1, the encoder module leverages the pre-trained BERT model to obtain two augmented entity-level relational features of each sentence for instance-wise contrastive learning, while the learning module retrieves hierarchical exemplars in a top-down fashion for exemplar-wise contrastive learning and updates the features of sentences iteratively according to the hierarchy. These updated features could be utilized to optimize the
parameters of encoders by a combined loss function noted as Hierarchical ExemNCE (HiNCE) in this work. To summarize, the main contributions of this paper are as follows:
- We develop a novel hierarchical exemplar contrastive learning framework HiURE that incorporates top-down hierarchical propagation clustering for URE.
- We demonstrate how to leverage the semantic structure of sentences to extract hierarchical relational exemplars which could be used to refine contextualized entity-level relational features via HiNCE.
- We conduct extensive experiments on two datasets and HiURE achieves better performance than the existing state-of-the-art methods. This clearly shows the superior capability of our model for URE by leveraging different types of contrastive learning. Our ablation analysis also shows the impacts of different modules in our framework.
The proposed model HiURE consists of two modules: Contextualized Relation Encoder and Hierarchical Exemplar Contrastive Learning. As illustrated in Figure 1, the encoder module takes natural language sentences as input, where named entities are recognized and marked in advance, then employs the pre-trained BERT (Devlin et al., 2019) model to output two contextualized entity-level feature sets  $H$  and  $H'$  for each sentence based on Random Span. The learning module takes these features as input, and aims to retrieve exemplars that represent a group of semantically similar sentences in different granularities, denoted as  $\mathcal{C}$ . We
leverage these exemplars to iteratively update relational features of sentences in a hierarchy and construct an exemplar-wise contrastive learning loss called Hierarchical ExemNCE which enforces the relational feature of a sentence to be more similar to its corresponding exemplars than others.
2.1 Contextualized Relation Encoder
The Contextualized Relation Encoder aims to obtain two relational features from each sentence based on the context information of two given entity pairs for instance-wise contrastive learning. In this work, we assume named entities in the sentence have been recognized in advance.
For a sentence  $x = [w_{1},..,w_{T}]$  with  $T$  words where each  $w_{i}$  represents a word and two entities Head and Tail are mentioned, we follow the labeling schema adopted in Soares et al. (2019) and argument  $x$  with four reserved tokens to mark the beginning and the end of each entity. We introduce  $[\mathrm{H_S}], [\mathrm{H_E}], [\mathrm{T_S}], [\mathrm{T_E}]$  to represent the start or end position of head or tail entities respectively and inject them to  $x$ :
$$
\begin{array}{l} x ^ {\prime} = \left[ w _ {1}, \dots , \left[ \mathrm {H} _ {\mathrm {S}} \right], \dots , w _ {i}, \dots , \left[ \mathrm {H} _ {\mathrm {E}} \right], \dots , w _ {S p a n 1}, \dots , \right. \\ w _ {S p a n P}, \dots , [ T _ {S} ], \dots , w _ {j}, \dots , [ T _ {E} ], \dots , w _ {T} ] \tag {1} \\ \end{array}
$$
where  $x'$  will be the input token sequence for the encoder and Span subscript indicates the Random Span words. Considering the relational features between entity pairs are normally embraced in the context, we use pre-trained BERT (Devlin et al., 2019) model to effectively encode every tokens in the sentence along with their contextual information, and get the token embedding  $\mathbf{b}_i = f_{\mathrm{BERT}}(w_i)$ , where  $i \in [1,T]$  including the special tokens in  $x'$  and  $\mathbf{b}_i \in \mathbb{R}^{b_R}$ , where  $b_R$  represents the dimension of the token embedding.
We utilize the outputs  $\mathbf{b}_i$  corresponding to  $[\mathrm{H_S}]$  and  $[\mathrm{T_S}]$  as the contextualized entity-level features instead of using sentence-level marker  $[CLS]$  to get embedding for target entity pair. For contrastive learning purposes, we randomly select  $P$  words as Random Span from all the context words in the whole sentence except for those entity words and special tokens to augment the entity-level features as  $\mathbf{b}_{Span}$ , where multiple different Random Span selections lead to different semantically invariant embedding of the same sentence. For every selection, we concatenate the embedding of the two entity and  $P$  Random Span words together to derive
a fixed-length relational feature  $\mathbf{h} \in \mathbb{R}^{(2 + P) \cdot b_R}$ :
$$
\mathbf {h} = \left[ \mathbf {b} _ {\left[ \mathrm {H} _ {\mathrm {S}} \right]}, \mathbf {b} _ {\left[ \mathrm {T} _ {\mathrm {S}} \right]}, \mathbf {b} _ {\text {S p a n 1}},..., \mathbf {b} _ {\text {S p a n P}} \right] \tag {2}
$$
where  $\mathbf{h}$  is the output of the Contextualized Relation Encoder which can be denoted as  $f_{\theta}(x, \text{Head}, \text{Tail}, \text{Span})$ . The Random Span strategy can get sentence-level enhanced relational features to construct positive samples directly and effectively, and its simplicity highlights the role of subsequent modules.
2.2 Hierarchical Exemplar Contrastive Learning
In order to adaptively generate more positive samples other than sentences themselves to introduce more similarity information in contrastive learning, we design hierarchical propagation clustering to obtain multi-level cluster exemplars as positive samples of corresponding instances. We assume the relation hierarchies are tree-structured and define hierarchical exemplars as representative relational features for a group of semantically similar sentences with different granularities. The exemplar-wise contrastive learning encourages relational features to be more similar to their corresponding exemplars than other exemplars.
The process is completed through Hierarchical Propagation Clustering (HPC) to generate cluster results of different granularities and Hierarchical Exemplar Contrastive Loss (HiNCE) to optimize the encoder. The main procedure of HPC consists of Propagation Clustering and Cross Hierarchy Attention (CHA), as is elaborated in Algorithm 1, which will be explained in detail below.
Propagation Clustering
We use propagation clustering to obtain hierarchical exemplars in an iterative, top-down fashion. Traditional clustering methods such as  $k$ -means cluster data points into specific cluster numbers, however, these methods could not utilize hierarchical information in the dataset and require the specific cluster number in advance. Propagation clustering possesses the following advantages: 1) It considers all feature points as potential exemplars and uses their mutual similarity to extract potential tree-structured clusters. 2) It neither requires the actual number of target relations in advance nor the distribution of relations. 3) It will not be affected by the quality of the initial point selection.
In practice, propagation clustering exchanges real-valued messages between points until a high-quality set of exemplars and corresponding clusters
Algorithm 1 Hierarchical Propagation Clustering
Input: Encoder outputs  $H = \{\mathbf{h}_1,\mathbf{h}_2,\dots ,\mathbf{h}_n\}$  Hierarchical cluster layers  $L$
Output: Hierarchical clusters results  $\mathcal{C}$
1:  $H^1\gets H$ $\mathcal{C}\gets []$
2: Initialize  $\{s_{ij}|i,j\in [1,n]\}$  by Eq. 3
3:  $\forall i\neq j:p_{\top} = \min (s_{ij})$ $p_\perp =$  median  $(s_{ij})$
4:  $ps = \left\{p_l \mid p_l = p_{\top} + \frac{p_{\bot} - p_{\top}}{L - 1} \cdot (l - 1), l \in [1, L] \right\}$
5: for  $l$  in  $[1,L]$  do
6: Update  $\{s_{ij}\}$  according to  $H^{l}$  by Eq. 3
7: Set diagonal to preference  $s_{ii} = p_l$
8: for all iterations do
9: Update  $\{r_{ij}\}$  and  $\{a_{ij}\}$  by Eq. 4 and 5
10:  $\hat{\mathbf{c}} = (\hat{c}_1,\dots ,\hat{c}_n),\hat{c}_i = \mathrm{argmax}_j(a_{ij} + r_{ij})$
11: Exemplar set  $E^{l} = \{\mathbf{e}_{\hat{c}_{i}}^{l}|\mathbf{e}_{\hat{c}_{i}}^{l} = \mathbf{h}_{\hat{c}_{i}}^{l},\hat{c}_{i}\in \hat{\mathbf{c}}\}$
12: if Changes of  $E^l$  have converged then
13: break
14: end if
15: end for
16:  $\mathcal{C}.\mathrm{add}(E^l)$
17:  $H^{l + 1}\gets (H^l,E^l)$  by Eq.8
18: end for
19: return  $\mathcal{C}$
are generated. Inspired by Frey and Dueck (2007), we adopt similarity  $s_{ij}$  to measure the distance between points  $i$  and  $j$ , responsibility  $r_{ij}$  to indicate the appropriateness for  $j$  to serve as the exemplar for  $i$  and availability  $a_{ij}$  to represent the suitability for  $i$  to choose  $j$  as its exemplar:
$$
s _ {i j} = - \left\| \mathbf {h} _ {i} - \mathbf {h} _ {j} \right\| ^ {2} \tag {3}
$$
$$
r _ {i j} = s _ {i j} - \max  _ {j ^ {\prime} \neq j} \left(s _ {i j ^ {\prime}} + a _ {i j ^ {\prime}}\right) \tag {4}
$$
$$
a _ {i j} = \left\{ \begin{array}{c} \sum_ {i ^ {\prime} \neq i} \max  \left(0, r _ {i ^ {\prime} j}\right), j = i \\ \min  \left[ 0, r _ {j j} + \sum_ {i ^ {\prime} \notin \{i, j \}} \max  \left(0, r _ {i ^ {\prime} j}\right) \right], j \neq i \end{array} \right. \tag {5}
$$
where  $r_{ij}$  and  $a_{ij}$  will be updated through the propagation iterations until convergence (Lines 8-15) and a set of cluster centers, which is called exemplar, will be chosen as  $E$  (Line 11). Then we wish to find a set of  $L$  consecutive layers of clustering, where the points to be clustered in layer  $l$  are closer to the corresponding exemplar of layer  $l - 1$ . We perform propagation clustering  $L$  times (Lines 5-18) with different preferences (Lines 2-4) to generate  $L$  different layers of clustering result, where a larger preference leads to more numbers of clusters
Figure 2: Overview of cross hierarchy attention. The first part shows original data. The second part divides data points into two clusters and utilizes attention to update every points which contribute to the next level of clustering. The dotted line indicates negative sample pair while solid line with positive in contrastive learning.
(Moiane and Machado, 2018). The Hyperparameter Analysis part provides a detailed explanation about how to select  $L$  and the reason for building the preference sequence  $ps$  according to the formula in Line 4.
Cross Hierarchy Attention
The traditional hierarchical clustering method either merge fine-grained clusters into coarse-grained one or split coarse cluster into fine-grained ones, which will both cause the problem of error accumulation. Meanwhile, note that the preference sequence leads to different cluster results in a hierarchical way but lost the interaction information between adjacent levels in propagation clustering. Based on this intuition, we introduce the CHA mechanism to leverage signals from coarse-grained exemplars to fine-grained clusters.
Formally, we derive a CHA matrix  $A^l$  at layer  $l$  where the element at  $(j,k)$  is obtained by a scaled softmax:
$$
\alpha_ {j k} ^ {l} = \frac {\exp \left(\lambda \mathbf {e} _ {j} ^ {l} \cdot \mathbf {e} _ {k} ^ {l}\right)}{\sum_ {k ^ {\prime}} \exp \left(\lambda \mathbf {e} _ {j} ^ {l} \cdot \mathbf {e} _ {k ^ {\prime}} ^ {l}\right)} \tag {6}
$$
where  $\lambda$  is a trainable scalar variable, not a hyperparameter (Luong et al., 2015). The attention weight  $\alpha_{jk}^{l}$  reflects the proximity between exemplar  $j$  and exemplar  $k$  in layer  $l$  and measures the influence and interactions to corresponding data points between these exemplars. Typically, exemplars that are visually close to each other would have higher attention weights. Then we derive attended point representation at layer  $l + 1$  by taking the attention weighted sum of its corresponding
exemplar from other exemplars:
$$
\hat {\mathbf {h}} _ {i} ^ {l + 1} = \sum_ {k} \alpha_ {j k} ^ {l} \mathbf {e} _ {k} ^ {l} \tag {7}
$$
where  $\mathbf{e}_j^l$  is the exemplar of  $\mathbf{h}_i^l$ . The attended representation aggregates signals from other exemplars weighted by how close they are to exemplar  $\mathbf{e}_j^l$  and transfer the signals from layer  $l$  to  $l + 1$ . They reflect how likely a neighboring cluster is relevant or the point will get close to it. Then we combine the attended representation with the original one to obtain the CHA based embedding  $\mathbf{h}_i^{l + 1}$ , defined as:
$$
\mathbf {h} _ {i} ^ {l + 1} = \mathbf {h} _ {i} ^ {l} + \lambda_ {\text {a t t}} \hat {\mathbf {h}} _ {i} ^ {l + 1} \tag {8}
$$
where  $\lambda_{att}$  is not a hyper-parameter, but a weighting variable to be automatically trained. As illustrated in Figure 2, the CHA mechanism helps data points to get closer with corresponding exemplars in previous layer and thus perform better in the current layer.
Hierarchical Exemplar Contrastive Loss
Given a training set  $X = \{x_{1}, x_{2}, \dots, x_{n}\}$  of  $n$  sentences, Contextualized Relation Encoder can obtain two augmented relational features for each input sentences by randomly sampling spans twice for the same entity pair. We do this for all sentences and obtain  $H = \{\mathbf{h}_1, \mathbf{h}_2, \dots, \mathbf{h}_n\}$  and  $H' = \{\mathbf{h}_1', \mathbf{h}_2', \dots, \mathbf{h}_n'\}$ . Traditional instance-wise contrastive learning treats two features as a negative pair as long as they are from different instances regardless of their semantic similarity. It updates encoder by optimizing InfoNCE (Oord et al., 2018; Peng et al., 2020):
$$
\mathcal {L} _ {\text {I n f o N C E}} = \sum_ {i = 1} ^ {n} - \log \frac {\exp \left(\mathbf {h} _ {i} \cdot \mathbf {h} _ {i} ^ {\prime} / \tau\right)}{\sum_ {j = 1} ^ {J} \exp \left(\mathbf {h} _ {i} \cdot \mathbf {h} _ {j} ^ {\prime} / \tau\right)} \tag {9}
$$
where  $\mathbf{h}_i$  and  $\mathbf{h}_i^{\prime}$  are positive samples for instance  $i$ , while  $\mathbf{h}_j^{\prime}$  includes one positive sample and  $J - 1$  negative samples for other sentences, and  $\tau$  is a temperature hyper-parameter (Wu et al., 2018).
Compared with the traditional instance-wise contrastive learning which unreasonably pushes apart many negative pairs that possess similar semantics, we employ the inherent hierarchical structure in relations. As illustrated in Figure 1, we perform the HPC algorithm iteratively at each epoch to utilize hierarchical relational features. Note that the relational feature  $\mathbf{h}_i$  will be updated in each batch while training, but the exemplars will not be retrieved until the epoch is finished. To maintain
the invariance of exemplars and avoid representation shift problems with the relational features in an epoch, we need to smoothly update the parameters of the encoder to ensure a fairly stable relational feature space. In practice, we construct two encoders: Momentum Encoder  $f_{\theta}$  and Propulsion Encoder  $f_{\theta'}$ , both of which is a instance of the Contextualized Relation Encoder.  $\theta'$  is updated by contrastive learning loss and  $\theta$  is a moving average of the updated  $\theta'$  to ensure a smoothly update of relational features (He et al., 2020). We leverage HPC on the momentum features  $\mathbf{h}_i = f_{\theta}(x_i)$  to obtain  $\mathcal{C}$  (Line 19), which contains  $L$  layers of cluster results with  $c_l$  exemplars respectively, where  $c_l$  is the number of clusters at layer  $l$ . In order to enforce the relational features more similar to their corresponding exemplars compared to other exemplars (Caron et al., 2020; Li et al., 2021), we define exemplar-wise contrastive learning as ExemNCE:
$$
\mathcal {L} _ {\text {E x e m N C E}} = - \sum_ {i = 1} ^ {n} \frac {1}{L} \sum_ {l = 1} ^ {L} \log \frac {\exp \left(\mathbf {h} _ {i} \cdot \mathbf {e} _ {j} ^ {l} / \tau\right)}{\sum_ {q} ^ {c l} \exp \left(\mathbf {h} _ {i} \cdot \mathbf {e} _ {q} ^ {l} / \tau\right)} \tag {10}
$$
where  $j \in [1, c_l]$ ,  $\mathbf{e}_j^l$  is the corresponding exemplar of instance  $i$  at layer  $l$  and  $q$  indicates all the exemplars from 1 to  $c_l$  at layer  $l$ . As we have explicitly constrained  $\mathbf{h}_i$  and  $\mathbf{e}_j^l$  into approximate feature space, so the temperature parameter  $\tau$  can be shared here. The difference between InfoNCE and ExemNCE is described in the second part of Figure 2, where the solid line represents positive while the dashed line represents negative.
Furthermore, we add InfoNCE loss to retain the local smoothness which could help propagation clustering. Overall, our objective named Hierarchical ExemNCE is defined as:
$$
\mathcal {L} _ {\mathrm {H i N C E}} = \mathcal {L} _ {\text {I n f o N C E}} + \mathcal {L} _ {\text {E x e m N C E}} \tag {11}
$$
After we update Propulsion Encoder  $f_{\theta'}$  with HiNCE, the Momentum Encoder  $f_{\theta}$  can be propelled by:
$$
\theta \leftarrow m \cdot \theta + (1 - m) \cdot \theta^ {\prime} \tag {12}
$$
where  $m \in [0,1)$  is a momentum coefficient. The momentum update in Eq. 12 makes  $\theta$  evolve more smoothly than  $\theta'$  especially when  $m$  is closer to 1.
We conduct extensive experiments on real-world datasets to prove the effectiveness of our model for
Unsupervised Relation Extraction tasks and give a detailed analysis of each module to show the advantages of HiURE. Implementation details and evaluation metrics are illustrated in Appendix A and B respectively.
Following previous work (Simon et al., 2019; Hu et al., 2020; Tran et al., 2020), we employ  $\mathrm{NYT + FB}$  to train and evaluate our model. The  $\mathrm{NYT + FB}$  dataset is generated via distant supervision, aligning sentences from the New York Times corpus (Sandhaus, 2008) with Freebase (Bollacker et al., 2008) triplets. We follow the setting in Hu et al. (2020); Tran et al. (2020) and filter out sentences with non-binary relations. We get 41,685 labeled sentences containing 262 target relations (including no Relation) from 1.86 million sentences.