Year: 2021

Abstract:
Recognizing relations between entities is a pivotal task of relational learning. Learning relation representations from distantly-labeled datasets is difficult because of the abundant label noise and complicated expressions in human language. This paper aims to learn predictive, interpretable, and robust relation representations from distantly-labeled data that are effective in different settings, including supervised, distantly supervised, and few-shot learning. Instead of solely relying on the supervision from noisy labels, we propose to learn prototypes for each relation from contextual information to best explore the intrinsic semantics of relations. Prototypes are representations in the feature space abstracting the essential semantics of relations between entities in sentences. We learn prototypes based on objectives with clear geometric interpretation, where the prototypes are unit vectors uniformly dispersed in a unit ball, and statement embeddings are centered at the end of their corresponding prototype vectors on the surface of the ball. This approach allows us to learn meaningful, interpretable prototypes for the final classification. Results on several relation learning tasks show that our model significantly outperforms the previous state-of-the-art models. We further demonstrate the robustness of the encoder and the interpretability of prototypes with extensive experiments.

heading: 1 INTRODUCTION
text: Relation extraction aims to predict relations between entities in sentences, which is crucial for understanding the structure of human knowledge and automatically extending knowledge bases (Cohen & Hirsh, 1994; Bordes et al., 2013; Zeng et al., 2015; Schlichtkrull et al., 2018; Shen et al., 2020). Learning representations for relation extraction is challenging due to the rich forms of expressions in human language, which usually contains fine-grained, complicated correlations between marked entities. Although many works are proposed to learn representations for relations from well-structured knowledge (Bordes et al., 2013; Lin et al., 2015; Ji et al., 2015), when we extend the learning source to be unstructured distantly-labeled text (Mintz et al., 2009), this task becomes particularly challenging due to spurious correlations and label noise (Riedel et al., 2010).
This paper aims to learn predictive, interpretable, and robust relation representations from large-scale distantly labeled data. We propose a prototype learning approach, where we impose a prototype for each relation and learn the representations from the semantics of each statement, rather than solely from the noisy distant labels. Statements are defined as sentences expressing relations between two marked entities. As shown in Figure 1, a prototype is an embedding in the representation space capturing the most essential semantics of different statements for a given relation. These prototypes essentially serve as the center of data representation clusters for different relations and are surrounded by statements expressing the same relation. We learn the relation and prototype representations based on objective functions with clear geometric interpretations. Specifically, our approach assumes prototypes are unit vectors uniformly dispersed in a unit ball, and statement embeddings are centered
∗Equal contribution. †Corresponding author. ‡http://ruizhang.info
-8.703243 38.913616 -11.877513 26.187891 -10.24494 31.439125 -11.185886 28.53963 -6.7643037 31.873375 -9.293796 30.790508 -7.6870537 36.014236 -11.216029 31.59263
-7.4329624 35.181496 -10.611754 35.345486 -12.039416 22.71971 -11.616351 26.774305 -8.657383 35.28101 -13.320659 29.52109 -11.843529 26.295746 -10.6841545 28.803246 -7.6381073 36.70645
-9.564307 31.78187 -7.315561 39.579956 -4.533672 34.883198 -8.275788 33.035698 -7.4689636 35.644993 -11.624744 30.369122
-9.997218 38.49715 -7.7783055 37.59266 -9.765418 38.476517 -9.501085 32.574696 -6.3372855 37.55847 -11.9366 27.271059 -12.867382 36.685673
-10.081516 36.11969 -11.364351 34.99714 -13.114578 37.7766 -12.3460045 36.313652 -10.533733 29.321291 -6.8389997 37.80524 -10.056862 37.98478 -11.23021 29.466564 -8.006758 36.853874 -9.294378 29.327047 -10.339967 35.70266 -10.410015 31.533749 -13.232315 35.18631 -13.210859 29.454363 -7.7240295 37.255066 -10.900574 32.073906 -7.5629783 36.904022 -10.104565 31.449959 -8.409077 33.879704 -4.9887133 35.23039 -9.434023 33.882618 -10.956362 38.222942 -12.509354 31.459927
-6.1397 37.212097 -8.249299 36.092 -8.1636095 38.060524 -13.816341 26.780077 -12.954381 37.67792 -8.914301 38.879467
-11.33116 32.713974
15.663114 12.8621025 17.03712 18.486742 17.551886 14.484679
18.931808 18.930895 15.572551 13.700756 10.558792 17.574535 17.613794 13.805676 17.844099 18.073603 10.654739 18.235968 16.538021 10.523316 17.641438 13.43363 14.079281 12.15079 18.864927 14.856975 15.856341 10.3096 14.077647 16.088354 8.66363 17.863062 13.279723 15.911679 19.156492 14.567751 16.535969 16.739618 17.02585 13.34684 18.596682 11.846535 12.980082 18.898485 16.939898 17.028992 11.612106 16.237764 16.605167 17.61748 17.126232 9.916799 18.854584 16.075922 10.663816 15.945908 15.273596 11.521369 13.061167 16.711267 17.905909 10.257096 12.815981 17.505522 15.421573 16.05703 17.434866 14.227275 14.178516 17.211475 16.826227 12.7907715 8.85155 18.680946 16.42301 15.614987 16.469488 15.520174 17.752037 11.194748 13.268249 11.668395 19.25654 19.09535 17.51284 8.887941 10.873803 14.245057 15.503976 14.112884 12.484748 12.027752 12.8171425 9.299756 10.16367 10.303493 19.28029 17.801203 6.9740515 14.092293 16.590027 14.853376 15.17265 18.155851 18.5615 18.52006 17.618689 14.498615 16.670238 16.880407 18.137753
16.441898 14.629623 19.16981 17.200861 18.588112 17.524498 11.210086 17.630442 18.84577
11.128795
-11.828977 -43.852882 -18.523153 -48.35973 -9.013127 -49.965767 -12.775725 -38.4372 -11.386365 -45.235485 -12.341138 -46.811172 -18.695421 -47.257236 -17.548359 -47.813652 -16.86908 -47.66306 -9.285677 -48.134 -15.553001 -47.495358 -14.435563 -47.35719 -15.938199 -44.728043 -10.96651 -50.96271 -9.333371 -47.840088 -8.612561 -50.579475
-9.66584 -45.757843 -14.620562 -36.031902 -13.302847 -47.38101 -12.612855 -44.99101 -11.566471 -48.70189 -14.438774 -45.744644 -13.297511 -51.752514 -12.276336 -48.390713 -8.143216 -49.363167 -14.794755 -45.36748 -9.096069 -49.784893 -9.096887 -48.303566 -11.601617 -40.474926 -8.760579 -48.474392 -8.8828 -51.211826 -14.739173 -13.053048 -14.749782 -44.379173 -13.720553 -47.30534 -13.980023 -38.663837 -16.943655 -45.568115 -7.284974 -47.792404 -13.98673 -38.59544 -15.000847 -42.772976 -11.206605 -39.627068 -14.621026 -37.741215 -5.989932 -49.660408 -7.066158 -50.164024 -13.359581 -41.030807 -11.044324 -48.147877 -10.257005 -41.0706 -8.1137085 -49.74866 -11.536943 -44.25597 -10.309574 -49.382477 -16.535332 -48.246834 -16.57508 -44.80348 -14.411162 -48.468327 -14.510945 -36.572926 -14.163412 -45.338158 -19.041832 -47.55801 -14.49797 -46.043823 -16.699863 -49.536724 -15.524808 -47.48149 -8.445343 -48.45605 -11.151114 -46.65739 -14.092925 -49.13657 -8.212766 -49.940895 -11.3532095 -46.101093 -11.26969 -45.414417 -7.291685 -49.634018 -15.727305 -43.962696 -14.725657 -36.18008 -8.530978 -48.039 -12.263783 -46.4457 -13.111706 -45.93163 -16.514885 -47.25092 -17.156609 -46.77544 10.73023 -60.08112 -8.636485 -48.777756 -15.849547 -46.368256 -14.301521 -36.94852 -13.472748 -47.12356
9.924656 -54.517498 -12.016547 -49.535435 -12.705193 -43.919346 -13.171877 -39.083767 -13.118616 -42.864147 -14.06309 -49.1256 -6.9750905 -51.342743 -10.250488 -47.86121 -6.3860655 -50.226997 -13.797769 -46.049805 -9.539331 -43.63843 -7.4702673 -50.618774 -14.247917 -38.567375 -15.689215 -35.869644 -11.770379 -46.01179 -7.974457 -48.025314 -15.115464 -41.50516 -17.751368 -47.27474 -11.07022 -47.56136 -12.364049 -45.718647 -7.192023 -51.772503 -16.014822 -36.732063 -11.9974165 -42.06392 -9.965427 -51.854122 -13.210664 -38.62058 -8.270923 -49.581097 -11.441111 -49.41309 -8.452865 -13.024228 -14.365504 -41.821922 -15.149126 -40.793488 -13.114061 -43.343094 -15.477967 -42.245445 -16.184212 -41.677425 -15.117426 -47.537777 -11.991396 -41.70241 -12.122491 -44.353127 -14.115153 -36.14243 -13.586221 -45.992226 -13.601769 -44.940388 -10.696865 -45.62354 -9.2235775 -48.957237 -8.406905 -50.76499 -18.666452 -47.53441 -7.9554815 -46.99664 -10.015536 -52.052444 -14.992561 -37.97665 -14.920621 -48.478596
at the end of their corresponding prototype vectors on the surface of the ball. We propose statementstatement and prototype-statement objectives to ensure the intra-class compactness and inter-class separability. Unlike conventional cross-entropy loss that only uses instance-level supervision (which could be noisy), our objectives exploit the interactions among all statements, leading to a predictively powerful encoder with more interpretable and robust representations. We further explore these properties of learned representations with extensive experiments.
We apply our approach using a pretraining fine-tuning paradigm. We first pretrain a relation encoder with prototypes from a large-scale distantly labeled dataset, then fine-tune it on the target dataset with different relational learning settings. We further propose a probing dataset, FuzzyRED dataset (Section 4.3), to verify if our method can capture the underlying semantics of statements. Experiments demonstrate the predictive performance, robustness, and interpretability of our method. For predictive performance, we show that our model outperforms existing state-of-the art methods on supervised, few-shot, and zero-shot settings (Section 4.4 and 4.2). For robustness, we show how the model generalize to zero-shot setting (Section 4.2) and how the prototypes regularize the decision boundary (Section 4.5). For interpretability, we visualize the learned embeddings and their corresponding prototypes (Section 4.4) and show the cluster clearly follow the geometric structure that the objective functions impose. The source code of the paper will be released at https://github.com/ Alibaba-NLP/ProtoRE.

heading: 2 RELATED WORK
text: Relation learning could be mainly divided into three categories. The logic-based methods reason relations via symbolic logic rules, with adopting probabilistic graphical models or inductive logic systems to learn and infer logic rules orienting to relations (Cohen & Hirsh, 1994; Wang et al., 2015; Yang et al., 2017; Chen et al., 2018; Kazemi & Poole, 2018; Qu & Tang, 2019). The graph-based methods encode entities and relations into low-dimensional continues spaces to capture structure features of KBs. (Nickel et al., 2011; Bordes et al., 2013; Yang et al., 2015; Wang et al., 2014; Nickel et al., 2016; Ji et al., 20 ; Lin et al., 15; Trouillon et al., 2016; Sun et al., 2019; Balažević et al., 2019). The text-based methods have been widely explored recently, focusing on extracting semantic features from text to learn relations.
The conventional methods to learn relations from the text are mainly supervised models, like statistical supervised models (Zelenko et al., 2003; GuoDong et al., 2005; Mooney & Bunescu, 2006). As deep neural networks have gained much attention then, a series of neural supervised models have been proposed (Liu et al., 2013; Zeng et al., 2014; Xu et al., 2015; Santos et al., 2015; Zhang & Wang, 2015; Verga et al., 2016; Verga & McCallum, 2016; Li et al., 2019; Distiawan et al., 2019; Ding et al., 2019). To address the issue of the insufficiency of annotated data, distant supervision has been applied to automatically generate a dataset with heuristic rules (Mintz et al., 2009). Accompanying with auto-labeled data, massive noise will be introduced into models. Accordingly, various denoising methods have been explored to reduce noise effects for distant supervision (Zeng et al., 2015; Lin et al., 2016; Jiang et al., 2016; Han et al., 2018a; Wu et al., 2017; Qin et al., 2018a;b; Feng et al.,
2
2018). Further efforts pay attention to learning relations from the text in various specific scenarios, such as zero-few-shot scenarios (Levy et al., 2017; Han et al., 2018b; Gao et al., 2019; Soares et al., 2019; Ye & Ling, 2019) and open-domain scenarios (Banko et al., 2007; Fader et al., 2011; Mausam et al., 2012; Del Corro & Gemulla, 2013; Angeli et al., 2015; Stanovsky & Dagan, 2016; Mausam, 2016; Cui et al., 2018; Shinyama & Sekine, 2006; Elsahar et al., 2017; Wu et al., 2019).
However, up to now, there are few strategies of relation learning that could be adapted to any relation learning tasks. Soares et al. (2019) introduces a preliminary method that learns relation representations from unstructured corpus. And the assumption that two same entities must express the same relation is imprecise, because it does not consider the semantics of contextual information. This paper abstracts the task from the text as a metric and prototype learning problem and proposes a interpretable method for relation representation learning. Apart from the differences in methods and task scenarios, row-less universal schema (Verga & McCallum, 2016) has a similar spirit with our method, where relation type embeddings guide clustering of statements for entity pair representations and embeddings of entity pairs are regarded as an aggregation of relations.
Although the intuition that assigns a representative prototype for each class is similar to some previous studies like Prototypical Networks (Snell et al., 2017) (ProtoNet), there exist some essential distinctions between our approach and the ProtoNet. The ProtoNet computes the prototype of each class as the average of the embeddings of all the instance embeddings, that could be regarded as a non-linear version of the nearest class mean approach (Mensink et al., 2013). The idea of mean-ofclass prototypes could also be traced to earlier studies in machine learning (Graf et al., 2009) and cognitive modeling and psychology (Reed, 1972; Rosch et al., 1976). In our method, prototypes and relation encoder are collaboratively and dynamically trained by three objectives, which divides the high-dimensional feature space into disjoint manifolds. The ProtoNet performs instance-level classification to update the parameters, which is not robust for noisy labels. Our method carries out a novel prototype-level classification to effectively regularize the semantic information. The prototype-level classification reduces the distortion caused by noisy labels of the decision boundary fitted by NNs. Moreover, the ProtoNet is designed for few-zero-shot learning and our methods are more like a semi-supervised pre-training approach that could be applied to supervised, few-shot, and transfer learning scenarios. Furthermore, the ProtoNet does not contain the metric learning among instances, and our method simultaneously optimizes the prototype-statement and statement-statement metrics. We also make a geometry explanation of our method (Section 3.1).

heading: 3 METHOD
text: Our method follows a pretraining-finetuning paradigm. We first pretrain an encoder with distantly labeled data, then fine-tune it to downstream learning settings. We start with the pretraining phase. Given a large-scale, distantly labeled dataset containing training pairs (w, r) where w = [w1, ..., wn] is a relation statement, i.e., a sequence of words with a pair of entities [h, t] marked, h being the head and t being the tail, ∃i1, i2, h = wi1:i2 ,∃j1, j2, t = wj1:j2 , r is the relation between h and t. r is a discrete label and is probably noisy. We aim to learn an encoder Encφ(·) parameterized by φ that encodes w into a representation s ∈ Rm, m being the dimension:
s = Encφ(w). (1)
A prototype z for relation r is an embedding in the same metric space with s that abstracts the essential semantics of r. We use [(z1, r1), ..., (zK , rK)], zk ∈ Rm to denote the set of prototype-relation pairs. K is the number of different relations. zk is the prototype for relation rk and superscripts denote the index of relation type. Given a batch B = [(s1, r1), ..., (sN , rN )], N is the batch size and subscripts denote batch index, the similarity metric between two statement embeddings d(si, sj), and between a statement embedding and a prototype d(s, z) are defined as:
d(si, sj) = 1/(1 + exp( si ||si|| · sj ||sj || )), d(s, z) = 1/(1 + exp( s ||s|| · z ||z|| )). (2)
Geometrically, this metric is based on the angles of the normalized embeddings restricted in a unit ball. We will explain the geometric implication of this metric in the next section. Inspired by Soares et al. (2019), we define a contrastive objective function between statements:
LS2S = − 1
N2 ∑ i,j exp(δ(si, sj)d(si, sj))∑ j′ exp((1− δ(si, sj′))d(si, sj′)) , (3)
where δ(si, sj) denotes if si and sj corresponds to the same relations, i.e., given (si, ri), (sj , rj), δ(si, sj) = 1 if ri = rj else 0. The computation of the numerator term forces that statements with same relations to be close, and the denominator forces that those with different relations are dispersed. When constructing the batch B, we equally sample all relation types to make sure the summation in the denominator contains all relations. Essentially, equation 3 ensures intra-class compactness and inter-class separability in the representation space. Additionally, equation 3 contrast one positive sample to all the negative samples in the batch, which puts more weights on the negative pairs. We find it empirically more effective than previous objectives like Soares et al. (2019).

heading: 3.1 LEARNING PROTOTYPES
text: Now we discuss the objective functions for learning prototypes. Denote S = [s1, ..., sN ] as the set of all embeddings in the batch B, given a fixed prototype zr for relation r, we denote Sr the subset of all statements si in S with relation r, and S−r the set of the rest statements. Z−r as the set of prototypes z′ for all other relations except r. We impose two key inductive biases between prototypes and statements: (a) for a specific relation r, the “distance” between zr and any statements with the same relation r should be less than the “distance” between zr and any statements with relations r′ 6= r. (b) the “distance” between zr and any statements with relation r should be less than the “distance” between any prototypes z′ ∈ Z−r and statements with relation r. To realize these two properties, we define two objective functions:
LS2Z = − 1
N2 ∑ si∈Sr,sj∈S−r [ log d(zr, si) + log(1− d(zr, sj)) ] , (4)
LS2Z’ = − 1
N2 ∑ si∈Sr,z′∈Z−r [ log d(zr, si) + log(1− d(z′, si)) ] , (5)
where equation 4 corresponds to (a) and equation 5 corresponds to (b). These objectives effectively splits the data representations into K disjoint manifolds centering at different prototypes. We further highlight the differences between equation 4 and 5 and a conventional cross-entropy loss: there is no interactions between different statements in the cross-entropy loss that solely relies on the instance level supervision, which is particularly noisy under a noisy-label setting. On the other hand, our loss functions consider distances between different statements and prototypes, which exploits the interactions between statements. This type of interaction would effectively serve as a regularization to the decision boundary, as we will empirically verify in section 4.5.
Combining equations 2, 3, 4 and 5, we further give a geometric explanation of the representation space: a prototype is a unit vector (because we normalize the vector length in equation 2) starting from the origin and ending at the surface of a unit ball, and statements for that prototypes are unit vectors with approximately same directions centering at the prototype (because our objective functions 3, 4, 5 push them to be close to each other). Under the optimal condition, different prototype vectors would be uniformly dispersed with the angles between them as large as possible (because the distance metric is based on the angle in equation 2 where the minimum is at π, and we maximize distances between prototypes in equation 4 and 5). Consequently, the dataset is clustered with each cluster centered around the end of one prototype vector on the surface of the unit ball. The illustration of the intuition is shown in Figure 2.
To further regularize the semantics of the prototypes, we use a prototype-level classification objective:
LCLS = 1
K ∑ k log pγ(r k|zk), (6)
where γ denotes the parameters of an auxiliary classifier. Our prototype essentially serves as a regularization averaging the influence of label noises. We further validate this regularization in Section 4.5 by showing that it reduces the distortion of the decision boundary caused by noisy labels. Finally, with hyper-parameters λ1, λ2 and λ3, the full loss is:
L = λ1LS2S + λ2(LS2Z + LS2Z) + λ3LCLS. (7)

heading: 3.2 FINE-TUNING ON DIFFERENT LEARNING SETTINGS
text: We apply our learned encoder and prototypes to two typical relational learning settings: (a) supervised; (b) few-shot. Under the supervised learning setting, having the pretrained encoder at hand, we finetune the model on a dataset of a target domain (which could again, be distantly labeled) containing new training pairs (w, r), we encode w to the representation and feed it to a feed-forward classifier:
p(r|w) = FF(Encφ(w)), (8) where FF(·) denotes a feed-forward layer, and we train it with the conventional cross-entropy loss. Under the few-shot learning setting, we assume a slightly different input data format. Specifically, the training set consists of the relations [r1, ..., rK ] and a list of supporting statements: sk1 , ..., s k L for all rk. Denote S? = {skl } k=1,...,K l=1,...,L all supporting statements for all relations. Given a query statement q, the task is to classify the relation r∗ that q expresses. During traing, we use the average distance to different prototypes as the logits of a feed-forward classifier and train the model with cross-entropy loss. During testing, we classify q according to the minimum similarity metrics:
sk ∗
l∗ = argmin skl ∈S?
d(skl ,Encφ(q)), r ∗ = k∗. (9)
Note this is equivalent to choosing the argmax of the logits because the logits are formed by the similarity scores. We note that model pre-trained by our approach performs surprisingly well even with no training data for fine-tuning, which is reported in Figure 3 and 4 in Section 4.2.

heading: 4 EXPERIMENTS
text: In order to evaluate the performance of prototypical metrics learning, we conduct extensive experiments on three tasks: supervised relation learning, few-shot learning and our proposed fuzzy relation learning evaluation. We make a comprehensive evaluation and analysis of our method, as well as full-scale comparisons between our work with previous state-of-the-art methods.

heading: 4.1 EXPRIMENTAL DETAILS
text: Pretraining dataset We prepare the weakly-supervised data by aligning relation tuples from the Wikidata database to Wikipedia articles. Specifically, all entities in wikipedia sentences are identified by a named entity recognition system. For an entity pair (h, t) in a sentence w, if it also exists in the knowledge base with a relation r, then w will be distantly annotated as r. After processing, we collect more than 0.86 million relation statements covering over 700 relations.
Baseline Models We primarily compare our model with MTB because it is a previous SOTA model. For fair comparison, we re-implement MTB with BERTbase (Devlin et al., 2018) and pretrain it on our collected distant data as same as our methods. We note under our setting, the reported number of MTB is smaller than its original paper because: (a) the original paper uses much larger dataset than ours (600 million vs 0.86 million); (b) they use BERTlarge encoder. In addition, multiple previous state-of-the-art approaches are picked up for comparison in all the three relation learning tasks (detailed later). For baseline models (PCNN, Meta Network, GNN, Prototypical Network, MLMAN) with publicly available source code, we run the source code manually, and if the produced results are close to those in the original paper, we would select and report the results in the original paper. For ones without open source code (BERTEM, MTB), we re-implement the methods with BERTbase pre-trained on our distant data and report the results. We also implement a baseline that directly trains BERTbase on our distant data with cross-entropy loss (LCE), namely BERTCE in the section (the encoder is pre-trained by directly predicting the distant label for each statement).
The IND baseline We additionally design a baseline, where the prototypes are pre-computed by vectorizing the extracted relation patterns independently (IND). In this way, for each statement, we pre-compute a fixed metric as its weight demonstrating the similarity with the corresponding prototype. The IND baseline is primarily for validating the effectiveness of trained prototypes over the rule-based prototypes. Given the distantly-labeled training set, we use the Snowball algorithm from Agichtein & Gravano (2000) to extract relation patterns. This algorithm takes a corpus of relational instances as inputs and outputs a list of patterns and their embeddings for each relation. After getting these patterns, we use patterns to match instances and calculate how many instances can each pattern match. For each relation label, we select the top-k patterns that match the most instances
and use the average of their embeddings as the prototype embedding. Note that these embeddings are not comparable to the sentence embeddings generated by our encoders, so the loss functions in our equation 4 and 5 are not usable for the prototypes generated here. To incorporate the information of the extracted prototype embeddings, we slightly modify equation 3 and use the similarity of instances to the embeddings to weight the instance embeddings. The modified loss function is:
LIND = − 1
N2 ∑ i,j exp(δ(si, sj)d(s ′ i, s ′ j))∑ j′ exp((1− δ(si, sj′))d(s′i, s′j′)) , (10)
s′i = sim(wi, z ′(wi)) · si, s′j = sim(wj , z′(wj)) · sj , s′j = sim(w′j , z′(w′j)) · s′j , (11)
where wi denote the corresponding sentence to si, z′(wi) denote the correspond prototype.
Implementation Details For encoding relation statements into a feature space, we adopt deep Transformers (Vaswani et al., 2017), specifically BERT, as Eθ. We take the input and the output of the BERT encoder as follows: given a statement s, we add special entity markers to mark Mh and Mt before the entities, then input the statement and special markers to Encφ to compute final representations. Note that our framework is designed independently to the encoder choice, and other neural architectures like CNN (Zeng et al., 2014) and RNN (Zhang & Wang, 2015) can be easily adopted as the encoder. We use PyTorch (Paszke et al., 2019) framework to implement our model. All the experiments run on NVIDIA Tesla V100 GPUs. The encoder is optimized with the combination of the prototype-statement and statement-statement metrics as well as the masked language model loss with the settings as follows. For the sake of saving resources, we choose BERTbase as the backbone instead of BERTlarge, although our method sacrifices a considerable part of the performance. We select AdamW (Loshchilov & Hutter, 2018) with the learning rate of 1e− 5 for optimization. Meanwhile, Warmup (Popel & Bojar, 2018) mechanism is used during training. The number of layers is 12, and the number of heads is 12. The hidden size is set to 768 and the batch size is set to 60. We train the model for 5 epochs in each experiment.

heading: 4.2 FEW-SHOT RELATION LEARNING
text: Dataset We use a large few-shot relation learning dataset FewRel (Han et al., 2018b) in this task. FewRel consists of 70000 sentences (about 25 tokens in each sentence) of 100 relations (700 sentences for each relation), on the basis of Wikipedia. We utilize the official evaluation setting which splits the 100 relations into 64, 16, 20 relations for training, validation, and testing respectively.
Experimental settings We use accuracy as the evaluation metric in this task. The batch size for few-shot training is 4, the training step is 20000 and the learning rate is 2e-5 with AdamW. We follow the original meaning of N-way-K-shot in this paper, please refer to (Han et al., 2018b) for details.
Results and analysis The results for few-shot relation learning are illustrated in Table 1. It is worth noting that we report the average score rather than the best score of each method for the sake of fairness. One observation is that the task is challenging without the effectiveness of pre-trained language models. Prototypical network (CNN) only yields 69.20 and 56.44 accuracies (%) for 5 way 1 shot and 10 way 1 shot respectively. Methods that have achieved significant performances on supervised RE (such as PCNN, GNN, and Prototypical Network) suffer grave declines. MTB learns the metrics between statements based on large pre-trained language models and yield surprising results on this task. Better performance gains by considering the semantic information by applying fixed-prototypes to MTB (IND strategy). The collaborative prototype-based methods are proven to be effective as shown in the last four rows. The results of COL (LS2S′ +LZ2S +LZ2S′ ) indicate that the loss of MTB have little influence for the performance. Finally, our final collaborative model achieve the best performance on all the N -way K-shot settings. Our method also outperforms human in 5 way 1 shot setting (92.41 vs 92.22) and 10 way 1 shot setting (86.39 vs 85.88).
Effect of the amount of training data In real-world scenarios, it is costly and labor-consuming to construct a dataset like FewRel containing 70,000 positive instances. Some relations rarely appear in public articles and obtain candidate instances via distance supervision introduces much noise. Therefore, the amount of instances to annotate may much larger than 70, 000. To investigate the ability of few-shot classification when lack of high-quality task-specific training data, we intentionally control the amount of instances in training data and run evaluations with the new training set. Figure 3 and Figure 4 respectively display the accuracy of the 5-way-1-shot task given a limited number
71.9
80.5
86.0
87.9 88.7
81.3 81.9
87.7 89.8
89.189.9
85.6
88.6 90.4
91.1
71.9
80.5
86.0
87.9 88.7
81.3 81.9
87.7 89.8
89.189.9
85.6
88.6 90.4
91.1
1
Figure 4: Impact of # of instances per type
of relation types or amount of instances of each relation in training data. As shown in the figures, pre-trained relation encoders outperform the basic BERTEM model when short of training data. Encoder learned with our proposed method shows large superiority over the others, when there is no training data (0), the absolute improvement of our framework is 17. 6. Besides, the accuracy increase with the increment of training data regardless of providing more relations or more instances of each relation. However, the diversity of relations is more important, since the performance in Figure 4 outperforms those in Figure 3 while the former is trained on a much smaller training data. Lack of diversity even hurts the performance when applying relation encoder which is learned with prototype-statement metric. Similar conclusions have been mentioned in Soares et al. (2019).

heading: 4.3 FUZZY RELATION EVALUATION
text: Dataset Aimed at evaluating relation representation learning on the noisy weakly supervised corpus, we propose a new evaluation dataset named FuzzyRED to quantitatively evaluate the capture of relational information. FuzzyRED is constructed aimed at the shortcoming of distant supervision, that is not all the statements contain the same entity pairs express the same relation. All the instances are from the weakly-supervised data and manually annotated binary labels to judge if the sentence semantically expresses the relation. For example, for “Jack was born in London, UK”, even if the relation Capital is correct between the entities in knowledge base, the sentence does not express the capital relationship and will be labeled as False in FuzzyRED. The dataset contains 1000 manually annotated instances covering 20 relations. For detailed information please refer to Appendix A.
Experimental settings The pre-trained relation encoders are directly utilized on FuzzyRED and expected to separate the false positive instances from true positives. For each relation, we calculate the minimum of statement-statement distances among all true positives as the threshold T . Then for
7
each instance s to be classified, we sample 1 K (k-shot) true positive instances s′ ∈ {s1, s2, ..., sK} and compare the average score of a = p(l = 1|s, s′) with T . If a > T , the instance is classified as positive. FuzzyRED is considerably challenging since it is designed for the issue of distant supervision and directly reflects the gap between semantic relations and annotation information.
Results The accuracies of FuzzyRED are reported in Table 2. The accuracy of labels annotated according to the DS assumption is denoted by DS, which indicates that the majority of the dataset are noise. From the experimental results, we could firstly observe that all the relation encoders suffer from the noise issue and get lower results. The impact of noise increases with the k increasing. But Table 2 still shows that pre-trained relation encoders do de-
tect a few noise data thus they all yield better performance than DS. Taking consideration of noise by prototypes leads to better performances, and the fixed prototypes play a small role to detect the complex semantics that express relations. Even though the COL model achieves remarkable improvement, the evaluation task is still very challenging because of the fuzziness of relation semantics.

heading: 4.4 SUPERVISED RELATION LEARNING
text: Dataset We use classic benchmark dataset SemEval 2010 Task 8 (Hendrickx et al., 2010) as the dataset for supervised relation learning. SemEval 2010 Task 8 contains nine different pre-defined bidirectional relations and a None relation, thus the task is a 19-way classification. For model selection, we randomly sample 1500 instances from the official training data as the validation set.
Experimental settings For evaluation, we use standard precision, recall, and Fmeasure for supervised RE task. Since the annotation schema of SemEval dataset differs from the data for the encoder training, we fine-tune the relation encoder in SemEval training data. The batch size is set to 20, the training epoch is 30, the learning rate is 1e-5 with Adam (Kingma & Ba, 2014). We note that the numbers for MTB are different from
their paper because their original setting uses BERTlarge with 600 million instances and is trained on TPUs. Given the restricted resources at hand, to make fair comparison, we re-implemented their model with BERTbase and pretrain it in our setting (0.86 million instances).
-35.306133 43.764988 -32.10036 47.11178 -34.310684 49.74119 -34.202057 48.316734 -34.083035 45.261963 -32.96691 49.549446 -34.994102 45.641056 -36.779217 49.506325 -34.004677 46.21778 -37.055138 50.32413 -33.685005 47.60056 -30.295895 49.543644 -36.494568 48.82859 -34.787422 51.660244
Results The results for supervised RE are reported in Table 3. As a classic benchmark task, results on SemEval 2010 Task 8 are already relatively high compared with other supervised RE tasks. The BERT based models outperform the traditional model significantly. By fine-tuning the relation encoder in Semeval training data, we gain 0.6 percent improvement over the original BERT model. Since the SemEval data contains rich semantical information, it is meaningful to visualize the relation representations produced by the trained relation encoders. In this part, we use tSNE (Maaten & Hinton, 2008) method to project the 768-dimensional relation representations of the test set of SemEval data to
2-dimensional points. The visualization is illustrated in Figure 5, where each round marker represents a statement and a star marker represents a prototype. First, the visualization shows that statements
1For each instance, we sample 100 times and report the average accuracy.
are well clustered and the prototypes are effectively learned for all the relations. Another interesting observation is that the visualization shows visible directions and it is identical to our geometric explanation Section 3.1.

heading: 4.5 A TOY EXPERIMENT OF DECISION BOUNDARIES FOR PROTOTYPICAL LEARNING
text: In section 3.1, we mention that the prototype-level classification (equation 6) reduces the disortion of the decision boundary caused by noisy labels. In this section, we carry out a toy experiment to explore the point. We perform a simple binary classification on the iris dataset (Blake, 1998). Here, we use the instance-level linear classifier and the prototype-level classifier. For the prototype-level classifier, the decision function could be written as g(x) = sign ‖(x− z−)‖2−‖(x− z+)‖2, where x is a data point and z− and z+ represent the prototypes for two classes. This equation is also equivalent to a linear classifier g(x) = sign(w>x+ b) with w = z+ − z− and b = ‖z−‖ 2−‖z+‖2 2 . Prototypes z are computed as the classical mean-of-class (Reed, 1972) method, thus the decision boundary could be easily depicted. We focus on the distortion of the decision boudaries, as shown in Figure 6, when the instances are wrongly labeled, the instance-level boundary is easily impacted and overfitted by noisy labels (Figure 6(b)). And the boundary of prototype-level classifier is hardly affected because of the regularization, it is an interesting phenomenon and we believe it worth further theoretical and empirical study. The results also demonstrate the potential of our model for the generalization of other standard/noisy/few-shot classification tasks.

heading: 5 CONCLUSION
text: In this paper, we re-consider the essence of relation representation learning and propose an effective method for relation learning directly from the unstructured text with the perspective of prototypical metrics. We contrastively optimize the metrics between statements and infer a prototype to abstract the core features of a relation class. With our method, the learned relation encoder could produce predictive, interpretable, and robust representations over relations. Extensive experiments are conducted to support our claim, and the method also shows the good potential of generalization.

heading: ACKNOWLEDGMENT
text: This research is supported by National Natural Science Foundation of China (Grant No. 61773229 and 6201101015), Alibaba Innovation Research (AIR) programme, the Shenzhen General Research Project (Grand No. JCYJ20190813165003837, No.JCYJ20190808182805919), and Overseas Cooperation Research Fund of Graduate School at Shenzhen, Tsinghua University (Grant No. HW2018002). Finally, we thank the valuable help of Xu Han and suggestions of all the anonymous reviewers.

heading: A FUZZY RELATION EVALUATION DATASET
text: A.1 DISTANT SUPERVISION
Even though our framework is adaptive for a variety of data types, weakly (distantly) supervised data is the most suitable in terms of quantity and quality. Generally, weakly supervised data is constructed on the basis of a large unstructured textual data (e.g. Wikipedia) and a large structured knowledge base (e.g. Wikidata) via the method of distant supervision. First, all the entities are identified by a named entity recognition system which could tag persons, organizations and locations, etc. Those entities form an entity set E that is mentioned in Section 2. Then if for a sentence I , the entity pair (h, t) also exists in the knowledge base with a relation r, then I will be distantly annotated as r. After the distant supervision, all the relations form a relation setR.
A.2 MOTIVATION OF FUZZYRED
The distant supervision assumption demonstrates that if two entities participate in a relation, then all the sentences that mention these two entities express that relation. While obtaining plenty of weakly (distant) supervised data, this assumption has also become the primary drawback of distant supervision. Generally, one entity pair in the distant dataset may correspond to multiple relations in different textual environments. For example, the entity pair (Steve Jobs, California) has three relations, which are Place of Birth, Place of Work, and Place of Residence. Thus, instances of these corresponding relations prone to introducing false-positive noise. We named such relations as fuzzy relations. Intuitively, conducting extraction on fuzzy relations is more challenging. To empirically evaluate if encoder trained on distant supervised could really learn the correct semantic from the noisy distant supervised corpus and perform well on Fuzzy relations, we present a novel dataset, namely, Fuzzy Relation Extraction Dataset (FuzzyRED). In this section, We will firstly introduce the construction process for FuzzyRED and report some statistics.
A.3 CONSTRUCTION PROCESS OF FUZZYRED
Broadly speaking, the dataset is selected from a distant supervised corpus and judged by humans to determine whether each instance really expresses the same relation as the distant supervision labels. For instance, sentence ”Jobs was born in California” really expresses the supervision label ”Place of Birth” while sentence ”Jobs lived in California” does not. The detailed process is as follows:
1. First, following the distant supervision assumption, we align the relation triples in Wikidata with Wikipedia articles to generate massive relation instances.
2. We count the number of corresponding relations for each entity pair, denote as CountR(pair). For example, the entity pair (Steve Jobs, California) has three relations, which are Place of Birth, Place of Work, and Place of Residence. Larger CountR(pair) implies a higher risk of introducing noise. Thus, we consider a pair whose CountR(pair) larger than two as an ambiguous seed of the relations that it is involved in.
3. Then, for each relation, we count the number of ambiguous seeds, denote as CountS(rel). The bigger CountS(rel) is, the more likely the relation will get the wrong data, which will lead to a worse learning effect of the relation.
4. We sort the relations according to CountS(rel) and select the top 20 relations as fuzzy relations of FuzzyRED. For each relation, 50 instances are randomly selected to annotate. For the specific annotation process, each annotator strictly judges whether the sentence could express the given relation according to the definition of the relation in Wikidata.
A.4 STATISTICS OF FUZZYRED
In FuzzyRED, there are 20 different kinds of relations, and 50 instances for each relation. As mentioned above, every instance is manually classified as true positive (TP) or false positive (FP). Table 4 shows the statistics of FP rates, which are calculated as count(FP)count(TP)+count(FP) . As shown in Table 4, the average FP rate is more than 50 percent, in other words, the majority of the data are FPs. Hence, it is challenging to learn a relation extraction model from FuzzyRED if it doesn’t take the
noise problem into consideration. In this paper, we empirically evaluate different relation encoders by conducting a binary classification task. A better encoder is expected to distinguish TPs from FPs. Specifically, a model based on such relation encoder should predict a higher probability for TPs.

heading: B ALGORITHMS
text: In Section 3.2, we introduce the adaptation of different downstream scenarios for our framework, which trains an encoder Encφ. In this section, we provide the algorithms for the three relation learning scenarios.
Algorithm 1 reports the training of supervied relation learning.
Algorithm 1 Training for Supervised Relation Extraction Input: Supervised dataset Sr = {s1:n}, statement encoder Encφ while not converge do
Sample mini-batches Sbatches from Sr. for Sbatch in Sbatches do
for s in Sbatch do s = Encφ(s) p(y|s, θ) = Softmax(Ws+ b) end for Update W , b and Encφ w.r.t. l(θ) =
∑ s∈Sbatch log p(y|s, θ)
end for end while
Similarly, Algorithm 2 illustrates the training algorithm for few-shot relation learning.
Algorithm 2 Training for Few-shot Relation Extraction Input: Few-shot dataset Sf , statement encoder Encφ repeat R′ = SampleRelation(N,R). S = ∅, Q = ∅ for r inR′ do {sri } = SampleInstance(k, r), i ∈ {1, 2...k} qr = SampleInstanc(1, r) S ⋃ {sri }, Q ⋃ {qr}
end for for qr in Q do
for sr ′
i in S do simr ′ i = Encφ(qr) · Encφ(sr ′
i ) end for r̂ = argmaxr′,r′∈R′(sim r′
i ) Update Encφ w.r.t. CrossEntropy(r, r̂)
end for until enough steps
Algorithm 3 reports the training of fuzzy relation learning. Note that FuzzyRED is designed for evaluation, so the algorithm shows the inference phase.
Algorithm 3 Training for Fuzzy Relation Evaluation Input: FuzzyRED Sz = {s1:n}, statement encoder Encφ while not converge do
for s in Sz do {si} = SampleInstance(k), i ∈ {1, 2, ..., k} Compute a w.r.t a = 1k ∑ p(l = 1|s, si)
if a > T then s is true positive (TP) else s is false positive (FP)
end if end for
end while

Review summaries:
This paper presents a pre-training method for encoders (i.e., BERT) of relation extraction, leveraging distant supervision data. The main idea is to introduce a prototype embedding for each relation in the distantly generated data. The loss function to pre-train the encoders involve several terms, aiming to exploit the intra-class compactness (e.g., the contrastive loss for statement pairs) and the inter-class separability (i.e., constraining the distances between the prototype for one relation and the embeddings of the statements for the same and different relations). The paper incorporates a prototype-level classification loss. Compared to few-shot learning models (e.g., prototypical networks), the paper claims that prototype-based losses can help the model to better address the noisy label issue of distantly generated data due to the exploitation of prototype and instance interactions. The pre-trained encoder is then applied on datasets for few-shot, supervised, and fuzzy relation extraction. The proposed method achieves competitive performance with other methods. Some analysis are also conducted to demonstrate the effectiveness of the proposed model.

The authors propose a novel method for learning prototype representation for relations which abstracts the essential semantics of relations between entities in sentences. The learned prototypes are learned based on an objective with clear geometric interpretation and have been shown to be interpretable and robust to noisy from distantly-supervised data. The method has been shown to be effective for supervised, few-shot, and distantly supervised relation extraction. The prototype embeddings are are unit vectors uniformly dispersed in a unit ball and statement embeddings are centered at the end of their corresponding prototypes. The training is done such that intra-class compactness and inter-class separability is increased. The results show that the proposed approach gives state-of-the-art results for few-shot, supervised relation extraction. The authors demonstrate the interpretability and robustness of the method.