Prototypical Representation Learning for Relation Extraction
Authors: Ning Ding
Xiaobin Wang
Yao Fu
Guangwei Xu
Rui Wang
Pengjun Xie
Ying Shen
Fei Huang
Hai-Tao Zheng
Rui Zhang
Venue: ICLR 2021 Poster (2021)
Keywords: NLP
Relation Extraction
Representation Learning

Abstract: Recognizing relations between entities is a pivotal task of relational learning.  
Learning relation representations from distantly-labeled datasets is difficult because of the abundant label noise and complicated expressions in human language.  
This paper aims to learn predictive, interpretable, and robust relation representations from distantly-labeled data that are effective in different settings, including supervised, distantly supervised, and few-shot learning. 
Instead of solely relying on the supervision from noisy labels, we propose to learn prototypes for each relation from contextual information to best explore the intrinsic semantics of relations. 
Prototypes are representations in the feature space abstracting the essential semantics of relations between entities in sentences.
We learn prototypes based on objectives with clear geometric interpretation, where the prototypes are unit vectors uniformly dispersed in a unit ball, and statement embeddings are centered at the end of their corresponding prototype vectors on the surface of the ball. 
This approach allows us to learn meaningful, interpretable prototypes for the final classification.
Results on several relation learning tasks show that our model significantly outperforms the previous state-of-the-art models.
We further demonstrate the robustness of the encoder and the interpretability of prototypes with extensive experiments.

Published as a conference paper at ICLR 2021
PROTOTYPICAL REPRESENTATION LEARNING FOR RE- LATION EXTRACTION
Ning Ding', Xiaobin Wang"*, Yao Fu², Guangwei Xu², Rui Wang", Pengjun Xie", Ying Shen', Fei Huang', Hai-Tao Zheng'| Rui Zhang
"Tsinghua University, AAlibaba Group, 'The University Edinburgh, *Sun Yat-sen University {dingn18 @mmails.. wiinguua edu. en, {yao. fueeed. ac. xuuanjie. wxb, kunka. xgw, chengchen.xipp f. (zheng.haitase www.:aaittaaae..oo 0sz tsinghua edu. en, frui. chang]@ieee..o
com
ABSTRACT
Recognizing relations between entities is pivotal task of relational learning. Learn- ing relation representations from distantly-labeles datasets is difficult because of the abundant label noise and complicated expressions in human language. This paper aims to learn predictive, interpretable, and robust relation representations from distantly-lbblled data that are effective in different settings, including super- vised, distantly supervised, and few-shot learning. Instead of solely relying the supervision from noisy labels, we propose to learn prototypes for each relation from contextual information to best explore the intrinsic semantics of relations. Pro- totypes are representations in the feature space abstracting the essential semantics of relations between entities in sentences. We learn prototypes based objectives with clear geometric interpretation, where the prototypes are unit vectors uniformly dispersed in a unit ball, and statement embeddings are centered the end of their corresponding prototype vectors on the surface of the ball. This approach allows to learn meaningful, interpretable prototypes for the final classification. Results on several relation learning tasks show that our model significantly outperforms the previous state-of-the-an -o-tte--rttttttte models. We further demonstrate the robustness of the encoder and the interpretability of prototypes with extensive experiments.
INTRODUCTION
Relation extraction aims predict relations between entities in sentences, which is crucial for under- standing the structure of human knowledge and automatically extending knowledge bases (Cohen & Hirsh, 1994; Bordes et al., 2013; Zeng et al., 2015; Schlichtkrull et al., 2018; Shen et al., 2020). Learning representations for relation extraction is challenging due to the rich forms of expressions human language, which usually contains fine-graid gre-ggained, complicated correlations between marked entities. Although many works are proposed to learn representations for relations from well-structured knowledge (Bordes et al., 2013; Lin et al., 2015; be unstructured Distantly-labeled text (Mintz et al., 2009), this task becomes particularly challenging due to spurious correlations and label noise (Riedel al., 2010).
al., 2015), when we extend the learning source to
This paper aims to learn predictive, interpretable, and robust relation representations from large-scale distantly labeled data. We propose prototype learning approach, where we impose a prototype for each relation and learn the representations from the semantics of each statement, rather than solely from the noisy distant labels. Statements are defined as sentences expressing relations between two marked entities. As shown in Figure 1, prototype is embedding in the representation space capturing the most essential semantics of different statements for given relation. These prototypes essentially serve the center of data representation clusters for different relations and surrounded by statements expressing the same relation. We learn the relation and prototype representations based on objective functions with clear geometric interpretations. Specifically, our approach assumes prototypes are unit vectors uniformly dispersed in a unit ball, and statement embeddings are centered
'Equal contribution.
Cooresponding author. 'http://ron zhang, info

---

Published as a conference paper at ICLR 2021
50
s; The subject of imply" the source of an implication while the subject of "infer" the recipient... (Cause-Fffeeet sy Lopez and Espinoza took plea agreements that resulted in diversion -- the completion of rereuirements... (Cause- Fffect)
34
sy The small photoreceptors of the retina (the inner surface (Component- Whole)
back the eye) sense...
s4: The cells inside nested table isolated from changes made to the outer table.
-18
(Component- Whole)
Came-Fiffict
8; That one of the many reasons that CEO's team includes an experienced general counsel.
Compenent
CColletton-Meembey
-4
CinloConspiint
Contmn-Commnnner
sg² As well wildlife, India vast treasure trove palaces, forts, temples, museums and... (Collection-Member
Collection-ernt
Protatypes for the relation
-70
-23
28
Figure I: The I-SNE visualization of relation representations and corresponding prototypes learned by our model. In the right part, $1.6 are examples of input statements, where red and blue represent the head and tail entities, and the italics in parenthesis represents the relation between them.
at the end of their corresponding prototype vectors on the surface of the ball. We propose statement- statement and poototypesstteennn objectives to ensure the intra-class compactness and inter-class separability. Unlike conventional cross-entropy loss that only uses instance-level supervision (which could be noisy), our objectives exploit the interactions among all statements, leading to a predictively powerful encoder with more interpretable and robust representations. We further explore these properties of learned representations with extensive experiments.
We apply our approach using pretraining fine-tuning paradigm. We first pretrain relation encoder with prototypes from a large-scale distantly labeled dataset, then fine-tune it on the target dataset with different relational learning settings. We further propose probing dataset, FuzzyRED dataset (Section 4.3), to verify if our method can capture the underlying semantics of statements. Experiments demonstrate the predictive performance, robustness, and interpretability of our method. For predictive performance, we show that our model outperforms existing state-of-the-as methods on supervised, few-shot, and zero-shot settings (Section 4.4 and 4.2). For robustness, we show how the model generalize to zero-shot setting (Section 4.2) and how the prototypes regularize the decision boundary (Section 4.5). For interpretability, we visualize the learned embeddings and their corresponding prototypes (Section 4.4) and show the cluster clearly follow the geometric structure that the objective functions impose. The source code of the paper will be released at https: :/ //t:://iii /tt///ttttt/ttt// com/ Alibaba NLP/P O...ProtORE.
RELATED WORK
Relation learning could be mainly divided into three categories. The logic-based methods reason relations via symbolic logic rules, with adopting probabilistic graphical models or inductive logic systems to learn and infer logic rules orienting to relations (Cohen & Hirsh, 1994; Wang et al., 2015; Yang et al., 2017; Chen al., 2018; Kazemi & Poole, 2018; Qu & Tang, 2019). The graph-based methods encode entities and relations into low-dimensional continues spaces to capture structure features of KBs. (Nickel et al., 2011: Bordes et al., 2013; Yang et al., 2015; Wang et al., 2014; Nickel et al., 2016; Ji et al., 2015; Lin et al., 2015; Trouillon et al., 2016; Sun et al., 2019; Balažević et al., 2019). The text-based methods have been widely explored recently, focusing on extracting semantic features from text to learn relations.
The conventional methods to learn relations from the text are mainly supervised models, like statistical supervised models (Zelenko et al., 2003; GuoDong et al., 2005; Mooney & Bunescu, 2006). As deep neural networks have gained much attention then, a series of neural supervised models have been proposed (Liu et al., 2013; Zeng et al., 2014; Xu et al., 2015; Santos et al., 2015; Zhang & Wang, 2015; Verga et al., 2016; Verga & McCallum, 2016; Li et al., 2019; Distiawan et al., 2019; Ding et al., 2019). To address the issue of the insufficiency of annotated data, distant supervision has been applied to automatically generate a dataset with heuristic rules (Mintz et al., 2009). Accompanying with auto-labeled data, massive noise will be introduced into models. Accordingly, various denoising methods have been explored to reduce noise effects for distant supervision (Zeng et al., 2015; Lin et al., 2016; Jiang et al., 2016; Han et al., 2018 Sa; Wu al., 2017; Qin et al., 2018a;b Feng et al.,
2

---

Published as a conference paper at ICLR 2021
2018). Further efforts pay attention to learning relations from the text in various specific scenarios, such as zero-few sero-few-shot scenarios (Levy et al., 2017; Han et al., 2018b: Gao et al., 2019; Soares et al., 2019; Ye & Ling, 2019) and open-domain scenarios (Banko et al., 2007; Fader et al., 2011; Mausam et al., 2012; Del Corro & Gemulla, 2013; Angeli et al., 2015; Stanovsky & Dagan, 2016; Mausam, 2016; Cui et al., 2018; 8; Shinyama & Sekine, 2006; Elsahar et al., 2017; Wu al., 2019).
However, up to now, there are few strategies of relation learning that could be adapted to any relation learning tasks. Soares et al. (2019) introduces a preliminary method that learns relation representations from unstructured corpus. And the assumption that two same entities must express the same relation is imprecise, because it does not consider the semantics of contextual information. This paper abstracts the task from the text as metric and prototype learning problem and proposes a interpretable method for relation representation learning. Apart from the differences in methods and task scenarios, row-less universal schema ( (Verga & McCallum, 2016) has similar spirit with our method, where relation type embeddings guide clustering of statements for entity pair representations and embeddings of entity pairs are regarded as aggregation of relations.
Although the intuition that assigns a representative prototype for each class is similar to some previous studies like Prototypical Networks (Snell et al., 2017) (ProtoNet), there exist some essential distinctions between our approach and the ProtoNet. The ProtoNet computes the prototype of each class as the average of the embeddings of all the instance embeddings, that could be regarded as non-linear version of the nearest class mean approach (Mensink et al., 2013). The idea of mfan-of- class prototypes could also be traced to earlier studies in machine learning (Graf et al., 2009) and cognitive modeling and psychology (Reed, 1972; Rosch et al., 1976). In our method, prototypes and relation encoder are collaboratively and dynamically trained by three objectives, which divides the high-dimensional feature space into disjoint manifolds. The ProtoNet performs instance-level classification to update the parameters, which is not robust for noisy labels. Our method carries out novel prototype-level classification to effectively regularize the semantic information. The prototype-level classification reduces the distortion caused by noisy labels of the decision boundary fitted by NNs. Moreover, the ProtoNet is designed for few-zero-shot learning and our methods are more like a semi-supervised pre-training approach that could be applicd to supervised, few-shot, and transfer learning scenarios. Furthermore, the ProtoNet does not contain the metric learning among instances, and our method simultaneously optimizes the prototype-statemen and statement-statemen metrics. We also make geometry explanation of our method (Section 3.1).
METHOD
Our method follows a cerraininn-finetuniin paradigm. We first pretrain an encoder with distantly labeled data, then fine-tune it to downstream learning settings. We start with the pretraining phase. Given a large-scale, distantly labeled dataset containing training pairs (w, r) where w = [w] wo] is a relation statement, i.e.. sequence of words with pair of entities [h, f] marked, / being the head and being the tail, 36, 12, h = Wi, discrete label and is probably noisy. We aim to learn encoder (-c parameterized by that encodes w into representation E I m being the dimension:
3j1. : f2, t = Wiiija r is the relation between h and t. r is a
8 s= -Enc(w))
A prototype z for relation r is an embedding in the same metric space with that abstracts the essential semanties of r. We use [(2|, ,'),... (zk r rr)]], ]], 2* E T. to denote the set of prototype-relation pairs. K is the number of different relations. =h is the prototype for relation rd and superscripts denote the
index of relation type. Given a batch B = (I), (NN,TN)] N is the batch size and subscripts denote batch index, the similarity metric between two statement embeddings d(si, 8j), and between statement embedding and a prototype d(s; 2) are defined as:
d(si, 85) 1/(1 + exp(
8,
|
$1 ))
(66,))11/1) = exp(
|>>|]|
|s||
Geometrically, this metric is based on the angles of the normalized embeddings restricted in unit ball. We will explain the geometric implication of this metric in the next section. Inspired by Soares et al. (2019), we define contrastive objective function between statements:
exp(6(0s, 83 (d(d) 8,))
Lsss = N Σ Σ En exp((1 - 5(5,, Syy sy)) )dd()) 8y))
(3)
WJ
3

---

Published as a conference paper at ICLR 2021
where d(s;; 85) denotes if 8; and sy corresponds to the same relations, i..., given (s,, rz), r3), 6(2 8y) = 1 if r; = rjj else O. The computation of the numerator term forces that statements with same relations to be close, and the denominator forces that those with different relations dispersed. When constructing the batch B, we equally sample all relation types to make sure the summation in the denominator contains all relations. Essentially, equation ensures intra-class compactness and inter-class separability in the representation space. Additionally, equation 3 contrast one positive sample to all the negative samples in the batch, which puts more weights on the negative pairs. We find it empirically more effective than previous objectives like Soares et al. (2019).
3.1 LEARNING PROTOTYPES
Now we discuss the objective functions for learning prototypes. Denote S = [s1,..., 8N] as the set of all embeddings in the batch B, given fixed prototype =" for relation r. we denote S the subset of all statements 9; in S with relation r, and S the set of the rest statements. Z as the set of prototypes z' for all other relations except r. We impose two key inductive biases between prototypes and statements: (a) for a specific relation r, the "distance" between z" and any statements with the same relation r should be less than the "distance" between x" and any statements with relations r' * r. (b) the "distance" between z" and any statements with relation should be less than the "distance" between any prototypes z' E 2-r and statements with relation r. To realize these two properties, we define two objective functions:
Ls27.
Σ
[log ss)) + log(1 d(z", 8;)]]
(4)
N2
&EST ,8EES-
Lsz
Σ
log d(z", s;) + log(1 d(2', s:)]]
(5)
N
&,E8" 1''2""'
where equation 4 corresponds to (a) and equation 5 corresponds to (b). These objectives effectively splits the data representations into K disjoins manifolds centering at different prototypes. We further highlight the differences between equation 4 and 5 and conventional cross-entropy loss: there is no interactions between different statements in the cross-entropy loss that solely relies on the instance level supervision, which is particularly noisy under a noisy-label setting. On the other hand, our loss functions consider distances between different statements and prototypes, which exploits the interactions between statements. This type of interaction would effectively serve as a regularization to the decision boundary. as we will empirically verify in section 4.5.
Combining equations 3, and 5, we further give geometric explanation of the representation space: prototype is a unit vector (because we normalize the vector length in equation 2) starting from the origin and ending at the surface of unit ball, and statements for that prototypes are unit vectors with approx- imately same directions centering at the prototype (because our objective functions 3, 4, push them to be close to each other). Under the optimal condition, different prototype vectors would be uniformly dispersed with the angles between them as large as possible (because the distance metric is based on the angle in equation 2 where the minimum is at n, and we maximize dis- tances between prototypes in equation and 5). Consequently, the dataset is clustered with each cluster centered around the end of one prototype vector on the surface of the unit ball. The illustration of the intuition is shown in Figure 2.
5
Figure 2: An illustration of the geometric explanation. Stars rep- resent prototypes and circles rep- resent statements.
To further regularize the semantics of the prototypes, we use a prototype-level classification objective:
Σ Logpy(r*1>>" Las = K
1
(6)
where denotes the parameters of an auxiliary classifier. Our prototype essentially serves as regularization averaging the influence of label noises. We further validate this regularization in Section 4.5 by showing that it reduces the distortion of the decision boundary caused by noisy labels. Finally, with hyper parameters X2, Xg and As, the full loss is:
L Cses + (2( Lsuz) + AsLCLs-
4

---

Published as a conference paper at ICLR 2021
3.2 FINE-TUNING ON DIFFERENT LEARNING SETTINGS
We apply our learned encoder and prototypes to two typical relational learning settings: (a) supervised; (b) fhotshot. Under the supervised learning setting, having the pretrained encoder hand, we fine- tune the model on dataset of a target domain (which could again, be distantly labeled) containing new training pairs (ur, r), we encode wo to the representation and feed to a feed-forward classifier:
p(rlw) = FF(Enc,(w))
(8)
where FF(.) denotes a feed-forward layer, and train it with the conventional cross-entropy loss.
Under the few-shot learning setting, we assume a slightly different input data format. Specifically, the training set consists of the relations fr',...,r' "]] and a list of supporting statements: .f.... sL for all ph Denote S* = {s/ ) all supporting statements for all relations. Given a query statement q. the task is to classify the relation r" that 9 expresses. During traing, we use the average distance to different prototypes as the logits of a feed-forward classifier and train the model with cross-entropy loss. During testing, we classify q according to the minimum similarity metrics:
st- arg min d(s) Encs(g)), r" r==
EES
Note this is equivalent to choosing the argmax of the logits because the logits are formed by the similarity scores. We note that model pre-trained by our approach performs surprisingly well even with no training data for fine-tuning, which is reported in Figure 3 and 4 in Section 4.2.
4 EXPERIMENTS
In order to evaluate the performance of prototypical metrics learning, we conduct extensive experi- ments on three tasks: supervised relation learning, few-shot learning and our proposed fuzzy relation learning evaluation. We make comprchensive evaluation and analysis of our method, as well as full-scale comparisons between our work with previous state-of-the-ar methods.
4.1 EXPRIMENTAL DETAILS
Pretraining dataset We prepare the weakly-supervised data by aligning relation tuples from the Wikidata database to Wikipedia articles. Specifically, all entities in wikipedia sentences are identified by a named entity recogmition system. For an entity pair (h, t) in sentence w, if also exists in the knowledge base with relation then will be distantly annotated as r. After processing, we collect more than 0.86 million relation statements covering over 700 relations.
Baseline Models We primarily compare our model with MTB because it is previous SOTA model. For fair comparison, we re-implement MTB with BERTbas (Devlin et al., 2018) and pretrain on our collected distant data same our methods. We note under our setting, the reported number of MTB is smaller than its original paper because: (a) the original paper uses much larger dataset than ours (600 million vs 0.86 million); (b) they use BERT large encoder. In addition. multiple previous state-of-the-ar approaches are picked up for comparison in all the three relation learning tasks (detailed later). For baseline models (PCNN, Meta Network, GNN, Prototypical Network, MLMAN) with publicly available source code, we run the source code manually, and if the produced results are close those in the original paper, we would select and report the results the original paper. For ones without open source code (BERTEN EM. MTB), we re-implement the methods with BERT baseeb pre-trained on our distant data and report the results. We also implement baseline that directly trains BERT base on our distant data with cross-entropy loss (LCE), namely BERT cg in the section (the encoder is pre-trained by directly predicting the distant label for each statement).
The IND baseline We additionally design a baseline, where the prototypes are pre-computed by vectorizing the extracted relation patterns independently (IND). In this way, for each statement, we pre-compute fixed metric as its weight demonstrating the similarity with the corresponding prototype. The IND baseline primarily for validating the effectiveness of trained prototypes over the rule- prototypes. Given the distantly-labeled training set, we use the Snowball algorithm from Agichtein & Gravano (2000) to extract relation patterns. This algorithm takes a corpus of relational instances as inputs and outputs a list of patterns and their embeddings for each relation. After getting these patterns, we use patterns to match instances and calculate how many instances can each pattern match. For each relation label, we select the top-k patterns that match the most instances
5

---

Published as a conference paper at ICLR 2021
and use the average of their embeddings the prototype embedding. Note that these embeddings not comparable to the sentence embeddings generated by our encoders, so the loss functions in our equation 4 and 5 not usable for the prototypes generated here. To incorporate the information of the extracted prototype embeddings, we slightly modify equation 3 and use the similarity of instances to the embeddings to weight the instance embeddings. The modified loss function is:
exp(6(st, sy)( $,))
LNN = N
Σ 3
(10)
exp((1 6(3;, 88 dd((,, ((y))
= sim(w,, ((:)) 8/,
b = (mj, z' (w²)) 83. , = sim ( 2 (w()) 8/j)
(11)
where W, denote the corresponding sentence to 8 2' (w;) denote the correspond prototype.
Implementation Details For encoding relation statements into a feature space, we adopt deep Transformers (Vaswani et al., 2017), specifically BERT, as F.g. We take the input and the output of the BERT encoder as follows: given a statement 8, we add special entity markers to mark M₁ and M, before the entities, then input the statement and special markers to Encş to compute final representations. Note that our framework is designed independently to the encoder choice, and other neural architectures like CNN (Zeng et al., 2014) and RNN (Zhang & Wang. 2015) can be easily adopted as the encoder. We use Py Torch (Paszke et al., 2019) framework to implement our model. All the experiments run on NVIDIA Tesla V100 GPUs The encoder is optimized with the combination of the prototype-statement and statement-statemenn metrics as well as the masked language model loss with the settings as follows. For the sake of saving resources, we choose BERT base the backbone instead of BERT although our method sacrifices considerable part of the performance. We select AdamW (Loshchilov & Hutter, 2018) with the learning rate of Le- - for optimization. Meanwhile, Warmup (Popel & Bojar, 2018) mechanism is used during training. The number of layers is 12, and the number of heads is 12. The hidden size is set to 768 and the batch size is set to 60. We train the model for 5 epochs in each experiment.
4.2 TEW-SHOT RELATION LEARNING
Dataset We use a large few-shot relation learning dataset FewRel (Han et al., 2018b) in this task. FewRel consists of 70000 sentences (about 25 tokens in each sentence) of 100 relations (700 sentences for each relation), on the basis of Wikipedia. We utilize the official evaluation setting which splits the 100 relations into 64, 16, 20 relations for training, validation, and testing respectively.
Experimental settings We use accuracy as the evaluation metric in this task. The batch size for few-shot training is 4, the training step is 20000 and the learning rate is 2e-5 with AdamW. We follow the original meaning of N-way-K-shot in this paper, please refer to (Han et al., 2018b) for details.
Results and analysis The results for few-shot relation learning are illustrated in Table 1. It is worth noting that we report the average score rather than the best score of each method for the sake of faimess. One observation is that the task is challenging without the effectiveness of pre-trained language models. Prototypical network (CNN) only yields 69.20 and 56.44 accuracies (%) for way shot and 10 way shot respectively. Methods that have achieved significant performances on supervised RE (such as PCNN, GNN, and Prototypical Network) suffer grave declines. MTB learns the metrics between statements based on large pre-trained language models and yield surprising results on this task. Better performance gains by considering the semantic information by applying exed-prrtotypes to MTB (IND strategy). The collaborative prototype-based methods are proven be effective as shown in the last four rows. The results of COL (Cszs' + Lzns + Lz2s') ) indicate that the loss of MTB have little influence for the performance. Finally, our final collaborative model achieve the best performance on all the N-way K-shot settings. Our method also outperforms human in 5 way shot setting (92.41 vs 92.22) and 10 way 1 shot setting (86.39 vs 85.88).
Effect of the amount of training data In real-world scenarios, it is costly and labor-consuming to construct a dataset like FewRel containing 70,000 positive instances. Some relations rarely appear in public articles and obtain candidate instances via distance supervision introduces much noise. Therefore, the amount of instances to annotate may much larger than 70, 000. To investigate the ability of fhotshot classification when lack of high-quality task-specific training data, we intentionally control the amount of instances in training data and run evaluations with the new training set. Figure and Figure 4 respectively display the accuracy of the 5-way- 1-shot task given limited number

---

Published as a conference paper at ICLR 2021
Table I: Few-shot classification accuracies (%) FewRel dataset. The last block is the results of the series of our models with prototypes. We use Lz to briefly indicate Zzes + Lzes Results with are reported as published, and other methods are implemented and evaluated by us. denotes outperformance over the main baseline MTB and denotes underperformance.
Method
5 way shot
5 way 5 shot
10 shot
10 way 5 shol
Finetune (PCNN)1 (Han et al., 2018b)
45.64
57.86
29.65
37.43
Meta Network (CNN)) (Han et al., 2018b)
64.46
80.57
53.96
69.23
GNN (CNN)) (Han al., 2018b)
66.23
81.28
46.27
64,02
Prototypical Networkf (Han et al., 2018b)
69.20
84.79
56.44
75.55
MLMAN (Ye & Ling, 2019)
82.98
92.66
73.59
87.29
BERTEM (Soares ct al., 2019)
88.70
95.01
81.93
90.05
MTB (Cs25:) (Soares et al., 2019)
89.09
95.32
82.17
91.73
BERTcr(CCr)
91.02 (1)
95.40 (1)
84.95 (1)
91.43
IND (LIND)
89.90 (1)
95.42
82.47 (
91.55
COL (C2)
90.40 (1)
94.73 (L)
84.27 (
91.58
COL (C2 + (CLS)
91.12 (1)
95.45 (1)
85.10 (
91.75
COL (Le28) + Lz Lcus)
91.08 (1)
95.52 (1)
85.83 (1)
92.18
COL Final (Lazs Lz. + Ccus)
92.51 (1)
95.88
86.39 (?)
92.76
95
95
90
90
Accuracy
Accuracy
85
A
= 96.96
85
A BERT
A BERT
75
D MTB
75
0 MTB
Ours
Ours
70
70
10
40
700
Number of relation types in training
Number of instances per type in training
Figure 3: Impact of # of relation types
Figure 4: Impact of instances per type
of relation types or amount instances of each relation in training data. As shown in the figures, pre-trained relation encoders outperform the basic BERTEM model when short of training data. Encoder learned with our proposed method shows large superiority over the others, when there is no training data (0), the absolute improvement of our framework is 17.96. Besides, the accuracy increase with the increment of training data regardless of providing more relations or more instances of each relation. However, the diversity of relations is more important, since the performance in Figure outperforms those in Figure 3 while the former is trained on a much smaller training data. Lack of diversity even hurts the performance when applying relation encoder which learned with prototype-statement metric. Similar conclusions have been mentioned in Soares et al. (2019).
4.3 FUZZY RELATION EVALUATION
Dataset Aimed evaluating relation representation learning on the noisy weakly supervised corpus, we propose new evaluation dataset named FuzzyRED to quantitatively evaluate the capture of relational information. FuzzyRED constructed aimed the shortcoming of distant supervision, that is not all the statements contain the same entity pairs express the same relation. All the instances are from the weakly-supervised data and manually annotated binary labels to judge the sentence semantically expresses the relation. For example, for "Jack born in London, UK", even if the relation Capital correct between the entities in knowledge base, the sentence does not express the capital relationship and will be labeled as False in FuzzyRED. The dataset contains 1000 manually annotated instances covering 20 relations. For detailed information please refer to Appendix A.
Experimental settings The pre- treitrained relation encoders directly utilized FuzzyRED and expected to separate the false positive instances from true positives. For each relation, we calculate the minimum of statement-statement distances among all true positives the threshold 7', Then for
7

---

Published as a conference paper at ICLR 2021
each instance to be classified, sample K (k-shot) true positive instances s' {si, $2..... SK and compare the average score of a = p(( = 1|s, with T. If > T, the instance is classified positive. FuzzyRED is considerably challenging since it is designed for the issue of distant supervision and directly reflects the gap between semantic relations and annotation information.
Results The accuracies of FuzzyRED are reported in Table 2. The accuracy of labels annotated accord- ing the DS assumption is denoted by DS, which
Table 2: Accuracies (%) on FuzzyRED.
Method
1-shot
3-shot
5-shot
indicates that the majority of the dataset are noise. From the experimental results, we could firstly ob- serve that all the relation encoders suffer from the noise issue and get lower results. The impact of noise increases with the k increasing. But Table 2 still shows that pre-trained relation encoders do de- tect few noise data thus they all yield better performance than DS. Taking consideration of noise by prototypes leads to better performances, and the fixed prototypes play a small role to detect the com- plex semantics that express relations. Even though the COL model achieves remarkable improvement, the evaluation task is still very challenging because the fuzziness relation semantics.
DS
46.3
46.3
46.3
MTB
51.3
50.7
50.7
IND
51.7
50.8
50.7
COL Final
53.2
52.1
52.1
4.4 SUPERVISED RELATION LEARNING
Dataset We use classic benchmark dataset SemEval 2010 Task 8 (Hendrickx et al., 2010) as the dataset for supervised relation learning. SemEval 2010 Task 8 contains nine different pre-defined bidirectional relations and a None relation, thus the task is a 19-way classification. For model selection, randomly sample 1500 instances from the official training data as the validation set.
Experimental settings For evaluation, we use standard precision, recall, and F- measure for supervised RE task. Since the annotation schema of SemEval dataset dif- fers from the data for the encoder training, we fine-tune the relation encoder in SemEval training data. The batch size set to 20, the training epoch is 30, the learning rate is le-5 with Adam (Kingma & Ba, 2014). We note that the numbers for MTB are different from their paper because their original setting uses BERT larg with 600 million instances and is trained on TPUs. Given the restricted resources at hand, to make fair comparison, re-implemented their model with BERT base and pretrain it in our setting (0.86 million instances).
Table 3: Accuracies (%) on SemEval 2010 Task 8.
Method
P
R
F
Bi-RNN (Zhang & Wang. 2015)
79.6
Self-attention (Bilan & Roth, 2018)
84.8
BERT (Soures et al., 2019)
86.0
89.1
87.4
MTB (Soares et al., 2019)
86.5
89.0
87.7
IND COL Final
86.5
89.7
88.0
87.9
88.2
88.0
Results The results for supervised RE are reported in Table 3. As classic benchmark task, results SvlEval 2010 Task 8 are al- ready relatively high compared with other supervised RE tasks. The BERT based models outperform the traditional model significantly. By fine-tuning the relation encoder in Semeval training data, we gain 0.6 percent improvement over the origi- nal BERT model. Since the SemEval data contains rich semantical information, it is meaningful to visualize the relation rep- resentations produced by the trained re- lation encoders. In this part, we t- SNE (Maaten & Hinton, 2008) method to project the 768-dimensional relation repre- sentations of the test set of SemEval data to 2-dimensional points. The visualization is illustrated in Figure 5, where each round marker represents a statement and star marker represents a prototype. First, the visualization shows that statements
Conaan Croand
35
Sers-Orp Oripe
Ohips
legs
Pedal Prokerr
-70
-100
-65
-32
36
70
Figure 5: I-SNE visualization of the relation repre- sentations and prototypes of SemEval data.
For each instance, we sample 100 times and report the average accuracy.
8

---

Published as a conference paper at ICLR 2021
well clustered and the prototypes effectively learned for all the relations. Another interesting observation is that the visualization shows visible directions and it is identical to our geometric explanation Section 3.1.
4.5 TOY EXPERIMENT OF DECISION BOUNDARIES FOR PROTOTYPICAL LEARNING
In section 3.1, we mention that the prototype-level classification (equation 6) reduces the disortion of the decision boundary caused by noisy labels. In this section, we carry out a toy experiment to explore the point. We perform simple binary classification on the iris dataset (Blake, 1998). Here, the instance-level linear classifier and the prototype-level classifier. For the prototype-level classifier, the decision function could be written as g(z) = sign (rr 21 - ( 2+) )ll where is data point and 2_ and 2+ represent the prototypes for two classes. This equation is also equivalent to a and b || _1 1| 1", Prototypes are (Reed, 1972) method, thus the decision boundary could be easily depicted. We focus the distortion of the decision boudaries, shown in Figure 6, when the instances are wrongly labeled, the instance-level boundary is easily impacted and overfitted by noisy labels (Figure 6(b)). And the boundary of prototype-level classifier is hardly affected because of the regularization, it is an interesting phenomenon and we believe it worth further theoretical and empirical study. The results also demonstrate the potential of our model for the generalization of other standard/noisyffew-hho classification tasks.
linear classifier g(z) sign(w x + b) with w 2+ computed as the classical
Instance-level Classification (Clean)
Instance-level Classification (Noisy)
vercechr
Sapai width
Sepal width
Sepal length
Sepal length
(a) Decision boundary on clean labels (ins-level).
(b) Decision boundary on noisy labels (ins-level).
Prototype-level classification(Clean)
Prototype- classification(Noiss))
setisa
veraa venakoner sebass protonype
Sepal width
ver slover protituse
Sepai width
vericaar-gre precccbrpreetppeee
11
Sepal length
Sepal length
(c) Decision boundary on clean labels (proto-level). (d) Decision boundary on noisy labels (proto-level).
Figure 6: A set of toy experiments on iris dataset to illustrate the distortion of decision boundaries for instance-level and prototype-level learning.
CONCLUSION
In this paper, we re-consider the essence of relation representation learning and propose an effective method for relation learning directly from the unstructured text with the perspective of prototypical metrics. We contrastively optimize the metrics between statements and infer prototype abstract the core features of a relation class. With our method, the learned relation encoder could produce pre- dictive, interpretable, and robust representations over relations. Extensive experiments are conducted to support our claim, and the method also shows the good potential of generalization.
ACKNOWLEDGMENT
This research is supported by National Natural Science Foundation of China (Grant No. 61773229 and 6201 620110,,,), Alibaba Innovation Research (AIR) programme, the Shenzhen General Research Project (Grand No. (CYJ20190813165000:
No.JCYJ2019080000 11000011880999 and Overseas Cooper- ation Research Fund of Graduate School at Shenzhen, Tsinghua University (Grant No. HW2018002). Finally, we thank the valuable help of Xu Han and suggestions of all the anonymous reviewers.

---

Published as a conference paper at ICLR 2021
REFERENCES
Eugene Agichtein and Luis Gravano. Snowball: Extracting relations from large plain-text collections. In Proceedings of the ACM DL. pp. 85-94, 2000.
Gabor Angeli, Melvin Jose Johnson Premkumar, and Christopher D. Manning. Leveraging linguistic structure for open domain information extraction. In Proceedings of ACL-IJCNLP,, pp. 344 354, 2015.
Ivana Balaževic, Carl Allen, and Timothy M Hospedales. Hypernetwork knowledge graph embed- dings. In Proceedings of NIPS, 553-565, 2019.
Michele Banko, Michael J Cafarella, Stephen Soderland. Matthew Broadhead, and Oren Etzioni. Open information extraction from the web. In Proceedings of IJCAI, volume 7, pp. 2670-2676, 2007.
Ivan Bilan and Benjamin Roth. Position-award self-attention with relative positional encodings for slot filling. In Proceedings of CoRR, 2018.
Catherine Blake. Uci repository of machine learning databases. http://ww:. ics. uci. edu/"
mleann/LLeepostorry html, 1998.
Antoine Bordes, Nicolas Usunier, Alberto Garcia-Duran, Jason Weston, and Oksana Yakhnenko. Translating embeddings for modeling multi-relational data. In Proceedings of NIPS, pp. 2787-2795, 2013.
Wenhu Chen, Wenhan Xiong, Xifeng Yan, and William Yang Wang. Variational knowledge graph reasoning. Proceedings of NAACL-HLT, pp. 1823-1832, 2018.
William W Cohen and Haym Hirsh. Learning the classic description logic: Theoretical and experi- mental results. In Principles of KR, pp. 121-133, 1994.
Lei Cui, Furu Wei, and Ming Zhou. Neural open information extraction. In Proceedings of ACL, pp. 407-413, 2018.
Luciano Del Corro and Rainer Gemulla. Clausie: clause-based open information extraction. In Proceedings of WWW, pp. 355-366, 2013.
Jacob Devlin, Ming-Was Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. Proceedings of NAACI, pp. 4171-4186, 2018.
Ning Ding, Ziran Li, Zhiyuan Liu, Haitao Zheng, and Zibo Lin. Event detection with trigger-aware lattice neural network. In Proceedings of EMNLP, pp- 347-356, 2019.
Bayu Distiawan, Gerhard Weikum, Jianzhong Qi, and Rui Zhang. Neural relation extraction for knowledge base enrichment. In Proceedings of ACI, pp. 229-240, 2019.
Hady Elsahar, Elena Demidova. Simon Gottschalk, Christophe Gravier, and Frederique Laforest. Unsupervised open relation extraction. In Proceedings of ESWC, pp. 12-16, 2017.
Anthony Fader, Stephen Soderland, and Oren Etzioni. Identifying relations for open information extraction. In Proceedings of EMNLP, pp. 1535-1545, 2011.
Jun Feng, Minlie Huang, Li Zhao, Yang Yang, and Xiaoyan Zhu. Reinforcement learning for relation classification from noisy data. In Proceedings of AAAI, pp- 5779-5786, 2018.
Tianyu Gao, Xu Han, Hao Zhu, Zhiyuan Liu, Peng Li. Maosong Sun, and Jie Zhou. Fewrel 2.0: Towards more challenging few-shot relation classification.. In Proceedings of EMNLP, pp. 6249-6254, 2019.
Amulf BA Graf, Olivier Bousquet, Gunnar Rätsch, and Bernhard Schölkopf. Prototype classification: Insights from machine learning. Neural computation, 21(1):272-300, 2009.
10

---

Published as a conference paper at ICLR 2021
Zhou GuoDong, Su Jian, Zhang Jie, and Zhang Min. Exploring various knowledge in relation extraction. In Proceedings of ACL, pp. 427-42444444 2005.
Xu Han, Zhiyuan Liu, and Maosong Sun. Neural knowledge acquisition via mutual attention between knowledge graph and text. In Proceedings of AAAI, pp. 4832-4839, 2018a.
Xu Han, Hao Zhu, Pengfei Yu, Ziyun Wang, Yuan Yao, Zhiyuan Liu, and Maosong Sun. Fewrel: A large-scale supervised few-shot relation classification dataset with state-of-the-art evaluation. In Proceedings of EMNLP, pp. 4803-4809, 2018b.
Iris Hendrickx, Su Nam Kim, Zornitsa Kozareva, Preslav Nakov, Diarmuid Ó Séaghdha, Sebastian Padó, Marco Pennacchiotti, Lorenza Romano, and Stan Szpakowicz. SemEval-2010 task 8: Multi- way classification of semantic relations between pairs of nominals. In Proceedings of SemEval, pp. 33-38, 2010.
Guoliang Ji, Shizhu He, Liheng Xu, Kang Liu, and Jun Zhao. Knowledge graph embedding via dynamic mapping matrix. In Pruceedings of ACL, pp. 687-696, 2015.
Xiaotian Jiang, Quan Wang, Peng Li, and Bin Wang. Relation extraction with nulti-instance multi-label convolutional neural networks. In Proceedings of COLING, pp. 1471-1480, 2016.
Scyed Mchran Kazemi and David Poole. Relnn: A deep neural model for relational learning. In
Proceedings of AAAI, pp. 6367-6375, 2018.
Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint, 2014.
Omer Levy, Minjoon Seo, Eunsol Choi, and Luke Zettlemoyer. Zero-shot relation extraction via reading comprehension. In Proceedings of CoNLL, pp. 333-342, 2017.
Ziran Li, Ning Ding, Zhiyuan Liu, Haitao Zheng, and Ying Shen. Chinese relation extraction with multi-grained information and external linguistic knowledge. In Proceedings of ACL, pp. 4377 4377-4386, 2019.
Yankai Lin, Zhiyuan Liu, Maosong Sun, Yang Liu, and Xuan Zhu. Learning entity and relation embeddings for knowledge graph completion. In Proceedings of AAAI, pp. 2181-2187, 2015.
Yankai Lin, Shiqi Shen, Zhiyuan Liu, Huanbo Luan, and Maosong Sun. Neural relation extraction with selective attention over instances. In Proceedings of ACL, volume I, pp. 2124-2133, 2016.
Chun Yang Liu, WenBo Sun, WenHan Chao, and Wanxiang Che. Convolution neural network for relation extraction. Proceedings of ADMA, pp. 231-242, 2013.
I Loshchilov and F Hutter. Fixing weight decay regularization in adam. In Proceedings of ICLR. 2018.
Laurens van der Maaten and Geoffrey Hinton. Visualizing data using t-sne. Journal of machine learning research, 9(Nov):2579-20 2605, 2008.
Mausam, Michael Schmitz, Stephen Soderland, Robert Bart, and Oren Etzioni. Open language learning for information extraction. In Proceedings of EMNLP- EMNLP-CONLL, pp. 533-534, 2012.
Mausam Mausam. Open information extraction systems and downstream application. In Proceedings of IJCAI, pp. 4074-4077, 2016.
Thomas Mensink, Jakob Verbeek, Florent Perronnin, and Gabriela Csurka. Distance-based image classes at near-zero cost. IEEE transactions pattern analysis
classification: Generalizing and machine intelligence, 35(11):2624-28 2013.
Mike Mintz, Steven Bills, Rion Snow, and Dan Jurafsky. Distant supervision for relation extraction without labeled data. In Proceedings of ACI. pp. 1003-1011, 2009.
Raymond J Mooney and Razvan C Bunescu. Subsequence kernels for relation extraction. In Proceedings of NIPS, pp. 171-178, 2006.
11

---

Published as a conference paper at ICLR 2021
Maximilian Nickel, Volker Tresp. and Hans-Peter Kriegel. A three- way model for collective learning on multi-relational data. In Proceedings of ICML, pp. 809 816, 2011.
Maximilian Nickel, Lorenzo Rosasco, and Tomaso Poggio. Holographic embeddings of knowledge graphs. In Proceedings of AAAI, pp. 1955-1961, 2016.
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An imperative style, pigh-peeformance deep learning library. In Proceedings of NeuRIPS, pp. 8024-8035, 2019.
Martin Popel and Ondřej Bojar. Training tips for the transformer model. The Prague Bulletin of Mathematical Linguistics, 110(1):3377,, 2018.
Pengda Qin, Weiran Xu, and William Yang Wang. Dsgan: generative adversarial training for distant supervision relation extraction. In Proceedings of ACI, pp. 496-505, 20.
Pengda Qin, Weiran Xu, and William Yang Wang. Robust distant supervision relation extraction via deep reinforcement learning. In Proceedings of ACL, pp. 2137-2147, 2018b.
Meng Qu and Jian Tang. Probabilistic logic neural networks for reasoning. In Proceedings of NIPS, pp. 7710-7720, 2019.
Stephen K Reed. Pattern recognition and categorization. Cognitive psychology, 3(3):382-407, 1972.
Sebastian Riedel, Limin Yao, and Andrew McCallum. Modeling relations and their mentions without labeled text. In Joint European Conference on Machine Learning and Knowledge Discovery in Databuses, pp. 148-163. Springer, 2010.
Eleanor Rosch, Carolyn B Mervis, Wayne D Gray, David M Johnson, and Penny Boyes-Braens Basic objects in natural categories. Cognitive psychology, 8(3):382 439, 1976.
Cicero Nogueira dos Santos, Bing Xiang, and Bowen Zhou. Classifying relations by ranking with convolutional neural networks. In Proceedings of ACL-IJCNLP, pp. 626-633 2015.
Michael Schlichtkrull, Thomas N Kipf, Peter Bloem, Rianne van den Berg, Ivan Titov, and Max Welling. Modeling relational data with graph convolutional networks. In Proceedings of ESWC, pp. 593-6077 2018.
Ying Shen, Ning Ding. Hai-Tao Zheng. Yaliang Li, and Min Yang. Modeling relation paths for knowledge graph completion. IEEE Transactions on Knowledge and Data Engineering, 2020.
Yusuke Shinyama and Satoshi Sekine. Preemptive information extraction using unrestricted relation discovery. In Proceedings of NAACL-IILT, pp. 304-311, 2006.
Jake Snell, Kevin Swersky, and Richard Zemel. Prototypical networks for few-shot learning. In Proceedings of NIPS, pp. 4077-4087, 2017.
Livio Baldini Soares, Nicholas FitzGerald, Jeffrey Ling, and Tom Kwiatkowski. Matching the blanks: Distributional similarity for relation learning. Proceedings of ACL, pp. 2895-2905, 2019.
Gabriel Stanovsky and Ido Dagan. Creating a large benchmark for open information extraction. In Proceedings of EMNLP, pp. 2300-2305, 2016.
Zhiqing Sun, Zhi-Hong Deng, Jian- Yun Nie, and Jian Tang. Rotate: Knowledge graph embedding by relational rotation complex space. In Proceedings of ICLR, 2019.
Théo Trouillon, Johannes Welbl, Sebastian Riedel, Éric Gaussier, and Guillaume Bouchard. Complex embeddings for simple link prediction. In Proceedings of ICML, pp. 2071-2080, 2016.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones. Aidan Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Proceedings of NIPS, pp. 5998 5998-6008, 2017.
Patrick Verga and Andrew McCallum. Row-less universal schema. Proceedings of ACL, pp. 63-68, 2016.
12

---

Published as a conference paper at ICLR 2021
Patrick Verga, David Belanger. Emma Strubell, Benjamin Roth, and Andrew McCallum. Multilingual relation extraction using compositional universal schema. In Proceedings of NAACL, pp. 886-896, 2016.
William Yang Wang, Kathryn Mazaitis, Ni Lao, and William W Cohen. Efficient inference and learning in a large knowledge base. Machine Learning, 100:101-126, 2015.
Zhen Wang, Jianwen Zhang, Jianlin Feng, and Zheng Chen. Knowledge graph embedding by translating on hyperplanes. In Proceedings of AAAI, pp. 1112-1119, 2014.
Ruidong Wu, Yuan Yao, Xu Han, Ruobing Xie, Zhiyuan Liu, Fen Lin, Leyu Lin, and Maosong Sun. Open relation extraction: Relational knowledge transfer from supervised data unsupervised data. In Proceedings of EMNLP-IJCNLP, pp. 219-228, 2019.
Yi Wu, David Bamman, and Stuart Russell. Adversarial training for relation extraction. In Proceedings of EMNLP, pp. 1778-1783, 2017.
Yan Xu, Lili Mou, Ge L00hang201 Srelation, Yunchuan Chen, Hao Peng, and Zhi Jin. Classifying relations via long short term memory networks along shortest dependency paths. In Proceedings of EMNLP, pp. 1794-1794, 2015.
Bishan Yang, Wen-tau Yih, Xiaodong He, Jianfeng Gao, and Li Deng. Embedding entities and relations for learning and inference in knowledge bases. In Proceedings of ICLR, 2015.
Fan Yang. Zhilin Yang, and William W Cohen. Differentiable learning of logical rules for knowledge base reasoning. In Proceedings of NIPS, pp. 2319-2328, 2017.
Zhi-Xiu Ye and Zhen-Hua Ling. Multi-level matching and aggregation network for few-shot relation classification. In Proceedings of ACL, pp. 2872-2881, 2019.
Dmitry Zelenko, Chinatsu Aone, and Anthony Richardella. Kernel methods for relation extraction. Proceedings of JMLR, 3(Feb):1083-111 2003.
Daojian Zeng, Kang Liu, Siwei Lai, Guangyou Zhou, Jun Zhao, al. Relation classification via convolutional deep neural network. In Proceedings of COLING, pp. 2335-2344 2014.
Daojian Zeng, Kang Liu, Yubo Chen, and Jun Zhao. Distant supervision for relation extraction via piecewise convolutional neural networks. Proceedings of EMNLP, pp. 1753-1762, 2015.
Dongxu Zhang and Dong Wang. Relation classification via recurrent neural network. Proceedings of CoRR, 2015.
13

---

Published as a conference paper at ICLR 2021
A FUZZY RELATION EVALUATION DATASET
A.1 DISTANT SUPERVISION
Even though our framework is adaptive for a variety of data types, weakly (distantly) supervised data is the most suitable in terms of quantity and quality. Generally, weakly supervised data is constructed on the basis of a large unstructured textual data (e.g. Wikipedia) and large structured knowledge base (e.g. Wikidata) via the method of distant supervision. First, all the entities are identified by named entity recognition system which could tag persons, organizations and locations, etc. Those entities form entity set that is mentioned in Section 2. Then if for sentence 7, the entity pair (h, 1) also exists in the knowledge base with a relation r, then will be distantly annotated as r. After the distant supervision, all the relations form relation set R.
A.2 MOTIVATION OF FUZZYRED
The distant supervision assumption demonstrates that if two entities participate in relation, then all the sentences that mention these two entities express that relation. While obtaining plenty of weakly (distant) supervised data, this assumption has also become the primary drawback of distant supervision. Generally, one entity pair in the distant dataset may correspond to multiple relations in different textual environments. For example, the entity pair (Steve Jobs, California) has three relations, which are Place of Birth, Place of Work, and Place of Residence. Thus, instances of these corresponding relations prone to introducing palse-positive noise. We named such relations as fuzzy relations. Intuitively, conducting extraction on fuzzy relations is more challenging. To empirically evaluate if encoder trained on distant supervised could really learn the correct semantic from the noisy distant supervised corpus and perform well Fuzzy relations, we present a novel dataset, namely, Fuzzy Relation Extraction Dataset (FuzzyRED). In this section, We will firstly introduce the construction process for FuzzyRED and report some statistics.
A.3 CONSTRUCTION PROCESS OF FUZZYRED
Broadly speaking, the dataset is selected from a distant supervised corpus and judged by humans to determine whether each instance really expresses the same relation as the distant supervision labels. For instance, sentence "Jobs was born in Califormia" really expresses the supervision label "Place of Birth" while sentence "Jobs lived in California" does not. The detailed process is as follows:
First, following distant supervision assumption, we align the relation triples in Wikidata with Wikipedia articles to generate massive relation instances.
2. We count the number of corresponding relations for each entity pair, denote as CountR(pair)) For example, the entity pair (Steve Jobs, California) has three relations, which are Place of Birth, Place of Work, and Place of Residence. Larger CountR (pair) implics a higher risk of introducing noise. Thus, we consider a pair whose CountR (pair) larger than two
ambiguous seed of the relations that it is involved in.
3. Then, for each relation, count the number of ambiguous seeds, denote Counts(rel), The bigger Counts(rel) (rel) is, the more likely the relation will get the wrong data, which will lead to worse learning effect of the relation.
4. We sort the relations according to Counts(rel) and select the top 20 relations fuzzy relations of FuzzyRED. For each relation, 50 instances are randomly selected to annotate. For the specific annotation process, each annotator strictly judges whether the sentence could express the given relation according to the definition of the relation in Wikidata.
A.4 STATISTICS OF FUZZYRED
In FuzzyRED, there are 20 different kinds of relations, and 50 instances for each relation. As mentioned above, every instance is manually classified true positive (TP) or false positive (FP). Table 4 shows the statistics of FP rates, which are calculated as (ount[P)) Table 4, the average FP rate is more than 50 percent, in other words, the majority of the data are FPs. Hence, is challenging to learn a relation extraction model from FuzzyRED if it doesn't take the
count(FP)
As shown in
14

---

Published as a conference paper at ICLR 2021
Table 4: Statisties of false positive (FP) rate of the data of FuzzyRED. An FP statement means that the statement does not express the relation but is distandy annotated the relation.
Median
Average
Maximum
Minimum
False Positive Rate (%)
49.0
54.1
94.0
19.0
noise problem into consideration. In this paper, we empirically evaluate different relation encoders by conducting binary classification task. A better encoder is expected to distinguish TPs from FPs. Specifically, a model based on such relation encoder should predict higher probability for TPs.
B ALGORITHMS
In Section 3.2, we introduce the adaptation different downstream scenarios for our framework, which trains encoder EncØ. In this section, provide the algorithms for the three relation learning scenarios.
Algorithm reports the training of supervied relation learning.
Algorithm Training for Supervised Relation Extraction
Input: Supervised dataset S, = {81m.). statement encoder Enca
while not converge do Sample mini-batches Shotches from S,..
for Sbatch in Statches do for & in Sbatch do (nnc(ss)
p 0) Softmax W8 b) end for
Update W, b and Enc, w.r.t. ((0)
Σ
logp(u/s s, 0)
SE Soatch
end for end while
Similarly, Algorithm 2 illustrates the training algorithm for few-shot relation learning.
Algorithm 2 Training for Few-shot Relation Extraction
Input: Few-shot dataset Sy, statement encoder Enc,
repeat
R' SampleRelation [N, R).
S = 20, Q = Ø
for r in R' do
{ = Samplelnstancee(k: r), {1, 21.2...k
q' = SampleInstanc(1,r))
SU($), QU(x))
end for
for qr in Q do
for st" in S do
sim; = Enco( Enc end for
argmax,", (simç') (simg
Update Encs w.r.t. CrossEntropy end for until enough steps
15

---

Published as a conference paper at ICLR 2021
Algorithm 3 reports the training of fuzzy relation learning. Note that FuzzyRED is designed for evaluation, so the algorithm shows the inference phase.
Algorithm 3 Training for Fuzzy Relation Evaluation
Input: FuzzyRED S: = {8imh), statement encoder Encs
while converge do for s in S do
{1, 2, .... k) p(1 = 1|s, s;) Compute a w.r.t a = 1: p( if a > T then
{st] =Smmleenstannce(k)))))
is true positive (TP)
else
is false positive (FP)
end end for end while
16


Reviews:
[{"review_id": "y7FGxG175rE", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper proposes a method for learning representations for relation statements as well as classes (prototypes) for relation extraction tasks. I think the key insight is that the relation statements and classes (corresponding to relation types) can be learned jointly using contrastive training objectives. The paper also proposes to use the similarity metric as 1 / (1 + exp(cos(u, v))) and claims that this will provide a clear geometric interpretation and more interpretability. In the experiments, they first train this framework on distantly-supervised data constructed from Wikidata + Wikipedia and then fine-tune it on several downstream tasks: FewRel, SemEval, and a new dataset they created focused on identifying false positives in distantly supervised data.\n\nOverall, I think the proposed approach is quite reasonable and also seems to work well in the evaluation tasks (learning prototype embeddings instead of taking the average of instance embeddings and adopting contrastive losses between relation statements and prototypes). However, I think the paper has many clarity and presentation issues that make it difficult to evaluate the significance of the work. \n\nFirst of all, I think this pre-training stage on weakly-supervised data is very crucial and the details of the data collection (which relations and how many instances have been used) should be moved to the main body of the paper instead of the Appendix. In realistic few-shot scenarios, you only have a very small number of examples for a new relation k so it is difficult to learn the r_k embedding from only a few examples. My interpretation is that for the FewRel evaluation, the relations must have been already seen in the pre-training stage (given both the weakly-supervised data and FewRel are collected based on Wikidata) unless I have misunderstood something (especially that the model can achieve a good accuracy when there is no training data used in Figure 3 & 4). For the SemEval evaluation, I assume the prototype embeddings must have been learned from the training data but it has 6k training examples so it is fine. I think this point really needs to be clarified and can be a weakness of the approach, especially in few-shot settings.\n\nFor the results in Table 1 & 3, it is not very clear to me whether the numbers of previous approaches are from their papers or re-run by the authors. This should be clarified. My main concern is the pre-training data used in this paper can be different from what has been used in (Soares et al, 2019 -- they didn’t use Wikidata and only consider Wikipedia and the links) and it makes the comparisons unfair. \n\nThe authors claimed that this similarity metric is crucial but there is no ablation study or comparison to other alternatives… How if you just compute the dot product between the two embeddings? I think the comparisons to commonly used similarity metrics need to be added to justify why this design choice is important.\n\nI also don’t know what the L_{CLS} training loss is used for. If z_k is just a set of learnable embeddings (one embedding per relation) and if it is used to predict the relation k, isn’t it just multiplied by another set of K embeddings? What is the benefit here?\n\nAlso, I don’t understand the fixed prototype baseline (denoted as IND). What does “fixed prototypes z that are pre-computed by vectorize extracted patterns for each relation” mean?\n\nTo sum up, I can’t recommend the acceptance of the paper if the above issues cannot be addressed. I am also concerned whether the highlight of the approach is the contrastive losses and prototype embeddings, or it has to be coupled with some type of pre-training (or even specifically pre-training on distantly-supervised data). \n\nMinor:\n- Equation (7): The second S2Z should be S2Z’.\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "KbuYyXpG85", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Summary:\n\nThis paper proposed a prototype-based relation learning method which infers a prototype vector for each relation type to help train a better text encoder. And authors introduced several loss functions including statement-statement similarities and statement-prototype similarities. \nExperimental results over both few-shot and supervised scenarios indicate that these additional training losses improve the performance significantly. Furthermore, the author provided a dataset to indicate that the proposed method can effectively reduce the noise from distant supervision. \n\n#####################\n\nReasons for score:\n\nI vote for marginally positive. Overall, the paper is interesting and the experiments are extensive and promising. However, the main concerns listed below prevent me from giving full acceptance. And I would like to hear further responses from the author.\n\n#####################\n\nMain concern:\n\nBesides promising results shown in the paper, there are some concerns about the concept of \"prototype\" in this paper: \n\n1. If I understand correctly, the pretraining and fine-tuning are using the same data for all the experiments because the author didn't mention extra corpus for pretraining. Therefore prototypes are consistent (by consistent I mean there is one-to-one correspondence) with the annotated relation types in the fine-tuning (This is a little confusing because pretraining usually means using a different corpus).\n\n2. And if prototype relation types have one-to-one mappings with relation types during fine-tuning, the key difference between the proposed method and direct training a relation classifier is that prototype vectors help to train the sentence (also called statement) encoder through S2S and S2Z losses, and then a separate classifier is trained over the statement vectors for classification. Then I'm curious why not directly add S2S and S2Z losses to the classifier? What is the reason behind generating prototype vectors? \nFrom my understanding, \"prototype\" usually means that, in the meta-learning and few shot scenarios, we can generate a new model from the prototype model. However, according to formula 9, during inference, the model uses a nearest neighbor classifier without any model generation. \n\n\n##############################\n\nSome minor comments:\n\n(1) Does formula 2 miss a negative sign for both similarity metrics? Otherwise the larger the cosine, the smaller the metric value, which is dissimilarity.\n(2) it is not clear to me how to get the baseline of fixed prototypes vectors from vectors of extracted patterns. Is it average?\n(3) This work shares similar spirits with Row-less Universal Schema (which has been cited in the reference) where relation type embeddings guide clustering of statement embeddings using distant supervision. A brief comparison will be useful in the related work for completeness.\n\n\n\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "Dcicqy6qR3c", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 7, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Summary:\nThe authors propose a novel method for learning prototype representation for relations which abstracts the essential semantics of relations between entities in sentences. The learned prototypes are learned based on an objective with clear geometric interpretation and have been shown to be interpretable and robust to noisy from distantly-supervised data. The method has been shown to be effective for supervised, few-shot, and distantly supervised relation extraction. The prototype embeddings are are unit vectors uniformly dispersed in a unit ball and statement embeddings are centered at the end of their corresponding prototypes. The training is done such that intra-class compactness and inter-class separability is increased. The results show that the proposed approach gives state-of-the-art results for few-shot, supervised relation extraction. The authors demonstrate the interpretability and robustness of the method. \n\nQuestions:\n1. It is a bit hard to follow how increasing the number of relation types in Figure 3 improves accuracy although the problem should become harder with more relations. \n2. Also, what is the intuition behind the initial decrease in performance in Figure 4 on increasing the number of instances per type although the same is not observed on other methods. \n", "extra_scores": {}, "extra_text": {}}, {"review_id": "2ap5IxQx_Rj", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper presents a pre-training method for encoders (i.e., BERT) of relation extraction, leveraging distant supervision data. The main idea is to introduce a prototype embedding for each relation in the distantly generated data. The loss function to pre-train the encoders involve several terms, aiming to exploit the intra-class compactness (e.g., the contrastive loss for statement pairs) and the inter-class separability (i.e., constraining the distances between the prototype for one relation and the embeddings of the statements for the same and different relations). The paper incorporates a prototype-level classification loss. Compared to few-shot learning models (e.g., prototypical networks), the paper claims that prototype-based losses can help the model to better address the noisy label issue of distantly generated data due to the exploitation of prototype and instance interactions. The pre-trained encoder is then applied on datasets for few-shot, supervised, and fuzzy relation extraction. The proposed method achieves competitive performance with other methods. Some analysis are also conducted to demonstrate the effectiveness of the proposed model.\n\nOverall, this paper is well written and the proposed method seems helpful for improving the performance of relation extraction on several settings according to the presented experiments. However, I have several concerns regarding the motivation and experiments of the papers as follow.\n\n\n1. The paper loss terms in Equations (2), (3), and (4) seem intuitive. However, they are all based on potentially noisy assignments of labels for statements (due to distant supervision). The paper mentions that as cross-entropy loss only considers instance-level supervision, it is particularly noisy (page 4). However, it is unclear why Equations (2), (3), and (4) can better handle the noise given that the information they rely on is still inherently noisy. Relatedly, a more direct baseline for the proposed model is to just fine-tune the encoder with cross-entropy loss using the standard supervised learning setting with distantly supervised data (i.e., directly predict distant labels from statements). As this baseline seems missing, it is unclear why the proposed losses are helpful and convincing.\n\n2. The paper uses the pre-training approach with distantly generated data. Another approach is to use multi-task learning where the encoder is trained simultaneously on both downstream task and distantly-generated data. How does this approach compare to the proposed method. Also, how does the relations in the distantly generated data differ from those in the few-shot and supervised experiments?\n\n3. In Appendix A, the paper mentions that the encoder is also fine-tuned with masked language model loss. This seems to be an important factor (given its effectiveness in BERT); however, it is not evaluated and discussed in the main paper. How does the performance changes if this masked language model loss is not used?\n\n4. The performance in Table 3 makes the proposed method less convincing. In particular, the proposed method (COL Final) essentially has the same performance as the baseline (IND) which raises a question about the benefits of the proposed techniques. There is no discussion between the performance difference between these two models to better justify the model. The performance gap between the proposed model and the baseline MTB seems minor too. Also, Soares et al., 2019 seem to report a different performance on SemEval than those indicated in this paper (i.e., the best score in there is 89.5). Can you clarify this detail?", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper proposes a method for regularizing the pre-training of an embedding function for relation extraction from text that encourages well-formed clusters among the relation types. Experiments on FewRel, SemEval 2010 Task 8, and a proposed FuzzyRed dataset show that the proposed prototype method generally outperforms prior state-of-the-art, including MTB (Soares et al., 2019), which was the strongest. The key, novel idea is to model prototype representations for target relations as part of the learning process. A contribution of the work is to show that learning prototype representations are useful in supervised deep learning architectures even beyond few-shot learning. This additional learning objective is useful as an inductive bias, and is perhaps of interest even beyond relation extraction research.

Reviewers generally found the proposed method sound and intuitive, and the original set of experiments promising. Some of the reviewers raised concerns about the setup of the experiments, including the relationship between the pre-training and target tasks, and the need for several additional baselines. The authors were able to address these concerns, and the reviewers did not raise any follow-up concerns.
Decision: Accept (Poster)