Open Set Relation Extraction via Unknown-Aware Training
Year: 2023

Abstract:
The existing supervised relation extraction methods have achieved impressive performance in a closed-set setting, where the relations during both training and testing remain the same. In a more realistic open-set setting, unknown relations may appear in the test set. Due to the lack of supervision signals from unknown relations, a well-performing closedset relation extractor can still confidently misclassify them into known relations. In this paper, we propose an unknown-aware training method, regularizing the model by dynamically synthesizing negative instances. To facilitate a compact decision boundary, “difficult” negative instances are necessary. Inspired by text adversarial attacks, we adaptively apply small but critical perturbations to original training instances and thus synthesizing negative instances that are more likely to be mistaken by the model as known relations. Experimental results show that this method achieves SOTA unknown relation detection without compromising the classification of known relations.

heading: 1 Introduction
text: Relation extraction (RE) is an important basic task in the field of natural language processing, aiming to extract the relation between entity pairs from unstructured text. The extracted relation facts have a great practical interest to various downstream applications, such as dialog system (Madotto et al., 2018), knowledge graph (Lin et al., 2015), web search (Xiong et al., 2017), among others.
Many efforts have been devoted to improving the quality of extracted relation facts (Han et al., 2020). Conventional supervised relation extraction is oriented to known relations with pre-specified schema. Hence, the paradigm follows a closedset setting, meaning that during both training and testing the relations remain the same. Nowadays,
*Equal Contributions. †Corresponding authors.
neural RE methods have achieved remarkable success within this setting (Wang et al., 2016; Wu and He, 2019); and in contrast, open relation extraction (OpenRE) is focused on discovering constantly emerging unknown relations. Common practices include directly tagging the relational phrases that link entity pairs (Zhan and Zhao, 2020), and clustering instances with the same relation (Hu et al., 2020; Zhao et al., 2021). However, relation extraction in real applications follows an open-set setting, meaning that both known and unknown relations are mixed within testing data.* This requires that a model can not only distinguish among the known relations, but also filter the instances that express unknown relations. The ability to filter these instances is also called none-of-the-above (NOTA) detection (Gao et al., 2019).
Unfortunately, a well-performing closed-set model can still confidently make arbitrarily wrong predictions when exposed to unknown test data (Nguyen et al., 2015; Recht et al., 2019). As
*Some sentences even express no specific relations.
9453
shown in fig. 1 (a), the decision boundary is optimized only on the known relational data (white points), leading to a three-way partition of the whole space. Consequently, the unknown relational data (black points), especially those far from the decision boundary, will be confidently classified into one of the known relations. By contrast, a more compact decision boundary (as shown in fig. 1 (b)) is desirable for NOTA detection. However, the compact decision boundary requires “difficult” negative data (red points in fig. 1 (b)) to be used, so strong supervision signals can be provided. It is important to note that synthesizing such negative data is a non-trivial task.
In this work, we propose an unknown-aware training method, which simultaneously optimizes known relation classification and NOTA detection. To effectively regularize the classification, we iteratively generate negative instances and optimize a NOTA detection score. During the testing phase, instances with low scores are considered as NOTA and filtered out. The key of the method is to synthesize “difficult” negative instances. Inspired by text adversarial attacks, we achieve the goal by substituting a small number of critical tokens in original training instances. This would erase the original relational semantics and the model is not aware of it. By using gradient-based token attribution and linguistic rules, key tokens that express the target relation are found. Then, the tokens are substituted by misleading normal tokens that would cause the greatest increase of NOTA detection score, thus misleading negative instances, which are more likely to be mistaken by the model as known relations, are synthesized. Human evaluation shows that almost all the synthesized negative instances do not express any known relations. Experimental results show that the proposed method learns more compact decision boundary and achieve state-of-the-art NOTA detection performance. Our codes are publicly available at Github.†
The contributions are threefold: (1) we propose a new unknown-aware training method for more realistic open-set relation extraction. The method achieves state-of-the-art NOTA detection, without compromising the classification of known relations; (2) the negative instances are more challenging to the model, when compared to the mainstream
†https://github.com/XinZhao0211/OpenSetRE.
synthesis method ‡ (e.g., generative adversarial network (GAN)-based method); (3) the comprehensive evaluation and analysis facilitate future research on the pressing but underexplored task.

heading: 2 Related Works
text: Open-set Classification: The open-set setting considers knowledge acquired during training phase to be incomplete, thereby new unknown classes can be encountered during testing. The pioneering explorations in (Scheirer et al., 2013) formalize the open-set classification task, and have inspired a number of subsequent works, which roughly fall into one of the following two groups.
The first group explores model regularization using unknown data. Larson et al. (2019) manually collect unknown data to train a (n + 1)-way classifier with one additional class, where (n+1)th class represents the unknown class. Instead of manually collecting unknown data, Zheng et al. (2020) generate feature vectors of unknown data using a generative adversarial network (Goodfellow et al., 2014). Zhan et al. (2021) use MixUp technique (Thulasidasan et al., 2019a) to synthesize known data into unknown data.
The second group approaches this problem by discriminative representation learning, which facilitates open-set classification by widening the margin between known and unknown classes. MSP (Hendrycks et al., 2017) is a maximum posterior probability-based baseline and ODIN (Liang et al., 2018) enlarges the difference between known and unknown classes by adding temperature scaling and perturbations to MSP. More recently, different optimization objectives such as large margin loss (Lin and Xu, 2019) and gaussian mixture loss (Yan et al., 2020) are adopted to learn more discriminative representations. Shu et al. (2017); Xu et al. (2020); Zhang et al. (2021) also impose gaussian assumption to data distribution to facilitate distinct unknown data. Open-set Relation Extraction: Open-set RE is a pressing but underexplored task. Most of the existing RE methods manually collect NOTA data and adopt a (n + 1) way classifier to deal with NOTA relations (Zhang et al., 2018; Zhu et al., 2019; Ma et al., 2021). However, the collected NOTA data with manual bias cannot cover all NOTA relations and thus these methods cannot effectively deal with open-set RE (Gao et al., 2019).
‡A quantitative analysis will be provided in Sec. 5.2.
Our method avoids the bias and the expensive cost of manually collecting NOTA data by automatically synthesizing negative data. Compared with general open-set classification methods, our method takes relational linguistic rules into consideration and outperforms them by a large margin.

heading: 3 Approach
text: We start by formulating the open-set relation extraction task. Let K = {r1, ..., rn} denote the set of known relations and NOTA indicates that the instance does not express any relation in K. Given a training set Dtrain = {(xi, yi)}Ni=1 with N positive samples, consisting of relation instance xi with a pre-specified entity pair § and relation yi ∈ K, we aim to learn a open-set relation extractor M = {pθ(y|x), sθ(x)}, where θ denote the model parameters. pθ(y|x) is the classification probability on the known relations (The NOTA label is excluded from pθ(y|x)). NOTA detection score sθ(x) is used to distinguish between known relations and NOTA. x is classified as NOTA if sθ(x) is less than the threshold α. Conversely, x is classified into a known relation ŷ = argmaxy pθ(y|x).

heading: 3.1 Method Overview
text: We approach the problem by an unknown-aware training method, which dynamically synthesizes “difficult” negative instances and optimizes the dual objectives of both known relation classification and NOTA detection. As shown in fig. 2, the training
§We assume that the entity recognition has already been done and an instance expresses at most one relation between the entity pair.
loop consists of two iteration steps: ① Synthesis Step: This step aims to synthesize “difficult” negative instances for model regularization. We draw inspiration from text adversarial attacks to achieve the goal. Specifically, B = {(xi, yi)}Bi=1 represents a training batch sampled from Dtrain. For each (x, y) ∈ B, we synthesize a negative instance by substituting the key relational tokens of x with misleading tokens. First, both the attribution method and relational linguistic rules are used to find key tokens expressing the target relation y. Second, the misleading token wmisi is searched for each key token wi, along the direction of the gradient ∇wisθ(x). By substituting wi with wmisi , it is expected for sθ(x) to experience its greatest increase, so it is difficult for the model to correctly detect the derived negative instance x′ as NOTA. ② Learning Step: This step aims to optimize the open-set relation extractor M = {pθ(y|x), sθ(x)}. Based on the training batch B from Dtrain, we optimize pθ(y|x) to accurately classify known relations. To effectively detect NOTA instances, we further synthesize negative batch B′ = {(x′i,NOTA)}Bi=1 and optimize the model to widen the gap of sθ(x) between x ∈ B and x′ ∈ B′. Consequently, instances with low sθ(x) scores are filtered out before being fed into pθ(y|x).
Next, we elaborate on the model structure of M (sec. 3.2) and the technical details of the synthesis step (sec. 3.3) and the learning step (sec. 3.4).

heading: 3.2 Open-set Relation Extractor
text: Instance Encoder and Classifier: Given an input instance x = {w1, .., wn} with four reserved
special tokens [E1], [\E1], [E2], [\E2] marking the beginning and end of the head and tail entities, the instance encoder aims to encode the relational semantics into a fixed-length representation h = enc(x) ∈ Rd. We adopt BERT (Devlin et al., 2018), a common practice, as the implementation of the encoder. We follow Baldini Soares et al. (2019) to concatenate the hidden states of special tokens [E1] and [E2] as the representation of the input instance.
w1, ..,wn = BERT(w1, .., wn) (1)
h = w[E1] ⊕w[E2], (2)
where wi,w[E1],w[E2] denotes the hidden states of token wi, [E1], [E2], respectively. ⊕ denotes the concatenation operator. The classification probability on known relations pθ(·|x) can be derived through a linear head η(·):
η(h) = Wclsh+ b (3) pθ(·|x) = Softmax(η(h)), (4)
where Wcls ∈ Rn×d is the weight matrix transforming the relation representation to the logits on n known relations and b is the bias. NOTA Detection Score: The goal of distinguishing between known and NOTA relations requires the modeling of the data density. However, directly estimating log p(x) can be computationally intractable because it requires sampling from the entire input space. Inspired by Liu et al. (2020) in the image understanding task, the free energy function E(h) is theoretically proportional to the probability density of training data. Considering that it can be easily derived from the linear head η(·) without additional calculation, the negative free energy function is used to compute the NOTA detection score as follows:
sθ(x) = −E(h) = log n∑
j=1
eη(h)j , (5)
where η(h)j denotes the jth logit value of η(h). The detection score has shown to be effective in out-of-distribution detection (Liu et al., 2020). Based on the classification probability pθ(·|x) and NOTA detection score sθ(x), the open-set relation extractor M works in the following way:
ŷ =
{ argmaxy pθ(y|x) S(x) > α
NOTA S(x) ≤ α, (6)
where α is the detection threshold.

heading: 3.3 Iterative Negative Instances Synthesis
text: “Difficult” negative instances are the key to effective model regularization. x = {w1, .., wn} is a training instance with a label y. To synthesize negative instance x′, we perturb each key token wi, which expresses the relation y, with a misleading token wmisi . The substitutions are expected to erase original relational semantics without the model being aware of it. Based on the attribution technique and relational linguistic rules, a score I(wi, x, y) is developed to measure the contribution of a token wi ∈ x to relation y as follows:
I(wi, x, y) = a(wi, x) · t(wi, y) · dp(wi, x), (7)
where a(wi, x) denotes an attribution score reweighted by two linguistic scores t(wi, y), dp(wi, x). We rank all tokens according to I(wi, x, y) in descending order and take the first ϵ percent of tokens as key tokens to perform substitutions. Next, we elaborate on (1) how to calculate the attribution score a(wi, x) and linguistic scores t(wi, y), dp(wi, x); (2) how to select misleading tokens for substitution. Gradient-based Token Attribution: Ideally, when the key tokens are removed, instance x will no longer express the original known relation y, and the NOTA detection score sθ(x) would drop accordingly. Therefore, the contribution of a token wi to relational semantics can be measured by a counterfactual:
c(wi, x) = sθ(x)− sθ(x−wi), (8)
where x−wi is the instance after removing wi. However, to calculate the contribution of each token in instance x, n forward passes are needed, which is highly inefficient. Fortunately, a firstorder approximation of contribution c(wi, x) can be obtained by calculating the dot product of word embedding wi and the gradient of sθ(x) with respect to wi, that is ∇wisθ(x) · wi (Feng et al., 2018). The contribution of n tokens can thus be computed with a single forward-backward pass. Finally, a normalized attribution score is used, in order to represent the contribution of each token:
a(wi, x) = |∇wisθ(x) ·wi|∑n j=1 |∇wjsθ(x) ·wj | . (9)
Linguistic Rule-based Token Reweighting: As a supplement to the attribution method, linguistic
rules that describe the pattern of relational phrases can provide valuable prior knowledge for the measure of tokens’ contribution. Specifically, the following two rules are used. Rule 1: If a token wi significantly contributes to relation y, it should appear more frequently in the instances of y, and rarely in the instances of other relations. By following this rule, tf-idf statistic (Salton and Buckley, 1987) t(wi, y)¶ is used to reflect the contribution of token wi to relation y (Appendix A.1 contains additional details about the statistic). Rule 2: Tokens that are part of the dependency path between the entity pair usually express the relation between the entity pair, while shorter dependency paths are more likely to represent the relation (ElSahar et al., 2018). Following the rule, stanza|| is used to parse the instance and the dependency score as calculated as follows:
dp(wi, x) =
{ |x|/|T | wi ∈ T
1, otherwise, (10)
where T denotes the set of tokens in the dependency path between the entity pair. |x|, |T | denote the number of tokens in instance x and set T , respectively. Eq. 10 indicates that the tokens in T are given a higher weight, and the shorter the path, the higher the weight. Misleading Token Selection: Negative instances are synthesized by substituting key tokens with misleading tokens. Note that we have obtained the gradient of sθ(x) with respect to each token wi in the attribution step. Based on the gradient vectors, a misleading token is selected from vocabulary V for each key token wi as follows:
wmisi = argmax wj∈V ∇wisθ(x) ·wj . (11)
Substituting wi with wmisi is expected to cause the greatest increase in sθ(x), so the synthesized negative instance is misleading to the model. To avoid that wmisi is also a key token of a known relation, the top 100 tokens with the highest tf-idf statistic of each relation are removed from the vocabulary V , when performing the substitution. Human evaluation results show that almost all the synthesized negative instances do not express any known relation. In addition, we provide two real substitution cases in tab. 6.
¶The statistic is based on the whole training set and does not change with a specific instance x.
||https://stanfordnlp.github.io/stanza/depparse.html

heading: 3.4 Unknown-Aware Training Objective
text: In this section, we introduce the unknown-aware training objective for open-set relation extraction. Based on the synthesized negative samples, an optimization of the dual objectives of both known relation classification and NOTA relation detection is performed. Specifically, at the mth training step, A batch of training data Bm = {(xi, yi)}Bi=1 is sampled from Dtrain. Cross entropy loss is used for the optimization of known relation classification:
Lcls = 1
B
B∑
i=1
(− log pθ(yi|xi)), (12)
where pθ(·|xi) is the classification probability on the known relations (eq. 4). For each instance x in Bm, we synthesize a negative sample x′ as described in sec. 3.3, and finally obtain a batch of negative samples B′m = {(x′i,NOTA)}Bi=1. To learn a compact decision boundary for NOTA detection, we use the binary sigmoid loss to enlarge the gap of detection scores sθ(·) between known and synthesized instances as follows:
LNOTA = − 1
B
B∑
i=1
log σ(sθ(xi))
− 1 B
B∑
i=1
log(1− σ(sθ(x′i))) (13)
where σ(x) = 1 1+e−x is the sigmoid function. The overall optimization objective is as follows:
L = Lcls + β · LNOTA, (14)
where β is a hyper-parameter to balance the two loss term.

heading: 4 Experimental Setup

heading: 4.1 Datasets
text: FewRel (Han et al., 2018). FewRel is a humanannotated dataset, which contains 80 types of relations, each with 700 instances. We take the top 40 relations as known relations. The middle 20 relations are taken as unknown relations for validation. And the remaining 20 relations are unknown relations for testing. Our training set contains 22,400 instances from the 40 known relations. Both the validation and test set consist of 5,600 instances, of which 50% are from unknown relations. Note that the unknown relations in the test set and the validation set do not overlap.
TACRED (Zhang et al., 2017). TACRED is a largescale relation extraction dataset, which contains 41 relations and a no_relation label indicating no defined relation exists. Similar to FewRel, we take the top 21 relations as known relations. The middle 10 relations are taken as unknown relations for validation. The remaining 10 relations and no_relation are unknown relations for testing. We randomly sample 9,784 instances of known relations to form the training set. Both the validation and test set consist of 2,000 instances, of which 50% are from unknown relations. Unknown relations in the validation set and the test set still do not overlap.
For the specific composition of relations in each dataset, please refer to Appendix A.4.

heading: 4.2 Compared Methods
text: To evaluate the effectiveness of the proposed method, we compare our method with mainstream open-set classification methods, which can be roughly grouped into the following categories: MSP (Hendrycks et al., 2017), DOC (Shu et al., 2017), ODIN (Liang et al., 2018), Energy (Liu et al., 2020), and SCL (Zeng et al., 2021) detect unknown data through a carefully designed score function or learning a more discriminative representation. No synthesized negative instances are used in these methods. MixUp (Thulasidasan et al., 2019b), and Convex (Zhan et al., 2021) use synthesized negative instances to regularize the model. Please refer to the appendix A.3 for a brief introduction to these methods.
We do not compare BERT-PAIR (Gao et al., 2019) because it is only applicable to the few-shot setting. We use DOC (Shu et al., 2017) with a BERT encoder as an alternative method for it.

heading: 4.3 Metrics
text: Following previous works (Liu et al., 2020; Zeng et al., 2021), we treat all unknown instances as one NOTA class and adopt three widely used metrics for evaluation. (1) FPR95: The false positive rate of NOTA instances when the true positive rate of known instances is at 95%. The smaller the value, the better. (2) AUROC: the area under the receiver operating characteristic curve. It is a threshold-free metric that measures how well the detection score ranks the instances of known and NOTA relations. (3) ACC: The classification accuracy on n known relations and one NOTA relation, measuring the overall performance of open-set RE.

heading: 4.4 Implementation Details
text: We use the AdamW as the optimizer, with a learning rate of 2e−5 and batch size of 16 for both datasets. Major hyperparameters are selected with grid search according to the model performance on a validation set. The detection threshold is set to the value at which the true positive rate of known instances is at 95%. The regularization weight β is 0.05 selected from {0.01, 0.05, 0.1, 0.15, 0.5}. See the appendix A.2 for the processing of sub-tokens. The dependency parsing is performed with stanza 1.4.2. All experiments are conducted with Python 3.8.5 and PyTorch 1.7.0, using a GeForce GTX 2080Ti with 12GB memory.

heading: 5 Results and Analysis

heading: 5.1 Main Results
text: In this section, we evaluate the proposed method by comparing it with several competitive open-set classification methods. The results are reported in tab. 1, from which we can observe that our method
achieves state-of-the-art NOTA detection (reflected by FPR95 and AUROC) without compromising the classification of known relations (reflected by ACC). In some baseline methods (e.g., MSP, ODIN, Energy, SCL), only instances of known relations are used for training. Compared with them, we explicitly synthesize the negative instances to complete the missing supervision signals, and the improvement in NOTA detection shows the effectiveness of the unknown-aware training. To intuitively show the changes of the decision boundary, we use the method of Yu et al. (2019) to visualize the decision boundary of the model in the input space. As can be seen from fig. 3, a more compact decision boundary is learned with the help of unknown-aware training. Although methods such as MixUp, and Convex also synthesized negative instances, our method is still superior to
them. This may be due to the fact that our negative instances are more difficult and thus beneficial for an effective model regularization (we provide more results in sec. 5.2 to support the claim).

heading: 5.2 Negative Instance Synthesis Analysis
text: In this section, the unknown-aware training objective is combined with the various negative instance synthesis methods to fairly compare the performance of these synthesis methods. The results are shown in tab. 2. Baseline means no negative instances are used. Gaussian takes Gaussian noise as negative instances and Gaussian† adds the noise to known instances. MixUp synthesizes negative instances by convexly combining pairs of known instances. Real means using real NOTA instances**. GAN synthesizes negative instances by Generative Adversarial Network (Ryu et al., 2018). Correlation between effectiveness and difficulty. (1) Gaussian with the largest ∆sθ performs even worse than Baseline in TACRED, suggesting that overly simple negative instances are almost ineffective for model regularization. (2) Our method synthesizes the second difficult negative instances (reflected by ∆sθ) and achieves the best performance (reflected by ACC, AUROC, FPR95), which shows that the difficult negative instances are very beneficial for effective model regularization. (3) The difficulty of negative instances of competitive methods (e.g., MixUp, Real, GAN) is lower than that of Ours, which indicates that it is non-trivial to achieve our difficulty level. (4) Although Gaussian† synthesizes
**We use the data from SemEval-2010 (Hendrickx et al., 2010). The overlap relations are manually removed.
the most difficult negative instances, our method still significantly outperforms Gaussian†. One possible reason is that overly difficult instances may express the semantics of known relations. This leads to the following research question.

heading: Do our synthetic negative instances really not
text: express any known relations? We conduct human evaluation to answer this question. Specifically, we randomly select 100 synthesized negative instances on each dataset and asked human judges whether these instances express known or NOTA relations. The evaluation is completed by three independent human judges. We recruit 3 graduates in computer science and English majors from top universities. All of them passed a test batch. Each graduate is paid $8 per hour. The results are shown in tab. 3, from which we can observe that: (1) More than 90% of the negative instances do not express any known relations (NOTA). (2) Very few instances remain in the original known relations (Known-Original) or are transferred to another known relation (Known-Other). (3) There are also some instances that are Controversial. Some volunteers believe that the instances express known relations, while others believe that the instances are NOTA. In general, our synthesis method achieves satisfactory results, but there is still potential for further improvement.

heading: 5.3 Ablation Study
text: To study the contribution of each component in our method, we conduct ablation experiments on the two datasets and show the results in tab. 4. First, the attribution score measures the impact of a token on NOTA detection of the model. The dependency score and tf-idf statistic reflect the matching degree between a token and the relational linguistic rules. When the three scores are removed, there may be some key relational phrases that can not be correctly identified and the performance decline accordingly. It is worth mentioning that the model parameters change dynamically with the training process, thus
iteratively synthesizing negative instances is crucial for effective regularization. When the practice is removed, the static negative instances can not reflect the latest state of the model, and thus the performance degrades significantly. Finally, we remove misleading token selection by substituting the identified key tokens with a special token [MASK] and the performance is seriously hurt, which indicates that misleading tokens play an important role in synthesizing difficult instances.

heading: 5.4 Hyper-parameter Analysis
text: We synthesize negative instances by substituting ϵ percent of key tokens with misleading tokens. In this section, we conduct experiments to study the influence of substitution ratio ϵ on NOTA detection. From fig. 4 we obtain the following observations. When the substitution ratio gradually increases from 0, the performance of NOTA detection is also improved (Note that the smaller the value of FPR95, the better). This means that an overly small substitution ratio is not sufficient to remove all relational phrases. The residual relational tokens are detrimental to model regularization. When the substitution ratio exceeds a certain threshold (i.e.,
0.2), a continued increase in the substitution ratio will lead to a decline in detection performance. One possible reason is that too high a substitution ratio can severely damage the original sentence structure, resulting in negative instances that differ too much from the real NOTA instances.

heading: 6 Conclusions
text: In this work, we propose an unknown-aware training method for open-set relation extraction, which is a pressing but underexplored task. We dynamically synthesize negative instances by the attribution technique and relational linguistic rules to complete the missing supervision signals. The negative instances are more difficult than that of other competitive methods and achieve effective model regularization. Experimental results show that our method achieves state-of-the-art NOTA detection without compromising the classification of known relations. We hope our method and analysis can inspire future research on this task.
Limitations
We synthesize negative instances by substituting relational phrases with misleading tokens. However, the relational semantics in some instances may be expressed implicitly. That is, there are no key tokens that directly correspond to the target relation. Therefore, we cannot synthesize negative instances based on these instances. Additionally, we consider substitution ratio ϵ as a fixed hyperparameter. It may be a better choice to dynamically determine ϵ based on the input instance. We leave these limitations as our future work.

heading: Acknowledgements
text: The authors wish to thank the anonymous reviewers for their helpful comments. This work was partially funded by National Natural Science Foundation of China (No.62206057,62076069,61976056), Shanghai Rising-Star Program (23QA1400200), Program of Shanghai Academic Research Leader under grant 22XD1401100, and Natural Science Foundation of Shanghai (23ZR1403500).

heading: A Appendix
text: A.1 Tf-idf statistic We consider a token wi to contribute significantly to a known relation y ∈ K if it occurs frequently in the instances of relation y and rarely in the instances of other relations. Tf-idf statistic (Salton and Buckley, 1987) can well characterize this property. Specifically, Tf-idf consists of term frequency and inverse document frequency. The term frequency tf(wi, y) describes how often a token wi appears in the instances of relation y:
tf(wi, y) = n(wi, y)∑
wj∈V n(wj , y) , (15)
where n(wi, y) denotes the number of times the token wi appears in the instances of relation y. Obviously, some tokens (e.g., the stop words) have high tf values in different relational instances. However, they do not contribute to the relational semantics. The inverse document frequency describes whether the token wi appears only in the instances of specific relations:
idf(wi) = log |K|
|{y : n(wi, y) ̸= 0}| , (16)
where |K| denotes total number of known relations and |{y : n(wi, y) ̸= 0}| denotes the number of known relations that token wi appears in their instances. Finally, we calculate t(wi, y) as follows:
t(wi, y) = tf(wi, y)× idf(wi). (17)
The tf-idf statistic t(wi, y) measures the contribution of token wi to the relation semantics of y. We calculate and store the statistics based on the entire training set Dtrain before the training loop start. During the training, the statistic of each token in the vocabulary is fixed.
A.2 How to Deal With Sub-tokens?
BERT adopts BPE encoding to construct vocabularies. While most tokens are still single tokens, rare tokens are tokenized into sub-tokens. In this section, we introduce how to deal with subtokens when performing the substitution. First, the tf-idf statistics and the dependency scores are calculated at the token level and require no additional process. If a token consists of n sub-tokens, we calculate its attribution score by summing the scores of all its sub-tokens. In addition, the misleading token of this token is only selected from the tokens that also have n sub-tokens according to argmaxwj∈Vn ∑n k=1∇wi,ksθ(x) · wj,k. Vn denotes a vocabulary, in which all tokens consist of n sub-tokens. wi,k denotes the embedding of the kth sub-token of the token wi.
A.3 Compared Methods
To validate the effectiveness of the proposed method, we compare our method with mainstream open-set classification methods. MSP (Hendrycks et al., 2017). MSP assumes that correctly classified instances tend to have greater maximum softmax probability than samples of unknown classes. Therefore, the maximum softmax probability is used as the detection score. DOC (Shu et al., 2017). DOC builds a 1-vs-rest layer containing m binary sigmoid classifiers for m known classes. The maximum probability of m binary classifiers is used as the detection score. ODIN (Liang et al., 2018). Based on MSP, ODIN uses temperature scaling and small perturbations to separate the softmax score distributions between samples of known and unknown classes. MixUp (Thulasidasan et al., 2019b). MixUp trains the model on convexly combined pairs of instances, which is effective to calibrate the softmax scores.
Energy (Liu et al., 2020). Instead of maximum softmax probability, this method uses the free energy E(x) = − log∑Kk=1 efk(x) as the detection score of the unknown data. Convex (Zhan et al., 2021). The method learns a more discriminative representation by generating synthetic outliers using inlier features. SCL (Zeng et al., 2021). SCL proposes a supervised contrastive learning objective, learning a more discriminative representation for unknown data detection.
A.4 Relations comprising the datasets In this subsection, we present the known relations contained in the training set, the unknown relations included in the validation set, and the unknown relations present in the test set, as shown in Table 5.
A.5 Additional Results Classification Accuracy: One of our key claims is that the proposed method achieves state-of-theart SOTA detection without compromising the classification of known relations. In this section, we provide an additional ACC metric, in which only the instances of n known relations are used to calculate the classification accuracy. The metric exactly indicates whether NOTA detection impairs the classification of known relations. From tab. 7 we can observe that our method is comparable to the existing method, which supports the key claim at the beginning of the paragraph. Two Real Substitution Cases: To intuitively show the effectiveness of the proposed synthesis method,
we conduct a case study based on the “Instrument” relation from FewRel and the “Spouse” relation from TACRED. The tokens with top-10 tf-idf statistics and a substitution case of each relation are shown in tab. 6, from which we can observe that: (1) the tokens with high tf-idf statistics have a strong semantic association with the target relation (such as Instrument-bass, Spouse-wife). (2) By substituting only two critical tokens in original training instances, the target relation is completely erased.
ACL 2023 Responsible NLP Checklist

heading: A For every submission:
text: 3 A1. Did you describe the limitations of your work?
Limitations section.
A2. Did you discuss any potential risks of your work? Not applicable. Left blank.
3 A3. Do the abstract and introduction summarize the paper’s main claims? abstract and introduction sections.
7 A4. Have you used AI writing assistants when working on this paper? Left blank.
B 3 Did you use or create scientific artifacts? section 4
3 B1. Did you cite the creators of artifacts you used? section 4
3 B2. Did you discuss the license or terms for use and / or distribution of any artifacts? section 4
B3. Did you discuss if your use of existing artifact(s) was consistent with their intended use, provided that it was specified? For the artifacts you create, do you specify intended use and whether that is compatible with the original access conditions (in particular, derivatives of data accessed for research purposes should not be used outside of research contexts)? Not applicable. Left blank.
B4. Did you discuss the steps taken to check whether the data that was collected / used contains any information that names or uniquely identifies individual people or offensive content, and the steps taken to protect / anonymize it? Not applicable. Left blank.
B5. Did you provide documentation of the artifacts, e.g., coverage of domains, languages, and linguistic phenomena, demographic groups represented, etc.? Not applicable. Left blank.
3 B6. Did you report relevant statistics like the number of examples, details of train / test / dev splits, etc. for the data that you used / created? Even for commonly-used benchmark datasets, include the number of examples in train / validation / test splits, as these provide necessary context for a reader to understand experimental results. For example, small differences in accuracy on large test sets may be significant, while on small test sets they may not be. section 4
C 3 Did you run computational experiments? section 4
3 C1. Did you report the number of parameters in the models used, the total computational budget (e.g., GPU hours), and computing infrastructure used? section 4
The Responsible NLP Checklist used at ACL 2023 is adopted from NAACL 2022, with the addition of a question on AI writing assistance.
3 C2. Did you discuss the experimental setup, including hyperparameter search and best-found hyperparameter values? section 4
3 C3. Did you report descriptive statistics about your results (e.g., error bars around results, summary statistics from sets of experiments), and is it transparent whether you are reporting the max, mean, etc. or just a single run? section 4
3 C4. If you used existing packages (e.g., for preprocessing, for normalization, or for evaluation), did you report the implementation, model, and parameter settings used (e.g., NLTK, Spacy, ROUGE, etc.)? section 3,4
D 3 Did you use human annotators (e.g., crowdworkers) or research with human participants? Left blank.
3 D1. Did you report the full text of instructions given to participants, including e.g., screenshots, disclaimers of any risks to participants or annotators, etc.? section 5
3 D2. Did you report information about how you recruited (e.g., crowdsourcing platform, students) and paid participants, and discuss if such payment is adequate given the participants’ demographic (e.g., country of residence)? section 5
D3. Did you discuss whether and how consent was obtained from people whose data you’re using/curating? For example, if you collected data via crowdsourcing, did your instructions to crowdworkers explain how the data would be used? Not applicable. Left blank.
D4. Was the data collection protocol approved (or determined exempt) by an ethics review board? Not applicable. Left blank.
3 D5. Did you report the basic demographic and geographic characteristics of the annotator population that is the source of the data? Left blank.

heading: Future Work
text: We hope our method and analysis can inspire future research on this task. Limitations
We synthesize negative instances by substituting relational phrases with misleading tokens. However, the relational semantics in some instances may be expressed implicitly. That is, there are no key tokens that directly correspond to the target relation. Therefore, we cannot synthesize negative instances based on these instances. Additionally, we consider substitution ratio ϵ as a fixed hyperparameter. It may be a better choice to dynamically determine ϵ based on the input instance. We leave these limitations as our future work