NS -Hunter: BERT -Cloze Based Semantic Denoising for Distantly Supervised Relation Classification (2021)

Abstract: Distant supervision can generate large-scale relation classification data quickly and economi-cally. However a great number of noise sentences are introduced which can not express their labeled relations. By means of pre-trained language model BERT’s powerful function in this paper we propose a BERT-based semantic denoising approach for distantly supervised relation classification. In detail we define an entity pair as a source entity and a target entity. For the specific sentences whose target entities in BERT-vocabulary (one-token word) we present the differences of dependency between two entities for noise and non-noise sentences. For general sentences whose target entity is multi-token word we further present the differences of last hid-den states of [MASK]-entity (MASK-lhs for short) in BERT for noise and non-noise sentences. We regard the dependency and MASK-lhs in BERT as two semantic features of sentences. With BERT we capture the dependency feature to discriminate specific sentences first then capturethe MASK-lhs feature to denoise distant supervision datasets. We propose NS-Hunter a noveldenoising model which leverages BERT-cloze ability to capture the two semantic features andintegrates above functions. According to the experiment on NYT data our NS-Hunter modelachieves the best results in distant supervision denoising and sentence-level relation classification. Keywords: Distant supervision relation classification semantic denoisingIntroduction

## introduction relation classification (rc) is a fundamental task in natural language processing. the goal of rc @xcite is to identify the relation type in a sentence for a given entity pair, which is particularly important for the construction of knowledge bases. in recent years, deep learning has performed very well in relation extraction, but the technique needs a great number of labeled data, which is very expensive for manual tagging. in order to obtain a large amount of labeled rc data quickly and cheaply, distant supervision (ds) @xcite was proposed to automatically generate data by aligning a knowledge base with an unlabeled corpus. it is built on a weak assumption that if an entity pair has a relation in a knowledge base, all the sentences containing this entity pair will express the corresponding relation and exist in the dataset as a bag @xcite . based on such an assumption, a large number of noise sentences are generated by ds because many sentences can not express their labeled relation in fact. for example, in figure 1 , sentence s-2 can not express the labeled relation "founder". these noise sentences such as s-2 will cause error propagation and may significantly reduce the performance of rc model. since the pre-trained language model bert was put forward @xcite , it has performed very well in the fully supervised rc datasets such as semeval 2010 task 8 @xcite . different from ds, it is manually labeled, so there is no noise-sentences in it. following these two works, even if we only keep two entities and delete the other parts of the sentence in test set, we can still get the f1-value of 49.99% (89.2% in mtb @xcite ) in 19-class fully supervised rc task. only keeping entity pair in every sentence of test set can get such a high f1-value, which shows that bert-based rc model will pay more attention to the entity pair itself rather than other words of c c l 2 0 2 1 the sentence. so, the model can not effectively discriminate noise and non-noise sentences, because in a bag generated by ds-method, noise and non-noise sentences have the same entity pair. therefore, this method can not be directly used in sentence-level ds-rc. predicting masked word is one of the two pre-training tasks of bert. cui et al. @xcite ) showed that there is a great deal of commonsense knowledge in bert. after our verification, bert can predict most blanks in general texts, which enables us to identify the noise-sentences in ds dataset. in this paper, we propose ns-hunter, a novel denoising model for ds-rc, which leverages bertcloze to capture semantic features of noise sentences in ds dataset (short for noise sentence hunter). we define the entity pair in each sentence as a source entity and a target entity, and the rest of the sentence as relation pattern. here the source entity is known, but the target entity needs to be predicted which may be head entity or tail entity. our ns-hunter is based on the following assumption: in a nonnoise sentence, the correct prediction of the target entity requires the both attendance of source entity and relation pattern. the assumption restricts the dependency of the source entity and target entity based on the relation pattern marked, which means for a sentence, if the target entity can be predicted only based on one of the either source entity or relation pattern, it will be regarded as a noise sentence. we regarded the dependency of the source entity and the target entity as the first semantic feature for detecting noise sentence in ds datasets. transformed sentences (with masked entity) s1-1 s1-2 jobs was the co-founder and ceo of [mask]. [mask] was the co-founder and ceo of [mask]. s2-1 s2-2 during jobs' tenure, [mask] released four iphones. during [mask]' tenure, [mask] released four iphones. apple [unk] apple apple bert bert-based results predicting according to the function of bert, if a word in a sentence was replaced by @xcite and then the revised sentence was input to bert, bert can make a reasonable prediction for the word at the position of @xcite according to other words in the sentence. to explain the rationality of our assumption, we transformed the s-1 in figure 1 into s1-1 and s1-2 in figure 2 , and also the s-2 in figure 1 into s2-1 and s2-2 in figure 2 . for s1 in figure 1 , when we do not mask jobs, i.e. s1-1 in figure 2 , bert can predict apple. when we mask jobs with [mask], i.e. s1-2 in figure 2 , bert can not predict apple. that means jobs and apple have closely related dependency in s-1, so we think s-1 can express the labeled relation. we use the same method to judge s-2, whether we mask jobs (i.e. s2-2) or not (i.e. s2-1), bert can predict apple, which shows that in s-2, jobs and apple are loosely related, so we think s-2 can not express the labeled relation, and it is a noise sentence. it is based on the first semantic feature, i.e. dependency feature, we can recognize a part of noise-sentences. however, this method is only applicable to the sentences whose target entities in bert-vocabulary (one-token word). in order to make our model can deal with multi-token word, we build some noise reducers, which are actually binary classifiers based on bert. their training sets and development sets are from the part that can be discriminated in the original training sets with our dependency feature, while their test sets are the whole original training set, so we can denoise the original training set for rc. for s2-1 and s2-2 in figure 2 , the prediction results of [mask] are all apple, but the prediction c c l 2 0 2 1 results of @xcite in s1-1 and s1-2 are different. so, we consider that there are semantic difference in the [mask]-represention between noise and non-noise sentences, which can be captured by fullyconnected layer. for the reason, we do not use the commonly used [cls] feature, but concatenate last hidden states of @xcite in transformed sentences as the feature (we call it as mask-lhs), and regard mask-lhs as the second semantic feature. we utilize mask-lhs feature to denoise general sentences. to the best of our knowledge, this is the first work of presenting semantic feature differences between noise and non-noise sentences, and implementing semantic denoising based on the features. we use the sentence-level test set in arnor tagged by jia et al. @xcite from nyt dataset for sentencelevel evaluation of our ns-hunter. overall, our contributions can be summarized as follows: neural network based models have performed very well in rc @xcite . however, training effective neural classifiers requires a large amount of labeled data, which is usually hard to obtain. ds @xcite provides a way to create massive weakly labeled data for rc. many studies train rc model in ds by applying multi-instance learning (mil) to reduce the impact of noise-sentences @xcite @xcite , which relaxes the label of each instance to a bag of sentences containing the same entity pair. some mil-based studies introduce adversarial training @xcite . mil assumes at least one sentence in a bag was labeled correctly. when all sentences in a bag are noise-sentences, mil still suffers from noise @xcite . moreover, these mil-based approaches are designed and tested for a pair of entities @xcite , and they are not suitable for sentence-level rc. alternatively, some studies evaluate and select training instances individually without relying on the at-least-one assumption @xcite @xcite . they usually rely on the classification effect to denoise. it is difficult to measure the denoising ability alone, and some of them need the pure data labeled manually @xcite . our approach uses the commonsense knowledge in the pre-training language model (plm) to measure whether the two entity pairs are closely related, does not rely on assumption of mil and needs no manually labeled data. our denoising model is independent of classification process. it is plug and play, so can be used with all the above models. recently, more and more plms have adopted the method of predicting masked word to learn grammar and semantics, such as roberta @xcite , electra @xcite , ernie @xcite etc. with more and more parameters and larger corpus for training, it can be expected that their cloze-accuracy will be higher and higher and the ds-rc model based on these plms will have stronger denoising ability. moreover, casrel @xcite is bert-based rc model, they extract entities and relations jointly. this model use ds datasets, but the sentences labeled as na were deleted. so, we can not compare the denoising ability of ns-hunter with it. ## model in this paper, we propose ns-hunter, a novel bert-based denoising model for ds-rc shown as figure 3 . for a ds dataset with n classes and a na (no relation) class, we construct a noise reducer with c c l 2 0 2 1 bert for every class except the na. finally, the denoised training set is composed of n classes after denoising and a na class in the original training set. in figure 3 , the left is the overall framework and the right is the detail of training k-th noise reducer for class k (1≤@xmath0. rc classifier new training set of all classes except na class 1 training class k training class n training the detail of class k original training set of class k sentences after graded noise reducer new training set of class k noise sentences that its entity in bert's vocab non-noise noise original training set of class na + ''dependency'' ''mask-lhs'' figure 3 : ns-hunter model. for each class, the lighter color on top means dataset that mixed with noise sentences and non-noise sentences, the gradual color in the middle means the sorted sentences, and the dark color below means the non-noise sentences we extracted. we do the same operation for each class and take the non-noise sentences of each class and the sentences labeled na in the original dataset as the new training set. ns-hunter consists of three parts (see ( 1 )∼(3) on the left side of figure 3 ). the first part is to discriminate the sentences whose target entities in bert-vocabulary, then we get n small-scale pure binary datasets, which is labeled as noise and non-noise. the second part is to train noise reducers with n small-scale datasets. in this part, we design a novel feature mask-lhs that can effectively capture the semantic differences between noise and non-noise sentences. it should be noted that the first two parts are for the single relation class, that is why we train a noise reducer for each class. the third part is the application of the denoised training set. after denoising, we get a training set with the same form as the original training set, but more pure. we introduce each part in section 3.1∼section 3.3, and finally give the algorithm description of ns-hunter. ## conclusion after carefully observing the rc dataset generated by the ds method, we present the two semantic features, i.e. dependency and mask-lhs feature, and propose a bert-based denoising model ns-hunter and a denoising approach based on the two semantic features for ds-rc. we present the dependency feature of the entity pair and use bert-cloze to discriminate some specific sentences with bert-vocabulary based on the dependency feature, which has strong interpretability. for general sentences generated by the ds method, we designed a novel mask-lhs feature to capture the semantic differences between noise and non-noise sentences for denoising. the performance of ns-hunter is better than several other denoising baselines based on cnn, pcnn and bilstm. significant improvements have been made in denoising and rc task. our denoising method can also be easily combined with other rc methods. because we train noise reducers for each class, even if the knowledge is updated (such as the president of a country changes), our noise reducers are still robust according to the existing similar relation patterns in the training set. however, our model may not perform well in some professional domain such as biological, because bert generates language representation from general corpus and lacks domainspecific knowledge @xcite . it may get better results if we continue to pre-training bert in a large scale of professional texts before applying our denoising model @xcite .