CETA : A Consensus Enhanced Training Approach for Denoising in Distantly Supervised Relation Extraction (2022)

Abstract: Distantly supervised relation extraction aims to extract relational facts from texts but suffers from noisy instances. Existing methods usually select reliable sentences that rely on potential noisy labels, resulting in wrongly selecting many noisy training instances or underutilizing a large amount of valuable training data. This paper proposes a sentence-level DSRE method beyond typical instance selection approaches by preventing samples from falling into the wrong classification space on the feature space. Specifically, a theorem for denoising and the corresponding implementation, named Consensus Enhanced Training Approach (CETA), are proposed in this paper. By training the model with CETA, samples of different classes are separated, and samples of the same class are closely clustered in the feature space. Thus the model can easily establish the robust classification boundary to prevent noisy labels from biasing wrongly labeled samples into the wrong classification space. This process is achieved by enhancing the classification consensus between two discrepant classifiers and does not depend on any potential noisy labels, thus avoiding the above two limitations. Extensive experiments on widely-used benchmarks have demonstrated that CETA significantly outperforms the previous methods and achieves new state-of-the-art results.

## introduction relation extraction (re), which aims to identify the relation between two specific entities in the text, is a fundamental task in natural language processing. most supervised re methods demand large-scale labeled training data, which is difficult to acquire manually. to alleviate the problem, distant supervision (ds) is proposed by @xcite to automatically generate the @xmath0:;<; =>?@, ?>ab, cd;@xmath1 in this section, we start with the learning setup and present the objective function in conjunction with our proposed theorem: the generalization error bound for dsre in subsection 3.1. then, in subsection 3.2, we will introduce the details of our proposed ceta, which aims to implement the proposed theorem for denoising. ## conclusion this paper goes beyond the typical instance selection approaches, and focuses on handling the noisy labels in the feature space. a theorem for denoising and the corresponding implementation, named consensus enhanced training approach (ceta), are proposed in this paper. by training the model with ceta, samples of different classes are separated in the feature space. thus the model can easily establish the robust classification boundary to prevent noisy labels from biasing wrongly labeled samples into the wrong classification space. besides, ceta achieves denoising does not depend on any potential noisy labels. therefore, ceta is not affected by common noisy instances. extensive experiments on the widely-used benchmarks have demonstrated that our proposed ceta outperforms previous methods.