De-biasing Distantly Supervised Named Entity Recognition via Causal Intervention (2021)

Abstract: Distant supervision tackles the data bottleneck in NER by automatically generating training instances via dictionary matching. Unfortunately, the learning of DS-NER is severely dictionary-biased, which suffers from spurious correlations and therefore undermines the effectiveness and the robustness of the learned models. In this paper, we fundamentally explain the dictionary bias via a Structural Causal Model (SCM), categorize the bias into intra-dictionary and inter-dictionary biases, and identify their causes. Based on the SCM, we learn de-biased DS-NER via causal interventions. For intra-dictionary bias, we conduct backdoor adjustment to remove the spurious correlations introduced by the dictionary confounder. For inter-dictionary bias, we propose a causal invariance regularizer which will make DS-NER models more robust to the perturbation of dictionaries. Experiments on four datasets and three DS-NER models show that our method can significantly improve the performance of DS-NER.

## introduction named entity recognition (ner) aims to identify text spans pertaining to specific semantic types, which is a fundamental task of information extraction, and enables various downstream applications such as relation extraction @xcite and question answering @xcite . the past several years have witnessed the remarkable success of supervised ner methods using neural networks @xcite @xcite , which can automatically extract effective features from data and conduct ner in an end-to-end manner. unfortunately, supervised methods rely on high-quality labeled data, which is very labor-intensive, and thus severely restricts figure 1: dictionary bias in ds-ner happens both at intra-dictionary and inter-dictionary aspects: (a) averaged likehoods of mentions in/not in the dictionary significantly diverge. (b) mention likehoods of the models using different dictionaries significantly diverge. the application of current ner models. to resolve the data bottleneck, a promising approach is distant supervision based ner (ds-ner). ds-ner automatically generates training data by matching entities in easily-obtained dictionaries with plain texts. then this distantly-labeled data is used to train ner models, commonly be accompanied by a denoising step. ds-ner significantly reduces the annotation cost for building an effective ner model, and therefore has attracted great attention in recent years @xcite @xcite @xcite . however, the learning of ds-ner is dictionarybiased, which severely harms the generalization and the robustness of the learned ds-ner models. specifically, entity dictionaries are often incomplete (missing entities), noisy (containing wrong entities), and ambiguous (a name can be of different entity types, such as washington). and ds will generate positively-labeled instances from the indictionary names but ignore all other names. such a biased dataset will inevitably mislead the learned models to overfit in-dictionary names and underfit out-of-dictionary names. we refer to this as intradictionary bias. to illustrate this bias, figure 1 (a) shows the predicting likelihood of a representait can be roughly divided into two parts: distant supervision (ds) and ner. from the scm, we identify that the intra-dictionary bias stems from the spurious correlations caused by backdoor paths, while the inter-dictionary bias stems from the over-fit on the dictionary characteristics. detailed explanations can be found in section 2. in this paper, we propose a causal framework to fundamentally explain and resolve the dictionary bias problem in ds-ner. we first formulate the procedure of ds-ner from the causal view with a structural causal model (scm) @xcite , which is shown in the left part of figure 2 . from the scm, we identified that the intra-dictionary bias stemming from the dictionary which serves as a confounder during the model learning. the dictionary confounder will introduce two backdoor paths, one from positively-labeled instances (x p ) to entity labels (y ) and the other from negatively-labeled instances (x n ) to entity labels. these backdoor paths introduce spurious correlations during learn-ing, therefore result in the intra-dictionary bias. furthermore, the current learning criteria of ds-ner models is to optimize over the correlations between the instances (x) and entity types (y ) given one specific dictionary (d), namely p (y |x, d). such criteria, however, diverges from the primary goal of learning a dictionary-free ner model (i.e., p (y |x)), and results in the inter-dictionary bias. based on the above analysis, unbiased ds-ner should remove the spurious correlations introduced by backdoor paths and capture the true dictionaryfree causal relations. to this end, we conduct causal interventions to de-bias ds-ner from the biased dictionary. for intra-dictionary bias, we intervene on the positive instances and the negative instances to block the backdoor paths in scm, then the spurious correlations introduced by dictionary confounder will be removed. specifically, we conduct backdoor adjustment to learn de-biased ds-ner models, i.e., we optimize the ds-ner model based on the causal distribution, rather than from the spurious correlation distribution. for inter-dictionary bias, we propose to leverage causal invariance regularizer @xcite , which will make the learned representation more robust to the perturbation of dictionaries. for each instance in the training data, causal invariance regularizer will preserve the underlying causal effects unchanged across different dictionaries. the proposed method is modelfree, which can be used to resolve the dictionary bias in different ds-ner models by being applied as a plug-in during model training. we conducted experiments on four standard ds-ner datasets: conll2003, twitter2005, webpage, and wikigold. experiments on three stateof-the-art ds-ner models show that the proposed de-biasing method can effectively solve both intradictionary and inter-dictionary biases, and therefore significantly improve the performance and the robustness of ds-ner in almost all settings. generally, the main contributions of this paper are: ## a causal view on ds-ner in this section, we formulate ds-ner with a structural causal model (scm), then identify the causes of both intra-dictionary bias and inter-dictionary bias using the scm. an scm captures the causal effect between different variables and describes the generative process of a causal distribution, which can be visually presented using a directed acyclic graph (dag). in scm, each node represents a random variable, and a directed edge represents a direct causal relationship between two variables. based on scm, the confounders and backdoor paths @xcite can be identified. in the following, we will describe the causal view of ds-ner and then identify the dictionary bias. ## de-biasing ds-ner via causal intervention in this section, we describe how to de-bias ds-ner. specifically, for intra-dictionary bias, we propose to use backdoor adjustment to block the backdoor paths. for inter-dictionary bias, we design a causal invariance regularizer to capture the dictionary-invariant evidence for ner. ## conclusions this paper proposes to identify and resolve the dictionary bias in ds-ner via causal intervention. specifically, we first formulate ds-ner using a structural causal model, then identity the causes of both intra-dictionary and inter-dictionary biases, finally de-bias ds-ner via backdoor adjustment and causal invariance regularizer. experiments on four datasets and three representative ds-ner models verified the effectiveness and the robustness of our method.