A Causal View of Entity Bias in (Large) Language Models (2023)

Abstract: Entity bias widely affects pretrained (large) language models, causing them to rely on (biased) parametric knowledge to make unfaithful predictions. Although causality-inspired methods have shown great potential to mitigate entity bias, it is hard to precisely estimate the parameters of underlying causal models in practice. The rise of black-box LLMs also makes the situation even worse, because of their inaccessible parameters and uncalibrated logits. To address these problems, we propose a specific structured causal model (SCM) whose parameters are comparatively easier to estimate. Building upon this SCM, we propose causal intervention techniques to mitigate entity bias for both white-box and black-box settings. The proposed causal intervention perturbs the original entity with neighboring entities. This intervention reduces specific biasing information pertaining to the original entity while still preserving sufficient semantic information from similar entities. Under the white-box setting, our training-time intervention improves OOD performance of PLMs on relation extraction (RE) and machine reading comprehension (MRC) by 5.7 points and by 9.1 points, respectively. Under the black-box setting, our in-context intervention effectively reduces the entity-based knowledge conflicts of GPT-3.5, achieving up to 20.5 points of improvement of exact match accuracy on MRC and up to 17.6 points of reduction in memorization ratio on RE.

## introduction entity bias @xcite @xcite @xcite refers to an undesirable phenomenon where models overly rely on prediction shortcuts triggered by specific entities to make spurious predictions. for example, given the sentence "bill gates went to microsoft building 99," models figure 1 : an example of entity bias in gpt-3.5. our in-context intervention mitigates the conflicts between parametric knowledge and contextual knowledge. may be misled by their memory of the entities bill gates and microsoft, saying the relation between them in this context is founder rather than visitor, as shown in fig. 1 . recent studies show that entity bias widely affects pretrained (large) language models (llms; @xcite @xcite . these models have a tendency to disregard contextual information that contradicts or is infrequently reported in the pretrained corpus, while excessively relying on (biased) parametric knowledge @xcite to make unfaithful predictions and perpetuate bias. prior studies have proposed multiple causalityinspired methods to mitigate entity bias @xcite @xcite . 2 despite their potential, the causal models underlying these methods are flawed in practice, primarily because of imprecise parameter estimation. for example, some causal models necessitate estimating the probability distribution over labels when given a sentence that is devoid of entities or contextual information @xcite . these methods either lose predictive information about entities, or are prone to erroneous representation without contextualization. the other critical problem is the difficulty of applying these methods to black-box llms, of which parameters are inaccessible and logits are uncalibrated. to address the aforementioned problems, the first contribution of this paper is a causal analysis of entity bias mitigation methods ( §3.1). we examine and compare the structured causal models (scms) behind existing methods. we find that, among the theoretically equivalent causal models @xcite , there exists a specific scm whose parameters are comparatively easier to estimate. as shown in fig. 2 , the proposed scm only requires to intervene input entities to mitigate the presence of spurious features before passing them to the subsequent neural layers. moreover, it retains the entity type information 3 at an appropriate level of granularity without requiring explicit entity typing. the second contribution of this paper is a training-time causal intervention technique for mitigating entity bias based on the proposed scm ( §3.2). specifically, we identify entities that are likely to share similar predictive information with the given entity. during training, we perturb embedding of the given entity within a convex hull constructed by embeddings of similar entities. during inference, we represent the entity with the center of the convex hull. taking advantage of the continuous nature of the embedding space, this intervention does not rely on models specifically trained on natural language to estimate the label distribution of unnatural text, nor does it sacrifice predictive entity or contextual information. the third contribution of this paper is to transform the training-time intervention into in-context intervention for black-box llms whose parameters are inaccessible, and logits are uncalibrated ( §3.3). a significant advantage of the proposed scm is that the causal intervention is carried out at the input layer, enabling its implementation within an in-context setting. specifically, we replace entities with placeholders and define each placeholder 3 entity type information plays a crucial role in entitydriven tasks. for example, without knowing a more specific location type, it is impossible to differentiate between relations born_in_city and born_in_country. by examples -a set of similar entities. for example, we can replace bill gates in fig. 1 with subject_entity and prepend the prompt, "assume that subject_entity can be any of steve jobs, bill gates, and jeff bezos", to the input. this in-context intervention can be applied to any black-box llm without additional cost. experiments on relation extraction (re) and machine reading comprehension (mrc) show that the proposed causal intervention techniques are effective for both white-box and black-box llms. ## method in this section, we first analyze methods for mitigating entity bias in a causal view and propose an easy-to-estimate scm as a theoretical basis ( §3.1). ## white-box experiments in this section, we evaluate our training-time intervention under the white-box setting. ## black-box experiments in this section, we evaluate our in-context intervention for mitigating entity bias from llms under black-box setting. ## conclusion in this paper, we analyze the entity bias in llms from a causal view. building upon an scm whose parameters are easier to estimate, we propose training-time causal intervention for whitebox llms and in-context causal intervention for black-box llms. both intervention techniques perturb the original entity with neighboring entities to mitigate spurious correlations between specific entities and predictions. experiments on relation extraction and machine reading comprehension show that the proposed intervention can effectively reduce the conflicts between parametric knowledge and contextual knowledge and significantly improve the performance of llms. future work can apply our causal intervention to more llms and tasks to achieve context-faithful answers.