Efficient Overshadowed Entity Disambiguation by Mitigating Shortcut Learning (2024)

Abstract: Entity disambiguation (ED) is crucial in natural language processing (NLP) for tasks such as question-answering and information extraction. A major challenge in ED is handling overshadowed entities—uncommon entities sharing mention surfaces with common entities. The current approach to enhance performance on these entities involves reasoning over facts in a knowledge base (KB), increasing computational overhead during inference. We argue that the ED performance on overshadowed entities can be enhanced during training by addressing shortcut learning, which does not add computational overhead at inference. We propose a simple yet effective debiasing technique to prevent models from shortcut learning during training. Experiments on a range of ED datasets show that our method achieves state-of-the-art performance without compromising inference speed. Our findings suggest a new research direction for improving entity disambiguation via shortcut learning mitigation.

## introduction entity disambiguation (ed) is an essential task in many natural language processing (nlp) applications, for instance, open-domain question answering @xcite @xcite , fact verification @xcite , and information extraction @xcite . the task is to identify the correct entity recorded in a kb, e.g., wikidata, for each ambiguous entity mention in a given text, which is a crucial capability when performing entity linking (el). in real-world ed applications, there are two important properties: figure 1 : the causal graph of ed models. due to the strong correlations between the spurious feature and training labels, typical ed models are prone to shortcut learning and fail to resolve overshadowed entities. michael jordan is widely regarded as one of the greatest basketball players of all time. intended feature intended feature hine learning. ext n surface n context ted entity 99.06 0.22 0.14 0.07 41421) 65029442) 6831716) q3308285) p(e | x m ) owed by q41421) gold label: q41421 figure 2: the system overview of the proposed method. compute similarity scores between mentions and entity descriptions. refined (ayoola et al., 2022b) enhanced the bi-encoder's performance by incorporating entity type classification and entity priors to re-rank the bi-encoder predictions. nonetheless, ed methods often struggle with overshadowed entities (provatorova et al., 2021), indicating a lack of context-awareness in current ed methods. kbed (ayoola et al., 2022a) improved refined's performance on overshadowed entities by leveraging kb facts. specifically, they extract relations between every pair of mentions in input and perform reasoning over external knowledge retrieved from kb to re-rank the refined's predictions. although this method has the potential to enhance context-awareness and reduce the overshadowing problem, it requires input to contain multiple mentions, and its computational burden grows as the number of mentions increases, hence compromising the scalability of the refined method. according to our empirical results, kbed slows down the throughput of refined from 3.3 to 0.6 queries per second (q/s) on standard ed datasets. this paper tackles the overshadowing issue by addressing shortcut learning @xcite during training, which does not impose a computational burden at inference. we introduce counterfactual training (cft) as a technique to prevent the models from learning shortcut solutions and to enhance context-awareness. as shown in figure 1 , each input text x to ed models contains two input features: the mention surface x m (spurious feature) and the mention context x c (intended feature). the intended solution is to use the contextual feature x c to determine entity e. nevertheless, the strong correlations between the spurious feature x m and training labels can induce the models to learn a shortcut (i.e., using the mention surface to determine entity e), obscuring the intended solution. this shortcut solution allows the models to achieve high performance on common entities but poor performance on overshadowed entities. we assess cft against existing methods on six standard datasets and three challenging datasets. the results show that cft achieves the best performance on seven out of nine datasets for overshadowed entities and six out of nine datasets for overall entities without compromising the throughput at inference. we find that cft performs surprisingly well on texts with limited contextual information (i.e., short sentences with a small number of mentions) while other methods struggle. ## qualitative analysis in failure case 1 reveals the limitations of current ed benchmarking. as shown in ## conclusion this paper addresses the challenge of handling overshadowed entities in entity disambiguation (ed). by formulating the ed problem as shortcut learning mitigation, the spurious correlation between mention surfaces and training labels can be mitigated via cft, which reduces the model's re-liance on surface forms for common entities. as opposed to the current sota (kbed), our solution does not impose additional inference time, making it 5 times faster than kbed. the empirical results show that cft achieves the best performance on overshadowed entities. these results support the new research direction of modeling the entity disambiguation problem with counterfactual learning.