Unexpected Phenomenon: LLM s’ Spurious Associations in Information Extraction (2024)

Abstract: Information extraction plays a critical role in natural language processing. When applying large language models (LLMs) to this domain, we discover an unexpected phenomenon: LLMs’ spurious associations. In tasks such as relation extraction, LLMs can accurately identify entity pairs, even if the given relation (label) is semantically unrelated to the pre-defined original one. To find these labels, we design two strategies in this study, including forward label extension and backward label validation. We also leverage the extended labels to improve model performance. Our comprehensive experiments show that spurious associations occur consistently in both Chinese and English datasets across various LLM sizes. Moreover, the use of extended labels significantly enhances LLM performance in information extraction tasks. Remarkably, there is a performance increase of 9.55%, 11.42%, and 21.27% in F1 scores on the SciERC, ACE05, and DuEE datasets, respectively.

## introduction information extraction (ie) plays a vital role in natural language processing (nlp), aiming to extract pre-defined types of information from unstructured text sources. typical tasks in ie include relation extraction (re) @xcite , named entity recognition (ner) @xcite , and event detection (ed) @xcite . despite its importance, ie often faces obstacles in limited-data scenarios, such as zero-shot or few-shot settings, where traditional models struggle to achieve effective performance @xcite . phenomenon definition. recently, large language models (llms) like chatgpt foot_1 have emerged as a fundamental backbone in the field of nlp. their remarkable capability lies in achieving impressive performance without parameter tuning, relying instead on a limited number of example instructions. hence, we also explore their potential in ie tasks. in this study, we uniformly define the ie task as the prediction of a-b pairs for the given textual data. "a" represents a pre-defined type label, while "b" refers to a single or multiple spans extracted directly from the text. specifically, these pairs in re, ner, and ed take the forms of relation-(head entity, tail entity), entity type-entity span, and event type-event trigger, respectively. during our exploration, we discover an intriguing phenomenon, i.e., llms' spurious associations. this phenomenon reveals that the models are capable of correctly predict the answer "b" even when confronted with a different "a ′ " that is semantically unrelated to the original type label "a". as illustrated in figure 1 , the model successfully identifies the entity pair (weak duration constraints, hmms) ("b"), even when provided with another relation like limitations of ("a ′ "), which is semantically unrelated to feature of ("a"). this phenomenon is not limited to re but is also observable in ner and ed tasks, as shown in figure 1 . phenomenon origins. we take re as an example to describe the process of discovering spurious association phenomena. we first feed a sentence and a pre-defined set of relations to chatgpt, and ask the model to generate triplets in the form of (head entity, relation, tail entity). both entities are derived from the provided sentence, and the relation originates from the relation set. when examining the error results, we observe that a significant portion of inaccuracies stems from generated relations that do not align with the pre-defined set. this is due to chatgpt's generative nature, which sometimes generates relations that differ significantly in semantics from the intended original relations. furthermore, in our attempts to employ these generated semantically unrelated relations for identifying the head and tail entities in other sentences with the original relation, we find that the large model can utilize these relations to effectively extract the correct head-tail entity pairs. ## related work related work of applying llms to ie tasks @xcite @xcite can be roughly divided into four categories: 1) directly employing llms for inference, 2) incorporating llms and small language models (slms), 3) leveraging slms with knowledge distilled from llms, and 4) utilizing llms with instruction tuning. the first branch is to directly employ llms for inference @xcite . typical works along this line include chatie @xcite and chatee @xcite . for example, chatie transforms the zero-shot ie task into a multi-turn question-answering problem with a twostage framework. in this framework, the method is designed to first determine relations, entity types, or event types, and then to extract the corresponding entity pairs, entity spans, or triggers from the given text. the second branch is to incorporate llms and slms for the ie tasks. for instance, the filter-then-rerank @xcite method is proposed, employing slms as filters and llms as extended relation (r') valid target entity pair valid text + d' extended relations ('') valid text + e' train text + trigger () re: train text + entity span (s) ner: entity extended entity type (e') ed: extended event type (d') forward label extension valid text + r' valid target entity span valid target trigger extended entity types extended event types (' rerankers. this is achieved by prompting llms to rerank a small subset of challenging samples identified by slms. the third branch is to use slms with knowledge distilled from llms for the tasks. this type of method regards llms as annotators and generates abundant samples with (pseudo) labels. then, slms are trained using augmented data to achieve superior performance @xcite . the fourth branch is to use llms based on supervised instruction tuning. for example, instruc-tuie @xcite ) is a multi-task learning framework for universal ie which enables the use of human-readable instructions to guide llms for ie tasks. ## study design in this section, we elaborate on the spurious association phenomenon in llms, including its definition, origins, and application. ## experiments we conduct extensive experiments to demonstrate the phenomenon of llms' spurious associations in ie tasks. then, we leverage this phenomenon to improve model performance in these tasks. ## conclusion in this paper, we observe an intriguing phenomenon: llms' spurious associations, when utilizing the llm-based method for accomplishing ie tasks. to explore this phenomenon, we design two strategies in this study, including forward label extension and backward label validation. moreover, we leverage these extended labels to enhance model performance. following the procedures described, we conduct extensive experiments to validate this intriguing phenomenon of llms with varying parameter sizes. furthermore, we perform experiments on downstream tasks, confirming that the extended labels have a positive impact on all ie sub-tasks.