Unexpected Phenomenon: LLMs’ Spurious Associations in Information Extraction (2024)

Abstract: Information extraction plays a critical role in natural language processing. When applying large language models (LLMs) to this domain, we discover an unexpected phenomenon: LLMs’ spurious associations. In tasks such as relation extraction, LLMs can accurately identify entity pairs, even if the given relation (label) is semantically unrelated to the pre-defined original one. To find these labels, we design two strategies in this study, including forward label extension and backward label validation. We also leverage the extended labels to improve model performance. Our comprehensive experiments show that spurious associations occur consistently in both Chinese and English datasets across various LLM sizes. Moreover, the use of extended labels significantly enhances LLM performance in information extraction tasks. Remarkably, there is a performance increase of 9.55%, 11.42%, and 21.27% in F1 scores on the SciERC, ACE05, and DuEE datasets, respectively.1

heading: 1 Introduction
text: Information Extraction (IE) plays a vital role in natural language processing (NLP), aiming to extract pre-defined types of information from unstructured text sources. Typical tasks in IE include Relation Extraction (RE) (Shang et al., 2022), Named Entity Recognition (NER) (Li et al., 2022), and Event Detection (ED) (Xie and Tu, 2022). Despite its importance, IE often faces obstacles in limited-data scenarios, such as zero-shot or few-shot settings, where traditional models struggle to achieve effective performance (Agrawal et al., 2022).
Phenomenon Definition. Recently, Large Language Models (LLMs) like ChatGPT2 have emerged as a fundamental backbone in the field of
∗Corresponding authors. 1The codes are publicly available at https://github.
com/TreMila/SaIE 2https://chat.openai.com/
Target D:
NLP. Their remarkable capability lies in achieving impressive performance without parameter tuning, relying instead on a limited number of example instructions. Hence, we also explore their potential in IE tasks. In this study, we uniformly define the IE task as the prediction of A-B pairs for the given textual data. “A” represents a pre-defined type label, while “B” refers to a single or multiple spans extracted directly from the text. Specifically, these pairs in RE, NER, and ED take the forms of relation-(head entity, tail entity), entity type-entity span, and event type-event trigger, respectively. During our exploration, we discover an intriguing phenomenon, i.e., LLMs’ spurious associations. This phenomenon reveals that the models are capable of correctly predict the answer “B” even when confronted with a different “A′” that is semantically unrelated to the original type label “A”. As illustrated in Figure 1, the model successfully identifies the entity pair (weak duration constraints, HMMs) (“B”), even when provided with another relation like limitations of (“A′”), which is semantically unrelated to feature of
9176
(“A”). This phenomenon is not limited to RE but is also observable in NER and ED tasks, as shown in Figure 1.
Phenomenon Origins. We take RE as an example to describe the process of discovering spurious association phenomena. We first feed a sentence and a pre-defined set of relations to ChatGPT, and ask the model to generate triplets in the form of (head entity, relation, tail entity). Both entities are derived from the provided sentence, and the relation originates from the relation set. When examining the error results, we observe that a significant portion of inaccuracies stems from generated relations that do not align with the pre-defined set. This is due to ChatGPT’s generative nature, which sometimes generates relations that differ significantly in semantics from the intended original relations. Furthermore, in our attempts to employ these generated semantically unrelated relations for identifying the head and tail entities in other sentences with the original relation, we find that the large model can utilize these relations to effectively extract the correct head-tail entity pairs.
Phenomenon Application. We utilize the spurious association phenomenon to enhance the model performance in IE tasks. We still consider RE as an illustrative case. First, we select the Top-K (K=1 in the experiments) extended relations based on the highest F1 scores on the verification dataset from those semantically unrelated to the original relation, yet capable of accurately identifying the correct head and tail entity pairs. Then, we integrate them with all pre-defined original relations to create a new set of relations. This augmented set, along with each test sample, is then fed into the model. To facilitate the extraction process, we design Chain-of-Thought (CoT) prompts that guide the model to extract triplets from the text. An improvement in the quality of the extracted triplets, compared to those obtained without incorporating the extended relations, confirms the positive impact of spurious association on model performance.
To investigate the aforementioned intriguing phenomenon, we conduct a comprehensive set of experiments using LLMs of varying parameter sizes: ChatGLM (6B) (Du et al., 2022), BaiChuan (13B) (Yang et al., 2023), Alpaca (33B) (Taori et al., 2023), LLaMA-2 (70B) (Touvron et al., 2023), ChatGPT, and GPT-4 (OpenAI, 2023). These experiments contain diverse IE tasks: RE, NER, and ED, and are conducted on datasets in both Chinese and English languages. After experimental
analysis, several significant conclusions have been drawn:
• Finding 1: Regardless of the size of LLMs, spurious associations occur in both Chinese and English datasets across the RE, NER, and ED tasks.
• Finding 2: The phenomenon of LLMs’ spurious associations is more pronounced in IE tasks. Despite over 60% of extended labels differing from original labels, the model accurately predicts entity pairs, spans, or triggers associated with original labels using these extended labels.
• Finding 3: The semantic representations of labels in spurious associations are closer to those of the original labels compared to other extended labels.
• Finding 4: Extended labels prove to be valuable for enhancing the LLMs’ performance on IE tasks. Notably, the model performance has improved by 9.55%, 11.42%, and 21.27% in terms of F1 scores on the SciERC (RE task), ACE05 (NER task), and DuEE (ED task) datasets, respectively.

heading: 2 Related Work
text: Related work of applying LLMs to IE tasks (Yu et al., 2023; Zhao et al., 2023; Wang et al., 2022; Xu et al., 2023) can be roughly divided into four categories: 1) directly employing LLMs for inference, 2) incorporating LLMs and small language models (SLMs), 3) leveraging SLMs with knowledge distilled from LLMs, and 4) utilizing LLMs with instruction tuning.
The first branch is to directly employ LLMs for inference (Min et al., 2022). Typical works along this line include ChatIE (Wei et al., 2023) and ChatEE (Gao et al., 2023). For example, ChatIE transforms the zero-shot IE task into a multi-turn question-answering problem with a twostage framework. In this framework, the method is designed to first determine relations, entity types, or event types, and then to extract the corresponding entity pairs, entity spans, or triggers from the given text. The second branch is to incorporate LLMs and SLMs for the IE tasks. For instance, the filter-then-rerank (Ma et al., 2023) method is proposed, employing SLMs as filters and LLMs as
extended relation (r’)
valid target entity pair
valid text + d’
extended relations (’’)
valid text + e’
train text + trigger ()
RE:
train text + entity span (s)NER:
Entityextended entity type (e’)
ED: extendedevent type (d’)
Forward Label Extension
valid text + r’
valid target entity span
valid target trigger
extended entity types
extended event types (’
outputinput
new
outputinput
(&)
($!)
'")
("
)"
$"!"
""
#"
%"
&"
Backward Label Validation Application of Extended Labels
new
+ test text
+ test text
+ test text +
'+
(
)+
filter
Triplet (ℎ, ', $)
((, &)
Event (), $!)
Phenomenon Origins Phenomenon Application
train text + entity pair (h, (ℎ, $)
Figure 2: Our study framework is designed for the RE, NER, and ED tasks, covering both spurious association phenomenon origins and application. Taking the RE task as an example, the process begins with phenomenon origins. Given the text and entity pair in the training set, we first perform the forward label extension, generating an extended relation set. Then, we move to the backward label validation step, which involves the selection of accurate extended relations capable of extracting the entity pair aligned with the target relation within the validation set. Finally, in terms of phenomenon application, we use the refined extended relation set to enhance the LLMs’ performance on the test set of the RE task.
rerankers. This is achieved by prompting LLMs to rerank a small subset of challenging samples identified by SLMs. The third branch is to use SLMs with knowledge distilled from LLMs for the tasks. This type of method regards LLMs as annotators and generates abundant samples with (pseudo) labels. Then, SLMs are trained using augmented data to achieve superior performance (Josifoski et al., 2023). The fourth branch is to use LLMs based on supervised instruction tuning. For example, InstructUIE (Wang et al., 2023) is a multi-task learning framework for universal IE which enables the use of human-readable instructions to guide LLMs for IE tasks.
In summary, the prevailing tendency is to employ large models for IE tasks, yet there remains room for performance enhancement. In this paper, we unveil an intriguing phenomenon, i.e., the spurious associations of LLMs, and leverage this discovery to improve the model’s performance on IE tasks.

heading: 3 Study Design
text: In this section, we elaborate on the spurious association phenomenon in LLMs, including its definition, origins, and application.

heading: 3.1 Phenomenon Definition
text: Definition 1: RE task. Given a text Cr = [c1, c2, . . . , cnr ] and the pre-defined relation types R = {r1, r2, . . . , rmr}, where nr denotes the number of tokens in Cr and mr is the number of relations in R, RE task aims to obtain a triplet set T = {(h, r, t)}m from Cr, where m is the number
of extracted triplets and r represents the relation between the head entity h and tail entity t.
Definition 2: NER task. Given a text Ce = [c1, c2, . . . , cne ] and the pre-defined entity types E = {e1, e2, . . . , eme}, NER task aims to detect the mention spans S = {s1, s2, . . . , swe} from Ce and the entity type e ∈ E (e.g., PERSON, LOCATION, etc) for each extracted span.
Definition 3: ED task. Given a text Cd = [c1, c2, . . . , cnd ] and the pre-defined event types D = {d1, d2, . . . , dmd}, ED task aims to identify the event trigger td for Cd and the event type d ∈ D of td.
Definition 4: Spurious associations. In RE, for a training sample with Cr and (h, r, t), LLMs would predict (h, t) based on Cr and r′, even if r′ is semantically unrelated to r. Similarly, in NER, with a sample containing Ce, s, and e, LLMs would predict s using Ce and e′, even when there is no semantic connection between e′ and e. In ED, when considering a sample comprising Cd, td, and d, LLMs would predict td given Cd and d′, even if d′ lacks semantic relevance to d.

heading: 3.2 Phenomenon Origins
text: As illustrated in Figure 2, we describe the origins of the spurious association phenomenon for three tasks: RE, NER, and ED. Since the phenomenon exhibits a uniform pattern across these tasks, we take RE as an example to detail the process, which is structured into two steps: 1) Forward label extension (on the training set), utilizing an LLM to extend the pre-defined original relations, and 2) Backward label validation (on the validation set),
The prompt of forward relation extension
Instruction
Expected output
Output : [LIMITATIONS-OF]
Demonstrations
Role definition % The prompt of forward relation extension
% instruction.
Your objective is to generate relation types for subject-object pairs within a given text, adhering to the following rules: 1. Utilize subject-object pairs, e.g., (noisy-channel architecture, transformation model). 2. Incorporate the context of the given text. 3. Generate relation types.
You are currently a senior relation generation expert.
Your objective is to generate relation types for head-tail entity pairs within a given text, adhering to the following rules: 1. Utilize head-tail entity pairs, e.g., (noisy-channel architecture, transformation model). 2. Incorporate the context of the given text. 3. Generate relation types.
The following is an example of instruction to help you understand the task: Input: "We present a text mining method for finding synonymous expressions based on the distributional hypothesis in a set of coherent corpora. "Head-tail entity pair: (text mining method, synonymous expressions) " Output: [USED-FOR]
Input: "Conventional HMMs have weak duration constraints." Head-tail entity pair: (weak duration constraints, HMMs)"
Figure 3: A prompt example in forward relation extension.
selecting the extended relations that can assist the model in precisely identifying head-tail entity pairs corresponding to the original relations.

heading: 3.2.1 Forward Label Extension
text: This step is designed to extend a new relation set R′r for each r ∈ R. Specifically, we select all samples with the original relation r from the training set. For each sample, we concatenate the sentence with its corresponding head-tail entity pair. This combination then serves as the input for the LLM, which is tasked with outputting a semantic relation (or “Na” if no relation is found) between the head-tail entity pair. To further enhance the model’s potential, we incorporate role definitions, instructions, and demonstrations. An illustrative example of this process is provided in Figure 3. Through the above process, we obtain the extended set R′r for the specific relation r. Notably, the relations extended by different r ∈ R may be the same, such as r1 → r′ and r2 → r′. If these identical relations emerge within the set R′ from which all relations are extended, it becomes challenging to ascertain their original corresponding relations. Hence, we eliminate these duplicate relations that appear across various extension sets for every r ∈ R, ensuring distinctiveness among the sets.

heading: 3.2.2 Backward Label Validation
text: In this step, we aim to evaluate the validity of each extended relation r′ ∈ R′r derived from the original relation r. Specifically, we select all samples associated with r from the validation set. For each r′, we concatenate the sentence from each selected
The prompt of backward relation validation
Instruction
Expected output Output : [LIMITATIONS-OF]
CoT Demonstrations
Role definition
Your objective is to extract head-tail entity pairs that correspond to the specified relation , adhering to the following rules: 1.Utilize the provided relation to merge the context of the given text. Extract subject-object pairs that could potentially exhibit the specified relation. 2.Construct evaluative sentences based on the relation and relevant keywords. Verify the accuracy of each evaluative sentence, outputting only "yes" or "no." 3.Create a catalog of triplets based on the judgment sentences, comprising (head entity, relation, tail entity). The relation must conform to the designated relation.
provide a text and a relation, subsequently
Here's an illustrative train of thought designed to guide your step-by-step understanding of the task:
Here's an example thought process that will guide you step by step in understanding the task:
You are currently a senior expert in relation extraction.
You objective is to extract head-tail entity pairs given a text and the specified relation, adhering to the following rules: 1. Utilize the context of the given text with the specified relation to extract head-tail entity pairs that represent the relation. 2. Verify the yes-no question sentence formed by concatenating the specified relation and the extracted head-tail entity pairs, and output only "yes" or "no". 3. Generate a list of triplet (head entity, relation, tail entity) based on the answer of yes-no question, where the relation must be the specified relation.
The following is an example of a chain of thought that guides you step by step to generate the results. Input: "We propose and analyze a block minimization framework for data larger than the memory size." Relation: COMPARE
Head-tail entity pair: [(data, memory size)] Answer: Is the relation between “data” and “memory size” "COMPARE"? Yes Generate a list of triplet: `` (data, COMPARE, memory size) `` Input: "Synchronous dependency insertion grammars are a version of synchronous grammars defined on dependency trees." Relation: LIMITATIONS-OF
Figure 4: An example of the CoT prompt in backward relation validation.
sample with r′ to form the input of the LLM and ask the model to generate the head-tail entity pair3. The model’s response is considered correct if its outputs exactly match the ground truth pairs. Accuracy is defined as the proportion of consistent responses to the total count of ground truth pairs. With this approach, we compute the F1 score for the extended relation r′ over all samples related to r. If the F1 score is zero, the extended relation r′ is removed from R′r. Otherwise, we retain it. Notably, in the LLM’s input, in addition to the sentence and extended relation, we introduce the CoT process in the demonstration, as illustrated in Figure 4. That is, we first ask the model to produce the head-tail entity pair. Then, we integrate this output with the extended relation and request the model to assess whether these two entities exhibit this extended relation. The model is expected to generate results that align with real-world facts.

heading: 3.3 Phenomenon Application
text: To verify the impact of the extended relations, we incorporate them to enhance model performance on the RE task. Specifically, for each r ∈ R, we first select the Top-k extended relations from R′r according to the F1 scores obtained in the validation
3Despite our requirement for the model to output entity pairs, the inherent generative nature of LLMs may lead to unexpected results, such as “Null”. See Appendix E for the detailed analysis.
Table 1: The statistics of six datasets used for RE, NER, and ED. “#” denotes the number of samples in the specific dataset. Note that “*” indicates that the dataset is preprocessed. That is, we select 10 pre-defined relations from the original 44 provided by CMeIE and then divide the samples in the training and validation set into three subsets based on the selected relations in a ratio of 8:1:1.
Task Dataset Lang. Type # Train # Valid # Test
RE SciERC en 7 1366 187 397 CMeIE* zh 10* 8680* 1053* 1053*
NER ACE05 en 7 7299 971 1060 CMeEE zh 9 15000 5000 3000
ED CASIE en 5 3751 778 1500 DuEE zh 9 11958 1498 3500
set. Next, we merge R with the selected extended relations of all relations in R. After this, we feed each sentence from the test set and the merged relation set into the LLM and design CoT to guide the LLM through the following steps to produce triplets. The model first identifies a set of head and tail entity pairs from the sentence. It then selects a relation for each pair from the provided relation set to form triplets. Finally, the model evaluates the reasonableness of each triplet. Only the triplets that are judged as reasonable are kept. The details of the prompt design are described in Appendix C.

heading: 4 Experiments
text: We conduct extensive experiments to demonstrate the phenomenon of LLMs’ spurious associations in IE tasks. Then, we leverage this phenomenon to improve model performance in these tasks.

heading: 4.1 Experimental Setup
text: Datasets. To illustrate the universality of the phenomenon in IE tasks, we conduct experiments on six public datasets: SciERC (Luan et al., 2018) and CMeIE4 (Guan et al., 2020) for RE, ACE055 (Walker et al., 2006) and CMeEE4 (Zhang et al., 2022) for NER, CASIE (Satyapanich et al., 2020) and DuEE1.0 (Li et al., 2020) for ED. The statistics of these datasets are detailed in Table 1. Notably, for every relation, entity type, or event type, we select 100 training samples in forward label extension to ensure efficiency. The entity pairs/entity spans/triggers6 are restricted to a single
4https://tianchi.aliyun.com/dataset/ dataDetail?dataId=95414
5catalog.ldc.upenn.edu/LDC2006T06 6Notably, every entity pair/entity spans/triggers cannot have other relations/entity types/event types in the training
label in each sample. In addition, we utilize 10 and 20 validation samples for the English and Chinese datasets separately in backward label validation.
Models. We experiment LLMs with various parameter sizes, including ChatGLM (6B) (Du et al., 2022), BaiChuan (13B)7 (Yang et al., 2023), Alpaca (33B) (Taori et al., 2023), LLaMA-2 (70B)8 (Touvron et al., 2023), ChatGPT9 and GPT410 (OpenAI, 2023). Notably, for GPT-4, due to its higher access costs, we randomly selected half of the total samples from every dataset for our experiments. Our experiments are conducted on a workstation running Ubuntu 20.04.6 LTS, with two Intel(R) Xeon(R) Platinum 8336C CPUs, four NVIDIA A800 GPUs, and 1.0TiB of memory.
Evaluation metrics. Following the previous works (Wei et al., 2023; Zhang et al., 2023), we employ three standard evaluation metrics, i.e., micro Precision (P), Recall (R), and strict Micro-F1 score (F1). Notably, in RE, a triplet is considered correct only if the relation type, along with the types and the boundaries of the head-tail entities are precisely determined. In NER, only when both the span and the type of the predicted entity are accurately predicted, we consider it correct. In ED, an event is considered correct only if both the event trigger and event type are accurately identified.

heading: 4.2 Study Results

heading: S1: Does the phenomenon of spurious associations manifest across various scales of LLMs?
text: After performing forward label extension and backward label validation, we present the results in Table 2. We observe that: 1) Spurious associations exist across various scales of LLMs in both Chinese and English datasets for the RE, NER, and ED tasks. This is evident from the consistently higher DIS-T results. 2) The results from SIM-T illustrate that even when the extended relation/entity/event types closely resemble the original, the performance based on these labels is inferior to that of the dissimilar label (DIS-T). This phenomenon appears counterintuitive and the underlying reasons will be explored in future work. 3) In scenarios where the extended labels diverge from the original, it is notable that the count of extended labels
sample. However, we observe that such instances are relatively rare in the IE datasets. Refer to Appendix A for more details.
7https://github.com/baichuan-inc/Baichuan2 8https://ai.meta.com/llama/ 9gpt-3.5-turbo
10gpt-4-0314
associated with accurate predictions significantly surpasses the count linked to inaccurate predictions. This shows that most of the labels extended and validated from the LLM are effective for identifying head-tail entities/entity spans/triggers.
S2: What is the extent of LLMs’ spurious associations? To analyze the extent of LLMs’ spurious associations, we conduct the following exper-
iments, continuing to employ the RE task as an example. First, we extract triplets from the samples in the test set using every relation extended by the validation set. Then, we retain the extended relation if at least one accurately extracted triplet is found in all results generated by the samples associated with this original relation. Finally, we ask the previous human annotators to determine
whether the pre-defined relation aligns with the remaining corresponding relations in semantics. The presence of a significant number of extended rela-
tions that semantically diverge from the original relations indicates the substantial extent of the phenomenon. The experimental results are shown in Table 3. By analyzing the results, we notice that a significant portion of the valid extended labels chosen in the test set are considered dissimilar to the pre-defined labels by human annotators. This observation highlights the noticeable presence of spurious associations in LLMs. In particular, the overall ratio of spurious associations for ChatGPT consistently exceeds 60% across the six datasets.
S3: How relevant are extension labels and original labels in specific contexts? We design the experiments as follows: First, the similarity between the original label and the three extended labels randomly selected from DC (referenced in Table 3) respectively, which can accurately predict the results on the validation set but are regarded as dissimilar to the original label, is calculated using the same text. The mean of these three similarity scores is denoted as SimDC . Second, the similarity assessment is repeated for the original label against three extended labels randomly selected from V-T (Table 3) respectively, which incorrectly predict the results and are considered dissimilar, using the same textual content. The average of these scores is recorded as SimV−T . In cases where there are fewer than three labels, additional labels are randomly selected from those extended by the validation set to complete the set of three. The difference, SimDC − SimVT , is then calculated for each predefined label and the results are illustrated in Figure 5. We observe that for the extended labels considered dissimilar to pre-defined labels by humans, the SimDC for most labels correctly
predicted by the model is higher than the SimV−T for those incorrectly predicted. This suggests that the labels in DC are closer in vector space to the original labels than the labels in V-T.

heading: S4: Do the extended labels improve the model
text: performance on the test set? To further evaluate the usefulness of the extended labels, we incorporate the Top-1 extended label of each type into the pre-defined set of all types to enhance the model performance on the test set (refer to Section 3.3). In this experiment, we design three baselines. The first one considers the text and all pre-defined types as the input, and the model predicts the results (triplets in IE, entity and its type in NER, and trigger and its type in ED). The second baseline adds the type definition derived from GPT-4 based on the first baseline. The third baseline is to use GPT-4 to paraphrase the pre-defined type also based on the first baseline. The experimental results are listed in Table 4. We observe that our method consistently outperforms all baselines across all datasets and metrics in the overall evaluation, which illustrates the effectiveness of our extended labels. In par-
ticular, compared with the baselines, our method achieves a substantial improvement in F1 score by 9.55%, 11.42%, and 21.27% on the SciERC (RE task), ACE05 (NER task), and DuEE (ED task) datasets, respectively. In addition, our method outperforms the baseline in terms of P, R, and F1 on most relationship/entity/event types. However, there are also cases where the extraction results based on extended labels are inferior to those produced by baselines, such as synonyms and phase in CMeIE.

heading: 5 Conclusion
text: In this paper, we observe an intriguing phenomenon: LLMs’ spurious associations, when utilizing the LLM-based method for accomplishing IE tasks. To explore this phenomenon, we design two strategies in this study, including forward label extension and backward label validation. Moreover, we leverage these extended labels to enhance model performance. Following the procedures described, we conduct extensive experiments to validate this intriguing phenomenon of LLMs with varying parameter sizes. Furthermore, we perform
experiments on downstream tasks, confirming that the extended labels have a positive impact on all IE sub-tasks.
Limitations
This study focuses on discovering the phenomenon of spurious associations in LLMs and utilizing this insight to enhance the model’s performance in IE tasks. However, it’s crucial to acknowledge a limitation: we do not provide an in-depth analysis of the underlying causes of this phenomenon. This limitation stems from the inherent black-box nature of LLMs. Therefore, we identify the exploration of the causes as a topic for future research. In addition, the phenomenon we discovered is also limited to the tasks that can be characterized as the A-B pair prediction problem, as described in the Introduction.

heading: Acknowledgments
text: We would like to thank the anonymous reviewers for their excellent feedback. This work was supported by the National Natural Science Foundation of China (No. 62306112), the Shanghai Sailing Program (No. 23YF1409400), and the Shanghai Pilot Program for Basic Research (No. 22TQ140010020).

heading: A Exploring Multi-Label Situations
text: In Footnote 5, we ignore the situation where an entity pair corresponds to multiple relations in a sample due to its infrequent occurrence. To prove this statement, we quantify the instances of this situation in the RE datasets. Similarly, we perform similar operations on the NER and ED datasets. The results are listed in Table 5. From the table, we observe that the number of samples that meet the above situation is very small, less than 1%. This shows that the strategy we adopted is reasonable.

heading: B Detailed Results of Table 3
text: This section reports the detailed results of Table 3 for ACE05, CMeEE, CASIE, and DuEE. The results are listed in Tables 6 and 7. We notice that the model utilizing extended labels effectively extracts the same results as the model employing actual labels in both NER and ED tasks. However, the annotation experts consider these two labels distinct, as indicated by a relatively high dissimilarity ratio. In particular, the overall ratios of unexpected associations for ChatGPT across the ACE05, CMeEE, CASIE, and DuEE datasets stand at approximately 64%, 69%, 91%, and 77%, respectively.

heading: C Prompt Details in Application
text: In this section, we provide an overview of the prompts used in Section 3.3. Taking RE as an example, the prompt is outlined in Figure 6. To enhance model performance, we augment the inputs
Table 8: Results of the application of extended labels on the NER task. △ represents the results of our method minus the results of the baseline with the highest F1 score.
Original Label Definition Paraphrase Our Method △ Test Sets
P(%) R(%) F1(%) P(%) R(%) F1(%) P(%) R(%) F1(%) P(%) R(%) F1(%) P (%) R (%) F1(%)
# ACE05 facility 38.46 31.25 34.48 40.00 25.00 30.77 12.50 6.25 8.33 63.64 50.00 51.85 +25.17 +18.75 +17.37 geographical soci. 63.64 73.68 68.29 83.33 52.63 64.52 60.87 73.68 66.67 90.00 63.16 64.52 +26.36 -10.53 -3.78 vehicle 87.50 50.00 63.64 70.00 50.00 58.33 35.29 42.86 38.71 90.00 64.29 75.00 +2.50 +14.29 +11.36 weapon 63.64 58.33 60.87 52.00 68.42 59.09 46.67 58.33 51.85 90.00 91.67 81.82 +26.36 +33.33 +20.95 organization 75.00 69.23 72.00 38.10 80.00 51.61 77.78 53.85 63.64 66.67 69.23 62.07 -8.33 +0.00 -9.93 person 60.00 24.00 34.29 50.00 24.00 32.43 70.00 28.00 40.00 100.00 44.00 51.28 +30.00 +16.00 +11.28 location 42.86 60.00 50.00 41.67 50.00 45.45 50.00 50.00 50.00 45.45 60.00 50.00 +2.60 +0.00 +0.00
overall evaluation 49.54 60.00 54.27 53.08 51.89 49.81 43.12 51.09 46.77 55.05 68.97 61.22 +1.97 +17.08 +11.42
# CMeEE medical department 7.55 34.78 12.40 41.03 69.57 51.61 48.39 65.22 55.56 50.00 73.91 58.62 +1.61 +8.70 +3.07 medical procedure 21.43 42.86 28.57 40.74 78.57 53.66 34.48 71.43 46.51 55.00 78.57 64.71 +14.26 +0.00 +11.05 body 41.46 65.38 50.75 42.86 57.69 49.18 42.55 76.92 54.79 80.00 88.46 76.00 +37.45 +11.54 +21.21 medical examinations 39.02 57.14 46.38 43.48 35.71 39.22 60.71 60.71 60.71 76.92 89.29 74.07 +16.21 +28.57 +13.36 medical equipment 43.48 66.67 52.63 66.67 66.67 66.67 76.92 66.67 71.43 76.92 93.33 74.29 +0.00 +26.67 +2.86 disease 58.82 80.00 67.80 59.26 64.00 61.54 78.26 72.00 75.00 75.00 96.00 75.00 -3.26 +24.00 +0.00 microorganisms 62.50 75.00 68.18 61.90 86.67 72.22 81.25 65.00 72.22 91.67 90.00 75.00 +10.42 +25.00 +2.78 clinical manifestations 53.13 70.83 60.71 53.57 62.50 57.69 55.56 83.33 66.67 60.87 100.00 68.85 +5.31 +16.67 +2.19 drug 100.00 90.91 95.24 100.00 90.91 95.24 100.00 90.91 95.24 100.00 100.00 100.00 +0.00 +9.09 +4.76
overall evaluation 63.98 35.10 45.33 63.98 53.60 58.33 72.04 57.26 63.81 81.18 65.09 72.25 +9.14 +7.82 +8.44

heading: 1. Generate a keyword pair list from the given text 2.Extract potential relations for keyword pairs from the given relation list
text: with role definition, instruction, and demonstration. It should be noted that we introduce CoT in the demonstration. That is, we first ask the model to produce the keyword pairs. Then, based on the keyword pairs, we instruct the model to identify potential relations from the given relation list. Next, we integrate the keyword pairs with the identified relations and ask the LLM to assess the factual accuracy of these three elements. Finally, the model retains the correct factual triplets as outputs. Note that in our prompt, we employ the term “keyword pair” instead of directly utilizing “entity pairs”. This strategy aims to stimulate ChatGPT to generate more candidate subject-object pairs in the initial step, effectively enhancing recall.

heading: D Detailed Results of Table 4
text: This section presents the detailed results of Table 4 for ACE05, CMeEE, CASIE, and DuEE. These results are shown in Tables 8 and 9. We notice that the model with our extended labels consistently outperforms the competitors, indicating the effectiveness of these labels. In particular, the model performance has improved by 11.42%, 8.44%, 13.55%, and 21.27% on ACE05 (NER task), CMeEE (NER task), CASIE (ED task), and DuEE (ED task) datasets, respectively. In addition, even when the F1 based on the original labels (such as location) is 0, the model optimized by extended labels also can extract the correct results.
However, there are also cases where the extraction results based on extended labels are inferior to those baselines, such as geographical social political and organization in ACE05, data breach and ransom in CASIE.

heading: E Unexpected Outputs in Backward
text: In footnote 3, we mention that, despite our requirement for the model to output entity pairs, the inherent generative nature of LLMs may lead to unexpected results. These unexpected results include two aspects: 1) LLMs would generate an empty
output, and 2) LLMs would fail to produce output in the expected format. We take ChatGPT as an example to provide statistics on these two situations in Tables 10, 11, and 12. The results reveal that regardless of the RE, NER, or ED datasets, both situations are present, with irregular proportions. Moreover, combining Table 2 and Table 4, we conclude that the spurious association phenomenon of LLMs and the positive effect of extended labels on downstream tasks remain unaffected by these situations.

heading: F Detailed Results of Extended Labels
text: In this section, we provide the three extended labels with the highest F1 scores for each pre-defined type during the application of the extended labels. The results for the RE, NER, and ED tasks are presented in Tables 13, 14, and 15, respectively. To facilitate better understanding, we also provide the original Chinese words for the extended labels on the three Chinese datasets CMeIE, CMeEE, and DuEE at the URL https://github.com/TreMila/SaIE.
SciERC
hyponym-of version-of 13.56 instance-of 10.96 exemplify 10.39
feature-of modifier-of 12.77 embedded-in 11.43 attribute 11.11
used-for applied-to 30.48 applies-to 29.47 use_as 28.57
conjunction coordination 14.81 sequence 12.90 coordinate 9.52
evaluate-for measure-of 18.60 result-from 12.24 indicator-of 9.09
part-of additional constraint 14.29 expand/extend 12.05 combine-and 11.90
compare outperform 51.85 outperforms 45.45 comparison/contrast 43.24
CMeIE
clinical manifestations possible symptoms 56.82 common symptoms 56.41 accompanying symptoms 55.81
age of onset predisposing age 56.00 onset time 48.28 time of occurrence 43.75
susceptible gender
sex differences in onset 57.14 incidence sex ratio 56.00 disease gender bias 56.00
synonyms analogy 16.67 subclass relationship 16.22 disease_alias 15.79
susceptible population
disease onset age 61.54 risk of disease 58.33 incidence group 50.00
drug therapy treatment measures 52.63 treatment programs 50.00 treatment equipment 43.90
auxiliary examination
diagnosis methods 52.17 confirmation methods 51.85 check for complications 51.61
phase disease level 40.00 duration 30.77 symptoms/manifestations 30.00
prevention substitute 41.38 prevention/treatment 40.00 slow down progress 36.36
high-risk factor
susceptible groups 72.22 uncertain relevance 56.41 comorbidities 55.56
Table 14: Three extended labels with the highest F1 in the application of NER task.
Entity Types Extended Entity Labels F1 (%)
ACE05
organization group 62.07 governmental organization 60.00 sports team 59.26
person living_being 51.28 person/organization 47.37 entity 44.44
geographical soci.
geopolitical location 64.52 geopolitical entity 64.52 other 63.16
vehicle type or vehicle 75.00 transportation 75.00 machine or equipment 69.57
location geographic_area 50.00 geographic location 47.62 geographical entity 45.45
weapon weapon category 81.82 weapon_type 81.82 weapon/tool 72.00
facility sentence 51.85 physical object 50.00 infrastructure 46.67
CMeEE
drug substance 100.00 brand 100.00 medicinal 100.00
body body parts 76.00 parts 75.47 human organs 73.33
medical procedure
medical behavior 64.71 route of administration 60.61 dosing method 58.82
clinical manifestations
abnormal behavior 68.85 indicator results 66.67 phenomenon 66.67
medical equipment
instrument 74.29 laboratory equipment 71.43 infrastructure 70.97
medical examinations
laboratory test results 74.07 biological indicators 71.64 biochemical indicators 69.84
medical department
field expertise 58.62 department/agency 58.18 academic area 55.17
microorganisms
microbial subtype 75.00 microbial drugs 75.00 source of infection 73.91
disease disease characteristics 75.00 vaccination history 74.07 disease cause 73.47
CASIE
phishing trick 62.07 deceive 52.94 trap 52.17
data breach steal 72.73 data theft 69.57 theft 66.67
ransom extortion 80.00 financial crime 80.00 event type: ransom 69.57
discover vulnerability detection 78.26 discover 72.73 vulnerability discovery 70.00
patch vulnerability update 63.64 solution 61.54 software_patch 60.87
DuEE
finance/ trading
transaction-pick 80.00 economic-transfer 66.67 capital markets-listing 64.29
product behavior
business activities-release 90.00 product-launch 90.00 financial business-launch 86.96
contact relationships-apology 81.82 emotion-visiting class 80.00 personal connection-thanks 80.00
disaster/ accident
traffic accident-distress 57.14 unexpected event-distress 55.56 natural disaster-accident 52.63
competitive behavior match result-beat 66.67 match-beat 66.67 contest result-defeated 64.00
organizational behavior personal relationships-exit 100.00 movement-leave 100.00 sports competition-exit 96.00
life personnel status-deceased 53.85 death-remains 53.33 health-death 51.61
judicial act legal action-detention 82.76 crime-arrested 76.92 behavior-arrested 76.92
organizational relation meeting-opening 86.96 sports competition-unveiling 86.96 events-unveiling 86.96

heading: Future Work
text: Therefore, we identify the exploration of the causes as a topic for future research. In addition, the phenomenon we discovered is also limited to the tasks that can be characterized as the A-B pair prediction problem, as described in the Introduction