Causal Intervention-based Few-Shot Named Entity Recognition
Authors: Zhen Yang
Yongbin Liu
Chunping Ouyang
Venue: EMNLP 2023 Findings (2023)
Keywords: Causal Intervention
Few-Shot Learning
Named Entity Recognition

Abstract: Few-shot named entity recognition (NER) systems aim to recognize new classes of entities with limited labeled samples. However, these systems face a significant challenge of overfitting compared to tasks with abundant samples. This overfitting is mainly caused by the spurious correlation resulting from the bias in selecting a few samples. To address this issue, we propose a causal intervention-based few-shot NER method in this paper. Our method, based on the prototypical network, intervenes in the context to block the backdoor path between context and label. In the one-shot scenario, where no additional context is available for intervention, we employ incremental learning to intervene on the prototype, which also helps mitigate catastrophic forgetting. Our experiments on various benchmarks demonstrate that our approach achieves new state-of-the-art results.

few-shot scenarios Causal Intervention-based Few-Shot Named Entity Recognition
Zhen Yang, Yongbin Liu; Chunping Ouyang
University of South China (uscyz094, yongbinliu03) @gmail.com, ouyangep 126.com
Abstract
label: animal
There too many pigeons in the square.
C: Confounders
Few-shot named entity recognition (NER) sys- tems aim recognize new classes of entities with limited labeled samples. However, these systems face significant challenge of overfit- ting compared to tasks with abundant samples. This overfitting is mainly caused by the spuri- ous correlation resulting from the bias in select- ing few samples. To address this issue, we propose causal intervention-bast NER method in this paper. Our method, based on the prototypical network, intervenes the context to block the backdoor path between context and label. In the one-shot scenario, where no additional context is available for in- tervention, we employ incremental learning to intervene on the prototype, which also helps mitigate catastrophic forgetting. Our experi- ments various benchmarks demonstrate that our approach achieves new state-of-te oate-off-hhe-rr re- sults.
E: Entty Presicted
(a]
dafl)
few-shot
(6)
(c)
Figure I: (a) An example of spurious correlation. "Pi- geons" are easily associated with "square", ", but not animals associated with "square". (b) Causal graph of the example. Contexts 7 such as the "square") "), enti- ties (such as "pigeons"), ) and class label (such as the "animal" class type), C means confounders brought by samples selection bias in the few-shot task. (c) Causal graph after do-operation
1 Introduction
and Katiyar, 2020: Hou et al., 2020). Moreover, span-based methods (Wang et al., 2021a; Yu et al., 2021; Ma et al., 2022) have emerged to facilitate the accurate identification of entity boundaries in few-shot named entity recognition.
Named entity recognition (NER) fundamen- tal task in information extraction, which involves identifying and classifying named entities in un- structured text. Several methods (Chiu and Nichols, 2016; Ma and Hovy, 2016; Lample al., 2016; Pe- et al., 2017) have been developed to achieve efficient results NER.
However, these existing methods tend to over- look the issue of overfitting caused by spurious correlation in few-shot tasks. While this problem less prominent tasks with abundant samples, it becomes crucial to address due to the bias introduced by the limited sample se- lection. As illustrated in Figure (()), we provide an example of spurious correlation. In this particular example, the context "square" "is is linked to the entity "pigeon" and the entity "pigeon" is associated with the label "animal". It is worth noting that in some NER tasks, there may be cases where the "animal" label is mistakenly associated with the semantic feature "square", but this correlation does not apply to all animal entities.
In practical applications, few-shot named entity recognition has gained significant attention due to challenges in label collection and the high cost of manual labeling. Recent years have witnessed numerous studies focusing on few-shot NER. The current approaches primarily revolve around metric learning models, particularly prototypical networks, These models compute prototype representation (Snell et al., 2017) for each class and assign labels to query samples based on their distance to the prototypes of each class (Fritzler et al., 2019; Yang
"Corresponding Author.

---

The aforementioned issues can be attributed to the overfitting of the model to confounders present in the limited number of samples. Specifically, the few samples selection bias acts as a confounder that leads the few-shot NER model to learn spuri- ous correlations between contexts and labels. For instance, in Figure (a),, the context "square" be- comes associated with the ground-truth entity "pi- geon". This denoted P [E] T),, tends to create positive association between the context and certain entities. When encountering the context "square", it increases the likelihood that belongs to the en- tity "pigeon". ". However, this association based on the confounder misguides the model to associate the non-causal but positively correlated context T In other words, the context "square" is incorrectly perceived a stable and intrinsic feature of the class label "animal", even though is a non-causal feature the "animal" class.
de Masson D' Auutume al., 2019) and employ the previous prototype to intervene the current pro- totype to obtain the final prototype. This approach also prevents catastrophic forgetting (Thrum, 1995) in the model. Our contributions can be summarized as follows:
From causal perspective, redefine the lask and introduce context-based intervention for 5-shot. This intervention replaces the con- text and effectively prevents overfitting. By reducing the spurious correlation caused by selection bias in few shot samples, our method significantly enhances the model's generaliza- tion.
with the class label Y, resulting in P(Y E, E
For 1-shot, we use prototype-based interven- tion to reduce the spurious correlation be- tween the current prototype and the label. We also apply sample reweighting to obtain more representative prototypes. Additionally, we utilize GPT for data augmentation to finetune the 1 --shot data and further improve the accu- racy.
Current metric- methods fail to address the underlying confounders present in the few samples, particularly the few samples selection bias. In or- der to tackle the aforementioned challenges, we approach few-shot named entity recognition from a causal perspective. Figure ((b) depicts the corre- sponding causal graph, which represents the causal relationships among the entity E, relevant context T, class label Y. and confounders C. The direct link between two nodes signifies the existence of causal relation between them.
Our model was evaluated on the Few_NERD and SNIP datasets, and the results demon- strate its superiority, surpassing previous state- of-the-art methods. Through comprehensive experiments, we observed significant im- provement in generalization, with an average increase of 2% and up to 13% improvement across all tasks.
In Figure ((b), a backdoor path T C is present, indicating that previous methods may erro- neously learn spurious correlations caused by this backdoor path. To address this issue, we propose a model based on causal interventions. Specifically, we utilize context-based interventions to mitigate the spurious correlation between contexts and class labels. From causal inference perspective, as depicted in Figure (c)), we employ PE do())) instead of P (') to eliminate the influence of confounders. The do-operation, denoted as do(T). allows us to establish the causal relationship be- tween the contexts T and the class labels Y with out the interference of confounders (Pearl, 2009). By calculating P(Y E, do)))) can block the backdoor path and eliminate the spurious correla- tion arising from confounders. For the one-shot task, obtaining additional contexts challenging. To overcome this, we draw inspiration from in- cremental learning (Thrun, 1995; French, 1999;
2 Related Work
2.1 Few-Shot Learning and Meta-Learning
Few-shot learning has gained significant popular- ity in natural language processing (NLP) research (Chen et al., 2019; Gao et al., 2020; Brown et al., 2020; Schick and Schütze, 2020; Lin et al., 2022). However, overfitting is common challenge few- shot learning. To mitigate this issue, researchers often incorporate source domain data (Han et al., 2018; Geng et al., 2019; Wang et al., 2021b). Ini- tially, meta-learning techniques were widely ap- plied in computer vision. The introduction of pro- totype networks led to the adoption of metric-based methods (Kulis al., 2013; Vinyals et al., 2016: Snell al., 2017). These methods involve encod- ing support set vectors, generating prototype rep- resentations, and calculating the distance between prototypes and query data using various metrics.

---

Finally, the query is classified based on the nearest prototype.
adjustment eliminate spurious correlations and identify causal effects between variables.
2.2 Few-shot Named Entity Recognition
3 Methods
Previous research in few-shot named entity recog- nition has explored various token-level approaches (Fritzler et al., 2019; Yang and Katiyar, 2020; Hou et al., 2020; Gong et al., 2021). One pop- ular method is the use of prototype networks (Snell et al., 2017). Building upon this, NNShot and StructShot (Yang and Katiyar, 2020) intro- duced feature extraction and nearest neighbors tech- niques. The development of Few-NERD (Ding few-
We give a few shot named entity recognition task descriptions in Appendix A. In the subsequent sec- tions, we outline our method for few-shot named entity recognition.
3.1 Causal Intervention
The few-shot named entity recognition task is plagued by significant selection bias in the data, leading the emergence of spurious correlations that can mislead the model's overfitting. To ad- dress this issue, we propose causal intervention- based approach. Figure depicts the problem the task, where 7 represents the context, E denotes the entity, Y signifies the predicted label, and C represents confounders, such the few samples selection bias.
al., 2021), )), a large-scale
shot NER dataset, allowed for the evaluation of ProtoBERT, NNShot, and StructShot methods. Ad- ditionally, prompt-based technologies (Cui et al.. 2021) have emerged in this field. However, these methods have shown limited generalization capabil- ities. To improve the accuracy of few-shot named entity recognition, the method proposed in (Das al., 2021) incorporates Gaussian embedding and contrast learning. Notably, these approaches of- ten overlook the integrity and boundaries of en- tities. In response, span-level approaches (Wang et al., a021a; Yu et al., 2021; Athiwaratkun et al., 2020; Wang et al., 2021c) have been proposed. For instance, ESD (Wang al., 2021a) leverages span representation and matching to enhance entity recognition completeness. Similarly, Decomposed- MetaNER (Ma et al., 2022) fine-tunes parameters using support instances and focuses solely on entity localization during span detection.
T E- Y The entity representation is learned in terms of contexts. The model uses the entity representation to get the predicted label.
T+ C-+ Y There exists a backdoor path in the model that creates a spurious correlation. The vari- able C encompasses the few samples selection bias. These biases cause the context to overly focus the current data, misleading the prediction of labels. This backdoor path results in spurious correlation between the context and the predicted class label.
To address the backdoor path T <- C-+ Y, we perform interventions on T. However, intervening on T directly is challenging due to the confounders C being difficult to capture. Therefore, employ front-door adjustment in the calculation, as shown in Equation 1.
However, the aforementioned methods primarily focus on the current support and query instances, neglecting the influence of context on entities. This oversight can result the overfitting of both con- texts and entities.
P(Y v/do(T ()) P v/E do(T ())
2.3 Causal Inference
P(E cldo(T ())
Causal inference, as described by Pearl (2009), aims to eliminate confounders and obtain causal effects between variables, enabling accurate predic- tions in tasks. In light of the challenges posed by fitting issues and causal effects in few-sample sce- narios, our objective is to leverage causal inference enhance the robustness and transferability of our model. Many prior works have employed causal theory to improve model robustness, employing techniques such as counterfactual analysis, unbi- ased estimation, visual cues, and front-back door
Based the principles of causal inference, we have derived Eq 2 as the final result. The detailed derivation process can be found in Appendix B, providing a comprehensive explanation the equa- tion.
P(Y = y v/do = 1)) - P(E=
F
P(Y y e,T LTP (2)

---

Causal Intervention
Sample
P(T=1)
Entity
Context
do(T)
P(Y=y(E=e,
ProtoNet
do(l)
1000 PP(T=1)
(a)
(b)
Figure 2: (a) The causal structure, where 7' is the context, E is the entity representation, Y is the predicted label. Besides, there exists the confounders, such as samples selection bias. (b) Our context-based intervention, block the spurious correlation between context and predicted label. We replace entities of the same type to get new sample and feed it into the prototype network to get the predicted label.
Based on the three terms on the right of Eq 2, we introduce three important components: entity detec- tion, context-based causal intervention, and sample reweighting. Each component has its specific role, and we will provide a detailed explanation of each below.
entity prototype, Y represents the predicted label, and C corresponds to the confounders. P The model calculates the Euclidean dis- tance through the entity prototype and gets the pre- dicted label.
Pt C- Y The presence of confounders, such as sample selection bias, can distort the calculation of the entity prototype, leading to spurious relation between the prototype and the predicted label. This spurious correlation, in turn, contributes to overfitting in the model.
In the expression EE P(E - is the entity of the current sample. is the context of If we consider E as a binary classification (entity vs. non-entity), this term denotes entity detection. It represents the probability of the entity E being equal to e given the context being equal to t.
To block the backdoor path P+ C-+ Y, we use the intervention which is similar to T. The interven- tion on P follows a similar approach in Egq 2.
In the expression Ee P(Y y = c, T= f'), i' refers the contexts of other entities that belong to the same type as e. This term indicates that to obtain the final prediction label y. we need to tra- verse t' when E is equal to e. So we propose the context-based causal intervention that involves replacing the contexts to observe the resulting pre- dicted labels. This allows us analyze the impact of different contexts the prediction when the entity fixed.
In the 1 1-shot, where we don't have additional entities to intervene on, we focus on intervening on the prototype P. Since there a direct causal path between P and Y, we can simplify the intervention by directly modifying the prototype. Specifically, we propose the prototype-based causal interven- tion consider both the previous prototype and the current prototype. Our experimental results confirms the importance of prototype intervention in improving the performance.
In the expression P(T = t'), with the calcu- lation of the prototype, we propose the sample reweighting to take into account the weights as- signed different samples. This is done to ad- dress the potential differences between the source domain and the target domain distribution. By cal- culating the weights, the prototype representative and takes into account the variations in the context distribution.
3.2 Entity Detection
In order to address the limitations of traditional pro- totype networks in capturing cross-class common- alitics, we propose an entity detection mechanism. While the traditional prototype network averages the vectors of entities belonging the same class to obtain class-specific prototypes, it overlooks the common features that may exist across different classes.
In the case of 1-shot, where no additional entities are available for intervention, we analyze the model. Figure 3 provides an illustration of the problem within the model. In this figure, denotes the
The entity detection approach, on the other hand,

---

Sample
do(P)
Previous:
ProtoNet
Prototype
Pro
Intervention
Current
ProtoNet
Pti
(0)
(b)
Figure (a) Prototype-based intervention, where is the entity prototype, the predicted label, C the confounders. (b) We intervene in the prototype to block the spurious correlation between the prototype and the predicted label. During the current episode, the previous prototype introduced to the current episode as knowledge. The two prototypes combined the current prototype. the episode, this prototype is introduced as the previous prototype.
aims to identify all named entities within the in- put sequence, without differentiating their specific classes. takes into account the potential simi- larities between entities of all classes. To achieve this, we feed all the entities as a whole into the prototype network, as well as all the non-entities, to identify and locate all entities in the sequence. This approach allows to uncover features that common to all entities, which can be used for the initial filtering of the input.
focuses on blocking the backdoor path. We achieve this by intervening on the context variable T, effee- tively blocking the influence the confounders C on the predicted label Y. We refer to this interven- tion strategy as context-based causal intervention.
The method outlined in Equation involves it- crating over each T calculate the final predicted label Y for a given entity E. To accomplish this, we employ a traversal process illustrated in Figure 2(b). During training, for each sentence, we replace the entities in the sentence with other entities of the same type, one at time, in order to explore all pos- sible combinations. This traversal process allows us consider the impact of different context-ent pairs on the predicted label. However, it's impor- tant note that this process is only applied during training. During testing, we use the original data without any entity replacements.
When integrating the predicted labels, it is im- portant to consider the dimensionality matching problem. Therefore, we expand the entity detection results to match the dimensionality of the class, suring compatibility for subsequent processing and classification tasks.
3.3 Context-based Causal Intervention
the few-shot task, limited samples can lead to overfitting and hinder generalization. This is es- pecially problematic when the entities are embed- ded within the contexts, as the model may overly adapt to specific context-entity relationships. To address this, we need to consider the causal rela- tionships between entities, contexts, and predicted labels. Causal inference helps mitigate overfitting by removing spurious correlations and improving generalization.
3.4 Prototype-based Causal Intervention
In 1 1-shot experiments, context- based causal inter- vention is not possible due to the lack of additional samples. Instead, we use prototype-based inter- vention (Figure 3). Previous methods average sup- port instances to calculate prototypes and rely on prototype-query distances for entity recognition. However, these methods can be inffuenced by con- founders, leading to spurious correlations and over- fitting.
In Figure 2(a), we present our approach for the few-shot named entity recognition task. The pres- ence of confounders, such as the few samples se- lection bias, creates a backdoor path T C-> between the context and predicted label. This back- door path introduces spurious correlations, leading to the overfitting of both the context and the pre- dicted label. To address this issue, our method
To overcome this, we propose prototype-based causal intervention. We intervene prototypes (P) to block the P C- Y path, reducing spuri- ous correlations. Our method incorporates prior knowledge by saving prior prototypes and combin- ing them with the current class representation. This approach mitigates overfitting by considering both

---

current and previous data.
the number of classes and K is the number of sup- port examples per class. The dataset includes four settings: 5-way 1~2-shot, 5-way 5~ 10-shot, 10- way 1~2-shot, and 10-way 5~100shot. We fine- tuned the model following (Ma et al., 2022) parti- tion. However, we found that their dataset contains overlapping portions. To provide better repre- sentation, we have included additional results after removing the overlaps, which are indicated in Table with 'f".
Prototype-based causal intervention effectively blocks spurious correlations between prototypes and predicted labels. improves generalization and prevents overfitting in prototype calculation.
3.5 Sample Reweighting
We propose sample reweighting address the is- sue of sample weighting. Instead of treating all samples equally, we assign different weights to each sample based on their relevance to the proto- type calculation. This is done by calculating the distances between support samples and the query and using Softmax to determine the weights.
SNIPS dataset (Coucke et al., 2018) focuses on slot filling and includes seven domains: D = (D1, D2, ... D7}. The leave-onc-out strategy is followed, where a target domain is tested using randomly se- lected validation domain, and the model is trained the remaining source domains. In the few-shot slot-filling task of SNIPS, K examples per class are used in the support set. The task has 1-shot and 5-shot settings.
We also incorporate Maximum Mean Discrep- ancy (MMD) into the loss calculation to reduce the distribution difference between the training and test support data. The Softmax and MMD Eq 3. By combining the MMD loss with the classification loss, we aim to improve domain adaptation.
4.2 Parameter Settings
Furthermore, we perform mee-tunng using the support instances from the test data to better adapt to the target domain. In the 1-shot scenario, where additional data is available for fine-tuning, use GPT for data enhancement before fine-tuning.
In implementation, we utilized the Bert-base- uncased model (Devlin et al., 2018) as our base model. We the maximum sequence length 32. For optimization, we used AdamW (Loshchilov and Hutter, 2017). For further information on the parameter settings, please refer to Appendix C.
0₁ softmaz(ha(xz)) he (xx)))
4.3 Evaluation metrics
For the evaluation of Few-NERD and SNIP, we followed ESD (Wang et al., 2021a), calculated the Fl Fl-score(FI).
N N
L(0)-
CrossEntropy ho (a)a)))
sup - (t)
(4)
4.4 Baselines
For a comprehensive comparison in Few_NERD We select the top six state-of-the-art methods such as ProtoBERT (Ding et al., 2021), NNShot (Ding et al., 2021), StructShot in Few-NERD (Ding et al., 2021), CONTAINER (Das et al., 2021), ESD (Wang et al., 2021a) and DecomposedMetaNNR (Ma et al., 2022). To ensure a fair comparison, we reproduce DecomposedMMttNER using our maxi- mum sequence length of 32. For SNIP, we choose the moethods including TransferBERT (Hou et al., 2020), MN+BERT (Hou et al., 2020), ProtoBERT (Hou et al., 2020), Ma2021 (Ma et al., 2021), L- TapNet+CDT (Hou et al., 2020), ESD (Wang et al., 2021a), Retriever (Yu et al., 2021) and CEx (Henderson and Vulić, 2020). The detailed base- line descriptions can be found Appendix D.
Where h is our network, y is the true label, denotes the distribution of the source domain s, 9 denotes the distribution of the target domain
4 Experiments
4.1 Dataset
Few-NERD (Ding et al., 2021) dataser consists of two tasks: FewNERD-INTHE and FewNERD- INTER. FeWNERD-INTRA has entities of differ- ent grained types in the training, valida- tion, and test sets, while TewNERD-INTER allows overlapping coarse coarse-grained types but disjoint fine- grained entity types. In FewNERD, the N-way K~2K shots approach is used, where N represents
"hups://apperswithcode.. http:///www...o.o/////////sottta

---

Intra
Inter
Models
1~~2-hoo
5~ 5~~10-shot
1~-2-shot
5~ 10-1ht
Avg.
Avg.
5 way
10 way
5 way
10 way
5 way
10 way
5 way
10 way
ProtoBERT
20.71+1.16
15.32+0.a
37.08-101 28.02+a56
25.28 37.49 26.98
+2.42 $6.29 43.30
NNShot"
21.58+070
15.72 25. 25.66=077 19.82+111 20.70 40.31+2.30 31.54_163 12.66=1.07 37.09 37.90
StructShot"
23.95
10m 29.68=111
17.10=175 20.76 38.78.15.0
22.61=00s
1.0
42.75 36.02
CONTAINER
40.00+a71
35.89
54,28=n64
48.48
44.66 52.17:2.74
61.11 50.00=146
59. 39.8440.36 55.78
ESD
37.121008
31.26:1053 50.50=179
33.381471
38.07 $7.56:22.5
52.89-1-11
69.922056
66.9010.0 61.82
DecomposedMetaNER
45.06+135
39.55+0.73
55.08=171
48.26+222
46,99
61.74++0.1
57.40-1.77
66.20+a.s
61.18 61.68
DecomposedMetaNNR
46.97
41.30+0..9
59,40=2.48
50.71 49.60
62.76++261
59.57=2a8
70.90
66.7640.4 64,70
Ours'
52.21 41.47:009
67.32_0044
59.78:42x
55,20
1.:3
=9,91_au1
79.21 2..38
73.47 10.59 69.48
Ours
54.00 43,58 70.34 -0.7 61.33
57.31 19.4 60.67=a75
81.89 75.86 71.96
OurS-GPT-Aug
73.721264 62.6T21.3
80.8512.14 67.63_n$5
Table 1: Performance of state-of-art models on Few-NERD. The "-" indicates that in order have sufficient data fine-tuning, we use GPT for data augmentation (Ours+GPT-Aug) and generate an additional example. The "*" indicates that the results were directly obtained from the original paper. The "4" represents the experimental results obtained after removing the overlapping portions in the set. The best results among the methods without using GPT are in Bold.
4.5 Results and Analysis
provement. Prototype-based causal intervention is particularly effective in 1-shot scenarios, contribut- ing to an 8% improvement. Sample reweighting enhances the model's performance by 4%. These results highlight the significance of each compo- nent in our proposed method.
Our method achieves state-of-the-an art results in both the FewNERD and SNIPS datasets, as shown in Table and Table 2. In FewNERD, we observe significant improvements of 11% for the 5-shot setting and 1-8% for the 1-shot setting. These im- provements demonstrate the effectiveness of our context-based causal intervention, entity detection, prototype-basee based causal intervention, and sample reweighting techniques. Similarly, in the SNIPS dataset, our method achieves an average improve- ment of approximately 13% for the 5-shot set- ting and an average improvement of 2% for the I-shot setting. These results highlight the abil- ity of our method to block spurious correlations, prevent overfitting, and improve overall perfor- mance. Meanwhile, GPT data enhancement im- proved our method by 2-20%. When we removed the overlapping portions, observed that our re- sults had smaller decrease compared to Decom- posedMetaNER, which further demonstrates the effectiveness of our model.
4.7 Experimental Analysis
How does entity detection improve entity recog- nition Entity detection is essential for dealing with boundary cases. For example, it helps distinguish instances where "the" is part of entity, like "the Porcellian Club" from cases where "the" is used common article, such as "the year". Similarly, en- tity detection can correctly identify specific entities like "American League" " while ignoring irrelevant words like airline names in phrases like "American airline". By performing entity detection, the model can improve the accuracy.
4.6 Ablation Study
The setting of the ablation study can be found in Appendix E. Table 3 demonstrates the contribution of each component in our method. Notably, we observed the following findings: Entity detection greatly enhances the effectiveness of context-based intervention, leading to an improvement of approx- imately 8% compared to not using entity detection in shot. Context- based causal intervention helps resolve spurious correlation and leads to a 10% im-
Figure 4: t-SNE visualization of entity replacing on Few-NERD Intra, 5-way 10-hh-
How does context-based causal intervention improve entity recognition In Figure we pro- vide t-SNE visualizations of the original proto- type network and the prototype network with the context-basee causal intervention. In the original

---

Models
We
Mu
Pl
Bo
Se
Re
Cr
Avg.
TransferBERT
55.82
38.0
45.65
1..32
21.96119
21.79
38.5
39.061336
MN+BERT
21.74
10.
39.71 +1::
58.15+0ss
24.21
32.88
69.66=16s
16.72
1-shot
ProtoBERT
46.72+1.0
40.07
50.78 +:.09
68.73+1.M
60.81 +:770
55.58
67.67
55.77
L-TapNet+CDT
74.53+404 60.56+0.7
66.27+2m
84.54+
76.27+1.2
70.79+1.0
62.89-1as
70941+197
ESD
78.25 54.7411.0
71.15
71.45
67.85
71.52.095
78.14 70.44...r
Ours
75.44
58.64
69.22.t1.4
84.61
:4.23
68.57
:0.4
74.73=1.9
78.79=a27 72.86=491
Ours+GPT-Aug
73.30
53.5622.17
18.981004
84.621os:
70.22
73.522
73.521nm1
76.57 20:8
74.402013
TransferBERT
59.41
10.30
42.00
46.07
14.52
20.74
28.20
1:.29
67.75
58.61
13.77 46. 1:229
MN+BERT
36.67 1364
33.67 12
52.60
69.09
2.36
38.42
33.2822299
72.10=1.6
1.44
ProtoBERT
67.8224.1
55.99
12.34
46.0211.19
12.17
73.59 1:
60. 18
66.8922.0
63. 23
5-shot
Retriever
82.95
71.75 km
81.65
73.10
51.35 79.54
7..72
ConVEx
7l.50(mk)
79.00kunk) 77.60kmk)
84.50
84.00
73.80
67. 67.40
76.80tunky
Ma2021
89.39/mk)
75.1 (ank)
77.18
16uuko
73,53
82.29
72.5 (wnk)
79. 19.17(ua
L-TapNet+CCD
71.64 13.62
67.16
75.8811.5)
84.3812.1
82.58
70.05 11.61
73.41 =261
12:01 12:46
ESD
84.50
66.6
42.00
79.69 1.35
82.57
82.22++o.1
80.44
=0080
81.
79.59
Ours
94.67
90.34...4 91.73
9..70 94.75
93.18 18:18::..
89.75
92.73=0.11
Table 2: Below are the Fl scores with standard deviations on 7 domains of SNIPS. Please note that some methods not provide deviation information their paper (marked as 'unk"). The baselines I-shot and 5-shot settings differ because they did not report the s-shot results in their paper.
Contest-husell Interveation
Sample Reweighting
Prototype-hused
fine the current classification. By performing the prototype-based causal intervention, we can rec- tify the error and accurately classify "German" as other-language in the current query.
Setting
Detection
Interveution
Score
2-
x
43.82
x
X
41.57
60,80
00.44
Table Ablation study: scores on Few-NERD
11
10
20
da
(siPato
prototype network, the embeddings are scattered and lack clear boundaries, making it prone to con- fusion. However, after applying the context-based causal intervention, the embeddings become more compact and exhibit clear boundaries. Entities of the same type are grouped closer together, while the distance between different types is increased. This optimization of the embedded space helps to distinguish and classify entities accurately.
25
do
A0
(N/ampp Beweightinn
Figure 5: histogram of sample reweighting Few- NERD Intra, 5-way 1~10o-hot.
How does sample reweighting improve entity recognition Figure 5, we present histograms depicting the data distributions of the source and the target domain before and after applying sam- ple reweighting. By comparing the histograms, we observe that after sample reweighting, the distri- butions of the source and target domains become more aligned, reducing the gap between them. This alignment beneficial for the model's ability transfer knowledge from the source domain to the target domain effectively.
How does prototype-based causal interven- tion improve entity recognition When analyzing the word "German" in the given query, the current prototype calculation may lead to misclassifica- tion. However, by intervening in the prototype and incorporating information from previous support data, we can improve the accuracy. For instance, if "German" was previously classified correctly as other-languee language, we can use that knowledge to re-

---

5 Conclusion and Future Work
Mingyang Chen, Wen Zhang, Wei Zhang, Qiang Chen, and Huajun Chen. 2019. Meta relational learning for few-shot link prediction in knowledge graphs. arXiv preprint arXiv: NrXtv/909.00515
this paper, we propose a method for few- shot named entity recognition that addresses the problem of overfitring by using causal interven- tions. We introduce entity detection, mentext-based and prototype-baae baadty interventions, and sample reweighting to improve performance. Our experi- ments show significant improvements in handling few-shot scenarios and transferring to new domains, Future work includes exploring dimensionality re- duction techniques to reduce memory usage.
Jason PC Chiu and Eric Nichols. 2016. Named entity recognition with bidirectional Istm-enns. Transac- tions of the association for computational linguistics. 4:357-370.
Alice Coucke, Alaa Saade, Adrien Ball, Théodore Bluche, Alexandre Caulier, David Leroy, Clément Doumouro, Thibault Gisselbrecht, Francesco Calta- girone, Thibaut Lavril, et al. 2018. Snips voice plat- form: an embedded spoken language understanding system for tey-aate-y-desiee voice interfaces. arXiv preprint arXiv: 1805. 10190.
Limitations
One limitation of our method is the increased mem- ory usage due context-based intervention. How- ever, our method still offers advantage compared other methods that require maximum length of 128, as we achieve good results with only 32. This helps mitigate the memory issue to some ex- tent. In addition, compared to the significant im- provement observed in the 5-shot, the boost for the I-shot relatively smaller. We can use GPT. GPT data enhancement proves to be particularly helpful in improving performance in 1-shot scenarios. By leveraging GPT enhance the limited amount of available data, we are able achieve better results.
Leyang Cui, Yu Wu, Jian Liu, Sen Yang, and Yue Zhang. 2021. Template-based named entity recognition us- ing bart. arXiv preprint 0rXiv:2/06.01770
Sarkar Snigdha Sarathi Das, Arzoo Katiyar, Rebecca Passonneau, and Rui Zhang. 2021. Container: Few- shot named entity recognition via contrastive learn- ing, arXiv preprint arXiv:2109.0758.
Cyprien de Masson D' Autume, Sehastian Ruder, Ling- peng Kong, and Dani Yogatama. 2019. Episodic memory in lifelong language learning. Advances Neural Information Processing Systems, 32.
Jacob Devlin, Ming-Was Wei Chang, Kenton Lee, and
Kristina Toutanova. 2018. Bert: Pre-training of deep bidirectional transformers for language understand- ing, arXiv preprint arXiv: 18t0.08800.
Acknowledgements
The State Key Program of National Natural Sci- ence of China, Grant/Award Number:61533018; National Natural Science Foundation of China, Grant/Award Number: 61402220; The Philos- ophy and Social Science Foundation of Hunan Province, Grant/Award Number: 16YBA323; Natural Science Foundation of Hunan Province, Grant/Award Number: 20200J4525.22221334955
Ning Ding, Guangwei Xu, Yulin Chen, Xiaobin Wang, Xu Han, Pengjun Xie, Hai-Tao Zheng, and Zhiyuan Liu. 2021. Few-nerd: A few-shot named entity recog- nition dataset. arXiv preprint 10..:2200.0076600
Robert M French. 1999. Catastrophic forgetting in con- nectionist networks. Trends in cognitive sciences, 3(4):128 135.
Alexander Fritzler, Varvara Logacheva, and Maksim Kretov. 2019. Few-shot classification in named en- tity recognition task. In Proceedings of the 34th ACM/SIGAPP Symposium on Applied Computing. pages 900-.
Scientific Research Fund of Hunan Provincial Education Department, Grant/Award Number:
188279,,9A439,22A0311
References
Tianyu Gao, Adam Fisch, and Danqi Chen. 2020. Making pre-trained language models better few-shot learners. arXiv preprint arXiv:2012. 157223.
Ben Athiwarackun, Cicero Nogueira dos Santos, Jason Krone, and Bing Xiang. 2020. Augmented natural language for generative sequence labeling. arXiv preprint urXiv:2009.13277
Ruiying Geng, Binhua Li, Yongbin Li, Xiaodan Zhu, Ping Jian, and Jian Sun. 2019. Induction net- works for few-shot text classification. arXiv preprint 1902 10482.
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020. Language models are few-shot learners. Advances in neural information processing systems, 1877-199..
Yuan Gong, Lu Mao, and Changliang Li. 2021. Few- shot learning for named entity recognition based on bert and two-level model fusion. Data Intelligence. 3(4):568-577.

---

Xu Han, Hao Zhu, Pengfei Yu. Ziyun Wang, Yuan Yao. Zhiyuan Liu, and Maosong Sun. 2018. Fewrel: A large-scale supervised few-shot relation classifica- tion dataset with state-of-the-art evaluation. arXiv preprint arXiv: 1af:::8810.1111 10147.
Oriol Vinyals, Charles Blundell, Timothy Lillicrap. Daan Wierstra, et al. 2016. Matching networks for one shot learning. Advances in neural information processing systems, 29.
Peiyi Wang, Runxin Xu, Tianyu Liu, Qingyu Zhou, Yunbo Cao, Baobao Chang, and Zhifang Sui. 2021a. An enhanced span-based decomposition method for few-shot sequence labeling. arXiv preprint arXiv:2109.13023.
Matthew Henderson and Ivan Vulić. 2020, Convex: Data-efficient and sew-shot slot labeling. arXiv preprint arXiv:2010.1 11711.
Yutai Hou, Wanxiang Che, Yongkui Lai, Zhihan Zhou, Yijia Liu, Han Liu, and Ting Liu. 2020. Few-shot slot tagging with collapsed dependency transfer and label- enhanced task adaptive projection network. arXiv preprint 20XY:22006.05702.
Peiyi Wang, Runxin Xun, Tianyu Liu, Damai Dai, Baobao Chang, and Zhifang Sui. 202 Behind the scenes: An exploration of trigger biases problem in few-shot event classification. In Proceedings of the 30th ACM International Conference on Information & Knowledge Management, pages 1969-1978.
Brian Kulis et al. 2013. Metric learning: sur- vey. Foundations and Trends@ in Machine Learning, 5(4):287-364.
Yaqing Wang, Haoda Chu, Chao Zhang, and Jing Gao. 2021c. Learning from language description: Low- shot named entity recognition via decomposed frame- work. arXiv preprint Wii::ii01.90:5::.
Guillaume Lample, Miguel Ballesteros, Sandeep Sub- ramanian, Kazuya Kawakami, and Chris Dyer. 2016. Neural architectures for named entity recognition. arXiv preprint 1oXivv1603.011
Yi Yang and Arzoo Katiyar. 2020. Simple and effec- tive few-shot named entity recognition with struc- tured nearest neighbor learning. arXiv preprint arXiv:2010.02405.
Qiang Lin, Yongbin Liu, Wen Wen, Zhihua Tao, Chun- ping Ouyang, and Yaping Wan. 2022. Ensemble making few-shot learning stronger. Data Intelligence, 4(3):529-551.
Dian Yu, Luheng He, Yuan Zhang, Xinya Du, Panupong Pasupat, and Li. 2021. Few-shot intent classifica- tion and slot filling with retrieved examples. arXiv preprint 2rXiv:2104.05700
nya Loshchilov and Frank Hutter. 2017. Decou- pled weight decay regularization. arXiv preprint
arXiv.171..5101.
Janqiang Ma. Zeyu Yan. Chang Li, and Yang Zhang. 2021. Frustratingly simple few-shot slot tagging. In Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pages 1028-1033.
A
Task Formulation
Named Entity Recognition (NER) is commonly formulated as sequence labeling problem, where given sequence (x] ./x./, the goal is to assign a label to each x,. These labels indicate whether x; belongs named entity category (e.g.. person, location) is not part of any entity (O class).
Tingting Ma. Huiqiang Jiang, Qianhui Wu, Tiejun Zhao, and Chin- Chin-Yew Lin. 2022. Decomposed meta- learning for few-shot named entity recognition. arXiv preprint 22X4:2200..0755..
Xuezhe Ma and Eduard Hovy. 2016. End-to-end sequence labeling via bi-directional lstm-enns-erf. arXiv preprint arXiv:1603.013::
Few-shot NER aims to identify unknown entity classes using only a limited amount of training data, typically employing N-way K-shot learning. In this approach, episodes are iteratively constructed to de- fine N-way K-shot training scenarios. During train- ing, each episode consists of N entity classes, with each class containing K samples. This construc- tion yields the support set, Strain = ((x(6),y ())) No*
Judea Pearl. 2009. Causality. Cambridge university press.
Matthew E Peters, Waleed Ammar, Chandra Bhaga- vatula, and Russell Power. 2017. Semi-supervised sequence tagging with bidirectional language models. arXiv preprint arXiv: Sodiv::775.001000
The query set is formed by sampling Q samples from each of the N classes, resulting in Qirain = I (x(), (y0))))) ) NN It should be noted that Strain n Qtrain =Ø.
Timo Schick and Hinrich Schütze. 2020. It's not just size that matters: Small language models also few-shot learners. arXiv preprint arXiv:2009.071
Jake Snell, Kevin Swersky, and Richard Zemel. 2017. Prototypical networks for few-shot learning. Ad- vances neural information processing systems, 30.
The model is trained based on predictions made on the query set Qtrain during training. During test- ing, a few samples Stest are used, and predictions are made for the corresponding query set Qteat- Similarly, Stest O @. Importantly, the entity
Sebastian Thrun. 1995. Is learning the n-th thing any easier than learning the first? Advances in neural information processing systems, 8.

---

classes present in the test set are not present in the training set, i.c., Yrrai n 28.
Similarly, according the previous causality theorem, we can also remove the do operation on E, we get the final Eq 11.
We expect to block the backdoor path T C-> Y and Pt- C- Y. So, we intervene on 7 and P. First, when we intervene with T, considering C as the confounders are hard to catch. Therefore, we use the front-door adjustment for the calculation. shown in the Eq 5. The intervention for P is similar.
P(Y yldo(T ()) Σ P(E e(T=eTT= =
Σ P(Y - y ()P(T =t) (11)
B
Causal Inference Formula
C Parameter Settings
P(Y v/do(TT = =t)
PPY P(Y vldo(T t), e)
Table 4: The hyperparameters experiments
E P(R eldo(T = =))
(5)
Name
Value
Intervention of E, taking T and as conditions, the conditional probability of constant. So we get the Eq 6.
Batch size
20
Max_lex Max_length
32
Learning rate
le-4
Embedding dimension
768
P(Y y vldo(1 =]) PPY-
P(Y yldo'T = f),
Dropout
0.1
E do( c)) eldo(T' t))
(6)
Given that there is direct causal path between T and E, the do-operation on T can be removed to get the Eq 7.
D Baselline
ProtoBERT (Ding et al., 2021) BERT to get the vector representation of each token, and then averages all vectors of the same type as the the class representation according to the prototype net- work. Finally, calculate the distance between each category representation and query, and judge based on the nearest class.
P(Y yldo(T 1)) PPY
P(Y = v|do(T= f),
E do(E= = c)) c =
(7)
Mcanwhile, considering that there is no direct causal path between T and Y. The do-operation on T will not affect the distribution of Y, so the formula can be further eliminated into Eq 8.
NNShot (Ding et al., 2021) gets the feature repre- sentation for each token and calculates the distance between the query and each representation. Finally, the class is judged based on the nearest distance. StructShot (Ding et al., 2021) adds additional Viterbi decoder to the NNShot.
P(Y TT = 1))
P(Y y =( (8) E
CONTAINER (Das et al., 2021) also uses BERT, and additionally uses contrast learning and Gaussian embedding to get the representation of each token. Then, fine-tuning on the support set and inference using the nearest neighbor method.
Next, apply the total probability formula again to get the Eq 9.
P(Y = y/do =)) =E>> P(Y P(Y = y/do(E c),
ESD (Wang et al., 2021a la) uses inter and cross- span attention based on prototypes to get span rep- resentations. Also, it constructs multi-prototypes for O label.
E
T = L')P(T = 1')do = P(E T
Reusing the previous direct causality between T and E, get the Eq 10.
DecomposedMetaNNR (Ma al., 2022) con- siders the few shot as a sequence labeling problem. MAML is used to initialize the model parameters, and meanwhile uses DecomposedMMttNNER to find the optimal embedding space for entity recognition.
y|oof = =)) = ΣΣ P(Y = [
E T- - ()P(E = = (10)

---

TransferBERT (Hou al., 2020) is a model that applies BERT directly to few-shot sequence labeling through fine-tuning.
6) We only use prototype-based intervention. For 5~ 10-shoh 1) Without sample reweighting, we make context-based intervention and the en- tity detection. 2) We only make a context-based intervention for the basic prototype network. 3) We only use the basic prototype network and do the entity detection.
Matching Net (MN)+BERT (Hou et al., 2020) is a model that combines the Matching Network (MN) approach with BERT for token classification. It is similar to ProtoBERT but utilizes the matching network instead of prototype-basee classification. This approach enhances the token classification task few-shot sequence labeling.
Ma2021 (Ma et al., 2021) approaches sequenee labeling by formulating it as machine reading comprehension problem. They propose method that involves generating specific questions tract slots from the query sentence. This approach allows for effective slot extraction in the sequence labeling task.
L-Ta (Hou et al., 2020) is an ad- vanced model that combines different techniques to improve classification in few-shot sequence label- ing. It uses task-adaptive projection, pair-wise bedding, and collapsed dependency transfer mech- anisms to enhance performance and capture task- specific information.
Retriever (Yu et al., 2021) is retrieval- method that performs classification by finding the most similar example in the support set. It uses retrieval mechanism identify the most relevant support instance and assigns the same label to the query based on this similarity.
ConVEx (Henderson and Vulié, 2020) is model that follows a fine-tuning approach. It is initially pre-trained the Reddit corpus using se- quence labeling objective tasks and then fine-tuned on both the source and target domain annotated data to perform few-shot sequence labeling.
E Ablation study setting
order to evaluate the contribution of the differ- ent components of the proposed method, we per- formed the following baseline as an ablation study: for 1~2-shot, 1) We use the basic prototype net- work and do the entity detection, prototype-based intervention and sample reweighting. 2) Without sample reweighting, we use the basic prototype network and do the entity detection and prototypr- based intervention. 3) and sample reweighting. 4) Without entity detec- tion, we only use the basic prototype network and do sample reweighting and prototype-based inter- vention. 5) We only do the sample reweighting.
the entity detection


Reviews:
[{"review_id": "nBA9AA5Vea", "reviewer": "Reviewer_djTo", "summary": "", "questions": "1.\tHow does the sample reweighting component reflect the expression P(T=t’) in Eq. 2?\n2.\tHow to understand the impact of prototype-based causal intervention on data bias? Could you provide some examples, since I think this is not as intuitive as context-based causal intervention.\n3.\tWhy not use the same baselines on both two datasets?\n4.\tCould you provide the intra-class distance and inter-class distance respectively for the t-SNE visualization in Fig. 4, since I think qualitative analysis is more persuasive.\n5.\tWhat do the horizontal and vertical axes in Figure 5 respectively represent?\n6.\tWhy does causal intervention show much better results when entity detection is performed? What would be the outcome if the entity detection component is added to other baselines?", "limitations": "", "rating": 3, "confidence": 3, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for taking the time to review our paper. We appreciate your insightful comments and valuable feedback, which have undoubtedly improved the quality of our paper. For your review, we would like to address each of your points in detail.\n\n1.The response to 'The novelty is limited. A causal-intervention based framework is commonly used to overcome superficial correlation in datasets, which I think lacks innovation in terms of methodology'.\n\nRegarding this question, we provide the following explanation.\n\n​\t(1)To begin with, causal intervention is indeed widely employed to address spurious correlations within tasks. **However, within the causal framework, the methods proposed in each paper differ due to the varying nature of tasks and causal graphs.** While our approach is rooted in the framework of causal intervention, the diversity in both the underlying causal intervention concepts and the tasks themselves leads to unique methodologies in each study. We also provide references to a few papers grounded in causal intervention below. Despite sharing the foundation of causal intervention, each study's methodology remains distinctive and innovative.\n\n- Yang X, Zhang H, Qi G, et al. Causal attention for vision-language tasks[C]//Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2021: 9847-9857.\n\n- Zhang W, Lin H, Han X, et al. De-biasing distantly supervised named entity recognition via causal intervention[J]. arXiv preprint arXiv:2106.09233, 2021..\n\n- Wu X, Li C, Miao Y. Discovering topics in long-tailed corpora with causal intervention[C]//Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021. 2021: 175-185.\n\n​\t(2) In terms of the innovation of our paper, there are several key aspects to consider. Firstly, in the entity detection part, we depart significantly from traditional prototype networks and existing methods. **We are the first to consider that all entity categories in few-shot named entity recognition should exhibit commonalities that differentiate them from non-entities.** Therefore, we introduce a novel approach of categorizing all entity classes into a single overarching class for input into the prototype network. Furthermore, our introduction of sample reweighting represents a departure from previous works in few-shot named entity recognition, which typically assumed equal weights of 1 for all samples during prototype computation. **We are the first to introduce the consideration of sample weights in this task.**\n\n2.The response to 'In the experiments, the latest LLM, e.g., ChatGPT is not compared. In addition, it can be observed that the role of entity detection in the performance improvement is significant, but its relevance to the perspective of causal intervention in this paper is not particularly strong. Similarly, it is difficult for me to ascertain whether certain implementation details have a significant impact on the model performance, such as incorporating MMD loss into the loss function or performing fine-tuning using the support instances from the test data'.\n\n​\t(1) Below, we list articles and results related to using GPT for few-shot NER tasks:\n\n​\t\t\tAshok D, Lipton Z C. PromptNER: Prompting For Named Entity Recognition[J]. arXiv preprint arXiv:2305.15444, 2023.\n\n（Supplement Table D1: Results in FewNERD-intra dataset 10-way 5-shot)\n\n| F1（%） | FewNERD-intra |\n| ------- | ------------- |\n| GPT3    | 62.33         |\n| GPT4    | 72.63         |\n| Ours    | 61.33         |\n\nThe table indicates that our results are comparable to GPT-3 but fall short of the performance achieved by GPT-4. However, it's important to note that our experimental execution time is significantly shorter when compared to utilizing GPT-4 in each data round. Additionally, our model has a smaller parameter size compared to GPT4, contributing to its efficiency.At the same time, GPT-4 also comes with a high economic cost.\n\n​\t(2) The following table presents the ablation results for each component, illustrating that each component has a substantial impact on the task performance.\n\n(Supplement Table D2): Results in FewNERD-intra dataset. w. and w.o. mean using the component or not.\n\n| F1（%）                 | FewNERD-intra |\n| ----------------------- | ------------- |\n| w.o. context-based      | 60.44         |\n| w context-based         | 70.34         |\n| w.o. prototype-based    | 46.09         |\n| w prototype-based       | 54            |\n| w.o. sample reweighting | 50            |\n| w sample reweighting    | 54            |\n| w.o. entity detection   | 62.8          |\n| w entity detection      | 70.34         |\n\n​\tEntity detection can assist the model in performing initial binary classification on sequences, facilitating more accurate classification in subsequent context-based causal intervention.\n\n​\t(3)Regarding your elaboration on the effects of MMD loss and fine-tuning on the results: **In the 5-shot scenario, we did not include the MMD loss, yet we still achieved promising results.** \n\n(Supplement Table D3): Results in FewNERD-intra and FewNERD-inter dataset 5-way 5-shot.\n\n| F1（%） | FewNERD-intra | FewNERD-inter |\n| ------- | ------------- | ------------- |\n| SOTA    | 63.23         | 74.14         |\n| Ours    | 70.34         | 81.89         |\n\nAdditionally, we extended our experiments in this direction by omitting the MMD loss and adopting the conventional loss. \n\n(Supplement Table D4): Results in FewNERD-intra and FewNERD-inter dataset 5-way 1-shot. w. and w.o. mean using MMD or not.\n\n| F1（%）  | FewNERD-intra | FewNERD-inter |\n| -------- | ------------- | ------------- |\n| SOTA     | 52.04         | 68.77         |\n| w.o. MMD | 52.97         | 69.15         |\n| w. MMD   | 54            | 69.41         |\n\n​\t**It can be observed that the impact of MMD on the results is not significant**.\n\n​\t(4) Concerning fine-tuning, **it's a common practice in the realm of few-shot learning and is often employed in various few-shot named entity recognition tasks**. For example, I have listed below two references that have been accepted by ACL. **It's undeniable that fine-tuning can indeed yield improvements in results. However, our approach combined with fine-tuning led to even greater enhancements.** \n\n- Das S S S, Katiyar A, Passonneau R J, et al. CONTaiNER: Few-shot named entity recognition via contrastive learning[J]. arXiv preprint arXiv:2109.07589, 2021.\n- Ma T, Jiang H, Wu Q, et al. Decomposed meta-learning for few-shot named entity recognition[J]. arXiv preprint arXiv:2204.05751, 2022.\n\n3.The response to 'The paper does not elaborate on the overall architecture and key details of the model. For example, how to bridge the different components and how to perform entity detection are not clearly illustrated. I suggest providing more precise formalized descriptions'.\n\n​\tDue to webpage constraints, it seems that we are currently unable to display the complete model diagram we have prepared. We will include this section in subsequent versions. For now, let me provide a brief description of the overall process. Within each round of data, there are support and query sets. \n\n​\t**In the entity detection component**, for the support data, we aggregate all entity classes into a single major category, denoted as the \"class prototype\" in the diagram. Simultaneously, all non-entity classes are grouped into another major category, corresponding to the \"other prototype\" in the diagram. Subsequently, adhering to the conventional prototype network methodology, we compute prototypes for both of these categories and employ the query data to calculate distances. In this phase, each query yields a two-dimensional outcome, signifying the probabilities for it belonging to the entity and non-entity classes.\n\n​\tFollowing this, **in the causal intervention segment**, for instances with 5-shot, we engage context-based intervention. For 1-shot instances where contextual replacement is unfeasible, we employ prototype-based intervention. Detailed depictions of context-based and prototype-based interventions are provided in the paper with illustrative diagrams. Through intervention, the dimensionality of the outcomes for each query becomes dependent on the number of categories, denoted as N. Resulting from intervention, the probability distribution is obtained for each category.\n\n​\t**In the section involving sample reweighting**, the similarity between each support data and query data is computed. This similarity calculation informs the weight assigned to the support data during prototype computation. Traditional prototype network computation is then conducted using these assigned weights. In this phase, akin to the previous stage, each query generates results with dimensions corresponding to the number of categories, N. \n\n​\tFinally, **during the concatenation of results**, due to the distinct dimensions of outcomes from entity detection compared to the other two components, we replicate the probabilities of entity classes the same number of times as N, mirroring the diagram's depiction. Upon achieving parity in dimensions, the results from the three components are summed element-wise to yield the overall outcome.\n\n**Q1.** The response to 'How does the sample reweighting component reflect the expression P(T=t’) in Eq. 2'.\n\n​\tIn this equation, T represents the context, and T=t' signifies the probability corresponding to different contextual scenarios. We posit that distinct contexts exert varying influences on the outcome. Therefore, we endeavor to assign differing levels of importance to different contexts. To achieve this, we leverage the similarity between the current sample and the query as a weighting factor, aimed at distinguishing the significance of various contexts in prototype computation.\n\n**Q2.** The response to 'How to understand the impact of prototype-based causal intervention on data bias? Could you provide some examples, since I think this is not as intuitive as context-based causal intervention'.\n\n​\tIn the results analysis section of the paper, titled \"How does prototype-based causal intervention improve entity recognition,\" you provide examples to illustrate the impact of prototype-based intervention. Due to the webpage's configuration, it appears that we are unable to display the properly formatted example diagram at the moment. We will incorporate this section in the upcoming versions. Here, let me briefly describe the illustration. Regarding the term \"German,\" if it is included in the current support data and categorized as the \"O\" class, then in the context of the current query data, a conventional prototype network would classify it as the \"O\" class as well. This illustrates that the model is only fitting to the immediate data and is forgetting previous data. However, when we incorporate prior knowledge, if the term \"German\" appeared in support data in previous rounds and was labeled as the \"other-language\" class, the model can correctly classify it as the \"other-language\" category. This is due to the fact that the previous prototypes retain relevant feature knowledge, which we reintroduce into the current scenario.\n\n**Q3.** The response to 'Why not use the same baselines on both two datasets‘.\n\tThe issue with the baseline models in the context of the datasets arises from the disparate timeframes of their introduction. FewNERD dataset was introduced at a later point. Consequently, the baseline model count is lower for FewNERD compared to the other dataset. We conducted comparisons based on the initial state-of-the-art (SOTA) baselines available on Paperwithcode. Additionally, certain baseline models were exclusively tested on one of the datasets, and we directly referenced these outcomes.\n\n**Q4.** The response to 'Could you provide the intra-class distance and inter-class distance respectively for the t-SNE visualization in Fig. 4, since I think qualitative analysis is more persuasive‘.\n\n​\t I'm not quite clear on whether you're referring to the intra-class and inter-class distances in terms of vector representations from the FewNER-intra dataset and the FewNERD-inter dataset, or if you mean the distances within each class and the distances between different classes in the vector representations obtained from any given dataset.\n\n​\t(1) If you are referring to the first scenario, involving vector representations from the FewNER-intra dataset and the FewNERD-inter dataset: We have provided the vector representations from the FewNERD-intra dataset in the paper. Here, we are presenting the vector representations from the FewNERD-inter dataset. (Due to webpage constraints, it seems that we are unable to display the completed diagrams. We will include this section in future versions. Here, I will provide a brief description of the diagrams.) The five color categories in the diagram represent five different classes. It can be observed that in a regular network, the classes are scattered and intertwined, making separation difficult. However, upon applying our method, the boundaries between classes become distinct, and instances of the same class start to cluster together.\n\n​\t(2) If your query pertains to the second scenario, involving distances within each class and distances between different classes in the vector representations obtained from any given dataset: We have already presented the vector representations from the FewNERD-intra dataset in the paper. The five colors represent five distinct classes. It can be noted that distances within the same color group have become closer, and there are evident boundaries between different colors.\n\n**Q5.** The response to 'What do the horizontal and vertical axes in Figure 5 respectively represent'.\n\n​\tRegarding this matter, I'd like to provide some further elucidation. Figure 5 manifests as a histogram, serving the purpose of delineating the congruity within data distribution across the task.  **Conventionally, the horizontal axis signifies distinct data types, while the vertical axis encapsulates the corresponding distribution magnitude. **\n\n**Q6**. The response to 'Why does causal intervention show much better results when entity detection is performed? What would be the outcome if the entity detection component is added to other baselines'.\n\n​\t (1) To begin with, our introduced entity detection indeed has significantly contributed to enhancing the task. It enables the model to identify non-entity tokens. Under this premise, causal intervention can manifest its effectiveness more profoundly. **In traditional prototype networks, they encompass two responsibilities: distinguishing entities from non-entities and further categorizing entities into specific classes. However, our entity detection solely requires the segregation of entities from non-entities, deferring the specific classification to subsequent stages of the task.** Thus, entity detection aids the model in preliminary screening and filtering, ultimately leading to improved performance.\n\n​\t(2) The incorporation of entity detection into other baselines isn't unprecedented. **Prior works have explored the concept of detecting entities and subsequently categorizing them, often termed as \"span\" methods. We provide references to two such studies below. Evidently, incorporating span-based approaches can elevate model performance, yet our results surpass the state-of-the-art (SOTA).** This substantiates the efficacy of our proposed causal intervention method.\n\n- Wang P, Xu R, Liu T, et al. An enhanced span-based decomposition method for few-shot sequence labeling[J]. arXiv preprint arXiv:2109.13023, 2021.\n\n- Ma T, Jiang H, Wu Q, et al. Decomposed meta-learning for few-shot named entity recognition[J]. arXiv preprint arXiv:2204.05751, 2022\n\n\n\n​\tFinally, we have provided detailed explanations for your suggestions and questions. We sincerely appreciate your valuable feedback and inquiries, as they have helped us enhance our paper. We hope that our responses have addressed your concerns. If there are any remaining points that we haven't clarified, please feel free to ask again.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. This paper focuses on the problem of data bias and model generalization in the few-shot NER task, which is a meaningful topic.\n2. The experimental results show the significant improvement over state-of-the-art baselines.", "reasons_to_reject": "1. The novelty is limited. A causal-intervention based framework is commonly used to overcome superficial correlation in datasets, which I think lacks innovation in terms of methodology.\n\n2. In the experiments, the latest LLM, e.g., ChatGPT is not compared. In addition, it can be observed that the role of entity detection in the performance improvement is significant, but its relevance to the perspective of causal intervention in this paper is not particularly strong. Similarly, it is difficult for me to ascertain whether certain implementation details have a significant impact on the model performance, such as incorporating MMD loss into the loss function or performing fine-tuning using the support instances from the test data.\n\n3. The paper does not elaborate on the overall architecture and key details of the model. For example, how to bridge the different components and how to perform entity detection are not clearly illustrated. I suggest providing more precise formalized descriptions.", "questions_to_authors": "1.\tHow does the sample reweighting component reflect the expression P(T=t’) in Eq. 2?\n2.\tHow to understand the impact of prototype-based causal intervention on data bias? Could you provide some examples, since I think this is not as intuitive as context-based causal intervention.\n3.\tWhy not use the same baselines on both two datasets?\n4.\tCould you provide the intra-class distance and inter-class distance respectively for the t-SNE visualization in Fig. 4, since I think qualitative analysis is more persuasive.\n5.\tWhat do the horizontal and vertical axes in Figure 5 respectively represent?\n6.\tWhy does causal intervention show much better results when entity detection is performed? What would be the outcome if the entity detection component is added to other baselines?", "paper_topic_and_main_contributions": "This paper aims to solve the overfitting problem (i.e., superficial correlation resulting from the data bias) of the few-shot NER task. It proposes a causal intervention framework to block the backdoor between the context and the label. Specifically, the authors introduce the components including entity detection, context-based intervention and sample reweighting. For the one-shot scenario, they replace the context-based intervention by a prototype-based one considering the lack of additional samples. The experiments on two datasets also demonstrate the effectiveness of the proposed method.", "excitement": 3, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {"Typos_Grammar_Style_and_Presentation_Improvements": "1.\tThe sentence “The Softmax and MMD in Eq 3.” in Line 404 seems to have a grammatical error.\n2.\tThere is a typo ”moethods” in Line 462.\n3.\tSome notations in Eq. 3 and Eq. 4 appear a bit abrupt, e.g., f."}}, {"review_id": "ohq2fq8d9U", "reviewer": "Reviewer_56nu", "summary": "", "questions": "1. In the tables of results, do +- indicate standard deviations across different random seeds?\n2. There seem to be 2 mistakes in Table 2: on the Mu and Se domains, the model with highest score is L-TapNet+CDT and not the model proposed in this paper, is this correct? \n3. Could you please address the issues from Reasons to Reject?\n", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "1.The response to 'The authors propose a causal model, in Fig 1, that aims to show the dependencies between predicted label, target entity, context, and the confounder (spurious factor due to sample selection bias). However, the way how a model captures the dependencies between such variables may not be causal. As a matter of fact, current models are statistical machines that capture statistical correlations. It is not clear if the models used here (prototype networks) are causal or not as it is not mentioned. Furthermore, even if the model could capture causal dependencies, that doesn't mean it is actually modeling such dependencies as proposed in Fig 1; i.e., there is no rationale, explanation, or supportive evidence for the validity of such a causal model.'\n\n​\t(1) First, I will explain the entire process of causal reasoning. In causal inference, **there is a very classic work**: Judea Pearl's \"The Book of Why,\" which includes a detailed diagram of the causal inference process. **In summary, researchers propose hypotheses based on background knowledge and construct causal models.** For issues that can be answered by the model, i.e., those that can be mathematically formulated, statistical estimation methods (commonly employed in machine learning) can be used. If the model cannot answer the question, i.e., if it cannot be formalized, researchers return to the hypothesis stage and reconstruct the causal model. In this process, hypotheses correspond to mathematical deduction, meaning that under a set of valid axioms, all conclusions drawn through mathematical deduction are certain to be correct. Model superiority stems from selecting the correct prior hypotheses—hypotheses that align with the real world. Additionally, in this process, statistical estimation corresponds to mathematical induction, which is the basis of contemporary machine learning. Based on this process, we proposed hypotheses about the few-shot named entity recognition task, drawing on our background knowledge and understanding. We hypothesize that sample selection bias exists in the task, causing false correlations between the current context and label in each training round. We provide corresponding examples in Figure 1 to illustrate these erroneous correlations. **Building on this hypothesis, we constructed a causal graph and mathematically defined the task (as in Equation 1) and carried out mathematical reasoning (as in Equation 2 and Appendix B). Subsequently, we employed statistical methods to calculate and estimate the probabilities in the equations, resulting in our causal method.** \n\n​\t(2) Following this process, we formally defined the task, provided a mathematical definition based on the causal graph, and derived the equations using causal inference theorems. Therefore, we believe our hypotheses are correct. **In Appendix B, we offer derivations under the causal inference theorem to support our theory.** Simultaneously, our method, based on this hypothesis and causal approach, achieved much higher results than the SOTA, which seems to further substantiate the effectiveness of our method. \n\n​\t(3) Below, we list several published articles on causal inference. Most of these works are based on task understanding, leading to the formulation of their hypotheses and construction of causal models. \n\n  - Niu Y, Tang K, Zhang H, et al. Counterfactual vqa: A cause-effect look at language bias[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2021: 12700-12710.\n  - Yang X, Zhang H, Qi G, et al. Causal attention for vision-language tasks[C]//Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2021: 9847-9857.\n\n- Wang W, Feng F, He X, et al. Clicks can be cheating: Counterfactual recommendation for mitigating clickbait issue[C]//Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval. 2021: 1288-1297.\n\n\n\n​\tOnce again, I appreciate your inquiry. In reality, while causal inference hypotheses and interventions have indeed propelled many tasks forward, the reliability of causal inference does present certain challenges, requiring more research to establish its credibility. We are continuously striving to provide more substantial explanations and evidence. I hope my response addresses your concerns.\n\n2.The response to 'The section on causal inference is too short and there is no deep elaboration on the connection between this theory and the work proposed in this paper. For example, under which grounds the causal model from Fig 1 is faithful to causal inference? (On the other hand, the authors could clearly eliminate many paragraphs from the paper that are highly repetitive.)\n\tWe will further expand the section on causal inference in the subsequent versions and remove redundant content. I'd like to provide a detailed explanation here to clarify the connection between the theory of causal inference and our work. \n\n​\t(1) Regarding the causal graph and Equation 1: In this paper, we start by analyzing the task of few-shot named entity recognition. We have elaborated on this task extensively in the fourth reason for rejection. Through an analysis of the entire process of few-shot named entity recognition, we propose that in each round of sample selection, due to the model's focus on a small subset of the current samples, overfitting to the present data might occur. To address this, we introduce causal intervention to block this path. Building upon this causal graph, we formulate Equation 1. The inspiration for Equation 1 stems from the common concept of front-door adjustment in causal inference. We provide the reference to previous works employing front-door adjustment in the text.\n\n  - Yang X, Zhang H, Qi G, et al. Causal attention for vision-language tasks[C]//Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2021: 9847-9857.\n\n​\t\t(2) **Addressing the faithfulness of the causal model in the graph**: In Appendix B, we present a derivation of Equation 1. The derivation relies on established theorems in causal inference. As long as these theorems are satisfied, the entire derivation process is valid and applicable to both the causal model and causal inference. Allow me to elaborate on the reasoning process. We provide a mathematical definition based on the causal graph, as demonstrated in Equation 1, also shown as Equation 5 in Appendix B. This equation is derived from the law of total probability. By analyzing the structure of the graphical model and assessing variable independence or conditional independence, we deduce that in the path T->E->Y, with T and E as conditions, Y remains unchanged. Therefore, P(Y=y|do(T=t),do(E=e)) and P(Y=y|do(T=t),E=e) are identical. Hence, Equation 5 can be deduced into Equation 6. Moreover, since the causal effect of T->E can be directly obtained from the data without confounding or backdoor paths, it can be derived directly. Consequently, P(E=e|do(T=t)) and P(E=e|T=t) are equivalent, resulting in the derivation of Equation 7. Further, due to the absence of a direct edge between T and Y, intervening on X does not affect Y. Consequently, P(Y = y|do(T = t), do(E = e)) and P(Y = y|do(E = e)) can be deduced as Equation 8. By employing the law of total probability once more and summing over all possible values of T, we obtain Equation 9. Lastly, based on the aforementioned explanation, P(T = t′|do(E = e)) and P(T = t′) are equivalent, resulting in Equation 10. Ultimately, based on the prior explanation, in the path T->E->Y, with T and E as conditions, Y remains unchanged. Therefore, P(Y = y|do(E = e), T = t′) and P(Y = y|E = e, T = t′) are equivalent, leading to Equation 11, also known as Equation 2 in the main text.\n\n​\t(3) **Establishing the connection between our proposed method and Equation 2**: Allow me to offer a comprehensive explanation below. After undergoing causal inference, Equation 2 no longer contains do-operations. Each component of the equation necessitates statistical methods from machine learning to obtain results. \n\n​\t**For the first component, P(E = e|T = t)**, we need to determine the probability of entity e given the context t. Viewing e within its own context renders it a complete, to-be-predicted sample, aligning with the conventional task of prototype-based networks where e corresponds to a specific entity category. This is why we initially employed traditional prototype-based networks to address this part. However, this approach yielded suboptimal results. Upon revisiting this component, we deemed it more appropriate to treat e as a binary classification task rather than associating it directly with specific categories. **We introduced an entity detection method based on this premise.**\n\n​\t **As for the second component, P(Y = y|E = e, T = t′)**, this signifies the probability of label Y given the current entity e and varying context t'. As t' needs to be iterated through, we maintained the entity e constant while considering the diverse contexts of same-type entities. **This notion gave rise to our context-based causal intervention.** Given the presence of two few-shot settings, namely 5-shot and 1-shot, where 5-shot entails each category having 5 sentences, allowing replacement of same-type contexts, and 1-shot consisting of only one sentence per category with no additional context, we proposed prototype-based causal intervention as an alternative for the latter. Rather than substituting same-type contexts, we considered same-type prototypes. We retained prototypes generated from each round and subsequently combined them when encountering the same-type prototypes. **This approach led to prototype-based causal intervention.** \n\n​\t**Regarding P(T = t′)**, where t' corresponds to context and context influences prediction, we assert that since context impacts prediction, different contexts should not default to equal weights when calculating prototypes. We computed the similarity between the vector representations of support and query data, then assigned distinct weights to samples for prototype computation. **This reasoning underpins the introduction of sample reweighting**. \n\n​\tTo conclude, by leveraging our understanding of the task's background knowledge, we formulated hypotheses concerning the causal graph. Subsequently, we mathematically defined the task based on the causal graph and employed causal inference to deduce the equation. Lastly, we proposed different methods to statistically estimate the probability outcomes of various components within the equation.I hope this detailed explanation addresses your queries.\n\n3.The response to 'Derived from the point above, there is no experimental proof that supports the causal model proposed in this paper. Is the sample selection bias tested? Is there really a causal connection for the form T <- C -> Y? Could the confounder be due to some other biases?\n\n​\t(1) **Regarding the issue of sample selection bias**, our initial skepticism arises from the nature of the few-shot named entity recognition task itself. In a few-shot task, such as a 5-way 5-shot setup, each round of data comprises five classes, with each class containing five sentences. This results in a total of 25 sentences per round (5 × 5 = 25). To learn class-specific features comprehensively, the model tends to heavily fit the patterns within these 25 sentences. The scarcity of samples can lead to overfitting on the current data. If the 25 chosen sentences in a round exhibit significant biases, like the repetition of certain words, the model might inaccurately correlate these occurrences with labels. We are thus concerned that the biased selection of samples could lead to such modeling inaccuracies. To demonstrate this point, we provide an illustration in Figure 1. It can be observed that the presence of the term \"square\" could spuriously link it to the label \"animal.\" While there might be some connection between \"square\" and \"pigeon,\" there is no inherent link between \"square\" and \"animal.\" Additionally, our context-based intervention method, which replaces contexts to alleviate the impact of sample selection bias, results in a notable improvement in accuracy. This suggests that sample selection bias indeed affects the model and our approach effectively addresses it. \n\n​\t(2) **Regarding the depicted relationships in the causal graph**, we have provided a detailed explanation of the causal inference process in your first rejection reason. We simply hypothesize the presence of such a relationship in the real task, and then mathematically define and deduce it based on the causal graph, employing statistical methods for calculation. The entirety of the formula derivation process serves as our theoretical foundation, and the ultimate results validate the effectiveness of our approach. \n\n​\t(3) **Regarding the confounding factors**, in fact, the model encompasses numerous confounding factors, many of which might remain latent or undefined. The sample selection bias that we address in the paper is just one of these factors. We specifically tackle this confounding factor to improve the model and achieve positive outcomes. Future researchers in the field of few-shot named entity recognition are likely to unearth additional factors influencing the model and further enhance its performance. We also aspire to identify more of these confounding factors in future endeavors.\n\n4.The response to 'The mechanism proposed is derived from Eq. 1, which is unclear if this equation is faithful to the theory of causal inference (no citation nor explanation is given).'\n\tRegarding Eq 1, it is a common form of front-door adjustment often used in causal inference. You can find information about front-door adjustment in \"Causal Inference in Statistics: A Primer\" by Judea Pearl, Madelyn Glymour, and Nicholas Jewell, Wiley, 2016, or in \"Causality: Models, Reasoning and Inference\" by Judea Pearl, Cambridge University Press, 2009. Additionally, in the field of computer science, researchers have explored the use of front-door adjustment, as demonstrated in the paper: \n\n - Yang X, Zhang H, Qi G, et al. Causal attention for vision-language tasks[C]//Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2021: 9847-9857.\n\n5.The response to 'The mechanism proposed consists of some factors, some of which are slightly unclear. For example, for the factor called prototype-based intervention, the intervention done to block the effect of the confounder on the prediction needs the signal from the previous prototype vector. It is unclear what is the previous prototype, what is the exact operation of this intervention, and what is the rationale behind such an intervention. Similarly, for the factor called sample reweighting, it is unclear the logic behind reducing the distribution difference between training and test support data (based on common sense, this operation seems, actually, undesirable since test data should not be used at training time, only at test time).'\n\tI will first explain the entire process of prototype-based few-shot named entity recognition (NER) using the prototype network, then provide a detailed explanation of the prototype-based intervention process and related details, and finally clarify the details of the sample reweighting approach.\n\n​\t(1) In **the process of few-shot NER**, data is divided into training, validation, and test sets. For clarity, let's temporarily disregard the validation set. Both the training and test sets have support data and query data. The support data from the training set is denoted as Strain, and the query data from the training set is denoted as Qtrain. The support data from the test set is denoted as Stest, and the query data from the test set is denoted as Qtest. During training, the model uses Strain to identify the current set of categories and learn their features, then predicts Qtrain data to optimize the model. During testing, the model uses Stest to identify the current set of categories and predicts Qtest data. The obtained loss is not used for model optimization but serves as the final accuracy. In this process, **both the support data from the training set and the support data from the test set are visible to the model to know which categories are present**. The query data from the training set is used for loss prediction and model optimization, while the query data from the test set is only used for prediction. The prototype-based few-shot NER approach refers to training the model's features using Strain (support data from the training set) based on a prototype network. The model averages the vectors of all entities in the same category to create a prototype representation for that category. During query data prediction, the model calculates the distance between the query and all category prototypes, classifying the query into the category with the nearest prototype.\n\n​\t(2) **Our prototype-based intervention involves saving the prototype vectors from each round and using them together in subsequent rounds when encountering the same category.** This means that two prototypes are averaged to obtain a vector representation, rather than considering only the prototype from the current round's data. This intervention stems from the observation of forgetting and overfitting phenomena in the task, as elaborated in the \"How does prototype-based causal intervention improve entity recognition\" section of the results analysis. If a word (such as \"German\") is present in the support data of the current round and assigned the category \"O,\" there is a high probability that the model will classify it as \"O\" in the query data of the current round due to overfitting the support data. However, the correct category for this query data might be \"other-language,\" leading to a model error. Fortunately, in previous rounds, \"German\" appeared as \"other-language.\" When we introduce the prototype of the \"other-language\" category from previous rounds to the current round, it helps the model correctly classify \"German.\"\n\n​\t(3) **Regarding sample reweighting**, the traditional prototype network averages all entities in the same category to create a prototype vector as the representation. However, we believe that each entity's contribution to the prototype should not be a default value of 1. If an entity is more similar to the query data, its contribution should be greater. Therefore, we calculate the similarity between the support and query data in the test set, then assign higher weights to the support data with higher similarity when calculating prototypes. It's important to emphasize that we calculate the similarity between support and query data using their vector representations, without considering their category labels, so there is no issue of visibility here. In subsequent prototype calculations, support data with different weights are used to calculate prototypes, and since the support data's weights are calculated based on similarity, it aids in achieving a better match between the distributions of support and query data. This reduces distribution discrepancies, enabling the model to better predict the knowledge learned from support data to query data. This approach aims to enhance model performance by aligning the distributions more effectively.\n\n\n\n**Q1.** In the tables of results, do +- indicate standard deviations across different random seeds?\n\tThe observed discrepancies in the results are attributed to the influence of random seed initialization. Indeed, these fluctuations stem from the random seed settings, thereby impacting the outcome. Moreover, to ensure the reproducibility and non-random nature of the outcomes, we conducted multiple experimental runs and subsequently calculated the average values.\n\n**Q2.** There seem to be 2 mistakes in Table 2: on the Mu and Se domains, the model with highest score is L-TapNet+CDT and not the model proposed in this paper, is this correct?\n\tTable 2 is indeed accurate. The experimental findings highlight that our outcomes might not be optimal for these two specific categories. However, considering the results across the other categories and amalgamating them, we presented the \"Avg\" figure. **On the aggregate scale, while these two categories may exhibit slight deficits, the overall average surpasses the state-of-the-art (SOTA) benchmarks.** This substantiates the efficacy of our proposed approach.     \n\n - Yang X, Zhang H, Qi G, et al. Causal attention for vision-language tasks[C]//Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2021: 9847-9857.    \n - Niu Y, Tang K, Zhang H, et al. Counterfactual vqa: A cause-effect look at language bias[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2021: 12700-12710.    \n - Chang C H, Adam G A, Goldenberg A. Towards robust classification model by counterfactual and invariant data generation[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2021: 15212-15221.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1.\tThe manuscript is clearly written\n2.\tMost of the results obtained surpass those from the literature, and in some cases by a wide margin.\n3.\tThe solution seems to be computationally feasible and derivation of the mechanism proposed is simple\n4.\tAblation experiments are provided to see the effect of each component in the proposed mechanism\n", "reasons_to_reject": "There are several aspects that remain unclear:\n1. The authors propose a causal model, in Fig 1, that aims to show the dependencies between predicted label, target entity, context, and the confounder (spurious factor due to sample selection bias). However, the way how a model captures the dependencies between such variables may not be causal. As a matter of fact, current models are statistical machines that capture statistical correlations. It is not clear if the models used here (prototype networks) are causal or not as it is not mentioned. Furthermore, even if the model could capture causal dependencies, that doesn't mean it is actually modeling such dependencies as proposed in Fig 1; i.e., there is no rationale, explanation, or supportive evidence for the validity of such a causal model.\n2. The section on causal inference is too short and there is no deep elaboration on the connection between this theory and the work proposed in this paper. For example, under which grounds the causal model from Fig 1 is faithful to causal inference? (On the other hand, the authors could clearly eliminate many paragraphs from the paper that are highly repetitive.) \n3. Derived from the point above, there is no experimental proof that supports the causal model proposed in this paper. Is the sample selection bias tested? Is there really a causal connection for the form T <- C -> Y?  Could the confounder be due to some other biases?\n4. The mechanism proposed is derived from Eq. 1, which is unclear if this equation is faithful to the theory of causal inference (no citation nor explanation is given).\n5. The mechanism proposed consists of some factors, some of which are slightly unclear. For example, for the factor called prototype-based intervention, the intervention done to block the effect of the confounder on the prediction needs the signal from the previous prototype vector. It is unclear what is the previous prototype, what is the exact operation of this intervention, and what is the rationale behind such an intervention. Similarly, for the factor called sample reweighting, it is unclear the logic behind reducing the distribution difference between training and test support data (based on common sense, this operation seems, actually, undesirable since test data should not be used at training time, only at test time).", "questions_to_authors": "1. In the tables of results, do +- indicate standard deviations across different random seeds?\n2. There seem to be 2 mistakes in Table 2: on the Mu and Se domains, the model with highest score is L-TapNet+CDT and not the model proposed in this paper, is this correct? \n3. Could you please address the issues from Reasons to Reject?\n", "paper_topic_and_main_contributions": "This paper proposes a mechanism for prototypical networks to help improve the correct labeling of entities in few-shot scenarios. It is claimed that this mechanism, a causal intervention, is based on causal analysis. The main aim of this mechanism is to remove a spurious dependency between context and predicted entity label (it is claimed that this spurious entity is due to a sample selection bias). The main contribution of this paper is to improve F1 scores on two datasets, namely Few-NERD (INTRA (1~2-shot, 5~10-shot), INTER(1~2-shot, 5~10-shot) and SNIPS (1-shot, 5-shot). Most of the results obtained surpass results from previous works.", "excitement": 4, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {"Typos_Grammar_Style_and_Presentation_Improvements": "It would be nice if the paper were less repetitive about the main point of the sample selection bias and showed more examples of the problem and how the mechanism proposed work on instances from the datasets used."}}, {"review_id": "28b5UnYZee", "reviewer": "Reviewer_c6Vr", "summary": "", "questions": "A. In Section 1, the author says “However, these existing methods tend to overlook the issue of overfitting caused by spurious correlation in few-shot tasks.” What elements are involved in this spurious association?\n \nB. In Section 4, The author only used GPT for data augmentation on the 1-shot setting, so what is the effect of using GPT for data augmentation on the 5-shot setting?\n \nC. In Section 4.2, the author describes utilizing the Bert-base-uncased model as the base model. If the author replaces Bert with Roberta and others pre-trained language models, will better results be achieved?\n\nD. In Section 4.7, the author just describes the word “German”, it is best to use visualization to demonstrate this case. And need give more description to analyze the reason.\n\nE. Besides the pre-trained method, will it effectively improve the accuracy of other models if the papers’ method is applied to other few-shot NER methods?\n", "limitations": "", "rating": 3, "confidence": 5, "soundness": 2, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "1.The response to 'Lack of case study analysis for more intuitive display‘.\n\n\n\n​\tIn the results analysis section of the paper titled \"How does prototype-based causal intervention improve entity recognition,\" we provide examples to illustrate the impact of our proposed method. **For further clarity, we have included an explanatory illustration below.**\n\n\n\n​\tDue to the webpage's configuration, it appears that we are unable to display the properly formatted example diagram at the moment. We will incorporate this section in the upcoming versions. Here, let me briefly describe the illustration. Regarding the term \"German,\" if it is included in the current support data and categorized as the \"O\" class, then in the context of the current query data, a conventional prototype network would classify it as the \"O\" class as well. This illustrates that the model is only fitting to the immediate data and is forgetting previous data. However, when we incorporate prior knowledge, if the term \"German\" appeared in support data in previous rounds and was labeled as the \"other-language\" class, the model can correctly classify it as the \"other-language\" category. This is due to the fact that the previous prototypes retain relevant feature knowledge, which we reintroduce into the current scenario.\n\n\n\n​\tIn addition to the above, to facilitate better understanding, **we have also provided additional examples to illustrate the functioning of the model. You can find these extra examples in our responses to the \"Questions For The Authors-D\" section that follows.**\n\n\n\n2.The response to 'The author only used GPT for data augmentation on the 1-shot setting, so what is the effect of using GPT for data augmentation on the 5-shot setting'.\n\n\n\n​\t(1) To address your first question, the decision not to employ GPT-based data augmentation for the 5-shot setting stems from the fact that in 5-shot scenarios, there are already five sentences for each category. This constitutes a substantial amount of data. When applying context-based intervention in the 5-shot setup, the number of sentences increases significantly. Each sentence for a category can be swapped with the other four sentences within the same category, yielding a total of 25 sentences. **This dataset volume is already substantial for the model to learn from, eliminating the necessity for additional data augmentation.** **Moreover, our model has achieved excellent results in this context, affirming our decision not to employ GPT augmentation for the 5-shot scenario.**\n\n\n\n​\t(2) **However, your concerns are valid and insightful. Therefore, we have taken the time to conduct supplementary experiments.** Due to time constraints, we've performed these experiments solely on the FewNERD-inter dataset for the 5-way 5-shot scenario. \n\n（Supplement Table D1: Results in FewNERD-intra dataset 5-way 5-shot)\n\n| F1（%）  | FewNERD-intra |\n| -------- | ------------- |\n| SOTA     | 59.4          |\n| Ours     | 70.34         |\n| Ours+GPT | 71.08         |\n\n​\t**This indeed confirms the effectiveness of utilizing GPT for data augmentation. This clarification underscores our commitment to comprehensive experimentation and validation of our proposed approach.**\n\n\n\n3.The response to 'In Section 4.6, the author did not compare the effectiveness of using the sample reweighting method under the 5-shot setting'.\n\n​\t**We did not employ sample reweighting in the 5-shot scenario. The reasons for not using it in the 5-shot context can be summarized as follows**:\n\n\n\n​\t(1) In the 5-shot setting, we utilized context-based intervention, where each sample's context was replaced with the context of other samples within the same category. This expansion resulted in an increased number of sentences per category, growing from 5 to 25 sentences. **This augmented dataset requires more memory to handle**. (It's worth noting that while the number of samples increased significantly, our sentence maximum length was set at 32, not the 128 used in SOTA methods, which mitigates memory challenges.) However, if we were to apply sample reweighting after expanding to 25 sentences, it would demand an even larger memory footprint. Therefore, we chose not to employ sample reweighting in the 5-shot setting and instead focused on scenarios with fewer samples, such as 1-shot tasks.\n\n\n\n​\t(2) We did experiment with combining context-based intervention and sample reweighting in the 5-shot setting on cards with larger memory capacity. **This did yield a modest improvement of 1-2 points in performance. However, the memory consumption was substantial, requiring a transition from 32GB to 80GB cards.** Although the 80GB cards were not fully utilized, the memory overhead remained significant. Given the marginal gain of 1-2 points in comparison to the extensive memory expenditure, we chose to forgo sample reweighting in the 5-shot scenario.\n\n\n\n4.The response to 'In Section 4.7, the author just describes the word “German”, it is best to use visualization to demonstrate this case'.\n\n\n\n​\t**Due to the webpage constraints, it seems that we are unable to display the prepared example images. We will incorporate this section in the subsequent versions.** **Here, I'll briefly describe the images.** For the term \"German,\" in the current set of support data, it is categorized as class \"O.\" Consequently, in the current query data, if the term \"German\" appears, a conventional prototype network would classify it as class \"O\" as well. This illustrates that the model is only fitting to the current data and forgetting the previous data. However, when we integrate prior knowledge, in previous rounds, the support data had instances of the term \"German\" labeled as the \"other-language\" class. This enables the model to accurately classify it as the \"other-language\" category when it appears in the query data. This effect is due to the prior prototypes retaining pertinent feature knowledge, which is then incorporated into the present context.\n\n\n\n5.The response to 'The paper needs further polishing and revision: There are many formatting errors and inconsistent correspondence before and after the paper'.\n\n\n\n​\tI appreciate your feedback, and we will certainly undertake thorough revisions and optimizations of both the grammar and content of the paper in subsequent versions. For the finalized published paper, we plan to enlist the assistance of native professional experts to further refine and polish the writing.\n\n\n\n**QA.** The response to 'A. In Section 1, the author says “However, these existing methods tend to overlook the issue of overfitting caused by spurious correlation in few-shot tasks.” What elements are involved in this spurious association'.\n\n\n\n​\tI appreciate your inquiry. In this context, we present an example as depicted in the figure below, which is also Figure 1 in the paper. In this example, \"pigeons\" belongs to the \"animal\" category, which is correct. However, in the same sentence, \"pigeons\" is situated closely to \"square.\" This proximity may lead the model to erroneously associate \"square\" with the \"animal\" category during feature representation and classification. **While this phenomenon has a limited impact in larger datasets due to the volume of samples, it becomes more pronounced in few-shot scenarios. With fewer samples, the model overfits to this limited data, making such errors more conspicuous.**This association is misleading – not all entities mentioned alongside \"square\" should be categorized as \"animal.\" **Therefore, we contend that a spurious correlation exists between context and labels, manifesting this false association.** \n\n\n\n**QB**.The response to 'B. In Section 4, The author only used GPT for data augmentation on the 1-shot setting, so what is the effect of using GPT for data augmentation on the 5-shot setting'.\n\n\n\n​\tThank you for sharing your concerns, and we truly understand your queries. Let me explain why we didn't utilize GPT for 5-shot tasks and then provide the additional results we've obtained by using GPT for 5-shot settings.\n\n\n\n​\t(1) Initially, the rationale behind not employing GPT data augmentation for 5-shot tasks is due to the fact that each category already has 5 sentences in the 5-shot scenario. When context-based intervention is applied in 5-shot tasks, the number of sentences increases significantly. Each sentence for a category can be swapped with the other four sentences within the same category, resulting in a total of 25 sentences. **This dataset volume is already substantial for the model to learn from, making the introduction of additional augmented data unnecessary. Furthermore, we've achieved excellent results in this context, which is why we didn't adopt GPT augmentation for 5-shot tasks.** To further clarify, although our method increases the number of sentences and might appear to consume more memory, our sentence maximum length is only 32, yet we've achieved remarkable outcomes. Unlike current SOTA methods that require a maximum length of 128, we've significantly conserved memory in this regard.\n\n\n\n​\t(2) However, your concerns are indeed insightful. Therefore, we've taken the opportunity to conduct supplementary experiments. Due to time constraints, we've exclusively conducted these experiments on the FewNERD-intra dataset for the 5-way 5-shot scenario. You can see the Supplement Table D1 in response 2. This comprehensive clarification underscores our commitment to rigorously exploring different aspects of our approach and validating its effectiveness.\n\n\n\n**QC.** The response to 'C. In Section 4.2, the author describes utilizing the Bert-base-uncased model as the base model. If the author replaces Bert with Roberta and others pre-trained language models, will better results be achieved'.\n\n\n\n​\tThank you for your insights, and your questions have indeed provided us with valuable perspectives. In the realm of vector representations, it's worth noting that BERT serves as a classic model but is interchangeable with other models. Due to time constraints, **our experimentation focused on Roberta**. \n\n（Supplement Table D2: Results in FewNERD-inter dataset 10-way 1-shot)\n\n| F1（%） | FewNERD-inter |\n| ------- | ------------- |\n| SOTA    | 60.47         |\n| Ours    | 60.27         |\n\n​\t We preliminarily attribute this observation to the model's focus on data fitting. During the training process, both BERT and Roberta's parameters are fine-tuned, and across multiple training rounds, these parameters tend to converge towards the dataset, yielding relatively similar final vector representations.\n\n\n\n**QD**.The response to 'D. In Section 4.7, the author just describes the word “German”, it is best to use visualization to demonstrate this case. And need give more description to analyze the reason‘.\n\n\n\n​\tI appreciate your inquiries. Allow me to elucidate the purpose of introducing prototypes and subsequently provide more examples for illustration.\n\n\n\n​\t(1) **For the example of \"German.\"** In few-shot named entity recognition tasks, the model learns from the current support data and predicts the category of entities in the query data. Given the minimal sample size per round in few-shot scenarios, the model tends to overfit the current data, leading to certain erroneous behaviors. When a word that exists in both support and query data is categorized as \"O\" (non-entity) in the support set, the model might wrongly classify the same word in the query as \"O.\" This constitutes an inaccurate prediction for a word that should belong to an entity category. Fortunately, if that word was correctly categorized as an entity in previous rounds, the prototype representation will retain this information. By storing prototype knowledge from previous rounds and incorporating this knowledge in subsequent rounds, the model can correctly classify the word, thereby mitigating the model's forgetting of past knowledge and overfitting to the current data.\n\n​\t(2)  **We have introduced new examples to further illustrate this phenomenon.** For instance, consider the word \"chocolate\" in the diagram below. (Due to constraints in webpage settings, we are currently unable to display the diagram here. We will include this section in future versions. For now, let me provide a brief description of the diagram.) In the current support data, \"chocolate\" is classified as category O. Consequently, when it appears in the query data, it is also categorized as O. However, in reality, it should belong to the \"product-food\" category. By incorporating previous prototype knowledge, which includes instances where \"chocolate\" was categorized as \"product-food\" in earlier rounds, the model is able to correctly classify it as \"product-food,\" thus rectifying the earlier misclassification.\n\n\n\n​\tThese explanations and examples collectively underscore how the introduction of prototypes facilitates correct classification and addresses challenges in few-shot scenarios.\n\n\n\n**QE.** The response to 'E. Besides the pre-trained method, will it effectively improve the accuracy of other models if the papers’ method is applied to other few-shot NER methods'.\n\n\n\n​\tThank you for your inquiry. **I'd like to start by clarifying that our proposed method is an independent approach, not a plug-in type of solution.** I will then explain that our method is based on improving prototype networks and has yielded significant enhancements. Below, I'll provide detailed explanations to address your concerns.\n\n\n\n​\t(1) **Our method consists of three main components**: entity detection, context-based causal intervention/prototype-based causal intervention, and sample reweighting. In the entity detection component, when dealing with five classes, the model doesn't treat the data as five specific classes plus an \"other\" class. Instead, it directly groups the five classes into an \"entity\" class, creating an \"entity class\" plus an \"other class\" setup. This step helps the model better understand the features of entity classes, aiding in the initial classification of entity categories in the sequence. In the context-based causal intervention part, for each entity, the model replaces the context of entities of the same type to reduce the influence of sample selection bias on the labels. In the case of 1-shot data, where each class has only one sentence, this type of context replacement is not feasible. Hence, we consider an alternative causal intervention approach, prototype-based causal intervention. The model can overfit the current data, which leads to excessive attention on the current prototypes. If there is bias in the current prototypes, it can affect the model's predictions on query data. To mitigate this, we introduce previous prototypes and combine them with the current ones. In practice, we store prototypes obtained by the previous prototype network, and when encountering prototypes of the same type later, we average them to form the feature representation for that type. Finally, in the sample reweighting part, we believe that different support samples should contribute differently to the prototypes. Support data more similar to the query should carry more weight. We calculate the similarity between the vector representations of support and query data, and assign different weights to calculate the prototypes. Overall, our proposed method is an independent approach rooted in prototype networks and causal inference ideas. It's not an application of a plug-in to other methods. \n\n\n\n​\t(2) It's worth noting that our method is based on prototype networks. By incorporating causal inference concepts into prototype networks, we construct both context-based and prototype-based causal interventions. In the ablation experiments, we compare results obtained using standard prototype networks and causal prototype networks. Table 3 indicates that causal inference significantly improves prototype networks. Figure 4 visually demonstrates that context-based causal intervention yields more compact and distinct entity categories compared to standard prototype networks. **These findings underscore how causal inference enhances prototype networks. Thus, we believe that applying causal inference concepts to other methods could also yield improvements.** \n\n\n\n​\tIn conclusion, I hope my explanation has addressed your concerns.\n\n\n\n**Missing References.**\n\n\n\n​\tThank you for your addition. In our upcoming versions, we will reference these sources and provide explanations accordingly.\n\n\n\n**Typos Grammar Style And Presentation Improvements**\n\n\n\n​\tFirst of all, thank you for your corrections. We will address these issues in subsequent versions to ensure greater compliance with standards. Regarding the example issue you raised, in various tasks, similar to the context you mentioned, background information can indeed lead to false associations with task labels. For instance, in common image-related problems, scenarios like pigeons being often present in squares are prevalent. **However, such associations are misleading since the presence of dogs on grass in images or horses being ridden by people in textual contexts doesn't inherently relate to the respective labels.** Below, we provide several papers that have been accepted at some top conferences to illustrate that these false associations exist both in natural language processing and computer vision domains.\n\n\n\n- Yang X, Zhang H, Qi G, et al. Causal attention for vision-language tasks[C]//Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2021: 9847-9857.\n- Niu Y, Tang K, Zhang H, et al. Counterfactual vqa: A cause-effect look at language bias[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2021: 12700-12710.\n- Chang C H, Adam G A, Goldenberg A. Towards robust classification model by counterfactual and invariant data generation[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2021: 15212-15221.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. This paper uses causal intervention to solve the overfitting problem caused by data selection bias in the few-shot scenario, the research motivation is reasonable.\n \n2. The author designs two causal intervention approach, named context-based causal intervention for 5-shot scenario and prototype-based causal intervention for 1-shot scenario. \n \n3. The author compared many novel few-shot NER methods in the experimental section and achieved SOTA results on the Few_NERD and SNIPS datasets, which have certain application values.", "reasons_to_reject": "Insufficient in the experimental part:\n1. Lack of case study analysis for more intuitive display.\n2. The author only used GPT for data augmentation on the 1-shot setting, so what is the effect of using GPT for data augmentation on the 5-shot setting?\n3. In Section 4.6, the author did not compare the effectiveness of using the sample reweighting method under the 5-shot setting.\n4. In Section 4.7, the author just describes the word “German”, it is best to use visualization to demonstrate this case.\n\nThe paper needs further polishing and revision:\nThere are many formatting errors and inconsistent correspondence before and after the paper.", "questions_to_authors": "A. In Section 1, the author says “However, these existing methods tend to overlook the issue of overfitting caused by spurious correlation in few-shot tasks.” What elements are involved in this spurious association?\n \nB. In Section 4, The author only used GPT for data augmentation on the 1-shot setting, so what is the effect of using GPT for data augmentation on the 5-shot setting?\n \nC. In Section 4.2, the author describes utilizing the Bert-base-uncased model as the base model. If the author replaces Bert with Roberta and others pre-trained language models, will better results be achieved?\n\nD. In Section 4.7, the author just describes the word “German”, it is best to use visualization to demonstrate this case. And need give more description to analyze the reason.\n\nE. Besides the pre-trained method, will it effectively improve the accuracy of other models if the papers’ method is applied to other few-shot NER methods?\n", "paper_topic_and_main_contributions": "This paper uses context-based causal intervention and prototype-based causal intervention to address the overfitting problem in the few-shot NER task. For 1-shot, the author uses prototype-based intervention to reduce the spurious correlation between the current prototype and the label. For 5-shot, the author introduces context-based intervention to replace the context to prevent overfitting. The experiments on various benchmarks demonstrate that this approach achieves new state-of-the-art results.", "excitement": 3, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {"Missing_References": "There are some causal intervention approaches that need to be referenced.[1-3]\n[1] Wu Y X, Wang X, Zhang A, et al. Discovering invariant rationales for graph neural networks[J]. arXiv preprint arXiv:2201.12872, 2022.\n[2] Wu Q, Zhang H, Yan J, et al. Handling distribution shifts on graphs: An invariance perspective[J]. arXiv preprint arXiv:2202.02466, 2022.\n[3] Fan S, Wang X, Mo Y, et al. Debiasing graph neural networks via learning disentangled causal substructure[J]. Advances in Neural Information Processing Systems, 2022, 35: 24934-24946.", "Typos_Grammar_Style_and_Presentation_Improvements": "1. In line 195, Reference “Pearl (2009)” has not been properly cited.\n\n2. In line 239, the author uses “Equation1”, while in line 243, the author uses “Eq 2”, this method of referencing formulas is not uniform.\n \n3. In line 420 the author uses “Few-NERD”, but in line 426, the author uses “FewNERD”, there exists the Inconsistent name of the dataset \"Few-NERD\".\n\n4. In line 055-059, the case of \"animal\" and \"square\" is very farfetched, it would be great if you could provide some stronger examples."}}]

Author rebuttal:

Meta-review: All reviewers agreed that the proposed approach for few-shot NER is sound and useful. The results on two benchmarks show clear benefits, some surpassing current results in the existing literature.

However, there are some raised legitimate concerns.
In particular, reviewers are concerned about the lack of clarity in the explanation of the causal model and details on the model component (e.g., sample reweighting component), some of which are partially addressed during the rebuttal phase. It is suggested the authors provide explicit connections to existing causal intervention approaches and use more case studies and visualizations to illustrate their approach.

Regarding presentation, the reviewers pointed out several typos, grammatical errors, and inconsistencies in the paper that need to be addressed.
Decision: Accept-Findings