Enhancing Graph Invariant Learning from a Negative Inference Perspective
Authors: Kuo Yang
Zhengyang Zhou
Qihe Huang
Wenjie Du
Limin Li
Wu Jiang
Yang Wang
Venue: ICML 2025 poster (2025)
Keywords: Graph learning
out-of-distribution generalization
environment awareness
negative inference
prompt learning.
Primary area: deep_learning->graph_neural_networks
TL;DR: In this work, we propose a negative inference graph OOD framework (NeGo)  to broaden the inference space for environment factors, effectively addressing the limitations that existing methods cannot tackle.

Abstract: The out-of-distribution (OOD) generalization challenge is a longstanding problem  in graph learning. Through studying the fundamental cause of data distribution shift, i.e., the changes of environments, significant progress has been achieved in addressing this issue. However, we observe that existing works still fail to effectively address complex environment shifts. Existing practices place excessive attention on extracting causal subgraphs, inevitably treating spurious subgraphs as environment variables. While spurious subgraphs are controlled by environments, the space of environment changes encompass more than the scale of spurious subgraphs. Therefore, existing efforts have a limited inference space for environments,  leading to failure under severe environment changes. To tackle this issue, we propose a negative inference graph OOD framework (NeGo)  to broaden the inference space for environment factors. Inspired by the successful practice of prompt learning in capturing underlying semantics and causal associations in large language models, we design a negative prompt environment inference to extract underlying environment information. We further introduce the environment-enhanced invariant subgraph learning to effectively exploit inferred environment embedding, ensuring the robust extraction of causal subgraph in the environment shifts. Lastly, we conduct a comprehensive evaluation of NeGo on real-world datasets and synthetic datasets across domains. NeGo outperforms baselines on nearly all datasets, which verify the effectiveness of our framework.

Enhancing Graph Invariant Learning from a Negative Inference Perspective
Kuo Yang 129
Zhengyang Zhou 121
Qihe Huang 12
Wenjie Du 12 Limin Li
Wu Jiang" Yang Wang 12*
Abstract
1. Introduction
Graph Neural Networks (GNNs) have emerged as the predominant approach for encoding graph data (Kipf Welling, 2016; Xu al., 2018), delivering notable achieve- ments in various research fields including molecular prop- erty prediction (Jumper et al., 2021; Yang et al., 2022), rec- ommendation systems (Wu et al., 2022b; Gao et al., 2022), and traffic flow forecasting (Liang et al., 2018: 8; Zhou et al., 2020). However, as real-world data evolving with com- plex patterns, the challenge of data distribution shift has become a major obstacle for GNNs (Gui et al., 2022; Ji et al., 2022; Wang al., 2023; Zhou al., 2022b; Zou et al., 2023; Sun et al., 2022c; 2024). Therefore, various studies concentrate on improving the Out-Of-Distribution (OOD) generalization ability of graph learning models (Chen et al., 2024; 2022; Gui et al., 2024; Miao et al., 2022; Sui et al., 2022; Li et al., 2022; Wu et al., 2022c; Sun al., 2022b).
The out-of-distribution (OOD) generalization challenge is longstanding problem in graph learning. Through studying the fundamental cause of data distribution shift, i... the changes of environments, significant progress has been achieved in addressing this issue. However, we observe that existing works still fail effectively address complex environment shifts. Existing practices place excessive attention on extracting causal subgraphs, inevitably treating spurious sub- graphs as environment variables. While spurious subgraphs are controlled by environments, the space of environment changes encompass more than the scale of spurious subgraphs. Therefore, existing efforts have limited inference space for environments, leading to failure under severe environment changes. To tackle this issue, we propose a negative inference graph OOD frame- work (NeGo) to broaden the inference space for environment factors. Inspired by the successful practice of prompt learning in capturing under- lying semantics and causal associations in large language models, we design negative prompt environment inference to extract underlying en- vironment information. We further introduce the environment -enannceddeeaceeeeeeeeee invariant subgraph Iearn- ing to effectively exploit inferred environment em- bedding, ensuring the robust extraction of causal subgraph in the environment shifts. Lastly, we conduct comprehensive evaluation of NeGo on real-world datasets and synthetic datasets across domains. NeGo outperforms baselines on nearly datasets, which verify the effectiveness of our framework.
Recently, environment--cnttere invariant learning methods achieved impressive OOD generalization performance with the aim of inferring underlying environment factors in data (Chen et al., 2024; Gui et al., 2024; Xia et al., 2023; Yuan et al., 2023). Those efforts demonstrate that the changes of environment are the fundamental reason for the shift of data distribution (Grice & White, 1961; Liu et al., 2021; Peters al., 2016). However, existing approaches still lack the ability to decouple causal subgraphs from complex en- vironments. As shown in Fig. I(a), we double the scale of spurious substructures in the STURIOUS-MOTIFAN and observe a significant decrease in the performance of current methods when they are re-conducted on this modi- fied dataset. The reason lies that current methods, even those claiming to model environments, focus much of their attention extracting causal subgraphs (Chen et al., 2022; Wu et al., 2022a;c; Sun et al., 2021). This results in the model being able to extract causal subgraphs only in known environments, and failing in modeling unseen complex envi- ronments. Therefore, this poses a challenging research ques- tion: how to brouden the inference space of environments, enabling model to handle complex environment shifts.
*Equal Contribution. Uniierssity of Science and Technol- ogy of China (USTC), Hefei, China "Suzhou Institute for Ad- vanced Research, USTC, Suzhou, China "China Mobile Com- munications Group Ltd, Correspondence to: Yang Wang* <angyan www.uuuccuuu. edu, Wenjie Du* <duwenjie@usse www.eenjje@ustccdu..co add......d......................
We argue that this limitation arises from the positive learning paradigm that focuses solely on extracting causal subgraphs as its primary objective. In contrast, negative inference paradigm, modeling the sample space except the invariant
Proceedings of the ndnd International Conference Machine Learning, Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

Enhancing Graph Invariant Learning from Negative Inference Perspective
Casual subgrophs
Environment factors
6-8
Lodder
Wheel
Tree
Accuracy
00D Methods
Mixing envirommeuss
Complex envirnmmnnss
Degradation in performance.
(b) Prediction failure in complex environments.
Figure 1. The motivation of our work. (a) We double the scale of spurious substructures in the 2019), and observe significant decrease the performance of current methods when they are re-conducted on this modified dataset. (h) The OOD methods, which treat spurious subgraphs as the environments, fail to address the shift of complex environments.
(Ying al.,
subgraph as environments, has the potential to broaden the perception scope of environment. As shown in Fig. I(b), the positive inference can only infer the specific ladder, wheel, and tree as environment variables, while the negative infer- ence approach can model all variable space except the cycle and house as environments. However, the inaccessibility of environment information pose challenges to implementing such negative inference. Specifically, (1) how to formu- late the negative inference learning to achieve environment awareness, and (2) how fully utilize environment informa- tion for facilitating causal invariant learning.
identify that limited environment space for positive in- ference is the main reason restricting the generalization capacity of existing OOD approaches.
We propose a novel invariant learning framework with negative inference NeGo. To be specific, we design an innovative environment inference strategy via negative in- ference, which effectively broadens the inference space of environment factors. Moreover, we introduce an attention- based residual connection offer our model with the ability to resist complex environment shifts.
In this work, we propose novel Negative inference Graph OOD framework (NeGo). NeGo aims to achieve causal invariant learning against complex environment shifts by a negative inference. Firstly, we design a negative prompt learning framework for inferring underlying environment factors. Given a specific sample with label Y, model all extra-class samples the environment space for this sample. This design enables the model to capture a broader scale of environments, no more limiting to in-sample spurious sub- graphs. Secondly, we introduce an wetrromment-enhance invariant learning strategy to effectively utilize inferred en- vironment variables. Specifically, we design an interactive decoding scheme that utilizes an attention-based residual connection to encapsulate environment embedding into node representations. Different from traditional approaches that neglect the information of environment variables during subgraph extraction (Chen et al., 2024; Gui et al., 2024), our design incorporates the underlying environment patterns into the process of invariant subgraph learning. Lastly, we conduct a comprehensive evaluation of NeGo real-world datasets across domains, and synthetic datasets. NeGo out- performs baselines on nearly all datasets. Our contributions can be summarized as follows:
We conduct extensive experiments both synthetic and real-world datasets with distribution shifts to evaluate the performance of NeGo. The results from both visualiza- tion and quantitative analysis indicate that our framework successfully achieves accurate prediction in complex en- vironmental scenarios.
2. Background
Preliminaries. A graph is denoted as G = (X, E G. where G is the observed graph dataset. A RNNN repre- sents the adjacency matrix and X' E Nxd denotes node features, where indicates the number nodes and d is the feature dimension. Each graph is associated with responding label Y. From the perspective of causal theory, the graph data can be partitioned into spurious subgraph Gg and a causal subgraph Gc, where Ge directly deter- mines its label Y. The spurious subgraph Gg is controlled by the spurious variable C, while the causal subgraphs Ge is controlled by the causal invariant factor C, as shown in Fig. 4. Based on the different interdependencies among C, S and Y, structural causal models (SCMs) be further classified into Full Informative Invariant Features (FIIF) and Partially Informative Invariant Features (PIIF) (Ahuja et al., 2021: Chen et al., 2022).
We observe that existing environment-centered OOD prac- tices encounter difficulties handling complex environ- ment shifts. Through a comprehensive investigation, we
Problem definition. Our work aims to address the lim- itations of existing approaches in handling complex data
2

---

Enhancing Graph Invariant Learning from a Negative Inference Perspective
distribution shifts. We specifically focus on broadening the inference scope of environments, enabling the network to handle intricate scenarios of environment shifts. Addition- ally, our framework is required to effectively tackle both FII and PI]F challenges.
various fields, due to its powerful interpretability (Chen et al., 2022; Li et al., 2022; Miao et al., 2022; Wu et al., 2022c). Most existing invariant learning methods focus on extracting the causal subgraph to achieve invariant learning. This strategy limits the inference space of the environments to the dimension of spurious subgraphs, which hinders the ability of models to capture the complex environment states. In this work, we propose an invariant learning mechanism based negative inference address this limitation.
Environment inference with negative prompt. Our nega- tive prompter proposed to achieve a broader inference scale of environments, which inspired by the success of prompt learning in language models (Brown et al., 2020; Gao et al., 2020). Prompt learning is designed to capture underlying semantic knowledge in language data, which improves the generalization ability of models by introducing appropriate prompt tokens to guide the network learn de- sired answers (Rao et al., 2022; Sordoni et al., 2024; White et al., 2023; Sun et al., 2023). For example, in the semantic emotion classification task, the language model constructs template such as "the emotion on expressed by this sentence is [class]", where [class] is trained to learn real label. In a similar way, our framework can be viewed as constructing a set of text prompts such as ronmentsst of current sample are [answer]* ", where [answer] can be guided capture the real environment states.
3.2. Prompt Learning
Prompt learning is proposed in NLP models to infer under- lying semantic and potential causal associations in linguistic data. Many effective prompt methods have developed with the introduction of large language models, including some hand-crafter prompts (Brown et al., 2020), discrete prompts (Gao et al., 2020; Shin et al., 2020), and learnable prompts design (Li & Liang, 2021). In recent years, prompt learning has also been developed in the graph learning field (Sun et al., 2022a; Li et al., 2024; Sun al., 2023; Zi al., 2024). Our approach is the pioneering effort to apply prompt learn ing to address the graph OOD generalization issue.
"the underlying
Different from random data augmentation techniques (Han et al., 2022; Li et al., 2021; Lu et al., 2024; Rong et al., 2019; Wang et al., 2021; You et al., 2020; Zhao et al., 2021) and distributionally robust optimization (DRO) methods (Staib& & Jegelka, 2019; Wu et al., 2024; Zhu et al., 2021), our prompt-based approach not only broadens the scale of environment inference but also deepens the understanding of underlying data generation process. Existing methods always expand the inference boundary of the model by in- corporating stochastic perturbations. However, the intro- duction of randomness prevents the model from capturing the underlying semantics and hinders its ability to deepen the understanding of generation process. In contrast, our prompt- based approach allows to deeply extract the un- derlying casual correlation of variables, which is the reason why we introduce the technique of prompt learning.
3.3. Comparisons to Existing Graph OOD Works
Environment--cnteree studies (Chen al., 2024; Gui et al., 2024; Li et al., 2022; Wu et al., 2022a; Yang et al., 2022) con- sider that the data distribution shifts stem from the changes of environments. Therefore, these practices enable the model to withstand data distribution shifts by inferring en- vironment variables. Concretely, the networks are often trained with the objective of equipping models effectively handle mixed environments scenarios. However, this design allows the networks to make narrow inference about the environments, and makes the networks unable to handle with distribution shifts in complex environments. We at- tribute this limitation of inference scale to the shortcomings of positive inference, which proved both empirically and theoretically. To this end, we propose a negative inference mechanism to broaden the inference space for environments, without relying the mixed environments hypothesis. Our approach, which represents pioneering practice in utilizing negative inference, is distinct from all existing practices in this field.
3. Related works
3.1. OOD Generalization.
onitt--Diittiuttion (OOD) generalization refers to the task of adapting a model that has been trained on specific distri- bution to effectively process data from potentially different distribution. Various approaches can be employed for OOD generalization, including data augmentation (Rong et al., 2019; Wang et al., 2021; You et al., 2020), domain adapta- tion (Wang& & Deng, 2018), and causal invariant learning (Sui et al., 2022; Wu et al., 2022c). Among them, causal invariant learning demonstrates impressive performance in
4. Graph OOD generalization via environment inference
Existing environment- practices aim to enable the networks with the ability to resist data distribution shifts. However, our empirical observations indicate that these approaches are insufficient in handling complex environ- ment shifts. To address this issue, we conduct theoretical
3

---

Enhancing Graph Invariant Learning from Negative Inference Perspective
Input data
Towards bronder cwv. space
Subgraph
Extractor
Graph Encuder
positive
Modelable deninonauu
anapiin
Negative Prompter
Unrevagnized PRLSFORStion
negarive loss
Leurnable Prompis
Env.
Figure 2. The architecture of NeGo. We implements an wewtoommnt--ehhancee graph learning framework in which the environment extracted through negative prompt mechanism. The training process is guided by both positive loss and negative loss, aiming broaden the modeling space for the environment.
analysis of these methods, and identify that their failures stem from the limited environment inference space by only positively treating spurious subgraphs as environments. In contrast, we propose a promising method based on negative inference.
they are formulated by K' independent D-dimensional Gaus- sian distributions N (1,, /), where μ4; E IR 1 Therefore, environment variables can be modeled from a vector space perspective, allowing us to approximate the environment space by exploring the mixture space of vectors.
4.1. Limited Environment Cognitive Space for Positive Inference
Given the Assumption 4. we can model the environments codebook 4 (M1 P2, ... PK) E RXD This environ- ment codebook serves as proxy for the environment space, representing the entire environment space through the mix- ture of vectors.
From the perspective of causal theory (Pearl, 2009; 2010), the variables of generating the graph data include causal subgraph Gc and spurious subgraph Gs, where Gg is con- trolled by environment variable E. As shown in Fig. 4, Ge + Y demonstrates stable casual relationship from Ge to Y in the data generation process. Consequently, the distribution shift between the training data and the test data can be attributed the shifts of environment E, which can be formally expressed as Pirein (G, E) (Ge E). Modeling environment variables becomes crucial for tack- ling OOD generalization issue (Chen et al., 2024; Gui et al., 2024; Xia et al., 2023; Yuan et al., 2023). With the observed training dataset G, environment-centered approaches strive to learn the distribution of the environment factor E,
Therefore, the scale of environments is modeled as a mix- ing space extracted environment variables. As a result, the new data G2 associated with the environment state Eş N(e, μ,, I), where e; E R1xk representing the mixing weight. This indicates that the latent variables
(e1, c,, eN) E N be regarded the proxy factor for the environment variable E, directly determin- ing the observed data & generation process. The posterior probability of the environments P(Elg) is transformed into,
N II P(e)P(G.))))) I=1
I P(e)
P(G,e)
1=1
E) P(GEE)(()) P(G)
G(E
(L)
P(G)
N E(e) P(G.e)) 111
P())
J PCG E)
E
(2)
The prior distribution P(E) and the likelihood E) =
The finite space of allows for the approximation strategy to be feasible. However, the limited scale of e may limit the capacity of model effectively counter complex environ- ment shifts, which is verified by our empirical results. We next delve into the reason contributing to this limitation. We first present definition of the basis and base environments of the environment space, similar to the concepts of basis and base vectors in the vector space.
N
II P(G.EE make the numerator thcoretically computable. (=1
However, due the uncertainty in the scale of environments E, the denominator of Eq. 1 involving integration becomes intractable. To tackle this issue, we impose distribution shift boundary based on following environment mixture assumption.
Assumption 4.1. (Li et al., 2022) K different environ- ment labels can extracted from the observed dataset G,
Assumption 4.2. Let E, = (Ei,... Ex} be the basis of environment space, and each element F, within it referred

---

Enhancing Graph Invariant Learning from Negative Inference Perspective
as the base environment. The linear combination of base environments can completely describe the entire environ- ment space.
live space for environments, with its upper limit being the ground-truth environment distribution.
Proposition 4.4 emphasizes that the negative inference paradigm enables a broader-scale environment inference space by cooperatively modeling both intra-class spurious subgraphs and extra-class samples. Detailed proof can be found in Appendix D.2.
Actually, the environment mixing assumption fundamen- tally relies on the expectation that extracted environment codebook cover the basis Eg, However, we observe that such goal cannot be achieved by existing methods. Given graph G, current anvironment-centere methods aim to de- compose it into causal subgraph Gc and spurious subgraph Gg. The spurious subgraph Gg is inferred as the environ- ment variable. Although Gş controlled by environment factor (E + Gg), the space of environment changes en- compass more than the scale of spurious subgraphs. For example, consider the substructure Ge that is causally as- sociated with one graph-level property 1, but the variants of such Ge act as enviromment factors for other properties. Existing methods fail to capture such other properties with invariance.
5. Graph Invariant Learning with Negative Inference
In this section, we introduce novel negative inference graph OOD framework NeGo to address the limitation of existing efforts in handling complex environments shifts. Specifically, NeGo is developed to design a negative infer- ence learning task to capture underlying environments, and leverage inferred environment embeddings to enhance graph invariant learning.
Proposition 4.3. Given observed graph dataset G, the inference process, considering Gs as the environment factor, fails to capture the basis Eg, that can represent the entire environment space.
5.1. Negative Prompt Learning for Environment Inference
Negative inference focuses on indirectly extracting invariant subgraph by investigating the information beyond the causal factors. This leads to the problem that the space of variables beyond causal information is fimiite-dimessional. Given the insight from Theorem 4.4, we decouple the process of modeling the environment through negative inference into two components: the extraction of intra-class spurious sub- graphs and the inference of extra-class samples. Modeling spurious subgraphs is relatively straightforward and exten- sively studied. The crucial challenge lies in achieving comprehensive understanding of extra-class sample space.
Proposition 4.3 indicates that the mixing of μ is unable to encompass the entire environment space. Existing enviromment-centered methods still have a narrow space of the environments, which leads to incapability of network to extract the causal graph from the complex environments. Therefore, the limitations of existing works are attributed to the narrow inference space of the model for environment variables. Detailed proof can be found in Appendix D.I.
4.2. The Enhancement of Negative Inference
Negative inference has major advantage in effectively ex- panding the cognitive boundary of models. For example, the positive inference can only infer the specific ladder, wheel, and tree as environment variables, as shown in Fig. 1(b), while the negative inference approach can infer all variable space except the cycle and house as environments. While the ultimate objective is extract invariant subgraphs, the negative inference mechanism prioritizes inferring the envi- ronment space, empowering the model with the capability to adapt to complex environment shifts. From the perspec- tive of information theory, the training objective of negative inference can be formalized as,
Formally, let the prior distribution of extra-class samples for G be denoted as P(Y). where G is with the label Y. We introduce a variational estimate of the environment variables denoted (E) (a.k.a,, fa), where & is the parame- terized network. Denoting KL-divergence as KL(-Il-), the training of Qo is to implement the first term of Eq. 3, which can be formalized as following optimization,
min E KL (Qo
(4)
Inspired by the success of prompt learning in capturing un- derlying semantic and causal associations in large language models (Floridi Chiriatti, 2020; Sordoni et al., 2024), we introduce a negative prompter to achieve this goal. Specif- ically, given sample G belonging to class 1, the negative prompter treats all extra-class samples as environments. De- signing appropriate prompt tokens to guide effective learn- ing is the primary question that needs to be addressed when employing the concept of prompt learning.
max I(E; Go| = max 1(E; GGll)
(3)
max T(E:G] - I(E;
where GY denotes the extra-class samples of the graph G which is labeled Y. Proposition 4.4. The learning objective of negative in- ference paradigm (Eq. 3) encompasses a broader cogni-
Given the proven efficacy of learnable prompts in various
5

---

Enhancing Graph Invariant Learning from a Negative Inference Perspective
practices, design class-specific learnable prompt tokens P == P v12) ,..., ..., o(0), where (( TR Ixd and I. is the number of classes. The class-specific design manner aims to capture the extra-class sample space for each graph, in order to achieve the objective defined by Eq. 4. The negative prompter f-) is guided to learn the prompt answers Av E RLxd by interacting the encoded graph embedding Z E Rixd and the learnalle prompts P.
the causal subgraph from environments. Existing methods often neglect the design of a graph-tailored environment ex- ploitation algorithm, which can lead to the failure in extract- ing causal subgraphs when environment becomes complex (Gui et al., 2024).
We propose an environment-enhance invariant learning mechanism that leverages perceived latent environment em- beddings achieve the extraction of causal subgraphs with resistance to complex environment disturbances. Different from the negative prompter that investigates the extra-class sample space, we concentrate on the disentanglement of causal invariant substructures within the intra-class samples in this subsection.
AN Lo( P).
(5)
The negative prompter fo is parameterized by cross- attention network in Transformer decoder (Vaswani et al., 2017), where Zg is obtained by GNN backbone encoder (.). For a sample G with label I, such negative prompts answers An should satisfy the following two properties:
Let the marginal distribution of the causal subgraph Ge be P(Gc). We introduce variational estimation of the sub- graph extraction (G E) (a.k.a., ge), where & is the parameterized networks. The model can make casual invari- ant predictions of the label distribution (Y(Y|G)) (a.k.a., ga). only when the causal graph is accurately extracted from complex environments. The learning objective for environment-ennhanee graph invariant learning Fg o Qc(() is implement the second term of Eq. 3, which can be formalized as following optimization,
The prompts answers AN should produce a low match with graphs whose labels are I.
The prompts answers AN should produce a high match with graphs whose labels are not
With the explanation in the language models, our negative prompl mechanism involves designing prompt tokens to learn the desired [answer] of "the underlying environments of current sample are [answer]". These two properties guide fa(+) to learn a positive answer when interacting with each extra- class sample and a negative answer when interacting with each intra-class sample. Therefore, the training objective of our negative prompl mechanism can be formulated as,
min ([ -lo Po(IGe)) (7) E,9
The environment embedding AN E RLdd is inferred at the graph level, but the extraction of substructures often requires node-level operations. Therefore, the primary focus of enviromment-enhanee
invariant learning to propagate the perceived environment embedding AN to individual nodes. We design an interaction-decodin module gt, (-) to address this issue.
([[()) [IKL((((TTIIQ)(((C))) Enega = E[
(6)
= E[log Po P& - log logGG(Y|G,
Specifically, 9s, (.) consists of three families of learnable
The environment variables we infer are class-specific, in contrast to the global environment factors constructed by previous methods. Our design is intuitively reasonable, specific subgraph may be perceived by one class as causal in- formation, while its minor variations are perceived by other classes as environments. Moreover, it worth noting that we never overlook the inference of the environments (spuri- ous subgraphs) within intra-class samples. Given that the intra-class environments are always intertwined with causal factors, we incorporate the inference of intra-class environ- ment variables into the discovery of the causal subgraph, which is provided in the next subsection.
parameters, i.e., Wº, W W E Rdxd (.) takes the node-level representation Z E Raxdd encoded by the GNN encoder (-) and the environment embedding AN E Lxd obtained by negative prompt as inputs. Three hidden state matrices are calculated by,
z = ZwQ, AK ANW* A = AN (8)
The node embedding with environment information ob- tained through residual connections is,
ZE = softmax ZA
JAV +
(9)
Vd
5.2. Environment-enhanced Graph Invariant Learning
We exploit subgraph extractor Gc = yea (Zx) to realize invariant subgraph discovery. Then, Ge is encoded by h.s()) (-) to obtain the causal representation for prediction. This representation is passed through an MLP layer ge to model the distribution of Y. Therefore, the training objective of
While inferring environment variables is a crucial step in understanding the data generation process, the ultimate goal of graph learning is to achieve casual invariant prediction. Thus, the next challenge to address is the disentanglement of
6

---

Enhancing Graph Invariant Learning from Negative Inference Perspective
environmeent-ehhanced invariant learning is,
Algorithm 1 The training process of NeGo
Input: training data G, negative prompts P. Initial: the GNN encoder hy. the negative prompter fø- environmentt-nhanneed invariant learning mechanism Øe- of epochs K.
Cpost = P.lloog
-Rllog + log (G,,saGell, (10)
final predictor go. learnable prompt tokens P. the number
5.3. Optimization Objective and Theoretical Analysis
for = 1 to K do
Our NeGo achieves a graph learning framework with a wider space of environment inference. This is accomplished through two sequential approaches. first focusing on con- structing the learning task for negative inference, and then leveraging the environment embeddings obtained from neg- ative inference to enhance graph causal invariant learning. Thus, the training objective of our NeGo is,
Z = hw(
AN = folZa, P)
Z ZW A AN WK AN A AANWV (A" )AV
Z softmax
vd
Y = go (G
Gc 9, (g 2)
Optimizing:
Lnago
-E[log Po G. E|KL(P(GY Q/ G)) (Y
C = Enega A Eposi:
(11)
Possi -Bllog Pg.o = = Elog GY
log P ( (Gc/G An)
where À is a hyperparameter, which is set to in the im- plementation. The training process of NeGo is provided in Alg. 1. Note that the two sub-challenges addressed by NeGo are not independent but closely intercomnected. The environment negative inference mechanism assists the net- work in comprehending the distribution shift of data, while the causal invariant learning with environment enhancement empowers the network to accurately extract causal invariant subgraphs even in complex environments. Thus, the former serves as a foundation for the latter. This design reflects the principle that understanding data generation process is erucial enhance the generalization of models. We also provide theoretical evidence supporting the ability of NeGo to effectively address both FIF and PHIF under both cases of (GG)) < H(G)(Y) and H (G))Y > (G)) Detailed proof can be found in Appendix D.3.
min L = Luega + Lepsi v.O.Bg.P
end for
Return hws for 95, 90 and P
IB IR-M (Ahuja al., 2021). The Empirical Risk Minimiza- tion (ERM) baseline is vanilla GNN with ERM objective, which is trained using the same settings with (Gui et al., 2024). Graph OOD approaches includes DIR (Wu et al.,
2022c), GSAT (Miao et al., 2022), CAL (Sui et al., 2022), CIGA (Chen et al., 2022), GIL (Li et al., 2022), LECI (Gui et al., 2024) and GALA (Chen et al., 2024).
6.2. Datasets
We adopt two synthetic datasets with distribution shift and six real-world scenario shift datasets from both molecular and social science domains. Synthetic datasets include GOOD-Motif (Wu et al., 2022c) and GOOD-CMNIS, (Gui et al., 2022). In molecular property prediction fields, we select the scaffold and size splits of GOOD-HIV dataset (Gui et al., 2022; Wu et al., 2018) and the assay and size splits of DrugOOD LBAP-core-icS0 dataset (Ji et al., 2022). We also choose two social sentiment graph datasets with distribution shifts, including GOOD-SST2 and GOOD- Twitter (Yuan et al., 2022). Detailed statistics on the number of graphs in those datasets provided in Tab. 6.
Theorem 5.1. Given the FIIF or PIIF assumptions under both cases when H (GclYY) < (GslY)) and H(GolY) > (Gs Y), the causal subgraph Ge can be extracted by optimizing Eq. 11.
6. Experiments
We evaluate the effectiveness of NeGo by answering the following questions. Q1. Does our approach effectively address the issue unresolved in existing works? Q2. Is our framework sufficiently interpretable? Q3. Does each component in our NeGo effectively enhance the generaliza- tion capacity? Q4. Does our framework operate with high efficiency?
6.3. Result Comparison and Analysis
We comprehensively evaluate the OOD performance of NeGo on both real-world and synthetic datasets to answer Q1. Tab. and 2 present the performance of NeGo on chemical and sentiment graph datasets. Tab. 8 showcases the performance of our framework on two synthetic datasets. Compared to existing methods, our method achieves opti- mal performance on almost all datasets. Besides, the per-
6.1. Baselines
We choose four representative OOD methods and seven graph-specific OOD approaches for comparison. The repre- sentative OOD frameworks we select consist of ERM, IRM (Arjovsky et al., 2019), V-Rex (Krueger et al., 2021), and
7

---

Enhancing Graph Invariant Learning from Negative Inference Perspective
Table 1. The ROC-AUC performance of Neão on four weal-world datasets in chemical research field. ID val and OOD val represent the results of OOD test set using the in in-distribution and outto--distribttion validation sets, respectively (Gui et al., 2024). The best results are shown in bold and the second best results are underlined. indicates statistical significance against the second- best results.
GOOD-HIV-scaffold
GOOD-HIV-size
DrugOOD-assay
DrugOOD-size
Model
I1D val
OOD val
II val
OOD val
ID val
OOD val
ID val
O0D vall
ERM
69.61+1.32
70.37+1.19
61.66+22.4
57.31+1.06
70.03+0.11
72.18+0.18
62.97+0.26
63,29+0..3
IRM
73.35
70.89-1.229
58.52:10.86
60.86112.78
71.5610.32
72.6910.29
63.24:10.26
63.46110.3
V-Rex
71.73=13.51
71.18:0..9
18.39:11.54
60.1012.09
70.22:10.6
72.32:10.58
63.87:10:22
64.1110.39
IB-IRM
67.56+2.31
66.25+0.93
57.45+00.74
56.65+1.22
69.34+0.48
71.32+00.6
64.03++061
64.59+0.70
DIR
1.78
68,5913.70
19.69:11.59
60,85:+0.52
67.29:10.73
69.70+0.65
63.85:+0.65
64.73:10,44
GSAT
71.55+33.58
11.39+1.41
60.92+1100
60.61+1.19
71.01+0.54
72.26++0.45
65.12
65.67=+0.5
CAL
73.48:42.64
1.03
1..83:+:.26
62.58++1.44
71.89+0.92
71.23+11.13
63.85:10049
64.22++074
CIGA
66,25+2.89
1..47=11.29
58.2441.77
6..56+1.76
67.68+1.14
70,54=0.59
14.16
64.83+0079
GIL
7..89+1.60
70.23=11223
1..74
1..29++1.3
70.45
70.73+1.36
64.91+0.51
65.4344064
LECI
74.04.0..5
1..4
64.83:12.59
65.44
72.67-0.46
73.45-10.17
65.93:10.43
66,4910.60
GALA 73.85+1.10
1..02-11.34
1..99
64.45:12.26
72.8310.73
73.23:10.29
6..23:10.2
65, 05.8410.52
NeGo
75.21
75.87
6..23
65.92.11.82
73.2010.18
73.94
65.49:10.73
66.91 0.84
count. As shown in Tab. 3, we observe that the advantage of our NeGo on common OOD datasets is persists in complex environments, with a 1. 1.29% performance improvement.
Table 2. The accuracy of NeGo on two sentiment graph datascts,
where * indicates statistical significance against the second-best
results.
Model
GOOD-SST2
GOOD TwittTwitter
Moreover, we evaluate the performance of our framework in the the complex environments scenario illustrated in Fig. I(a). NeGo achieves 87.34% and 80.29% on the original and adjusted dataset, respectively. There is only a minor decrease in performance, suggesting that our method ef- fectively tackles the limitations encountered by existing
val
OOD val
ID val
OOD
ERM
78.3742.64
80.41
S1.930.96
17.0
IRM
19.73:1145
80.17+1552
55.27=1.19
5..72++1.03
V-Rex
19.31+1140
10.33++1009
56.46=0.93
56.37740.66
IB-IRM
18.93++1.23
80.22++0.55
$1.23=1.21
56.73++1.02
DIR
77.65+00.71
81.50++0.55
55.32=1.85
56.81+0.91
GSAT
1.09
80.46++00.8
55.09-0066
56.07++0.53
CAL
81.20.11.21
82.34.10.67
56.77-0.86
57.82.10.44
methods in handling complex environments.
CIGA
1.43
82.93++0.75
57.51=1.336
17.19+
GIT.
81.43+1.00
83.31++0.50
58.21-1.24
1.18
To answer the Q2, we visually represent the causal sub- graphs extracted by NeGo on the modified dataset in Fig. (a)). As depicted in Fig. 3, our method consistently extracts the ground-truth subgraph. The visualized results further validate the effectiveness of our proposed negative inference method. By modeling extensive extra-class samples as en- vironments, our approach offers undeniable advantages in handling complex environment shifts.
LECI
0.29
83.44.10.27
1.44
59.64110.15
GALA 02.60:+0.66
82.98+0042
59.03-0.65
60.45+1.36
NeGo
82.72.10.51
84.16:00.23*
60.82.10.22
61,25
formance of cuvironment-centered OOD methods, such as LECI and GALA, often achieves suboptimal or even op- timal results on various datasets. This demonstrates the effectiveness of modeling environment factors in addressing data distribution shifts.
6.4. Ablation Studies
To answer Q3, we investigate each component of NeCo. Specifically, we conduct ablation studies to explore the ef- fectiveness of negative prompter and interactive decoding component. Tab, 4 shows that the performance drops sig- nificantly when there is either negative prompter or in- teractive decoding component. NeGioNoPro refers to the framework that eliminates negative prompter and negative loss, which causes the most performance drop. Therefore, the negative inference mechanism plays a vital role in en- hancing the capability of environment perception. This further validates the rationale of our motivation for incorpo- rating negative inference. NeGo-NoEny indicates that the casual subgraphs are extracted directly using node embed- ding without integrating inferred environment information.
Table The performance comparison (ID val) of NeGo on new
environment scenarios.
Model
GSAT
CAL
CIGA
LFCI
GALA 77.93
NeGo 79.75
(%)
70.13
75.21
71.82
78.46
To futher validate the effectiveness of our NeGo in new environment scenarios (nonlinear environment mixing). we conduct additional experimental discussion. Considering that the environments in SPURIOUS-MOTH MOTIF (Ying al., 2019) include five specific label-irrelevant base subgraphs (wheel, tree, ladder, star, and path), we increase the envi- ronment complexity by diversifying these base subgraphs. Specifically, we randomly add connections to the base sub- graphs in each sample, totaling 10% of the original edge
8

---

Enhancing Graph Invariant Learning from Negative Inference Perspective
Figure 3. The causal subgraphs extracted by NeGo the modified dataset in Fig. ((a).
The performance decline emphasizes the significance en- vironment utilizing strategies overlooked by existing works.
enhance the ability to remove irrelevant information. Exten- sive experiments on real-world datasets across domains and synthetic datasets validate the effectiveness of NeGo.
Table 4. Ablation studies of NeGo.
Model
DrugOOD(assay)
GOOD(Twitter)
Impact Statement
NeGo-NoPro
70.37
58.41
NeGo-NoEny
71.71
59.17
This paper investigates the advanced application of graph learning in chemistry and social science domains, utiliz- ing publicly available datasets. Consequently, there are no ethical concerns in this work.
NeGo
73.20
60.82
6.5. Efficiency Analysis
To address Q4, we explore the training efficiency of NeGo from both theoretical and practical perspectives. The time complexity of NeGo is O([V) x d² - [V] x d x h + [) x d), where | represents the number of nodes, |9| denotes the number of edges, is the feature dimension, and h represents the number cross-attention heads. Our method has linear time complexity with high training efficiency.
A Acknowledgment
This work was supported by the National Natural Science Foundation of China (No.12227901 Natural Science Foun- dation of Jiangsu Province (0244024,4 BK.20240461) the grant from State Key Laboratory Resources and En- vironmental Information System. The Al-driven experi- ments, simulations and model training were performed on the robotic Al-Scientist platform of Chinese Academy of Science.
We empirically compare the training efficiency of NeGo with other baselines DrugOOD-sizz dataset as shown in Tab. 5. Compared with some earlier invariant learning methods (DIR and GSAT), the minor increase in running time of our menthod brings out the substantial performance boost. Additionally, our approach demonstrates greater competitiveness both training efficiency and performance compared existing environment-centered methods.
References
Ahuja, K., Caballero, E., Zhang, D., Gagnon-Audet, J.-C., Bengio, Y., Mitliagkas, I., and Rish, I. Invariance princi- ple meets information bottleneck for out-of-distribution generalization. Advances in Neural Information Process- ing Systems, 34:3438-344 -4088, 2021.
Table 5. The training efficiency of NeGo with other baselines on DrugOOD-size (s/epoch).
Antol, S., Agrawal, A., Lu, J., Mitchell, M., Batra, D., Zitick, C. L., and Parikh, D. Vqa: Visual question answering. In Proceedings of the IEEE international conference on computer vision, pp. 2425-2433 225-22333,015 2015.
Models Time 51.6
GSAT
DIR
CIGA
IECCI
GALAA
NeGio 58.7
52.6
54.2
59.1
62.3
7. Conclusion
Arjovsky, M., Bottou, L., Gulrajani. L. and Lopez- Paz, Invariant risk minimization. arXiv preprint arXiv: arXiv:/907.0 /907 1907.02893, 2019.
In this work, we propose negative inference graph OOD framework NeGo handle complex environment shift in OOD scenarios. By inheriting the practices of prompt leam- ing in large language models, we design a negative prompter to model the environment on larger scale. We then intro- duce an ieroomment-enhannee invariant learning strategy to eliminate spurious subgraphs from the data. This strategy effectively leverages the inferred environment variables to
Brown, T., Mann, B., Ryder, N., Subbiah, M., Kaplan, J. D., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. Language models are few-shot learners. Advances in neural information processing systems, 33: 1877-1901, 2020.

---

Enhancing Graph Invariant Learning from Negative Inference Perspective
Chen, Y., Zhang, Y., Bian, Y., Yang, H., Kaili, M., Xie, B., Liu, T., Han, B., and Cheng, J. Learning causally invari- ant representations for out-of-distribution generalization on graphs. Advances in Neural Information Processing Systems, 25:22131-221444 2022.
Kipf, T. N. and Welling, M. Semi-supervised classifica- tion with graph convolutional networks. arXiv preprint arXiv: I 009.02907, 2016.
Krueger, D., Caballero, E., Jacobsen, J.-H., Zhang, A., Bi- nas, J., Zhang, D., Le Priol, R., and Courville, A. Out- of-distribution generalization via risk extrapolation (rex). In International Conference on Machine Learning, pp. 5815-588 5826. PMLR, 2021.
Chen, Y., Bian, Y., Zhou, K., Xie, B., Han, B., and Cheng. J. Does invariant graph learning via environment augmenta- tion learn invariance? Advances in Neural Information Processing Systems, 36, 2024.
Li, H., Zhang, Z.,, Wang, X., and Zhu, W. Learning invariant graph representations for out-of-disrribttoon generaliza- tion. Advances in Neural Information Processing Systems, 35:11822 18888-18888 11841, 2022.
Floridi, L. and Chiriatti, M. Gpt-3: Its nature, scope, limits, and consequences. Minds and Machines, 30:681-688 694, 2020.
Gao, C., Wang, X., He, X., and Li, Y. Graph neural networks for recommender system. In Proceedings of the Fifteenth ACM International Conference on Web Search and Data Mining, pp. 1623-1625, 2022.
Li, J., Sun, X., Li, Y., Li, Z.., Cheng. H., and Yu, X. Graph intelligence with large language models and prompt learn- ing. In Proceedings of the 30th ACM SIGKDD Con- ference on Knowledge Discovery and Data Mining, pp. 6545-66 6554, 2024.
Gao, T., Fisch, A., and Chen, D. Making pre-trained lan- guage models better few-shot learners. arXiv preprint arXiv:2012. 15723, 2020.
Li, P., Li. D., Li, W. Gong, S., Fu, Y., and Hospedales, T. M. A simple feature augmentation for domain general- ization. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 8886-8895, 2021.
Grice, H. P. and White, A. R. Symposium: The causal theory of perception. Proceedings of the Aristotelian Society, Supplementary Volumes, 35:121-168, 1961.
Li, X., Gui, S., Luo, Y., and Ji, S. Graph structure extrapola- tion for out-of- generalization. In Forty-first International Conference on Machine Learning.
Gui, S., Li, X., Wang, L., and Ji, S. Good: A graph out-of- distribution benchmark. Advances in Neural Information Processing Systems, 35:2059 2073, 2022.
Li, X. L. and Liang, P. Prefix-tuning: Optimizing continuous prompts for generation. arXiv preprint 2000092001.00190 2021.
Gui, S., Liu, M., Li, X., Luo, Y., and Ji, S. Joint learning of label and environment causal independence for graph out-of-distribution generalization. Advances in Neural Information Processing Systems, 36, 2024.
Liang, Y., Ke, S., Zhang. J., Yi, X., and Zheng, Y. Geoman:
Multi-level attention networks for geo-sensory time series prediction. In IJCAI, volume 2018, pp. 3428 222844444,,4 2018.
Han, X., Jiang, Z., Liu, N., and Hu, X. G-mixup: Graph data augmentation for graph classification. In Interna- tional Conference on Machine Learning, pp. -230- PMLR, 2022.
Liu, C., Sun, X., Wang, J., Tang, H., Li, T. Qin, T., Chen, W., and Liu, T.-Y. Learning causal semantic representation for dut-o--distribution prediction. Advances in Neural Information Processing Systems, 34:6155 61700,66601661 2021.
Ji, Y.. Zhang, L., WU, J., Wu, B., Huang, L.-K., Xu, T., Rong, Y., Li, L., Ren, J., Xue, D., et al. Drugood: dustof-distribution (ood) dataset curator and benchmark for ai-aided drug discovery-a focus on affinity predic- tion problems with noise annotations. arXiv preprint MrXiv:2201.09633 2022.
Lu, B., Zhao, Z., Gan, X., Liang, S., Fu, L., Wang, X., and Zhou, C. Graph out- duitt--disti generalization with controllable data augmentation. IEEE Transactions on Knowledge and Data Engineering, 2024. 2024.
Jia, T., Li, H., Yang, C., Tao, T., and Shi, C. Graph invariant learning with subgraph co-mixup for disttribuitiitti generalization. In Proceedings of the AAAI Conference Artificial Intelligence, volume 38, pp. 1562-8570 2024.
Miao, S., Liu, M., and Li, P. Interpretable and generaliz- able graph learning via stochastic attention mechanism. International Conference on Machine Learning, pp. 15524-15543. PMLR, 2022.
Jumper, J., Evans, R., Pritzel, A., Green, T., Figurnov, M., Ronneberger, O., Tunyasuvunakool, K., Bates, R., Židek, A., Potapenko, A., et al. Highly accurate protein structure prediction with alphafold. Nature, 596(7873):583-588) 2021.
Monti, F., Boscaini, D., Masci, J., Rodola, E.,, Svoboda, J., and Bronstein, M. M. Geometric deep learning on graphs and manifolds using mixture model cnns. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp- 5115-5124, 2017.
10

---

Enhancing Graph Invariant Learning from Negative Inference Perspective
Pearl, J. Causal inference in statisties: An overview. 2009.
Sun, Q., Li, J., Peng, H., Wu, J., Ning, Y., Yu, P., and He, L. Sugar: Subgraph neural network with reinforcement pooling and self-supervised mutual information mecha- nism. In Proceedings of the web conference 2021, pp. 2081-2091, 2021.
Pearl, J. Causal inference. Causality: objectives and assess- ment, pp. 39-58, 2010.
Peters, J., Bühlmann, P., and Meinshausen, N. Causal in- ference by using invariant prediction: identification and confidence intervals. Journal of the Royal Statistical So- ciety Series B: Statistical Methodology,7888 78(5):947-1012, 2016.
Sun, Q. Li, J., Peng, H., Wu, J., Fu, X., Ji, C., and Philip, S. Y. Graph structure learning with variational informa- tion bottleneck. In Proceedings of the AAAI Conference Artificial Intelligence, volume 36, pp. 4165-4172 2022b.
Piao, Y., Lee, S., Lu, Y., and Kim, S. Improving out-of- distribution generalization in graphs via hierarchical se- mantic environments. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition, pp. 27631- 77631-27640, 2024.
Sun, Q.. Li, J., Yuan, H., Fu, X., Peng, H., Ji, C., Li, Q.. and Yu, P. S. Position-aware structure learning for graph by relieving under-reaching and over- squashing. In Proceedings of the 3 ACM International Conference Information & Knowledge Management, pp. 1848-1857, 2022c.
ropology-imballne
Rao. Y., Zhao, W., Chen, G., Tang, Y., Zhu, Z., Huang, G., Zhou, J., and Lu, J. Denseclip: Language-guidee dense prediction with context-an awwre prompting. In Proceedings of the IEFF/CVF conference on computer vision and pattern recognition, pp. 18082-18091, 2022.
Sun, Q., Chen, Z., Yang, B., Fu, X., Zhou, S., Peng, H., Li, J., and Yu, P. Ge-bench: An open and uni- fied benchmark for graph condensation. In Advances in Neural Information Processing Systems, 2024.
Rong, Y., Huang, W., Xu, T., and Huang, J. Dropedge: Towards deep graph convolutional networks on node clas- sification. arXiv preprint arXiv: /907. 10903, 2019.
Sun, X., Cheng, H., Li, J., Liu, B., and Guan, J. All in one: Multi-task prompting for graph neural networks. In Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp. 2120-2131, 2023.
Shin, T., Razeghi, Y., Logan IV, R. L., Wallace, E., and Singh, S. Autoprompt: Eliciting knowledge from lan- guage models with automatically generated prompts. arXiv preprint arXiv:2010. 15980, 2020.
Vaswani, A., Shazeer, N.. Parmar, N., Uszkoreit. J., Jones, L.., Gomez, A. N., Kaiser, E., and Polosukhin, I. At- tention is all you need. Advances in neural information processing systems, 30, 2017.
Sordoni, A., Yuan, E., Côté, M.-A., Pereira, M., Trischler, A., Xiao, Z., Hosseini, A., Niedtner, F., and Le Roux, N. Joint prompt optimization of stacked IIms using vari- ational inference, Advances in Neural Information Pro- cessing Systems, 36, 2024.
Wang. M. and Deng, W. Deep visual domain adaptation: survey, Neurocomputing, 312: 2018
Staib, M. and Jegelka, S. Distributionally robust optimiza- tion and generalization in kernel methods. Advances in Neural Information Processing Systems, 32, 2019.
Wang, Y., Wang, W., Liang, Y., Cai, Y., and Hooi, B. Mixup for node and graph classification. In Proceedings of the Web Conference 2021, pp. 3663-3674, 2021.
Sui, Y., Wang, X., Wu, J., Lin, M., He, X., and Chua, T.-S. Causal attention for interpretable and generalizable graph classification. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp. 1696-1705, 2022.
Wang. Z., Liu, X., Li. H., Sheng, L., Yan, J., Wang, X., and Shao, J. Camp: Cross-modal adaptive message passing for text-image retrieval. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 5764 5773, 2019.
Sui, Y., Wu, Q., Wa, J., Cui, Q., Li, I.., Zhou, J., Wang, X., and He, X. Unleashing the power of graph data augmentation on covariate distribution shift, Advances in Neural Information Processing Systems, 36, 2024.
Wang. Z., Chen, Y., Duan, Y., Li, W., Han, B., Cheng, J., and Tong, H. Towards -fiit--distiibution generalizable predictions of chemical kinetics properties. arXiv preprint arXiv:2310.03152. 2023.
Sun, M., Zhou, K., He, X., Wang, Y., and Wang, X. Gppt: Graph pre-training and prompt tuning to gener- alize graph neural networks. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp. 1717-1727, 2022a.
White, J., Fu, Q.. Hays, S., Sandborn, M., Olea, C., Gilbert, H., Elnashar, A., Spencer- Smith, J., and Schmidt, D. C. A prompt pattern catalog to enhance prompt engineering with chatgpt. arXiv preprint arXiv:2302. 1382, 2023.

---

Enhancing Graph Invariant Learning from Negative Inference Perspective
Wu, J., Chen, J., Wu, J., Shi, w. Wang, X., and He, X. Understanding contrastive learning via distributionally robust optimization. Advances in Neural Information Processing Systems, 36, 2024.
In Proceedings of the aaai conference on artificial intelli- gence, volume 35, pp- 11015-11023, 2021.
Zhou, K., Yang. J., Loy, C. C., and Liu, Z. Learning to prompt for vision-language models. International Jour- nal of Computer Vision, 130(0):3377-244, 2022a.
Wu, Q. Zhang, H.. Yan, J., and Wipf. Handling distribu- tion shifts on graphs: An invariance perspective. arXiv preprint ar Xii:2222.002666 2022a.
Zhou, Y., Kutyniok, G., and Ribeiro, B. Ood link predic- tion generalization capabilities of message- passing gnns in larger test graphs. Advances Neural Information Processing Systems, 35:20257-20272, 2022b.
Wu, S., Sun, F., Zhang, W., Xie, X., and Cui, B. Graph neural networks in recommender systems: a survey, ACM Computing Surveys, 55(5):1-37, 2022b.
Zhou, Z., Wang, Y., Xie, X., Chen, L., and Liu, H. Riskora- cle: A minute-level citywide traffic accident forecasting framework. In Proceedings of the AAAI conference on artificial intelligence, volume 34, pp. 1258-1265, 2020.
Wu, Y.X.., Wang, X., Zhang, A., He, X., and Chua, T.- S. Discovering invariant rationales for graph neural net- works. arXiv preprint arXiv:2201. 12872, 2022c.
Wu, Z., Ramsundar, B., Feinberg, N., Gomes, J., Ge- niesse, C., Pappu, A. S., Leswing, K., and Pande, V.
Zhu, J.JJ., Jitkrittum, W., Diehl, M., and Schölkopf, B. Ker- nel distributionally robust optimization: Generalized du- ality theorem and stochastic approximation. In Interna- tional Conference on Artificial Intelligence and Statistics, pp. 280-288. PMLR, 2021.
Moleculenet: benchmark for molecular machine learn- ing. Chemical science, 9(2):513-530, 2018.
Xia, Y. Liang, Y., Wen, H., Liu, X., Wang, K., Zhou, Z.,, and Zimmermann, R. Deciphering spatio-temporal graph forecasting: A causal lens and treatment. arXiv preprint arXiv:2309. 13378, 2023.
7i, C., Zhao, H., Sun, X., Lin, Y., Cheng, H., and Li, Prog: A graph prompt learning benchmark. arXiv preprint arXiv22406.05344 2024.
Xu, K. Hu, W., Leskovec, J., and Jegelka, S. How powerful are graph neural networks? arXiv preprint 2018.
Zou, D., Liu, S., Miao, S., Fung, V., Chang, S., and Li, P. Gdl-ds: A benchmark for geometric deep learning un- der distribution shifts, arXiv preprint 2:10023310.88677 2023.
Yang, N., Zeng, K., Wu, Q., Jia, X., and Yan, J. Learning substructure invariance for out-of-distribution molecular representations. Advances in Neural Information Process- ing Systems, 35:12 12978. 2022.
Ying, Z., Bourgeois, D., You, J., Zitnik, M., and Leskovec, J. Gnnexplainer: Generating explanations for graph neural networks. Advances in neural information processing systems, 32, 2019.
You, Y., Chen, T., Sui, Y., Chen, T., Wang. Z., and Shen, Y. Graph contrastive learning with augmentations. Advances in neural information processing systems, 33:5812-5823, 2020.
Yuan, H., Yu, H., Gui, S., and S. Explainability in graph neural networks: A taxonomic survey. IEEE transactions puttern analysis and machine intelligence, 45(5):5782- 2022.
Yuan, H., Sun, Q., Fu, X., Zhang, Z.,, Ji, C., Peng, H., and Li, J. Environment- -ware dynamic graph learn- ing for out-of-distribution generalization. arXiv preprint 2rXiv:2311.11114, 2023.
Zhao, T., Liu, Y., Neves, L., Woodford, O., Jiang, M., and Shah, N. Data augmentation for graph neural networks.
12

---

Enhancing Graph Invariant Learning from a Negative Inference Perspective
A. More Related works
A.1. OOD Generalization.
citto--Distribution (OOD) generalization learning refers to the task of adapting a model that has been trained on a specific distribution to effectively process data from a potentially different distribution. This study holds significant importance because the issue of data distribution shifts is common occurrence in the real world. External factors, such as changes in environmental conditions, technological advancements, evolving user preferences, can lead to shifts in the data distribution. Various approaches can be employed for OOD generalization, including data augmentation (Rong et al., 2019; Wang et al., 2021; You et al., 2020), domain adaptation (Wang & Deng, 2018), and causal invariant learning (Sui et al.. 2022; Wu et al., 2022c). Jia et al. (Jia et al., 2024) innovatively proposes a mixup-based environment modeling framework, IGM, to enhance graph invariant learning. IGM focuses on expanding the environment space through a mixing generation scheme, while our NeGo aims to mine environmental space as much as possible from the novel perspective of negative learning. Piao et al. creatively proposes hierarchical environment inference paradigm to enhance graph invariant learning methods (Piao et al., 2024). This work focuses on generating sample-level hierarchical environments expand the modeling of the environment space. Unlike this method, our NeGo focuses on class-level environment augmentation, collaborating with extra-class environment modeling and inter-class invariant learning to achieve global inference of environment space.
Among them, causal invariant learning demonstrates impressive performance in various fields, due to its powerful inter- pretability (Chen et al., 2022; Li et al., 2022; Miao et al., 2022; Wu et al., 2022c). Our NeGo is aligned with this research line, as an enviroemment-eentered invariant learning method based on causal theory. However, in the field of graph learning, most existing invariant learning methods focus on extracting the causal graph to achieve invariant learning. This strategy limits the inference space of the environments the dimension of spurious subgraphs, which hinders the ability of models to capture the complex environment states. In this work, we propose an invariant learning mechanism based on negative inference to address this limitation.
A.2. Prompt Learning
Prompt learning is proposed in NLP models to infer underlying semantic and potential causal associations in linguistic data. Many effective prompt methods have developed with the introduction of large language models, including some hand-crafted prompts (Brown et al., 2020), discrete prompts (Gao et al., 2020; Shin et al., 2020), and learnable prompts design (Li & Liang, 2021). There have been various works on the interaction of computer vision and natural language processing fields, e.g., text-to-image retrieval text-to-image retrieval (Wang et al., 2019), visual question answering(Anto) al., 2015; Rao et al., 2022; Zhou et al., 2022a) and so on.
In recent years. prompt learning has also been developed in the graph learning field (Sun al., 2022a; Li et al., 2024; Sun et al., 2023; Zi et al., 2024). Our approach is the pioneering effort to apply prompt learning to address the challenge of graph OOD generalization issue.
A.3. Comparisons to Existing Graph OOD Works
Environment--cntered studies (Chen et al., 2024; Gui et al., 2024; Li et al., 2022; Wu et al., 2022a; Yang et al., 2022) consider that the data distribution shifts stem from the changes of environments. Therefore, these practices enable the model to withstand data distribution shifts by inferring environment variables. Concretely, the networks are often trained with the objective of equipping models to effectively handle mixed environments scenarios. However, this design allows the networks to make narrow inference about the environments, and makes the networks unable to handle with distribution shifts in complex environments. We attribute this limitation of inference scale to the shortcomings of positive inference. which is proved both empirically and theoretically. Therefore, we propose negative inference mechanism to broaden the inference space for environments, without relying on the mixed environments hypothesis.
Our approach, which represents a pioneering practice in utilizing negative inference, is distinct from all existing practices in this field. DIR (Wu et al., 2022c) aims to identify causal patterns that are stable across different distributions and filter out spurious patterns that are unstable. This work is a classic work in the carly application of causal theory to address the challenge of graph OOD generalization. It focuses on obtaining invariant subgraphs with a positive inference manner. GIL (Li al., 2022) aims to capture the invariant relationships between predictive graph structural information and labels in mixture of latent environments through jointly optimizing three mutually promoting modules. This method relies on the mixing environment hypothesis and has limited inference space for environments. CIGA (Chen et al., 2022) build
13

---

Enhancing Graph Invariant Learning from a Negative Inference Perspective
three Structural Causal Models (SCMs) characterize the distribution shifts that could happen on graphs: one is model the graph generation process, and the other two are to model two possible interactions between invariant and spurious features during the graph generation, i.e., FlIF and PIIF. This work provides fresh perspective existing research on out-of- cuttof-diitributionn generalization based on causality. However, it still falls within the framework of positive inference, aiming to extract causal subgraphs. GALA (Chen et al., 2024) utilized proxy prediction mechanism to infer environment label. It is worth noting that the negative samples mentioned in this work are different from our negative inference, and their design is also to improve performance under the mixed environments hypothesis. Thus, it essentially follows positive inferring process for environment variables. LECI (Gui et al., 2024) primarily focused on spurious substructures space to model the environment variables. Such environment inference strategy still relies on a positive inference with narrow cognitive space of the environments. Unlike existing graph OOD researches that centers environment awareness, our work presents greater technical challenges, and targets scenarios that are more complex. G-Splice (Li et al.) focuses on linear mixed scenarios with spurious structures, whereas our work aims to address the effectiveness of models in nonlinear mixed scenarios. AIA (Sui et al., 2024) aims to model the environment factors in graph through the lens of data augmentation. Unlike AIA that focuses only on modifications within spurious subgraphs inside samples, our NeGo more concerned with environment extraction in mixed scenarios at the class level. Thus, in scenarios where the spurious subgraphs within a sample are highly complex, we believe that our method more effective.
B. Detailed Datasets
GOOD-Motif (Wu et al., 2022c) is synthetic dataset designed for studying structure shifts. Each graph in the dataset is created by connecting base graph and motif, where the label is determined by the motif. This accessible ground-truth substructure brings a lot of convenience to the invariant subgraph learning with interpretability. This dataset include five label-irrelevant base graphs (wheel, tree, ladder, star, and path) and three label-determining motifss (house, cycle, and crane) are used to generate the graphs in the dataset. In environment-centers invariant learning, such base graphs can be seen as environment factors and such motifs are be consider as the casual factors.
GOOD-CMNIST (Gui et al., 2022) is a semi-synthetie dataset that has been purposefully created to evaluate node feature shifts. It comprises graphs constructed from hand-written digits extracted from the MNIST database, with the transformation applied using superpixel techniques (Monti et al., 2017).
GOOD-HIV (Gui et al., 2022) is a compact and real-world molecular dataset that has been derived from (Wu et al., 2018). It comprises molecular graphs, where atoms represent nodes and chemical bonds represent edges. The primary task associated with this dataset is to predict a molecule's potential for inhibiting HIV replication. Its distribution shift scenario is developed into two, i... the scaffold, and the size of nodes in a molecular graph.
(Ji et al., 2022) is utilized in the Ligand- Affinity Prediction (LBAP) task, where the core noise level and IC50 measurement type serve as domain features. Its distribution shift scenario is developed into three, i.e., the scaffold, the size, and the assay.
TrugOOD(LBAP-core-e
GOOD-SST2 (Yuan et al., 2022) is real-world social sentiment dataset derived from natural language. This dataset represents each sentence as graph, where individual words are treated as nodes, and their corresponding word embeddings serve as node features. The primary task in this dataset involves binary classification, aiming to predict the sentiment polarity of each sentence.
GOOD-Twitter (Yuan et al., 2022) is a real-world natural language sentiment dataset that shares the same transfor- mation process as the SST2 dataset. The classification task of this dataset involves predicting one of three sentiment polarities for each sentence. Similar to the GOOD-SST2 dataset, the sentence lengths are chosen as the domains.
C. Detailed Baselines
DIR (Wu et al., 2022c) an early work using causal theory address the distribution shifts issue in graph data. This work provides detailed theoretical proofs that demonstrate the feasibility of extracting invariant subgraph from graph data.
GSAT (Miao et al., 2022) employ information bottleneck theory to select causal subgraphs under onlythe FIIF assumption. The proposed stochastic attention mechanism in this paper is highly robust in extracting casual subgraphs,
14

---

Enhancing Graph Invariant Learning from Negative Inference Perspective
Table 6. Statisties the number of graphs in the datasets.
Dataset
Training
ID validation
ID test
OOD validation
OOD test
GOOD-HIV-Scattole
24682
4112
4112
4113
4108
GOOD-HIV-Size
26169
4112
4112
2773
3961
GOOD-SST2-Length
24744
5301
5301
17206
17490
GOOD-Twitte-LLnnggh
2590
554
554
1785
1457
GOOD-CMNIST- Color
42000
7000
7000
7000
7000
GOOD-Motif-Banl
18000
3000
3000
3000
3000
GOOD-Motif-Sizz
18000
3000
3000
DrugOOD-assay
34179
11314
11683
19028
19032
DrugOOD-size
36597
12153
12411
17660
16415
Table 7. Comparison of existing methods on addressing OOD generalization issuc.
Methods
SCMs
Inferred Environment Space
DIR
FIIF
Spurious subgraphs
GSAT
FIIF
Spurious subgraphs
CIGA
FII & PIIF
Spurious subgraphs
GALA
FII & PIIF
Spurious subgraphs
LECI
FIIF & PHIF
Spurious subgraphs
Intra-class spurious subgraphs and
NeGo
&
extra-class sample space
and has emerged as a backbone model in numerous methods. Actually, the subgraph extractor used in our work also inspired by GSAT.
CAL (Sui et al., 2022) is guided by the backdoor adjustment principle derived from causal theory. encourages the Graph Neural Networks (GNNs) to focus exploiting causal features while disregarding shorteut connections.
CIGA (Chen et al., 2022) is the first graph OOD method considering both Fully Informative Invariant Feature (FIIF) and Partially Informative Invariant Feature (PIIF) assumptions. This work presents OOD algorithm for graphs that is provably generalizable under different types of distribution shifts.
GIL (Li et al., 2022) is designed to capture invariant graph patterns in a mixture of underlying environments and handle the distribution shift issue. This work introduces GNN- subgraph generator to identify potentially invariant subgraphs from the complex interaction between invariant and variant patterns.
LECI (Gui al., 2024) comprehensively reviews existing OOD approaches and identifies the current causalsubgraph discovery challenges. This work jointly optimize label and environment causal independence achieve powerful causal subgraphs learning.
GALA (Chen et al., 2024) designs additional assistant model to enhance model with more powerful OOD gen- eralization ability without explicit environment labels. Theoretical proofs establish that GALA possesses robust out-of-distribution generalization capabilities under the FIIF and PIIF assumptions.
Y
G
Y
G
G
G
GE
G
G
G
E
E
E
(a). Traditional SCM
(b). FIF SCM
(c). PIIF SCM
Figure 4. Illustrations of three structural causal models (SCMs).
15

---

Enhancing Graph Invariant Learning from Negative Inference Perspective
D. Detailed Proofs
D.I. Proof of Proposition 4.3.
The basis Es represents a set of fundamental components or features that can accurately represent the entire environment space. These components capture the essential variations, patterns, and characteristics present in the environment. However, if the inference process fails capture this basis, it implies that the process is unable fully understand and model the complexities of the environment. Thus, next investigate that whether the environment variable inferred from Gş covers such base environments. We consider two SCMs hypotheses FIIF and PIIF as shown in Fig. 4.
Under the FIIF assumption, YLGs|Ge. we have P(Y, assumption leads to an equivalent expression: G) = P(Y|Gs; Gc) = P(Y|Gc). Therefore, the process of extracting the causal subgraph Ge is equivalent to the process of modeling the spurious correlations Gg. Traditional positive casual learning methods are capable of handling the FIIF assumption.
= P PGGs(Gc). This conditional independence
Under the PIIF assumption, YAGs Gc. we have P(Y, Gs Gc) & P(Y]Gc) P(Gs|Gc). Furthermore, we can obtain P(Y[G) P(Y|Gs, Gc) + Gc). Thus, the process of extracting the causal subgraph Ge cannot be used to infer the labels of samples. More formally, using mutual information theory, we derive the following.
GslGG) = H(YIGe) - H(YIGs, Gc) > 0,
(12)
H(Y|Gc)> H(Y|Gs, Gc).
(13)
This indicates that, given the causal subgraph Gc. the uncertainty of Y is higher than when both the spurious subgraph Gs and the causal subgraph Go are given. It suggests that the spurious subgraph Gg contains additional information of Y.
Therefore, the causal subgraph Ge learned by the model with the positive learning manner contains components of the spurious subgraph, i.c. Gg n Gc # 0. At this point, we can obtain the basis for the environment space, the model should be able to infer the spurious subgraph Gs and treat it as part of the environment E. The extracted causal subgraph Go should be able to effectively remove the spurious subgraph, i.e., G3 GC = 0. This clearly contradicts the PIIF assumption, indicating that the model currently lacks the capability obtain basis for the environmental space. Therefore, simply inferring the causal subgraph with a postive manner is not sufficient to address the PIIF assumption. Since E - Gs. modeling the spurious subgraph Gş requires modeling and understanding its root E. Existing methods that simply model G - Gc also lack the capability to address the PIIF assumption.
D.2. Proof of Proposition 4.4.
The optimization of Eq. 3 enables a broader scale environment inference space by cooperatively modeling intra-class spurious subgraphs and extra-class samples. Given that max (EE;; GclY) ) max (EE; Gs(Y), maximizing (EE Gs(Y)
implements the inference process for intra-class spurious subgraphs. Consider ()) = Σ G(E; maximizing
WEY
G| implements the modeling of extra-class sample space. The optimization procedure of max I (EE; GIGY)
indicates that all other extra-class samples (GO) y/s E YY are modeled as environment variables when making environment inference on samples with label Y. Therefore, the optimization process for Eq. 3 encompasses a broader cognitive space for environments, with its upper limit being the ground-truth environment distribution.
D.3. Proof of Theorem 5.1.
Given that PIIF shifts in the absence of environment labels are more challenging (Chen et al., 2024), our work focuses on the ability of NeGo the PIIF assumption, namely PIIF implies that the causal variable Ge indirectly influences the spurious variable Gs through the mediator Y. In the following analysis, analyze the two specific scenarios under PIIF assumption, i.e., H(GelY) < H(GslY) and H(GelY) > HGGs\Y). NeGo aims to comprehensively capture the underlying environment space by inferring the extra-class sample space and the intra-class spurious subgraphs. The learning objective of extracting casual subgraph Gc can be rewritten as follows,
arg max (T(G)) G2IC) T(Gc.GIV) Ve,e,EE
(14)
= arg max (-I ()) 1 GZI1). ee,e,EE
16

---

Enhancing Graph Invariant Learning from Negative Inference Perspective
where Gg denotes the extracted causal subgraph under any environmental scenario e,. The first term represents the constraint of negative inference, meaning that NeGo models all extra-class samples as environmental space. The second term represents the constraint of positive causal inference, meaning that the causal subgraph extracted under any environmental condition remains consistent, and is most useful for label prediction. Next, we will demonstrate that NeGo can address the two scenarios of the PIIF assumption.
For the case of ((Cc;)) > H(Gc) - H(Gs), we can get following derivation,
H(Gs|Y) H(GelV),
(15)
H(Gs) I(Gs;Y) > H(Gc) - I(Go;Y),
(16)
H(Gs) H(GG) (GG; T(Gs;Y) 0,
(17)
I(Go;Y) > H(Gc) - H(s)).
(18)
We can get that inferring Gc from is more effective and seamless compared to simply separating causal and spurious substructures based on entropy differences. Thus, our positive inference approach, arg max IGG Gg Y), is sufficient to Ve,,aEEE
achieve the decoupling of Ge from the label Y.
For the case of T(Gc; < H(Gc) - HGGs), we get T(Ge; Y) < H(GC) H(s)). This means that we need to consider entropy differences in the data composition assess the differences between causal and spurious relationships. In other words, positive inference arg max I G (Y) alone may result in Gc containing spurious subgraph information,
Ee
meaning Gs Ge. Fortunately, our negative inference strategy further refines Ge by considering entropy differences ((Gc))HHCGS to better distinguish between causal and spurious relationships. Specifically, our Go is also subject to this constraint through a negative inference approach to learn Gs.
Go E G - arg max
(19)
E. More Experiment Results
E.1. Implementation Details
We implement our Neqo and parts of baselines with PyTorch 1.10.1 on a server with NVIDIA A100-PCIE-40GE All experiments are repeated with 10 different random seeds of 22,334,,,667,88,9,10, The reported results include the mean and standard deviation obtained from these runs. During the training stage, we employ the Adam optimizer. We set the maximum number of training epochs 200. The batch size of training uses batch size of 64. For GOOD MMtif, GOOD-CME CMNIST and GOODSST2, the learning rate is set to 5 x 10-4, For GOOD HIV, GOODD-wittee and DrugOOD, we exploit learning rate of 10-4, Additionally, we utilize weight decay of 10-4 to help with regularization and prevent overfitting. The experiment setup of all baselines same as (Gui et al., 2024).
32 except for GOOD-CMNIST, which
E.2. Performance on the Synthetic Datasets
As shown in Tab. 8, our method achieves optimal performance on almost all datasets. Besides, the performance of environment-centered OOD methods, such as LECI and GALA, often achieves suboptimal or even optimal results on various datasets. This demonstrate the effectiveness of modeling environment factors in addressing data distribution shifts.
E.3. Hyperparameter Sensitivity Analysis
To investigate the sensitivity of À to performance in Eq. 11, we conducted experiments on both GOOD-HIV-scafoold and BrugOOD-assay As illustrated by the results from Fig. 5(b), the performance of NeGo improves significantly when = 1. We also find that neither smaller nor larger values of A lead to improvements in performance. Therefore, only by jointly utilizing both positive learning and negative inference can better performance achieved.
17

---

Enhancing Graph Invariant Learning from Negative Inference Perspective
Table 'The accuracy of NeGo on synthetic datasets, where GOOD-Motif has structure shift and GOOD-CMNIST has feature
shift. indicates statistical significance against the second second-best results.
GOOD-Motif
GOOD-CMNIST
Model
basis
size
color
covariate
ERM
60.93:11.111
56.63:17.12
26.64:12.37
57.56:19..5
IRM
64.9414.85
54.52:3.27
29.6342.06
58.11#5.14
V-Rex
61.59+6.58
55.85+19.42
27.13++2.90
48.78+7.811
IB-IRM
63.45:1:5.2
52.76:14.67
28.95:11.98
50.56:16.62
DIR
34.39=2.02
43.11
22.53:2.56
44.67=0.00
GSAT
62.27+8.79
50.03+5.771
35.02+2..78
68.22+7.23
CAL
59.4513.34
51.2742.50
28.87
52.59-2.76
CIGA
37.81
51.87:5.15
25.06:13.07
56.78+2.99
GIL
68.4812.46
63.61:2.75
47.3242.27
57.61
LECI
84.56+2.22
71.43#1.96
51.80#2.71
83.20
GALA
80.95+1.31
7..45+1.30
52.68
81.23
NeGo
8399
72.65
53.28
82.43
E.4. Case Studies
We also explore whether incorporating prompt learning can enhance the model's s performance, rather than our overall negative prompt framework. To this end, develop a variant of our NeGo framework, referred as PoGo, which incorporates the positive prompt practice. We evaluate the effectiveness (ROC-AUC) of PoGo four distribution shift datasets. We present the final performance by averaging the results from two runs conducted on an NVIDIA A A100-PCIR-40GB with different random seeds. As shown in Fig. 5(a), the performance of PoGo is competitive with recent successful practices like LECI and GALA, demonstrating that the design of positive prompt can still obtain excellent generalization. However, our framework of negative prompt shows superior performance.
We further investigate the reason of such performance of positive prompt practice PoGo. We modify PoGo by masking the posi (the original Negative Loss Lmage), obtaining PoGo (w/o. Lposi). With all other configurations remaining the same, observe significant decrease in the performance of PoGo (w/o. Epozi). Our analysis is follows: although both Cpost and Lpred are positive losses in PoGo, we argue they serve different purposes and convey distinct information. pprompt- as guidance strategy for the positive prompt, guides the prompt module to learn more potential environment semantics, while pree enhances prediction accuracy. Without prompt guidance fprompt, the advantage of prompt learning is not released. Therefore, we argue that positive prompt may also enhance the model capture broader scale environments. A more in-tepth investigation will be left for our future work.
GOOD-HIV-ccfflil DrugOOD-assa
Perform Perfommance Metric
Brg
Faleors Gpocal
Hyperparameter
Figure 5, Left: the performance comparison of PoGo, which is a variant of our NeGo framework by incorporating the positive prompt practice. Right: hyperparameter sensitivity analysis.
18

---

Enhancing Graph Invariant Learning from Negative Inference Perspective
F. Future Works
Our design can effectively solve the existing challenges, but there still exist a limitation. The negative prompter in our approach learns class-specific environment embeddings by considering all extra-class samples as environment variables. This results in our method relying on the class information of the dataset. With larger number of classes, the model is better equipped to capture and recognize complex underlying environment factors. When the dataset is limited to binary classification task, environment factors always present within the in-class samples. In this case, our negative prompter may have reduced capability to expand the environment inference space. The reason for this limitation is that the model is sensitive to the characteristics of dataset. Actually, we can realize that environment variables are often shareable across datasets. Therefore, it is promising research direction to study cross-task graph OOD work to capture broader environmental information. In the future, we aim investigate transferable multi-task graph OOD generalization learning, which is not discussed in existing works.
19


Reviews:
[{"review_id": "0LZjWPc5VN", "reviewer": "Reviewer_cMu6", "summary": "Through the experimental exploration, this paper observe that the existing approaches lack the ability to decouple causal subgraphs from complex environments. To tackle this problem, a negative prompt environment inference framework is proposed. The theoretical analysis, along with the excellent experimental performance, effectively demonstrates the superiority of the proposed method.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear Reviewer cMu6,\n \nWe appreciate your time on reviewing our work. We will provide further clarification to address your concerns. \n\n**W1 and Q1. Complex environments.**\n\nOur proposed NeGo remains effective in other complex environment scenarios, not limited to the size-dependent environment cases. Actually, we have discussed the effectiveness of NeGo in scenarios where spurious subgraph styles become more complex in Sec 6.3. As shown in Tab. 3, we observe the superior performance of NeGo in style-dependent complex environment cases, achieving a 1.29% performance improvement. \n\n\n**W2 and Q2. Prompt learning and negative learning.**\n\nPrompt learning and negative learning collaboratively contribute to performance improvement, which has been studied in Apendix E.4. Given your valuable suggestion, we will move the discussion from the Appendix to the main text in the revised version of our manuscript. Specifically, we designed two variants of NeGo: PoGo uses a positive learning mechanism, and PoGo (w/o. prompt) masks the prompt learning module. The performance comparison results we obtained are as follows: NeGo > PoGo > PoGo (w/o. prompt). Therefore, using only negative learning mechanism or only the prompt module does not achieve the best performance.\n\n\n**W3. Detailed discussion of related works.**\n\nWe will emphasize the researches on graph OOD from an environment perspective with more details in Sec 3. \n\n*GALA investigates the problem of invariant graph representation learning from the perspective of environment augmentation. LECI introduces the environment discrimination module and utilizes an adversarial learning mechanism to achieve invariant learning. Our proposed NeGo share the same research goal with those works, achieving better invariant learning mechanisms through environment modeling. However, our research persepctive and techniques for achieving this goal are completely different with them. Specifically, we not only focus on environment shift issues in linear mixed scenarios, but also place greater emphasis on model performance under complex and nonlinear environment shifts. Technically, we use prompt learning and negative learning mechanisms to capture the environment.*\n\nThank you again for your positive comments on our work, which are invaluable for improving our manuscript. We look forward to your further feedback.", "claims_and_evidence": "Yes. All claims made in the submission supported by clear and convincing evidence.", "methods_and_evaluation_criteria": "Yes. The proposed method and evaluation criteria make sense for the problem at hand.", "theoretical_claims": "Yes, I have checked the correctness of all proofs for theoretical claims. All theoretical analysis is relatively solid.", "experimental_designs_or_analyses": "Yes, I have checked the validity of all experimental designs. It is better to provide more interpretable analysis.", "relation_to_broader_scientific_literature": "The contributions of this work are both technical and innovative, and it is an excellent scientific literature.", "essential_references_not_discussed": "The works discussed in this literature is very fresh and representative.", "other_strengths_and_weaknesses": "Strengths: \n\n1. The topic on studying graph earning methods in the scenario of complex environments shift is important to the community. The authors provide new insights to show the relationship between positive and negative learning.\n\n2. The paper’s core contribution, a negative inference graph OOD framework (NeGo), is thoroughly tested and demonstrates strong empirical performance. This paper compares the performance of NeGo with fresh baselines, and conducts thorough discussions under the cases of complex environment shifts. \n\n3. The presentation of the paper is clear. The overall logical structure of the paper is clear and easy to follow.\n\nWeaknesses:\n\n1. While the paper presents a compelling model for using negative learning mechanism and prompt learning, a discussion on whether NeGo can handle any of the complex environment shift challenges. For example, when the environment does not refer to the size of the graph but to other complex cases, whether the proposed method is still valid.\n\n2. It is well known that prompt learning has achieved success in many fields. Therefore, I believe it is important to explore in more detail whether prompt learning or negative learning specifically contributes to the performance improvements presented in this paper.\n\n3. It would be beneficial to highlight studies that are closely related to this paper and allocate more space for a detailed discussion of these works in Section 3.", "other_comments_or_suggestions": "No Other Comments Or Suggestions.", "questions_for_authors": "1. Whether the model is valid after the environment factors in the graph becomes more complex?\n\n2. Which plays a more critical role, prompt learning or negative inference?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "zh8IQJUuFk", "reviewer": "Reviewer_sxw9", "summary": "In this work, the authors tackle the complex environment shift challenge in graph learning. Through theoretical analysis and an in-depth discussion of the challenge, the authors propose the negative prompt graph learning framework NeGo. Extensive experiments on real-world datasets across domains and synthetic datasets validate the effectiveness of NeGo.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear Reviewer sxw9,\n\nThank you for taking valuable time to review our work. We have carefully considered your comments, and the following are our detailed responses. We hope these details could address your concerns.\n\n\n**W1. The environment in the graph data.**\n\nThe environment factors in graph data are highly informative and extend well beyond just graph scale. In graph-structured data, environment variables are typically defined as all information (spurious subgraph) beyond the causal subgraph. In synthetic datasets, researchers commonly simulate environment variations by increasing the size of spurious subgraph. For example, in GOOD-Motif dataset, base graphs (label-irrelevant subgraphs) can be seen as environment factors and  motifs (label-determining subgraphs) are be consider as the casual factors. In real molecular datasets, the environment variables are often characterized by identifying molecules with complex structure and a diverse range of atomic types. \n\nActually, the complex environments discussed in our manuscript encompass not only changes in scale but also variations in subgraph types. The environment variables for each dataset are detailed in the table below.\n\n\n|      | Environment factor | \n|   :--:   | :--:   | \n| GOOD-Motif |  Label-irrelevant base graphs | \n| GOOD-CMNIST | Color | \n| GOOD-HIV |  The scaffold and the size  | \n| DrugOOD(LBAP-core-ic50) | The scaffold, the size, and the assay | \n| GOOD-SST2  | Sentence lengths | \n| GOOD-Twitter   | Sentence lengths | \n\n\n**W2. A discussion of experiment phenomenon.**\n\n\nThank you for your insightful comment. In real-world tasks, causal variables extend beyond the subgraph itself and also incorporate additional contextual information. We aruge that negative learning may cause the model to actively suppress certain features that are beneficial for ID scenarios but ineffective in OOD scenarios. The phenomenon where our NeGo achieves suboptimal performance on ID val but delivers the best results on OOD val validates our view. \n\n**W3. FIIF and PIIF.**\n\nFIIF and PIIF represent two possible interactions between invariant and spurious features during graph generation, as modeled by SCMs [1]. FIIF and PIIF have been accepted by many studies for verifying whether the proposed model demonstrates comprehensive invariant learning ability [2,3]. \n\nUnder the FIIF assumption, the invariant feature $C$ fully contains all the information about the label $Y$, i.e., $(S, E) \\perp Y \\mid C $. This means that given $C $, $S$ and $E $ are independent of $Y$. In the structural causal model under FIIF, we have $Y := f_{\\text{inv}}(C)$, $S := f_{\\text{spu}}(C, E)$, $G := f_{\\text{gen}}(C, S)$. Here, $f_{\\text{inv}}$ is the label generation function, $f_{\\text{spu}}$ describes how $S$ is influenced by $C$ and $E$, and $f_{\\text{gen}}$ is the graph generation function. Under the PIIF assumption, the invariant feature $C$ only partially contains the information about the label $Y$, i.e., $S$ is indirectly dependent on $C$ through $Y$. In the structural causal model under PIIF, we have $Y := f_{\\text{inv}}(C)$, $S := f_{\\text{spu}}(Y, E)$, and $G := f_{\\text{gen}}(C, S)$. Here, $f_{\\text{inv}}$ is the label generation function, $f_{\\text{spu}}$ describes how $S$  is influenced by $Y$ and $E$, and  $f_{\\text{gen}}$ is the graph generation function.\n\nIn both FIIF and PIIF, the interaction between $C$ and $S$ is different, leading to distinct distribution shift patterns. In FIIF,  $C$ directly controls  $S$ , while in PIIF, $C$ indirectly controls $S$ through $Y$ . Understanding these differences is crucial for addressing distribution shifts. In our work, we also validate that our NeGo possesses comprehensive causal learning ability through theoretical proof. In the next version of our manuscript, we will introduce more discussions on the importance of studying causal learning under the FIIF and PIIF assumptions.\n\n\n**Typos.**\n\nMany thanks for your careful review. We will certainly correct the typos you pointed out and proceed to check our manuscript in greater detail. We will carefully consider your comments and revise our manuscript to improve it further. Looking forward to your feedback!\n\n**References:**\n\n[1] Chen, Yongqiang, et al. \"Learning causally invariant representations for out-of-distribution generalization on graphs.\" Advances in Neural Information Processing Systems 35 (2022): 22131-22148.\n\n[2] Chen, Yongqiang, et al. \"Does invariant graph learning via environment augmentation learn invariance?.\" Advances in Neural Information Processing Systems 36 (2023): 71486-71519.\n\n[3] Gui, Shurui, et al. \"Joint learning of label and environment causal independence for graph out-of-distribution generalization.\" Advances in Neural Information Processing Systems 36 (2023): 3945-3978.", "claims_and_evidence": "Yes. The claims of this manuscript are supported by clear observations and convincing experiment results.", "methods_and_evaluation_criteria": "Yes. The research motivation of this manuscript stems from the limitations of existing works, and an innovative approach is proposed to address these issues. As a result, it contributes to advancing the field.", "theoretical_claims": "Yes. The theoretical claims of this manuscript are proved in the Appendix.", "experimental_designs_or_analyses": "Yes. The experimental designs of this manuscript are comprehensive, including comprehensive datasets, a robust set of baselines, and a wealth of experimental results.", "relation_to_broader_scientific_literature": "The topic explored in this manuscript is one that has been extensively studied in recent scientific literatures. As a result, the proposed framework can offer novel ideas (negative prompt) to advance scientific research in the field.", "essential_references_not_discussed": "No. Section 3 and Appendix A provide a comprehensive analysis of relevant works, including many classic studies of graph learning, and recent researches of graph OOD learning.", "other_strengths_and_weaknesses": "The strengths I found are listed as below,\nS1. The motivation of this research is significant. Section I presents many intuitive examples and descriptions of existing work, effectively helping reviewers and readers grasp the focus of this research. \n\nS2. The background is comprehensive, including prompt learning and two distribution shift hypotheses. Besides, this paper provides a clear analysis of the differences between the proposed method and existing works.\n\nS3. The experiment design in this paper is thorough. The four Questions preset in the experiment can fully prove that the proposed method is effective in tackling existing issues. In the comparison of model performance, it is convincing that the authors used the significance hypothesis for validation.\n\nS4. The reproducibility of this research is trustworthy, and the technical implementation is provided.\nThe weaknesses are listed as follows,\nW1. Environmental variables in OOD scenarios typically refer to external factors that influence model performance and data distribution, which may differ from the distribution of the training data. Are the environment variables in graph-structured data limited solely to the size of the graph, or do they also encompass other factors such as node attributes, edge relationships, or external contextual information that may influence the graph's behavior?\n\nW2. In the experiment section, I observe that the suboptimal performance of NeGo is often concentrated in the 'ID val' setting. This suggests that the same model tends to exhibit different preferences under 'ID val' and 'OOD val' conditions. The authors do not provide a thorough discussion of this phenomenon.\n\nW3. The paper provides extensive theoretical Proof to verify NeGo can handle both the FIIF and PIIF assumptions. What is the significance of distinguishing FIIF and PIIF two assumptions?", "other_comments_or_suggestions": "There are also a few typos and statements require to be corrected. Line 204: delete ‘first’; Line 268: some references should be added after ‘various practices’; Line 351: ‘supporting’ should be modified to ‘to support’.", "questions_for_authors": "Please answer the Weaknesses (W1-W4).", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Ft30eYKpcN", "reviewer": "Reviewer_wCN4", "summary": "This paper introduces a negative inference graph OOD framework (NeGo) to broaden the inference space for environment factors. Technically, this paper is inspired by the concept of prompt learning and, from the perspective of negative learning, propose an innovative approach called negative prompt. Experimentally, this paper focuses on the graphs with complex environments, and demonstrates the generalization ability of NeGo in complex environments by comparing it with baselines.\n\n## update after rebuttal\nI appreciate the authors' response. I will maintain my positive score.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear Reviewer wCN4,\n\nThank you for taking valuable time to review this work. We have carefully considered your comments, and the following are our detailed responses. We hope these details could address your concerns.\n\n**W1. Parameterized networks**\n\n${f_\\phi }$ represents the negative prompter to learn negative embedding, ${g_\\xi }$ and ${g_\\theta }$ are subgraph extractor and final predictor. We will update Fig. 2 by adding the parameter networks to the structure figure in the next version.\n\n**W2. Negative learning and negative sampling**\n\nThe differences between our negative learning (NL) and negative sampling (NS) is as follows:\n\n- NS primarily aims to optimize training efficiency, while NL focuses on enhancing the model's generalization and distinguishment ability.\n\n- NS involves randomly selecting a small number of negative samples from all negative samples for training. In contrast, our NL design negative loss to explicitly guide the model in learning how to distinguish between positive and negative samples, and use prompt mechanism to increase the contribution of negative samples to the model's generalization.\n\n\n**W3&4. Quantitative evaluation and Training efficiency**\n\nWe will add the following in the description: *Our NeGo achieves optimal performance across four datasets, with the highest performance improvement of 1.17%.* In the next version of the manuscript, we will incorporate Appendix E.5 (Efficiency Analysis) into the main text.\n\n\n**W5. typo**\nMany thanks for your careful review. We will certainly correct the typos you pointed out and proceed to check our manuscript in greater detail.\n\n\n**Q1. Unique design of graph-structured data**\n\nThe unique prompt design of our NeGo lies that the prompt outputs serves as a latent embedding for extracting subgraphs, rather than being directly used for prediction. Given that label-related information in graph-structured data corresponds to subgraphs, we employ prompt learning with the goal of disentangling causal subgraph from spurious subgraph. \n\n\n**Q2. Intuitive validations** \n\nThe effectiveness of negative prompting in NeGo can be intuitively understood from following several perspectives:\n\n- Broadening the Environment Inference Space. Negative prompting allows the model to consider all extra-class samples as potential environments. This significantly expands the scope of environment factors that the model can infer, beyond just the spurious subgraphs within the same class.\n\n- Our Case Studies and Visualization. In Appendix E.4, we show that the model consistently identifies the ground-truth subgraphs even in complex environments. This demonstrates that negative prompting helps the model accurately distinguish between causal and spurious factors.\n\n\n**Q3. Transferability**\n\n\n- Transferability Across Tasks. The negative inference mechanism of NeGo not only focuses on extracting causal subgraphs but also learns environment factors through negative prompting, a mechanism that may have a certain degree of universality across different tasks. Experimental results show that NeGo performs well in different tasks, such as molecular property prediction (e.g., GOOD-HIV, DrugOOD) and social sentiment analysis (e.g., GOOD-SST2, GOOD-Twitter). This shows that the design of NeGo can accommodate the demands of different tasks and has a certain level of transferability across tasks.\n\n- The design of NeGo is based on causal theory and negative inference, theories that have broad applicability across different fields (such as chemistry, biology, social sciences). Experimental results indicate that NeGo performs well in molecular property prediction tasks in the chemical domain and social sentiment analysis tasks in the social domain. This suggests that the design of NeGo can adapt to data distribution shifts in different fields and has a certain level of transferability across domains.\n\n\nThanks again for your constructive comments, which are valuable in helping us further improve our work. Look forward to your further feedback!", "claims_and_evidence": "Yes. The claims made in the submission are supported by clear and convincing evidence. The authors provide a solid theoretical foundation, backed by empirical experiments that reinforce their assertions.", "methods_and_evaluation_criteria": "Yes, the proposed methods and evaluation criteria are well-suited for the challenge in this field. The chosen baselines align with the goals of the research, and the evaluation criteria is appropriate for assessing the performance and effectiveness of the proposed approach.", "theoretical_claims": "Yes. I have reviewed the correctness of the theoretical proofs presented in the manuscript. The proofs for Proposition 4.3., Proposition 4.4. and Theorem 5.1. appear to be sound and logically consistent.", "experimental_designs_or_analyses": "Yes. The overall experimental setup, including the choice of datasets and evaluation metrics, seems well thought out and appropriate for this research.", "relation_to_broader_scientific_literature": "The key contributions of the paper build upon prior research in graph OOD. For instance, the authors extend the works [1-3], all of which focus on solving the distribution shift issue on graphs. Note that the contributions of this paper are quite novel, addressing a gap in the literature that has not been fully explored before.\n\n[1] Discovering invariant rationales for graph neural networks, ICLR 2022;  \n[2] Does invariant graph learning via environment augmentation learn invariance? NeurIPS 2023;  \n[3] Joint learning of label and environment causal independence for graph out-of-distribution generalization, NeurIPS 2023.", "essential_references_not_discussed": "The authors offer a thorough discussion of the related works, covering nearly all relevant references.", "other_strengths_and_weaknesses": "This paper is a good work with a solid motivation, clear writing, and novel method. Specifically, the strengths of this work excel in following aspects,\n\n- This paper tackles a novel and important issue, with limited prior work on focusing graph OOD challenge under complex environment shifts. \n\n- The authors empirically investigate the performance of existing models in complex environments shifts, effectively verify the significance of the work studied.\n\n- The technical contribution is substantial and innovative. To tackling existing issue, proven effective techniques and insights are successfully applied in this study. \n\n- The writing is clear, and the analyses are compelling. \n\nThe paper still has some notable weaknesses. If the following weaknesses are addressed, the paper could be significantly improved:\n\n- The paper includes numerous network architecture designs ($f_\\phi}$, $g_\\xi $, $g_\\theta$), but these parameterized networks are only described in text, making them hard to understand and visualize. For example, in Figure 2, it is difficult to understand at which stage $f_\\phi$, $g_\\xi $, and $g_\\theta$ appear and what roles they play. \n\n- The mechanism of negative inference aims to capture invariant subgraphs by modeling extra-class samples, which is similar with the concept of negative sampling in traditional machine learning. I'm confused about the distinction between these two concepts.\n\n- The abstract section should contain both an introduction to the framework and an evaluation of the effectiveness of the model. But the abstract of this work lacks a quantitative evaluation of the performance of NeGo. \n\n- The experimental comparison of training efficiency should be presented in the main text, not in the appendix. \n\n- Minor typo. line95 “while the causal subgraphs”-> “while the causal subgraph”", "other_comments_or_suggestions": "No", "questions_for_authors": "- Prompt learning has been widely studied in text and image structured data. What unique design or contributions do the authors make when transferring the mechanism of prompt learning to graph-structured data in this work?\n\n- I am confused about why negative prompts are so effective. Could you provide some intuitive validations to offer more insights? \n\n- What is the transferability of the proposed method? Could it inspire insights for other research, such as self-supervised learning?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "zagemgCLrE", "reviewer": "Reviewer_vzxf", "summary": "The paper presents NeGo, a negative inference graph OOD architecture designed to address complex environmental shifts in OOD scenarios. Extensive experiments demonstrate the effectiveness of proposed architecture.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear Reviewer vzxf,\n\nThank you so much for taking your valuable time to review our manuscript. We greatly appreciate the positive feedback on our work. Here we will carefully address your concerns.\n\n**W1&Q1. Intuitive analysis**\n\nPrompt learning and negative learning collaboratively contribute to performance improvement, which is studied in Apendix E.4. Given your valuable comment, we will move the discussion from the Appendix to the main text in the revised version of our manuscript. Specifically, we designed two variants of NeGo: PoGo uses a positive learning mechanism, and PoGo (w/o. prompt) masks the prompt learning module. The performance comparison results we obtained are as follows: NeGo > PoGo > PoGo (w/o. prompt). Therefore, using only negative learning mechanism or only the prompt module does not achieve the maximum performance.\n\n\n**W2&Q2. Learnable parameters**\n\nThe reasons for choosing learnable parameters as prompt guidance are as follows:\n\n- Limitations of manual design. Traditional manual prompts require expert knowledge and may not generalize well to new data or tasks. Especially in scenarios with data distribution shifts, manually designed prompts often struggle to generalize across all distributions.\n\n- A data-augmented perspective. The learnable prompt enables implicit enhancement by dynamically (learnably) updating the input prompts without directly altering the original data distribution. This design is particularly well-suited for tasks involving data distribution shifts.\n\n\n**W3&Q3. The efficiency of model**\n\nWe support that model efficiency is a crucial evaluation criterion. In the next version of the manuscript, we will incorporate Appendix E.5 and following table into the main text.\n\n| Models   | GSAT | DIR | CIGA | LECI | GALA | NeGo |\n|   :--:   | :--:   |  :--:   |  :--:   |  :--:   |  :--:   |   :--:   | \n| Time(s/epoch) | 51.6 |  52.6  |  54.2 |  59.1 |  62.3 | 58.7 | \n\n\nThank you again for taking the time to review our work! We will carefully consider your suggestions and further revise our manuscript to satisfy the high standards of ICML community.", "claims_and_evidence": "Yes, the Yes, this work brings a new perspective and method to graph OOD learning.laims are supported by both theoretical analysis and empirical experiments.", "methods_and_evaluation_criteria": "Yes, this work brings a new perspective and method to graph OOD learning.", "theoretical_claims": "Yes, the theoretical proofs in the manuscript are solid.", "experimental_designs_or_analyses": "Yes, the experiment section is structured around four Questions and provides a comprehensive discussion on the effectiveness of NeGo.", "relation_to_broader_scientific_literature": "This paper aims to solve the complex environment shift issue that can not be solved by the existing works, which is a further improvement on the existing methods.", "essential_references_not_discussed": "The introduction to related work is relatively thorough.", "other_strengths_and_weaknesses": "Strengths:\n1. This manuscript proposes a novel approach to address complex environment shift in OOD scenarios. By introducing an interesting perspective on negative learning, and a novel technique for prompt learning, the authors construct approaches that specifically address existing challenges.\n\n2. Both the technical details and experiments are very well explained and organized. This manuscript is well-written, the technical design is well-articulated, and the motivation is well-stated.\n\n3. The theoretical analysis presented in the paper is sound and rigorous. The reproducibility of the paper is good.\nWeaknesses:\n1. The technical contributions of this manuscript can be viewed in two aspects: the perspective of negative learning and the utilization of prompt learning. The manuscript lacks an intuitive analysis of their individual roles, such as whether these two elements must be combined jointly, what each contributes when combined.\n\n2. The learnable parameters (learnable prompt) is the input to the negative learning module and serves as a key component. Although this design has been validated by many previous works, the authors should provide a more thorough analysis of its function. \n\n3. Although the proposed framework does not involve excessive intermediate computations, it is necessary to discuss the efficiency of model.", "other_comments_or_suggestions": "No Other Comments.", "questions_for_authors": "1. Is it necessary for negative learning and the utilization of the prompt learning mechanism to be combined, and what does each contribute when they are used together?\n\n2. Why can the learnable prompt guide negative learning? Do the authors offer insights that differ from previous works?\n\n3. What is the training efficiency of the proposed model?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: In the initial submission, the reviewers maintained a generally positive attitude toward both the content and quality of this manuscript. Following the authors' detailed responses, the reviewers' overall sentiment remained highly favorable. I believe this manuscript makes a valuable contribution to the community. After careful evaluation and consideration, I am inclined to accept this paper.
Decision: Accept (poster)