Adaptive Mixture of Disentangled Experts for Dynamic Graph Out-of-Distribution Generalization
Authors: Haibo Chen
Xin Wang
Guanheng Chen
Yuan Meng
Haoyang Li
Yang Yao
Zeyang Zhang
Zhiqiang Zhang
JUN ZHOU
Ling Feng
Wenwu Zhu
Venue: ICLR 2026 Poster (2026)
Keywords: Dynamic Graph Neural Network; Out-of-Distribution Generalization; Mixture of Experts
Primary area: learning on graphs and other geometries & topologies
TL;DR: We propose a novel adaptive mixture-of-experts framework that dynamically routes disentangled architecture experts to evolving distribution shifts for dynamic graph.

Abstract: Dynamic graph out-of-distribution (OOD) generalization has drawn an increasing amount of attention in the research community, given its wide applicability in real-world scenarios. Existing methods typically employ a fixed-architecture design to extract invariant patterns. However, there may exist evolving distribution shifts in dynamic graphs, leading to suboptimal performance of fixed-architecture designs. To address this issue, we propose a novel adaptive-architecture design to handle evolving distribution shifts over time, to the best of our knowledge, for the first time. The proposed adaptive-architecture design introduces an adaptive mixture of architecture experts to capture invariant patterns under evolving distribution shifts, which imposes three challenges: 1) How to detect and characterize evolving distribution shifts to inform architectural decisions; 2) How to dynamically route different expert architectures to handle varying distribution characteristics; 3) How to ensure that the adaptive mixture of experts effectively discovers invariant patterns. To solve these challenges, we propose a novel **Ada**ptive **Mix**ture of Disentangled Experts (**AdaMix**) model to adaptively route architecture experts to varying distribution shifts and jointly learn spatio-temporal invariant patterns. Specifically, we propose a spatio-temporal distribution detector to infer evolving distribution shifts by jointly leveraging historical and current information. Building upon this, we develop a prototype-guided mixture of disentangled experts that adaptively routes experts with disentangled factors to different distribution shifts. Finally, we design a distribution-aware intervention mechanism that discovers invariant patterns based on expert selection of nodes. Extensive experiments on both synthetic and real-world datasets demonstrate that our proposed **AdaMix** model significantly outperforms state-of-the-art baselines.

Published as a conference paper at ICLR 2026
ADAPTIVE MIXTURE OF DISENTANGLED EXPERTS FOR DYNAMIC GRAPH CUT-OF-DISTRIBUTION GENERAL-
IZATION
Haibo Chen', Xin Wangl2; Guanheng Chen', Yuan Mengl Haoyang Li',
Yang Yao', Zeyang Zhang', Zhiqiang Zhang', Jun Zhou', Ling Feng', Wenwu Z2
"Department of Computer Science and Technology. Tsinghua University BBejjing National Research Center for Information Science and Technology
3 Ant Group
[chb24, chengh24 2 yaoyang2l, zy-zhang20 t twwnggui.....e............
xxinn_ wang, fengling, wwzhu @tsinnhua. edu. en, 1ihy21 (hbyy188gmmal 88gmail com iingaac. axgaaat comm, zhou jun com
ABSTRACT
Dynamic graph out-of-distribution (OOD) generalization has drawn an increasing amount of attention in the research community, given its wide applicability in real- world scenarios. Existing methods typically employ a fixed-architecture design to extract invariant patterns. However, there may exist evolving distribution shifts in dynamic graphs, leading to suboptimal performance of -rrhttcceeee designs. To address this issue, we propose novel adaptive-architettue design to handle evolving distribution shifts over time, to the best of our knowledge, for the first time. The proposed adaptive-aree architeccttree design introduces an adaptive mixture of architecture experts to capture invariant patterns under evolving distribution shifts, which imposes three challenges: 1) How to detect and characterize evolving distribution shifts to inform architectural decisions; 2) How to dynamically route different expert architectures to handle varying distribution characteristics: 3) How to ensure that the adaptive mixture of experts effectively discovers invariant patterns. To solve these challenges, we propose a novel Adaptive Mixture Disentangled Experts (AdaMix) model adaptively route architecture experts to varying distribution shifts and jointly learn spatio-temporal invariant patterns. Specifically, we propose spatio-temporal distribution detector infer evolving distribution shifts by jointly leveraging historical and current information. Building upon this, we develop a prototype-guice mixture of disentangled experts that adaptively routes experts with disentangled factors to different distribution shifts. Finally, we design distribution-awart intervention mechanism that discovers invariant patterns based on expert selection of nodes. Extensive experiments on both synthetic and real-world datasets demonstrate that our proposed AdaMix model significantly outperforms state-of-the-art baselines. Codes will be available at huttps//giitubb..om/aabbbb http://hhaaaao//aaiaiaaaaa 2/AdaMix
INTRODUCTION
Dynamic graph out-of-distribution (OOD) generalization (Zhang et al. 2022 2023a: Yuan al.
2023 Yang et al. 2024) aims to tackle distribution shifts and ensure effective generalization for dynamic graphs, whose structures and features evolve over time (Li et al. 2019 You et al. 2019 Wu et al. 2020). Existing methods for dynamic graph OOD generalization typically attempt to extract invariant patterns, i.e.. structures and features whose predictive abilities remain stable across shifts. For example, DIDA (Zhang et al. 2022) employs a disentangled spatio-temporal graph attention network encode node trajectories into invariant and variant representations, and then applies random interventions on the variant part to force predictions to rely on the invariant patterns.
"Corresoonding Authors.

---

Published as a conference paper at ICLR 2026
However, existing methods typically rely on exed-architectuue designs to extract invariant patterns, overlooking that distribution shifts in dynamic graphs are continuously evolving and may require adaptive architectures over time to extract optimal invariant patterns. For instance, in academic collaboration networks, the distribution of research topics may evolve with certain regularities. Typical phenomena include the growth in the number of publications (expanding graph size), the increasing density of citation relationships (rising node degrees), and the diversification of research fields (increasing feature diversity). This example illustrates a broader phenomenon where the characteristics of distribution shifts themselves evolve, exemplifying common distribution shifts in graphs (Gui al., [022] Li et al., (025a). Furthermore, the evolution of these distribution shifts may require changes in architectural requirements over time, as the required architectures are inherently shaped by the underlying data distributions (Niu et al., 2021 Wu et al. 2024).
In this paper, we propose novel adaptive-architecue design to address evolving distribution shifts in dynamic graphs, leveraging a mixture of experts (MoE) that dynamically adjusts the model architecture over time to capture invariant patterns more effectively. This design to handle evolving distribution shifts remains largely unexplored in the literature, and is highly non-trivial, presenting several key challenges: 1) How to detect and characterize evolving distribution shifts to inform architectural decisions? 2) How to dynamically route different expert architectures to handle varying distribution characteristies? 3) How to ensure that the adaptive mixture of experts effectively discovers invariant patterns?
To address these challenges, we propose a novel Adaptive Mixture of Disentangled Experts (AdaMix) method to adaptively route expert networks to different distribution shifts for jointly learning spatio- temporal invariant patterns. Specifically, we propose spatio-temporal distribution detector infer evolving distribution shifts based on historical and current information, which includes a memory vector for storing historical distribution information. Then, we develop a prototype-guides mixture of disentangled experts that adaptively routes experts to varying distribution shifts. Each expert associated with a disentangled prototype that captures a distinct factor of variation. Finally, we intervention mechanism that encourages nodes to be intervened upon
design distribution-awan by others from different distributions, leveraging expert-based interventions to discover invariant patterns. Extensive experiments on real-world and synthetic datasets demonstrate the effectiveneess of our proposed method, outperforming state-of-the-ar baselines. The contributions of this paper are summarized follows:
We propose a novel cllapivv--rrcitecture design Adantivv Mixture of Disentangled Experts (AdaMix)~to handle evolving distribution shifts in dynamic graphs, where the adaptive- architecture defined relative to the underlying data distribution. To the best of our knowledge, this is the first work to address dynamic graph distribution shifts from an architectural perspective. We observe that different timestamps under evolving distribution shifts may require distinct architecture designs, and further provide a theoretical analysis demonstrating the advantages of adaptive architectures over fixed ones in such cases.
We propose three key components to address adaptive mixture of experts for dynamic graph OOD generalization: i) spatio-temporal distribution detector; ii) prototype-guided disentangled experts; and iii) distribution-aware intervention mechanism.
We conduct extensive experiments on real-world and synthetic datasets, demonstrating the effec- tiveness of our proposed method, which outperforms state-of-the-at baselines.
2 PROBLEM FORMULATION AND NOTATIONS
In this section, we present the fundamental concepts and notations used throughout the paper, focusing on dynamic graphs and distribution shifts within them. Random variables are denoted using bold letters (e.g., G), while their realizations are denoted using italic letters (e.g.. 9).
Dynamic Graphs. We denote a dynamic graph as G = (G*) where T denotes the total number of timestamps. Each snapshot G (V*', corresponds to the graph at time f, with node set Y and edge set 3.. For simplicity, a snapshot can also be expressed as g' = (X', A'), where X is the node feature matrix and A' is the adjacency matrix. The prediction task graphs aims to leverage historical snapshots to make future predictions, i.e., p( ' Itt,, where G (G), .c.,, GO) represents the graph trajectory up time l,, and Y! denotes the target labels
dynamic
2

---

Published as a conference paper at ICLR 2026
(e.g., node properties future links) at time t + 1. Following (Zhang al. 2022) the entire trajectory can be factorized into ego-grapp gro-graph trajectories: pl
the distribution of
II. ply; G.²).
Distribution Shifts Dynamic Graphs. The standard learning objective is to optimize a predictor
under empirical risk minimization (ERM): ming (ygt::::"~<p<<('""" (y). (() vf)). where fo a parameterized dynamic graph neural network. However, under distribution shifts, the predic- tor trained the training distribution Par may not generalize to the test distribution Pie. since
(tr G o Pre(Y'. G (.)... Following the OOD generalization literature (Arjovsky et al. 2019
Wu et al. 2022b Gagnon-Audes
al. 2022 /hang al.] 2022), we adopt the following assumption regarding invariant and variant patterns under distribution shifts in dynamic graphs:
Assumption For a given task, suppose there exists a predictor f()) such that, for any distribution, each sample (Gh yb) be decomposed into two parts: invariant pattern Pf (v) and variant pattern PV (w). These patterns are required to satisfy: (I) the invariant patern alone is sufficient for prediction, i.e., y'= ((P( + e, where denotes noise; (2) the invariant pattern can be obtained by excluding the variant pattern frotternfrrm the observed trajectory, i.e., Pi(1) = G (P (u); (3) the effect of the variant pattern on the label fully shielded by the invariant pattern, i.e. y 1 (v) Pf(v).
MOTIVATION
In this section, illustrate the motivation of our proposed method. We begin by introducing the phenomenon of evolving distribution shifts in dynamic graphs, and then discuss the limitations of existing methods from an architectural perspective.
Evolving Distribution Shifts. In real-world dy- namic graphs, the underlying data distribution evolves continuously over time, ie., p( (G') # v(G') for I. + L'. To empirically validate this phenomenon, we analyze several dynamic graph datasets. As il- lustrated in Figure []] on the Collab academic col- laboration network, key graph statistics such as the number of nodes and the average node degree exhibit a continuous upward trend over time. This evolution reflects typical forms of graph distribution shifts (Gui) et al. 2022; Li et al. 2025a). Importantly, the pres- ence of some trends implies that historical trajectories contain valuable signals for inferring the current dis- tribution. Furthermore, because the graph distribution continuously evolves, the joint distribution of labels and historical graph data also changes accordingly, i.e. PYY, Gza)) p(Y G.: Additional analyses on other datasets are provided in Appendix B.3
5008
Number of Nodes
Average Degree
-8.0
Number of Nodes
4000
-1.5
Degret Degre
-7.0
3000
66.5
Aver Average
2000
-6.0
1000
Timestamp
Figure 1: Visualizations of the number of nodes and average degree in each graph snap- shot.
Architectures Impact. Existing methods for dy- namic graphs typically rely on fixed architectures extract invariant patterns. However, the optimal ar- chitecture may be inherently tied to the underlying data distribution (Niu et al., 202T) Wu et al., 20044)
AUC4
When the data distribution continuously evolves, as is the case in dynamic graphs, single fixed archi- tecture may become suboptimal over time. We hy- pothesize that different timestamps under evolving distribution shifts may require distinct architecture designs. To validate this hypothesis, we evaluate two GNN architectures GAT (Veličković et al. (017) and GATv2 (Brody al. 2021). built upon dynamic graph OOD method SILD (Zhag et al.) 2023a). Fig- ure presents the performance of each architecture various timestamps the Collab dataset. As
GATv2
Timestamp
Figure 2: The solid lines indicate the average AUC across timestamps, with the shaded re- gion representing the standard deviation.
3

---

Published as a conference paper at ICLR 2026
Spatio-temporee Distribution Detector
Prototype-guided Disentangled Experts
GIN
Weights i.
GCN
Z'
Node Embedcing
GAT
Routing Similarity Matching
Archifactura Exports with Disered Prototypes
Rouling Vecoo from Snapshot and Memory
Router GNN
Invariant
Patterms
X
CII CIR
R
RC
7
Var
Variant Patlerms
M
M
M
Invariant/Varian Pattorn
Zi Distrbution Intervention
CIR
Mamory Update with Routing Vector
Extraction
Distribution-aware
Intervention Mechanism
RC R
M M
calculate similarity
maxmize the distance
prototypes of experts
Figure 3: Overview framework of AdaMix. Given a snapshot at time t, the spatio-temporal dis- tribution detector employs Router GNN to infer the current distribution shift based the current snapshot and historical information stored in the memory vector, producing a routing vector. In the prototype-gee guidedd disentangled experts module, the routing vector is used to compute the weights of different experts by measuring the similarity between the routing vector and the experts' prototypes. simultaneously disentangled to ensure minimal similarity, and the pert outputs are combined according to their weights to obtain the dixture-of--xxpee embeddings. Finally, in the distribution-awe prior to time f are decoupled into invariant and variant patterns for the current time t, and nodes are selected for intervention training based on the expert weights.
The experts' prototypes
(MoE) node intervention mechanism, all MoE node embeddings
can be seen, GAT and GATv2 alternately achieve superior performance over different time periods, supporting that different timestamps may require distinct architectures. We further provide theo- retical discussion showing that, under the invariance constraint y': .L ((((w)) P((()) specified in
Assumption | when different timestamps require distinct architecture designs, adaptive architectures can capture invariant and variant patterns more effectively than any fixed architecture. We formalize this in the following proposition, with the proof deferred to Appendix C.2
Proposition 1. Under the invariance constraint y'. T (v) (v)),, if there exist two timestamps, t and 12, for which the optimal architectures differ when tasked with discovering invariant patterns at in and 12, then adaptive architecture can capture invariant patterns Pf (t) and variant patterns (v) more effectively than a fixed architecture.
4 METHOD
In this section, we propose Adaptive Mixture of Disentangled Experts (AdaMix) for dynamic graph OOD generalization. The method comprises three key components: prototype-guide disentangled experts, spatio-temporal distribution detector, and a distribution-awaree intervention mechanism. The overall framework of AdaMix is depicted in Figure 3
4.1 PROTOTYPE-GUIDED DISENTANGLED EXPERTS
To route experts adaptively under evolving distribution shifts, it crucial that each expert specializes in distinct factor of variation in the data distribution, thereby ensuring alignment with the underlying distribution. However, in standard MoE frameworks, experts operate independently and lack explicit relational modeling, failing to encourage that each expert corresponds to meaningful disentangled factor of variation. To address this, we propose prototype-guides disentangled experts, which associate each expert with corresponding prototype. These prototypes mutually disentangled, capturing distinct factors of variation, and act as anchors to guide the routing process.
Disentangled Experts. We adopt K. independent GNN architectures as experts, such as GAT /Veličković et al. 2017) and GCN (Kipf & Welling 2016), denoted by (GNN, R k== Each
4

---

Published as a conference paper at ICLR 2026
expert encodes the input graph independently, generating node embeddings:
H GNNN, (X', A').
(1)
where Hy ( k E V) E VRlV denotes the node representation matrix produced by expert l: at time t, and X and A' are the node feature matrix and adjacency matrix at time respectively, and dy is the hidden dimension. We utilize a set of learnable prototypes {P* for K experts, where each prototype P&: E TR represents a distinct factor of variation. During training, we encourage each expert specialize in capturing information associated with its corresponding prototype. To this end, we introduce similarity loss that promotes disentanglement among prototypes:
P& P |pall IPw
Las
R=
Minimizing this loss enforces mutual dissimilarity among prototypes, thereby fostering disentangle- ment across experts and encouraging each develop distinct area of specialization.
Prototype-guided Routing. Let r denote the routing vector for node v at time t, which captures the current distribution (introduced the next section). To route experts in alignment with the underlying distribution, we calculate the similarity between the routing vector r and the prototypes {P& [= associated with each expert. The resulting similarity scores are then transformed into expert weights , via softmax:
exp(â
r Pk
0
6
K -w= exp(at,.))
|
When the routing vector re is more similar to the prototype of particular expert, that expert is assigned a higher weight. The outputs of all experts are then aggregated obtain the node embedding 2t, for each node at time t:
z
Σ
MoE node embeddings = {zt E V, 1 1, ................................ R vixda are subsequently processed
to extract both invariant and variant patterns.
4.2 SPATIO-TEMPORAL DISTRIBUTION DETECTOR.
To infer specific distribution shifts within the context of evolving distribution shifts, we propose spatio-temporal distribution detector that leverages both historical and current information. Specifi- cally, our goal is to capture node-level distribution by jointly considering the current ego-graph and historical distributional information.
Snapshot Graph Trajectories Modeling. We adopt GNN, to learn a node-level routing embed- ding rf for each node v at time t from its ego-graph trajectory G (V (V,,e)), which encodes the structural and feature information of the current snapshot. Formally,
(5) (u, v) Et. High-order structural information can be captured by stacking multiple GNN layers or employing advanced architectures such as GAT (Veličković et al. 2017).
# = GNN, E),, [o)
& = [(u, E gt (uEV);
where ('(())
This produces the node-level routing embedding matrix R' (rr, E V*) E RIV[xds for all nodes in the snapshot at time t.
Memory-augmented Vector: To infer the distribution from historical information, we utilize a memory bank M {m. all nodes. For each node v, we denote the memory vector m E dd that accumulates its historical information up time t. At each step, the routing vector r is generated by GNN,, which integrates the initial node embedding x!, with the previous memory vector m
V} E RV which stores historical distributional information for
r (N AO) X Linear ([x]] m -
(6)
5

---

Published as a conference paper at ICLR 2026
where x Rdr is the combined feature of node time t, dy denotes the dimension of the initial node features, || denotes the node-wise concatenation operation, and Linear(.) is a linear transformation to project the concatenated feature the input dimension of GNN,. We then obtain the routing weights a and MoF node embeddings z for each node using the prototype-guided routing mechanism described in Eq. and Eq. respectively. Finally, we update the memory bank through a gate-controleed mechanism:
m = g 2 + (1 g.) m g. = a (Linear (() m E [0, 1)th,
(7)
where 3 denotes element-win wise multiplication, g.. E [0. 1]dh is a gate vector that controls the update rate of the memory, and o(.) is the sigmoid function. We set the initial memory vector m. zero vector. This mechanism allows the memory to adaptively incorporate new information while retaining relevant historical context, thereby allowing the routing vectors r, to infer current distribution shifts from both present and past information.
4.3 DISTRIBUTION-AWAN INTERVENTION MECHANISM
Previous dynamic graph OOD methods typically rely on randomly sampling variant patterns to replace those of other nodes to discover invariant patterns. However, such interventions may be inefficient when some nodes are intervened upon by others from the same distribution. To address this, we leverage the expert weights from previous steps to distinguish nodes from different distributions better, and we apply interventions using nodes sampled from distinct distributions.
Invariant and Variant Patterns. We first extract the invariant and variant patterns based on MoE node embeddings Z = (z() Txxixxd,, in Eq To account for distribution shifts that may be unobservable in the time domain but become apparent in the spectral domain (Zhang et al. 2023a), apply a Fast Fourier transform (FFT) to project Z into the spectral domain. Let Re(Z) = [Re(z.)} E Tx and Im (Z) = (Im(z[)) E R x denote the real and imaginary parts of the transformed embeddings, respectively. We then derive disentangled invariant and variant spectrum masks m, and my as follows:
m
my o m
my
m = MLP(Re(Z)) Iz())
(8)
where (-)) denotes the sigmoid function, multi-layer perceptron. Finally, the invariant and variant patterns Z, and Zv are obtained follows:
temperature hyperparameter, and MLP(.)
Z, IFFT (Re(Z) my + Im(Z) O mi), Zy IFFT (Re(Z) my + Im (Z) my).
where (-) denotes the inverse Fast Fourier transform, and is the imaginary unit. Z, = {z E R MV]x and Zv (zt, v } E Tx(Vlxd represent the invariant and variant patterns for all nodes across all timestamps, respectively.
Distribution-Aware Intervention. Since experts are assigned nodes according their underlying distributions, a large difference in dominant experts between two nodes strongly suggests that they follow different distributions. We first identify the dominant expert et, for each node based on the routing weights a' in Eq.
arg max 0
(10)
To ensure that nodes are intervened upon by others from distinct distributions, we intervene nodes by sampling other nodes with different dominant experts to replace their variant patterns. Specifically, at each time step f, we randomly sample a set nodes u from the invariant patterns Z, (e.g., e and then replace the variant pattern of node if its dominant expert at time t differs from that of at time t'(t' S t). Consequently, the invariance loss is defined as follows:
Lim Var Cl zn, E Zv),
(11)
s.t. C/ = Σ1 (fi(2h.)) (o(2 1 lettet + 11.1. les=eg)).!)
(12)

---

Published as a conference paper at ICLR 2026
where fr(-) is a classifier based invariant patterns, ((--) is the cross-entropy loss, and Lo is an indicator function that equals if a' # a; and otherwise. In this way, ensure that nodes intervened upon by others from different distributions, thereby enhancing the effectiveness of the intervention mechanism. Then, calculate the final loss follows:
== L1 +
Linv
a Lais
(13)
Eq. 12
Eq.
where L1 the empirical risk based on invariant patterns, and À and a are hyperparameters that balance the three loss terms. The overall training procedure of AdaMix is summarized in Algorithm
5 EXPERIMENTS
In this section, we conduct extensive experiments to demonstrate that our proposed method effectively handles distribution shifts on dynamic graphs through an adaptive MoF framework. Additional details regarding experimental settings and supplementary results are provided in the Appendix B
5.1 EXPERIMENTAL. SETUP
Datasets. We evaluate our method on three real-world dynamic graph datasets that exhibit evolving distribution shifts. For the task of link prediction, we use two datasets: Collab (Tangg al. (0122), an academic collaboration network spanning papers published from 1990 to 2006, and Yelp (Sankar ct al., (2020), which contains customer reviews of businesses over 24-month period. For both datasets, the data is partitioned such that the test set contains different fields from those used in training, thereby simulating a real-world distribution shift. For node classification, we use the Aminer dataset (Tang et al. 2008 Sinha et al.) 2015). citation network covering papers published from 2001 to 2015. In addition, we employ synthetic datasets (Zhang et al., 2023a) with different levels of distribution shifts(0.4, 0.6, 0.8) to further validate the effectiveness of our method. Figures in Appendix B.3 illustrate the evolving distribution shifts across these datasets. Additional details about the datasets are provided in Appendix B.I
Baselines. We compare our proposed AdaMix with three categories of baselines: (1) representative
dynamic GNNs, including GCRN (Seo et al. 2018). EGCN (Pareja al. 2020), and DySAT (Sankar
eet al. 2020): (2) general OOD generalization methods, including IRM (Arjovsky et al.) 2019).
Group DRO (Sagawa et al. 2019), and V-REx (Krueger al. [02T): y (3) static graph MoE methods, including GMoE (Wang et 2023) and GraphMETRO (Wu et al. 2024); and (4) dynamic graph
OOD generalization methods, including DIDA (Zhang al. (022). EAGLE (Yuan et al. 2023) and
SILD ZZang et al. 2023a). To ensure that the performance gains are not merely due to introducing specialized architectures, we replace the original architecture in SILD (Zhang al.) (203a) with
the same architecture experts used in our method for comparison, including GCN (Kipf & Welling 2016), GAT (Veličković al. 2017), GIN (Xu et al. 20188) and GATv2 (Brody et al. 2021). l). More details about the baselines are provided in Appendix B.2
5.2 MAIN RESULTS
Real-world Datasets. Following Zhang et al. (2023a), we evaluate the performance of different methods on real-world datasets with distribution shifts split, details of which are provided in Ap- pendix B. Table T presents the results of different methods on real-world datasets. From Table T we have the following observations: (1) Dynamic graph OOD methods generally achieve better performance than both dynamic GNNs and general OOD methods, highlighting the importance of incorporating temporal information when addressing distribution shifts in dynamic graphs. However, results on the Aminer dataset show that dynamic graph methods cannot guarantee optimal perfor- mance at all time periods. For instance, when SILD employs GATv2 as its backbone, it achieves the best performance on Aminer15 but underperforms GAT Aminer17. This suggests that distribution shifts may differ across time, necessitating different architectures to handle them effectively. (2) Our proposed AdaMix achieves superior competitive performance most datasets, often surpassing existing baselines. These results highlight its effectiveness in handling distribution shifts in dynamic graphs, with the adaptive MoE framework enabling better adaptation to evolving distribution shifts.
7

---

Published as a conference paper at ICLR 2026
Table 1: Performance of different methods on real-world link prediction and node classification datasets. The best results are highlighted in bold, and the second-best are underlined. For the Aminer dataset, the year indicates the test split, e.g., 'Aminer15" refers to the average test accuracy in 2015.
Task
Link Prediction (AUC%)
Node Classification (ACC%)
Avg.
Dataset
Collab
Yelp
Aminer15
Aminer16
Aminer17
GCRN
69.72.045
54.68=7.59
4..96+1.12
51.33
42.93+0.1
57.27
EGCN
76.15
53.82=2.06
44.14
46.28=1.84
37.71+
57.56
DySAT
76.59:020
66.09=142
48.41:081
49.76=0996
42.39:002
63.18
IRM
75.42:2087
56.02:16.08
48.44
50. 30.18=0.73
42.4010.27
59,48
VREx
76.24:10.7
=1.4
48.70
49,
49.24-0:27
42.5944.37
63.16
GroupDRO
76.33
=6.97=0611
48.73
49.74=0:26
42.80
63.46
GMoE
56.45
12.53
49.17
50.89
43. 13.1440.61
58.90
GraphMETRO
57.92.0.11
15.66
50.05
1..12
42.29
50.58
DIDA
81.87=0040
75.92=0.00
50.34:1081
51.43
44.694006
68.87
EAGLE
84.41
77.26=0.44
41.48
54.87
00.31
45.97+0.23
70.81
SILD
84.09
78.65
52.35
0.662
1..94
71.14
SILD-GON
79.53
43.74=0.21
50.54:10.87
53,47
41.64122.66
57.27
SILD-GAT
83,82:1025
50.18
51.68:1181
1..93
44.87:+1.2
61.39
SILD-GIN
15.18:4042
81.55
49.04-11.2
51.15=1163
23.68
66.01
SILD--xN22
83.97:0012
47.84=1.96
52.70++154
54.15=0993
43.354:3.14
60.63
AdaMix
84.85
82.65
12.95
54.58
46.50
72.95
Table 2: Performance of different methods on synthetic link prediction and node classification datasets.
The best results are highlighted in bold, and the second-best are underlined. A larger shift indicates higher level of distribution shift.
Dataset Shift
Link Synthetit (AUC%)
Node-Synthetic (ACC%)
Avg.
0.4
0.6
0.8
0.4
0.6
0.8
GCRN
72.57.0.72
72.29.0..
67.261002
27.191218
25.951080
29.261009
49.09
EGCN
69.00=0.51
62.70=1114
24
22.75
24.98+1.2
43.93
DySAT
70.24+1.26
64.01=0.19
62.19
40.95=2w
37.94#1m
30.90
51.04
IRM
69.40.10.0
63.97.1039
02.6610.35
33.23
30.29
29.43
48.16
VRFx
70.4421.08
63,99=0a1
62.21 20.40
41.78=1.0
38. 11:81
29.5620.4
51.02
GroupDRO
70.30
64.05=001
62.13
41.35=219
35.74
31.03+1.24
50.77
GMoE
55.3911.92
34,97 14.94
56.30
83.33
80.83
72.08
67.15
GraphMETRO
59.53=00.0
59.28=000
58.72=0.12
75.82=433
78.19=3.5
75.25+3382
67.80
DIDA
85.20=n.54
82.89
72.59
43.33
39.48
28.14443.07
58.60
EAGLE
88.32:1061
87,29
12.30.0.75
10
35 1.005
28.5010.15
61.55
SILD
85.9510.18
84.69
78.01
43.62.1274
39.7813356
38.641177
61.78
SILD-GeN
69.43
63.16-0011
60.64
73.21 78.59=100
65.93
68.49
SILD-GAT
85.97
84.69=1.11
43.15
15.15
38.51
61.75
SILD-GN
60.73
58,99
55.22.0.0
17.89
74.65
63.36
65.14
SIL.D-GAT2
86. 19.49
83.82=004
68.43+0.59
41.48=0a5
40.18:2.00
38.08
59.70
AdaMix
90.21
89.64=0.26
88.86
83.63=140
81.50
76.1914922
85.00
Synthetic Datasets. Table 2 reports the results on six synthetic datasets. (1) We observe that AdaMix outperforms most baselines across the datasets, indicating that its adaptive architecture effectively captures invariant patterns under varying levels of distribution shift. (2) As the degree of distribution shift increases, the performance of all baselines degrades significantly. In contrast, AdaMix shows smaller performance drop, further demonstrating its strong ability to handle distribution shifts. (3) The performance of SILD varies significantly when using different GNN architectures as experts, indicating that single architecture is optimal for all distribution shifts. In contrast, AdaMix consistently achieves strong performance by adaptively selecting the most suitable architectures for each node at each time step.
8

---

Published as a conference paper at ICLR 2026
5.3
ABLATION STUDY
100
AUC/Accumayy
80
60
40
Collah
Ydlp
Arzinet
Lak-wa0
Licksgud00
Pull
wro dis
pup rou
w/o mamm
pvp inv
wio
wio fft
welp memarrou
Figure 4: Comparisons of different ablated versions of AdaMix on real-world and synthetic datasets.
To verify the effectiveness of each designed component in AdaMix compare different ablated versions on each dataset: 1) w/o dis: we remove the disentanglement loss Lais in Eq. 2 by setting replace the prototype-guidee routing mechanism with a simple linear router that directly maps the routing embeddings to expert weights; 3) w/o mem: we remove the memory-auu -ugmeente mechanism by setting the memory vector always inv: we replace the invariance loss in Eq. T2 with a random intervention mechanism that randomly samples nodes from the invariant patterns to replace the variant patterns; 5) w/o inv: remove the invariance loss Linv in Eq. 12 by setting X = 0 in Eq. 13 6) w/o fft: we remove the FFT and IFFT operations in Eqs. when extracting invariant and variant patterns. 7) w/o mem&rou; we remove both the deeper linear router.
0 in Eq. 13: 2) rep
vector; 4) rep
mechanism and replace the prototype serootype-giiedd routing mechanism with a
The results are shown in Figure We have the following observations: i) The ablated versions w/o dis and rep rou exhibit significant drop and unstable performance on some datasets, indicating that disentangled prototypes help the router better distinguish different distributions, thereby selecting the appropriate experts. ii) The ablated version w/o mem and w/o mem&rou leads to a noticeable perfor- mance decrease, indicating that leveraging the historical distribution information stored in memory vectors enables better inference of the current distribution. Moreover, disentangled prototypes allow the router to distinguish more effectively between different distributions. iii) The ablated version rep inv and w/o inv yield suboptimal performance, demonstrating the effectiveness of the expert-based based interventions in discovering invariant patterns. iv) The ablated version w/o fft, which relies solely on time-domain information, shows noticeable declines compared to the full model. This demonstrates that spectral-domain invariant pattern modeling effectively captures distribution shifts that may be unobservable in the time domain but become evident in the spectral domain.
5.4 TIME COMPLEXITY ANALYSIS
Let MI. [&), and denote the number of nodes, edges, and time steps, respectively. We denote the dimensions of input features and hidden embeddings by d, and dn, respectively, and let [S] represent the number of intervention times. The time complexity of AdaMix mainly consists of the following components: the time complexity of K experts is OKKTI dg + KT))'dd,d,>> the time complexity of the router GNN is O(T)E|d, + TIV the time complexity of computing MoE weights is (dT)) and the time complexity of computing MoE node embeddings is also | In addition, the time complexity of extracting invariant and variant patterns O(T)V)d, d, log 7), and the time complexity of distribution-aue
interventions V(d)).
Therefore, the overall time complexity of AdaMix is: OKKT(E|,, + KTTV/dzdh + T|E(d,
+ + [d,, log T SyT)v/d, ). which scales linearly with the number of edges and nodes in the dynamic graph, which is comparable existing dynamic graph OOD generalization methods (Zhang al. 2023a Yuan et al. 2023).
RELATED WORK
Dynamic Graph Neural Networks.
Dynamic graphs are pervasive numerous real-world scenar-
ios (Deng et al. 2020 Wang et al. 2021 Cai al. 2021 Zhang al., 2023b 2024c
Wang et al..

---

Published as a conference paper at ICLR 2026
2025b), ranging from social interactions and recommendation systems event prediction (Skarding et al. 2021 Zhu et al. 2022 Chen et al. 2023a). One paradigm employs snapshot- GNNs to learn node representations at each time step and then applies temporal modules such as recur- rent or attention-based models to capture temporal evolution (Yang et al. 2021] [Sun et al. 2021 Hajiramezanali et al]] 2019 (Seo et al. 2018). Another paradigm integrates temporal encoding mechanisms that directly embed temporal information into time-aware representations, which are
2020). Despite these advances, the impact of distribution shifts on dynamic graphs has received limited attention. Some recent works (Zhang al., 2022: Yuan et al., [023] Y Yang al. [024] Tieu et al. 2025) have
then processed with GNNs or memory architectures (Cong et al. 202T, Xu al.
begun to explore this area. For instance, SILD (Zhang et al., (023a) proposes spectral-domain method to disentangle invariant and variant spectral patterns in dynamic graphs, thereby achieving generalization against distribution shifts (especially those that are unobservable in the time domain). However, existing methods typically rely on a single model architecture, which may not be optimal for handling evolving distribution shifts over time.
Graph Mixture of Experts. Mixture of experts (MoE) models have recently been applied to graph
learning to handle the diverse structures and features inherent to graph data (Hu al. 202T Liu
et al. 2023 Rumiantsev & Coates 2024 Han et al.) 2024, Yao et al. 2025 Ye et al. (2025) An MoE architecture comprises multiple expert networks specialized for different patterns, along with a gating network that selects or weights their outputs. For instance, GMoE (Wang et al.. (203) proposes that each node dynamically routes to one of several information aggregation experts, each with differing hop sizes, 2023) takes a different perspective, utilizing a weak MLP and a strong GNN expert, with a confidence gate that per-node decides how much to rely on feature-only the context of OOD generalization, GraphMETRO (Wu et al. (024) uses a Mixture-of- architecture to decompose complex distribution shifts into multiple components. A gating network infers which shifts affect each graph, and each expert is trained to produce representations invariant to its designated shift. However, these methods focus on static graphs, whereas we propose an adaptive MoE framework for dynamic graphs, routing experts based on historical and current information to discover invariant patterns more effectively under evolving distribution shifts.
to better adapt to local graph structure in large-scale settings. Mowst (Zeng et al.,
structure-awant prediction. In
Invariant Representation Learning. Deep invariant representation learning aims to achieve out-of- distribution generalization by capturing stable relationships between data and tasks, thereby enabling
more robust prediction (Arjovsky et al.] 2019; [Ahuja et al., 2020a Li et al.] [202c]] [2033] Xia
et al. 2023 Zhang et al. 2024b; [Sun et al., 20244 Wang et al. 2024; Chen et al.) 2025 Li et al.
2025b) For instance, DIR (Wu et al.] 2022b) discovers causal rationales that remain invariant across
different distributions, while suppressing spurious patterns that are unstable. EERM (Wu et al. (022a) proposes an invariant learning framework that employs adversarially trained graph structure editors to simulate virtual environments, enabling GNNs to extrapolate beyond the single observed environment and thus achieve robust node-level prediction. However, existing invariant representation learning methods adopt fixed model architectures to learn invariant representations, overlooking the fact that dynamic graphs with evolving distribution shifts may require adaptive architectures.
7 CONCLUSION
In this paper, we study distribution shifts in dynamic graphs from an architectural perspective. We propose AdaMix, a novel adaptive mixture-of-experts framework that dynamically selects the most suitable architecture for each node at every time step based on its inferred distribution. Specifically, AdaMix employs a spatio-temporal distribution detector to infer the underlying distribution each node by leveraging both historical and current information. It then incorporates prototype-guiee guided disentangled experts module, which ensures that each expert specializes in a distinct factor of variation, thereby enabling effective routing. Finally, a distribution-aware intervention mechanism is introduced to enhance the discovery of invariant patterns by intervening nodes with others from different distributions. Extensive experiments on both real-world and synthetic datasets demonstrate the effectiveness of our proposed method. One limitation of our work is that we mainly focus on node-level tasks, and we leave the exploration of graph-level tasks for future work.
10

---

Published as a conference paper at ICLR 2026
ACKNOWLEDGEMENTS
This work is supported by the National Key Research and Development Program of China No.2023YFF120000..
ETHICS STATEMENT
All authors of this work have adhered to the ICLR Code of Ethics. In preparing this manuscript, have ensured that no human subjects were directly involved, and all data used are publicly available benchmark datasets. To assist with language clarity and grammatical correctness, large language model (LLM) was employed for proofreading and text refinement; however, all scientific content, ideas, analyses, and conclusions are solely the work of the authors. We have carefully considered potential biases, faimess, and reproducibility of our methods, and we confirm that our research does not involve applications or insights that could cause harm. All experiments comply with applicable legal and ethical standards in machine learning research.
REPRODUCIBILITY STATEMENT
We have made every effort to ensure the reproducibility of the results reported in this paper. All datasets used in experiments are publicly available, with data preprocessing steps detailed in Appendix [.] The proposed Adaptive Mixture of Disentangled Experts (AdaMix) model is described with detailed algorithmic steps in Algorithm hyperparameter settings in Appendix C. and experimental environment configurations in Appendix C.2
REFERENCES
Kartik Ahuja, Karthikeyan Shanmugam, Kush Varshney, and Amit Dhurandhar, Invariant risk minimization games. In International Conference on Machine Learning, pp. 145-155. PMLR, 2020a.
Kartik Ahuja, Jun Wang, Amit Dhurandhar, Karthikeyan Shanmugam, and Kush R Varshney. Empirical invariant risk minimization? sample complexity perspective. arXiv preprint arXiv:2010.16412, 2020b.
Martin Arjovsky, Léon Bottou, Ishaan Gulrajani, and David Lopez-Paz. Invariant risk minimization. arXiv preprint www.:9907.009999 2019.
JunnHyun Bae, Inchul Choi, and Minho Lee. Meta-learned invariant risk minimization. arXiv preprint arXiv:2103.12947, 2021.
Beatrice Bevilacqua, Yangze Zhou, and Bruno Ribeiro. Size-invariant graph representations for graph classification extrapolations. In ICML, pp. 837-851, 2021.
Shaked Brody, Uri Alon, and Eran Yahav. How attentive are graph attention networks? arXiv preprint arXiv:2105.1449) 2021.
Lei Cai, Zhengzhang Chen, Chen Luo, Jiaping Gui, Jingchao Ni, Ding Li, and Haifeng Chen. Structural temporal graph neural networks for anomaly detection in dynamic graphs. In Proceedings of the 30th ACM international conference on Information & Knowledge Management, pp. 3747- 3756, 2021.
Chao Chen, Haoyu Geng, Nianzu Yang, Xiaokang Yang, and Junchi Yan. Easydgl: Encode, train and interpret for continuous-time dynamic graph learning, arXiv preprint 1233:22303.12341 2023a.
Haibo Chen, Xin Wang. Zeyang Zhang, Haoyang Li, Ling Feng, and Wenwu Zhu. Autogfm: Automated graph foundation model with adaptive architecture customization. In Forty-second International Conference Machine Learning, 2025.

---

for Published as a conference paper at ICLR 2026
Yongqiang Chen, Yonggang Zhang, Han Yang, Kaili Ma, Binghui Xie, Tongliang Liu, Bo Han, and James Cheng. Invariance principle meets out-of-distribution generalization on graphs. arXiv preprint arXiv.2202.05441, 2022.
Yongqiang Chen, Yatao Bian, Kaiwen Zhou, Binghui Xie, Bo Han, and James Cheng. Does invariant graph learning via environment augmentation learn invariance? Advances in Neural Information Processing Systems, 36:71486-71519, 2023b.
Zhengyu Chen, Teng Xiao, Kun Kuang, Zheqi Lv, Min Zhang, Jinluan Yang, Chengqiang Lu, Hongxia Yang, and Fei Wu. Learning to reweight for generalizable graph neural network. In Proceedings of the AAAI conference on artificial intelligence, volume 38, pp. 8320-8328, 2024.
Kyunghyun Cho, Bart van Merrienboer, Çaglar Gülçehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. Learning phrase representations using rnn statistical machine translation. In EMNLP. 2014.
Weilin Cong, Yanhong Wu, Yuandong Tian, Mengting Gu, Yinglong Xia, Mehrdad Mahdavi, and Chun-cheng Jason Chen. Dynamic graph representation learning via graph transformer networks. arXiv preprint arXiv:2111.10447, 2021.
Songgaojun Deng, Huzefa Rangwala, and Yue Ning. Dynamic knowledge graph based multi-event forecasting. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 1585-1595, 2020.
Shaohua Fan, Xiao Wang, Chuan Shi, Peng Cui, and Bai Wang. Generalizing graph neural networks on out-of-distribttion graphs. arXiv preprint arXiv:2111.100 10677 2021.
Jean-Christophe Gagnon-Audet, Kartik Ahuja, Mohammad-Javal Darvishi-Bayazi Guillaume Du- mas, and Irina Rish. Woods: Benchmarks for out-of-distribution generalization in time series tasks. arXiv preprint 2XXi:22033.09778 2022.
Chendi Ge, Xin Wang, Ziwei Zhang, Yijian Qin, Hong Chen, Haiyang Wu, Yang Zhang, Yuekui Yang, and Wenwu Zhu. Behavior importance-awee graph neural architecture search for cross-domain recommendation. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pp. 11708-11716. 2025.
Shurui Gui, Xiner Li, Limei Wang, and Shuiwang Ji. Good: A graph out-of-distribution benchmark. Advances in Neural Information Processing Systems, 35:2059-2077 20009,2203322200 2022.
Shurui Gui, Meng Liu, Xiner Li, Youzhi Luo, and Shuiwang Ji. Joint learning of label and environment causal independence for graph out-of-distribution generalization. Advances in Neural Information Processing Systems, 36:3945-397 2023.
Ehsan Hajiramezanali, Arman Hasanzadeh, Krishna Narayanan, Nick Duffield, Mingyuan Zhou, and Xiaoning Qian. Variational graph recurrent neural networks. Advances in neural information processing systems, 32, 2019.
Haoyu Han, Juanhui Li, Wei Huang, Xianfeng Tang, Hanqing Lu, Chen Luo, Hui Liu, and Jiliang Tang. Node-wise filtering in graph neural networks: A mixture of experts approach. arXiv preprint :xXiv:2406003464, 2024.
Xiaotian Han, Zhimeng Jiang, Ninghao Liu, and Xia Hu. G-mixup: Graph data augmentation for graph classification. In ICML, pp. 8230-8224, 2022.
Sepp Hochreiter and Jürgen Schmidhuber. Long short-term memory. Neural computation, 9())
1735-1780, 1997.
Paul W. Holland, Kathryn Blackmond Laskey, and Samuel Leinhardt. Stochastic blockmodels: First steps. Social Networks, 5(2):109-137, 1983. ISSN 0378-8733 doi: https://doi.org 111101600378-8733333)))00000 Dtticlee//11/131777333333338
URL https::/ www. sciencedireet..............iee.. com/ science/
Fenyu Hu, Liping Wang, Shu Wu, Liang Wang, and Tieniu Tan. Graph classification by mixture of diverse experts. arXiv preprint arXiv:2103.1 15622, 2021.
12

---

Published as a conference paper at ICLR 2026
Tianrui Jia, Haoyang Li, Cheng Yang, Tao Tao, and Chuan Shi. Graph invariant learning with subgraph co-mixup for out-of-distriuttion generalization. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pp. 8562-8570, 2024.
Thomas N Kipf and Max Welling. Variational graph auto-encoders. arXiv preprint arXiv:1611.077 2016.
David Krueger, Ethan Caballero, Joern-Henrie Jacobsen, Amy Zhang, Jonathan Binas, Dinghuai Zhang, Remi Le Priol, and Aaron Courville. Out-of-distribution generalization via risk extrapola- tion (rex). In International Conference on Machine Learning, pp. 5815588252 PMLR, 2021.
Haoyang Li, Peng Cui, Chengxi Zang, Tianyang Zhang, Wenwu Zhu, and Yishi Lin. Fates of microscopic social ecosystems: Keep alive or dead? In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 668-676, 2019.
Haoyang Li, Xin Wang, Ziwei Zhang, and Wenwu Zhu, Ood-gnn: Out-of-distribution generalized graph neural network. IEFE Transactions Knowledge and Data Engineering. 2022a.
Haoyang Li, Xin Wang, Ziwei Zhang, and Wenwu Zhu. Out-of-distribution generalization on graphs:
A survey. arXiv preprint arXiv:2202.07987, 2022b.
Haoyang Li, Ziwei Zhang, Xin Wang, and Wenwu Zhu. Learning invariant graph representations for -iiittitdittiittnn generalization. Advances in Neural Information Processing Systems, 35: 11828-11841, 2022c.
Haoyang Li. Ziwei Zhang, Xin Wang, and Wenwu Zhu. Invariant node representation learning under distribution shifts with multiple latent environments. ACM Transactions on Information Systems, 42(1):1-30, 2023.
Haoyang Li, Xin Wang, Zeyang Zhang, Haibo Chen, Ziwei Zhang, and Wenwu Zhu. Disentangled generalization. In Forty-first International
graph sele-suue learning for Conference on Machine Learning, 2024.
Haoyang Li, Xin Wang, Ziwei Zhang, and Wenwu Zhu. Duitof-dist generalization on graphs: A survey. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2025a.
Haoyang Li, Xin Wang, Xueling Zhu, Weigao Wen, and Wenwu Zhu. Disentangling invariant
subgraph via variance contrastive estimation under distribution shifts. In Forty-second second International Conference on Machine Learning. 2025b.
Haoyang Li, Haibo Chen, Xin Wang, and Wenwu Zhu. Out-of-distribution generalization in graph foundation models. arXiv preprint arXiv:2601 2tV.:2601..2066.. 21067, 2026.
Peiwen Li, Xin Wang, Zeyang Zhang, Ziwei Zhang, Fang Shen, Jialong Wang, Yang Li, and Wenwu Zhu, Causal-aware graph neural architecture search under distribution shifts. Proceedings the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining 2. pp. 1458-1469, 2025c.
Yong Lin, Hanze Dong, Hao Wang, and Tong Zhang. Bayesian invariant risk minimization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 16021-16030, 2022.
Zheyuan Liu, Chunhui Zhang, Yijun Tian, Erchi Zhang, Chao Huang, Yanfang Ye, and Chuxu Zhang. Fair graph representation learning via diverse mixture-of-expprts. In Proceedings of the ACM web conference 2023, pp. 28-38, 2023.
Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. Efficient estimation of word representa- tions in vector space. arXiv preprint arXiv: 3301.7781, 2013.
Christopher Morris, Martin Ritzert, Matthias Fey, William L Hamilton, Jan Eric Lenssen, Gaurav Rattan, and Martin Grohe Weisfeiler and leman go neural: Higher-order graph neural networks, In Proceedings of the AAAI conference artificial intelligence, volume 33, pp. 4602-4609, 2019.
13

---

Published as a conference paper at ICLR 2026
Shuaicheng Niu, Jiaxiang Wu, Guanghui Xu, Yifan Zhang, Yong Guo, Peilin Zhao, Peng Wang, and Mingkui Tan. Adaxpert: Adapting neural architecture for growing data. International conference on machine learning, pp. 8184-8199 PMLR, 2021.
Aldo Pareja, Giacomo Domeniconi, Jie Chen, Tengfei Ma, Toyotaro Suzumura, Hiroki Kanezashi, Tim Kaler, Tao Schardl, and Charles Leiserson. Evolvegen: Evolving graph convolutional networks for dynamic graphs. In Proceedings of the AAAI Conference Artificial Intelligence, volume 34, pp. 5363-5370, 2020.
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An imperative style, segh-performance deep learning library. Advances in neural information processing systems, 32, 2019.
Yijian Qin, Xin Wang, Ziwei Zhang, Pengtao Xie, and Wenwu Zhu. Graph neural architecture search under distribution shifts. In International Conference on Machine Learning, pp. 18083-18095. PMLR, 2022.
Pavel Rumiantsev and Mark Coates. Graph knowledge distillation to mixture of experts. arXiv preprint arXiv:2406. 1919, 2024.
Shiori Sagawa, Pang Wei Koh, Tatsunori B Hashimoto, and Percy Liang. Distributionally robust neural networks for group shifts: On the importance of regularization for worst-case generalization. arXiv preprint 1911 1. 108731, 2019.
Aravind Sankar, Yanhong Wu, Liang Gou, Wei Zhang, and Hao Yang. Dysat: Deep neural rep- resentation learning on dynamic graphs via self-attention networks. In Proceedings of the 13th International Conference on Web Search and Data Mining, pp. 519-527, 2020.
Youngjoo Sco, Michaël Defferrard, Pierre Vandergheynst, and Xavier Bresson. Structured sequence modeling with graph convolutional recurrent networks. In International Conference on Neural Information Processing, pp. 362-373. Springer, 2018.
Zheyan Shen, Jiashuo Liu, Yue He, Xingxuan Zhang, Renzhe Xu, Han Yu, and Peng Cui. Towards out-of-distribution generalization: A survey, arXiv preprint arXiv:2108. 10008.13660 13624, 2021.
Arnab Sinha, Zhihong Shen, Yang Song, Hao Ma, Dariin Eide, Bo-june Paul Hsu, and Kuansan Wang. An overview of microsoft academic service (mas) and applications. In Proceedings of the 24th international conference world wide web, pp. 243 236-246. ACM, 2015.
Joakim Skarding, Bogdan Gabrys, and Katarzyna Musial. Foundations and modeling of dynamic networks using dynamic graph neural networks: A survey. IEEE Access, 9.79143-79168 2021.
Li Sun, Zhongbao Zhang, Jiawei Zhang, Feiyang Wang, Hao Peng. Sen Su, and Philip S Yu. Hyperbolic variational graph neural network for modeling dynamic graphs. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pp. 4383-4383 2021.
Xin Sun, Liang Wang, Qiang Liu, Shu Wu, Zilei Wang, and Liang Wang. Dive: subgraph disagreement for graph ofitt--diitiibuttin generalization. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp. 2794-2805, 2024.
Jie Tang, Jing Zhang, Limin Yao, Juanzi Li, Li Zhang, and Zhong Su. Arneminee: extraction and mining of academic social networks. In Proceedings of the 14th ACM SIGKDD international conference on Knowledge discovery and data mining, pp. 990-998, 2008.
Jie Tang, Sen Wu, Jimeng Sun, and Hang Su. Cross-domain collaboration recommendation. In 2012, 2012.
Sibo Tian, Xin Wang, Zeyang Zhang, Haibo Chen, and Wenwu Zhu. Out-of-distribution generalized graph anomaly detection with homophily-aware environment mixup. In The Thirty-ninth Annual Conference on Neural Information Processing Systems.
Katherine Tieu, Dongqi Fu, Jun Wu, and Jingrui He. Invariant link selector for out-of-distribution problem. arXiv preprint arXiv:2505.2413 2025.
14

---

Published as a conference paper at ICLR 2026
Petar Veličkovié, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, and Yoshua Bengio. Graph attention networks. arXiv preprint arXiv:1710.10903, 2017.
Binwu Wang. Jiaming Ma, Pengkun Wang, Xu Wang, Yudong Zhang, Zhengyang Zhou, and Yang Wang. Stone: A spatio-temporal ood learning framework kills both spatial and temporal shifts. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp. 2948 2959, 2024.
Haotao Wang. Ziyu Jiang, Yuning You, Yan Han, Gaowen Liu, Jayanth Srinivasa, Ramana Kompella, Zhangyang Wang, et al. Graph mixture of experts: Learning on large-scale graphs with explicit diversity modeling. Advances in Neural Information Processing Systems, 36:50825-50837, 2023,
Xin Wang, Haoyang Li, Haibo Chen, Zeyang Zhang, and Wenwu Zhu. Modular machine learn- ing: An indispensable path towards new-generation large language models. arXiv preprint arXiv.2504.20020, 2025a.
Xin Wang. Haoyang Li, Zeyang Zhang, Haibo Chen, Tong Xiao, Kehan Li, and Wenwu Zhu. Aincertiitt-awwar disentangled dynamic graph attention network for out-of-distribution general- ization. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2025b.
Xin Wang, Zeyang Zhang, Linxin Xiao, Haibo Chen, Chendi Ge, and Wenwu Zhu. Towards multimodal graph large language model. Science China Information Sciences, 68(11):1-20, 2025c.
Yanbang Wang, Yen-Ye Yu Chang, Yunyu Liu, Jure Leskovec, and Pan Li. Inductive representation learning in temporal networks via causal anonymous walks. arXiv preprint 101.::200000997 2021.
Yuanchao Wang, Zhao-Rong Lai, and Tianqi Zhong. Out-of-distribution generalization for total variation based invariant risk minimization. arXiv preprint arXiv:2502.19665, 2025d.
Jiapeng Wu, Meng Cao, Jackie Chi Kit Cheung, and William L Hamilton. Temp: Temporal message
passing for temporal knowledge graph completion. arXiv preprint arXiv:2010.03526 2020.
Qitian Wu, Hengrui Zhang, Junchi Yan, and David Wipf. Handling distribution shifts on graphs: An invariance perspective. arXiv preprint arXiv:2202.02466 2022a.
Shirley Wu, Kaidi Cao, Bruno Ribeiro, James Zou, and Jure Leskovec. Graphmetro: Mitigating complex graph distribution shifts via mixture of aligned experts. Advances in Neural Information Processing Systems,
2024.
Yihan Wu, Aleksandar Bojchevski, and Heng Huang. Adversarial weight perturbation improves generalization graph neural networks. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pp. 10417-10425, 2023.
Ying-Xin Wu, Xiang Wang, An Zhang, Xiangnan He, and Tat-Seng Chua. Discovering invariant rationales for graph neural networks. arXiv preprint #xXiv:2211.12872 2022b.
Donglin Xia, Xiao Wang, Nian Liu, and Chuan Shi. Learning invariant representations of graph neural networks via cluster generalization. Advances in Neural Information Processing Systems, 00:45602-456133 2023.
Da Xu, Chuanwei Ruan, Evren Korpeoglu, Sushant Kumar, and Kannan Achan. Inductive representa- tion learning on temporal graphs. arXiv preprint arXiv:2002.0796. 2020.
Keyulu Xu, Weihua Hu, Jure Leskovec, and Stefanie Jegelka. How powerful are graph neural networks? arXiv preprint 1xXi:18100000222 2018.
Wujiang Xu, Qitian Wu, Runzhong Wang, Mingming Ha, Qiongxu Ma, Linxun Chen, Bing Han, and Junchi Yan. Rethinking cross-domain sequential recommendation under open-world assumptions. In wWW, 2024.
Kuo Yang. Zhengyang Zhou, Qihe Huang, Limin Li, Yuxuan Liang, and Yang Wang. Improving generalization dynamic graph learning via environment prompt. Advances in Neural Information Processing Systems, 17:70048-700775 2024.
15

---

Published as a conference paper at ICLR 2026
Menglin Yang, Min Zhou, Marcus Kalander, Zengfeng Huang, and Irwin King. Discrete- time temporal network embedding via implicit hierarchical learning in hyperbolic space. In Proceedings of the 27th ACM SIGKDD Conference Knowledge Discovery & Data Mining, pp. 1975-1985, 2021.
Huaxiu Yao, Yu Wang. Sai Li, Linjun Zhang, Weixin Liang, James Zou, and Chelsea Finn. Improv- ing out-of-distribution robustness via selective augmentation. In Proceeding of the Thirty-ninth International Conference on Machine Learning, 2022.
Tianjun Yao, Yongqiang Chen, Zhenhao Chen, Kai Hu, Zhiqiang Shen, and Kun Zhang. Empowering graph invariance learning with deep spurious infomax. arXiv preprint arXiv:2407.1108. 2024a.
Yang Yao, Xin Wang, Yijian Qin, Ziwei Zhang, Wenwu Zhu, and Hong Mei. Data-augmented curriculum graph neural architecture search under distribution shifts. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pp. 16433-16441, 2024b.
Zelin Yao, Mukun Chen, Chuang Liu, Xianke Meng, Yibing Zhan, Jia Wu, Shirui Pan, Huiting Xu, and Wenbin Hu. Da-moe: Addressing depth-sensitivity in graph-level analysis through mixture of experts. Neural Networks, pp. 108064, 2025.
Junda Ye, Zhongbao Zhang, Li Sun, and Siqiang Luo. Mose: Unveiling structural patterns in graphs via mixture of subgraph experts. arXiv preprint arXiv:2509.09333 093377,, 2025.
Jiaxuan You, Yichen Wang, Aditya Pal, Pong Eksombatchai, Chuck Rosenburg, and Jure Leskovec. Hierarchical temporal convolutional networks for dynamic recommender systems. In The world wide web conference, pp. 2236-2246, 2019.
Haonan Yuan, Qingyun Sun, Xingcheng Fu, Ziwei Zhang, Cheng Ji, Hao Peng, and Jianxin Li. Environment-aware dynamic graph learning for out-o--distribution generalization. Advances in Neural Information Processing Systems, 36:49715 49747, 2023.
Hanqing Zeng, Hanjia Lyu, Diyi Hu, Yinglong Xia, and Jiebo Luo. Mixture of weak & strong experts on graphs. arXiv preprint arXiv2231100511 2023.
Zeyang Zhang, Xin Wang, Ziwei Zhang, Haoyang Li, Zhou Qin, and Wenwu Zhu. Dynamic graph neural networks under spatio-temporal distribution shift. In Advances in Neural Information Processing Systems, 2022.
Zeyang Zhang, Xin Wang, Ziwei Zhang, Zhou Qin, Weigao Wen, Haoyang Li, Wenwu Zhu, et al. Spectral invariant learning for dynamic graphs under distribution shifts. In Thirty-seventh Conference on Neural Information Processing Systems, 2023a.
Zeyang Zhang, Ziwei Zhang, Xin Wang. Yijian Qin, Zhou Qin, and Wenwu Zhu. Dynamic hetero- geneous graph attention neural architecture search. In Proceedings of the AAAI conference on artificial intelligence, volume 37, pp. 11307-11315, 2023b.
Zeyang Zhang, Xin Wang, Haibo Chen, Haoyang Li, and Wenwu Zhu. Disentangled dynamic graph attention network for out-of-distribution sequential recommendation. ACM Transactions on Information Systems, 43(1):1-4) 2024a.
Zeyang Zhang, Xin Wang, Yijian Qin, Hong Chen, Ziwei Zhang, Xu Chu, and Wenwu Zhu. Disen- tangled continual graph neural architecture search with invariant modular supernet. In Forty-first International Conference on Machine Learning, 2024b.
Zeyang Zhang, Xin Wang, Ziwei Zhang, Haoyang Li, Yijian Qin, and Wenwu Zhu. Llm4dyg: Can large language models solve spatial-temporal problems on dynamic graphs? In Proceedings of the 30th ACM SIGKDD conference on knowledge discovery and data mining, 2024c.
Qi Zhu, Natalia Ponomareva, Jiawei Han, and Bryan Perozzi. Shift-robust gnns: Overcoming the limitations of localized graph training data. Advances in Neural Information Processing Systems, 34, 2021.
Yuecai Zhu, Fuyuan Lyu, Chengming Hu, Xi Chen, and Xue Liu. Learnale encoder-decoder architecture for dynamic graph: survey, arXiv preprint arXiv:2203. 10480, 2022.
16

---

Published as a conference paper at ICLR 2026
LLM USAGE STATEMENT
In this work, we leveraged large language model (LLM) to assist with checking for grammatical errors and improving the clarity and readability of the manuscript. Specifically, the LLM was used to proofread sentences, suggest stylistic improvements, and ensure that the text adhered to formal academic writing standards. All scientific content, ideas, and results presented in this paper are solely the work of the authors.
A NOTATIONS AND ALGORITHM
Table 3: The summary of the notations and their descriptions used in this paper.
Notations
Descriptions
(P.E)
Dynamic graph with node set V and edge set & Graph snapshot at time t with node set V and edge set
G (VL, 8
G (DM E
Ego-graph trajectory of node v at time
Historical ego-graph trajectory of node from time to
dz dn
Dimensions of input features and hidden embeddings
X lxd, A E RIVE X[V|
Node feature matrix and adjacency matrix at time Label of node at time t
y
H E Mixx
Node embedding matrix produced by expert & time Learnable prototype for expert & Routing vector for node at time Memory vector for node
E Rdh.
r Rd.
m Rd
time f Weight of expert for node at time Invariant and variant masks
my and my
x Rd.
MoE node embedding for node time Invariant variant patterns for nodes Dominant expert for node time
2, Zy Tx(N/dd
timestamps
Cs Cum: Cais N,
Empirical risk hased on invariant patterns, invariance loss, and disentanglement loss Hyperparameters to balance different loss terms
Algorithm 1 Training pipeline for AdaMix
Input: data D (( y number of experts K, hyperparameters and a Initialize: experts (GNN, & I= prototypes [P*] initial memory bank M, distribution detec- tor GNN,
for each epoch do
Reset memory bank M for each time step 1 to T do for each node E V do
Obtain routing vector rt, using Eq.
Calculate expert weights a using Eq.
Obtain MoE node embedding zi, using Eq. 4
Update memory vector m using Eq.
end for
Calculate invariant and variant masks my and my using Eq.
Extract invariant and variant patterns Z, and Zv using Eq. Calculate loss L using Eq. 13
end for
end for
B EXPERIMENT DETAILS AND ADDITIONAL RESULTS
B. DATASETS DETAILS
We summarize the dataset statistics in Table and describe the dataset details follows.
17

---

Published as a conference paper at ICLR 2026
Table 4: Dataset statistics
Dataset
Task
# Nodes
# Edges
# Snapshots
Time Granularity
Features
Evolving Features
Collab
Link
23,035
151,790
16
Year Month
32
No
Yelp
Link
13,095
65,735
Aminer
Node
43.141
851,527
17
Year
128
No
Link-Synthen
Link
151,790
18,974
16
64
Yes
Node-Synthetie
Node
5,000
11,252,385
100
No
Collab (Tang et al. 2012) is an academic collaboration dataset comprising 16 graph snapshots of co-authored papers published between 1990 and 2006. Nodes represent authors, and edges denote co-authorship relationships. Each edge is annotated with one of five domain-specific attributes: "Data Mining". "Database", "Medical Informatics", "Theory", and "Visualization". For OOD generalization experiments, we designate "Data Mining" as the shifted attribute. The dataset is chronologically split into 10/1/5 graph snapshots for training, validation, and testing, respectively. The full dataset comprises 23.035 authors and 151,790 co-authorship links in total.
Yelp (Sankar et al. 2020) is a business review dataset where nodes represent customers or businesses, and edges denote review behaviors. We utilize data from January 2019 to December 2020 (24 graph snapshots), selecting users and reviews with more than 10 interactions. Node features are extracted using word2vee (Mikolov et al. 2013) from reviews, averaged to form 32-dimensional representations for both users and businesses. The distribution shift arises from the COVID-19 pandemic and differing business categories, including "Pizza", "American (New) Food", "Coffee & Tea", "Sushi Bars", and "Fast Food". : We designate "Pizza" the shifted attribute and use 15/1/8 chronological graph slices for training, validation, and testing, respectively. The dataset comprises 13,095 nodes and 65,375 links in total.
Aminer Tang et al.) 2008; Sinha et al. 2015) is a citation network constructed by aggregating data from multiple academic sources, including DBLP, ACM, MAG, and others. The dataset comprises research papers and their citation relationships. For our experiments, we focus on predicting the publication venue of paper. We select the top 20 venues in the dataset as target categories. We use word2vec (Mikolov et al. 2013) to extract 128-dimensional features from paper abstracts and average to obtain paper features. The distribution shift in this task might be attributed the significant rise of deep learning research. Therefore, we use papers published between 2001 and 2011 for training, those published between 2012 and 2014 for validation, and papers published from 2015 onwards for testing.
Link-Synthetic (Zhang et al. 2022 is a synthetic dataset designed to evaluate OOD generalization under controlled spatio-temporal shifts. It is constructed by augmenting the COLLAB dataset. We generate a synthetic feature set X; by training embeddings to reconstruct future links Á A++1 using cross-entropy loss (X)) 11). This ensures X., encodes strong, spurious correlations with future link patterns. The input features are X = [X X where X are the original COLLAB features. The intensity of the distribution shift is controlled by vime-ng sampling probability p(t) = clip(p + a cos(t), 0, 1) where p is set to 0.4, 0.6, or 0.8 for training and 0.1 for testing. We preserve the dataset division method of training. validation, and testing time steps of 10, 1, and 5.
(Zhang et al.) (223a) is designed to simulate distribution shifts in node classification tasks by explicitly modeling frequency components dynamic graphs that exhibit invariant correla- tions with labels, while others do not. To construct this dataset, we employ a stochastic block model (SBM) (Holland et al.) 1983) generate links between nodes, where the link probability between nodes depends on their class labels. Specifically, the SBM is parameterized as SBM Pour). with Pin E [0, x denoting the intra-class link probability and Pou representing the inter-class link prob- ability. We set C = 5 classes for the node labels. Each node is associated with two types of frequency parameters: flow E {0.02, 0.04, 0.08, O. 10, 0.12) and frigh E {0.22, 0.24, 0.28, 0.30, 0.32). The cor- relation between flow and labels is varied across training (0.4), validation (0.6), and testing (0.8) splits, while fiigh maintains fixed correlation of with labels across all splits. At each time step I, the dynamic graph G' is constructed by aggregating multiple subgraphs: (1) random graph G generated from Gaussian noise, (2) an invariant graph G SBM(pp high (t), Poat) derived from high-frequency parameters, and (3) variant graph G SBM (L), Pous) based on low-frequency parameters.
Node-Synthetic
The temporal evolution of these parameters is governed by low (1, f) Si (2 + cos(2 ft)) and
18

---

Published as a conference paper at ICLR 2026
high () = S2 (2 cos(22ft))) where Pouts S1, S2 set to 1e-3, le-2, 5e-3 respectively. Each node is assigned 4-dimensional random features to enhance realism. To ensure generalization under distribution shifts, models must identify and prioritize the invariant graph component (9)) for accu- rate predictions, as the variant component (gv) exhibits unstable label relationships across training and testing phases. This design enables rigorous evaluation of model's ability to disentangle and leverage invariant spectral patterns in dynamic graphs.
Pin
B.2 BASELINES DETAILS
We adopt several representative dynamic GNNs and Out-oo-DiisributionnOOO00 generaliztion meth- ods as our baselines:
Dynamic GNNs: GCRN (Seo et al. 2018) integrates a spatial graph convolutional network (GCN) Kipf & Welling 2016) with temporal gated recurrent unit (GRU) (Cho et al. 2014)
to capture both structural and temporal dependencies in dynamic graphs. EGCN (Pareja ct al. 2020) dynamically evolves GCN parameters over time by incorporating an LSTM (Hochreiter & Schmidhuber, 1997) or GRU (Cho et al.] (014). enabling adaptive modeling of network evolution. DySAT (Sankar et al. (0200) employs structural self- mechanisms to aggregate neighborhood information at each timestamp and uses temporal self-attention to model dynamic network patterns.
general OOD generalization methods: IRM (Arjovsky et al,, 2019) seeks to learn a domain- invariant predictor by minimizing the maximum empirical risk across training domains. Group- DRO (Sagawa et al. 2019) prioritizes domains with higher prediction errors during training, reducing worst-case risks across heterogeneous environments. V-REx (Krueger al., 2021 minimizes the variance of empirical risks across training domains to enhance generalization under distributional shifts. Although these methods focus on static graphs, they are adapted here by leveraging the best-performing DGNNs as backbone architectures for dynamic graph tasks.
static graph MoE methods: GMoE (Wang et al. 2023) utilizes a mixture-of-expper architecture where each expert captures information at different hop sizes, allowing dynamic routing based on local graph structures. GraphMETRO (Wu al. 2024) employs a mixture-of-exxprtt framework to decompose complex distribution shifts into multiple components, with each expert learning representations invariant to its designated shift.
dynamic graph OOD generalization methods: DIDA ZZhagg al. 2022 captures invariant and variant patterns by utilizing disentangled attention in the spatial-temporal domain, and conducts a spatial-temporal intervention mechanism to let the model abandon spurious features and turning to utilizing invariant features to make predictions. EAGLE (Yuan et al. 2023 uses an
EA-DGNN to disentangle multi-channel environments. Then, an ECVAE infers and generates diverse environment samples for fine-grat fine-grained causal interventions. SILD (Zhang et al. 2023a) disentangles the frequency components of node feature trajectories the spectral domain, and then captures invariant patterns by masking out variant frequency components. SII.DGGON, SILD-GAT, SILD-GAIv2, SILD-GIN apply the SILD framework using the GCN (Kipf & Welling 2016),GAT (Veličkovćć et al. 2017). GATv2 (Brody al., 2021) and GIN (Xu et al. 2018) backbone, respectively. We implement these variants to ensure a fair comparison with our AdaMix model.
B.3 CASE STUDY OF EVOLVING DISTRIBUTION SHIFTS
We visualize the evolving distribution shifts in real-world dynamic graphs in terms of the number of nodes and average degree in Figure 5 The distributions of these key graph statistics change significantly over time, confirming the presence of continuous distribution shifts. While some datasets, such as Collab, exhibit a consistent monotonic trend (e.g., continuous growth), this observation suggests that analyzing historical trends can be crucial for inferring the current graph distribution.
B.4 CASE STUDY OF ARCHITECTURE IMPACT
To effectively demonstrate that different time periods in dynamic graph require distinct optimal architectures, conduct a case study using two GNN architectures: GAT (Veličković al. 2017)
19

---

Published as a conference paper at ICLR 2026
50.0
Number of Nodes
Number Noded
Averape
Averaee Degroe
Numbar ho000
3537
Numeer of Nodes
-12
R2433
42343
Number of Nodes
1000
Arcrog Dogros
8
4123
Timestarpp
(a) Collab
(b) Yelp
(c) Aminer
Figure 5: Visualizations the number of nodes and average degree each graph snapshot.
and GATv2 (Brody et al. 2021 Built upon the SILD framework (Zhang al. 2023a), visualize the timestamp- wise performance of both architectures in Figure Our results reveal that the optimal architecture is not static; GAT outperforms GATv2 certain timestamps, while GATv2 demonstrates superior performance at others. This finding indicates that no single fixed architecture sufficient for all time periods, underscoring the critical need for adaptive architectures to handle evolving distribution shifts in dynamic graphs effectively.
C4R/2
nPV
Tiaanars
Timasarss
Timatoon
(a) Collab
(b) Aminer
(c) Link-syn(0.4)
Figure 6: Performance comparison of GAT and GATv2 on the real-world dynamic graphs. The solid lines indicate the average AUC across timestamps, with the shaded region representing the standard deviation.
75
22
12
60
10
10
10
10
10
10
10
10
10
10
(a) Collab
(b) Yelp
(c) Link Laik-SSythetii(O....
Figure 7: Sensitivity analysis of hyperparameter real-world and synthetic datasets. The solid line represents the average AUC (%). with the shaded area showing the standard deviation. The dashed line indicates the average AUC (%) of baseline SILD.
B.5 HYPERPARAMETERS SENSITIVITY ANALYSIS
We conduct the sensitivity analysis two key hyperparameters: the weight of the invari- ance loss, X. and the weight of the disentanglement loss, 3. We vary both À and from {10-6, 10 100 10-3 10 2 20101 1000-1100)) while keeping all other hyperparameters fixed. The results on both real-world and synthetic datasets are presented in Figure 7 and Figure 8 respectively. The hyperparameter in Eq. controls the trade-off between minimizing the empirical risk from
20

---

Published as a conference paper at ICLR 2026
94
81
75
78
10
10
16"
10"
10'
10
10
10"
10" 10"
10"
10"
10
10
(a) Collab
(b) Yelp
(c) Link- Lunn-SSythetic(O..4
Figure 8: Sensitivity analysis of hyperparameter on real-world and synthetic datasets. The solid line represents the average AUC (%), with the shaded area showing the standard deviation. The dashed line indicates the average AUC (%) of baseline SILD.
predictions (I1) and enhancing generalization through learning invariant patterns (Lin)), as defined in Eq. 12 A large value of X could lead to over-emphasis invariance, potentially causing Eq. 13 controls the trade-off
underfitting of the invariant patterns. Similarly, the hyperparameter between L, and the disentanglement loss (Lais), defined in Eq. Lati is crucial for encouraging each expert to learn distinct factors, which is necessary for capturing diverse distribution shifts in dynamic graphs. As shown in Figure our model yields stable performance across wide range of values, demonstrating that the contribution of the disentanglement loss is robust to its hyperparameter selection.
B.6 SHOWCASE OF ADAPTIVE ARCHITECTURES
Koke
6AX2
7.0
Node
1ppor
604
GAN2
Nide
boom
GAN2
760
Figure 9: Illustration of the final architectures for Nodes 0-2 from Aminer across different time steps.
As shown Figure 2 present the final architectures discovered for Nodes 0-2 in the Aminer dataset. Several observations support our earlier hypotheses: architectures searched for different nodes at the same time step can differ, and architectures for the same node can change over time. Notably, Node exhibits significant architectural changes between early and later stages. In contrast, some nodes maintain consistent architectures, such as Node O, while Node shows only minor variations, suggesting that the underlying distribution for some nodes may not experience substantial shifts.
B.7 TRAINING AND INFERENCE TIME
21

---

Published as a conference paper at ICLR 2026
As shown in Table evaluated the training and inference cost of our approach compared with competitive baselines. The training time is measured on the portion of the "W/O DS dataset that requires loss computation and backpropagation (i.e., its training split). The inference time measured on the remaining parts of the dataset, including the validation and test splits of "W/O DS" as well follows. The table reports, for each method and
the entire "W DS" dataset. The results each dataset, the average per-epoch training and inference cost (in seconds). All measurements obtained under the same hardware configuration for fair comparison. We can observe that prrromment-modellng methods such as EAGLE incur substantially higher time cost compared to non-environment- modeling approaches. contrast, our AdaMix introduces only modest overhead relative to SILD.
Table 5: Training and inference time (in seconds) of different methods on various datasets.
Dataset
EAGLE Train
EAGLE Inf
SILD Train
SILD Inf
AdaMix Train
AdaMix Inf
Yelp
6.84
0.87
0.93
0.74
0.78
0.67
Collab
14.16
3.88
0.37
0.20
0.45
0.56
Link-syn (0.4)
4.96
1.23
0.35
0.36
0.47
0.71
Link-syn (0.6)
7.77
1.99
0.27
0.31
0.32
0.62
Link-syn (0.8)
11.02
2.20
0.53
0.34
0.48
0.69
Aminer
9.22
0.18
0.31
0,27
1.03
1.15
Node-syn (0.4)
3,44
0.43
0.29
0.16
1.04
0,55
Node-syn (0.6)
3,47
0.44
0.23
0.12
1.17
0.60
Node-syn (0.8)
3.48
0.44
0.22
0.12
1.14
0.60
B.8 PERFORMANCE WITH MORE EXPERTS
To evaluate sensitivity the number of experts, add GraphConv (Morris et al. 2019) to the original set of four expert architectures and examined the effect of increasing the number of experts. As shown Table 6 the results indicate that using five experts still achieves comparable performance.
Table Performance comparison with different numbers of experts.
Dataset
Collab
Yelp
Link-syn (0.4)
Link-syn (0.6)
Link-syn (0.8)
Four experts
84.85 + 0.39
82.65 0.87
90.21 + 0.13
89.64 + 0.26
88.86 + 0.13
Five experts
85.24 + 0.27
83.59 0.19
88.71 + 0.41
89.34 + 0.77
88.78 = 0.42
C IMPLEMENTATION DETAILS
C.1 HYPERPARAMETERS
For all baseline models, their official implementations and carefully hyperparameters ensure the best possible performance. For the SILD framework (Zhang et al. 2023a), we replace its original backbone with several widely-used GNN architectures: GCN (Kipf & Welling 2016). GAT (Velikkvić et al.) 2017). GATv2 (Brody et al. 2021). and GIN (Xu et al. 2018). For our method, we also use these four GNNs as our experts to ensure a fair comparison, maintaining consistent layer and dimension configurations. To ensure fair comparison, we adopt the same hyperparameter search spaces as the baselines for shared parameters, including the number attention heads, normalization methods, and dropout rates. For our method's specific hyperparameters, the invariance loss weight À and the disentanglement loss weight 3, we perform a grid search over the set {10-9 6, 10-5 10 10 10 10 100} and empirically select the optimal values for each dataset. We use different learning rates for the expert network and other modules, and we fine-tune both the learning and weight decay for each dataset via grid search the validation split.
22

---

Published as a conference paper at ICLR 2026
C.2 CONHIGURATIONS
We conduct all experiments in the following configurations.
Operating System: Ubuntu 24.04.3 LTS
CPU: AMD EPYC 7543 32-Core Processor
GPU: NVIDIA 1100-SXM4-40GB and NVIDIA A100-SXM4-80G1
Software: Python 3.9, CUDA 11.7, Pytorch
(Paszke al. 2019) 2.0.1
D PROOF
Proposition Under the invariance constraint y' (v) Pf()), if there exist two timestamps, t and tg, for which the optinal architectures differ when tasked with discovering invariant patterns at t and t2, then an adaptive architecture can capture invariant patterns Pj(1) (0) and variant patterns (v( more effectively than fixed architecture.
We provide a proof from the perspective mutual information. Consider dynamic graphs {G Y " where G lst (G', G? ..... G') denotes the sequence of graph snapshots up to time t, and Y represents the labels at time t. At each timestamp t, an encoder architecture of extracts two invariant patterns and variant patterns (v( and (v)::
(f( =
Pv(()) fv'
(14)
We denote the K candidate architectures as on for lo = 1,2,...... K, and define the following two sets:
Fixed architectures Φnx = foj t = 1,2, T:: : single architecture ør is shared across all timestamps We denote the containing all Φax as Lax.
Adaptive architectures Dada (d's to vary with G where a routing variable that depends on G. We denote the set containing all Φadn as Lnda.
1.. T): the architecture ó is allowed
To satisfy the invariance constraint in Assumption [ we aim to minimize the conditional mutual information (PP) (o); (v)).. For each timestamp we aim find a of that achieve
I(Pr (()) = I(fv !G!!") fr(0' ( (G!")) =,
(15)
where is a sufficiently small constant. We then define the constraint-sattiifing subsets Fex(e) and Fsda(E) of dix and aada. respectively, as follows:
Fe)(e) { (orx E Pax (PP(()):) Pf()) =, Vt
(16)
Fata(E) (Data 4nx (o) P)()) =, Vt
(17)
Clearly, Pis C Pata (since setting to a constant recovers fixed architecture), which implies Fix(E) Futlle (E). Then we apply the chain rule of mutual information:
I P(()))
((P)( + (P)()) (+)))
(18)
Therefore, ((P)()) P (u)): equivalent to maximizing (PP) Overall, for any &b E F(e). we have:
under the invariance constraint 7 (w)a&) P(())
e, maximizing
T
T
sup F(e) t=1
(())))))) sup F(e)
[(P(()):)) + [PP ((P(()))) P(())]
(19)
sup F(c) 1=1
I(P(()):Y)) +c]
(20)
23

---

Published as a conference paper at ICLR 2026
The adaptive architectures can capture both invariant and variant patterns least effectively as fixed architectures under the same invariance constraint. Since Fix(E) C (a)a it follows that:
r
sup Flll
& sup Fall))
Σ (i)
(21)
Similarly, for any subset T S (I......T) T}, we can derive an analogous result:
sup E (( S sup ((P) Fal(e) LET
(22)
Pa(E)
We next show that adaptive architectures strictly outperform fixed architectures when there exist two timestamps, 4 and l2, for which the optimal architectures differ when tasked with discovering in- variant patterns at 1 and t2, ie.. arg max, [(P) (u): & arg max (v); Specifically, let o; and Φy denote the optimal architectures at t and 22, respectively, with i # j. By the definition of optimality, have:
sup IP)) (v):yt) sup ((P) sup 1 (u):y)) sup / (P)
(23)
Sup/ (u) s
The first inequality follows from the fact that Φy outperforms Φ; 12, while the second follows because Φ; outperforms Φj at t1. Consequently, an adaptive architecture that applies 6; at t and oj tz strictly outperforms any fixed architecture. For the remaining time steps, we can regard them as we therefore obtain:
a subset T S f By the result in Eq. 22
T
T
Σ (U)(())))
(PP(
sup Faale) t=1
sup Φ
1=
T
(24)
((P())))))
sup
om [d)
Combining Eq. 20 and Eq.
24
we obtain:
T
T ( ((P)(())
Σ
sup
(a) t=1
(25)
Σ (L)
sup on (d) 1=
In summary, adaptive encoder architectures are at least as effective as fixed architectures in capturing both invariant and variant patterns under the same invariance constraint. Moreover, they can strictly encoder differs across times-
outperform fixed architectures when the optimal tamps, as this allows the model to adapt to varying distribution shifts, thereby capturing more total information and yiclding invariant patterns and variant patterns more effectively.
E ADDITIONAL RELATED WORKS
Out of Distribution Generalization. Most existing machine learning approaches rely the assumption that training and test datasets are independently and identically distributed, an assumption
often violated in practical scenarios (Arjovsky et al. 2019; Ahuja et al. 22020b; (hen et al.) 2021
Lin et al. 2022 Bae et al. 2021 et al., 2024; Ge et al. 2025; Wang et al. 2025c Tian et al.)
al. (025c).. In such cases. distribution shifts between training and test data severely undermine
model performance. To mitigate this, the study Out-of- Distribution (OOD) generalization has
gained substantial attention across wide range of domains Yao al. 2022 Xu et al.
2024 Wang
24

---

Published as a conference paper at ICLR 2026
et al. 2025d). Representative methods include Invariant Risk Minimization (IRM) (Arjovsky et al. 2019), which seeks predictors invariant across training domains by minimizing empirical risks jointly, thereby achieving consistent performance across diverse environments. GroupDRO (Sagawa et al. 2019) instead emphasizes robustness to worst-case groups by focusing optimization on domains with the highest error rates. Similarly, VREx (Krueger al.) (021) reduces risk variance across domains, alleviating sensitivity to distributional changes. However, these methods fail to consider the unique challenges posed by graphs, such as complex relational structures and dependencies, which are crucial for effective OOD generalization in graph-based tasks.
Graph Out Distribution Generalization. Graph out-of-distributton generalization must account for distribution shifts not only in node features but also in complex structural dependencies and relational patterns (Zhu et al. [022] | Fan et al.
[2021; Li et al.
2022a] Chen et al.] [022 Qin et al.
2022
Gui al. 2023 Chen al. [023bb Wu al. 2023 Jia et al. 2024 Yao et al.) 2024a| Chen
et al.
2026), where the challenges often stem from variations in topology, such graph size structural attributes. For example, Bevilacqua
2024 Yao et al.
2024b Wang et al. 2025a Li et al.
2021) employ structural causal models under independence assumptions to learn representations transferable across different graph size distributions. G-mixup (Han al. (002)) proposes data augmentation strategy that interpolates node features and structures in embedding space to enhance robustness. Advances in graph self-supervised learning and graph explanation techniques further contribute to addressing structural distribution shifts (Li et al. (022b)) However, these methods often overlook the evolving nature of distribution shifts dynamic graphs, which can significantly impact model performance over time.
al. (Bevilacqua et al.)
25


Reviews:
[{"review_id": "7lml76010r", "reviewer": "Reviewer_CVZa", "summary": "This paper addresses dynamic graph representation learning under evolving distribution shifts, a setting where both graph topology and features change over time and the nature of distribution shifts itself evolves. The authors propose AdaMix, an adaptive mixture-of-experts (MoE) framework that dynamically adjusts model architectures to capture invariant patterns across time. Extensive experiments on real-world (Collab, Yelp, Aminer) and synthetic datasets show that AdaMix consistently outperforms both standard dynamic GNNs (e.g., DySAT, EGCN) and prior OOD methods (DIDA, EAGLE, SILD).", "questions": "1. What are the training/inference time compared with baselines?\n2. Could you add a “no-FFT” variant (time-domain only) and a “no-memory but deeper router” variant to disentangle spectral vs. temporal-memory contributions.", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "1. The paper argues that evolving shifts make fixed architectures sub-optimal, which is well-supported.\n2. The paper proposes a three component framework to dynamically adjust the architecture", "weaknesses": "1. Some baselines show very high variance (e.g., Aminer with SILD has huge std), while AdaMix gains on real data are sometimes modest. For examples, on Aminer15, Aminer16 and Aminer17, Adamix does not have evident improvements. The reported mean ± std of baselines often overlap or even exceed AdaMix in mean. \n2. Adaptive MoE at node-time granularity plus FFT-domain masking and memory updates may increase training/inference cost.\n3. Although an ablation study is reported, the performance drop after removing individual components (e.g., memory module, prototype disentanglement, distribution-aware intervention) is not quantitatively large or clearly demonstrated in the main paper. The results do not convincingly show that each component is essential to the final performance.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "HMTyk7lmBX", "reviewer": "Reviewer_fPRk", "summary": "This paper aims to address the challenge of dynamic graph representation learning under evolving distribution shifts, a limitation of existing fixed-architecture methods. It proposes an adaptive-architecture framework for this task, comprising three core components: a spatio-temporal distribution detector, prototype-guided disentangled experts, and a distribution-aware intervention mechanism. Extensive experiments show the proposed model outperforms state-of-the-art baselines (e.g., SILD, EAGLE) in link prediction and node classification.", "questions": "Please refer to the weakness.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1. Using MoE technology to solve the problem of distribution shifts on dynamic graphs is interesting and effective.\n\n2. The distribution-aware mechanism avoids inefficient random interventions by leveraging dominant experts, enhancing invariant pattern extraction.\n\n3. Experiment results on diverse synthetic datasets are good.", "weaknesses": "1. Apart from the MoE part, there is existing work on both disentangling (e.g., DIDA, SILD) and intervention (e.g., SILD, EAGLE) on dynamic graphs. Therefore, the overall architecture has a piecemeal feel. It is impossible to discern any fundamental changes in these sections compared to existing work.\n\n2. The MoE section (section 4.1) has relatively low innovation.\n\n3. The performance improvement on real datasets is very limited (less than 1%).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "mRCZkR0PUm", "reviewer": "Reviewer_1zib", "summary": "This paper investigates dynamic graph representation learning under evolving distribution shifts and introduces an adaptive-architecture framework, AdaMix, designed to handle such changes over time. The proposed approach employs a mixture of architecture experts, guided by a spatio-temporal distribution detector, a prototype-based expert routing mechanism, and a distribution-aware intervention module to capture invariant spatio-temporal patterns. The authors identify and address three main challenges: detecting evolving shifts, dynamically routing experts, and ensuring effective invariant learning. Experimental results on synthetic and real-world datasets are reported to demonstrate the method’s performance compared with existing approaches.", "questions": "Please see the weaknesses.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "1. The problem studied in this paper is important.\n\n2. The experimental datasets are sufficient.\n\n3. The work has a certain degree of theoretical support.", "weaknesses": "1. Some techniques are applied too directly without sufficient explanation. For example, why is invariant pattern modeling performed in the spectral domain?\n\n2. The comparison methods in the experiments are not up to date (latest from 2023), lacking evaluations against GraphMoE-type approaches such as GMoE and GraphMETRO.\n\n3. Considering that the proposed method follows an ensemble learning paradigm and incorporates more GNN encoders than the baselines, the performance gains on real-world datasets are relatively small—mostly within a 1% range. This raises the question of whether such limited improvement justifies the increased model complexity.\n\n4. The method performs better on synthetic datasets but only moderately on real-world ones. This discrepancy suggests that the paper’s assumptions might be overly idealized and not well aligned with the characteristics of real-world data distributions.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "w7PBYEze31", "reviewer": "Reviewer_h4tg", "summary": "This paper addresses the challenge of dynamic graph learning under evolving distribution shifts. The authors propose AdaMix, an adaptive MoE framework that dynamically routes nodes to different GNN experts based on spatio-temporal distribution characteristics. The model includes a memory-augmented distribution detector, prototype-guided disentangled experts, and a distribution-aware intervention mechanism. Experiments on several real-world and synthetic dynamic graph datasets are reported to show improved OOD generalization compared to existing methods.", "questions": "1. How is the proposed adaptive routing different in principle from a standard MoE gating mechanism conditioned on node embeddings?\n\n2. During inference, does the model truly change its computation path, or merely reweight pre-trained experts?\n\n3. What ensures that each expert captures a distinct distributional pattern rather than overlapping ones?\n\n4. How sensitive is AdaMix to the number of experts? Were there cases where performance degraded with more experts?\n\n5. Could the observed performance gains stem from increased model capacity rather than intrinsic adaptability to distribution shifts?\n\n6. How does the proposed intervention mechanism differ from data augmentation or cross-domain mixing used in prior OOD graph works?", "limitations": "", "rating": 4, "confidence": 5, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1. The proposed adaptive mixture formulation is interesting and aligns with the idea of conditional computation in dynamic systems.\n\n2. Experimental results show consistent improvements over several baselines, including recent OOD graph learning models.\n\n3. The inclusion of ablation studies and visualizations helps demonstrate the contribution of individual components.", "weaknesses": "1. Novelty is limited. The framework closely follows the structure of prior dynamic OOD works with mostly terminological changes (more like a combination of DIDA, SILD and EAGLE).\n\n2. The “adaptive architecture” claim is somewhat overstated. The architecture is fixed after training, and only expert weights are adaptively combined during inference.\n\n3. The motivation for modeling distribution shifts via expert routing, rather than latent environments or causal factors, is underdeveloped and lacks theoretical grounding.\n\n4. The mathematical formulation remains descriptive rather than rigorous. The objective functions and routing updates are introduced heuristically without clear derivation.\n\n5. The experiments do not provide sufficient diagnostic evaluation (e.g., temporal shift severity, expert specialization visualization) to validate the “adaptive” behavior claimed.\n\n6. Writing style is occasionally redundant and derivative, with repeated phrases and limited conceptual clarity in the theoretical section.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)