Continuous Domain Generalization
Authors: Zekun Cai
Yiheng YAO
Guangji Bai
Renhe Jiang
Xuan Song
Ryosuke Shibasaki
Liang Zhao
Venue: NeurIPS 2025 poster ()
Keywords: Domain Generalization
Continuous Domain Generalization
Distribution Shift
Neural Operator
Lie Group
Primary area: deep_learning
TL;DR: We propose a principled framework for Continuous Domain Generalization that models the geometric and algebraic structure of parameter evolution, enabling robust generalization across domains with continuous variation.

Abstract: Real-world data distributions often shift continuously across multiple latent factors such as time, geography, and socioeconomic contexts. However, existing domain generalization approaches typically treat domains as discrete or as evolving along a single axis (e.g., time). This oversimplification fails to capture the complex, multidimensional nature of real-world variation. This paper introduces the task of Continuous Domain Generalization (CDG), which aims to generalize predictive models to unseen domains defined by arbitrary combinations of continuous variations. We present a principled framework grounded in geometric and algebraic theories, showing that optimal model parameters across domains lie on a low-dimensional manifold. To model this structure, we propose a Neural Lie Transport Operator (NeuralLio), which enables structure-preserving parameter transitions by enforcing geometric continuity and algebraic consistency. To handle noisy or incomplete domain variation descriptors, we introduce a gating mechanism to suppress irrelevant dimensions and a local chart-based strategy for robust generalization. Extensive experiments on synthetic and real-world datasets, including remote sensing, scientific documents, and traffic forecasting, demonstrate that our method significantly outperforms existing baselines in both generalization accuracy and robustness.

Continuous Domain Generalization
Zekun Cail.4 Yiheng Yao²
Guangji Bai
Renhe Jiang?
Xuan Songl" Ryosuke Shibasaki" Liang Zhao
1 Jilin University, Changehun, China "The University of Tokyo. Tokyo, Japan
Emmory University, Atlanta, GA, USA *LocationMind, Tokyo, Japan {caizekun, wongxuann jll.. edu. en, yihengyao@g. ecc. www.uou.tooocc..jj.. [guangji. bai, 1lang. zhao) Gemory edu, {jiangrh, shiba]@csii wtt.a/astossss....eee..... uctokyo.ac.
Abstract
Real-world data distributions often shift continuously across multiple latent factors such as time, geography, and socioeconomic contexts. However, existing domain generalization approaches typically treat domains discrete a single axis (e.g., time). This oversimplification fails to capture the complex, mul- tidimensional nature of real-world variation. This paper introduces the task of Con- tinuous Domain Generalization (CDG), which aims to generalize predictive models unseen domains defined by arbitrary combinations of continuous variations. We present a principled framework grounded in geometric and algebraic theories, showing that optimal model parameters across domains lie on low-dimensional manifold. To model this structure, we propose a Neural Lie Transport Operator (NeuralLio), which enables Liuettrr-presenve geometric continuity and algebraic consistency. To handle noisy or incomplete do- main variation descriptors, we introduce gating mechanism to suppress irrelevant dimensions and a local chart-based strategy for robust generalization. Extensive experiments on synthetic and real-world datasets, including remote sensing, scien- tific documents, and traffic forecasting, demonstrate that method significantly outperforms existing baselines in both generalization accuracy and robustness. cww Code is available at: https::/giit com/Zekun http:/ZeuunCCaaeeureeee Cai/NeuralLio
evolving along
parameter transitions by enforcing
Introduction
Distribution shift refers to changes in data distributions between training and deployment environ- ments, where the input-output relationship learned during training longer holds at test time. This discrepancy fundamentally compromises the reliability of learned models and motivates the study of Domain Generalization (DG)
which aims train models multiple source domains that generalize to unseen domains without accessing target-doman data. While early DG methods treat domains as independent entities, recent work recognizes that distribution shifts over time follow evolutionary patterns and thus leverages temporal information to capture such regularities. This perspective has led to the Temporal Domain Generalization (TDG) which models domain evolution along a temporal axis and seeks to generalize to future domains.
(52 49
23
14
43
[56] [9 6].
A foundational modeling assumption in TDG is that domain evolution can be projected onto single latent axis. typically instantiated as time. This simplification reduces domain generalization model extrapolation problem, enabling the use of well-established sequence modeling techniques. For instance, state transition matrices simulate domain progression 44 59
recurrent neural networks capture temporal dependencies [2]; and ordinary differential equations model the continuous-tin time domain dynamics 6 58). While the temporal formulation offers tractability, reducing domain evolution to unidimensional sequence imposes a fundamental limitation. In practice, distribution shifts rarely unfold along single direction but are instead continuously driven by multiple interacting factors such as climate conditions, socioeconomic status, infrastructure development, and population structure, each governing one aspect of the generative process. These factors jointly define
Corresponding author
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

continuous variation space, over which distribution shifts co-evolve smoothly. Collapsing such evolution onto single temporal axis inevitably induces information loss and structural distortion.
Formally, this paper introduces problem termed Continuous Domain Generalization (CDG), which aims to generalize predictive models across domains that evolve over an arbitrary continuous space. This formulation captures a more general and realistic set- ting where domain variation arises continuously from multiple underlying factors. For instance, as shown in Fig. [ in remote sensing, image semantics vary with seasonal climate and regional development level, which jointly shape land appearance: climate governs vegetation coverage and coloration, and development determines infrastructure density and land use. A col- lcction of observed domains, spanning combinations such as urban areas in winter rural areas in summer, provides supervision to calibrate predictive models across the variation space. A new region with mod- erate development and autumn conditions naturally resides within this space, and its corresponding pre- dictive model should emerge through evolution along the continuous variation field, yielding predictions that are both semantically coherent and structurally grounded. This mechanism similarly governs many real-world applications. In healthcare, patient dis- tributions may vary continuously with age, blood pressure, glucose, cholesterol, and body mass index. In urban analytics, regional traffic patterns evolve jointly with time and economic activity density. In visual recognition, image domains shift with illumination, viewpoint, and camera parameters such as focal length. All of these factors continuously alter the underlying decision function.
Prediction Model
Function Surface
3
7
From Lindendeveloped to Advanced Region
2 1
22
From Cool Warm Climate
(2 zg) Observed Domains
* Ag: Unseen Domain
Figure 1: Illustration of continuous domain generalization. Real-world distributions are shaped by continuously varying factors. The observed domains provide sparse supervision over the joint variation-mode space. Continu- ous domain generalization studies how predic- tive functions evolve over this space, enabling the model to generalize to unseen domains across the underlying continuous fields.
Continuous domain generalization introduces novel and broadly applicable formulation. However, it poses three fundamental challenges: 1) Identifying domain variation to model evolution in general spaces. Continuous domain generalization requires characterizing how domain variation over a high-dimensional space induces associated transitions in model behavior. However, this relationship remains theoretically unclear, with no principled constraints to ensure the coherent evolution of models across domains. 2) Modeling without inherent structural priors of domains. Temporal domain generalization benefits from the chronological structure of domains, which enables the use of mature sequential modeling techniques. In contrast, general continuous space lacks such inherent topology, making it considerably challenging recover the global structure of domains and to infer continuous variation fields from sparse and irregular observations. 3) Learning under imperfect representations of domain variation. While continuous domain generalization leverages readily available contextual variables as descriptor signals to guide generalization. these proxies are often noisy, incomplete, misaligned with the true underlying variation. Such discrepancies introduce uncertainty into the learned parameter transitions and undermine generalization performance.
To address these challenges, we propose a generic continuous domain generalization framework. which generalizes the model parameters to unseen domains using observed domains across continuous space by transport operator techniques. We theoretically show that the family of domain-optimal models constitutes now-dimensional geometric manifold, providing principled foundation for modeling continuous domain generalization. Building on this insight, we integrate Lie Group theory with neural transport operator to enforce both geometric continuity and algebraic consistency. To ensure robust generalization under real-world imperfections, we incorporate a gating mechanism and local chart-based strategy. Our framework jointly resolves the theoretical, structural, and practical challenges in continuous domain generalization. Extensive experiments on synthetic and real-worle datasets, including remote sensing, scientific documents, and traffic forecasting, demonstrate that our method significantly outperforms existing baselines in both generalization accuracy and robustness.

---

2 Related Works
Domain Generalization (DG) and Domain Adaptation (DA). Domain Generalization (DG) aims to train models on multiple source domains that generalize to unseen ones without access to target data Noomain-invariin
39
38
29
3 [3 [7 Approaches include data augmentation [88 49
33
43
559,
representation learning [17 30; 19; (3], and meta-learning strategies [36
27
14
7 23], Domain
Adaptation (DA), by contrast, requires access to target data during training, using adversarial training
I5 BI], and ensemble methods Despite their effectiveness, these methods assume domains are categorical and independent, overlooking scenarios where domain shifts arise from continuous, structured variation.
17 50, 37 51, 53 feature alignment to bridge source-target gaps 44 22
Modeling Relationships Across Domains. Beyond treating domains as independent categories, a line of research seeks to model relationships or structures among domains. Before deep learning. early works modeled intermediate domain shifts via feature flows, such as geodesic flow kernels 18 and manifold alignment [20]. These methods relied on handcrafted features and linear subspace, limiting their applicability to modern deep, high-dimensional representations. Temporal Domain 59 extends the idea by assuming domains follow a sequential order, enabling recurrent or extrapolative generalization. However, these methods are restricted to the temporal axis and fail to generalize to continuous domains characterized by multi-dimensional. non-sequential variation, limiting their applicability in broader settings. Other approaches leverage partial domain relationships based on domain indices. For instance, AdaGraph [37] constructs graphs over domain indices, while CIDA [5]] defines domain relations through pairwise distances of domain indices. However, graphs are discrete, and distance- tissance-bbasd formulations only induce a metric space, all of which lack the ability to express global mapping over the domain variation space. Overall, while these methods capture partial domain relationships, they are limited in scope and fail to capture the full structure of continuous domain variation.
Generalization (TDG) 28
Geometric and Algebraic Structures in Learning. Geometric deep learning aims uncover the low-dimensional structures and regularities underlying woaldworld data, formulating learning problems through unified geometric and algebraic principles [5]. It views data and models as entities living on structured spaces rather than in Euclidean coordinates, allowing learning to exploit intrinsic relationships. A broad spectrum of mathematical tools, including topology, geometry, group theory. graphs, and differential operators, provide rigorous formalisms to express physical and statistical regularities such as symmetry, invariance, equivariance, continuity, and conservation. These foundations have catalyzed major advances in scientific machine learning and physies-informed modeling 45 21 8 16: 34 24], enabling neural architectures to obey governing equations and preserve physical symmetries. Building on this foundation, we introduce a structural modeling perspective across domain-speciff models for continuous domain generalization.
3 Problem Definition
Continuous Domain Generalization (CDG) CDG addresses the task of generalizing predictive models across domains whose data distributions evolve continuously with respect to underlying latent factors. Formally, let (Drj be a collection of domains, where each domain D; consists of a dataset (X,, Y)) containing input-output pairs sampled from distribution over X' x V, with X' and ) denoting the input and output spaces, respectively. In addition, each domain is associated with a descriptor 2; e Rd, with 2; lying in a continuous descriptor space Z that captures domain-specifer attributes such as spatial, temporal, demographic, economic, or environmental factors. We assume that the underlying data distribution of each domain continuously governed by its descriptor, that is, the conditional distribution P(Y X, 2;) evolves smoothly over 2 as 2, varies. Accordingly, small perturbations in 2,, either along individual coordinates or jointly, induce proportionally small changes in the conditional distribution.
The goal of CDG is, during training. we are provided with set of N domains (Di, Dz, DN }. associated with their descriptors {21, .2..... 2N.). For each domain D., we learn predictive model g(:; 0:) X' + V. where 0; E I denotes the model parameters learned from D.. We want to learn the co-variations between domain descriptors (21 ,., _N) ) and model parameters {0,, 0z. ON), and to infer the parameters 0, corresponding to an unseen domain D, given a new descriptor 2, & (21....,ZN) ). This enables the learned framework to generalize to arbitrary domains across the descriptor space Z, yielding a predictive model g(: 02)) for any given descriptor 2g.
3

---

CDG assumes that data distributions evolve continuously, consistent with the continuum hypothesis in physics and engineering, which posits that macroscopic systems change smoothly with respect to their underlying causal factors. Such smooth evolution is widely observed in physical, biological, and socioeconomic systems. Consequently, modeling domain evolution as continuous process offers a principled and tractable abstraction of real-world dynamics. Nevertheless, generalizing predictive models over such continuous spaces poses significant challenges. First, domain variation and model behavior are often entangled, making it difficult to establish stable correspondences. Second, the absence of inherent chronological organization leaves no on-hand structural guidance for modeling cross-domain relationships. Third, domain-level information is typically noisy, incomplete, or misaligned, introducing uncertainty that undermines reliable generalization. In the following sections, tackle these challenges sequentially.
4 Methodology
In this section, we present our framework for continuous domain generalization through structural model transport. Specifically, we first identify the geometric structure underlying model parameters across domains, and formally prove that the collection of domain-wise parameters forms an embedded submanifold. (Section 4.1). This establishes the geometric foundation for representing model evolution as a continuous mapping on the manifold. We then derive the necessary structural constraints that should be satisfied for model evolution, and propose a Neural Lie Transport Operator (NeuralLio) grounded in Lie Group theory to enable equivariant parameter transitions across domains (Section 4.2). Finally, we handle imperfections in the descriptor space, including degeneracy and incompleteness, using gating and local chart-based based modeling (Section 4.3).
4.1 Identifying the Parameter Manifold of Continuous Domains
This subsection provides theoretical perspective the geometric structure of optimal model param- eters across continuous domains, revealing a smooth and coherent mapping from domain descriptors model parameters. Preserving this structure is fundamental for continuous generalization.
Theorem 1 (Parameter Manifold). In continuous domain generalization, the function mapping domain descriptors to their corresponding predictive model parameters, 0(2) 2 -> O continuous. Moreover, let Z S. IRd denote theoretical descriptor space that provides a complete and non- degenerate representation of all latent factors governing domain variation, then the image set M = {0(=) E Z} forms a d-dimensional embedded submanifold R...
Proof. Under continuous domain distribution shift, the variation of the distribution can be described by a continuous vector field f. Since the optimal predictor g(: (())) serves as functional represen- tation of the conditional distribution P( I 2). the evolution of the predictive model function space can be modeled by:
V V (=((((())) = f(g(; 2),
where Vg denotes the gradient operator with respect to z.
Applying the chain rule to the above PDE yields:
v= (; 0()) = (0(2)) (26(2)
((0(2) J1((()) (((:; 2).
(2)
where J denotes the Moore-Penrose pseudoinverse of the Jacobian of with respect to 0. This shows that 0 - θ is differentiable, and hence continuous.
the descriptor space Z provides complete and non-degenerate representation of all latent factors governing domain variation O then the Jacobian J=0(z) RR is of full rank d, and the mapping z + P(Y [[||X; z) is injective. Since the model architecture g is fixed, and each domain-specific parameter (22) is uniquely determined by the corresponding conditional distribution P(Y|X; 2), the mapping + ((2) is also injective. Then, by the Constant Rank Theorem, the image set
M = ((2) 2 E Z}
(3)
forms d-dimensional embedded submanifold of R.P
'The image set of function refers the set of all values it outputs.
SSuc a descriptor space theoretically exists; in practice, observed descriptor serves as approximations without affecting the validity of the theorem.

---

Z x Theorem carries several important implications as follows:
Property (Unified Representation). Modeling in parameter space yields unified, comparable representation, abstracted away from lask-specific feature engineering.
Property 2 (Geometric Regularization), The manifold structure regularizes the parameter space by construining optimization to a geometrically valid subset, reducing the searching complexity in RRD.
Property 3 (Analytical Operability). The manifold structure enables principled geometric and algebraic operattins over model facilttatng interpretable and controllable generalization.
4.2 Neural Domain Transport Operator under Structural Constraints
Eq. reveals that a differential equation governs the evolution of model parameters. In the temporal case where the domain descriptor is one-dimensional, prior work 6 models such evolution with NeuralODEs [9]. A natural extension is to lift the ODE to a PDE framework for multi-dimensional cases. However, this introduces substantial challenges: (1) Classical numerical PDE solvers, such as finite difference or finite element methods, require known values of ((2) on structured grid to specify boundary or initial conditions. However, in continuous domain generalization, the (()) for each training domain is not directly known and must be learned through iterative optimization, leaving such conditions undefined. (2) The training domains are sparsely and irregularly distributed, precluding natural ordering or integration paths, making it infeasible to reduce the PDE to a set of ODEs along any direction. (3) Modern neural PDE solvers, such as Physics-Informe
Neural Networks [45], rely symbolic PDE formulations to define residual losses, yet the governing equations describing distributional cvolution remain unknown in continuous domain generalization.
Structural Constrained Domain Transport Rather than solving the PDE or explicitly regressing the parameter field ((2), we propose to learn a structure- strueture-preserving transport operator T
2 + θ, which maps parameters at one descriptor 2; to another target descriptor 2j. This structural operator circumvents limitations of classical and neural PDE solvers by directly learning pairwise parameter transitions. Specifically, given ((2,) at and the target point 2j, the operator produces
0(z,) = T(((z), 24, 2z).
To ensure that T yields meaningful and generalizable parameter transitions, explore and for- mulate the necessary conditions that the transport operator needs to satisfy for continuous domain generalization: geometric continuity structure and algebraic group structure.
Definition 1 (Geometric Structure). The neural transport operator T is said to satisfy geometric structure if it is continuous in all of its inputs.
Definition 2 (Algebraic Structure). The neural transport operator T said to satisfy algebraic structure if the following properties hold:
Closure: T(0(=:), 2i, 2z) E O, ensuring transported parameters remain within the valid space.
Identity: T(0(2,), 24, 24) ((2)), ensuring that self-transport leaves parameters unchanged.
Associativity: ((T(((())) =4, 25), øj. =x) =T(((=;) Zi; 2k), ensuring that sequential transports are equivalent to direct ones.
Invertibility: T T (T(2)) 24, 24) 0 (24). ensuring that each transport can be exactly reversed.
The geometric structure ensures that ((z) forms a smooth manifold, as established in Theorem | The algebraic structure defined via Group-like like axioms imposes Equivariance. Equivariance characterizes a symmetry relation between transformations in the input and output spaces. In this context, it ensures that applying transformation in the descriptor space and then mapping parameters yields the same result as transforming parameters directly. Without equivariance, the learned operator may produce parameter trajectories that deviate from the true evolution of the underlying data distribution.
The geometric and algebraic structure jointly imply that the parameter family M (8(2) 2 3} forms Lie Group-i.e a smooth manifold with a compatible group operation. The transport operator induces parameter Lie Group transitions along directions specified by domain descriptors. This motivates us to propose the Neural Lie Transport Operator io), a learnable operator grounded in Lie theory and parameterized by a neural network. NeuralLio characterizes the local variation of (2)) using the Lie algebra g generated from the domain
Neural Lie Transport Operator
5

---

descriptor. The Lie algebra g serves as tangent space that captures the differential structure of the parameter manifold. Given source descriptor 2; and corresponding parameter ((2;), we generate a Lie algebra element (==) E g using neural network. For a target descriptor 2j. we apply the offset (2) -2) 2,) to V(z,) to transport (zz)) to ((z)):
(22)) = T((2,), 4, 23) =
(z) ((2)).
(5)
where the exponential map exp: : g + MM lifts the Lie algebra vector to valid group transformation.
In practice, the parameter vector ((2) E R.D is high-dimensional. We first encode it into a compact latent representation e(z) = fe()(z)) E IR. using an encoder fe- We then perform cascaded transport the latent space. Specifically, we define a set of neural Lie algebra fields {V*,... .,. Vd), where each V E mm produced by a field network 1 Rd - Rm conditioned on the source descriptor 2₁- Each f can be instantiated with generic deep neural network. Given descriptor shift (zj zi), we compute the cumulative transformation as:
2 (E(=))) e((z). e(zy) ii exp ((0) -xa)
(6)
The latent representation e(z,) is then decoded by fa : Rm + ED to obtain the final ((2)).
4.3 Handling Imperfect Descriptors Space
Theorem I discusses a theoretically grounded descriptor space. In the real world, however, the descriptors accessible for model training are observable projections of this theoretical space. They may contain noise, redundancy, or missing factors, leading to imperfect representations. These imperfections introduce two challenges: (1) Degeneracy: Some directions in the observed 2 fail to affect (2) or redundantly encode variations. (2) Incompleteness: The observed descriptors partially capture the degrees of freedom governing domain shifts, leaving some latent factors unmodeled.
Suppressing Degeneracy via Descriptor Gating We introduce gating mechanism to suppress irrelevant or redundant directions in Z. Specifically, we apply a dimension- gate to modulate the influence of each feature in z, enabling the model to focus on informative directions for parameter vari- ation. The gate consists of two components: a data-dependent gate defined as g(2;) = Sigmoid (Wa 24) with W E IRdxd, and a global trainable gate vector E Rd shared across all domains. This yields a shared gating vector m g(z;) @ w used to modulate both source and target descriptors:
2
2 m (2)), 2j = @ m (2:).
where @ denotes element- wise multiplication. This gating suppresses degenerate directions before computing the transport operator.
Mitigating Incompleteness via Local Chart When the space 2 is incomplete, learning globally consistent transport function becomes underdetermined. We adopt localized modeling strategy grounded in the theory of differential geometry: the parameter manifold is represented atlas, ie., a collection of overlapping local charts. For each descriptor 2,, we define its neighborhood as:
N(z)) == {zj E Z žj is k-NN of =c);
and restrict the transport operator to locally adjacent domains:
((2,) = ((((2,), 2i 23), Vz, E A2x))
(9)
The localized construction enables the mapping + ((2) the union of charts compensates for global non-smoothness from incompleteness.
(z) to remain smooth within each chart, while
Optimization NeuralLio is optimized by supervising the transport operator T over local neigh- borhoods in the descriptor space. For each training descriptor 2;, we sample nearby descriptors 2y E (2;))) and train T to transport model parameters from 2; 2j so that the resulting predictions and latent representations remain consistent. During inference, parameters for an unseen domain 2, are inferred by transporting from the nearest training descriptor using the learned operator. Detailed procedures for both training and inference are summarized in Appendix We also provide a detailed model complexity analysis in Appendix A.4

---

Table 1: Performance comparison on continuous domain datasets. Classification tasks report error
rates (%) and regression tasks report MAE. 'N/A' implies that the method does not support the task.
Classification
Regression
Model
2-Moons
MNIST
[MoW
ArXiv
Yearbook
Traffic
Descriptor-Annosie
ERM
34.7 0.2
31,8 + 0,9
27.7 = 1.6
0.1
8.6 1.0
16.4 = 0.1
IRM
34.4 0.2
33.0
41.5 2.8
37.4 1.0
8.3
16.6 0.1
V-REx 26
34.9+
3.4
3.6
0.7
8.9
20.9
GRO
46
3..5+0.1
1.0
-
+.1
8.0
16.2 - 0.1
Mixup 54
3.1
3.90
2..1
3.2
7.5
16.1 = 0.1
DANN
3..1
0.6
2.70
36.5
8.9
18.1 0.2
MI.GG 27
34.6+
8..1
29.2
+.2
7.7
16.9 - 0.1
CDANN 32
35.0
0.8
27.6
3622
8.7 0.4
17.3 0.2
URM 25
34.7 0.1
31.8 = 1.3
26.9 1 1.0
35.5 1.5
8.0 0.3
16.2 0.2
Descriptor-Aware
ERM-D
13.1 1.5
31.7 + 0.5
28.9 + 1.8
38.1 1 0.6
7.4 0.5
15.9 0.1
NDA
25.4 + 0.3
26.3
0.7
31.2 + 1.4
35.6 + 0.6
11.0
17.2 0.2
CIDA 51
14.2 + 1.1
27.4 + 0.5
27.1 + 0.9
35.3 + 0.4
8.4
16.6 = 0.1
TKNets
59
N/A
N/A
N/A
N/A
8.4
N/A
DRAIN
N/A
N/A
N/A
N/A
1.0
N/A
Koodos
N/A
N/A
N/A
N/A
6.6
N/A
NeuralLio (Ours)
3.2 + 1.2
9.5 1.1
24.5 + 0.5
34.7 + 0.4
4.8 0.3
15.1 0.1
5
Experiment
In this section, we evaluate the effectiveness of the proposed framework across diverse continuous domain generalization tasks. Our experimental study is designed answer the following key questions: 1) Can the frumework effectively generalize predictive models across continuously evolving domains? 2) Can the frumework faithfully recover the underlying parameter manifold shaped by domain variation? 3) Can the framework robustly handle descriptor imperfections? 4) Can the learned transport operator exhibit the desired structural properties? More detailed results (i.e., dataset details, baseline details, model and hyperparameter configurations, ablation study, scalability analysis, sensitivity analysis, and convergence analysis) are demonstrated in Appendix A
Synthetic Datasets. Two synthetic datasets are employed simulate continuous domain shifts under interpretable variations. In the 2-Moons dataset, each domain is generated by applying scaling and rotation to the base moon shape. The descriptor = [=] E IR² encodes the scale factor and rotation angle, respectively. We train the model on 50 randomly sampled domains, and evaluate on 150 additional randomly sampled domains, together with extra test domains uniformly sampled over mesh grid in the descriptor space (see Fig. 5). In the MNIST dataset, each domain constructed by applying rotation, color shift (red to blue). and occlusion to digits. The descriptor [2', 22, 2 I encodes the intensity of each transformation. We use 50 randomly selected domains for training and another 50 for testing. Visual illustrations are shown Fig. 6
Real-world Datasets. We further evaluate our framework on diverse collection of multimodal real-world datasets: IMoW, ArXiv, and Yearbook for classification, and Traffic for regression. For each dataset, we construct continuous descriptors using auxiliary information derived from publicly available domain metadata contextual variables. Comprehensive details of dataset composition, processing, and descriptor construction are provided in Appendix I.
Baselines. We employ two categories of baselines. The Descriptor-Agnosti group operates purely on input-output pairs without domain-level descriptor: ERM, IRM, V-REx, GroupDRO, Mixup, DANN, MLDG, CDANN, and URM. The Descriptor Aware group incorporates explicit domain descriptors to guide generalization: ERM-D, NDA, (1], TKNets 55),, DRAIN 22), and Koodos Details of all comparison methods are provided in Appendix A.2
Metrics. Error rate (%) is used for classification tasks. Mean Absolute Error (MAE) is used for regression tasks. All models are trained training domains and evaluated on all unseen test domains. Each experiment is repeated five times, and we report the mean and standard deviation. Full hyperparameter settings and implementation details provided in Appendix A.3
7

---

Continuous Domain Variation Field
Sample [3.7. 24.17)
Sample [18.7. 40.09)
Sample 127.9 13,04]
Sample (41 37.151
50
40
X
X
ERM-D
(Rotation)
30
X
20
NDA
20
30
40
50
CIDA
(Scaling)
Figure 2: Visualization of generalization behavior of baseline models on the 2-Moons dataset. Left: All training domains (black dots) and selected test domains (blue crosses) in the variation space. Right: Decision boundaries of baseline methods (rows) evaluated at the four test domains (columns).
10.0, 15.01
110.0. 15.07
120.0, 15.01
190.0. 00
[40.0. 15.01
150.0, 15.01
g!
140.0. 0.9]
[40.0. 10.0]
44..0, 20.01
[40.0, 30.0)
[40.0, 40.0)
[40.0. 50.0]
Ap el (etatioon
Dirction
30
10
210
(Scaling)
30
10
7.5]
15.0]
122.5 22.51
10.0,
[37.5
[45.0. 45.0]
Figure 3: Visualization of the learned parameter manifold and the corresponding generalization behavior. Left: PCA projection of predicted parameters f(2) over the entire descriptor space. Right: Visualization of decision boundaries and data samples along selected direction.
5.1 Quantitative Analysis
We present the performance of our proposed method against baseline methods, highlighting results from Table Our method demonstrates strong generalizability across wide range of continuous domains, with particularly large improvements on 2-Moons (3.2% vs. 13.1 1%) and MNIST (9.5% vs. 26.3%), indicating its effectiveness in modeling continuous domain variation. Beyond synthetic data, NeuralLio also achieves the best results on fMoW, ArXiv, Yearbook, and Traffic, indicating its robustness in handling diverse and irregular real-world domain variation. Table | also reveals several insights into how existing methods behave: (1) Descriptor-awar methods generally outper- form secriptor-agnostte ones, yet naively incorporating descriptors (e.g.. ERM-D) risks spurious correlations that hinder generalization. (2) Temporal methods are insufficient for general continuous domains. Models designed for temporal domain generalization (i.c.. TKNets, DRAIN, Koodos) rely on domain sequence and fail when the descriptor space lacks such an ordering structure. (3) Con- tinuous domain generalization is intrinsically challenging. Even among descriptor-awarc methods, the performance of baselines on synthetic datasets is unsatisfactory. No existing method achieves reasonable accuracy. This highlights that prior methods largely overlook the underlying structure of domain variation, which is essential for modeling continuous generalization.
5.2 Qualitative Analysis
To further understand how domain variation influences model behavior, we visualize both baseline and learned models the 2-Moons dataset. Fig. 2 provides global view of the descriptor space,

---

where the two axes correspond to scaling (monotonic variation) and rotation (periodic variation). The background color encodes the continuous variation field across domains, All training domains are shown as black dots, with several illustrated in small inset panels. Four unseen test domains are randomly sampled for visualization and marked as blue crosses in the descriptor space. The right panel visualizes the classification boundaries of three baseline methods evaluated on the four test domains. Each row corresponds to a method, and each column to a test domain. ERM-D embeds descriptors directly into the model input, leading entangled representations. This often induces spurious correlations and overfitting, resulting in highly irregular and fragmented decision boundaries. NDA relies solely on chance overlap between training and test domains, lacking any generalization mechanism. As a result, it produces unstable decision boundaries with no guarantee. CIDA performs adversarial alignment based on descriptor distances and partially captures the scaling effect, visible as a ring-like expansion. However, fails to model rotational variation because its metric-driven objective lacks structural fidelity. This limitation highlights the inherent weakness of distance-based objectives in preserving domain geometry, particularly under non-isometric transformations.
Fig. shows an intrinsic view of the parameter manifold learned by NeuralLio on the 2-Moons dataset. Specifically, we train the model using the given training domains, and then densely generalize (2)) across the full descriptor space. The left panel visualizes the parameter manifold obtained by projecting all the predicted model parameters ((2) via PCA. The resulting "twisted surface" aligns with the monotonic- porrooiic-periodic geometry of scaling and rotation, respectively. The right panel demonstrates that traversing along different descriptor directions corresponds smooth, consistent transformations in decision boundaries, showing that the learned operator preserves geometric continuity and captures the co-evolution of model parameters and domain distributions. In particular, the mixed-direction case reveals the model's ability to jointly encode multiple domain variations. In summary, the visualization results firmly demonstrate that our method captures the structural geometry of domain variation and enables interpretable generalization across the descriptor space.
Noisy
Redundant
Incomplete
C Rd+N
gnn Drop
I
E cR RM--
Midd M
where I is the index set of N dropped dimensions
2-Moons
2-Moons
2-Moons
Error Rates (96)
Galling Garting
Error Rates (%)
Gallig
Error Rates (%)
25
Gotting
20
20-
Chart
No-Chart
30-
10-
Nolse Dimension
Redundancy Dimension
Missing Dimensions
Figure 4: Robustness to imperfect descriptors. Top:
Visualization of noisy, redundant, and incomplete descriptor constructions. Bottom: Error rates under increasing imperfection on the 2-Moons dataset.
5.3 Imperfections Analysis
We design three controlled experiments to simulate the common imperfections of the domain de- scriptor: noise, redundancy, and incompleteness, as discussed in Section 4.3 To assess the noise, we augment z with randomly sampled noise dimensions, forming descriptors with uninformative components. To assess redundancy, we apply redundant projections to the original descriptor to obtain a higher-dimensional,, ever-parameterized version. This setup mimics real-world cases where descriptors are overly entangled. To study incompleteness, we simulate missing information by randomly dropping n dimensions from the redundant descriptor. As n increases, the retained dimen- sions provide less faithful representation of the underlying domain variation. An overview of the construction is illustrated in the top row of Fig.
The bottom row of Fig. shows the error rates under increasing levels of descriptor imperfection across three settings. In the Noisy setting (left), gradually append up to five random noise

---

dimensions to the original descriptor. As noise increases, the performance of the model without gating rapidly deteriorates, reaching over 30% error at the highest noise level. In contrast, the model equipped with the gating mechanism remains significantly more stable. This indicates that although the noise dimensions are uninformative, the remaining structure the descriptor space still guides transport reasonably well, provided that irrelevant dimensions are actively suppressed. In the Redundant setting (middle), we expand the descriptor by applying random linear projections to the original descriptor, introducing redundancy without discarding any information. Both models maintain reasonably low error rates (under 10%), since the underlying domain variation remains fully preserved. However, the model equipped with gating achieves better and more stable performance. This suggests that while redundancy itself is not harmful, the gating mechanism helps by softly recovering the original independent factors of variation, effectively regularizing the projection space and improving robustness. In the Incomplete setting (right), we construct an 8-dimensional descriptor space embedding two latent factors of variation, and simulate increasing incompleteness by progressively removing to dimensions. The model without charting (No-Chart) fails once key structural information is removed. In contrast, the chart-based variant exhibits controlled performance decline. This validates the benefit of local manifold modeling, as restricting transport computation descriptor neighborhoods allows the chart-based approach preserve continuity even when global descriptor integrity is compromised.
5.4 Structure Property Analysis
To validate the structure properties introduced in Section 4.2 we evaluate the learned operator on the 2-Moons dataset. We measure the consistency between theoretically equivalent transformations by computing the cosine similarity of their resulting parameters across all test domains.
Identity. For each test descriptor 2;, we apply
the self-transport 0(2;) -- (z;) and com- pute the cosine similarity: c0s()) 0()))))
Table 2: Empirical verification of the structure properties the learned operator.
Associativity. We consider all triplet com- binations (zi, 22) from the test descrip- tors. For each triplct, we compute two trans- ports: 8(2)) 3- 8'(2)) Sy-ten, ('(2,, and ((z) 2.tt ("(x),, then evaluate cos(0'(zz), (zxx))
Property
Cosine Similarity (%)
Identity Associativity
99.9
99.1
Invertibility
98.3
Invertibility. We consider all pairs (25, 2z) from the test descriptors. For each pair, we apply the round-trip transport (2:)) Rs A (22)) By B 0' (2:). and compute cos()(2;), (z:)..
Closure. This property is inherently satisfied, as the operator always maps within the R.P.
As shown in Table the learned operator fulfills the expected properties with high precision, which empirically confirms that NeuralLio realizes the Lie- Lie-Group- based transport formulation, providing principled foundation for structurally consistent model evolution.
Conclusion
This paper presents a unified framework for continuous domain generalization, which aligns structural model parameterization with domain- wise continuous variation. We identify the geometric and algebraic foundations for continuous model evolution and instantiate them through a neural Lie transport operator, which enforces fructure-pressevvie parameter transitions over the variation space. Beyond idealized settings, we further address realistic imperfections in the descriptor space by introducing a gating mechanism to suppress noise, and a local chart construction to handle partial information. We provide both theoretical analysis and extensive empirical validation on synthetic and real-world datasets, demonstrating strong generalization performance, robustness, and scalability.
Building upon these insights, several directions emerge beyond the current setting. First, many seemingly discrete domain generalization tasks may in fact represent sparse samples of an underlying continuous process governed by latent semantics. Discovering these latent descriptors through unsupervised or generative modeling is appealing. Second, sealing the framework toward more expressive model structures offers a promising avenue for extending its applicability to broader domains. Third, exploring arithmetic operations between models in the functional space may further deepen the understanding of continuous generalization.
10

---

References
[1] Martin Arjovsky, Léon Bottou, Ishaan Gulrajani, and David Lopez-Paz. Invariant risk mini- mization, arXiv preprint arXiv:1907.02893, 2019.
[2] Guangji Bai, Chen Ling, and Liang Zhao. Temporal domain generalization with drift-aware dy- namic neural networks. In The Eleventh International Conference on Learning Representations, 2023.
[3] Yogesh Balaji, Swami Sankaranarayanan, and Rama Chellappa. Metareg: Towards domain generalization using meta-regularization. Advances in neural information processing systems, 31, 2018.
[4] Konstantinos Bousmalis, Nathan Silberman, David Dohan, Dumitru Erhan, and Dilip Krishnan. Unsupervised pixel-level domain adaptation with generative adversarial networks. In Proceed- ings of the IEEE conference on computer vision and pattern recognition, pages 3722-3731, 2017.
[5] Michael M Bronstein, Joan Bruna, Taco Cohen, and Petar Veličković. Geometric deep learning: Grids, groups, graphs, geodesics, and gauges. arXiv preprint 21V::2200413477 2021.
[6] Zekun Cai, Guangji Bai, Renhe Jiang, Xuan Song, and Liang Zhao. Continuous temporal domain generalization. In Advances Neural Information Processing Systems, volume 37, pages 127987-128011 2024.
[7] Zekun Cai, Renhe Jiang, Xinyu Yang, Zhaoman Wang, Diansheng Guo, Hill Hiroki Kobayashi, Xuan Song, and Ryosuke Shibasaki. Memda: Forecasting urban time series with memory-based drift adaptation. In Proceedings of the 32nd ACM International Conference on Information and Knowledge Management, pages 193 202, 2023.
[8] Kathleen Champion, Bethany Lusch, J Nathan Kutz, and Steven L Brunton. Data-driven discovery of coordinates and governing equations. Proceedings of the National Academy of Sciences, 116(45):2244--2244 2019.
[9] Ricky TQ Chen, Yulia Rubanova, Jesse Bettencourt, and David K Duvenaud. Neural ordinary differential equations. Advances in neural information processing systems, 31, 2018.
[10] Gordon Christie, Neil Fendley, James Wilson, and Ryan Mukherjee. Functional map of the world. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. pages 6172-6180, 8.
[11] Colin Clement, Matthew Bierbaum, Kevin P O' Oeeefffe, and Alexander A Alemi. On the use of arxiv dataset. arXiv preprint arXiv: oett:9955000770 2019.
[12] Li Deng. The mnist database of handwritten digit images for machine learning research [best of the web). IEEE signal processing magazine, 29(6): 141-142, 2012.
[13] Qi Dou, Daniel Coelho de Castro, Konstantinos Kamnitsas, and Ben Glocker. Domain gener- alization via model-agnosiee learning of semantic features. Advances in neural information processing systems, 32, 2019.
[14] Yingjun Du, Jun Xu, Huan Xiong, Qiang Qiu. Xiantong Zhen, Cees GM Snoek, and Ling Shao. Learning to learn with variational information bottleneck for domain generalization. In European Conference on Computer Vision, pages 200-216, 2020.
[15] Yuntao Du, Jindong Wang, Wenjie Feng, Sinno Pan, Tao Qin, Renjun Xu, and Chongjun Wang. Adarnn: Adaptive learning and forecasting for time series. In Proceedings of the 30th ACM International Conference on Information & Knowledge Management (CIKM), 2021.
[16] Mare Finzi, Samuel Stanton, Pavel Izmailov, and Andrew Gordon Wilson Generalizing convolutional neural networks for equivariance to lie groups on arbitrary continuous data. In International Conference Machine Learning, pages 3165-3176. PMLR, 2020.

---

[17] Yaroslav Ganin, Evgeniya Ustinova, Hana Ajakan, Pascal Germain, Hugo Larochelle, François training of neural networks.
Laviolette, Mario March, and Victor Lempitsky. Journal of machine learning research, (59):1 1-35, 2016.
[18] Boqing Gong, Yuan Shi, Fei Sha, and Kristen Grauman. Geodesic flow kernel for unsupervised domain adaptation, In 2012 IEEE conference on computer vision and pattern recognition, pages IEEE, 2012.
[19] Rui Gong, Wen Li, Yuhua Chen, and Luc Van Gool. Dlow: Domain flow for adaptation and generalization. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 2477-2486, 2019.
[20] Raghuraman Gopalan, Ruonan Li, and Rama Chellappa. Domain adaptation for object recogni- tion: An unsupervised approach. In 2011 international conference on computer vision, pages 999-1006. IEEE, 2011.
[21] Samuel Greydanus, Misko Dzamba, and Jason Yosinski. Hamiltonian neural networks, Advances in neural information processing systems, 32, 2019.
[22] Huan He, Owen Queen, Teddy Koker, Consuelo Cuevas, Theodoros Tsiligkaridis, and Marinka Zitnik. Domain adaptation for time series under feature and label shifts. In International Conference on Machine Learning, pages 12746-12774. PMLR, 2023.
[23] Zeyi Huang, Haohan Wang, Eric P Xing, and Dong Huang. Self-challenging improves cross- domain generalization. In 16th European Conference on Computer Vision, ECCV 2020, pages 1 144.10..50 Springer, 2020.
[24] Nikola Kovachki, Zongyi Li, Burigede Liu, Kamyar Azizzadenesheli, Kaushik Bhattacharya, Andrew Stuart, and Anima Anandkumar. Neural operator: Learning maps between function spaces with applications to pdes. Journal of Machine Learning Research, 24(89):1-97 2023.
[25] Kiran Krishnamachari, See-Kiong Ng, and Chuan Sheng Foo. Uniformly distributed feature representations for fair and robust learning. Transactions on Machine Learning Research, 2025.
[26] David Krueger, Ethan Caballero, Joern- -enrik Jacobsen, Amy Zhang, Jonathan Binas, Dinghuai Zhang, Remi Le Priol, and Aaron Courville. Out-of-distribution generalization via risk ex- trapolation (rex). In International conference on machine learning, pages 5815-5826. PMLR, 2021.
[27] Da Li, Yongxin Yang, Yi-Zhe Song, and Timothy Hospedales. Learning generalize: Meta- learning for domain generalization. In Proceedings of the AAAI conference on artificial intelligence, volume 38, 2018
[28] Da Li, Yongxin Yang, Yi-Zhe Song, and Timothy Hospedales. Sequential learning for domain generalization. In European Conference on Computer Vision, pages 603-619. Springer, 2020.
[29] Da Li, Yongxin Yang, Yi-Zhe Song, and Timothy Hospedales. Deeper, broader and artier domain generalization. In Proceedings of the IEEE international conference on computer vision, pages 5542-5550. 5550, 2017.
[30] Wen Li, Zheng Xu, Dong Xu, Dengxin Dai, and Luc Van Gool. Domain generalization and adaptation using low rank exemplar svms. IEEE transactions on pattern analysis and machine intelligence, 40(5):1114-1122 2017.
[31] Wendi Li, Xiao Yang, Weiqing Liu, Yingce Xia, and Jiang Bian. Ddg-da: Data distribution generation for predictable concept drift adaptation. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pages 4092-4100, 2022.
[32] Ya Li, Xinmei Tian, Mingming Gong. Yajing Liu, Tongliang Liu, Kun Zhang, and Dacheng Tao. Deep domain generalization via conditional invariant adversarial networks. In Proceedings of the European conference on computer vision (ECCV), pages 624 639, 2018.
[33] Alexander H Liu, Yen-Cheng Liu, Yu- Ying Yeh, and Yu-Chiang Frank Wang. A unified feature disentangler for multi-domain image translation and manipulation. Advances in neural information processing systems, 31, 2018.
12

---

[34] Lu Lu, Pengzhan Jin, Guofei Pang, Zhongqiang Zhang, and George Em Karniadakis. Learning nonlinear operators via deeponet based on the universal approximation theorem of operators. Nature machine intelligence, 3(3):218-223 2021.
[35] Wang Lu, Jindong Wang, Xinwei Sun, Yiqiang Chen, and Xing Xie. diiti-idist representation learning for time series classification. In International Conference on Learning Representations, 2023.
[36] Massimiliano Mancini, Samuel Rota Bulo, Barbara Caputo, and Elisa Ricci. Best sources forward: domain generalization through source-specific nets. In 2018 25th IEEE international conference on image processing (ICIP), pages 1353-1357. IEEE, 2018.
[37] Massimiliano Mancini, Samuel Rota Bulo, Barbara Caputo, and Elisa Ricci. Adagraph: Uni- fying predictive and continuous domain adaptation through graphs. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 6568 0088-6577,22 2019.
[38] Saeid Motüann, Marco Piccirilli, Donald A Adjeroh, and Gianfranco Doretto. Unified deep supervised domain adaptation and generalization. In Proceedings of the IEEE international conference on computer vision, pages 5715-5725, 2017.
[39] Krikamol Muandet, David Balduzzi, and Bernhard Schölkopf. Domain generalization via invariant feature representation. In International conference on machine learning, pages 10-18.
PMLR, 2013.
[40] Anshul Nasery, Soumyadeep Thakur, Vihari Piratla, Abir De, and Sunita Sarawagi. Training for the future: A simple gradient interpolation loss generalize along time. Advances in Neural Information Processing Systems, 14:19998-19209 2021.
[41] Zheyi Pan, Yuxuan Liang, Weifeng Wang. Yong Yu, Yu Zheng, and Junbo Zhang. Urban traffic prediction from spatio-temporal data using deep meta learning. In Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining, pages 1720-1730, 2019.
[42] Jason Priem, Heather Piwowar, and Richard Orr. Openalex: A fully-open index of scholarly works, authors, venues, institutions, and concepts. arXiv preprint arXiv:2205.0188 018333, 2022.
[43] Fengchun Qiao, Long Zhao, and Xi Peng. Learning to learn single domain generalization. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 12556-12565, 2020.
[44] Tiexin Qin, Shiqi Wang, and Haoliang Li. Generalizing to evolving domains with latent structure-aware sequential autoencoder. In International Conference on Machine Learning. pages 18062-18082. PMLR, 2022.
[45] Maziar Raissi, Paris Perdikaris, and George E Karniadakis. Physics-informet neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. Journal of Computational Physics, 378:686-707, 2019.
[46] Shiori Sagawa, Pang Wei Koh, Tatsunori Hashimoto, and Percy Liang. Distributionally robust neural networks for group shifts: On the importance of regularization for worst-case 2019.
generalization. arXiv preprint arXiv:
[47] Kuniaki Saito, Yoshitaka Ushiku, and Tatsuya Harada. Asymmetric tri-training for unsupervised domain adaptation. In International conference on machine learning, pages 2988-2997. PMLR, 2017.
[48] Josh Tobin, Rachel Fong, Alex Ray, Jonas Schneider, Wojeiech Zaremba, and Pieter Abbeel. Domain randomization for transferring deep neural networks from simulation to the real world. In 2017 IEEE/RS/ international conference on intelligent robots and systems (IROS), pages 23-30. IEEE, 2017.
[49] Jonathan Tremblay, Aayush Prakash, David Acuna, Mark Brophy, Varun Jampani, Cem Anil, Thang To, Eric Cameracci, Shaad Boochoon, and Stan Birchfield. Training deep networks with synthetic data: Bridging the reality gap by domain randomization. Proceedings of the IEEE conference computer vision and pattern recognition workshops, pages 969-977, 2018
13

---

[50] Eric Tzeng, Judy Hoffman, Kate Saenko, and Trevor Darrell. Adversarial discriminative domain adaptation, In Proceedings of the IEEE conference on computer vision and pattern recognition. pages 7167-7176, 2017.
[51] Hao Wang, Hao He, and Dina Katabi. Continuously indexed domain adaptation. In International Conference on Machine Learning, pages 9898 9907. PMLR, 2020.
[52] Jindong Wang. Cuiling Lan, Chang Liu, Yidong Ouyang. Tao Qin, Wang Lu, Yiqiang Chen, Wenjun Zeng, and S Yu Philip. Generalizing to unseen domains: A survey domain general- ization. IEEE transactions on knowledge and data engineering, 35($):8052-8072, 2022.
[53] Zihao Xu, Guang- Yuan Hao, Hao He, and Hao Wang. Domain-indexing variational bayes: Interpretable domain index for domain adaptation. Eleventh International Conference on Learning Representations (ICLR), 2023.
[54] Shen Yan, Huan Song, Nanxiang Li, Lincan Zou, and Liu Ren. Improve unsupervised domain adaptation with mixup training. arXiv preprint arXiv:2001.0066 2020.
[55] Huaxiu Yao, Caroline Choi, Bochuan Cao. Yoonho Lee, Pang Wei W Koh, and Chelsea Finn. Wild-time: A benchmark of in-the-wild distribution shift over time. Advances in Neural Information Processing Systems, 35: 10309-13324, 2022.
[56] LIN Yong, Fan Zhou, Lu Tan, Lintao Ma. Jianmeng Liu, HE Yansu, Yuan Yuan, Yu Liu, James Y Zhang, Yujiu Yang, al. Continuous invariance learning. In The Twelfth International Conference on Learning Representations, 2023.
[57] Dazhou Yu, Guangji Bai, Yun Li, and Liang Zhao. Deep spatial domain generalization. In 2022 IEEE International Conference on Data Mining (ICDM), pages 1293-1298 IEEE, 2022.
[58] Qiuhao Zeng, Changjian Shui, Long-Kat Kai Huang, Peng Liu, Xi Chen, Charles Ling, and Boyu Wang, Latent trajectory learning for limited timestamps under distribution shift over time. In The Twelfth International Conference on Learning Representations, 2024.
(59) Qiuhao Zeng, Wei Wang, Fan Zhou, Gezheng Xu, Ruizhi Pu, Changjian Shui, Christian Gagné, Shichun Yang. Charles X Ling, and Boyu Wang. Generalizing across temporal domains with koopman operators. Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 16651 16651, 2024.
14

---

adopt single- A Experimental Details
A.1 Dataset Details
To establish a comprchensive evaluation protocol, we construct a suite of synthetic and real-world datasets designed to cover diverse transformation types, domain structures, data fields, and prediction tasks. The synthetic datasets support controlled analysis of continuous distribution shifts through interpretable transformations. For real-world data, collect reorganize large-scale datasets across four domains in vision, language, and spatiotemporal modalities. Each domain is anchored to a real-world geography, time, semantics, economic context.
Synthetic Datasets Prior temporal domain generalization studies [1
40 2 59
factor domain variations. We instead introduce multiple co-occurring transformations into their widely adopted synthetic dataset, whose combinations induce complex distribution shifts and substantially increase the difficulty of generalization.
1. 2-Moons This dataset is a variant of the classic 2-entangled moons dataset, where the lower and upper moon-shaped clusters are labeled 0 and 1, and each contains 500 instances. We construct two types of transformations jointly: scaling, which enlarges the spatial extent of the moons by 10% per unit, and rotation, which rotates the entire structure counter- clockwise by 18 per unit. These transformations induce a two-dimensional descriptor z == [ == E 122. We train on 50 randomly sampled domains, and test on 150 randomly sampled others plus a fixed mesh grid of descriptor points. Examples from the train set are shown in Fig. 2 and the distribution of train and test domains visualized in Fig. 3 The continuous domain shift arises from smooth transformations of the moon-shaper clusters.
2. MNIST This dataset is variant of the classic MNIST dataset 12). where each domain consists of 1,000 digit images randomly sampled from the original MNIST. combina- tion of three transformations is applied to all samples within each domain: rotation (18 counterclockwise per unit), color shift (from red to blue over the transformation space), and occlusion (increasing coverage ratio per unit). These transformations induce three- dimensional domain descriptor = = [=to 200 2000] R-3, We randomly sample 50 domains for training and another 50 for testing. Fig. 6 illlustrates representative training and test domains. The visual appearance of digits exhibits substantial variation due to combinations of transformations, posing significant challenge for models to generalize.
Real-world Datasets
3. IMoW The IMoW dataset 10 consists of over million high-resolution satellite images collected globally between 2002 and 2018. Each image is labeled with the functional purpose of the region, such as airport, aquaculture, or crop fields. We select ten common categories to construct a multi-class classification task and define each country domain, resulting in 96 domains. For each country, we collect publicly available climate statistics from the CRU climate datase(") including seasonal-term averages of temperature, precipitation, humidity, and solar radiation. These climate factors influence land use deci- sions and introduce meaningful distribution shifts driven by environmental and geographic variability. We randomly select 50 domains for training and use the remaining for testing.
separate
4. Arxiv The arXiv dataset [T] comprises 1.5 million pre-prints over 28 years, spanning fields such as physics, mathematics, and computer science. We construct a bitle-based classification task to predict the paper's subject. Domains are defined by the publisher that eventually accepted the article, yielding 83 domains. This captures semantic variation across venues; for example, the term "neural" refers to artificial networks in IEEE, but to biological systems in Nature Neuroscience. Publisher metadata is obtained from the open- source OpenAlex 42 We randomly select 40 domains for training and use the remaining for testing.
5. YearBook The Yearbook dataset [55] contains frontal portraits from U.S. high school yearbooks spanning 1930 to 2013. The task is to classify gender from facial images. Same as the setting in (6), we sample 40 years from the 84-year range, treating each year separate domain. The resulting domains are temporally ordered. The first 28 domains are
15

---

used for training and the remaining for testing. This dataset serves as a one one-dimensional temporal testbed, where time is treated special case of continuous domain generalization. It enables existing temporal domain generalization methods to be comparable with ours.
6. Traffic We use a real-world taxi flow dataset [4] collected from Beijing, covering the period from February to June 2015. The city is partitioned into 1,024 (32 x 32) regions, and the task is time-series forecasting of future traffic flow based on past hourly taxi inflow and outflow observations. Each region associated with the distribution of Points of Interest (POIs). We treat each region separate domain and use the POIs distribution its descriptor, as it reflects functional differences land use that potentially influence local traffic dynamics. We select 100 domains for training and use the remaining for testing.
Distribution
Domain Distritution
Train
Random
40
Grid Test
(Rotation)
20-
10
25
30
20
20
20 (Scoling)
|Scaling!
Figure 5: Train and test domain descriptors for the 2-Moons dataset. The left plot shows the 50 training domains, while the right plot shows the 150 randomly sampled test domains and additional test domains uniformly distributed over a fixed mesh grid in the descriptor space.
A.2 Comparison Methods
Descriptor-Agnose Methods. These methods operate purely on input-output pairs and do not utilize domain-level descriptor information:
ERM (Empirical Risk Minimization): Trains single model across all domains by minimizing average training loss.
IRM [ Encourages invariant predictive relationships across environments by penalizing gradient dependence between input and loss.
V-REx [6] Enforces risk extrapolation to approximate invariant predictors, offering robustness to covariate shift and enabling partial causal identification.
GroupDRO 46 Adopts worst-case optimization over domain-spectio risks to ensure robustness.
Mixup 64 Applies data interpolation across domains to promote smoothness and generalization for unsupervised domain adaptation.
DANN 17 Introduces domain adversarial training align latent representations across domains. Performs meta-learning by simulating domain shifts during training via meta- train/meta-test splits.
MLDG 27
CDANN 32
An adversarial method that conditions domain labels enforce domain-invarian
features.
A recent invariant learning method that enforces uniformity across both data distribu- tions and feature representations.
URM |25
Deseriptor-Aware Methods. These methods incorporate explicit domain descriptors to guide model generalization across domains:
ERM-D: An extension of Empirical Risk Minimization that incorporates the domain descriptor as an auxiliary input. Specifically, the deseriptor is first encoded via a shallow MLP and then concatenated with the feature representation of the input data. This allows the network condition its prediction on domain-specific signals.
NDA (Nearest Domain Adaptation): This method first trains global model on all training domains, then fine-tunes individually for each domain. For test-time prediction, selects the closest training domain in descriptor space and directly uses its fine-tuned model.
16

---

Train Domains 18 of 50 Domain [16.6, 7.651
Domain 19.0, [49.7, 40.92, 12.00] [32.0, 10.57]
Domain
Domain
Domain [4.1, 14.52)
Domain
[26.8, 12.61]
Domain 19: 3 [37.7, 8.82, 12.54]
Domain 20: [13.9, 29.05, 13.35)
Domain 24: [30.2, 41.80, 6.98]
Domain 28: [14.2, 8.20, 10.34)
Demain 33: 3 [32.0, 0.15, 8.36)
Domalnn 34: [17.4, 21.57, 7.36]
Domain 36: 1 [3.6, 48.28, 8.72)
Domain 37: 3 [3.6, 2.75, 12.26]
Domain 39: [5.6. 14. 14.05, 14.40]
Domain 44: [19.0. 2.23, 14.80]
Domain 45: 7 [4.7, 4.22. 7.60]
Domain 49: [39.8. 29.09, 9.89]
Test Domains
50
Domain 50:4 [34.5, 26.39 26.39, 9.62]
Domein 51:2 [27.2, 40.49, 13.34]
Domain 56: [3.9, 36.37, 7.69]
Domain
Domain 61: [25.5, 44.46, 6.93]
Domein
[41.8, 14.48, 12.85)
(25.1, 14.60)
Domain 73: [1.5, 26.08, 12.96] [37.3, 49.75, 13.39]
Domain 78:
Domain 84: (23.6, 11.73, 8.04]
Domain 89: [21.5, 10.81, 8.11]
Domain 90: 1 [22.6, 26.28, 10.39)
Domain 91: 6 [19.9, 47.41, 8.67]
Domain (41.1, 38.85, 7.41]
Domain 95: (30.3, 46.23, 5.55]
96:
97: [36.9, 47.45, 13.26]
Domaln 13.20]
[10.7, 7.66, 6.81]
[37.7, 0.22, 13.89]
Figure 6: Visualization of the MNIST dataset. Each domain corresponds a unique combination of three transformations: rotation, color shift, and occlusion. One representative image is sampled from each domain for visualization. The title above each image indicates its domain index and ground-truth label, while the descriptor is shown below. The top and bottom panels respectively show 18 randomly sampled training and domains.
CIDA 51:: CIDA addresses domain adaptation for continuously indexed domains. It employs descriptor-conditionn discriminator that predicts the domain index value from encoded features. The encoder is trained adversarially to obfuscate this prediction, encouraging domain- inmain-invariant representations across domains. our setup, we adopt the milei-dimeeiional extension of CIDA, adapting the discriminator regress continuous descriptors.
TKNets (59
Learns the temporal evolution the model by mapping them into a Koopman operator-drivent driven latent space, where the dynamics are assumed to be linear. Each temporal domain has shared latent linear operator that governs its transitions.
Models the temporal evolution predictive models under distributional drift using recurrent neural architecture. Specifically, DRAIN employs an LSTM-bassed controller to dynamically generate model parameters conditioned on temporal dependency, enabling the model to adapt to continuously shifting data distributions without access to future domains.
DRAIN 2
17

---

Models the continuous evolution of model parameters with neural ODEs, where the dynamics are driven by temporal dependency. It synchronizes the progression of data distributions and model parameters through latent learnable dynamics.
Koodos [6]:
A.3 Model Configuration
We provide full implementation details and code in our repository Our model comprises four jointly trained components: a per-domain predictive model, parameter encoder-decoder pair, a neural transport operator, and a gating mechanism. We adopt a task-specific predictive architecture (detailed below) shared across all domains. Each training domain is associated with its own parameter vector, while maintaining structural consistency. The parameters encoder and decoder are four-layer MLPs with ReLU activations. The gating mechanism consists of linear transformation and a learnable mask. The neural transport operator implemented as an exponentiated vector field over the descriptor space. It includes two sub-networks: a field network that maps the source descriptor z; to set of basis matrices, and a coesficient network that maps the descriptor difference A: = 2j 2j to set of scalar weights. The resulting transformation is computed by sequentially applying matrix exponentials of the scaled basis matrices as described in Eq. 6 For models with large parameter sizes, introduce shared feature extractor and infer only the remaining domain-specinn specific parameters. All experiments are conducted on a 64-bit machine with two 20-core Intel Xeon Silver 4210R CPUs @ 2.40GHz, 378GB memory, and four NVIDIA GeForce RTX 3090 GPUs. Results are averaged over five runs with different random seeds using the Adam optimizer. We specify the task-specific predictive architecture for each dataset follows:
1. 2-Moons The predictive model a three-layer MLP with 50 hidden units per layer and ReLU activations. The encoder and decoder are both four-layer MLPs with layer dimensions [1024, 512, 128, 32). The transport operator consists of 32-dimensional linear field network with 2 basis matrices. The learning rate is set to 1 x 10-3
2. MNIST The shared feature extractor is convolutional backbone composed of three convo- lutional layers with channels [32, 32, 64), each followed by a ReLU activation and max pooling layer with kernel size 2. The resulting features are flattened and passed through a dropout layer. The per-domain predictive model is a two-layer MLP with hidden dimen- sion of 128 and an output dimension of 10. The encoder and decoder are four-layer MLPs with layer dimensions [1024, 512, 128, 32). The neural transport operator implemented as 32-dimensional linear field network with 3 basis matrices. The learning rate is set to 1 x 10 for all components.
3. MMoW ResNet-50 backbone pretrained on ImageNet as the shared feature extractor to capture high-level semantics. The extracted features are fed into per-domain predic- tive model implemented as a three-layer MLP with hidden dimensions [128, 64] and an output dimension of 10. The encoder and decoder are four-layer MLPs with dimensions [1024, 512, 128, 32). The neural transport operator is implemented as 128-dimensional linear field network with 5 basis matrices. The learning rate is set to 1 x 10-3
4. Arxiy Each paper title is first embedded using a 384-dimensional representation. The per-domain predictive model is three-layer MLP with hidden dimensions [50, 50] and output dimension 10. The encoder and decoder are four-layer MLPs with layer dimensions [1024, 512, 128, 32). The neural transport operator is implemented as a 32-dimensional linear field network with 5 basis matrices. The learning rate is set to 1 10-3 for all components.
encoder, resulting in
5. YearBook The shared feature extractor is convolutional backbone composed of three convolutional layers with channels [32, 32, 64], each followed by a ReLU activation and max pooling layer. The output is flattened and passed through a dropout layer. The per-domain predictive model is a three-layer MLP with hidden dimensions [128, 32] and output dimension 2. The encoder and decoder four-layer MLPs with dimensions [1024, 512, 128, 32). The neural transport operator is implemented as a 32-dimensional linear field network with 5 basis matrices. The learning rate is set to 1 x 10-3
6. Traffic The predictive model is three-layer MLP that takes as input a flattened 96- dimensional vector representing 48 historical inflow and outflow pairs, and outputs dimensional vector corresponding 3-step future predictions. The hidden dimension is set
18

---

to 64. The encoder and decoder are four-layer MLPs with dimensions [1024, 512, 128, 32). The neural transport operator is implemented as a 32-dimensional linear field network with 5 basis matrices. The learning rate is set to x 10-3 3 for all components.
A.4 Complexity Analysis
In our framework, all domains share common feature extractor, while each domain is associated with a domain-specifii parameter vector 0 E RDD This vector is first encoded into a low-dimensional latent embedding& through neural autoencoder implemented as MLP. The embedding is then updated by a neural transport operator defined over the descriptor space and subsequently decoded back into 0 to produce the generalized model.
The overall complexity of this transformation is O(2(Dn + E) F), where is the width of the first encoder layer, E' is the total number of parameters in the remaining layers of the encoder, and F denotes the parameters in the transport and gating modules. Specifically, F includes the field network, which maps the source descriptor to a small set of basis matrices in the latent space, and the coefficient network, which maps descriptor differences scalar weights over those basis matrices. Since both networks operate in a low-dimensional space (i.e., the same dimension as e), their parameter sizes remain small. The gating module is implemented as a lightweight mask vector over descriptor dimensions and adds negligible overhead. In practice, D is also small because most model parameters, such those in convolutional transformer- -basdd feature extractors, are shared across domains. Domain-specific parameters are usually applied only to a few final layers. Consequently, the overhead introduced by our framework remains controlled.
A.5 Ablation Study
To understand the contribution of each core component in our framework, we conduct a series of ablation studies on the 2-Moons and Traffic datasets. We focus on the impor- tance of structure-aware transport design, gating mechanisms, and local chart-based infer- ence. All variants are trained and evaluated under the same protocol as the main experiments. We achieve the following variants, results can be found in Table 3 (1) Plain: replaces the neural Lie transport operator with a plain MI.P that directly maps (24, 2z, e;) to ej. termed as Plain. The performance of the T Plain variant demonstrates the limitations of removing geometric and algebraic constraints, as it consis- tently underperforms our full model across all datasets. The lack of structural bias causes the transport function overfit training tuples, failing to extrapolate to unseen domains. (2) noLie: bypass the exponential mapping that projects Lie algebra elements the Lie Group. Instead, the descriptor difference As is first applied to the field network to produce a raw linear operator, which is then applied directly to the source embedding e, to obtain ey. T noLie variant reduces the transformation a descriplor-conddtioneed linear mapping without enforcing any group structure. It leads to severe performance degradation, especially on the 2-Moons dataset, and remains suboptimal on the real-world dataset. This highlights the importance of algebraic structure, without which the learned transport fails to align reliably with the underlying domain variation. We further ablate two structural modules designed to handle imperfect descriptors. Disabling the gating mechanism leads to performance drops under noisy or redundant descriptors, indicating its role in suppressing irrelevant dimensions. Removing charting similarly impairs gener- alization under incomplete descriptors, as transport lacks global consistency. Both components are critical for robust modeling under imperfect descriptor conditions. More detailed evaluations of these modules are presented in Section 5.3
Table 3: Ablation test results for different datasets.
Ablation
2-Moons
Traffic
T Pluin
13.3 1 0.4
16.5 + 1.0
T - noLie
34.8 + 0.5
16.4 + 0.3
w/o Gating
16.3 + 0.4
w/o Chart
16.1 = 0.3
NeuralLio
3.2 + 1.2
15.1 + 0.1
A.6 Scalability Analysis
We evaluate the scalability of our framework with respect to two key factors: the number of prediction model parameters and the number of domains. Computation time is measured under increasing model and domain complexity, normalized by the shortest time to provide consistent basis.
Fig. 7 (a) reports computation time on the fMoW dataset as a function of prediction model size, varied from 80K to 9M parameters by adjusting network depth and width. The growth in runtime is smooth and nearly linear, which aligns with the theoretical complexity described in Section A.4
19

---

Computation Time vs, Parameter Size
Computation Time vi. Number of Domains
2.5
Relative Time
Relative
2.0
200
6.0
800
1000
Predictive
Number Domains
(a) Scalability for Parameter Size
(b) Scalability for Domain Number
Figure Scalability analysis w.r.f the number of parameters and the number of domains.
Fig. 7 (b) presents the runtime function of the number of domains, evaluated on the 2-Moons dataset with domain counts ranging from 10 to 1000. Domains are sampled from the descriptor space with fixed number of data points per domain to ensure comparability. The runtime increases linearly with domain count, reflecting the scalability of our per-domain inference pipeline and supporting its applicabillit to large-scale domain generalization tasks.
Table 4: Cost of training and testing time.
Model
Train Time (s)
Test Time (s)
CIDA
711
0.6
DRAIN
144
0.2
TKNets
1250
0.3
Koodos
540
0.1
Ours
694
0.1
We further benchmark the wall-clock training time of our method against representative baselines that explicitly model inter-domain relationships, including CIDA (5T), DRAIN(), TKNets (59), and Koodos [6]. To support a broader set of bascline methods, we conduct this comparison on the YearBook dataset. All models are trained under identical hardware and epoch configurations. As shown in Table
4 our method matches the training efficiency of prior structure-awar models (e.g.. Koodos, CIDA) while achieving the lowest testing cost. This reflects a well-baland balancee trade-off of our model between computational efficiency and effectiveness.
2
Mooos
Moons
Noons
2 2-Moons
Error Rates (%)
Error (%6)
Error Rates (9)
Error Rates 190
Number of Field Basis
Dim of Field Network
Number
Dim of Paramter Embeding
Figure 8: Sensitivity analysis. From left to right: number of field basis vectors used in the transport operator, hidden dimension of the field network, number of neighbors K in the local chart module, and the dimension of the parameter embedding e.
A.7 Sensitivity Analysis
We conduct sensitivity analysis on the 2-Moons dataset to evaluate the robustness of our method to key hyperparameters. Specifically, we vary four components: (1) the number of field network I in the operator, denoted as the Number of Field Basis; (2) the hidden dimension of the MLP implementing the field network, denoted as the Dim of Field Network; (3) the number neighbors
K used in the local chart module; and (4) the embedding dimension of predictive model parameters after encoding, denoted as the Dim of Parameter Embedding.
As shown in Fig. g our model exhibits strong robustness across a broad range of values. The error rate remains stable when varying the number of basis vectors and the hidden dimension of the field network, indicating flexibility in the transport capacity. For K, performance is consistent for moderate neighborhood sizes (c.g., K ~ 10), while very small K leads to degradation due to
20

---

mmoer-smoothing.. For the embedding dimension of e, we observe an optimal range below 128, which show that overly large values introduce overparameteriiatton. These results show that our model outperforms the baseline model over wide range of hyperparameters, confirming the robustness and adaptability of our framework.
2 Moons
2 Moons
2 Moons
40
100
60
7am
10
Train Loss
40
Rates
40
20-
Training
Training
20
10-
20
10
Test
20
60
80
100
100
150
200
200
400
600
Number of Training Domains
Epoch
Epoch
Test error (%) on 2- M-oons as function of the number of training domains. Middle: Training loss curves for different numbers of training domains. Right: Training loss and test error over long epochs.
Figure 9: Convergence analysis. Left:
A.8 Convergence Analysis
To evaluate the learning behavior and convergence properties of our framework, we conduct a series of controlled experiments the 2-Moons dataset. We design three complementary analyses: (1) convergence with respect to the number of training domains, (2) convergence trajectories across epochs under varying the number of domains, and (3) joint dynamics of training loss and test performance over extended training.
Convergence results across different domain counts. We evaluate convergence under increasing numbers of training domains on the 2-Moons dataset. Specifically, we vary the number of training domains from 5 to 100, while fixing the test domains evaluation. As shown in the left panel of Fig. g the test error drops rapidly as the number of training domains increases. With few 20 training domains, the model already achieves test error below 10%, and beyond 50 domains, the error consistently falls below 5%. At 80 domains, the model shows near-perfect generalization. These results demonstrate that our model can recover the underlying parameter field even under very sparse supervision, and increasing the number of training domains is crucial for capturing the variation structure.
predefined mesh grid ensure consistent
Convergence trajectory across different domain counts. The center panel of Fig. 9 illustrates the training loss trajectories over epochs under different numbers of training domains (10.30.50,70) Across all settings, the loss consistently decreases and stabilizes after around 120 epochs, demonstrat- ing the convergence stability of our framework. Moreover, as the number of domains increases, the model not only converges to a lower final loss (insight from the previous paragraph) but also reaches convergence more rapidly. This confirms that greater domain coverage not only improves model accuracy but also accelerates optimization.
Joint dynamics of training loss and test performance. The right panel of Fig. 9 plots both the training loss and test eттоr under our main experimental setting, with over 600 epochs. The model converges rapidly and maintains consistent test performance throughout training, showing no signs of performance degradation. This stability arises from the structural constraints imposed by the architecture, which regularize domain-wise variation and prevent the model from memorizing individual domains. This observation aligns with our theoretical analysis in Property 2 suggesting that enforcing structural constraints naturally limits model overfitting and supports robust generalization.
A.9 Limitations
Our framework assumes the availability of continuous descriptors to model domain variation. Two practical challenges arise in this context. First, many domain generalization tasks are still formulated under discrete domain assumptions. However, we contend that such seemingly discrete observations reflect limited observations over implicit continuous process. For example, real, cartoon, and sketch domains likely lie along a visual continuum from realism to abstraction; formal and informal text styles mark the endpoints of stylistic spectrum; and related languages such English, German,
21

---

and Dutch reflect gradual linguistic evolution. The success of recent diffusion models also supports this view. For such cases, the challenge is not discontinuity, but how to coordinate observed domains within the implicit continuous space. Second, in more extreme scenarios, descriptors may entirely unavailable or unobservable. While this uncommon in structured environments, automatic descriptor inference is possible. Recent advances in contrastive representation learning and adversarial domain embedding provide promising tools to estimate latent descriptors directly from the input. These mechanisms can be readily integrated into our framework additional descriptor discriminators or encoder networks.
Overall, we view domain descriptor construction as a key and promising direction. Building on our current framework, such extensions can further expand the applicability of continuous domain generalization to more -spervsupe environments.
22

---

B Optimization Details
The joint training strategy follows the general optimization paradigm established in [6], which has been shown to be principled and empirically validated for coordinating the temporal evolution of model parameters. Our method generalizes its idea a structural transport operator across descriptor space, extending its applicability beyond purely temporal settings. Specifically, we jointly optimize the per-domain parameters ((2)), encoder fe, decoder Sa, neural transport operator T, and gating mechanism. For each training descriptor 24, supervise its neighborhood ((2x) by aligning predictions, latent representations, and transported parameters between (æ 21) pairs. This formulation ensures local consistency and smooth transitions across the parameter manifold. During inference, we identify the nearest training descriptor for a test domain and apply the transport operator to infer its model parameters. The detailed training and inference procedure summarized in Algorithm
Algorithm 1: Training and Inference Procedure of NeuralLio
Input:
Domain set {(X,, Y)) with descriptors {2, ) Predictive model architecture g; Test data Xs with descriptor 2n
Output: Learned parameters {0(z))i=i Encoder fe. Decoder fa, Operator T; Gating g; Test prediction Y,
// Training phase initialize per-domain parameters {0(2)) encoder fe, decoder fa. transport operator T, and gating g
Initialize modules:
Build neighborhoods: for each 2,, compute k-nearest neighbors (22;) using Euclidean distance foreach training iteration do
Sample minibatch B of domain indices
foreach i & B do
Fetch (X,, Y,, ((:)); encode e(²,) fe((2z))); reconstruct 0(z,) = fa(e(zi))
Compute losses: Ciped (()(Xx;((2))) Yi). C [10(z)) (22
foreach 2, E (22,) do
Fetch (Xj, Y,, (zz))
Gating: m (2;) = Sigmoid 2:) 3 w, = m(z), 2 o m(=:)
Predict ((2y) T(e(=)), &, Rj); decode 0(2,) = fa(ē(z;))
Compute Cr
pred [(g(X;; (2,)), Yj)
Compute L consist ([(2z)) (²
Compute C [l((2;) - (0(()))))
Update all modules with total loss:
C E( + Ci + Σ (is + C
C
11 Inference phase
Find nearest 2; 2, and fetch ((2);; encode e(2;) fe((i)))
Gating: m(2) = Sigmoid W 2,) O w, & = 2, 3 mm2)
Generalize: é(z,) T(e(z;), E,, 2,): 0(z.) fa(é(zz))
Predict: Y, = g(X:; 0(z)))
23

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: Yes, the main claims in the abstract and introduction accurately reflect the paper's contributions and scope.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: Discuessed in Sec. A.9
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]

---

Justification:
The paper provides a complete set of assumptions along with full and correct
proofs.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: We fully discloses necessary information to reproduce the main experimen- tal results, also will publish code and data.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, it may be necessary either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, or other means that appropriate to the research performed.
While NeurlPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome deseribe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
25

---

Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Question:
Answer: [Yes)
Justification:
We provide link access the code and data.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips... cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https
/ "cipps.c/puullc//guu /otee//ggudddeeeedSuuuubmiibii ssionPolicy) for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones omitted from the script and why.
At submission time, preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (c.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: Yes, the paper specifies all relevant training and test details.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification:
We report error bars appropriately and includes relevant statistical information.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
26

---

The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
OK to report 1-sigma error bars, but one should state The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative eтror rates).
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: Yes, we report it in the appendix.
Guidelines:
The answer NA means that the paper does include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g.. preliminary or failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics httpp:///aarrrs...cppppppi/hhhea
Answer: [Yes]
Justification:
We respect with the NeurlPS Code Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive socictal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: Yes, the paper discusses the potential positive societal impacts the introduc- tion and problem definition. As the work is theoretical in nature, there are no foresecable negative societal impacts.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
27

---

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: We are conducting basic theoretical research, which does not involve this item, and all the data we use is public data.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g.. code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: All external assets used explicitly acknowledged and respected.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the batasets.papersstt paperswi thcode. cww/ datasets package should be provided. For popular datasets,
has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
28

---

If this information is not available online, the authors are encouraged to reach out to the asset's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer:
[NA]
Justification:
Out scope.
Guidelines:
The answer NA means that the paper does release new assets.
Researchers should communicate the details the dataset/codeemmodll as part their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification: Out scope.
Guidelines:
The answer NA means that the paper does involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Bthics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: This paper does not involve human subjects or user studies and therefore does not require IRB approval.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, not include any information that would break anonymity (if applicable), such the institution conducting the review.

---

16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology. scientific rigorousness, originality of the research, declaration is not required.
Answer: [NA]
Justification: LLMs are not used as component of the core methods in this research.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
Please refer to L.LM policy httt:p///eerrrree.cc////feeereee for what should or should not be described.
30


Reviews:
[{"review_id": "5w6lF8DKqW", "reviewer": "Reviewer_rCQN", "summary": "This paper introduces Continuous Domain Generalization, extending domain generalization from discrete/temporal settings to arbitrary continuous descriptor spaces. The authors propose a Neural Lie Transport Operator (NeuralLTO) that learns to transport model parameters between domains while enforcing geometric continuity and algebraic consistency. They prove that optimal parameters form a low-dimensional manifold and include mechanisms for handling noisy/incomplete descriptors.", "questions": "1. How does NeuralLTO perform when P(Y∣X,z) changes drastically  (e.g., sim-to-real gaps)? Have you measured cycle-consistency error in such cases?\n\n2. The method fixes several important hyperparameters: k=5 neighbors for local charts (Equation 9) and a 32-dimensional latent space for parameter embedding. How sensitive are the results to these choices? Specifically, can you provide sensitivity analysis showing performance.", "limitations": "Yes", "rating": 4, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 3, "strengths_and_weaknesses": "**Strength:** \n\n1. Generalizes beyond temporal domain shifts to multi-dimensional continuous spaces\n2. Theorem 1 provides geometric justification for parameter manifold structure\n3. Lie group theory provides elegant mathematical framework\n\n\n**Weaknesses:** \n\n1. Evaluation on toy or lightly filtered datasets. Experiments focus on 2-Moons with a synthetic “rotation” descriptor and a small Climate-fMoW split . These shifts resemble data augmentation more than uncontrolled, multi-factor drift, so real-world value is unclear.\n2. In the middle of the proof of Theorem 1, the authors assume Jz θ(z) has rank d. This pre-condition is not listed up front, and real descriptors rarely satisfy it; the manifold guarantee may therefore not apply. Please put assumption Cleary visible.\n3. Definition 2 demands closure, identity, associativity and invertibility, yet the NeuralLTO implementation is a neural approximation of the exponential map. However, the paper provides no experimental validation that these group properties actually hold in practice for the learned transport operator.\n4. Each domain stores its own parameter vector θᵢ plus a private head inflating model size. Appendix A.4 claims overhead is “controlled” .", "was_revised": true, "final_justification": "My concerns have been addressed, and I kept my score.", "rebuttal": "Thanks for reviewing and acknowledging our work. Please find our response as follows:\n> W1. Evaluation on toy or lightly filtered datasets. Experiments focus on 2-Moons with a synthetic “rotation” descriptor and a small Climate-fMoW split . These shifts resemble data augmentation more than uncontrolled, multi-factor drift, so real-world value is unclear.\n\n1) Our evaluation includes four large-scale real-world datasets with **uncontrolled and multi-factor domain shifts**:\n\n- fMoW involves complex shifts from factors as temperature, precipitation, humidity, and radiation;\n- ArXiv captures semantic drift across disciplines and publishers;\n- Yearbook spans decades of social, technological, and stylistic change;\n- Traffic reflects variation in POI distributions, urban function, and socio-economic activity.\n\nThese shifts are **inherent, naturally occurring, and not based on data augmentation**. All datasets are widely recognized for real-world distribution shift: fMoW, ArXiv, and Yearbook are from paper in NeurIPS21 DB Track [55] (citation #121), one of the most comprehensive and widely adopted datasets for distribution shift. Traffic from paper in KDD19 [41](citation #647), and has been used in numerous urban analytics and spatiotemporal generalization studies.\n\n2) The use of synthetic datasets (2-Moons, MNIST) follows standard ML practice, providing concise, interpretable ground-truth variation that contributes to rigorously evaluating the mathematical behavior of models. Importantly, existing methods all perform poorly, **confirming the non-triviality of the task even under such seemingly ‘toy’ settings**.\n\n> W2. In the middle of the proof of Theorem 1, the authors assume Jz θ(z) has rank d. This pre-condition is not listed up front, and real descriptors rarely satisfy it; the manifold guarantee may therefore not apply. Please put assumption Cleary visible.\n1) We appreciate the comment. To clarify, **Theorem 1 does not assume the descriptors used in practice**. Rather, it expresses that a complete, non-degenerate descriptor space theoretically exists and characterizes the model parameter structure. Such a theoretical descriptor space is grounded in reality, as distribution shifts are ultimately governed by a finite set of latent factors. \n2) Practical descriptors are imperfect. We fully acknowledge this, but  **practical descriptors serve as projections of the theoretical space**, so having partial descriptors is still better than having none. Moreover, the underlying manifold established in Section 4.1 **exists independently of practical descriptor quality**. Our method explicitly addresses the imperfect problem in Section 4.3. Experiments in real-world datasets and Section 5.3 further demonstrate that these partial descriptors are informative and effective.\n3) We will revise the statement of Theorem 1 to clarify this point and remove ambiguity.\n\n> W3. Definition 2 demands closure, identity, associativity and invertibility, yet the NeuralLTO implementation is a neural approximation of the exponential map. However, the paper provides no experimental validation that these group properties actually hold in practice for the learned transport operator.\n1) NeuralLTO is grounded in Lie group theory and is defined via exponential maps in Lie algebras, which theoretically satisfy group properties by definition.\n2) To validate it, we get the $\\theta$ for all test domains of 2-Moons, then conduct the following evaluations:\n- **Identity**: For each test descriptor $ z_i $, we apply $\\theta(z_i) \\xrightarrow{z_i \\rightarrow z_i} \\theta'(z_i)$ and evaluate the cosine similarity: $\\cos(\\theta(z_i), \\theta'(z_i))$.\n- **Associativity**: We consider all triplet combinations $(z_i, z_j, z_k)$ from the test descriptors. For each triplet, we get two results: $\\theta(z_i) \\xrightarrow{z_i \\rightarrow z_j} \\theta'(z_j) \\xrightarrow{z_j \\rightarrow z_k} \\theta'(z_k)$ and $\\theta(z_i) \\xrightarrow{z_i \\rightarrow z_k} \\theta''(z_k)$, then compute $\\cos\\left( \\theta'(z_k),\\ \\theta''(z_k) \\right)$\n- **Invertibility**: We consider all pairs $(z_i, z_j)$ from the test descriptors. For each pair, we apply $\\theta(z_i) \\xrightarrow{z_i \\rightarrow z_j} \\theta'(z_j) \\xrightarrow{z_j \\rightarrow z_i} \\theta'(z_i)$, then compute $\\cos\\left( \\theta(z_i),\\ \\theta'(z_i) \\right)$.\n- **Closure** is inherently satisfied, as the transport operator always maps within the valid parameter space.\n\nResults are as follows:\n| Property| Cosine Similarity|\n|-|-|\n| Identity| 99.9%|\n| Associativity|99.1%|\n| Invertibility|98.3%|\n\nThese results indicate that the learned operator satisfies the group axioms with high fidelity.\n\nWe appreciate your constructive comments and will incorporate the above content into the camera-ready version.\n\n> W4. Each domain stores its own parameter vector $\\theta_i$ plus a private head inflating model size. Appendix A.4 claims overhead is “controlled” .\n1) Domain-specific parameters are confined to the final layers (e.g., lightweight MLPs), while the feature extractor (e.g., CNN backbone), which accounts for the vast majority of model parameters, is fully shared. This greatly limits the additional cost per domain.\n2) After training, we can store only the parameter embedding vector $e$ of each domain, which further reduces cost.\n3) The transport operator is also lightweight, just consisting of a few MLP layers.\n\n> Q1. How does NeuralLTO perform when P(Y∣X,z) changes drastically (e.g., sim-to-real gaps)? Have you measured cycle-consistency error in such cases?\n1) We clarify that CDG assumes distribution continuous shifts, in line with the **continuum hypothesis** widely adopted in physics and engineering: every real-world macroscopic systems evolve smoothly over time, with their physical quantities modeled as continuous fields governed by differential equations.\n2) What may appear as a \"gap\" reflects **coarse or sparse sampling** over an underlying continuous variation field. This is not a limitation for our method: as shown in Figures 5 left and 6, the training domains are intentionally designed to be **extremely sparse in the descriptor space**, yet the model generalizes smoothly and performs robustly across the full variation range.\n\n> Q2. Sensitivity Analysis \n1) We have already conducted a sensitivity analysis in Appendix A.7 (Figure 8), varying the number of neighbors $k$ and the dimensionality of the parameter embedding.\n2) The results show that the performance remains stable across a range of values, demonstrating that our method is not sensitive to these choices."}, {"review_id": "yjRhPh04Qq", "reviewer": "Reviewer_aVTi", "summary": "Authors propose a new task named Continuous Domain Generalization (CDG) and a method named Neural Lie Transport Operator (NeuralLTO) to address the new task. They conduct theoretical justifications and empirical analyses.", "questions": "See weaknesses.", "limitations": "See weaknesses.", "rating": 1, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 1, "clarity": 2, "significance": 1, "originality": 3, "strengths_and_weaknesses": "Strengths: The problem is overall new, with some theoretical and empirical analyses as support. \n\nWeaknesses:\n\n- The theoretical analyses seem to be insufficient. Theorem 1 assumes that the d-dimensional descriptor provides a complete and non-degenerate representation, which makes the conclusion and proof a little trivial. Meanwhile, it could be a strong assumption that a descriptor providing a complete and non-degenerate representation is available in reality. \n- Experiments do not seem quite convincing. \n  - Improvements over baselines on most real-world datasets (FMoW, ArXiv, Traffic) are marginal, with less than 1 pp.\n  - More advanced descriptor-agnostic DG methods (i.e. standard DG methods) could be added as baselines for comparison.  For example, Fishr [1] and SWAD [2].\n  - Network backbones used in experiments (listed in Appendix A.3) are too small, making the experiments toy. In standard DG literature, ResNet-50 and ViT are commonly adopted, while only a three-layer CNN is adopted in experiments of image classification (MNIST, fMoW, YearBook). It is reasonable to use small models like MLP for tabular data like 2-Moon and Traffic since larger models usually do not work well on tabular data. However, for image tasks, using such small models does not really make sense.\n- The problem of CDG assumes access to multi-dimensional descriptors. However, such metadata of domains is often not available in real-world applications. This could severely limit the application range of the proposed problem setting and method, even making the setting a bit artificial. More realistic examples could be provided to justify the practical usage of the proposed problem setting and method.\n- The title of the paper in the openreview system is \"Continuous Domian Generalization\", which is an eye-catching typo. Such a severe typo weakens the reliability of the paper to some extent. \n\n\n\n### References\n\n[1] Rame A, Dancette C, Cord M. Fishr: Invariant gradient variances for out-of-distribution generalization[C]//International Conference on Machine Learning. PMLR, 2022: 18347-18377.\n\n[2] Cha J, Chun S, Lee K, et al. Swad: Domain generalization by seeking flat minima[J]. Advances in Neural Information Processing Systems, 2021, 34: 22405-22418.", "was_revised": true, "final_justification": "I appreciate efforts of authors made in rebuttal. However, the paper still seems to be far below the bar of the conference to me. \n\nFor theoretical analyses, even if this is not an assumption of practical descriptors, such an assumption still makes the proof of theorem too straightforward, which significantly weakens the theoretical contributions. \n\nFor experimental improvement, it is too tricky to look at relative improvement mentioned in the rebuttal. \n\nFor model backbones, I suggest all experiments should be conducted again with modern backbones like ResNet-50 or ViTs. Otherwise, such experimental results are not really persuasive nowadays. \n\nTherefore, I decide to keep my score.", "rebuttal": "Thanks for reviewing. Please find our response as follows:\n> W1. The theoretical analyses seem to be insufficient. Theorem 1 assumes that the d-dimensional descriptor provides a complete and non-degenerate representation, which makes the conclusion and proof a little trivial. Meanwhile, it could be a strong assumption that a descriptor providing a complete and non-degenerate representation is available in reality.  \nW3. The problem of CDG assumes access to multi-dimensional descriptors. However, such metadata of domains is often not available in real-world applications. This could severely limit the application range of the proposed problem setting and method, even making the setting a bit artificial. More realistic examples could be provided to justify the practical usage of the proposed problem setting and method.\n1) We would clarify that **Theorem 1 does not assume the descriptors used in practice**. Rather, it expresses that a complete, non-degenerate descriptor space theoretically exists and characterizes the model parameter structure. Such a theoretical descriptor space is grounded in reality, as distribution shifts are ultimately governed by a finite set of latent factors. \n2) Practical descriptors are imperfect. We fully acknowledge this, but  **practical descriptors serve as projections of the theoretical space**, so having partial descriptors is still better than having none. Moreover, the underlying manifold established in Section 4.1 **exists independently of practical descriptor quality**. Our method explicitly addresses the imperfect problem in Section 4.3. Experiments in real-world datasets and Section 5.3 further demonstrate that these partial descriptors are informative and effective.\n3) We will revise the statement of Theorem 1 to clarify this point and remove ambiguity.\n\n> W2.1 Improvements over baselines on most real-world datasets (FMoW, ArXiv, Traffic) are marginal, with less than 1 pp.\n\nWe respectfully clarify that in terms of relative improvement, our method achieves 12% on average over descriptor-agnostic baselines, and 9% on average over SOTA approaches. Such gains are widely regarded as significant in the domain generalization literature.\n\n> W2.2 More advanced descriptor-agnostic DG methods (i.e. standard DG methods) could be added as baselines for comparison. For example, Fishr [1] and SWAD [2].\n\nWe have added the more recent and standard DomainBed benchmark method Fishr to the experimental comparisons across all six benchmark datasets. The updated results are shown below:\n\n| Method | 2-Moons | MNIST | fMoW | ArXiv | Yearbook | Traffic |\n|--------|-------------|-----------|----------|-----------|---------------|--------------|\n| Fishr  | 34.9 ± 0.2 | 29.5 ± 1.4 | 37.5 ± 0.8 | 35.5 ± 0.1 | 7.1 ± 0.9 | 18.7 ± 0.2 |\n| **Ours**   | **3.2 ± 1.2** | **9.5 ± 1.1** | **35.9 ± 0.5** | **34.7 ± 0.4** | **4.8 ± 0.3** | **15.1 ± 0.1** |\n\nThese results confirm that our method continues to achieve state-of-the-art performance across all real-world and synthetic datasets.\n\n> W2.3 Network backbones used in experiments (listed in Appendix A.3) are too small, making the experiments toy. In standard DG literature, ResNet-50 and ViT are commonly adopted, while only a three-layer CNN is adopted in experiments of image classification (MNIST, fMoW, YearBook). It is reasonable to use small models like MLP for tabular data like 2-Moon and Traffic since larger models usually do not work well on tabular data. However, for image tasks, using such small models does not really make sense.\n\nTo address concerns regarding the backbone size, we further evaluate our method on the fMoW dataset using ResNet-50 pretrained on ImageNet as the backbone. As shown in the table below, our method continues to outperform all descriptor-agnostic and descriptor-aware baselines, validating that our framework can generalize well under more complex network configurations.\n\n|Type | Method     | fMoW (ResNet-50 as backbone) |\n|-----------------|------------|-------------------|\n| Descriptor-Agnostic | ERM        | 27.7 ± 1.6        |\n|                 | IRM        | 41.5 ± 2.8        |\n|                 | V-REx      | 32.1 ± 3.6        |\n|                 | GroupDRO   | 28.6 ± 1.9        |\n|                 | Mixup      | 27.1 ± 1.5        |\n|                 | DANN       | 26.0 ± 0.7        |\n|                 | MLDG       | 29.2 ± 1.0        |\n|                 | CDANN      | 27.6 ± 0.9        |\n|                 | URM        | 26.9 ± 1.0        |\n| Descriptor-Aware   | ERM-D      | 28.9 ± 1.8        |\n|                 | NDA        | 31.2 ± 1.4        |\n|                 | CIDA       | 27.1 ± 0.9        |\n|                 | **Ours**   | **24.5 ± 0.5**    |\n\n> W4. typo\n\nWe sincerely apologize for the typo. This error exists only in the OpenReview system and does not appear in the actual manuscript. We will correct it accordingly. We emphasize that this mistake does not affect the technical content, contributions, or reliability of the manuscript."}, {"review_id": "zdEMnOUjMG", "reviewer": "Reviewer_sRq5", "summary": "The paper introduces Continuous Domain Generalization (CDG), which assumes each domain is specified by a real-valued descriptor vector (e.g., climate, season, socioeconomic indices) rather than by a discrete label. To address the issue of time-evolving pattern in each domain, the authors proposed a Neural Lie Transport Operator to capture the complex, multi-dimensional nature of real-world variation", "questions": "- Why the existing methods only capture partial domain relationships, and they are limited in scope and fail to capture the full structure of continuous domain variation? \n- How does the model perform on larger datasets, such as domain net, which is commonly used by other DG methods.\n- How does the proposed method compare to continual learning? And why the setting of DG is needed.\n- What is the computation time if the proposed method is applied on a larger network (e.g. ResNet)?\n- The motivation for using Structured Constrained Domain Transport rather than PDE is not clear.", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 2, "originality": 3, "strengths_and_weaknesses": "- Strength\n  - Solid theoretical background\n  - Covers divers application domain\n- Weakness\n  - Benchmark dataset and backbone network is small.\n  - Motivation is not quite clear for the proposed adaptation scenario (continual DG) and the proposed transport operator\n  - Relationship to continual learning is not very clear.", "was_revised": true, "final_justification": "Most of my concerns has been addressed in the rebuttal", "rebuttal": "Thanks for reviewing. Please find our response as follows:\n> W1. Benchmark dataset and backbone network is small.\n\n**Dataset**: Our evaluation includes four large-scale real-world datasets with uncontrolled and multi-factor domain shifts. All datasets are widely recognized for distribution shift benchmark: fMoW, ArXiv, and Yearbook are from paper in NeurIPS21 DB Track [55] (citation #121), one of the most comprehensive and widely adopted evaluations for distribution shift. Traffic from paper in KDD19 [41](citation #647), and has been used in numerous urban analytics and spatiotemporal generalization studies.\n\n**Backbone**: To address concerns regarding the backbone size, we further evaluate our method on the fMoW dataset using ResNet-50 pretrained on ImageNet as the backbone. As shown in the table below, our method continues to outperform all descriptor-agnostic and descriptor-aware baselines, validating that our framework can generalize well under more complex network configurations.\n\n|Type | Method     | fMoW (ResNet-50 as backbone) |\n|-----------------|------------|-------------------|\n| Descriptor-Agnostic | ERM        | 27.7 ± 1.6        |\n|                 | IRM        | 41.5 ± 2.8        |\n|                 | V-REx      | 32.1 ± 3.6        |\n|                 | GroupDRO   | 28.6 ± 1.9        |\n|                 | Mixup      | 27.1 ± 1.5        |\n|                 | DANN       | 26.0 ± 0.7        |\n|                 | MLDG       | 29.2 ± 1.0        |\n|                 | CDANN      | 27.6 ± 0.9        |\n|                 | URM        | 26.9 ± 1.0        |\n| Descriptor-Aware   | ERM-D      | 28.9 ± 1.8        |\n|                 | NDA        | 31.2 ± 1.4        |\n|                 | CIDA       | 27.1 ± 0.9        |\n|                 | **Ours**   | **24.5 ± 0.5**    |\n\n\n> W2. Motivation is not quite clear for the proposed adaptation scenario (continual DG) \n\n1) We would like to first clarify that our work 'Continuous Domain Generalization' is not 'continual DG'. **'Continuous' and 'Continual' have different meanings in the context of DG**. The term “continuous” refers to a process that changes smoothly, while “continual” describes something that occurs repeatedly or intermittently over time.\n2) The motivation for CDG lies in that domain distribution shifts are **continuous over the underlying governed factor space** (e.g., space, time, environment), yet existing methods largely treat domains as discrete entities. Addressing this gap is both principled and practical: \n- **Principled motivation**: CDG is grounded in the **continuum hypothesis**, widely adopted in physics and engineering: every real-world macroscopic system evolves smoothly, with its quantities modeled as continuous fields governed by differential equations.\n- **Practical motivation**: In the third paragraph of Introduction, we decompose a real-world example, satellite land-use classification to illustrate how gradual environmental and developmental changes induce smooth shifts in distribution. We further present three additional realistic cases (i.e., healthcare, urban analytics, and visual recognition) to support the broad applicability of CDG. \n3) Prior DG methods ignore this continuum and treat domains as independent entities, leading to fragmented and inefficient generalization. These limitations strongly motivate our proposed formulation.\n\nWe appreciate your constructive comments and will incorporate the above content into the camera-ready version to improve clarity.\n\n> W2 & Q5. Motivation is not quite clear for the proposed transport operator. The motivation for using Structured Constrained Domain Transport rather than PDE is not clear.\n\nWe clarify that although model parameters $\\theta$ follow a PDE-like evolution as Eq. 2, CDG poses the following challenges:\n- (c1) The ground-truth $\\theta_i$ for each training domain is unobservable. They are implicit and can only be obtained through iterative optimization.\n- (c2) The training domains are sparsely and non-uniformly sampled in the descriptor space.\n- (c3) The analytical form of the distribution shift governing PDE is unknown.\n\nAs a result, existing standard PDE-solving techniques are not applicable:\n- Numerical methods require known boundary or initial values (unavailable due to c1);\n- Separation of variables methods rely on decomposing the PDE into a set of ODEs along a continuous direction (infeasible due to c2);\n- Physics-Informed Neural Networks require explicit PDE forms to define the loss (unavailable due to c3).\n\nThese limitations motivate our use of neural operators. Directly learn mappings between function spaces, which are widely studied in scientific machine learning.\n\nWe have discussed this rationale in the first paragraph of Section 4.2, and will further clarify it in the camera-ready version.\n\n> W3 & Q3. Relationship to continual learning is not very clear. How does the proposed method compare to continual learning? And why the setting of DG is needed.\n\n1) We respectfully clarify that continual learning and continuous domain generalization refer to fundamentally different problems:\n- **Continual learning** assumes a temporal task sequence, where the objective is how to learn on the current domain while retaining performance on previously learned ones, typically addressing the catastrophic forgetting problem under constraints such as no access to past data.\n- **Continuous domain generalization** addresses scenarios where domains vary continuously over a descriptor space, and the objective is to model how predictive models co-evolve as a function of distribution variation, enabling generalization to any domain in the continuum.\n2) Although both involve the word ’continue’, they differ substantially in formulation, assumptions, and objectives. \n\n> Q1. Why the existing methods only capture partial domain relationships, and they are limited in scope and fail to capture the full structure of continuous domain variation?\n\n1) As we discussed earlier, real-world processes are continuous, resulting domain distribution shifts are continuous. This implies the existence of a structural mapping: how predictive models $\\theta$ continuously evolve as a function of the distribution variation. **Capturing this globally evolving functional relationship** defines what we refer to as the full structure of continuous domain variation.  \n2) By contrast, prior methods are limited in scope:\n- Traditional DG methods ignore all domain relations but assume domains are categorical and independent. \n- The temporal DG method compresses the general multi-dimensional variation space into a one-dimensional temporal sequence. This reduction prevents it from capturing the intrinsic geometry of domain shifts.\n- Some methods leverage partial domain index relationships.  AdaGraph [37] constructs graphs over domain indices; however, graphs are discrete and non-differentiable, making them incapable of modeling a global continuous mapping over the domain space. CIDA [51] utilizes domain indices distances to model domain relationships; however, distance functions induce only a metric space, and they cannot capture the smoothness, directional derivatives, or curvature inherent in geometry.\n3) Our method, by modeling $\\theta$ as a continuous function learned via a structurally constrained neural operator, enables capturing the **full geometric structure** of domain variation beyond the limitations of discrete or partial relationship methods.\n\n> Q2. How does the model perform on larger datasets, such as domain net, which is commonly used by other DG methods.\n\nWhile DomainNet is a popular benchmark in traditional DG, it is not suitable for our problem setting due to the absence of domain descriptors. We acknowledge that handling distribution shifts without explicit descriptors is an important and practical extension. We view this as a natural future direction of our work, and have discussed it in detail in the Limitation section.\n\n> Q4. What is the computation time if the proposed method is applied on a larger network (e.g. ResNet)?\n\nWe report runtime results on fMoW using ResNet-50 (pretrained on ImageNet). Since baselines such as DRAIN, TK-Nets, and Koodos reported in Table 3 (running time analysis) are not compatible with fMoW, CIDA remains the only applicable descriptor-aware baseline.\n\n| Method | Train Time (min) | Test Time (s) |\n|--------|------------------|----------------|\n| ERM    | 17               | 6.0            |\n| CIDA   | 176              | 12.0           |\n| Ours   | 128              | 8.1            |\n\nThese results indicate that our method maintains competitive efficiency compared to existing domain relationship modeling approaches."}, {"review_id": "WndUfA6QXw", "reviewer": "Reviewer_DmZF", "summary": "This paper extends predictive modeling frameworks by introducing a generalization from a single latent variable (e.g., time) to multiple latent variables. To this end, the authors propose a neural Lie transport operator designed to ensure smooth transitions over a learned manifold, subject to certain regularity conditions. To handle imperfect input descriptions, such as those that are noisy or sparse, the method incorporates gating mechanisms and local coordinate charts to maintain robustness. Experimental results on both synthetic and real-world datasets demonstrate the effectiveness and generalizability of the proposed approach.", "questions": "Questions:\n\n1. In case of the Yearbook dataset, if the descriptor is a single-axis, which existing baselines assume, why does the proposed method perform the best even in these scenarios?\n2. It would be helpful for the authors to provide a more detailed, intrinsic interpretation of the visualizations presented in Figures 2 and 3. Beyond evaluating the quality of the decision boundary, is there any additional insight that the visualization in Figures 2 and 3 is intended to convey? For example, in Figure 3, how does the visualization by altering the direction of the moon contribute to understanding the learned manifold or the behavior of the proposed model?\n3. Does the performance enhancement difference, such as high performance gain in the synthetic dataset, while relatively low gain in the real-world dataset, relate to non-degenerate representation of the descriptions?\n4. In equation 6, how can $(z_j^k - z_i^k) \\in \\mathbb{R}^d$ and $f_v^k(z_i) \\in \\mathbb{R}^{m \\times m}$ be multiplied? Could the authors clarify about this equation?", "limitations": "yes", "rating": 4, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n\n- The paper is easy to follow.\n- The motivation for extending predictive models to incorporate multiple latent variables, rather than relying solely on time as a single axis of variation, is well-justified and compelling.\n- The performance of the synthetic dataset affects the need to incorporate multiple latent variables.\n\nWeaknesses:\n\n- Some visualizations are challenging to interpret (Figure 2).\n- The performance gain on the real-world dataset is marginal compared to the synthetic dataset.\n- Assuming non-degeneracy may be unrealistic in real-world settings, where input descriptors are frequently noisy or ambiguous.", "was_revised": true, "final_justification": "Most of my concerns have been adequately addressed in the rebuttal, and I intend to maintain my positive evaluation.", "rebuttal": "Thanks for reviewing and acknowledging our work. Please find our response as follows:\n> W1 & Q2. Some visualizations are challenging to interpret (Figure 2). It would be helpful for the authors to provide a more detailed, intrinsic interpretation of the visualizations presented in Figures 2 and 3. Beyond evaluating the quality of the decision boundary, is there any additional insight that the visualization in Figures 2 and 3 is intended to convey? For example, in Figure 3, how does the visualization by altering the direction of the moon contribute to understanding the learned manifold or the behavior of the proposed model?\n1) Figures 2 and 3 are not only intended to show decision boundary quality, but more importantly, to provide intrinsic insights into the learned manifold and **how model parameters co-evolves with domain distribution variation**:\n2) Figure 2 left provides a visualization of the domain descriptor space of the 2-Moons dataset, where the two axes represent rotation (periodicity) and scaling (monotonicity). The background colors represent the periodicity and monotonicity. The black dots indicate training domains, while the white insets show that the black dots correspond to domain data. We randomly select four test domains (marked with blue crosses) to show the prediction of the baseline methods in Figure 2, right. All baseline models generalize very poorly.\n3) In Figure 3, the left plot visualizes the learned **global parameter manifold** via PCA, showing a ‘twisted sheet-like manifold surface’ that aligns with the periodicity of rotation and the monotonicity of scaling. **The right plot shows that varying the domain descriptor corresponds to traversing on the manifold surface**. The continuous evolution of decision boundaries along single and mixed directions confirms that the learned manifold is **continuous and geometrically consistent**. In particular, the mixed-direction case highlights the model’s ability to **capture multiple types of variations jointly**.\n4) We will include this clarification in the camera-ready version.\n\n> W2 & Q3. The performance gain on the real-world dataset is marginal compared to the synthetic dataset. Does the performance enhancement difference, such as high performance gain in the synthetic dataset, while relatively low gain in the real-world dataset, relate to non-degenerate representation of the descriptions?\n1) While the performance gains are more pronounced on synthetic datasets, this does not diminish the significance of our gains on real-world benchmarks: our method achieves 12% on average over descriptor-agnostic baselines, and 9% on average over SOTA approaches. Such gains are widely regarded as significant in the domain generalization literature.\n2) Synthetic datasets are intentionally designed with clear and interpretable domain variations to rigorously evaluate a model’s mathematical behavior. Our method nearly solves these benchmarks, demonstrating strong theoretical soundness. However, directly comparing gains on synthetic and real-world datasets is unfair. While synthetic benchmarks isolate specific mathematical challenges, real-world datasets involve a coupling of high-dimensional, noisy, and entangled factors, which makes improvements inherently more difficult. \n3) Naturally, descriptor quality affects performance. Our method is explicitly designed to mitigate such imperfections (Section 4.3). As shown in Section 5.3, improving descriptor quality can further enhance model performance. We regard this as an important follow-up direction built upon the current work.\n\n> W3. Assuming non-degeneracy may be unrealistic in real-world settings, where input descriptors are frequently noisy or ambiguous.\n1) We appreciate the comment. To clarify, **Theorem 1 does not assume the descriptors used in practice**. Rather, it expresses that a non-degenerate descriptor space theoretically exists and characterizes the model parameter structure. Such a theoretical descriptor space is grounded in reality, as distribution shifts are ultimately governed by a finite set of latent factors. \n2) Practical descriptors are imperfect. We fully acknowledge this, but **practical descriptors serve as projections of the theoretical space**, so having partial descriptors is still better than having none. Moreover, the underlying manifold established in Section 4.1 **exists independently of practical descriptor quality**. Our method explicitly addresses the imperfect problem in Section 4.3. Experiments in real-world datasets and Section 5.3 further demonstrate that these partial descriptors are informative and effective.\n3) We will revise the statement of Theorem 1 to clarify this point and remove ambiguity.\n\n> Q1. In case of the Yearbook dataset, if the descriptor is a single-axis, which existing baselines assume, why does the proposed method perform the best even in these scenarios?\n\nOur method is **the only one to jointly encode both geometric continuity and algebraic consistency**, leading to a more complete modeling of the evolution structure, no matter in single-axis or multi-axis scenarios:\n1) Traditional descriptor-agnostic models treat domains as independent entities, ignoring any relational structure.\n2) CIDA utilizes distances between domain indices to model domain relationships; however, distance functions induce only a metric space, and they cannot capture the smoothness, directional derivatives, or curvature inherent in geometry.\n3) TKNets and DRAIN model temporal dependencies via sequence-based mechanisms, but lack both geometric and algebraic constraints.\n4) Koodos recognizes the continuity of distributional shifts, yet it does not incorporate algebraic structure into parameter evolution.\n\n> Q4. In equation 6, how can they be multiplied? \n\nWe confirm that Equation (6) is mathematically correct. Specifically, although $(z_j - z_i) \\in \\mathbb{R}^d $, the term $(z_j^k - z_i^k)$ is a scalar (i.e., the $k$-th coordinate difference of the $d$-dimensional vector), and $f_v^k(z_i) \\in \\mathbb{R}^{m \\times m}$, so their product is a valid scalar-matrix multiplication."}]

Author rebuttal:

Meta-review: The reviewers acknowledged the interest and novelty of the problem addressed, the elegant mathematical framework, the diverse application domains covered, and the motivation behind the approach. Nevertheless, they expressed concerns related to some aspects of the theoretical derivations, and some aspects of the empirical evaluation. The authors' feedback addressed the concerns of most of the reviewers, with the exception of Reviewer aVTi. Specifically, Reviewer aVTi expresses concerns about the relationship between Theorem 1 and practical descriptors, the performance gains reported by the authors, and the lack of experiments with modern backbones. The AC nonetheless believes that the authors' feedback satisfactorily addresses these points:
- Making assumptions in the proof of a theorem and not focusing on specific descriptors is standard, and the authors have clarified this in their answer;
- The performance improvements reported by the authors are typical of what can be found in the literature;
- Experiments with a ResNet-50 backbone were provided in the rebuttal. The AC acknowledges that this point could be further strengthened with, e.g., a ViT backbone. However, the current set of experiments already shows that the method is effective.

Altogether, the AC therefore believes that the positive aspects listed by the reviewers outweigh the negative ones, and that the paper is sufficient for acceptance. The authors are nonetheless strongly encouraged to incorporate elements of their answers in the final version of the paper.
Decision: Accept (poster)