Title: Continuous Domain Generalization
Venue: NeurIPS 2025
Accepted: yes
Keywords: Domain Generalization

Continuous Domain Generalization

Distribution Shift

Neural Operator

Lie Group
Abstract: Real-world data distributions often shift continuously across multiple latent factors such as time, geography, and socioeconomic contexts. However, existing domain generalization approaches typically treat domains as discrete or as evolving along a single axis (e.g., time). This oversimplification fails to capture the complex, multidimensional nature of real-world variation. This paper introduces the task of Continuous Domain Generalization (CDG), which aims to generalize predictive models to unseen domains defined by arbitrary combinations of continuous variations. We present a principled framework grounded in geometric and algebraic theories, showing that optimal model parameters across domains lie on a low-dimensional manifold. To model this structure, we propose a Neural Lie Transport Operator (NeuralLio), which enables structure-preserving parameter transitions by enforcing geometric continuity and algebraic consistency. To handle noisy or incomplete domain variation descriptors, we introduce a gating mechanism to suppress irrelevant dimensions and a local chart-based strategy for robust generalization. Extensive experiments on synthetic and real-world datasets, including remote sensing, scientific documents, and traffic forecasting, demonstrate that our method significantly outperforms existing baselines in both generalization accuracy and robustness.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper extends predictive modeling frameworks by introducing a generalization from a single latent variable (e.g., time) to multiple latent variables. To this end, the authors propose a neural Lie transport operator designed to ensure smooth transitions over a learned manifold, subject to certain regularity conditions. To handle imperfect input descriptions, such as those that are noisy or sparse, the method incorporates gating mechanisms and local coordinate charts to maintain robustness. Experimental results on both synthetic and real-world datasets demonstrate the effectiveness and generalizability of the proposed approach.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\n- The paper is easy to follow.\n- The motivation for extending predictive models to incorporate multiple latent variables, rather than relying solely on time as a single axis of variation, is well-justified and compelling.\n- The performance of the synthetic dataset affects the need to incorporate multiple latent variables.\n\nWeaknesses:\n\n- Some visualizations are challenging to interpret (Figure 2).\n- The performance gain on the real-world dataset is marginal compared to the synthetic dataset.\n- Assuming non-degeneracy may be unrealistic in real-world settings, where input descriptors are frequently noisy or ambiguous.\n\nQUESTIONS:\nQuestions:\n\n1. In case of the Yearbook dataset, if the descriptor is a single-axis, which existing baselines assume, why does the proposed method perform the best even in these scenarios?\n2. It would be helpful for the authors to provide a more detailed, intrinsic interpretation of the visualizations presented in Figures 2 and 3. Beyond evaluating the quality of the decision boundary, is there any additional insight that the visualization in Figures 2 and 3 is intended to convey? For example, in Figure 3, how does the visualization by altering the direction of the moon contribute to understanding the learned manifold or the behavior of the proposed model?\n3. Does the performance enhancement difference, such as high performance gain in the synthetic dataset, while relatively low gain in the real-world dataset, relate to non-degenerate representation of the descriptions?\n4. In equation 6, how can $(z_j^k - z_i^k) \\in \\mathbb{R}^d$ and $f_v^k(z_i) \\in \\mathbb{R}^{m \\times m}$ be multiplied? Could the authors clarify about this equation?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 2, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-23"}

comments: SUMMARY:
The paper introduces Continuous Domain Generalization (CDG), which assumes each domain is specified by a real-valued descriptor vector (e.g., climate, season, socioeconomic indices) rather than by a discrete label. To address the issue of time-evolving pattern in each domain, the authors proposed a Neural Lie Transport Operator to capture the complex, multi-dimensional nature of real-world variation

STRENGTHS AND WEAKNESSES:
- Strength
  - Solid theoretical background
  - Covers divers application domain
- Weakness
  - Benchmark dataset and backbone network is small.
  - Motivation is not quite clear for the proposed adaptation scenario (continual DG) and the proposed transport operator
  - Relationship to continual learning is not very clear.

QUESTIONS:
- Why the existing methods only capture partial domain relationships, and they are limited in scope and fail to capture the full structure of continuous domain variation? 
- How does the model perform on larger datasets, such as domain net, which is commonly used by other DG methods.
- How does the proposed method compare to continual learning? And why the setting of DG is needed.
- What is the computation time if the proposed method is applied on a larger network (e.g. ResNet)?
- The motivation for using Structured Constrained Domain Transport rather than PDE is not clear.

LIMITATIONS:
yes
TITLE: Official Review
CLARITY: 2
IMPACT: 2
ORIGINALITY: 3
DATE: 2025-06-30

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nAuthors propose a new task named Continuous Domain Generalization (CDG) and a method named Neural Lie Transport Operator (NeuralLTO) to address the new task. They conduct theoretical justifications and empirical analyses.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths: The problem is overall new, with some theoretical and empirical analyses as support. \n\nWeaknesses:\n\n- The theoretical analyses seem to be insufficient. Theorem 1 assumes that the d-dimensional descriptor provides a complete and non-degenerate representation, which makes the conclusion and proof a little trivial. Meanwhile, it could be a strong assumption that a descriptor providing a complete and non-degenerate representation is available in reality. \n- Experiments do not seem quite convincing. \n  - Improvements over baselines on most real-world datasets (FMoW, ArXiv, Traffic) are marginal, with less than 1 pp.\n  - More advanced descriptor-agnostic DG methods (i.e. standard DG methods) could be added as baselines for comparison.  For example, Fishr [1] and SWAD [2].\n  - Network backbones used in experiments (listed in Appendix A.3) are too small, making the experiments toy. In standard DG literature, ResNet-50 and ViT are commonly adopted, while only a three-layer CNN is adopted in experiments of image classification (MNIST, fMoW, YearBook). It is reasonable to use small models like MLP for tabular data like 2-Moon and Traffic since larger models usually do not work well on tabular data. However, for image tasks, using such small models does not really make sense.\n- The problem of CDG assumes access to multi-dimensional descriptors. However, such metadata of domains is often not available in real-world applications. This could severely limit the application range of the proposed problem setting and method, even making the setting a bit artificial. More realistic examples could be provided to justify the practical usage of the proposed problem setting and method.\n- The title of the paper in the openreview system is \"Continuous Domian Generalization\", which is an eye-catching typo. Such a severe typo weakens the reliability of the paper to some extent. \n\n\n\n### References\n\n[1] Rame A, Dancette C, Cord M. Fishr: Invariant gradient variances for out-of-distribution generalization[C]//International Conference on Machine Learning. PMLR, 2022: 18347-18377.\n\n[2] Cha J, Chun S, Lee K, et al. Swad: Domain generalization by seeking flat minima[J]. Advances in Neural Information Processing Systems, 2021, 34: 22405-22418.\n\nQUESTIONS:\nSee weaknesses.\n\nLIMITATIONS:\nSee weaknesses.", "TITLE": "Official Review", "RECOMMENDATION": 1, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 1, "ORIGINALITY": 3, "DATE": "2025-06-30"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces Continuous Domain Generalization, extending domain generalization from discrete/temporal settings to arbitrary continuous descriptor spaces. The authors propose a Neural Lie Transport Operator (NeuralLTO) that learns to transport model parameters between domains while enforcing geometric continuity and algebraic consistency. They prove that optimal parameters form a low-dimensional manifold and include mechanisms for handling noisy/incomplete descriptors.\n\nSTRENGTHS AND WEAKNESSES:\n**Strength:** \n\n1. Generalizes beyond temporal domain shifts to multi-dimensional continuous spaces\n2. Theorem 1 provides geometric justification for parameter manifold structure\n3. Lie group theory provides elegant mathematical framework\n\n\n**Weaknesses:** \n\n1. Evaluation on toy or lightly filtered datasets. Experiments focus on 2-Moons with a synthetic “rotation” descriptor and a small Climate-fMoW split . These shifts resemble data augmentation more than uncontrolled, multi-factor drift, so real-world value is unclear.\n2. In the middle of the proof of Theorem 1, the authors assume Jz θ(z) has rank d. This pre-condition is not listed up front, and real descriptors rarely satisfy it; the manifold guarantee may therefore not apply. Please put assumption Cleary visible.\n3. Definition 2 demands closure, identity, associativity and invertibility, yet the NeuralLTO implementation is a neural approximation of the exponential map. However, the paper provides no experimental validation that these group properties actually hold in practice for the learned transport operator.\n4. Each domain stores its own parameter vector θᵢ plus a private head inflating model size. Appendix A.4 claims overhead is “controlled” .\n\nQUESTIONS:\n1. How does NeuralLTO perform when P(Y∣X,z) changes drastically  (e.g., sim-to-real gaps)? Have you measured cycle-consistency error in such cases?\n\n2. The method fixes several important hyperparameters: k=5 neighbors for local charts (Equation 9) and a 32-dimensional latent space for parameter embedding. How sensitive are the results to these choices? Specifically, can you provide sensitivity analysis showing performance.\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 2, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-02"}