Title: Efficient Parametric SVD of Koopman Operator for Stochastic Dynamical Systems
Venue: NeurIPS 2025
Accepted: yes
Keywords: Koopman operator

spectral decomposition

singular value decomposition

neural networks

low-rank approximation

DMD

VAMP

DPNets
Abstract: The Koopman operator provides a principled framework for analyzing nonlinear dynamical systems through linear operator theory. Recent advances in dynamic mode decomposition (DMD) have shown that trajectory data can be used to identify dominant modes of a system in a data-driven manner. Building on this idea, deep learning methods such as VAMPnet and DPNet have been proposed to learn the leading singular subspaces of the Koopman operator. 
However, these methods require backpropagation through potentially numerically unstable operations on empirical second moment matrices, such as singular value decomposition and matrix inversion, during objective computation, which can introduce biased gradient estimates and hinder scalability to large systems.
In this work, we propose a scalable and conceptually simple method for learning the top-$k$ singular functions of the Koopman operator for stochastic dynamical systems based on the idea of low-rank approximation. Our approach eliminates the need for unstable linear-algebraic operations and integrates easily into modern deep learning pipelines. Empirical results demonstrate that the learned singular subspaces are both reliable and effective for downstream tasks such as eigen-analysis and multi-step prediction.


Peer reviews:
comments: SUMMARY:
The authors propose a row rank approximation method to approximate Koopman operators. The simplicity of the algorithm makes it easy for the method to integrate with modern deep learning pipelInes by eliminating the need for unstable linear
algebraic operations.

STRENGTHS AND WEAKNESSES:
Finding good low-dimensional space for representing Koopman operators is a nonobvious and important problem. This paper try to tackle this problem, and the topic is related to the community.

There are existing methods that aim to learn low-dimensional spaces for representing Koopman operators. The advantage of learning a singular space is not clear for me. Could you clarify the advantage of learning a singular space instead of learning general linear spaces for low-dimensional approximation of Koopman operators?

Minor comments:  
- Line 49: what do you mean by the identity $x_0=x_0$?
- Line 102: it seems that the limit is taken in the topology of the function space where the Koopman operator is defined, not the strong operator topology for operators.
- Line 115: "for even for " should be "even for"?

QUESTIONS:
The singular vectors should be orthogonal each other. How do you guarantee the orthogonality of the vectors $f_1,\ldots, f_k$?

LIMITATIONS:
Investigating what type of neural networks can be used for constructing $\mathbf{f}$ and $\mathbf{g}$ would be interesting.
TITLE: Official Review
CLARITY: 2
IMPACT: 3
ORIGINALITY: 3
DATE: 2025-06-17

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis manuscript presents a novel method, termed LoRA, for learning the top-k singular functions of the Koopman operator for stochastic dynamical systems. The authors identify a critical issue in existing deep learning-based approaches like VAMPnet and DPNet: their reliance on numerically unstable operations such as matrix inversion and singular value decomposition within the training loop, which can lead to biased gradients and optimization challenges. To address this, the proposed LoRA method is based on directly minimizing the Hilbert-Schmidt norm of the low-rank approximation error. This formulation results in a loss function that is a simple polynomial of empirical second-moment matrices, thereby avoiding unstable computations during optimization. The method's efficacy and stability are demonstrated through a series of experiments on synthetic and molecular dynamics datasets, where it is shown to outperform baseline methods.\n\nSTRENGTHS AND WEAKNESSES:\nThe primary strength of this work lies in its clear identification of a key technical bottleneck in existing Koopman analysis methods—namely, numerical instability during optimization—and its proposal of a mathematically rigorous and algorithmically elegant solution. The LoRA objective function is well-motivated, and its stable polynomial form represents a significant advantage, facilitating straightforward implementation and optimization within modern deep learning frameworks. The derivation provided in the appendix confirms its theoretical soundness. The experimental results, particularly the successful application to the complex Chignolin molecular dynamics system where baseline methods reportedly failed to converge, compellingly demonstrate the superior stability and practical utility of the proposed algorithm.\n\nHowever, the rigor and persuasiveness of the work are undermined by several significant flaws in its reporting and execution. Firstly, the reporting on the crucial Chignolin experiment contains multiple errors in the main manuscript, including incorrect units, data splits, and table labels, which, while noted in an appendix errata, reflect a lack of rigor in the report itself. Secondly, the work’s central conclusion relies heavily on a post-hoc physical justification in Appendix G.4.3 to bridge the gap between their experimental data and a reference value from a different temperature; this justification is qualitative and lacks quantitative support, making it appear speculative. Most critically, the authors admit in Appendix G.4.4 that their choice of a short lag time \"inherently biases the model towards resolving faster dynamic events,\" which is a concession of a systemic flaw in the experimental design that makes it unsuitable for validating the theoretical goal of finding the system's slowest physical processes.\n\nQUESTIONS:\nBased on the analysis, we pose the following questions to the authors. First, in Theorem C.3 of Appendix C.2, there is a clear discrepancy. The theorem states that for a left eigenfunction $\\zeta$, the relationship is $\\mathcal{K}^{*}\\zeta=\\lambda\\zeta$. However, the final step of the proof on line 832 derives the result as $\\overline{\\lambda}\\zeta(x')$, where $\\overline{\\lambda}$ is the complex conjugate of $\\lambda$. This conclusion from the proof, $\\mathcal{K}^{*}\\zeta=\\overline{\\lambda}\\zeta$, is mathematically correct for adjoint operators but contradicts the theorem's statement. We ask the authors to clarify and correct this imprecision.\n\nSecond, the authors acknowledge in Appendix G.4.4 that the short lag time used in the Chignolin analysis \"inherently biases the model towards resolving faster dynamic events.\" This raises a critical question about the method's general applicability: for a new system, how should a practitioner select an appropriate lag time to ensure the discovery of the most critical, slow dynamics, rather than being misled by parameter choices into focusing on faster, less relevant modes? Does this imply that the successful application of LoRA requires extensive prior knowledge or tedious sensitivity analyses?\n\nThird, while the stability of LoRA is well-demonstrated, the experimental validation relies significantly on systems that could be considered \"toy\" or methodologically \"tricky.\" The Noisy Logistic Map, for instance, is a 1D system chosen specifically because its finite-rank kernel is analytically tractable. Similarly, the Ordered MNIST experiment uses a synthetic process with simple, deterministic dynamics layered on top of image data. While valuable for controlled comparisons, how confident are the authors that the strong performance on these simplified or contrived benchmarks will translate to more complex, high-dimensional, and chaotic real-world systems where the underlying dynamics are not low-rank or neatly structured?\n\nLIMITATIONS:\nThe most fundamental limitation of this research is that, as a data-driven method, the quality of the learned dynamical model is entirely beholden to the quality and characteristics of the input data. The Chignolin experiment serves as a primary example: if the available data, due to factors like trajectory length or sampling frequency, does not sufficiently contain the signal of the slowest physical processes (e.g., microsecond-scale global folding), the method cannot recover them, regardless of its mathematical optimality or algorithmic stability. Therefore, the method cannot guarantee the discovery of the absolute slowest physical timescale in all cases; rather, it finds the most prominent observable slow process within the given data. This data-dependency defines the principal boundary for the method's applicability when applied to new systems.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-30"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis work studies a deep-learning–based framework for modeling dynamical systems by approximating the Koopman operator through a low-rank singular-value decomposition. Its principal contributions are evaluation of the Low-Rank (LoRa) contrastive loss in the context of the (stochastic) Koopman operator approximation, systematic comparisons with existing related methods, careful elaboration  of implementation details, and  and extensive experiments.\n\nThe chief advantage of LoRa is that its computation avoids the inversion of latent-space covariance matrices—objects that are often ill-conditioned during training. By avoiding these unstable inverses, LoRa produces unbiased estimates of the loss and its gradients, delivering more reliable optimization and superior performance compared with previous state-of-the-art losses.\n\nSTRENGTHS AND WEAKNESSES:\n### __Strengths__\n\n- The paper provides a solid review of related deep-learning approaches for stochastic Koopman operators and provides their comparisons.\n- The paper is self contained, well written and accessible.\n- The variants of the proposed algorithm appear to consistently beat state-of-the-art baselines.\n\n### __Weakenesess__\n\n- __Novelty:__ Although the paper cites [29] and [15], Section 3.1 should explicitly acknowledge that the LoRa loss was *first* introduced for low-rank conditional-expectation operator approximation in those works, while paper [14] proposed truncated SVD as the approach for representation learning for stochastic processes estimation. Additionally, I strongly suggest to place LoRa in the wider context of self-supervised contrastive learning and density ratio fitting by citing \n    \n    • Wang *et al.* (2022), “Spectral Representation Learning for Conditional Moment Models”\n    \n    • Sugiyama *et al.* (2012), “Density Ratio Estimation in Machine Learning”\n    \n- __Inconclusive empirical results:__ While the paper promises an experimental evaluation of design choices LoRa based low-rank approximation of the Koopman operator, many results appear inconclusive:\n    - **Left vs. right singular functions**\n    Figure 1 shows no clear performance gap, and Figure 2 reports no statistically significant difference. These results could be summarized in prose, omitting the corresponding curves to reduce visual clutter.\n        \n        The claim\n        \n        > “...we empirically found that using the right singular basis can sometimes yield better results.”\n        > \n        \n        is not convincingly supported by the current data.\n        \n        Since theory predicts both subspaces converge to the same Koopman-invariant space once training stabilizes, this design choice may not warrant prominent discussion; consider moving it to the appendix.\n        \n    - **Joint vs. sequential nesting**\n    The implementation of sequential nesting is unclear, given that all left/right functions are parameterized by the same encoder / delayed-encoder. Please clarify the procedure. Furthermore in the joint nesting, it is unclear how the weights are chosen given that the during learning the left-and-right data representations are not ordered by relevance of the singular space. It seems that if the nesting technique is going to be one of the main contributions of the paper, it warrants both better introduction and more conclusive results.\n\n    - **Hyper-parameter study**\n        In the synthetic experiment, Figure 1 shows no discernible trend for learning rate or batch size. It is hard for the reader to draw any conclusion from these plots, beyond the better performance of LoRa against baselines. Either try to pursue a deeper ablation study with more conclusive results and trends, or simplify this plot to avoid the clutter.  \n  \n- __Minor issues:__  Font sizes in most plots are too small, compromising readability. Use the available whitespace to enlarge labels and legends. The x-axis of Figure 3 is unlabeled; specify the quantity and units. The paper does not report the scalar weighting the DPNet regularization term, nor whether this parameter was tuned.\n\n\n__In summary, I believe that the paper is solid contribution with interesting empirical results, but my concern lies in clear and sufficient novelty w.r.t. the existing literature.__\n\nQUESTIONS:\n- **Figure 1 (top row):** Why does the singular-value error *increase* as the number of singular functions grows? Intuitively, adding more modes should *reduce* the approximation error. Please clarify the underlying reason.\n- **Left vs. right singular spaces:** The paper compares the span of the left-right-singular-functions on downstream inference tasks, yet theory predicts that both should converge to the same Koopman-invariant subspace. What practical insight is gained from this comparison?.\n- **Scope of the empirical study:** The work largely integrates results from prior literature and promises a thorough ablation of LoRa-based Koopman approximation. In practice, the only clear conclusion is LoRa’s superiority over DPNet and VampNet losses. To strengthen the empirical contribution, consider adding experiments that yield actionable insights on:\n    - **Latent dimensionality:** How do all models scale as the latent space grows? We know that large latent spaces will lead to unstable DPNet and Vamp scores. Is LoRa based optimization unaffected by this?\n    - **Orthonormal regularization:** Is orthonormal regularization influencial for numerical stability and/or convergence speed of the training process?\n    - **Batch-size sensitivity:** How does batch size affect optimization stability of these methods?\n\nLIMITATIONS:\nSection limitation is present, but the discussion is quite superficial and not discussing limitations per se, but rather plans for future work.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 5, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 1, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces a scalable and stable method for learning the top-k singular subspaces of the Koopman operator in stochastic dynamical systems using LoRA. Unlike prior methods such as VAMPnet and DPNet, which rely on backpropagation through numerically unstable operations like SVD or matrix inversion, LoRA formulates a variational objective entirely in terms of polynomial forms of moment matrices, enabling unbiased gradient estimation and efficient training using minibatches. The proposed method is validated on multiple benchmark systems, including synthetic chaotic maps, Langevin dynamics, molecular simulations (Chignolin), and Ordered MNIST, demonstrating superior performance in general.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\n- The paper is clearly structured, with a coherent flow from motivation, problem formulation, method, theory, and experiments. \n- I’m not much familiar with the field, but the proposed approach’s motivation is convincing. e.g., challenges on VAMPnet and DPnet (e.g., unstable backpropagation, scalability) are clearly discussed.\n- Notation is mostly consistent, and figures are helpful for understanding convergence and performance trends.\n\nWeaknesses and Questions\n- While LoRA seems not been applied to dynamic mode decomposition, it looks like a simple integration of existing LoRA method to existing problem to a certain extent. \n- While the theoretical background and motivation are sound, some of the proof sketches and intuition for why LoRA performs better (especially under non-normal dynamics) could be clarified in the main text. Please correct me if it was presented in section 3.\n- I agree that VAMPnet and DPnet would be more difficult to optimize. However, I'm not sure the experimental results effectively exhibit such limitations. In terms of rel. error in Figure 1, LoRA achieves the lowest error comparing others, but it’s not always true when evaluating Hausdorff (the lowest distance achieved by DPNet).\n- Nesting (seq or jnt) results do not seem consistent. For example, LoRA_jnt is superior in Figure 2, but LoRA_seq is better in Table 1. \n- Also, I expected results showing LoRA is significantly faster and scalable than VAMPnet and DPnet, but such experiments seem not considered.\n\nAs I’m only little familiar with the field, I’ll have to see other reviewers score to finalize mine.\n\nQUESTIONS:\nN/A\n\nLIMITATIONS:\nN/A", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 2, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-07"}