Title: Learning long range dependencies through time reversal symmetry breaking
Venue: NeurIPS 2025
Accepted: yes
Keywords: physical learning

continuous adjoint method

temporal credit assignment

Hamiltonian systems

analog computing

training algorithms

state space models
Abstract: Deep State Space Models (SSMs) reignite physics-grounded compute paradigms, as RNNs could natively be embodied into dynamical systems. This calls for dedicated learning algorithms obeying to core physical principles,  with efficient techniques to simulate these systems and guide their design. We propose \emph{Recurrent Hamiltonian Echo Learning} (RHEL), an algorithm which provably computes loss gradients as finite differences of physical trajectories of non-dissipative, \emph{Hamiltonian systems}. In ML terms, RHEL only requires three ``forward passes'' irrespective of model size, without explicit Jacobian computation, nor incurring any variance in the gradient estimation. Motivated by the potential to implement our algorithm in non-digital physical systems, we first 
introduce RHEL in \emph{continuous time} and demonstrate its formal equivalence with the continuous adjoint state method.
To facilitate the simulation of Hamiltonian systems trained by RHEL, we propose a \emph{discrete-time} version of RHEL which is equivalent to Backpropagation Through Time (BPTT) when applied to a class of recurrent modules which we call \emph{Hamiltonian Recurrent Units} (HRUs). This setting allows us to demonstrate the scalability of RHEL by generalizing these results to hierarchies of HRUs, which we call \emph{Hamiltonian SSMs} (HSSMs). We apply RHEL to train HSSMs with linear and nonlinear dynamics on a variety of time-series tasks ranging from mid-range to long-range classification and regression with sequence length reaching $\sim 50k$. We show that RHEL consistently matches the performance of BPTT across all models and tasks. This work opens new doors for the design of scalable, energy-efficient physical systems endowed with self-learning capabilities for sequence modelling.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper extends and generalizes Hamiltonian Echo Backprop (HEB) into the proposed Recurrent Hamiltonian Echo Learning (RHEL). The method is shown to be formally equivalent to the continuous-time adjoint state method (ASM) and Backpropagation Through Time (BPTT) in discrete time. The authors introduce Hamiltonian Recurrent Units (HRUs) and stack them into Hamiltonian State Space Models (HSSMs) to build deep sequence models that are compatible with RHEL.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n-This paper is well written with precise exposition of the RHEL method and supporting results.\n-The experiments on long-range tasks demonstrate that RHEL matches the performance of BPTT while being useful for continuous learning or alternate hardware implementations, with potential memory and computation savings.\n- The theoretical equivalences to both BPTT and ASM are well developed in the limit of small perturbations and supported with clear empirical results.\n\nWeaknesses:\n- As presented this method is only applicable to reversible Hamiltonian  systems which can be a restrictive class, and relies on symplectic \nintegration.\n- The theoretical equivalence between RHEL and BPTT and ASM is  achieved as the perturbation strength, epsilon, goes to zero. However accurate gradient computations rely on epsilon > 0, particularly in lower precision, which may result in practical implementation being sensitive. It would be useful to see some consideration of continuity / similarities for a non zero perturbation.\n-The claim of eliminating / reducing exploding and vanishing gradients would be strengthened by a supporting result. There is no \ndemonstrated improvement over BPTT (which is expected and fine) but could be the case when BPTT experience gradient maladies.\n-Some of the phrasing in the introduction and conclusion, motivating this method through connections with hardware advancements, is a \nlittle odd and does not seem to fully compliment the method begin presented.\n\nSome of the language, particularly in sentence one of the abstract, could be toned down. Other examples: \"\"This paper takes an extreme view in the world of ai hardware...\". The clarity score would be higher if these were addressed.\n\nQUESTIONS:\n- Can you discuss the computational cost of running RHEL vs BPTT in a conventional setting?\n- Is there a recommended heuristic for setting epsilon in practice?\n\nLIMITATIONS:\nyes.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces Recurrent Hamiltonian Echo Learning (RHEL), a novel and elegant algorithm for training a class of recurrent models known as Hamiltonian State Space Models (HSSMs). The core idea is to leverage the physical principle of time-reversal symmetry, which is inherent to conservative Hamiltonian systems. Instead of using backpropagation, RHEL computes gradients through a clever, physics-inspired procedure: it runs the model forward, applies a small \"nudge\" to the final state in the direction of the loss gradient, and then lets the system evolve backward in time (an \"echo\"). The deviation of this echo trajectory from the true time-reversed path provides a direct, unbiased estimate of the loss gradients.\n\nThe authors make several key contributions:\n1.  They formalize RHEL first in continuous time, proving its equivalence to the continuous adjoint state method.\n2.  They derive a practical discrete-time version for a specific class of models they call Hamiltonian Recurrent Units (HRUs), proving its equivalence to Backpropagation Through Time (BPTT).\n3.  They generalize the algorithm to deep, stacked models (HSSMs) via a procedure called RHEL chaining.\n4.  They provide compelling empirical evidence that RHEL perfectly matches the gradients computed by BPTT and achieves equivalent training performance on challenging long-range sequence modeling tasks.\n\nUltimately, the work provides a powerful new perspective on credit assignment, framing it as a physical process and paving the way for the design of novel, energy-efficient \"self-learning\" physical hardware.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths:**\n\n1.  **Exceptional Originality and Significance:** The paper's central contribution is not an incremental improvement but a paradigm-shifting idea. By elegantly connecting the mathematical formalism of backpropagation to the physical principle of time-reversal symmetry, it opens a genuinely new avenue for research in AI. The ultimate vision of building physical systems that learn *in-situ* using their own dynamics is profound and could have a major long-term impact on the field of energy-efficient AI and neuromorphic computing.\n\n2.  **High Technical Quality and Rigor:** The paper is built on a solid theoretical foundation. The authors provide clear, formal theorems (Theorems 3.1, 3.2, 3.3) that establish the equivalence of RHEL with standard gradient-based methods (Adjoint State Method and BPTT). The derivation is logically sound, progressing from the continuous-time idealization to a practical, discrete-time algorithm and finally to deep hierarchical models.\n3.  **Outstanding Clarity and Presentation:** The paper is exceptionally well-written and easy to follow. The core concept of RHEL is introduced with clear intuition, supported by excellent illustrations (Fig. 1 and 2). The logical flow from problem statement to continuous theory, discrete algorithm, and stacked models makes the complex material highly accessible. The authors do a great job of situating their work with respect to prior art like HEB while clearly articulating their novel contributions.\n\n**Weaknesses:**\n\n(These are minor and mostly represent avenues for future work, which the authors acknowledge.)\n\n1.  **Limited Model Class:** The primary limitation is that RHEL is applicable only to conservative Hamiltonian systems that exhibit time-reversal symmetry. This is a restrictive subset of dynamical systems, and many real-world processes (and potentially more expressive models) are dissipative. While the authors suggest workarounds, extending the core theory to non-conservative systems remains an open challenge.\n\n2.  **Practicality on Conventional Hardware:** On current digital hardware (GPUs/TPUs), RHEL requires three \"forward-like\" passes (one free evolution and two parallel echo passes) to compute gradients. This may not offer a significant speed or memory advantage over a standard BPTT implementation (one forward, one backward pass). The true promise of the algorithm lies in its potential implementation on novel analog or physical hardware, a point the authors rightly emphasize.\n\n3.  **Offline Nature:** The current formulation of RHEL is offline, as it requires processing the entire sequence forward before initiating the backward-in-time echo pass. This makes it unsuitable for online or real-time learning scenarios.\n\nQUESTIONS:\n1.  **Extending to Dissipative Systems:** The restriction to conservative Hamiltonian systems is the most significant limitation. Could the authors elaborate on the fundamental challenges of extending this framework to dissipative systems where time-reversal symmetry is broken? Does the echo mechanism break down completely, or could it be adapted, perhaps by having the echo system model the dissipation in reverse (i.e., by actively pumping energy back into the system)?\n2.  **Sensitivity to the Nudging Parameter `ε`:** The algorithm introduces a new hyperparameter, the nudging strength `ε`. Could you provide more intuition on its role and sensitivity? How was it selected in your experiments? Is there a principled way to set it, perhaps by adapting it based on the magnitude of the loss or the learning rate?\n3.  **Path to a \"True\" Physical Implementation:** The vision of a self-learning physical system is compelling. However, Algorithm 1 still requires explicitly computing the gradient of the Hamiltonian with respect to the parameters (`∇_θ H`), which may be non-trivial for a physical system to compute. Could you expand on the feasibility of achieving a truly \"black-box\" physical implementation, perhaps by elaborating on the \"homeostatic control\" idea mentioned in the discussion? What would this look like in practice?\n\nLIMITATIONS:\nYes. The authors have done an excellent job of discussing the limitations of their work in the discussion section. They are upfront about the restriction to non-dissipative systems, the offline nature of the algorithm, the computational cost on digital hardware, and the potential bias from the finite nudging strength. Their discussion is honest, thorough, and points clearly to important directions for future research.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 4, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 4, "DATE": "2025-07-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a novel algorithm for learning continuous-time sequence models on non-dissipative physical device (Hamiltonian systems). In addition, it introduces a discrete-time version called Hamiltonian Recurrent Units (HRUs), providing a means for simulation. Furthermore, by proposing Hamiltonian State Space Models (HSSMs), which scale and extend HRUs in a hierarchical manner, the paper demonstrates that the proposed model can be expanded into sequence models such as State Space Models (SSMs), which have recently gained attention in ML. It also shows that HSSMs can learn very long sequence tasks through simulation.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths** \n- The paper is very clearly written and rigourous proof for each theorem.\n- The paper also compare their algorithm with the previous algorithm in detail and address the important limitation of the previous algorithm.\n- This paper is an excellent contribution as it develops a real on-chip self-learning system that reduces the high energy cost of GPUs while demonstrating scalability, such as with deep SSMs.\n\nQUESTIONS:\n- In line 173, for the time derivative of $\\pi_1$ and $\\pi_4$ in Equation 8, it would be helpful to add explanations about $u$ (external force) and $y$ (target). As shown in Figure 2B, explicitly specifying the external force ($u$) and target ($y$) for the object would make it easier to understand.\n- In line 48, the text mentions three conditions: i) use “forward passes” only (i.e., no backward passes), ii) do not use explicit state-Jacobian (i.e., Jacobian of the system’s dynamics). However, only two conditions are described earlier. This seems to be a typo.\n- Regarding HSSMs with multiple layers, it is stated that activation patterns need to be stored to compute gradients. Does this mean physical implementation is impossible? Or is this limitation only applicable to the discretized time model, while the continuous-time model does not face such physical constraints?\n- The title, \"Learning long-range dependencies through time reversal symmetry breaking,\" seems misleading as there is no mention of time-reversal symmetry breaking in the main text. This could confuse readers. According to Lopez-Pastor & Marquardt (PRX 2023), \"*The self-learning device obeys time-reversal symmetry.*\" The title appears contradictory. Was the authors' intention to highlight that the system being studied has time-reversal symmetry, and that external driving forces applied when starting the backward trajectory create symmetry breaking?\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 4, "IMPACT": 4, "ORIGINALITY": 3, "DATE": "2025-07-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes Recurrent Hamiltonian Echo Learning (RHEL), a forward-only gradient-estimation algorithm for non-dissipative Hamiltonian systems. RHEL works by perturbing a time-reversed copy of the forward trajectory with a small ``nudge'', and shows that the resulting finite differences converge to the continuous adjoint-state gradients. A discrete variant is derived, and this paper shows its equivalence with BPTT. Empirical validation occurs on six UEA long-sequence classification datasets and the PPG-DaLiA regression task. HRUs trained with RHEL match BPTT (although mean performance is a little lower, this is typically not statistically significant).\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n- The method proposed offers a hardware-friendly alternative to BPTT and only uses three forward passes,\n- Theorems are also supported with proofs (some sketched in text, others in Appendix).\n- The diagrams in the paper are good, very useful for illustrating the method. \n\nWeaknesses:\n- There are no results/values of runtime, and this reduces the selling point of the paper being \"forward-only\".\n- The writing is not very clear at times and sometimes seem a bit overdramatic. For example, \"the physical realization of our algorithm\" in the abstract is not very clear. and \"this paper embraces an extreme view\" is quite dramatic. \n- The structure and value of the Supplementary materials need to be made clearer. The informal versions of the theorems are stated in the paper and the formal versions are not signposted.\n- Many systems in reality are not conservative, which may reduce the impact.\n\nQUESTIONS:\n- Why is it surprising/need to be posited that \"some SSMs could inherently be mapped onto dynamical systems\"?\n- Theorem 3.1, if you take $\\epsilon$ to 0, wouldn't both methods give the echo trajectory simply be the forward trajectory in reverse for both cases?\n\nMinor:\n- Page 2, Line 48: three -> two? I think I only saw two conditions.\n- Page 3, Line 127: let -> left\n- Page 6, Line 219: rewrite -> can be rewritten\n- Page 7, Line 221: spurious \"reading\"?\n- Page 9, Line 296: Remove \"Yet\" \n- It is a bit misleading to bold font all the averaged numbers in Table 1.\n- Recommend to use hyperref so that the refs and cross-refs are clickable.\n\nLIMITATIONS:\nYes, the authors are upfront about the limitations of the method and that it slightly underperforms BPTT.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-03"}