Title: Detecting Invariant Manifolds in ReLU-Based RNNs
Venue: NeurIPS 2025
Accepted: no
Keywords: Dynamical Systems

Recurrent Neural Networks

Attractors

Nonlinear Dynamics

Analysis
Abstract: Recurrent Neural Networks (RNNs) have found widespread applications in machine learning for time series prediction and dynamical systems reconstruction, and experienced a recent renaissance with improved training algorithms and architectural designs. Understanding why and how trained RNNs produce their behavior is important for scientific and medical applications, and explainable AI more generally. An RNN's dynamical repertoire depends on the topological and geometrical properties of its state space. Stable and unstable manifolds of periodic points play a particularly important role: They dissect a dynamical system's state space into different basins of attraction, and their intersections lead to chaotic dynamics with fractal geometry. Here we introduce a novel algorithm for detecting these manifolds, with a focus on piecewise-linear RNNs (PLRNNs) employing rectified linear units (ReLUs) as their activation function. We demonstrate how the algorithm can be used to trace the boundaries between different basins of attraction, and hence to characterize multistability, a computationally important property. We further show its utility in finding so-called homoclinic points, the intersections between stable and unstable manifolds, and thus establish the existence of chaos in PLRNNs. Finally we show for an empirical example, electrophysiological recordings from a cortical neuron, how insights into the underlying dynamics could be gained through our method.


Peer reviews:
comments: SUMMARY:
The authors present a novel algorithm for computing stable and unstable manifolds in piecewise-linear recurrent neural networks (PLRNNs) with ReLU activations. By exploiting the piecewise-linear structure of PLRNNs, the algorithm constructs stable and unstable manifolds using eigenvalues and eigenvectors. It is also capable of detecting homoclinic and heteroclinic orbits. The paper also analyzes both synthetic systems and real neural data.

STRENGTHS AND WEAKNESSES:
This is the first algorithm which can detect stable/unstable manifolds in ReLU-based RNNs. The authors conduct a comprehensive analysis with both simulated data and neural data.  However, one of the main challenges in analyzing RNNs lies in their nonlinearities, such as ‘tanh’, however, the authors only focused on Relu-based RNN (PLRNN in the paper), which are piecewise-linear and allow fixed points to be more readily identified via eigenvalues and eigenvectors.
The method assumes invertibility of the RNN map, which may not hold for all trained networks. They mitigate this via a regularization term, but it still limits generality. (As acknowledged in the paper.)
Therefore, my score will be borderline.

QUESTIONS:
1. It might be better to reorder citation in the paper.
2. How sensitive is the algorithm to slight perturbation of invertibility?
3. Could this method be extended to non-piecewise-linear activations?
4. Are there specific types of dynamics where the algorithm fails?

LIMITATIONS:
yes
TITLE: Official Review
CLARITY: 3
IMPACT: 2
ORIGINALITY: 3
DATE: 2025-06-30

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors propose a well-motivated heuristic algorithm for finding the stable and unstable manifolds of piecewise linear RNNs. Finding these is useful for illuminating the behavior of dynamical systems, as it can be used to illustrate e.g., the different basins of attractions for different choices in a decision making model. The authors also nicely illustrate the emergence of chaos without relying on mean-field theories / infinitely large networks.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**\n\nThis paper tackles an important open question in the field. The contributed method will be a useful tool to have available when reverse engineering dynamical systems. The paper is original, as there are not many alternatives for obtaining the manifolds. The empirical results are encouraging, and include a nice real world application to a single-cell recording.\n\n**Weaknesses**\n\n1. I would have liked a bit more concrete discussion on which parts of the algorithm are heuristic and which are exact. And when they are heuristic, how do potential hyper parameter choices affect the outcome, and how are they picked? I will give some concrete examples here:\n- Use of approximate algorithm for finding the fixed points: Is there any concern in case one doesn't find all fixed points?\n- Sampling seed points: how many are sampled? How are they sampled? It doesn’t seem trivial (or even always possible) to me to e.g., uniformly sample within one region of linear dynamics.\n- Propagating seed points. I think one is not always sure we reach all reachable manifolds from a given sub-region? (E.g., if we undersample seed points, or all seed points / most of the space ends up converging in one particular direction).\n- The fallback algorithm 3 still doesn’t have any guarantees of finding the right subspace, does it?\n\n2. From results shown it is unclear to me that the regularisation (Eq. 6) actually reaches the desired effect. Besides the effect on training, does it actually lead to networks that are more likely to be invertible? Is there some way to verify this, maybe by comparing the average condition numbers of the to be inverted matrices when running your manifold construction algorithm on a model trained with and without the regulariser?\n\n\nNote that while I do think some clarity on these will make the paper stronger, both the good empirical results, and the originality, make this a paper a valuable contribution!\n\n------\n\nMinor:\n- Line 45: Maybe it can be useful to name the term separatrix (formed by the stable manifolds)?\n- Line 113: In Pals et al., NeurIPS, 2024, it has been shown that the amount of regions in shPLRNNs is $\\sum_m^M {H \\choose m}$ ($\\leq$ the stated $2^H$). This means your lorenz-63 model has only ~1300 instead of over a million regions, might be useful.\n- Line 208: I think a regularisation does not *enforce* (it is a soft constraint).\n\nQUESTIONS:\nCould you explain why PCA is used? Is it simply more convenient to work with an orthonormal basis? \n\nI also don’t understand the use of kernel-PCA, could you elaborate on this? If it is just to get a real basis PCA([Re(vecs), Im(vecs)]) should be sufficient, no?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 4, "ORIGINALITY": 4, "DATE": "2025-06-30"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents a novel semi-analytical method for identifying invariant manifolds (stable/unstable manifolds of saddle points) in piecewise-linear RNNs. By exploiting the network’s piecewise-linear structure, the method computes the local manifold in the linear region of a saddle via eigen-decomposition, then extends it globally across regions by sampling and iteratively propagating points. The algorithm thus delineates basin boundaries revealing multistability and detects homoclinic/heteroclinic intersections revealing chaos, as demonstrated on toy maps, classical dynamical systems, and neural data.\n\nSTRENGTHS AND WEAKNESSES:\n# Strength:\n1. This work introduces the ​​first dedicated algorithm​​ for computing invariant manifolds in ReLU-based PLRNNs, leveraging their piecewise-linear structure to bypass traditional limitations of numerical continuation methods. By analytically deriving manifolds within linear subregions and propagating them iteratively, the approach achieves polynomial-time scalability beyond the 5D barrier of prior techniques, filling a critical gap in high-dimensional dynamical systems analysis.\n2. The study establishes rigorous mathematical foundations - formally defining invariant manifolds, basins of attraction, and homoclinic chaos - while deriving closed-form solutions for dynamics in affine subregions. The novel invertibility regularization ensures theoretical consistency with minimal computational overhead, demonstrating principled innovation throughout.\n3. Validated across synthetic and real-world benchmarks, the algorithm robustly handles high-dimensional models with sublinear scaling in subregion sampling. Its efficiency enables previously infeasible analyses, such as identifying attractor basins in decision-making tasks or chaos signatures in electrophysiological data, bridging theoretical insights with empirical applications.\n4. Unifying traditionally disjoint analyses, the framework simultaneously delineates basins of attraction, detects chaotic regimes via homoclinic intersections, and extracts mechanistic principles. This integration provides a versatile toolkit for dissecting RNN dynamics, from memory states to edge-of-chaos computational properties.\n\nWeakness:\n1. Medical applications require guarantees that chaotic dynamics won't amplify adversarial perturbations, yet no experiments probe this vulnerability. The discussion in Sec. Limitations ignore adversarial threats in high-stakes settings, focusing solely on invertibility constraints.\n2. The paper exhibits ​​significant bias in application scope validation​​, disproportionately focusing on cortical neuron dynamics and classical physical systems, while neglecting critical and general domains explicitly mentioned in the introduction. Claimed applications in \"time series prediction\" in the abstract remain untested for ​​sensor networks or fault detection​​.\n3. The algorithm's theoretical O(2^P) complexity risks intractability in high-dimensional industrial systems. Validation remains narrowly focused on low-dimensional models and neuroscience tasks, omitting critical industrial scenarios. It lacks the stress-testing on high-noise/long-horizon data and real-time control benchmarks.\n\nQUESTIONS:\nPlease refer to the above-mentioned weaknesses.\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 1, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-03"}

comments: SUMMARY:
This paper introduced an algorithm to detect stable and unstable manifolds in ReLU-based piecewise-linear RNNs. By exploiting the network's piecewise-linear structure, the method iteratively constructs global invariant manifolds via local eigenvectors and adaptive sampling, tracing boundaries between basins of attraction. The authors also introduced an invertibility regularization to ensure the RNN map remains invertible, enabling robust manifold tracing even in high dimensions. Demonstrations on synthetic and real neural data show the method’s utility for characterizing multistability, detecting chaos, etc.

STRENGTHS AND WEAKNESSES:
**Strengths**
- The authors provide comprehensive mathematical background, detailed derivations, and clear pseudocode, enabling full understanding and reproducibility of their approach.
-  Empirical Validation and Utility: The method is validated on both synthetic dynamical systems and real neural data,  providing mechanistic insight into complex dynamics.

**Weakness**
- The approach is specifically designed for ReLU-based, piecewise-linear RNNs and may not generalize to RNNs with smooth or other types of nonlinear activations, limiting its broader applicability.

QUESTIONS:
As mentioned in the weaknesses, your algorithm relies on the RNN being piecewise linear in order to define and track linear subregions. Do you see any possible ways to approximate or extend your approach so that it could be applied to more general RNNs with smooth or non-piecewise-linear activation functions?

LIMITATIONS:
Yes
TITLE: Official Review
CLARITY: 3
IMPACT: 3
ORIGINALITY: 4
DATE: 2025-07-03