Venue: NeurIPS 2021 Conference

Reviews:

final_score: rating: 5: Marginally below the acceptance threshold
review_content: summary: The paper presents a method to learn the dynamics of stable systems. The authors first exploit Neural Delay Differential Equations to model the partial observability and time delay of the system. They then employ Lyapunov-Razumikhin Function to ensure the stability of the learned dynamics. The authors evaluate their methods by presenting the sampled stable trajectories and loss plots of the learned model against other selected models.
main_review: [Strength] The paper has the following strength.
- The paper studies the modeling of the interesting type of time-delayed system and extends to partially observed stable systems. 
- The proposed method is learning stable dynamics based on sampled trajectories in a data driven way. 
- The method can be combined with policy learning to jointly learn a stabilizing policy and the dynamics of the closed loop system.

[Weakness] However, It has the following weakness about the paper which has stopped me from recommending the paper for publication in NeurIPS.
- The paper discusses little and conducts no comparison against the line of work from recurrent neural networks (RNNs) based effort on modeling system dynamics. Even though RNNs may lack the capability of modeling continuous dynamics, I think a more thorough discussion and empirical experiments here would increase the completeness of the work.
- The model introduces a few parameters to tune: K, samples of historical observations, r, the length of history to consider, and alpha, q for the exponential stability of the learned system. The parameters are highly tailored with the system dynamics to learn. That is, you would not be able to learn a good model for a (internally) higher dimensional partially observed systems with a small K (this is mentioned by authors as there would not be a one-to-one mapping from internal states to the coordinates). I am expecting a discussion on choosing the parameters and a demonstration/analysis of performance of the method against these parameters.
- The method bases on optimizing a loss function where a loss term is introduced to encourage stability. This only indirectly ensures stability. I am concerned that unseen observations could lead to non-stabilizing trajectories. If any theoretical guarantee could be established on this, it would increase the theoretical contribution of the work.
- The experiments has only compared the method against ANODE where ANODE's failure is shown through non-matching phase plots, but the success of the proposed method is only shown through lower loss. This alone is insufficient in showing a better performance of the proposed method as the worse performing ANODE with learned IC achieves lower loss than ANODE with true IC. More experiments could help make the point clearer.
- The method ports the established method of Neural Delay Differential Equations to the dynamics learning problem. The theoretical analysis of the method bases on established results clearly given in other papers -- The only theorem in this paper is given by (Efimov and Aleksandrov, 2020). The experiments have not shown any special benefits of the method against existing methods. Therefore, I am concerned that the novelty of the work is not up to the standard of NeurIPS.

_________________________________
Post rebuttal:
I have read the responses from the authors and appreciate the author's responses to my questions. However, I would like to stick with my original reviews and scoring.
limitations_and_societal_impact: See main review.
review_title: null

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: This paper aims to learn stable deep dynamics models for partially observed or delayed dynamical systems. The authors build their method upon Neural Delay Differential Equations (NDDEs) [1], where the main idea is to augment the state with the past history, thus avoiding the estimation of unobserved augmented states. To stabilize the learned dynamics, the authors then propose to combine NDDEs with a Lyapunov-like regularization term.

The authors show that NDDEs are powerful tools to model partially observed dynamical systems where the dynamics are non-Markovian. Furthermore, they demonstrate that the proposed method outperforms Augmented Neural ODEs (ANODE) in partially observed systems and is able to learn a stabilizing feedback policy for control systems with input delays.

[1] Qunxi Zhu, Yao Guo, Wei Lin, "Neural Delay Differential Equations"
main_review: [Strength]

This paper targets an important research question of learning stable deep dynamics models for partially observable scenarios. Deep neural networks are powerful tools to fit the observation data, yet additional treatments have to be in place to ensure that the learned model satisfies certain properties like stability, energy preservation, etc.

This paper is on the line of adding structures to the learning procedure based on our understanding of the dynamical systems. The authors take a step forward by extending to partially observed or delayed systems, which I believe has the potential to apply to safety-critical systems where it is crucial to have guarantees on convergence.

The authors provide theoretical analysis and convergence guarantees on the proposed method. 


[Weakness]

My primary concern of this paper is its novelty. This work builds on top of Neural Delay Differential Equations (NDDE) [1] by jointly learning a neural network Lyapunov-Razumikhin function. However, similar techniques for stabilizing dynamical systems with neural Lyapunov functions have already been investigated in [2, 3] and many others. Thus, to me, the proposed method seems to be a direct combination of existing methods, making this paper a bit insufficient in technical novelty.

My second concern is the lack of evaluation tasks. For evaluation, the authors only consider a simple hand-designed system, a partially observed damped double pendulum, and a friction-less inverted pendulum. However, for papers that line the similar direction like [3], they considered much more challenging cases, including wheeled vehicles and humanoid balancing. In [4], they investigated the n-link pendulum and high-dimensional systems like video texture generation. The current set of evaluating environments may be a bit too simple, limiting the significance and potential impact of this paper.


[1] Qunxi Zhu, Yao Guo, Wei Lin, "Neural Delay Differential Equations"

[2] Spencer M. Richards, Felix Berkenkamp, Andreas Krause, "The Lyapunov Neural Network: Adaptive Stability Certification for Safe Learning of Dynamical Systems"

[3] Ya-Chien Chang, Nima Roohi, Sicun Gao, "Neural Lyapunov Control"

[4] Gaurav Manek, J. Zico Kolter, "Learning Stable Deep Dynamics Models"


================================

Post-rebuttal comments

I have read the reviews from other reviewers and the rebuttal from the authors. I appreciate the authors' efforts in providing the additional experimental results and the detailed discussions about the novelty of the proposed method and the comparison with previous methods. I encourage the authors to include them in the revised version of the paper.

The rebuttal has sufficiently addressed my concerns, and I raised my score from 5 (Marginally below the acceptance threshold) to 6 (Marginally above the acceptance threshold).
limitations_and_societal_impact: Please see my main review for the limitations, and I do not see any obvious potential negative societal impact of this work.
review_title: null

final_score: rating: 7: Good paper, accept
review_content: summary: This paper extends prior work on stable deep neural network dynamics models to learn stable models with partially observed states, and to learn (simple) control policies with time-delayed feedback. To achieve this, the authors minimize a Lyapunov-Razumikhin loss term when training _neural delay differential equations_. They demonstrate the capabilities of their system on a few tasks.

main_review: The work is the novel application of Lyapunov-Razumikhin stability to Neural DDEs. Applicable related work is cited.

## Quality

The derivation of the Lyapunov-Razumikhin loss term proceeds methodically from the background section. The special care taken with the Lyapunov NN construction and the LRF loss discretization is especially appreciated.

The authors are forthright in their evaluation of their own work: they clearly highlight potential pitfalls (such as how the stability certification is for each trajectory and not global, etc.) 

### Experiments

Their choice of a two-link pendulum with partially-observed states is a reasonable evaluation task to highlight how the LRF loss prevents blow-up. This is only a single evaluation, though, and it is not obvious to me that ANODE (at least, when used with true initial conditions) will fail at solving the two-link pendulum.

Having more experiments would make this much better. In particular, if they can find a condition in which ANODE/vanilla NDDEs fail to learn but stable NDDEs succeed, this would further support their claim on LRF acting to regularize training (line 294). One way to do this would be to try this on three- or more-link pendulums as well.

The delayed feedback control experiment is well-selected.

The experiment tasks are all relatively simple and small. The paper would be much better served by presenting either a more complex “showcase” task, or a comparison to a common benchmark.

## Clarity

The submission is well-organized and mostly clear. Some key experimental details are in the appendix, but given that the focus of this paper is on the methodology, this is reasonable. (For the delayed feedback control experiment, I suggest mentioning the selection of $\tau=0.03$ in the experimental section itself.)

There is a little ambiguity in the creation of the initial history: in particular, lines 248-254, I’m not sure which data is used to train/fit the GP-mean, and if this is replaced/appended to at each iteration.

As this is a stability-by-loss-function approach, there is also some concern on how well the stability generalizes. (Answering the question “how likely are other trajectories produced by the model to be stable?”) Presenting some experiment quantifying this would alleviate this concern.

## Significance
This work joins a body of previous work on stable neural-network dynamics models using loss terms. Its ability to deal with partially-observable states and delayed control is somewhat interesting. The experimental results don’t showcase what the method could do in practice.

To make the results compelling depends on what this method can do in practice. Perhaps this could be showcased by performance on a more complicated common control-theory benchmark task (like a delayed or partially-observed cartpole?) 

(Minor formatting issue: please consider allowing Latex to position figures instead of embedding them in text flow, It improves readability.) 


limitations_and_societal_impact: The paper's technical limitations have been addressed in the review. This is basic research, and has no direct negative societal impact.
review_title: null

Meta-review: Thank you for your submission to NeurIPS.  Although there is still some disagreement between reviewers after the rebuttal period, overall the proposed paper seems to be a solid contribution to the emerging field of modeling continuous-time dynamical systems with delays and/or partial observability.  The work is perhaps a bit incremental in that it is largely a combination of past approaches, including past work in Neural DDEs and past work on ensuring the stability of dynamical systems specified by Lyapunov functions.  But it also combines these past methods in new and rather interesting ways, and I believe is worthy of highlighting at NeurIPS.  Thus, while the paper is somewhat borderline, I am recommending it be accepted, with the strong comment that the authors address the reviewer concerns that _can_ be addressed (i.e., not just the general concerns about significance) as they did in the rebuttal.

Decision: Accept (Poster)