Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: In this paper, the authors studied the stabilization problem of the systems described by stochastic delay-differential equations, and the main contribution is the designed framework of neural deterministic and stochastic control with the analysis of stability and safety. Experiment results are given to compare the performance of  the proposed NSC/NDC method and baseline methods.
strength_and_weaknesses: Strength:

[1] The problem of control of nonlinear stochastic delay-differential equations is very interesting and also extremely challenging

[2] The proposed methodology and main results were presented/explained reasonably clear, and overall speaking I found the manuscript friendly to readers.

[3] The proposed framework is novel, and the provided case studies are helpful to demonstrate the advantages and efficacy of the proposed neural-based method in comparison with the typical/traditional existing control methods.

Weakness:

[1] It was claimed in the paper abstract that the proposed NSC and NDC methods "guaranteeing the stochastic stability in SDDEs". However, based on my understanding, such proposed methods for training neural network Lyapunov functions for control synthesis of nonlinear systems are heuristic in nature, and extremely hard to obtain rigorous stability or safety guarantees for the closed-loop system. For the main theoretical results in Theorem 5.1, it was mentioned that "Assume the controlled system satisfies the similar conditions in Theorem 2.2 and Remark 3.1 with Ker(w1 − w2) = 0", but it was not clear to me how the authors guarantee that the closed-loop control system would indeed satisfy all the conditions in Theorem 2.2. Similar for other main theoretical results (e.g., Theorem 5.2). Simply adding stability conditions and safety conditions to the loss function of the NNs is not sufficient to guarantee the closed-loop controlled system would be stabilized, since there are a lot of different things affect the NN training and the NN training might even possibly diverge, and there is no much rigorous theoretical guarantee for such NN method. So I'm not convinced by the authors' claim about the proposed methods  "guaranteeing the stochastic stability in SDDEs" from the current version manuscript. More clarifications/elaborations from the authors are required to justify such claims. This is the biggest limitation/concern in my mind for the current version manuscript.

[2] It is not comprehensively discussed/explained when to use NSC and when to NDC. For the specific case study in Section 4.1, I saw a brief statement "We can see that the ranking of the comprehensive performance is that NSC is greater than NDC than QP". But is such conclusion generic for all cases, or it is just an observation for this special case study? I.e., is that using NSC rather than NDC always the recommended method by the authors (when there are randomness there), and what are the pros and cons? It would be helpful for the authors to elaborate more and share more comprehensive discussions for generic cases.

clarity,_quality,_novelty_and_reproducibility: The paper is written reasonably clear, and it contains some valuable novel results. But I have some major concerns about the authors' claim regarding theoretical stability guarantee of the closed-loop system using their proposed method, and I think that's a over-claim.


summary_of_the_review: This paper proposed a novel framework for neural deterministic and stochastic control with the analysis of stability and safety, for systems described by stochastic delay-differential equations. The empirical cases studies are helpful to demonstrate the effectiveness of the proposed method. 

However, my main concern about the current version manuscript is that, I'm not convinced by the authors' claim that the proposed NSC and NDC methods "guaranteeing the stochastic stability in SDDEs", and I think the proposed methods are heuristic in nature and hard to have rigorous stability or safety guarantees for the closed-loop system. Some more clarifications/elaborations about the theoretical theorems (5.1, 5.2, etc.) for the closed-loop controlled systems would be needed to justify the authors' claim about the theoretical stability guarantee of the closed-loop system.

If the authors can't rigorously prove theoretical stability guarantee of the closed-loop system, the proposed method and the empirical case studies are still somewhat valuable, though the contribution of the paper would be less significant.  In that case, it would be a boarder-line paper (slightly below acceptance threshold), and at the very least, the over-claim about the theoretical stability guarantee needs to be corrected/clarified.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper provides a safety-aware control approach, based on neural networks, for stabilizing stochastic delay differential equations (SDDEs). In particular, the authors construct a controller with two components - a deterministic control $u_f$ tied to the time-evolution of the system, and a stochastic control $u_g$ tied to the Brownian motion of the system. The authors both build off existing theorems and derive new theorems to characterize conditions under which the controllers will stabilize the system. They then construct their neural networks to satisfy some of the assumptions within these theorems, as well as constructing loss functions that incentivize satisfaction of the rest. The authors demonstrate the superiority of their method compared to baselines on small-scale experiments. 
strength_and_weaknesses: Strengths: 
* The amount of theoretical and empirical work represented within this submission is seemingly immense. The authors derive new stability theorems, and structure neural network controllers that play well with the assumptions associated with both existing theorems and these new theorems. The authors also run numerous experiments to demonstrate the efficacy of their framework.

Weaknesses:
* While the authors claim they obtain a safety guarantee through their neural network control framework, as far as I can tell, this is not the case. In particular, while some assumptions within the theorems they build upon are satisfied through the structure of their neural network framework, other assumptions and preconditions are only _incentivized_ (not guaranteed) to be satisfied through the design of loss functions. As such, I don't think the authors can actually claim that their controller "guarantees" safety.
* In part because of the immense quantity of the work in the submission, the paper as written is rather inaccessible for an ICLR audience and is not self-contained. In particular, the authors could spend much more time describing the intuition behind their approach, providing algorithm diagrams and flow charts, etc. in the main text to really impart the big ideas; right now, the main body of the paper is hard to follow without this clearly articulated conceptual framework. The experimental results are also not self-contained within the main paper. 
* The mathematical notation is not adequately defined, given that some of it is not standard or usual for the ICLR audience.

Questions:
* Is the problem statement, and particularly the formulation in Equation (4), standard - or is it fully newly proposed by the authors? If the latter, what are the strengths and limitations of the proposed formulation itself (in addition to the solution technique)?
clarity,_quality,_novelty_and_reproducibility: The work seems novel (though this paper is outside my area, so I'm not able to fully evaluate this). The quality seems reasonable except for the over-claim of guaranteed stability, and my uncertainty regarding whether the problem formulation itself is standard or novel (if the latter, it needs to be further justified). The authors provide code to improve reproducibility, and additional details are given in the appendix. The clarity could be significantly improved.
summary_of_the_review: The authors provide a seemingly novel, interesting framework for constructing neural network controllers for SDDEs. However, the claim that this framework "guarantees" safety is seemingly an overclaim, and the submission needs to be significantly improved in terms of its clarity and accessibility for the ICLR audience. In addition, the authors should further clarify where their problem formulation in Equation (4) comes from, and further justify it if it is a novel problem formulation.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: The paper proposed a method for generating stable and safe policies for stochastic delay-differential equations. Specifically, the paper achieves this by learning a LaSalle's type stability certificate as an MLP and extending it to a stochastic control barrier function.
strength_and_weaknesses: # Strengths:
- The paper relies on minimal assumptions (twice continuously differentiable systems and policies).
- The paper introduces a method for an important problem (stable and safe control in stochastic systems)

# Weaknesses:
Generally, I feel the writing of the paper could be a bit clearer. For instance, the function in Eq (5) looks quite similar to the neural network learned in Chang et al. 2019.
Moreover, the failure case of the NDC is not entirely clear to me. How exactly does the limit of x at infinity cause the limitation of the NDC? 
What is the role of the L2 regularization in the method and the flaw of NDC specifically?
Overall, the structure of the paper is good though.

The contributions seem rather incremental. In particular, the difference to Chang et al. 2019 seems to be only in the time-delay of the model. The main novelty of the paper compared to stochastic control barrier functions is mentioned to be its simplicity. Both arguments seem rather minor advancements.

The scale of the experiment seems rather limited. Particularly, the bicycle model and the inverted pendulum task are both use cases that much simpler linear or LQR control policies can potentially solve.
This raises the question of what bottlenecks the method.

How does the LaSalle's type loss ensure the fulfillment of the conditions of Theorem 2.2 for all (x,y), especially when the function V is expressed by an MLP and trained by gradient descent on random samples?

To obtain safety guarantees, the paper uses an input convex neural network. I don't see how this is then more expressive than a QP-based approach to synthesizing control barrier functions.
Can you elaborate?

clarity,_quality,_novelty_and_reproducibility: As mentioned in the weaknesses section, the writing is unclear at times.
summary_of_the_review: Overall interesting work simplifying existing approaches model and adding time-delay to the model. Contributions compared to existing methods may be incremental and experimental evaluation is limited to simpler control tasks.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: The reviewers found the problem studied in the paper important and challenging, and the proposed solution interesting and novel. However, there are concerns about 1) certain claims in the paper (e.g., safety guarantee), 2) the lack of clarity in writing, which makes it difficult to properly evaluate the results, 3) novelty compared to existing methods, and 4) some limitations in the experiments. The authors addressed (or partially addressed) some of these concerns during the rebuttal, which eventually convinced the reviewers to increase their score. I would strongly recommend the authors to take the rest of the reviewers' comments into consideration, revise their work, especially better (and more careful) structuring and writing in their next revision.


justification_for_why_not_higher_score: lack of clarity in writing, especially in expressing certain claims, and limited novelty compared to existing methods
justification_for_why_not_lower_score: The problem studied in the paper is challenging and the proposed solution is interesting and novel.
Decision: Accept: poster