Venue: NeurIPS 2021 Conference

Reviews:

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: This paper studies a real-time iteration scheme for solving nonlinear MPC. The paper provides an adaptive way of Newton stepsize selection and establishes global convergence. The paper also provides abundant numerical results.
main_review: The paper is well-written and the results are important. One major challenge in nonlinear control is the computation burden and one major strength is that the proposed algorithm can ensure global convergence for time-varying cost functions and dynamical systems only by one iteration of Newton update per time step, which greatly reduces the computation time per stage while still ensuring certain performance guarantee. Some comments are provided below.

Firstly,  It would be better if the authors can comment more on the quality of the stationary points where the RTI converges. For example, how does the converged stationary point compare with solving MPC completely? Is there any way to improve the performance of the stationary points? How do the initial values affect the quality of the stationary points?

Secondly, can the authors provide more discussions on the technical novelty? Right now, the results seem to be a straightforward application of line search plus online Newton updates. 

Further, the stability of NMPC by implementing the RTI scheme proposed in this paper should be discussed. 

Lastly, can the authors comment on how to extend this to the model-free case?
limitations_and_societal_impact: Yes
review_title: null

final_score: rating: 7: Good paper, accept
review_content: summary: This
 paper proposes an adaptive extension of a real-time iteration (RTI) 
scheme used to solve online non linear optimal control problems, with 
time-varying dynamics.

It introduces a parametric merit function, named exact augmented 
Lagrangian, which is used to adapt the step-size of each unique Newton 
steps involved to solve the succession of MPC sub-problems. Such 
step-size follows an Armijo-type condition. Before each back-tracking 
line search, the penalty parameters of this merit function are adjusted 
according to a well-defined scheme.

Under generic assumptions, authors prove the global convergence of their
 approach. That is, the KKT residual of t-th sub-problem converges to 
zero as t, the time horizon, increases. Furthermore, for large t, 
penalty parameters of the merit function used are no longer changed. 

Finally, the paper proposes two types of numerical experiments. First, 
it compares the performances (in terms of speed and precision) of their 
approach with the one using a $l_1$ merit function to perform a 
backtracking scheme. Secondly, it analyses the robustness of their 
assumptions with respect to a quadratic penalty parameter $\mu$, which 
is tuned manually in their approach. 

Numerical results show indeed that this adaptive RTI has a greater 
performance, especially for time-varying problems, and is robust to 
$\mu$ parameter choice.
main_review: The
 paper is original by the subject it tackles, which is to 
propose an algorithm ensured to globally converge when used with an 
online non-linear optimal control problem with time-varying dynamics. 
This task is indeed hard. 

Its theoretical contribution is valuable and of some 
significance for the community as it provides a first global 
convergence analysis of RTI schemes in this setting.

The paper is well written and organized. Explanations are clear. The 
following remarks could enhance the quality and clarity of the 
contribution :
  - There should be a reference, or a proof, to the assertion of 
equivalence between being a strict local minimizer of the exact 
augmented Lagrangian and problem 2 (cf. lines  135 to 137),
  - A reference to the mean value theory and back-tracking line 
search would be appreciable (at lines 142-143),
 - The argument for tuning $\mu$ manually (lines 150-151) is not really convincing as other schemes could be used (not involving solving several times the KKT systems). This part should be more developed.
  - It is not clear whether or not for large $t$ the augmented 
Lagrangian is ensured to be exact with final $\eta_1$ and $\eta_2$ found
 (cf. theorem 4.4). Is it the case ? If yes, it should be mentioned somewhere, otherwise, does it mean the property is useless for this global convergence analysis ? It would be worth precising it.
   - In line 235, I guess authors want to write "one Newton step" 
and not "one step Newton".
   - Simulation setups should be enhanced. 5 independent runs are too little to show results of any statistical significance. The mean 
value of at least 1000 trajectories (for each different initial point) 
with standard deviations should be provided. The same applies finally to the robustness analysis of the $\mu$ parameter. 5 different parameters with a very little scale difference (from 1 to 5) are not significant to prove your claim. The analysis should be done for larger $\mu$ choices 
(at least $\mu = 1000$). Indeed, assumption 4.3 involves a parameter 
$c$, which authors "hope" being "small" (line 207). Or, the global 
convergence theorem 4.7 assumes $c\leq 1/\kappa^3$, with $\kappa = \mu /
 \gamma_{RH}$. It is hence of some importance to analyze numerically the scope of assumptions imposed for $c$. 


limitations_and_societal_impact: Authors
 provides limitations and extensions to their work, including more 
precise constants to be found in theorem 4.4, and several remarks 
provided in section 6.

Yet, as developed in the previous section, the scope of assumptions made
 on parameter $c$ must be more investigated numerically. If it appears 
that for some problems, a too large $\mu$ compromises global 
convergence, then such limitation must be developed consequently.
review_title: null

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: This paper gives an algorithm for nonlinear MPC based on augmented Lagrangians, and demonstrates that the iterates converge globally to a critical point of the KKT equations. The key insight is a rule for tuning the regularization parameters of the augmented Lagrangian that ensures that each algorithm step reduces the KKT residuals sufficiently to overcome any increase that would arise due to taking a step in the dynamics. As a result, after alternating between algorithm steps and dynamics steps for several iterations, all iterates produced by the algorithm will be near critical points. 
main_review: The paper is well written, the topic is interesting, and the theory appears to be sound. However, I do have several concerns. 

Fit
My most basic concern with this work is its fit with NeurIPs. This appears to be a work on model predictive control, with very little overlap with the machine learning community. While many control-related topics such as reinforcement learning and system identification have broad overlap with NeurIPs, this work does not. I think a controls conference or journal would be a much better fit. 

Significance
It is not clear what global convergence really implies in this context. The KKT residuals go to zero, but what does this mean from a control perspective? It appears that the after many iterates, the solution will at least be nearly feasible. But it is unclear whether the solution is converging to a local minimum. 

Along with this, the assumption that the state and actions stay within a compact set place some stringent requirement a priori that the MPC method stabilizes the system. 

Edit after Response: The general consensus seems to be that this is within the scope of NuerIPs, which was my main concern. I do think the work has merit, but there are some limitations, such as closed-loop stability, which are not addressed.
limitations_and_societal_impact: The authors have addressed some of the basic limitations of the work in the conclusion. There does not appear to be any obvious negative societal impact.
review_title: null

Meta-review: Dear authors,

The authors reached a consensus and positively evaluated the paper. Hence I recommend acceptation to the SAC.
I encourage you to pay special care to clarifying the points raised during the discussion regarding stability, MPC and RTI when preparing the final version of the paper.
Decision: Accept (Poster)