Global Convergence of Online Optimization for Nonlinear Model Predictive Control
Authors: Sen Na
Venue: NeurIPS 2021 Poster (2021)
Keywords: Global convergence
Nonlinear optimization
Model predictive control
Real-time iteration

Abstract: We study a real-time iteration (RTI) scheme for solving online optimization problem appeared in nonlinear optimal control. The proposed RTI scheme modifies the existing RTI-based model predictive control (MPC) algorithm, by selecting the stepsize of each Newton step at each sampling time using a differentiable exact augmented Lagrangian. The scheme can adaptively select the penalty parameters of augmented Lagrangian on the fly, which are shown to be stabilized after certain time periods. We prove under generic assumptions that, by involving stepsize selection instead of always using a full Newton step (like what most of the existing RTIs do), the scheme converges globally: for any initial point, the KKT residuals of the subproblems converge to zero. A key step is to show that augmented Lagrangian keeps decreasing as horizon moves forward. We demonstrate the global convergence behavior of the proposed RTI scheme in a numerical experiment.

Global Convergence of Online Optimization for Nonlinear Model Predictive Control
Sen Na
Department of Statistics University of Chicago Chicago, IL 60637 senna@uchicago.co edu
Abstract
We study a real-time iteration (RTT) scheme for solving online optimization problem appeared in nonlinear optimal control. The proposed RTI scheme modifies the existing RTI-based model predictive control (MPC) algorithm, by selecting the stepsize of each Newton step at each sampling time using a differentiable exact augmented Lagrangian. The scheme can adaptively select the penalty parameters of augmented Lagrangian on the fly, which are shown to be stabilized after certain time periods. We prove under generic assumptions that, by involving stepsize selection instead of always using full Newton step (like what most of the existing RTIs do), the scheme converges globally: for any initial point, the KKT residuals of the subproblems converge to zero. A key step is to show that augmented Laagrangian keeps decreasing as horizon moves forward. We demonstrate the global convergence behavior of the proposed RTI scheme in numerical experiment.
Introduction
We consider the following time- marriig nonlinear optimal control problem
N
P(xo) min
Σ Ou (II, up) - (N(EN),
k=0
s.t. 0++1 fe(a,, h E (N
(1)
To Zo.
where E RR= are the state variables, uy E Th are the control variables, Ok R x RR - R_ (gN TR= -> R2,) are the nonnegative cost functions, fs. R x TR. - RR are the dynamical constraint functions, To is the given initial state, and N is the horizon length. We suppose that gx, fa are twice continuously differentiable and, without loss generality, that the origin is steady state, ie. f0, 0) O, and is also a stationary point with zero loss, i.e. gs (0, 0) The well-known linear quadratic regulator (LQR) satisfies this setup and the same setup is commonly used in the literature (8). Problem (1) is also called dynamic program in control community, which has a close relation to reinforcement learning (RL) (see (6). The main difference is that the dynamic fx in our paper is known, which is the in many industrial applications and model-based RL studies.
and V (0, 0) 0.
In modern applications such as energy and autonomous control, tends to be extremely large or even infinity, which stimulates the interest of solving ①1) in real time. Let fi< ux) = fa (Ix, ug; de) where dę is data at stage k. then a more realistic setting is collect dy as a data stream. That is, instead of knowing fo all the way to fN-1 and solving [I] offline, we only know fa sequentially and have to solve | online. Model predictive control (MPC), also known as receding horizon
35th Conference on Neural Information Processing Systems (NcurIPS 2021).

---

control, is an important feedback control technique that is used in online optimization of control problems, Given the current system state, MPC looks ahead multiple stages on the prediction horizon. solves the subproblem on the prediction horizon to optimality, and provides the first optimal control action to the system as the feedback. Then the system evolves to a new state based on the control action and the procedure is repeated. However, the classical nonlinear MPC is being regarded as theoretical concept rather than a practical strategy. It is not as widely used as linear MPC, and is mostly investigated in slow dynamics. This is due to the intensive computations of solving the control subproblems when we have nonlinear models. Fortunately, the design of real-time iteration (RTI) and its predecessors (26 37 have enabled RTI- based nonlinear MPC to be applied on fast dynamics and
achieve good performance [48
38
15
24
18 22
20
16
19 39
Instead of using optimal feedback control, RTI schemes compute a cheap approximation of control action and provide back to the system as fast as possible. The existing RTT schemes perform a single, full Newton step for each subproblem, and the output iterate of the previous subproblem is used to warmly initialize the current subproblem 88 42.. Thus, RTT schemes significantly reduce the computations of each subproblem, which makes them succeed in modern industrial applications. RTI schemes exploit the fact that two successive subproblems are closely related though different, so that the iteration can achieve the convergence "on the fly" (i.e. the error goes to zero as horizon moves forward). However, the RTI-based MPC is challenging to analyze. The main difficulty lies in the fact that the prediction horizon is shifted each time. Each subproblem is appended by a new stage that has never been scanned before and may introduce perturbation into the system. Different RTI-based MPC schemes have been studied 9
with an improved local analysis established in 31 recently.
Our paper complements the literature on RTI-based MPC by investigating its global convergence. We emphasize that by global convergence we mean the convergence to stationary points (i.e. KKT points) for any initialization, instead of the convergence to global optimality. The latter is not achievable for nonlinear problems without strong assumptions [34] Instead of using a full Newton step, we design an adaptive scheme to involve the stepsize selection procedure. In fact, the stepsize selection has been suggested as way to improve the global behavior of RTI in shrinking (not moving) horizons in an old paper (25). The authors implemented a & penalized merit function with a watchdog line search. However, the global convergence theory remains open. Different from the suggestion of [25], we use a differentiable exact augmented Lagrangian as the merit function. This function mainly has two differences to the & penalized merit function: (i) it depends
both primal and dual variables, so that dual variables are updated using the same stepsize as the primal variables (f, merit function only updates the primal variables); (ii) a simple backtracking line search can effectively overcome Maratos effect (29) and accept the unit stepsize locally (&1 merit function requires a more time-consuming line search method such watchdog or second-order correction). We conduct experiments to show the superiority of augmented Lagrangian over ε penalized function, especially for time-varying problems. Moreover, our algorithm adaptively selects the penalty parameters of augmented Lagrangian on the fly. When the unit stepsize is accepted. the algorithm simply boils down to the standard RTI-based MPC, and the existing local theories [8] [1] apply seamlessly. We notice that the same merit function has been employed in a recent work (32 to achieve the global convergence for solving (1) in offline fashion, with a horizon decomposition strategy. In that work, the authors showed the unit stepsize is accepted even with an inexact Newton direction. Our paper is in online regime and does not investigate the local behavior.
Related work and contribution: Our work is closely related to a growing literature on online optimization via RTI. The theories of RTI have been investigated over the past decade. [10] and g established the local convergence of RTI in shrinking horizons and moving horizons, respectively.
36 all showed similar results. A different view of RTI-based MPC is via the parametric optimization. where subproblems are parameterized by continuous time index and single Newton step performed at each time instant. Similar stability results in this setup are shown in [1] 45 [3] Recently, [I]] studied a lag- L. RT'-bbsed MPC (i.e. one shifts stages each time), and proved that RTI iterates on each stage converge linearly with a rate decaying exponentially in the prediction horizon length. Their analyses rely on the sensitivity analysis An important consequence of convergence of online iteration is the dynamic regret For example, the convergence rate in [31 and the regret order in 27 are both exponential prediction horizon length. This is due to the widely-used Lipschitz continuity assumption the objective.
When we have different models, 3 42
35]
of 1 30 33
analysis 47 27
43
'The merit function refers to the function that is used in the line search.
2

---

However, all above theories are local results. The algorithms always take a full (Newton) step for each subproblem, without involving any stepsize selection procedure. Thus, the analyses require to have an initial point that is sufficiently close to the local solution, or require that the first Newton step is sufficiently small. Obviously, for Problem ( when | is small enough, the origin is a good initial point and RTT íterates stably track the origin. However, if | is large or we may want to track a nontrivial MPC policy, the origin is not a reasonable choice for the initial point. Based on these considerations, the global convergence of RTI addressed in this paper complements the aforementioned rich local analyses. Again, by global convergence we mean that the KKT residuals horizon moves forward for any initialization. The known, local stability
of subproblems go to of tracking error of RTI is not addressed in this paper, but referred to [45] Theorem 4.2].
The challenge of our analysis is that: the problem varies with time instants and only single Newton step is performed each time instant. Such a setup contrasts to online Newton (with line search) analysis, where single Newton step targeting the same problem is performed. A key, technical step of our analysis is to show that the exact augmented Lagrangian merit function keeps decreasing as horizons shift. To our knowledge, this is the first paper studying the global behavior of online iteration under RTI setting, via a tool of the exact augmented Lagrangian. Another contribution to nonlinear MPC is that we consider time-varying dynamies, while most of the literature considered invariatt dynamics (e.g. [8 21 40)). The subproblems time- invariaat dynamics only differ in the initial state, which results in much simpler analysis. We notice that [3]] allowed mare-varying dynamics, but required to use a larger lag when shifting the horizon. We allow lag one instead.
For two vectors a, b, (a: b) denotes long column vector that stacks and b together. For two scalars a, b, a 5 = max (a, b). Without specification, || denotes the lg norm for vectors and operator norm for matrices. We also reserve the following notations. O denotes zero vector or matrix, and denotes the identity matrix. Their dimensions are clear from the context, M denotes the prediction horizon length, / indexes the sampling time, and z denotes the state-control pair at stage k.
Notations:
(Ix; uk) E " with n2 f2y + nu
Preliminaries
We introduce MPC subproblems formulation, the Newton step, and the augmented Lagrangian merit function in this section. At time I, the prediction horizon shifts to [t, Mr] with M, = + M denoting the last stage, and the corresponding subproblem p'(Z)) is given by
M
p'(a)):
min Mu
Σ
9k (akk, uk) + 9M, (mras 0) + ~ a 11:
k=1
s.t. 0&+1
f ux), [t, 1].
Et.
Here, l is quadratic penalty parameter independent from I; o. is the current system state, which is given and determined by the control feedback provided to the system before 1. The formulation (2) is used in (1 for local analysis, while alternative subproblem formulations are available as well 23
22. RTI schemes perform a single Newton step for pt (T() and move to the next time instant. To ease notations, we let ã; Jr.M, and û; Ut denote the state and control variables of the f-th subproblem, respectively. We also let 2; = Zt:M. = (2z;... ..: as, denote the whole ordered state-control variables, with slightly abusing the notation MM, to let z M. 2M,- We sometimes also express &, = (me, üt) when we specify each component of žt. Then, the Lagrangian function of ( is
Given subproblem
M -1
C², \1;
Σ
go (xr. ux) + gM, ( 0) + " 2
| 112
M.
X ( - fa (x)))))))) Xx_ (x; Tt),
where 2, A.-1,M,-- -1 the dual vector. Here, > E R is associated to the k-th constraint of for k E [t, Mr 1], and A _== is associated to the initial state constraint.
3

---

Given the input iterate (x), x,),
the Newton direction (Ažt, Aλ,) is solved by the KKT system
(G
B
(G")
A AA
(3)
where B' is an approximation of the Hessian H' Va
L is the Jacobian matrix; and all matrices in (3) are evaluated at (z), 2.). The left hand side matrix is called the KKT matrix.
=0 In what follows, H² B¹, G are always evaluated at the input iterate (2), X, if no evaluation point is specified. Taking a closer look 3 (cf. Appendix [A]. we note that the KKT matrix and Vg, L² independent from the given initial state Tt. This is a crucial observation. In reality, the realization of the system state may lag behind the sampling time, since the system has to apply the feedback control
to evolve. Thus, DG may not be known soon as the input iterate (ž X²) However, promising property RTI is that most computations of (3) can be prepared without knowing öt.
Given the Newton direction (Aæ,. Adx) from 33. we update the iterate by involving a stepsize ot
2
AZ A
+
at
(4)
X2
In our paper, 0₁ is selected by forcing Armijo condition hold on differentiable exact augmented Lagrangian, defined as
(2,, dx; T.) = (2, (.: 2 V f'. Ar: 2.)
1/2 V2, do: a:) (5)
2
Here 17 = (m, 122) are parameters of the quadratic penalties. 5 is called exact augmented Lagrangian since one show that, for sufficiently large 7/1 and sufficiently small 1/2, strict local minimizer of (-) is also a strict local minimizer Problem (2) and vice versa (see [2] Proposition 4.5]). When V22 = 0, C (-) reduces to standard augmented Lagrangian, while it does not enjoy exactness property anymore, and may cause issues when it is used as merit function. For simplicity, we sometimes suppress "exact" and call (-) augmented Lagrangian. Based on (5) ar is selected by making the following Armijo condition.
Vz, Ci &. (, S C) X;: a,
10
T
A AAx
(6)
V S, X/:
hold for prespecified parameter 3 (0, Note that, by mean value theory, Ot can always be found by backtracking line search provided the inner product in (6) is negative (cf. Theorem 4.5
3 An Adaptive RTI-based MPC Scheme
We now set the stage to propose our algorithm. The algorithm inherits the flavor of RTI schemes: it performs single Newton step for each subproblem. Differently, it involves a stepsize selection procedure based on (-))) instead of using a full step (i.e. a₁ 1).
The algorithm adaptively selects the parameter on the fly (i.e. automatically selects suitable for each subproblem). However, since μ is required in the problem formulation, the adaptivity on may require to solve the KKT system multiple times for each subproblem. Thus, the gain from the adaptivity on for procedure is limited. We hence prefer to tune a manually in practice. Our experiments show that our algorithm is robust in (see Figure T.
Let us now illustrate how transit from the triple (2) X): wx) to (x) (2++1 S4 T.-1). To further ease notations, let C,,11 f1, L L.0 denote the function values of (,)()) and (²(-) at the output iterate ( x and the input iterate (=) X,)). respectively. addition, when we specify the iterate a : for Id 0 (similar for 2/4 x, A&,, and Ad₂). For example, a/+1. denotes the output iterate of the (-th subproblem at stage +
on each stage, let ax!d
Given the input iterate (2 X.) we perform four steps:
(1) We compute B¹, Gt. V CL.. and Vx t except the first row. After observe Tt, we compute

---

Algorithm An Adaptive RTI-based MPC Scheme
1: Input: initial iterate (#): Xg). initial state Zo, and parameters μ > p > 1, 1/ (n/, 77)
(1² 1), and 3 (0,1);
2: for O, 1, 2......... do
3: Compute B¹, G", CE, V Lt.,, and solve the KKT system 3 to obtain (Ažt, A};);
4: while
T
AA,
AE
>
zh
Vz, V2,
Vx, c..
do
5: Let 77 = p m/, 7/2
= n
6: end while
7: Select a2 by backtracking line search to make (6) hold, and update iterates by (4)
8: Define (#1+1; ((+1) and #4-1 according to (8 and let 17 =ny²;
9:
end for
the first of V Ct.,, Then, we compute the Newton direction (Ažr, Adx) by solving 33)
(2) We select of (ni v/2) such that
C V
T
A AX,
(
Vz, C2. 2 f
V L
V
4
As shown in Algorithm| 1 we can use While loop to increase r/ and decrease 7/ until 7 is satisfied. (3) Using of from last step, we select or by backtracking line search to make Armijo condition hold. We then obtain the output iterate (2) x by (4)
(4) We transit to the next horizon by letting
t+1 = F441:M441.111 = (1 2442 ; :
2h = -u 141:M--1-1H = (ut+1 2 1, 0):
(8)
x
2+1 = -NiMM11-114 = (A A AM.
T1+1 (x) ut,))
The above four steps are displayed in Algorithm] [ If we remove Step (2) and use a, = 1 in Step (3), it reduces to standard RTI schemes [8 41 42 BT]. However, shown later, the difference that select Or based on augmented Lagrangian C, (.) enables Algorithm to converge globally.
We discuss the overhead of the algorithm. For each subproblem, the computational complexity is dominated by solving the KKT system. The KKT matrix square matrix with dimensions 2(M + 1)nz + Mn,- Directly inverting the matrix results in ((M n flops. Fortunately, we can exploit the sparse structure of the KKT matrix and apply sparse LDL" factorization did in Section C of [41), which results in (M n.,) 3) flops instead. So the computations are linear in terms of the prediction horizon length M.
Remark 3.1.
4 Global Convergence Analysis
We conduct global convergence analysis for Algorithm We show that, for any initial iterate (#)) X the KKT residual the t-th subproblem, |/ CV converges to zero as f increases. The significance of the result is that only perform single Newton step for each time-varyine varying subproblem, and our result complements the local results of RTI. We should mention that, like most deterministic methods for nonlinear problems (34), Algorithm i is not selective stationary points, and may converge to saddle points. Converging to local minimum is only possible under stronger, local assumptions, which addressed in this paper but refer to [31.
4.1 Assumptions
We require the following assumptions.
5

---

Assumption 4.1 (Compactness). We assume all iterates lie in convex compact set that contains the origin. In particular, we assume there exist convex compact sets 0 E Z C R th and 0 E A C R such that (= x Z x A for any 2 0 and any stage k of the t-th subproblem.
Assumption 4.2 (Boundedness condition). We assume there exist absolute constants T, YRH 0 such that (i) for any Z 0, B' satisfies =" B x 2 7RH | V2 E {= G² = 0, # 0). and
|B' H' 8; (ii) for any & and 2 E 2, we have
V an (I+)) V V un [x)) V 2 (2x))l V 1 (+) S
(9)
[ [V. 9 (2+) 2 (... S T;
(10)
(iii) for any 2 0, (G 1/T2.
Assumption 4.3. We assume there exist absolute constants c, C > such that for any 2 (i)
| S cll( A ; AX)Il if i M,, * O. (ii) || S C/ Dell if a * Zt.
A 4.1 and 4.2 are standard in the literature on Newton analysis 34 We note that Jacobian G
always has full row rank (see (3) in Appendix A). so the linear independence constraint qualification (LICQ) holds. Together with LICQ, A i. implies the KKT matrix is nonsingular [34] Lemma 16.1]. In fact, we can simply let 8 = pll, which satisfies all conditions trivially; is sufficient for our global analysis; and is indeed used in our experiments. More generally, 3* can approximate the Hessian /1' using any methods in [34]. Different from existing analyses which only require the bounded Hessian, (TO) requires one more derivative -te third derivative to be bounded. This is
32], because the Hessian standard when using augmented Lagrangian (,,() in the algorithm 46 2
of (,,(.) requires the third derivatives of gk: JR- We mention that the third derivatives are only used in the analysis, and do not have to be computed in the implementation. ( is directly implied by the compactness in A 4.1 A 4.2 iii) is also reasonable to hold since G has full row rank. For time-invariant problems where Ok = 9. fk = f. A 4.2 ii) and (iii) hold trivially.
A 4.3 is particularly imposed for MPC analysis; condition with the same flavor is in g At the first glance, it seems to be a restrictive assumption. Fortunately, it guaranteed to hold when je is sufficiently large (the threshold depends on problem parameters), and is easily checkable during the iteration. In particular, A (1) assumes that the last dual step AÂM, is bounded by the full Newton step (Až,, A).). It holds trivially for 1, however, hope is small. One can expect this condition to be satisfied for large enough μ. since AÅM,-1; measures the disturbance of x M away from #0 . O. When μ is large, know from the quadratic penalty in that
TM, RJ O. Hence, AAM, becomes arbitrarily small. See more discussions in [8] Assumption 3] for local analysis of MPC. A fii allows any constant C as long C' S μ²², so it holds well for large enough μ.. In addition, A 4.3 ii) is not required if a,= = 1 is selected, which is always true
when (2): x;) is close a stationary point of the subproblem? This is because that, if at 1, then zi,-" = He for any input iterate x²,--
4.2 Global convergence
We first show that Algorithm well-posed, that is, each iteration can be performed successfully. The computation of the Newton direction in Line 3 ensured by the nonsingularity
the KKT matrix, implied by A 4.2 as discussed before. The backtracking line search in Line 7 is guaranteed by the mean value theorem 5 It hence suffices to show that the While loop in Line terminates in finite time.
Theorem 4.4. Consider Algorithm under Assumptions T1, T2, Ta > 0 by
4.1 and 4.2
Let us define thresholds
25
32
YRE 28
T2
YRH
2RH
For any t 2 0, if
2 TI, 12
T2, 11722 2 T3, then
V Ct.0 x
T
/ A
1/2
2z, Lro
c/.0
4
x
2 128) showed under certain conditions that the backtracking line search would always select unit stepsize locally when using exact augmented Lagrangian the merit function.

---

Further, there exist an integer 7 > 0 and positive constants = (71 F2)) satisfying
32 p2 u²
YRH
in S
72
32p
such that r/' 17. 14 T.
Theorem 4.4 shows that the While loop indeed terminates in finite time, and more importantly the penalty parameter 1/' stabilizes after certain iterations. This result is critical for global convergence since, after r subproblems, all the succeeding subproblems try to decrease a much similar merit function. We note that fj1 depends on μ.. so the penalty of the objective connects to the penalty of augmented Lagrangian. We mention that the constants in {Ta J=1 and î7, f72 can be significantly improved by a careful calculation. However, we leave for future work since it's not main target here.
is also worth mentioning that the stabilized parameters (7, ft2) may not ensure the equivalence between stationary points of C (-) and KKT points of ( because the stabilized values rely particular iteration sequence. The parameters thresholds for the exactness property of L ensure that any stationary points of augmented Lagrangian are equivalent to KKT points, which is a stronger guarantee. However, the stabilized parameters indeed ensure that our particular iteration converges to a KKT point, not just stationary point of augmented Lagrangian.
We now study how the merit function decreases from subproblem another.
Theorem 4.5. Consider Algorithm | Junder Assumptions 4.1 and 4.2 There exists constant
such that 2 ā. Vt 2 7 where is from Theorem 4.4
Moreover,
< 0.0
1
4
Theorem 4.5 shows that Armijo condition is satisfied for sufficiently small ag with threshold independent of t. Moreover, after one Newton step, we can observe sufficient decrease of the merit function. However, since we only perform one Newton step and transit to the next subproblem immediately, we cannot claim the decrease is sustained. This is indeed the main challenge of global analysis of RTI schemes. Different subproblems target different merit functions, even v is shared.
We show in the next lemma that, the transition between subproblems may increase the merit function, but not too much. Overall, the merit function C keeps decreasing.
and let k Hessian matrix. Suppose the constants c, C in Assumption 4.3 satisfies S 7RH/K² and C' S n², Then
Lemma 4.6. Under Assumptions
denote the condition number of the
12033
|V
8
Combining Lemma 4.6 with Theorem 4.5 we immediately have
TzāB
one-step error recursion:
C,,100 S
(11)
8
Using 11 we sum over and finally establish the global convergence in the next theorem.
Theorem 4.7. Consider Algorithm under Assumptions 4.1
4.3
Suppose e S YRH/K² and C n²
then for any input iterate (20, Ag). we have V
0 as
5 Numerical Experiment
We apply Algorithm Jo dynamic program demonstrate the global behavior of RTI-based MPC. We compare augmented Lagrangian merit function with & penalized merit function, defined as
Mi-1
(2:;;
Σ
go (ar, we) + gM, (M,,, 0) + H || 2
k=t
M.-1 +
[xtt1 fa ux)ll1 + || - æell1
3ingh means the inequality holds up problem- inniem--edddeeddentt constant.
7

---

Table 1: Simulation Setups
CASE
N
(ag, be: Ck)
j
111
1/2
8
V
Jo
CASE
100
(5, 10, 15)
(1,1,1)
25
1
0.4
1
1.5
10
10 8
CASE 2
250
(5, 10, 15)
(k, k, k)
1
0.4
1
1.5
10
10
CASE 250
(5, 10, 15)
(2,,2, A.AP,A k*)
20
100
0.4
1.5
10
10-8
Using (.) as the merit function, Armijo condition is rewritten C < f. +ag-- Až₁).
where D(C5 A the directional derivative of (-) at # along the direction A given by
Me-1
D A2,) =
Σ
ga (x + Van 9M, (2 0) 0)
k=
| faL ux)l11 + +1 Tzll2 DI -VD2.
To ensure D(E,, A2t) to be negative, we let o' = max(D1) ((111))2) ( ,1-1 with E (1, 2). Thus, RTI-based MPC using & penalized merit function (called E MPC) is obtained by replacing Lines 4 and 7 of Algorithm | the above corresponding steps.
Simulation setups. We consider ID trigonometric perturbed LQR problem. In particular, we let ge = ax.z + bb + sin (xx) and fs = I + ug + sin( We study three cases summarized in Table For each case, we perform 1000 independent runs with randomly generalized initial iterate (=g. %0) by letting (x u2.0= 20)) ~ (0, 257), Vk. We stop the iteration either t >> M 10-8 (i.e. attains the error threshold). We let B' = =ul, vt. For each case, we try three prediction horizon lengths.
(i.e. attains the iteration threshold) [V < [ on μ. For Case 1, we let dg by C2 1,
5, 10, 15. We also vary μ
in difffrrntt cases to test the robustness of Algorithm which results in a time-invarian problem. For Cases and 3, we let a₃ = b2 = Cj = & and k2, which results in time-vane vare-narying problems. The linear and quadratic increase of coefficients bring significant challenges for solving subproblems online. The code is implemented in Julia 1.5.4 and is publicly available (with high resolution figures) at wttph::////// httpggittt.....caaaaaaaaaaaaaeae 1281 GGtballollllllooaoooooooooolllla -RTa--RRI---
Evaluation of results. For each case, we compare the average running time between two merit functions, and compare the maximum (over 1000 independent runs) of the number of iterations. The comparisons are shown Table 2 From Table 2 we see that two merit functions work equally well for the Lime-invariant problem Case 1, while augmented Lagrangian works significantly better than
& MPC for the time-vare varying problems in Cases 2 and 3. The latter fails to attain 10 KKT residuals within 250 iterations in both Case and Case 3.
Moreover, we plot C*, v.s. t log scale for cases. To have a better visualization, among 1000 initializations, we only plot the first in Figure [ but draw an error bar plot for all initializations in Figure 2 We see from Figure [ (a)-(f) and Figure 2 (a)-(f) that both MPC schemes achieve global convergence within 25 iterations for the time-invariant problem in Case 1. Their (linear) convergence speed is comparable as well. However, for the time-varying problems in Cases and 3, Figure (g)-(r) and Figure (g)-(r) suggest that augmented Lagrangian MPC performs much better than f MPC. Although both schemes decrease KKT residuals as t increases, which validates our global convergence theory. augmented Lagrangian MPC decreases the residuals linearly in t, and below the threshold e = 10 within a moderate number of iterations. As comparison, & MPC decreases the residuals only to [10- 10 using 250 iterations. Moreover, it is clear to see that E MPC has slow convergence speed locally (because the tail of [/V'' is flat), which does not appear in augmented Lagrangian MPC. Therefore, for time- varying problems, the proposed augmented Lagrangian MPC much preferable than ε MPC.
We also take Case 2 as example to show our algorithm is robust to n.. We vary 1, 5, 10, 100, 500, 1000 and randomly generate an initial point for each μ.. We see from Figure (s)-(x) that the performance of our algorithm is stable for different choices of μ.

---

196
192211
(a) Case 1, M 5, Aug (b) Case 1, .M 10, Aug (c) Case 1, M 15, Aug
(d) Case 1, M 5, 4
10.2
(e) Case M. = 10, &
(f) Case 1, M 15. E1 (g) Case 2, M 5, Aug (h) Case 2, M 10, Aug
Case 2, .M 15, Aug
i Case 2, M 5.
(k) Case 2, M 10,
(l) Case M -15, E1
1922n
(m) Case 3, M = 5, Aug (n) Case 3, .M 10, Aug (0) Case 3, M 15, Aug
(p) Case 3, M = 5,
(g) Case 3, M = 10, E1
(r) Case 3, M 15. 81
(5) Case 2, M = 5, Aug (L) Case 2, M 10. Aug
20
-10
16
(u) Case M 15, Aug
(v) Case M (w) Case 2, M 10, &1
(x) Case 2, M 15,
KKT residual plot. Each panel corresponds a combination of Case, M and the merit function. The figures (a)-(r) correspond to the settings in Table In each panel, the five lines correspond to five different initial points that are randomly generalized. The figures (s)-(x) tests the robustness of using Case 2. In each panel, the five lines correspond to five μ varying from to 5, with randomly generated initial point.
Figure 1:

---

Table 2: Simulation Results
Case
Aug
log (Time)
Aug
of
Case
log (Time) Aug
Aug
of
Case
Aug
log (Time)
Aug
10
Case
-7
-7
22
Case
157
241
Case 3
-4
156
241
15
-6
-5
25
25
-4
164
236
-3
3
217
236
(the boldface mumber indicites the leration stops attaining the iteration thresholld
(a) M = 5, Aug
(b) M 10, Aug
(c) M 15, Aug
(d) M = 5, 5.
(c) M = 10, 4
(f) M 15.4
100
g M 5, Aug
(h) M 10, Aug
i1) M 15, Aug
( M = 5,61
(k) M 10,
()) M 15
(m) M Aug
(n) M 10,
(0) M
15, Aug
(p) M 5,4
(q) M 10.4
(1) M 15.8
Figure 2: KKT residual bar plot. Each panel corresponds to a combination of Case, M and the merit function. The first, second, third lines correspond to Case 1,2.3, respectively. each panel, the circle corresoonds to the mean among 1000 initializations, while the length of the vertical line of each point corresponds to the standard deviation. The low bound suppressed it is negative.
6 Conclusion
We studied the global behavior of RTI-based nonlinear MPC algorithm. Different from the existing schemes, we imposed line search for each subproblem select the stepsize, using a differentiable exact augmented Lagrangian the merit function. We proved that the proposed algorithm enjoys global convergence: for any initial iterate, the KKT residuals of subproblems converge to zero. Our analysis complements the existing local analyses of RTI-based MPC, which require sufficiently good initial point.
One extension is to replace the Newton step by the quasi- step. This may lead to a slow local convergence, however, the global convergence should be preserved. Another important extension is to study stochastic dynamics, like 40). We can evaluate the iteration by either regret 227 E V L Then even the local stability property RTI-based MPC is not well understood. Furthermore, studying RTI schemes for model-based RL is attractive research direction and, to some extent, our analysis is still applicable. However, how to apply RTI on model-free RL is still not clear to our knowledge. Finally,
experiments show that RTI-based MPC converge globally even for moderate (c.g. = 1), which in some sense suggests Assumption 4.3) is not necessary. How to remove Assumption 4.3 is worth investigating.
References
[1] Thivaharan Albin, Dennis Ritter, Dirk Abel, Norman Liberda, Rien Quirynen, and Moritz Diehl. Nonlinear mpe for two-stage turbocharged gasoline engine airpath. In 2015 54th IEEE Conference Decision and Control ( (CDC), pages 849-856. IEEE, 2015.
10

---

[2] Dimitri P Bertsekas. Constrained optimization and Lagrange multiplier methods. Academic press, 2014.
[3] H. G. Bock, M. Diehl, E. A. Kostina, and P. Schlöder. Constrained optimal feedback control of systems governed by large differential algebraic equations. In Real-Time and Online PDE-Cone Constraiidd Optimization, pages 3-22. SIAM, 2007.
[4] J. Bonilla, R. D. Keyser, M. Diehl, and J. Espinosa. Fast NMPC DC-DC converter: An exact Newton real-time iteration approach. In Nonlinear Control Systems, 2007.
[5] Stephen Boyd and Lieven Vandenberghe. Convex Optimization. Cambridge University Press, 2004.
[6] Lucian Busoniu. Reinforcement learning and dynamic programming using function approxima- tors. CRC Press, Boca Raton, FI., 2010.
[7] Ahing-An Cheng, Jonathan Lee, Ken Goldberg, and Byron Boots. Online learning with continuous variations: Dynamic regret and reductions. In International Conference on Artificial Intelligence and Statistics, pages 2218-228.. PMLR, 2020.
[8] M. Diehl, R. Findeisen, H.G. Bock, F. Allgöwer, and J.P. Schlöder. Nominal stability of real- time iteration scheme for nonlinear model predictive control. IEE Proceedings Control Theory and Applications, 152(3):296-308 may 2005.
[9] Moritz Diehl. Real-time optimization for large scale nonlinear processes. PhD thesis, 2001.
[10] Moritz Dichl, Hans Georg Bock, and Johannes P. Schlöder. A real-time iteration scheme for nonlinear optimization in optimal feedback control. SIAM Journal on Control and Optimization, 43(5):1714-1736 jan 2005.
[11] Moritz Diehi, H.Georg Bock, Johannes P. Schlöder, Rolf Findeisen, Zoltan Nagy, and Frank Allgöwer. Real-time optimization and nonlinear model predictive control of processes governed by differentiallalgebrric equations. Journal of Process Control, 12(4):577-585, jun 2002.
[12] Moritz Dichl, Lalo Magni, and Giuseppe De Nicolao. Efficient nmpe of unstable periodic systems using approximate infinite horizon closed loop costing. Annual Reviews in Control, 28(1):37-45, 2004.
[13] Quoe Tran Dinh, Carlo Savorgnan, and Moritz Diehl. Adjoint-based predictor-corrector sequen- tial convex programming for parametric nonlinear optimization. SIAM J. Optim., 22(4): 1284, 2012.
[14] H. Ferreau, P. Ortner, P. Langthaler, L. del Re, and M. Diehl. Predictive control of a real- world diesel engine using extended online active set strategy. Annual Reviews in Control, 31(2):293-301, 2007.
[15] Hans Joachim Ferreau, Boris Houska, Kurt Geebelen, and Moritz Diehl. Real-time control of a kite-model using an auto- generatee nonlinear MPC algorithm. In Proceedings of the 18th IFAC World Congress, 2011.
[16] J. V. Frasch, A. J. Gray, M. Zanon, H. J. Ferreau, S. Sager, F. Borrelli, and M. Diehl. An auto-generated nonlinear MPC algorithm for real-time obstacle avoidance of ground vehicles. In Proceedings of the European Control Conference (ECC), pages 4136-4141, 2013.
[17] S. Gros, M. Vukov, and M. Diehl. A real-time MHE and NMPC scheme for wind turbine control. In Proceedings of the IEEE Conference on Decision and Control (CDC), 2013.
[18] Sébastien Gros, Rien Quirynen, and Moritz Dichl. Aircraft control based on fast non-linear mpc & multiple-shooting. In 2012 IEEE 51st IEEE Conference on Decision and Control (CDC), pages 1142-1147. IEEE, 2012.
[19] Sébastien Gros, Rien Quirynen, and Moritz Diehl. An improved real-time economic nmpc scheme for wind turbine control using ppline-interpolated aerodynamic coefficients. In 53rd IEEE Conference Decision and Control, pages 935-940. IEEE, 2014.

---

[20] Sébastien Gros, Mario Zanon, and Moritz Diehl. Control of airborne wind energy systems based on nonlinear model predictive control & moving horizon estimation. In 2013 European Control Conference (ECC), pages 1017-1022. IEEE, 2013.
[21] Sébastien Gros, Mario Zanon, Rien Quirynen, Alberto Bemporad, and Moritz Diehl. From linear to nonlinear MPC: bridging the gap via the real-time iteration. International Journal of Control, (3(1):62-80, sep 2016.
[22] Sébastien Gros, Mario Zanon, Milan Vukov, and Moritz Diehl, Nonlinear mpc and mhe for mechanical multi-body systems with application to fast tethered airplanes. IFAC Proceedings Volumes, 45(17):86-93, 2012.
[23] Lars Grüne, Jürgen Pannek, Martin Seehafer, and Karl Worthmann. Analysis of unconstrained nonlinear mpe schemes with time varying control horizon. SIAM Journal on Control and Optimization, 48(8))4938-499) 49699938496622-8 2010.
[24] Boris Houska, Hans Joachim Ferreau, and Moritz Diehl. An auto-generated real-time iteration algorithm for nonlinear MPC in the microsecond range. Automatica, 47(10))2279-222) oct 2011.
[25] Daniel B Leineweber, Irene Bauer, Hans Georg Bock, and Johannes P Schlöder. An efficient multiple shooting based reduced sqp strategy for large-scale dynamic process optimization, part 1: theoretical aspects. Computers & Chemical Engineering, 27(2):157-166, 2003.
[26] Wei C Li and Lorenz T Biegler. A multistep, newton-type control strategy for constrained, nonlinear processes. In 1989 American Control Conference, pages 1526-1527. IEEE, 1989.
[27] Yingying Li, Xin Chen, and Na Li. Online optimal control with linear dynamics and predictions: Algorithms and regret analysis. In NeurIPS, pages 14858- 14858-14870, 2019.
[28] S Lucidi. Recursive quadratic programming algorithm that exact augmented lagrangian function. Journal of Optimization Theory and Applications, 67(2):227-245, 1990.
[29] Nicholas Maratos. Exact penalty function algorithms for finite dimensional and control opti- mization problems. 1978.
[30] Sen Na and Mihai Anitescu. Exponential decay in the sensitivity analysis of nonlinear dynamic programming. SIAM Journal Optimization, 3022):157711554, jan 2020.
[31] Sen Na and Mihai Anitescu. Superconvergence of online optimization for model predictive control. arXiv preprint arXiv:2001.037077 2020.
[32] Sen Na, Mihai Anitescu, and Mladen Kolar. fast temporal decomposition procedure for long-horizon nonlinear dynamic programming. arXiv preprint 157. 2021.
[33] Sen Na, Sungho Shin, Mihai Anitescu, and Vietor Zavala. Overlapping schwarz decomposi- tion for nonlinear optimal control. arXiv preprint arXiv:2005.06677 2020.
[34] Jorge Nocedal and Stephen J. Wright. Numerical Optimization. Springer Series in Operations Research and Financial Engineering. Springer New York, 2nd edition, 2006.
[35] Armin Nurkanovic, Andrea Zanelli, Sebastian Albrecht, and Moritz Diehl. The advanced step real time iteration for NMPC. In 2019 JEEE 58th Conference on Decision and Control (CDC), pages 5298-5305. IEEE, IEEE, dec 2019.
[36] Armin Nurkanović, Andrea Zanelli, Gianluca Frison, Sebastian Albrecht, and Moritz Diehl, Contraction properties of the advanced step real-time iteration. In Proceedings of the IFAC World Congress, volume 51, 2020.
[37] Toshiyuki Ohtsuka. continuation/GGMRSS method for fast computation of nonlinear receding horizon control. Automatica, 40(4):563-574, apr 2004.
12

---

[38] Q. Tran-Dinh, C. Savorgnan, and M. Dichl. Real-time sequential convex programming for nonlinear model predictive control and application to a hydro-power plant. In Proceedings of the IEEE Conference on Decision and Control and European Control Conference (CDC-ECC), pages 5905-5910, December 2011.
[39] Milan Vukov, S Gros, G Horn, Gianluca Frison, Kurt Geebelen, John Bagterp Jørgensen, Jan Swevers, and Moritz Dichl. Real-time nonlinear MPC and MHE for large-scale mechatronic application. Control Engineering Practice, 45:64-78, 2015.
[40] Nolan Wagener, Ching-An Cheng, Jacob Sacks, and Byron Boots. An online learning approach model predictive control. arXiv preprint arXiv:1902.08867, 2019.
[41] Yang Wang and Stephen Boyd. Fast model predictive control using online optimization. IEEE Transactions on Control Systems Technology, 18(2):66-2278 2010.
[42] Andrew Wynn, Milan Vukov, and Moritz Diehl. Convergence guarantees for moving horizon estimation based on the real-time iteration scheme. IEEE Trans. Automat. Control, 59(8):2215- 2221, 2014.
[43] Jianjun Yuan and Andrew Lamperski. Trading-off static and dynamic regret in online least- squares and beyond. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pages 671 6712-6719, 2020.
[44] Andrea Zanelli, Rien Quirynen, Gianluca Frison, and Moritz Diehl. A partially tightened real-time iteration scheme for nonlinear model predictive control. In 2017 IEEE 56th Annual Conference on Decision and Control (CDC), pages 4388-4393, IEEE, IEEE, dec 2017.
[45] Victor M. Zavala and Mihai Anitescu. Real-time nonlinear optimization as generalized equation. SIAM J. Control Optim., 48(3):5444-5477 2010.
[46] Victor M Zavala and Mihai Anitescu. Scalable nonlinear programming via exact differentiable penalty functions and trust-region newton methods. SIAM Journal on Optimization, 24(1):528- 558, 2014.
[47] Lijun Zhang, Tianbao Yang, Zhi-Hua Zhou, et al. Dynamic regret of strongly adaptive methods. In International conference on machine learning, pages 5882-5891. PMLR, 2018.
[48] J. Zhao, M. Dichl, R. Longman, H.G. Bock, and J.P. Schlöder. Nonlinear model predictive control of robots using real-time optimization. In Proceedings of the AIAA/AAS Astrodynamics Conference, Providence, RI, August 2004.
13


Reviews:
[{"review_id": "9Nbi-jkjXG3", "reviewer": "Reviewer_3owW", "summary": "This paper studies a real-time iteration scheme for solving nonlinear MPC. The paper provides an adaptive way of Newton stepsize selection and establishes global convergence. The paper also provides abundant numerical results.", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The paper is well-written and the results are important. One major challenge in nonlinear control is the computation burden and one major strength is that the proposed algorithm can ensure global convergence for time-varying cost functions and dynamical systems only by one iteration of Newton update per time step, which greatly reduces the computation time per stage while still ensuring certain performance guarantee. Some comments are provided below.\n\nFirstly,  It would be better if the authors can comment more on the quality of the stationary points where the RTI converges. For example, how does the converged stationary point compare with solving MPC completely? Is there any way to improve the performance of the stationary points? How do the initial values affect the quality of the stationary points?\n\nSecondly, can the authors provide more discussions on the technical novelty? Right now, the results seem to be a straightforward application of line search plus online Newton updates. \n\nFurther, the stability of NMPC by implementing the RTI scheme proposed in this paper should be discussed. \n\nLastly, can the authors comment on how to extend this to the model-free case?", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Yes", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "2"}, "extra_text": {}}, {"review_id": "6gO2SEUMPxx", "reviewer": "Reviewer_ZaaT", "summary": "This\n paper proposes an adaptive extension of a real-time iteration (RTI) \nscheme used to solve online non linear optimal control problems, with \ntime-varying dynamics.\n\nIt introduces a parametric merit function, named exact augmented \nLagrangian, which is used to adapt the step-size of each unique Newton \nsteps involved to solve the succession of MPC sub-problems. Such \nstep-size follows an Armijo-type condition. Before each back-tracking \nline search, the penalty parameters of this merit function are adjusted \naccording to a well-defined scheme.\n\nUnder generic assumptions, authors prove the global convergence of their\n approach. That is, the KKT residual of t-th sub-problem converges to \nzero as t, the time horizon, increases. Furthermore, for large t, \npenalty parameters of the merit function used are no longer changed. \n\nFinally, the paper proposes two types of numerical experiments. First, \nit compares the performances (in terms of speed and precision) of their \napproach with the one using a $l_1$ merit function to perform a \nbacktracking scheme. Secondly, it analyses the robustness of their \nassumptions with respect to a quadratic penalty parameter $\\mu$, which \nis tuned manually in their approach. \n\nNumerical results show indeed that this adaptive RTI has a greater \nperformance, especially for time-varying problems, and is robust to \n$\\mu$ parameter choice.", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The\n paper is original by the subject it tackles, which is to \npropose an algorithm ensured to globally converge when used with an \nonline non-linear optimal control problem with time-varying dynamics. \nThis task is indeed hard. \n\nIts theoretical contribution is valuable and of some \nsignificance for the community as it provides a first global \nconvergence analysis of RTI schemes in this setting.\n\nThe paper is well written and organized. Explanations are clear. The \nfollowing remarks could enhance the quality and clarity of the \ncontribution :\n  - There should be a reference, or a proof, to the assertion of \nequivalence between being a strict local minimizer of the exact \naugmented Lagrangian and problem 2 (cf. lines  135 to 137),\n  - A reference to the mean value theory and back-tracking line \nsearch would be appreciable (at lines 142-143),\n - The argument for tuning $\\mu$ manually (lines 150-151) is not really convincing as other schemes could be used (not involving solving several times the KKT systems). This part should be more developed.\n  - It is not clear whether or not for large $t$ the augmented \nLagrangian is ensured to be exact with final $\\eta_1$ and $\\eta_2$ found\n (cf. theorem 4.4). Is it the case ? If yes, it should be mentioned somewhere, otherwise, does it mean the property is useless for this global convergence analysis ? It would be worth precising it.\n   - In line 235, I guess authors want to write \"one Newton step\" \nand not \"one step Newton\".\n   - Simulation setups should be enhanced. 5 independent runs are too little to show results of any statistical significance. The mean \nvalue of at least 1000 trajectories (for each different initial point) \nwith standard deviations should be provided. The same applies finally to the robustness analysis of the $\\mu$ parameter. 5 different parameters with a very little scale difference (from 1 to 5) are not significant to prove your claim. The analysis should be done for larger $\\mu$ choices \n(at least $\\mu = 1000$). Indeed, assumption 4.3 involves a parameter \n$c$, which authors \"hope\" being \"small\" (line 207). Or, the global \nconvergence theorem 4.7 assumes $c\\leq 1/\\kappa^3$, with $\\kappa = \\mu /\n \\gamma_{RH}$. It is hence of some importance to analyze numerically the scope of assumptions imposed for $c$. \n\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Authors\n provides limitations and extensions to their work, including more \nprecise constants to be found in theorem 4.4, and several remarks \nprovided in section 6.\n\nYet, as developed in the previous section, the scope of assumptions made\n on parameter $c$ must be more investigated numerically. If it appears \nthat for some problems, a too large $\\mu$ compromises global \nconvergence, then such limitation must be developed consequently. \n\n\n", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "8"}, "extra_text": {}}, {"review_id": "4nDJm1IkgHv", "reviewer": "Reviewer_fJhh", "summary": "This paper gives an algorithm for nonlinear MPC based on augmented Lagrangians, and demonstrates that the iterates converge globally to a critical point of the KKT equations. The key insight is a rule for tuning the regularization parameters of the augmented Lagrangian that ensures that each algorithm step reduces the KKT residuals sufficiently to overcome any increase that would arise due to taking a step in the dynamics. As a result, after alternating between algorithm steps and dynamics steps for several iterations, all iterates produced by the algorithm will be near critical points. ", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The paper is well written, the topic is interesting, and the theory appears to be sound. However, I do have several concerns. \n\nFit\nMy most basic concern with this work is its fit with NeurIPs. This appears to be a work on model predictive control, with very little overlap with the machine learning community. While many control-related topics such as reinforcement learning and system identification have broad overlap with NeurIPs, this work does not. I think a controls conference or journal would be a much better fit. \n\nSignificance\nIt is not clear what global convergence really implies in this context. The KKT residuals go to zero, but what does this mean from a control perspective? It appears that the after many iterates, the solution will at least be nearly feasible. But it is unclear whether the solution is converging to a local minimum. \n\nAlong with this, the assumption that the state and actions stay within a compact set place some stringent requirement a priori that the MPC method stabilizes the system. \n\nEdit after Response: The general consensus seems to be that this is within the scope of NuerIPs, which was my main concern. I do think the work has merit, but there are some limitations, such as closed-loop stability, which are not addressed.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The authors have addressed some of the basic limitations of the work in the conclusion. There does not appear to be any obvious negative societal impact.", "review_text": "", "extra_scores": {"ethical_concerns": "No concerns.", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "2"}, "extra_text": {}}]

Author rebuttal:

Meta-review: Dear authors,

The authors reached a consensus and positively evaluated the paper. Hence I recommend acceptation to the SAC.
I encourage you to pay special care to clarifying the points raised during the discussion regarding stability, MPC and RTI when preparing the final version of the paper.
Decision: Accept (Poster)