Disentangling Linear Quadratic Control with Untrusted ML Predictions
Authors: Tongxin Li
Hao Liu
Yisong Yue
Venue: NeurIPS 2024 poster (2024)
Keywords: Linear Quadratic Control
Disentanglement
Competitive Analysis
Primary area: optimization
TL;DR: Our work introduces DISC, a novel policy that merges online control with learning disentangled predictions, effectively optimizing performance in dynamic systems and maintaining robustness through competitive ratio guarantees.

Abstract: Uncertain perturbations in dynamical systems often arise from diverse resources, represented by latent components. The predictions for these components, typically generated by "black-box" machine learning tools, are prone to inaccuracies. To tackle this challenge, we introduce DISC, a novel policy that learns a confidence parameter online to harness the potential of accurate predictions while also mitigating the impact of erroneous forecasts. When predictions are precise, DISC leverages this information to achieve near-optimal performance. Conversely, in the case of significant prediction errors, it still has a worst-case competitive ratio guarantee. We provide competitive ratio bounds for DISC under both linear mixing of latent variables as well as a broader class of mixing functions. Our results highlight a first-of-its-kind "best-of-both-worlds" integration of machine-learned predictions, thus lead to a near-optimal consistency and robustness tradeoff, which provably improves what can be obtained without learning the confidence parameter. We validate the applicability of DISC across a spectrum of practical scenarios.

Disentangling Linear Quadratic Control with Untrusted ML Predictions
Tongxin Li
Hao Liu
Yisong Yue²
School of Data Science
The Chinese University of Hong Kong, Shenzhen, China litongxin@cuhk. edu. cn
CCmmptting Mathematical Sciences California Institute Technology, Pasadena, USA fhliu3, yyue)@caltech... edu
Abstract
Uncertain perturbations in dynamical systems often arise from diverse resources, represented by latent components. The predictions for these components, typically generated by "black-box" machine learning tools, are prone to inaccuracies. To tackle this challenge, we introduce DISC, novel policy that learns confidence pa- rameter online to harness the potential of accurate predictions while also mitigating the impact of erroneous forecasts. When predictions are precise, DISC leverages this information to achieve near-optimal performance. Conversely, in the case of significant prediction errors, it still has a worst-case competitive ratio guarantee. We provide competitive ratio bounds for DISC under both linear mixing of latent variables as well as a broader class of mixing functions. Our results highlight a first-of-in kind "best-of- soof--othhworrr both-worlds" integration of machine-learne's predictions, thus lead to a near-optimal consistency and robustness tradeoff, which provably improves what can be obtained without learning the confidence parameter. We validate the applicability of DISC across a spectrum of practical scenarios.
Introduction
We study the problem of online decision meaiiinnnmmkinggg with predictions. Such settings are increasingly popular with the use of machine learning for predictive modeling, with applications to power grids and robotics [ 2 3 However, in many real-world settings, such black-box predictors can be unreliable due to practical problems such as high model variability [5] 6 and out-of-distribution generalization issues 1788. In settings that require making reliable and high-quality decisions, recent research has been working on designing the decision-makeee policy that can be adaptively in response to receiving low-quality predictions (4.
In this paper, we consider the setting where there are different sources of disturbance, each represent- ing distinct (and often independent) resources or factors. For instance, in electricity grids, machine fluctuations, electricity
learning (ML) forecasts of variables like power consumption//eneraaie prices, and battery states can facilitate near-optimal operational decisions on the one hand, but on the other hand, latent variables such as power injections on certain loads can be highly unpredictable due to their volatility 99. Another example is the drone navigation task [00] where the challenge
*Corresonndence to: Tongxin Li <litongxin@cuha edu, en>, Tongxin Li contributed to the design of the algorithms and experiments, and the writing of the paper. Hao Liu focused on conducting the experiments and data analysis. The contributions are equal.
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

lies in managing external perturbations caused by mixture of predictable elements like air flows and less predictable factors like raindrops. When performing online decision- making in this setting. one important technical challenge is to be able to combine disentanglement with an adaptive online learning algorithm to result in a more exact estimation of the trustfulness of the disturbance. To address this challenge, in this work, we focus on linear quadratic control problem where system perturbations originate from unidentified and possibly heterogencous latent resources/componens.
Contributions. We develop novel policy DIsC that does the disentanglement and learns the adaptive parameter online simultaneously. We first introduce a policy, Å-CON, which extends the X-confident approach in [4] by adapting vectorized confidence parameter À E [0, 1J where each X()) represents the estimated trustworthiness of the MI. prediction for the i-th latent variable in the dynamical system. We then show that a static À cannot guarantee an optimal consistency and robustness tradeoff (see Definition 2.2 for formal description) for X-CON: if X-CON is (1 + o(1)- consistent, then it is at least w (robbrs.stt
To circumvent this limitation, we propose the dynamic policy DISC (Section 3.2 This algorithm leverages online learning to optimize the confidence parameter X at each time L. We establish competitive ratio guarantees for DISC in both linear and general mixing scenarios, as shown in Theorems 4.2 and 4.3 (Section (4) respectively. Under Assumption [ [ndd The competitive ratio bound for Disc, outlined informally as follows, incorporates term that embodies our "best- "best-of-botts of-both- worlds utilization" of untrusted MI. predictii:ons::
( E [CR(Dis))) s 11 1 o()) +0 (p²)) +0 ((T/W) + 3(1)
Σ
Best-of-both-worldd atilization
where the o(1) term hides quantities that vanish when the total number of steps T increases; is the number of latent variables generating the perturbations; p (0, 1); w denotes the prediction window size and each e(i) (for = 1....,1, k) denotes the prediction error corresponding to each latent component. The last term in this result highlights the desired performance guarantee: When a component-wise prediction error E(i) is small, the individual term E(i)/ (S(T/w) + E(i) will be negligible; otherwise, it always holds that Ei)) (M(T/w) /w) + +3(1)) S regardless of how high the prediction error becomes. Our proposed DISC is both (1 + o -coossctn and 0(1) (r)bust with a sufficiently large prediction window size. We offer the first bound of this nature, grounded in a term as function of the prediction error, without restrictive assumptions on the errors (E(1)....,
Proving our main result above is nontrivial due to the fact that despite it is known that input- disturbed linear system can be reduced to an online convex optimization (OCO) with structured memory M. the connection between the problem with A-CON and memoryless online optimization is not previously discovered. In Lemmal I we provide a result that decouples the loss terms in the total regret in (9) and future perturbations, thereby reducing the problem of choosing A an online optimization instance. Then we use a two-stage analysis as depicted in Figure 7 that combines a dynamic regret bound for our control policy and static regret bounds induced by online learning algorithms to derive the main result.
We demonstrate the practicality of DIsC through two real-world examples (Section 5): drone naviga- tion problem with mixed external disturbances and voltage control in a power grid with heterogeneous power injections. We demonstrate that DIsc, when applied with disentangled ML predictions, out- performs baselines that do not distinguish between underlying latent variables. Additionally, DIsc shows remarkable adaptability to rapid changes in various scenarios, such those involving ML models trained with out-of distriibbttii data in non-stationary environment.
Related Work. Our work contributes to the growing community of algorithms with predictions while also incorporating ideas from adaptive control, online learning, and disentangled representation learning. We summarize the detailed related work in Appendix A
2 Problem Formulation
Notational conventions. Throughout this paper, || denotes the fn-n for vectors and the matrix norm induced by the norm. We use subscript (-), represent length-k vector s; =

---

Disentangled Latent Predictions
Previous Perturbation
f()11;00
(n((1) :
K)
Control Policy
Azol Ax, + Bu, +
Untrusted ML Agent
Observed State
Dynamical Systems
Figure 1: Overview of the system model considered in this work. Left time series: Disentangled latent variable time series predictions provided by an untrusted ML agent at time E [T]. represented by (f)e(0) : =.,...... l.....k): k); Right time series: Observed mixed perturbations (ʃ (8ri 0) () at time 1.
.,. , (k))) at time whose i-th coordinate is written as st)) We use Aos (X()ss()))) k) 1,...,
E IR to denote the Hadamard product between vectors X, RR
2.1 Linear Quadratic Control with Latent Perturbations
We consider a finite-time linear dynamical system with latent perturbations. Let [T] == (o...., . 7 --13 be the set of time steps. Denote by xa R. system state and ue I an action from state feedback policy Ne at time f E [7]. The system update rule is given by
T++1 Ax, + Bu, + (s;;0, [T
where 5, == (se(1), (A)) E R (A: S n) contains k latent variables (components) that together generate a perturbation at time t [T] via mixing function f R > R" parameterized by observed after being generated, while the mixing function
I E θ C RRd. The states and the actions I transforms the unobservable latent variables in 8 to an additive system perturbation. Throughout this paper, we focus on the nontrivial regime that fs; 0) O,, for all [T].
In (2. A E gaxx and B E gaxm are system matrices. We consider the standard regime that the pair (A, B) is stabilizable [ Without loss of generality, we also assume the system is initialized with some fixed zy E RR. The goal of control is to minimize the following quadratic costs given matrices B, R:
((T)= Σ (x) Qu, + un Rui) + x PET,
where (me : [T]): Q 1n E mmxm are positive definite matrices, and P is symmetric positive definite cost-to-go solution of the following discrete algebraic Riccati equation (DARE), which must exist because (A, B) is stabilizable and Q, R are positive definite [12]
P Q A PA-A" A PB(R B (BB) BB PA
Given P. we define == (R+ P PA, and u4 -rx i is the feedback control law of a linear quadratic regulator (L.QR), which is optimal in the case of zero disturbances (w, = 0 for all t [T). Further, let == A BK be the closed-loop system matrix. The Gelfand's formula implies that there must exist constant C' > 0 and spectral radius pF (0, 1) such that ||F*|| < Crpp for E [T')
In this work, we focus on designing near-optimal policy π = (Te t (T') that minimizes the total quadratic cost in (3) subject to the linear dynamics in (2). At time E [T). each n. R2 x R2* + IR fate-fee policy that produces an action #4 Ih with observed state 3;-1, and MI. predictions (t) i = 1, 1,) of the latent variables 8, E IR= for future w time steps with a prediction window size w > (denoting F := min(t- + w - 1, T 1)). Figure above summarizes system model. We defer detailed introduction of the ML predictions in Section 2.2 together with the performance benchmarks considered in this paper. We relegate concrete latent perturbation
modelling examples and real-world applications Appendix B
In light the existing nonlinear ICA models summarized in Appendix B throughout this paper, assume the continuity and invertibility conditions hold for the mixing function f to guarantee
3

---

our theoretical results. This assumption is weaker compared to those required in the nonlinear ICA literature [3]
16 T7 to guarantee identifiability, as summarized Table IT (see Appendix B noting that our goal is instead to provide near-optimal competitive ratio bound regardless of the error of disentanglement and latent variable predictions (see our performance benchmarks defined in Section 2.2
14
15
Assumption 1. The mixing function f IR x θ + I2 is Lipschitz continuous and bijective with respect to E 1
2.2 Performance Benchmark
Let Srjt be the latent variable time series value at time т, predicted by an untrusted ML agent at time
I I r. The prediction window size is an integer wo > 0. We write == min(L + w 1, 1}. The
estimated mixing parameter at each time t [I'] is denoted by 0.. Define the following error terms for S r and E [T]]
(E+)e Er (k)) = Etjt == s, 3t/t T == 8 A.
(4)
To assess the cumulative impact of prediction error over all T time steps, we define the total prediction error corresponding to the component-we latent variable time-series prediction and the mixing parameter estimation as follows:
T E(i) = (o (e)())))))) 1=
7=
|lnel
t=0
where pF E (0,) 1) represents the spectral radius of the matrix F. The term PF in (5) accounts for the exponential decay phenomenon in the impact of component-wise errors, implying that predictions further into the future contribute less to the total error. This approach to error measurement is analogous to methods employed in linear quadratic control models with adaptive offline adversarial perturbations, as discussed in [4), which provides foundational understanding of error evaluation in our context.
Our performance benchmark is the competitive ratio for given prediction error defined as follows. To be more precise, we focus on the competitive ratio defined for deterministic online algorithms with an adaptive offline adversary that selects the system parameters A, B, Q, R, f. 0 and latent time
series (st : / E [T]). Write ε == (E(1)= : 1, ....., ,,... k) and denote by J e) the total cost obtained by implementing π (T, E (T]) with some fixed prediction error s.
Definition 2.1. Fix some prediction error e. The competitive ratio CR(T;; is defined as the smallest constant C 2 such that (TT;E) < C- for all A, B, Q, R, f.0, and (st tt [T]) satisfying the model assumptions.
The following notions of consistency and robustness with respect to the competitive ratio offer concise characterization to measure the algorithmic performance in the presence of prediction error e. It aligns with the benchmarks used in the growing literature on algorithms with predictions [I8 19
20 21
(see further discussions provided in Section A).
Definition 2.2. A policy π is 7-consistent if its competitive ratio satisfies CR(T))) S 7 for and k-robust if CR(m;)) < for all
3 Disentangled Confident Policy
In this section, we present our policy, DIsC, which takes disentangled and untrusted ML predictions and achieves near-optimal consistency and robustness tradcoff (see Definition 2.2
We define pairs (s, 0) and (s', 0') consistent if they yield identical outputs in the mixing function i.e. f(s; 0) f(s': 0'). For simplicity in our notation, we will treat consistent pairs (s, and (s',8') as indistinguishable when they produce the same perturbation effect in the system. Here, at each time step t E [2'] the estimated mixing parameter and the time series predictions are compared against the consistent pairs 0 and with the smallest prediction error.

---

Algorithm 1: DiSentangled Confidence (DISC) policy
for 0,..., 1 do
[ f- min
1, - 1/
/+ Online learning of A
if = 0 then Initialize Ào I else Set A DONLINEPROCEDURE (Ge é (f]) through (12
13 with Çe defined in
/+ Obtain disentangled predictions
Get (ā+je 1 + S t) and an estimated mixing parameter 0: /* Implement disentangled X--confident policy
Take My as (6) with a learned )y such that
Kri Y Σ ( (
Pf (Ae 001110)), (A--CoN POLICY)
Update and observe Te+1 following
end
3.1 Warm-Up: Disentangled X-Confident Policy (A-CoN)
Before proceeding to introduce DISC, we first consider a policy that balances consistency and robustness by combining an MPC policy that fully utilizes the untrusted MIL predictts,,ns, and an LQR without considering any predictions.
Denote by == min(t+ + w - 1, 1) and let Y = BT++-PPD-11 Recall that as defined in Section 2.2 each time E [T]. (57je: :t S S t) represents sequence of predictions for the future values of the latent variables (s, : S T S !). with these predictions spanning a predetermined window size w > O. Similarly, the sequence (f, : [T]) corresponds to the estimated mixing parameters. At each time t E [7'). an estimate 0g is obtained by applying some disentanglement algorithm (see those examples Section B). With this setup, we proceed define an MPC-type action as follows:
Ur KK Σ (FT)T* PS PS (A03)1100)). (A-CON POLICY)
(6)
which specifies a disentangled X-confident policy, denoted by X-CON, where X I= [0, 1] fixed trust parameter. The term o 871 denotes the Hadamard product of and 5-kt- is worth noting that it is well known that (6) is an optimal solution of the following MPC scheme 4 2
min mm
Σ
( Qx, + u. Ru,) 2 PTz+1
x++1 Ar, + Bu, + (x08 E [t,7]
(8)
In particular, if is an all-one vector 1; in 77. trusts the ML predictions. Note that it generalizes the X-confident policy in [4], which linearly combines actions from an MPC policy and an LQR. In Section 4.1, we present a negative result for A-CON, indicating that it cannot achieve the optimal consistency and robustness tradeoff (see Definition 2.2 This motivates the disentangled confident policy discussed in the next section.
3.2 Disentangled Confidence Policy (DIsc)
At each time t E [T], the algorithm adaptively learns a trust parameter X₂ II [0, 1]a, and uses a Hadamard product of X and the ML learned latent variable X 5 estimate future perturbations. The update rule of the trust parameter Ae follows an ONLINE-PROS PROCEDURE, which can be constructed by the following explicit form of the overall dynamic regret (see 22 with fixed À E I:
J(m(A)) - J Σ (e)
(9)
5

---

where = B(R B PB) P 'B" and deer (A) is defined as
Σ (F p (f((s):))) - (FT) (10
(E)
(10)
Consider the application of online optimization strategies to minimize total regret depicted in At each time I, we obtain (A). tr.e For (A), since the summation is over TEl {e, min{e + - 1, 13}. V'e, formulation of the offline optimization (9) does not conform to a canonical framework suitable for online optimization. This necessitates a tailored approach for online learning, which is facilitated by Lemmaa 1, allowing for the online learning of X. Below we assume the following to regulate the [T]. which holds for typical online learning algorithms with stationary learned sequence (^g Note that when T < O, we consider A = λo
IR" is function that depends / [T]. Thus, the
environments 23 24 25
Assumption 2. If > is a constant, then (Ae E (T]) satisfies that ΣT- AAe A o(T).
Define & max to an online optimization problem. The proof can be found in Appendix D.3
1.. The following lemma helps convert (9) to form that can be reduced
Lemma 1. Define a function Çe I R as Ce(A) = Er (T, ()HgTT where
g(t, = (FT) p (s(:)))))
- ( (x03
(11)
Then for any t E [T]. it follows that Ei (A)HH S Et=os Se.
Note that each (e()) defines convex funetion of X, as verified in Appendix D.1 Therefore, the selection of X, at each time t E [T] follows from an online learning procedure ONLINE- PEILINE-ROOEDUREE whose concrete realizations include the follow-the-regee www.tthe-rgglarizee-eeeeee (FTRL) approach with an f2-norm regularizer (23 24], which is equivalent to online mirror descent (OMD) when f is convex; and the follow-the-perture pertrr-eeeerrrrelledeeee (FTPL) 26 27 28 25 for online non-convex learning, taken previous quantities (Se & E [f]) as the input. Next, we will discuss theoretical guarantees for linear and general mixing models with specified online learning procedures, as detailed later in Section Our policy is summarized in Algorithm
4 Main Results
In this section, we present technical results, proving that the scheme in DISC achieves near- optimal competitive ratio bounds for both linear and general mixing cases. Proving our main result above is nontrivial due to the fact that despite it is known that input-disturbed linear system can be reduced to an online convex optimization (OCO) with structured memory [. the connection between the problem with A-CON and memoryless online optimization is not previously discovered. In Lemma [ we provide a result that decouples the dependency between cost functions in our problem that depends on previous actions via linear dynamical system (see the dynamical system defined in (2), thereby reducing the problem of choosing A to an online optimization instance, then use two-stage analysis (see Figure] 7 that combines the dynamic regret analysis of X-CON and static regret bounds corresponding to applying online optimization algorithms to learn a confidence parameter.
4.1 A Fundamental Gap
online. Based on Definition 2.2 the following theorem reveals a negative result of X-CON, since for any fixed non-zero X, there always exists predicted time series (žyje :l S T, (T)) such that the competitive ratio CR(A-CON) for X-CON can be unbounded (when 7 goes to infinity).
First, we motivate the necessity of learning (Ae : [T))
Theorem 4.1 Mansiisenny-Robustneen Impossibility for CR(X-CON)). If the X-confident policy X-CON is (1 + o(1))-consisten, then least -(o)srobust, even if the mixing parameter estimate is perfect, i.e., π 0.

---

Here, o(-) and w(.) characterize the asymptotic asymptotic growth rates with respect to the time horizon length T. Next, we show in both the linear and general mixing settings, CR(DISC) can be bounded as previewed in Section 1 thus the gap between CR(A-CON) and CR(DISC) can be arbitrarily large as implied by Theorem 4.1 above. Especially, Corollary B.1 Section (..2 implies that DISC is both (1 + o(1))-consistem) and -robust, with a sufficiently large prediction window size w w11), revealing that online learning of the confidence parameter provides a significant improvement of the consistency and robustness tradeofT.
4.2 Linear Mixing
We consider linear mixing setting, where the mixing function linear such that f(; As for all E R where 0 = (0:z) denotes x l full column rank mixing matrix. Suppose (s, t (T]) and 9 bounded such that || lI I s for all & [T'] and |@]]| < B. For this particular instance, we implement (follow-the tew-ttherrre hitee--gularized--eeee
FTRL procedure that sets
Ae E arg min XEZ
Σ VI (co())) A
1 J soll -011)
(FTRL FOR X-LEARNING)
(12)
at each time E [T] for some 3 > O that can be optimized. The DIsC policy satisfies the following competitive ratio bound, whose proof can be found in Appendix
Theorem 4.2 (DISC for Linear Mixing). With our model assumptions, the competitive ratio of DIsC
with linear mixing function satisfies
Σ S(T/W) E(i)
k
E(I)
k
2t0 π T
CR(DISC) S $1+0
+0
W
+ O
PF
where k is the number of latent variables; T is the time horizon length; E (0, 1) is the spectral radius of F; (E(i) : I.....k) .)) and πj are defined in (5 O(-) and 526.) hide multiplicative constants (see the details in Appendix
This result highlights a consistency and robustness tradeoff DISC, as claimed in Section [ which offers dual advantage: it exploits accurate predictions when available, and safeguards aganst the ramifications of trusting inaccurate forecasts, thus ensuring system performance reliability. Besides, our analysis can be carried out to convex mixing functions, as the online learning of X, in (12) via FTRL guarantees sub-linear static regret as long as (e(A) convex in for all E [T]. We further provide bound for sample efficient ICA in the appendix.
4.3 General Mixing
In this section, we extend the linear assumption on f by considering general setting where the mixing function f is an arbitrary Lipschitz continuous function (see Assumption []. To deal with the
non-convexity of f. we revisit the (follow-the- tethe-perturbedleeaae leader) FTPL approach (see 28 25 and consider the following online learning procedure for tuning (Ae t E [T]):
EG((A) (G(A))ooT)) +OTA) L=D
Xe E arg min
(FTPL FOR X-LEARNING).
(13)
XEI
Here, oe is length-k random vector with each coordinate (i) (i = 1,..., variable from an exponential distribution. The DISC policy satisfies the following competitive ratio bound.
. k) being IID random
Theorem 4.3 (DISc for General Mixing), With our model assumptions, the expected competitive ratio of DISC. satisfies
E [CR(DISC) S 1+0
+0 30F
17
+
wk? VT
(14)
(TTWw) + E1)
where the parameters pF, w. k T.H and (e(1) = 1....,K ...., are the same as in Theorem and ||se|| for all t [T], The expectation is taken the randomly sampled (or [T]).
7

---

The notation (-) and ()-) above in Theorem 4.3
hide multiplicative constants such as the Lipschitz constant of the mixing function f (details are provided in Appendix H.. The result above implies the best-of-both-wee worr-booth-words prediction utilization previewed in Section except that the online learning of d converges slower with a rate k², causing the term O(k² ( in the competitive ratio bound (14). Unlike the linear setting, ensure near-optimal expected competitive ratio, the latent time series (st : [T]) is not necessarily bounded, as long as the prediction window size is sufficiently large (e.g., w = (log T) so that the term 8 vanishes. Finally, note that the online learning procedure of X is not limited to FTPL. For example, the non-convex online optimization algorithm in [29] can be used to optimize (A, : [7]) online and leads to a similar guarantee. The proof of Theorem 4.3 can be found in Appendix
5 Experiments
Experimental Setup. To generate ML predictions, we use the FastICA method in (30] to decompose the mixed perturbations and train multi- -lyter-larrrrrr perceptron neural network with 4 hidden layers to pre- dict the future latent variables. FastICA a simple and efficient linear ICA method that aims to find an orthogonal rotation of the observation that maximizes the rotated components' non-Gaausiannty using fixed-point iteration. Furthermore, in our experiments, we implement a follow-the-regulan follow-tieerrgullrizeedeadd
(FTRL) optimization with fz-regularizer || Ao 2 which is equivalent to the following equivalent
online mirror descent (OMD) implementation [24] Tr = Tt-1 z
_ V (s, HG). A = (T.)
The detailed parameters and hyper-parameters used in our experiments be found in Appendix C.3
The code is available at
5.1 Experiment Drone Navigation with Mixed Disturbances
Self-turing polley
Leaening stabiize
(untrunted)
Windy and Rainy
12
DICS (thic work)
intensity
Ratio of Costs JG3l
f (71)
Wind speed
Offiine
Online tracking trojectory
0.25
04
0.75
Scaling Factor
Figure Drone Navigation under challenging windy and rainy weather conditions, with drone's target path illustrated by dotted curve. The external perturbations impacting the drone's flight are modeled by two independent, tire-varying latent variables: we for wind speed and r, for rain intensity at time E [T]. The trajectory produced by the DIsC policy (Algorithm I in Section 3
represented by the blue curve, while that from the offline optimal policy is traced in red. More comparison results with other baseline policies can be found in Appendix C.2 Right: Ratio of Costs J(x)/J* (y-axis) between DISC (red), linear quadratic regulator (LQR, orange), model predictive control with untrusted ML predictions (MPC (UNTRUSTED), green), and the self-tuning policy (blue), with a varying scaling factor y from 0.25 to 8. Shadow area depicts the range of standard deviations for 5 random tests.
We first consider the drone piloting task described in Example [ detailed description is delegated to Appendix B to track an unknown trajectory, shown on the left of Figure
Competitive Ratio Analysis. We present a comparative analysis of the ratio of costs achieved by DISC policy (detailed in Algorithm i Section on the right of Figure against several benchmarks: the Linear Quadratic Regulator (LQR), Model Predictive Control (MPC) using untrusted ML predictions (i.e., setting
1, in Equation 77), and the self-tuning policy outlined [4] and [I].
We examine the impact of varying scaling factor in the range 0.25 to 8, adjusting the additive perturbation rz02 in (T6) (ra/w)c2. This scaling factor v serves as a proxy for different rain intensi- ties, influencing the level of environmental unpredictability- larger is, the more predictable the

---

environment. With the theoretically guaranteed best-of siat-of-bothh bothhworkde utilization of disentangled and untrusted ML predictions, DIsC consistently outperforms existing baselines in terms of cost ratios across varying levels of environmental predictability.
5.2 Example B: Voltage Control with Heterogeneous Power Injections
We consider the voltage control task outlined in Example 2 for distribution power grid with 11 nodes, as depicted in Figure S Specifically, nodes 3, 5, and are active, each supplying dynamic active power injections (p((t)
3, 5, 8) at every time step t [T]. Collectively, these nodes induce the composite perturbation Gp, + 2n The objective is to apply the DISC algorithm to determine near-optimal controllable reactive power injections that will effectively mitigate voltage fluctuations across the grid. More details of the problem setting can be found in Appendix C.4
Consumption
Incodder Toaiier Wind Generation
Solar Integration
10
Figure 3: Voltage Control with a dynamic environment. This figure illustrates the heterogeneity and variability of power injections at different nodes within electrical grid, including residential area, solar photovoltaic (PV) system, and a wind turbine. Right: Example B. Voltage Control (Section 5.2) Left column: Convergence of confidence parameters X(1), A(2), A(3) corresponding to latent components. Right column: Temporal dynamics of latent time series st in R. We illustrate
the time series data for three key latent variables in the energy system (see Figure 3 Gaussian- distributed residential consumption (bottom), real-world photovoltaic (PV) integration (top), and wind generation (middle). The red dotted line marks critical transition point where there is notable shift in the power generation patterns. We use time series before the blue dotted line as the buffered data to warm start the ML model, learn the mixing matrix, and obtain disentangled predictions. The x-axis in each graph marks the time steps, while the y-axis denotes the values of the time series.
We consider realistic scenario when and use this to demonstrate the adaptivity of DIsC in changing environments. Figure 3 shows the change of patterns corresponding to the solar integration and wind generation. We set T 200. At time t 110, the solar generation switches from regular pattern to generating random Gaussian noise, representing miscommunication or system faults. Similarly, at time f 100, the wind generation is recovered from the irregular mode regular mode. For the regular solar and wind generation time series, we use the real solar PV generation time series data from the DTU-Data in 2021
32 and wind generation from the U.S. Virgin Islands Wind Resources from the National Renewable Energy Lab (NREL) 33
Adaptability Amidst Environmental Variability. Figure exhibits the adaptability of our ap- proach in response to environmental shifts in renewable energy generation and consumption patterns. Specifically, the figure presents the time series for solar generation (L) : E [T]), wind generation (2x(2): E [T]), and residential power consumption (sr(3) E [T]). The transition points in the time series patterns are marked by dotted red lines in the figure.
notable observation the reactive behavior of the confidence parameters to the changes in predictability within each time series. For instance, as the solar generation transitions to an irregular Gaussian noise profile > 110, this unpredictability precipitates decline the corresponding confidence parameter (1). Conversely, the wind generation time series exhibits move towards a more predictable pattern past 2 100, resulting in a progressive increase the confidence parameter >e (2). In the case of residential consumption, which is consistently modeled Gaussian

---

noise, the algorithm adaptively learns to reduce the confidence parameter A(3). The resilience of the Disc algorithm in dynamically adjusting confidence levels demonstrates its robustness against environmental shifts, critical feature for real-world applications where conditions are subject to sudden and unpredictable changes. This further highlights the efficacy of DIsC in maintaining system stability and performance through intelligent adaptability to the reliability of input data, as verified by the theoretical results in Section
Concluding Remarks
Our work combines online control and the concept of learning disentangled predictions. In this paper, we have introduced novel policy DISC that achieves the best-of-both-wordd utilization of untrusted ML predictions of latent variables for linear control problem, where the advantage of disentanglement is theoretically validated. The practicality of our method has been validated through two real-world applications, which exemplify its relevance and adaptability to complex, real-world scenarios.
Limitations and Future Directions. Despite that our main results, i... the competitive ratio bounds sy (k)) are
in Theorem 4.2 and 4.3 do not rely on the assumption that the latent variables independent, most of the disentanglement methods do (like sample efficient ICA algorithms used to derive Corollary B.I and the detailed assumptions are summarized in Table T It would be interesting explore more carefully designed algorithms to guarantee both sample efficiency and identifiability based on our control model. Looking ahead, our results open several intriguing paths. An immediate extension of particular interest adapting methodology nonlinear dynamical systems, which could strengthen our current results. Furthermore, our work contributes to the growing community of algorithms with predictions, compelling question arises: can the best-of-bothwworkde competitive ratio bounds we have achieved be replicated across different online decision -making problems? Obtaining either positive or negative results would potentially lead to broad implications for the field of robust control and online learning.
10

---

Acknowledgement
We would like to thank all the anonymous reviewers for their helpful comments.
Tongxin Li was supported in part by the National Natural Science Foundation of China (NSFC) under grant No.72301234, the GuangDong Basic and Applied Basic Research Foundation, NSFC under grant No. 62336005, the PengCheng Peacock Supporting Scientific Research Fund Category C (2024TC0024), the GunngDong Provincial Key Laboratory of Mathematical Foundations for Artificial Intelligence (2023B 121 122010001), and the Shenzhen Key Lab of Crowd Intelligence Empowered Low-Carbon Energy Network (No. 2 010S522022000000000000002 00601002).
References
[1] Guanya Shi, Yiheng Lin, Soon-Jo Chung, Yisong Yue, and Adam Wierman. Online optimization with memory and competitive control. Advances in Neural Information Processing Systems, 14:20636-20647 2020.
[2] Chenkai Yu, Guanya Shi, Soon-Jo Chung, Yisong Yue, and Adam Wierman. The power of predictions in online control. Advances in Neural Information Processing Systems, 33, 2020.
[3] Tongxin Li, Yue Chen, Bo Sun, Adam Wierman, and Steven Low. Information aggregation for constrained online control. ACM SIGMETRICS Performance Evaluation Review, 49(1):7-8, 2021.
[4] Tongxin Li, Ruixiao Yang. Guannan Qu, Guanya Shi, Chenkai Yu, Adam Wierman, and Steven Low. Robustness and consistency in linear quadratic control with untrusted predictions. ACM SIGMETRICS Performance Evaluation Review, 50( 1:107-108, 2022.
[5] Tobias Johannink, Shikhar Bahl, Ashvin Nair, Jianlan Luo, Avinash Kumar, Matthias Loskyll, Juan Aparicio Ojea, Eugen Solowjow, and Sergey Levine. Residual reinforcement learning for robot control. In 2019 International Conference on Robotics and Automation (ICRA), pages 0023-6029. JEEE, 2019.
[6] Benjamin Recht. A tour of reinforcement learning: The view from continuous control. Annual Review of Control, Robotics, and Autonomous Systems, 2:253-279, 2019.
[7] Chaochao Lu, Yuhuai Wu, José Miguel (tenánddezLLobtto and Bernhard Schölkopf. Invariant causal representation learning for duitr--diitribution generalization. International Conference on Learning Representations, 2021.
[8] Florian Wenzel, Andrea Dittadi, Peter Gehler, Carl-Johann Limon-Gabriel, Max Horn, Dominik Zietlow, David Kernert, Chris Russell, Thomas Brox, Bernt Schiele, et al. Assaying out-of- distribution generalization in transfer learning. Advances in Neural Information Processing Systems, 35:7181-7199 2022.
[9] Evan D Sherwin, Max Henrion, and Inés ML Azevedo. Estimation of the Sear-on-yeas volatility and the unpredictability of the united states energy system. Nature Energy, 3(4):341-346 2018.
[10] Tiago Oliveira, A Pedro Aguiar, and Pedro Encarnacao. Moving path following for unmanned aerial vehicles with applications to single and multiple target tracking problems. IEEE Transac- tions on Robotics, 1265):102 2016.
[11] Sungho Shin, Yiheng Lin, Guannan Qu, Adam Wierman, and Mihai Anitescu. Near-optimal distributed linear-quadratic regulator for networked systems. SIAM Journal on Control and Optimization, 61(3): (113):111311111 113-1 2023.
[12] Geir E Dullerud and Fernando Paganini. A course in robust control theory: a convex approach, volume 36. Springer Science & Business Media, 2013.
[13] Aapo Hyvarinen and Hiroshi Morioka. Unsupervised feature extraction by time-contrastive learning and nonlinear ica. Advances in neural information processing systems, 29, 2016.
[14] Aapo Hyvarinen and Hiroshi Morioka. Nonlinear ica of temporally dependent stationary sources. In Artificial Intelligence and Statistics, pages 460-469. PMLR, 2017.
11

---

[15] Ilyes Khemakhem, Diederik Kingma, Ricardo Monti, and Aapo Hyvarinen. Variational au- toencoders and nonlinear ica: A unifying framework. In International Conference on Artificial Intelligence and Statistics, pages 2207-2217. PMLR, 2020.
[16] Xiaojiang Yang, Yi Wang, Jiacheng Sun, Xing Zhang, Shifeng Zhang, Zhenguo Li, and Junchi Yan. Nonlinear ica using volume-preserne preserviig transformations. In International Conference on Learning Representations, 2021.
[17] Yujia Zheng, Ignavier Ng, and Kun Zhang. On the identifiability of nonlinear ica: Sparsity and beyond. Advances in Neural Information Processing Systems, 35:164 (1641111660 1-16422, 2022.
[18] Manish Purohit, Zoya Svitkina, and Ravi Kumar. Improving online algorithms via ml predictions. In Advances in Neural Information Processing Systems, pages 9661 9661-9670, 2018.
[19] Alexander Wei and Fred Zhang. Optimal robustness-consisstnnyy trade-off's for learning- augmented online algorithms. arXiv preprint arXiv:20100/1443, 2020.
[20] Shom Banerjee. Improving online rent-or- buy algorithms with sequential decision making and ml predictions. Advances in Neural Information Processing Systems, 13:21072-21088 2020.
[21] Nicolas Christianson, Tinashe Handina, and Adam Wierman. Chasing convex bodies and functions with black-box advice. In Conference on Learning Theory, pages 867-908. PMLR, 2022.
[22] Chenkai Yu, Guanya Shi, Soon-Jo Chung, Yisong Yue, and Adam Wierman. Competitive control with delayed imperfect information. In 2022 American Control Conference (ACC), pages 2604-2610. IEEE, 2022.
[23] Adam Kalai and Santosh Vempala. Efficient algorithms for online decision problems. Journal of Computer and System Sciences, 71(3):291-307 2005.
[24] Elad Hazan and Satyen Kale. Extracting certainty from uncertainty: Regret bounded by variation costs. Machine learning, 80:165-188, 2010.
[25] Arun Sai Suggala and Praneeth Netrapalli. Online non-convex learning: Following the perturbed leader is optimal. In Algorithmic Learning Theory, pages 845-861. PMLR, 2020.
[26] James Hannan. Approximation to bayes risk in repeated play. Contributions to the Theory of Games, 3:97-139, 1957.
[27] Walid Krichene, Maximilian Balandat, Claire Tomlin, and Alexandre Bayen. The hedge algorithm continuum, In International Conference on Machine Learning, pages 824-832. PMLR, 2015.
[28] Naman Agarwal, Alon Gonen, and Elad Hazan. Learning in non-convex games with an optimization oracle. In Conference on Learning Theory, pages 18-29. PMLR, 2019.
[29] Lin Yang, Lei Deng, Mohammad H Hajiesmaili, Cheng Tan, and Wing Shing Wong. An optimal algorithm for online non-convex learning. Proceedings of the ACM on Measurement and Analysis of Computing Systems, 2(2):1 2018.
[30] Aapo Hyvärinen and Erkki Oja. Independent component analysis: algorithms and applications. Neural networks, 13(4-5):41144 2000.
[31] Yiheng Lin, James A Preiss, Emile Timothy Anand, Yingying Li, Yisong Yue, and Adam Wierman. Online adaptive policy selection in time-varying systems: No-regret via contractive perturbations. In Thirth-seventh Conference on Neural Information Processing Systems, 2023.
[32] Matti Juhani Koivisto and Juan Pablo Murcia Leon. Solar PV generation time series (PECD 2021 update). DTU-data, 5 2022.
[33] Joseph Owen Roberts and Adam Warren. Us virgin islands wind resources update 2014. Technical report, National Renewable Energy Lab.(NREL, Golden, CO (United States), 2014.
12

---

[34] Mohammad Mahdian, Hamid Nazerzadeh, and Amin Saberi. Online optimization with uncertain information, ACM Transactions on Algorithms (TALG), 8(1):1-29, 2012.
[35] Dhruv Rohatgi. Near-optimal bounds for online caching with machine learned advice. In Proceedings of the Fourteenth Annual ACM-SIAM Symposium on Discrete Algorithms, pages I 1834-1845. SIAM, 2020.
[36] Thodoris Lykouris and Sergei Vassilvitskii. Competitive caching with machine learned advice. Journal of the ACM (JACM), 68(4):1-25, 2021.
[37] Sungjin Im, Ravi Kumar, Aditya Petety, and Manish Purohit. Parsimonious learning-augmenter caching. In International Conference on Machine Learning, pages 9588-9601 PMLR, 2022.
[38] Antonios Antoniadis, Christian Coester, Marek Elias, Adam Polak, and Bertrand Simon. Online metric algorithms with untrusted predictions. In International Conference Machine Learning, pages 345-355. PMLR, 2020.
[39] Etienne Bamas, Andreas Maggiori, and Ola Svensson. The primal-dual method for learning augmented algorithms. arXiv preprint arXiv:2010. /632, 2020.
[40] Keerti Anand, Rong Ge, Amit Kumar, and Debmalya Panigrahi. Online algorithms with multiple predictions, In International Conference on Machine Learning, pages 582-598. PMLR, 2022.
[41] Pengfei Li, Jianyi Yang, Adam Wierman, and Shaolei Ren. Robust learning for smoothed online convex optimization with feedback delay. Advances in Neural Information Processing Systems, 36, 2024.
[42] Tongxin Li, Yiheng Lin, Shaolei Ren, and Adam Wierman. Beyond black-box advice: learning- augmented algorithms for mdps with q-value predictions. Advances Neural Information Processing Systems, 36, 2024.
[43] Jianyi Yang, Pengfei Li, Tongxin Li, Adam Wierman, and Shaolei Ren. Anytime-competitvve reinforcement learning with policy prior. Advances in Neural Information Processing Systems, 36, 2024.
[44] Yiheng Lin, Yang Hu, Guannan Qu, Tongxin Li, and Adam Wierman. Bounded-regret mpc via perturbation analysis: Prediction error, constraints, and nonlinearity. Advances in Neural Information Processing Systems, 35:36174 37187,,677,11 2022.
[45] Tongxin Li, Ruixiao Yang, Guannan Qu, Yiheng Lin, Adam Wierman, and Steven H Low. Certifying black-box policies with stability for nonlinear control. IEEE Open Journal of Control Systems, 1.49-62, 2023.
[46] Tongxin Li, Bo Sun, Yue Chen, Zixin Ye, Steven H Low, and Adam Wierman. Learning- based predictive control via real-time aggregate flexibility. IEEE Transactions on Smart Grid, 1 426))4487744113, 2021.
[47] Nicolas Christianson, Christopher Yeh, Tongxin Li, Mahdi Torabi Rad, Azarang Golmoham- madi, and Adam Wierman. Robustifying machine-learned algorithms for efficient grid operation. In NeurIPS 2022 Workshop on Tackling Climate Change with Machine Learning, 2022.
[48] Tongxin Li. Ccarning-Augmenne Control and Pecision-Making Theory and Applications in Smart Grids. PhD thesis, California Institute of Technology, 2023.
[49] Tongxin Li and Chenxi Sun. Out-of cileeof-ddstrriution-aae aware electric vehicle charging. IEEE Trans- actions on Transportation Flectrification, 2024.
[50] Runyu Zhang, Yingying Li, and Na Li. On the regret analysis of online Iqr control with predictions. arXiv preprint arXiv:2 102.01309, 2021.
[51] Jean-Jacques E Slotine, Weiping Li, et Applied nonlinear control, volume 199, Prentice hall Englewood Cliffs, NJ, 1991.
13

---

[52] Yasin Abbasi- Yadkori and Csaba Szepesvári. Regret bounds for the adaptive control of linear quadratic systems. In Proceedings of the 24th Annual Conference on Learning Theory, pages 1 1-26. JMLR Workshop and Conference Proceedings, 2011.
[53] Max Simchowitz and Dylan Foster. Naive exploration is optimal for online lqr. In International Conference on Machine Learning, pages 8937-8948 PMLR, 2020.
[54] Sarah Dean, Horia Mania, Nikolai Matni, Benjamin Recht, and Stephen Tu. On the sample complexity of the linear quadratic regulator. Foundations of Computational Mathematics, pages 1-47, 2019.
[55] Naman Agarwal, Brian Bullins, Elad Hazan, Sham Kakade, and Karan Singh. Online control with adversarial disturbances, International Conference on Machine Learning, pages 111-119. PMLR, 2019.
[56] John Doyle, Keith Glover, Pramod Khargonekar, and Bruce Francis. State-space solutions to standard h2 and oo control problems. In 1988 American Control Conference, pages 1691-1696. IEEE, 1988.
[57] Kemin Zhou and John Comstock Doyle. Essentials of robust control, volume 104. Prentice hall Upper Saddle River, NJ, 1998.
[58] Alberto Bemporad and Manfred Morari. Robust model predictive control: A survey. In Robustness in identification and control, pages 207-226. Springer. 1999.
[59] Pierre Comon. Independent component analysis, a new concept? Signal processing. 36(3):287-
3 314, 1994.
[60] Aapo Hyvärinen and Petteri Pajunen. Nonlinear independent component analysis: Existence and uniqueness results. Neural networks, 12(3):429 439, 1999.
[61] Sébastien Lachapelle, Pau Rodriguez, Yash Sharma, Katie E Everett, Rémi Le Priol, Alexandre Lacoste, and Simon Lacoste-Julien. Disentanglement via mechanism sparsity regularization: A new principle for nonlinear ica. In Conference on Causal Learning and Reasoning, pages 428-484. PMLR, 2022.
[62] Aapo Hyvarinen, Hiroaki Sasaki, and Richard Turner. Nonlinear ica using auxiliary variables and generalized contrastive learning. In The 22nd International Conference on Artificial Intelligence and Statistics, pages 859-868. PMLR, 2019.
[63] Aapo Hyvärinen, Ilyes Khemakhem, and Hiroshi Morioka. Nonlinear independent component analysis for principled disentanglement in unsupervised deep learning. Patterns, 4(10), 2023.
[64] Yingying Li, Xin Chen, and Na Li. Online optimal control with linear dynamics and predictions: algorithms and regret analysis. In NeurlPS, pages 14858-14870, 2019.
[65] Wen-Hus Chen, Donald J Ballance, Peter J Gawthrop, and John O' O'Reilly. A nonlinear dis- turbance observer for robotic manipulators. IEEE Transactions on industrial Electronics, (7(4):932-938) 2000.
[66] Mesut E Baran and Felix F Wu. Network reconfiguration in distribution systems for loss reduction and load balancing. IEEE Transactions on Power delivery, 4(2):1401-1407, 1989.
[67] Na Li, Guannan Qu, and Munther Dahleh. Real-time decentralized voltage control in distri- bution networks. In 2014 52nd Annual Allerton Conference on Communication, Control, and Computing (Allerton), pages 582-588. IEEE, 2014.
[68] Masoud Farivar, Lijun Chen, and Steven Low. Equilibrium and dynamics of local voltage control in distribution systems. In 52nd IEEE Conference on Decision and Control, pages 4329-4334. IEEE, 2013.
[69] Navin Goyal, Santosh Vempala, and Ying Xiao. Fourier pea and robust tensor decomposition. In Proceedings of the forty-sixth annual ACM symposium on Theory of computing, pages 584-593, 2014.
14

---

[70] Mikhail Belkin, Luis Rademacher, and James Voss. Blind signal separation in the presence of gaussian noise. In Conference on Learning Theory, pages 270-287. PMLR, 2013.
[71] Santosh Vempala and Ying Xiao. Max vs min: Tensor decomposition and ica with nearly linear sample complexity. In Conference on Learning Theory, pages 1710-1723. PMLR, 2015.
[72] Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.
[73] Joshua S Richman and Randall Moorman. Physiological time-series analysis using approx- imate entropy and sample entropy. American journal of physiology-heart and circulatory physiology, H2039-HH2099-244 2000.
[74] Peter Auer, Thomas Jaksch, and Ronald Ortner. Near-optimal regret bounds for reinforcement learning. Advances neural information processing systems, 21, 2008
[75] Mohammad Gheshlaghi Azar, Ian Osband, and Rémi Munos. Minimax regret bounds for reinforcement learning. In International Conference Machine Learning, pages 263-272
PMLR, 2017.
[76] Elad Hazan. Introduction to online convex optimization. MIT Press, 2022.
15

---

Broader Impacts. As demonstrated in our experiments (see Appendix C. the ability to incorporate and correctly utilize latent variable predictions in control systems can drastically improve decision- making processes in critical areas such as autonomous driving, aerospace navigation, and healthcare systems, leading to more efficient and robust system performance under varying conditions. Despite that by leveraging disentangled predictions, our approach can potentially reduce the need for extensive data in making accurate predictions, concerns exist when implementing our method in real systems. For example, in the voltage control task, disentangling power generation resources including the residential consumption, solar injection, wind generation, etc may lead to discrimination among the system users.
A Related Work
Our work contributes to the growing community of algorithms with predictions, while also incorpo- rating ideas from adaptive control, online learning, and disentangled representation learning.
Online Decision-Making with Predictions. The challenge posed by untrusted ML advice in
online decision-making processes has attracted increasing attention, as evidenced by recent studies
(e.g..
34
18 for various models such as ski rental [18
19 20],
caching [35 36 37 bipartite
matching 38 online covering [39 40. online optimization
reinforcement learning 42 43
control 4
with focuses on exploring the implications and potential solutions for handling unreliable ML predictions. The most relevant result this work [4). which establishes a consistency-robustne
45), and real-world applications [46
tradeoff for the linear quadratic control problem with untrusted predictions. However, the self-tuning algorithm proposed in 4 has a competitive ratio that suffers from high variability of system perturbations and predictions, yielding a sub-optimal consistency and robustness tradeoff provided there.
Linear Quadratic Control Our work intersects with and extends upon established research in the area of linear quadratic control (LQC) systems and the analysis of the linear quadratic regulator (LQR), particularly in the context of integrating predictions with control strategies. The existing literature primarily explores the balance between achieving optimal control and managing the uncertainties inherent in predictive models. Significant work has been conducted in the realm of LQR systems, especially regarding model predictive control (MPC). For instance, Yu et al. in 2 have analyzed LQR regret in MPC under the assumption of accurate perturbation predictions. The implications of imprecise predictions are explored in 22 In addition, there are notable contributions to the discourse on regret and competitive ratio in MPC, as evidenced by works like 1 50 4 Our work is closely related to (4) which reveals a consistency and robustness tradeoff in LQC, with self-tuning policy that updates 1-dimensional confidence parameter based on past observations.
Adaptive and Robust Control Moreover, our approach also aligns with the adaptive control paradigm, especially in its recent intersection with learning theory to address non-asymptotic metrics. While the adaptive control community has traditionally concentrated on Lyapunov stability and asymptotic convergence [5T], the newer trend represented in [52 53
employs learning- theoretic measures such as regret and dynamic regret for finite-time horizon analysis. Our results contribute to this evolving field by presenting adaptive control policy that achieves the first of-its- kind near-optimal consistency and robustness tradeoff, as a novel endeavor in the context of LQR systems that also incorporates the unique challenge raised by untrusted predictions. Besides, robust control is a large area that concerns the design of controllers with performance guarantees that are robust against model uncertainty or adversarial disturbances [12. Tools of robust control include Ho synthesis [56] (7) and robust MPC [58). Our work diverges from these existing approaches by considering an LQC model that employs disentangled predictions. While it adheres to the robust control principle of resilience to uncertainties and perturbations, our work also aims to achieve near-optimal performance when predictions become accurate. This best-of-both-wooll utilizing untrusted predictions introduces a novel dimension to the robustness-consiste tradeoff, a concept not extensively explored in the existing robust control literature.
54
551
strategy in
Learning Disentangled Latent Variables Disentanglement, a key goal of which is to separate the typically independent effects of latent variables on observations, is well-established concept within the field of representation learning. For instance, when the underlying mixing function that
16

---

mixes latent independent signals to observations is linear with respect to $t, i.e., f(s;; 0) = Osp- for a matrix 0 E R linear independent component analysis (ICA) [59 30] can be effectively employed to identify the latent components. This linear approach, while powerful its simplicity, is often insufficient for more complex, real-world data structures. In contrast, advanced nonlinear ICA methods [60] have been developed to address these complexities, providing more nuanced tools for disentangling latent perturbations. These methods extend the traditional ICA framework to accommodate nonlinear mixing functions, thereby enhancing its applicability
wider range of scenarios. State-of of-the-art techniques in nonlinear ICA have introduced novel algorithms that can effectively manage the intricacies of nonlinear relationships between observed and latent variables. Furthermore, nonlinear ICA has been explored under various settings, with multiple assumptions ensuring the identifiability of the model, such as sparsity [61 17. auxiliary information [2] [5]. and others. Recent work by Hyvärinen et al. [63] has utilized contrastive learning identify the nonlinear ICA model a time-series context, demonstrating the evolving capabilities of these methods. Table in Section B provides summary of key assumptions of recent nonlinear ICA methods. Note that these methods often incorporate deep neural networks as critical black-box elements within their algorithmic structures. We focus on optimally harnessing the predictions of latent variables derived from those aforementioned disentanglement methods, regardless of their trustworthiness. Therefore, work uniquely represents strategic shift towards effectively utilizing uncertain or unverified predictions in downstream decision-making tasks, a challenge not directly addressed by existing representation learning techniques.
B Examples and Applications
B.1 Latent Perturbation Modelling
Understanding how the observed variables relate set of latent variables s, ..).....: se()) has been extensively studied in machine learning. When the mixing function f is linear, standard statistical assumptions (see Corollary B.1 on 8t can guarantee the desired identifiability of f. up to multiplicative scaling and permutation. For the case when f is nonlinear, recent advances such in nonlinear independent component analysis (ICA) leverage additional assumptions on the latent variables in 8; and the mixing function f. to make the model identifiable. Below in Table [
as [3 14
[5
[6
17
we summarize the key assumptions (besides the common assumptions of mutual independence between latent variables and at most one latent variable is Gaussian) made in those models.
Table 1: Standard assumptions on f and s for a subset of nonlinear ICA models.
Nonlinear ICA Models
Key Assumptions
Identifiable VAE [SS]
Mixing function f is bijective and smooth
Contrastive learning
B 14
Mixing function f is bijective and smooth Support of the Jacobian (s) f is sparse
Structural sparsity model
Volume-pree preserving model |6
Mixing function is bijective and (s)] =
Nonlinear ICA Models
Key Assumptions on
Identifiable VAE 15
(sc(1),.. 81 (k)) are conditionally independent given a variable u
Contrastive leamning
(si 1 E (T]) is nonstationary or temporal dependencies
B.2 Real-World Applications
Our model represents a classical control framework with extensive applicability across variety of real-world decision-maie making problems. In the following section, we illustrate the versatility of our model through selection of examples that demonstrate our dynamical model in (2) (see Section subject to latent perturbations. These applications not only exemplify our model, but also set the stage for the comprehensive experimental results we discuss later in Section C where we validate the model's practical efficacy real-world scenarios.
Example (Piloting a Drone with mixed disturbances). Imagine the challenge of piloting a drone to follow an unpredictable path (ye R²2 :t [T]) on windy and rainy day. This dynamic scenario
17

---

can be mathematically represented in a discrete- time counterpart of the kinematic model 66 2
A
=
A
Bug + C.us
Ut+1
Vt
for some matrices A E R1x4, B E R1x2 C' E Rix1 The column vector wi
(w:(1), (2(2), (3), wx)4)) contains latent variables where we(1) ye(1) yt+1 (1); wx(2) ==
yt (2) (2); w2(3) is the environmental wind speed and wx(4) is the rainfall intensity at time
t E TT. More details are discussed in Section 5.1
Besides, is well-known that any controllable system can be transformed into a canonical linear input-disturbed systems with dynamics described by T++1 Ax, B(u, + w,) [65] 1 that can be used in many real-world applications with disturbances added to the control inputs. Now, we introduce another example in electricity grid, with latent variables representing power injections from distinct buses of a distribution network.
Example 2 (Heterogeneous Voltage Control). Consider a Voli/Var control task with the Simplified Distflow model in a distribution power network consisting of n branch buses and a substation bus 166 67V:
(15) wherein imaginary constants of the complex power injection at time [T] are separated into two parts of E Th" and of E RR,, representing the controllable reactive power injection and the uncontrollable external injection (perturbation) respectively. The fixed and known matrices X, G E g=x= are positive definite and positive (Lemma in (68)) with their entries depend on the line impedance and the grid topology. Our goal is design a Volt/Var controller regulate squared volage magnitudes Ut+1 = (V)+1(1)..... Ut+1(n)) (n)) to their nominal values by provisioning the reactive power injection of for each time step E (T).
Ut+1 0++1= = X + Gpe + Xqe + vgln:
For the above example, we focus on a scenario with a set of n heterogeneous users corresponding to branch nodes that generate a sequence of time-varying active and reactive injections (p:(t) i = L..... .n) and (qf
= ..). The injection at the branch nodes i = 1.... n may be generated by various types of renewable resources and power consumption patterns, e.g., solar/wind generations, residential, and commercial activities, energy storage management, or charging EVs, etc. Together they form set of latent variables s, in (2) with l and in this special case, the mixing function (8;; 0) = Gpt + Xxf + In is affine.
In Section 5.1 and 5.2 we explore in greater detail the two examples previously discussed, and demonstrate the efficacy of our proposed method through case studies using real-world data.
B.3 Sample efficient ICA
Furthermore, there are sample efficient ICA algorithms for independent latent variables (69 70
71. The following corollary be derived from Theorem 4.2 and the sample complexity result in 71). which shows that vj is sub-linear in T, thereby concluding the utilization claimed in Section 0
prediction
Corollary B.1. Suppose the mixing matrix is full column rank, and the latent variables (s)
i = 1....., k) are mutually independent for all t E [T.. Let (log T). Under the classic assumptions such that E. [s=(()) = 0, F. (0) = 1, E [i]] < M, and the fourth cumulant (s)) 2 A for all t E [I'] and i 1, 1,..., k. there exists a polynomial-time
ICA algorithm that provides sequence of mixing matrix estimates (0. it E [T]) of f such that with high probability, the competitive ratio of DISC satisfies
CR(DISC) 11+0
Σ
k
E()
+0
(101 log T
h T
+ +0
pス + (log h log 20(0"
log T + E(())
T
Therefore, for all E (I..... E(i) is negligible, CR(DISC) will converge to the optimal competitive ratio 1 with an increasing time horizon length T and a sufficiently large prediction window w (e.g., w = (log 7)); otherwise, for any a(i) > O, CR(DISC) will be asymptotically bounded from above by O(k) + 1 where & is the number of latent variables. It is worth noting that the statistical assumptions on the latent variables (s+(1) = I,...,
......,) k) are classic in the ICA
analysis (69 70 71].
18

---

C Case Studies
In this section, we delve deeper into the practical applications of main results by using two real-world examples (see Example [ 2 to demonstrate the efficacy of DISC. We first describe our experimental setup in Section [
We then demonstrate the effectiveness of the proposed method in Section 5.1 and compare DISC with several baselines. In Section 5.2, we show the desired resilience with respect to non-stationary environment in voltage control task.
Online Learning Confidence Parameters. We illustrate the convergence of the confidence pa- rameter (Ae : E [T]) with respect the latent dimensions. In Figure 4 we exemplify the convergence behavior of the confidence parameters associated with each of the four latent components shown in (16) on the left column, together the time series for the four latent components on the right.
Notably, the first three components display learnable patterns while the last component, representing rainfall intensity is some unpredictable Gaussian noise. This distinction is reflected in the convergence behaviors of the corresponding confidence parameters: A(1), X(2), and A(3), associated with the more predictable components, quickly converge to the optimal value of 1. Conversely, the last parameter X(4) associated with the erratic rainfall component exhibits a decreasing trend due the unpredictability of the fourth component, highlighting the resilience of the proposed DISC policy against the prediction error.
C.1 Implementation Details of Experiment A
The drone's location at each moment, Pr+1 E IR2, is determined by its current position and velocity 2t E R², evolving as Pt-1 = Pt + c,2/. The drone controller can adjust its velocity every time step through a control imput u,, leading to the updated velocity 2t+1 = + Ug The coefficients Cp. Ce are determined by control time intervals and physical parameters like the drone's 's weight and the drag coefficient. Defining Ar, Pt Y/ the location mismatch at time f, this piloting task can be mathematically represented in a linear control framework as shown in (
R.a 3
Am++1 1t+1
A
An V1
+
But +
yt 02x1
ye++1
+ wyet + r202,
(16)
02x1
latent components
[
Cp
0
0
0
with system matrices
0
Cp
B
0
0
0
1
0
CV
0
0
Cy
The term wyCr + T202 represents external perturbations from the environment. The column vectors CI C2 E I& quantify the effects of the stochastic time-varying wind speed and rainfall intensity, represented by two time series (w,, / [T]) and (r, [T]). We assume the wind speed time series forms sinusoidal function and the rain intensity time series is Gaussian, which are visualized together with the ML predictions in Figure Table 2 in Appendix C.3 presents detailed choices of
the matrices A, B, Q, R and vectors C1,
C.2 Additional Experimental Results
We provide further experimental results that complement and extend the results presented in Section C
In Figure 5
we display the trajectories in the drone navigation task (Section 5.T corresponding to DISC and the three baseline policies: the LQR, the self-tuning policy in 4 and the MPC with untrusted predictions (setting À = 1 in (7). The trajectory for DISC closely aligns the best with the offline optimal trajectory upon convergence.
In Figure 6 for the voltage control task (see Section 5.2). we compare the average total costs J(x)) (defined in 33) corresponding to DISC and the three baseline policies. Note that DISC consistently achieves the lowest average cost with the smallest variance computed from random tests. The scaling parameter v of the Gaussian component at node 8 is set as 1.25.
19

---

0.20
1.0
1ake
0.15
0.30
0.9
0.05
0.00
0.4
0.2
-0.20
M141
50
100
150
200
250
100
200
300
400
1.0
2(2)
0.9
0.2
0.8
0.7
0.0
-0.1
-0.2
0.3
92xl
50
100
150
200
250
100
2000
300
400
( volk
-0.1
-0.2
PHH1
-0.3
[2xt]
250
200
0.325
0.300
0.4
0250
0.20
-02-
0175
-0.-
6150
5F2
-0.6
50
100
150
200
250
1.00
200
300
400
Figure 4: Example A. Drone Navigation (Section 5. Left column: Convergence of confidence parameters ((), X(4) corresponding latent components. Right column: Temporal dynamics of latent components fs, in R.. We illustrate the time series for the latent components in (16), with each sub-figure featuring four distinct curves representing the dimensions of the respective 4-dimensional component. Post the dotted red line, the additional blue curves visualize imperfect ML predictions from the multi-layer neural network, elaborated in Appendix C.3 The r-axis in each graph marks the time steps, while the y-axis denotes the values of the time series.
C.3 Detailed Setup and Hyper-Parameee
MI Model Setup We train ML model online generate predictions of (s+ : S at each time / E [7]. Here, ((i) denotes the i-th column of the true mixing parameter matrix 0. At each time step. the prediction model is trained and updated using previously collected disturbances. The warm-start buffer size is set as 100 and 50 respectively for the drone navigation and voltage control applications. In particular, and use length-b subsequence of the collected time series (s- (z) max(0, the input to predict the future w (w is the prediction window size defined in Section ( steps
update the ML prediction model each time step E [T]
b} < T < f) as
of perturbations, as the output. Note that (e) is the i-th column of the estimated mixing parameter matrix 0,-1 provided by the FastICA method in (30 at time
[T).
prediction models are
formed via a fully-connected neural network with 4 hidden layers with width 80 and LeakyReL.U
as the activation function except the final layer, and are trained using Adam [72 as the optimizer with learning rate 1 for 500 epochs.
20

---

Windy and Rainy
Rain intensity
0 (T))
Wind speed (
Offline optimal
LQR
MPC (untrusted)
Self-tuning
DISC
Unknown trajectory
Closer to the offline optimal
Figure 5: Supplementary results showcasing drone navigation in windy and rainy conditions, further illustrating the scenarios discussed in Section 3 depiicee in Figure 2 The external perturbations impacting the drone's s flight are modeled by two independent, time-varying latent variables: w; for wind speed and r, for rain intensity time E [T]. The path navigated by the drone using the DIsc policy (detailed in Algorithm iin Section 3 is traced in red. Notably, this trajectory closely aligns best with the offline optimal trajectory upon convergence, demonstrating the effectiveness of the DISC policy in adapting to complex environmental conditions.
Disentanglement Implementation A well- recognized issue in independent component analysis (ICA) and its nonlinear variants is the permutation indeterminacy of sources. This implies that the ordering of disentangled com-- ponents can vary, such that the i-th compo- nent identified at one time step may not cor- respond to the i-th component in subsequent al- gorithm outputs. To mitigate this "permutation unidentifiability", our methodology involves pre- processing the time series data by ranking the components according to their sample entropy, as suggested in (73] This step precedes the ap- plication of the ML prediction model, thereby enhancing the consistency of component identi- fication across time steps.
1040
100
80
Corl Ket
60
DISC
Safftuning
MPC (untrusted)
Figure 6: Average total cost J(T) with error bars for four policies in the voltage control task (Sec- tion 5.2
Table 2 below summarizes the detailed param- eters used in Section [ In our experiments, the parameters are not subject to extensive optimiza- tion.
C.4 Detailed Setup in Section 5.2
The matrices X and G are generated from the following an impedance matrix ClIIl with the following non-zero entries:
7, 71, 7,2 72,1 +
Z2 = Za Z2,4 Z4,2 = 1 2i,
Za9 Zg,5 Z5,10 Zi0,5 2 4i,
Za, = Zs.3 = Z3,6 = 7 = Z4,7 = Z7,4 = 74,8 = Zz.4 = 2 21,
where denotes the imaginary unit.
21

---

Parameter
Value
Time horizon length
240 (Section 5.1 200 (Section 5.2
Warm start buffer length
100 (Section
1 50 (Section
State dimension
4 (Section
5.
10 (Section 5.2
Action dimension m
2 (Section
5.1 10 (Section
5.2
Prediction window size w
5
Coefficients Co. Cp
0.2
Mixing Column C1
(0.8, 1.6, 1.25, 0.75)
Mixing Column C2
(1.2, 0.4, 0.75, 1.25)
12x2
02x2
Q, A (Section 5.1
,2222
O2x2
02x2
Q, R (Section 5.2
110x00
0.1 x 110x10
Initial confidence parameter Ao
0.3
(a) Basic Control Problem Setup in Section 5.1 and 5.2
Hyper-Parameter
Value
Number of hidden layers
Hidden layer width
80
Covariate size
Input dimension
1 x
Output dimension
X w
Activation function
LeakyReLU
Optimizer
Adam 72
Learning
Ic-3
(b) Neural Network Hyper- flyper-parameters.
Table 2: Parameters and hyper-parameters used in Section
D
Useful Lemmas
We first present some preliminary results that will be used when proving our main theorems.
D.1 Convexity of Çe
Below we verify that (e) is convex function of À E [0, 1 By definition,
8 EirTy-n (15E ( 0z(A (A0
( (0ss - 5-(2 o Sar)))
H
Σ
(17)
r-
equals to (since H is symmetric)
POst
H
(T
POss
Constant independent of A
+
Σ FT TT p (No
H
Σ (0
r=
-2
H
Σ
( (0, (2 o 30/T)
T=
22

---

Now, we simplify the remaining two terms. We first notice that Σ (FT))=TPP PO, (A - = AA
for some matrix A E Raxx To see this, denote de, == ( P0, E Bu We get
81
1 Popy--PP,.00 (A 81/7) -= = Σ ser (117))) = (1, (k)
Σ (FT) 7 g
T
SkAA
(1) (k)),
: Ah
Therefore, suffices to validate that the matrix H is positive
show (AA)TH(A))
T H(A) is convex, where we denote 0 == == (F PAsz E R". Note that B(R PTB Since R is positive definite by assumption and the DARE solution P is also positive definite, (AA) (AA)TH(AX) H(A)) 29 (AA) is summation of quadratic form and a linear function X, validating the convexity of the term in (17). For the same reason,
(e(A) = "T, Hg(T, (A) is also convex.
D.2 Generalized Lower Bound on J
The following lemma is a generalized result from 4. which provides lower bound on the offline optimal cost J function of system perturbations.
Lemma 2 (Generalized Lower Bound on J (4. Let (Qin(Q) and Xmiu (P) denote the smallest eigenvalues of positive definite matrices Q and P, respectively. follows that
T-1
T-1
J' 20
Σ
Σ T=t
PF (
(18)
t=0
for some constant 0 < Cp <
(1- (l-pr)² min (Amin (P), Amiu (R)/[BII,, Amin (Q)/ m2 [[]]]
2
Next, present the proof of Lemma]
D.3 Proof of Lemma] 1
We restate the lemma below, which reduces the online learning objective Σe- (2) to canonical online optimization formulation.
Lemma 3. Define Se I - R" as in
Then for any t E [T], follows that
(A)
< w Σ C((A).
20
8=0
Denote by g(T, E) = (FT)T= p (f((r:0)) ( (203 We first note that
1-1 F ΣΣ 0( Hg(T,)) D(( Hg((,)) Σ ((A).
b-
(19)
t=
Since H B(R + B PB) BT is positive semi-definite as we have shown Appendix D.I the function M(z) = x Hr is convex. Applying the Jensen's 's inequality.
7
1
h
g(T,))
<
h (g(T,
7=8
23

---

Summing over 6 = 0...,--1
Proceed from / = - to e
(PT) (PT)-PP( (fe-
(Ft)-t Ft (Be-
w
Adding a new term 5z
max (E - 1.0} l = min(e + - 1,2]
Need modify previous Ve.y
Figure 7: Illustration of the equivalence in T9p. When é increases from / L, the reduced form adds a new term S. while the original form needs to update w previous functions thz for È w. t
Therefore,
t-1 7 Σ (E) S w EE -
= E
(20)
t=
where (20) follows from (9).. The proof illustrated in Figure 7 We now proceed to show our main results. The key structure of the lemmas and theorems is outlined in Figure
Dynamic Regret Bound for À-CON
Lemma
(A-COON) - 1 S AJ(2 = O
Σ
we(1)
(2(1)) 2 +
(1 (1- (1-
22(1)
||
1m1
J(i*-CON) - j
æ(1)
S min AJ(2)
Σ
(2(T/w) + z(i)
Theorem 4.2 Theorem 4.3
J(*--CON) == min J (i-CON)
Static Regret Bound for DISCO
ICA: Lemma 5
(DISCO) - / (A* CON) = o(T)
GM: Lemma 7
Figure 8: Outline of key steps in the proofs of Theorem 4.2 and 4.3
Arrows denote implications.
E
Proof of Theorem 4.2
We first state a detailed version of Theorem 4.2 below with explicit multiplicative constants.
Theorem E.1 (DISC for Linear Mixing). Ler H = B(R + B B PB)-BBT With our model assump- tions, the competitive ratio of DISC with linear mixing function satisfies
CR(DISC) S1+ J
C1
27 + (0p%) T) + 8C2w
Σ
Cooz min (0)E (3))) +
7
32
V KT
Jo
where mmin (0) > is the smallest singular value of 0; 3 and defined in Section
4.2 Co is
defined in Lemma 2
C = 2||l|| (rep |P|13) and C2 = ||] (CI))
24

---

We now proceed to prove Theorem
E.1
Step Consistency- Robensssss Analysis of X-CON We first show the following lemma, which highlights tradeoff between consistency and robustness. The following definition of dynamic regret will be used as an alternative worst-case performance metric in our main results. Dynamic regret is a more general (and often more challenging to analyze) measure than classical static regret, which has been mostly used for stationary environments [74 75).
Definition E.1 (Dynamic regret). The dynamic regrer of policy = (t E [T]) is defined as the difference between the quadratic cost induced by the policy n, J(T) in ( and the offline optimal cost J inf, (x) subject to 2 i.e., DR(=T) J(T) -
Lemma "Consiseency-Robuttnes Robustness Lemma). With a fixed trust parameter X I, the disentangled X-confident policy in Section 3.7 has a dynamic regret of at most
H| (21P1]Crs) (1 (0p))
8
Σ
fgee + gtau))
(21)
where P, CF.TT Ti. dF are defined in Section and are defined Section tt are functions of x defined as
4.2 The terms g.th and
7 (FT)
F 7-t p (0 (A0E (A0EE1))))
7-4 (0. ((1x - A) o)0
8711)
Pr com
gt
Proof of Lemma 4 Denote by X-CON the X-confident policy, and J (A-CON) the corresponding total cost induced by taking actions (uo,..., UT-1) generated by A-CON. Similarly, denote by J the optimal total cost. Denote T == min(t explicit form of (A-CON) JJ:
1,7 1}. Lemma 3 in 4 implies the following
T-1
Σ
F (FT)²=
T- P ((0,,, k(T)0) (A 3n11)) H
Σ (FT) r-t p (0,s, (
(A 33
(22)
where <s() = when T. The sum of quadratie costs
22 can further bounded by
(A-CON) l
T
T
F (FT)== T-T P (Ast k(T)0. (A o 80(())
l|
1=0
r=t
T-1 Σ (F (FT)²- past 1/).
0s, 0, (A 3 1|
<
(23)
short-term error s! (lincar setting)
long-term hrnx (linear setting)
In above, the first term characterizes a total cost gap induced by inaccrurate ICA identifiability and predictions. The second term in (23) is the long-term error for disentangled times series predictions that are at least w steps away from the current time steps, denoted by hefro,, which becomes when t 2 T 1. It follows that
(CCF
CF pp |P| |1
2 p
1
eTror
<
Σ
CFpt 11P1108
<
(24)
25

---

since by our assumption on F, the Gelfand's formula implies that there must exist a constant Cp > 0, Pr E (0, s.t. ||F'| I Crpps for all 2 0. Moreover, for Jt we can bound from above by
Pi o
F 1 P (As, P(As,
01s- + tes, 0s (A (o 8112)
I (FT) P (8e (s+ (5 8
<2
Σ (FT) r-t PriST
+2
Σ
A
(25)
where t7 0: 1 is the mixing parameter error in (4) defined Section 2.2 = Erjt == 8 St/6, we obtain
Furthermore, noting
T
T
emer 9t
2
2
Σ
(FT)²
Pmsst
+2
Σ
(
p p(0 (AoEt1 + (1x - A) o
T=
r=t
2
T
2
CFPF
|PP
Σ (F -ut P 0. (AoE+1))
consistency emaur n (ICA xelling)
(FT) P P(.. (11 -) (1c-)))s)))
(26)
robusiness error 9 (ICA setting)
The right hand side of 26 contains two types of errors the consistency error, denoted by gfth that occurs due to the estimated time series; and the robustness error, denoted by g gtth(t that arises when A becomes small. Therefore, combining 23 (A-CON)- J":
(24). and (26) implies the following upper bound on
J (A-CON)- - J < H| Σ
(oTTTr herror
0=
CF (1-p
2 ||||$
CF
2 210
<4|||]|
17 H
P||Ss
8 H
Σ
cun gi
1- pF
t=0
where 7j is defined in
Step 2: Component-Wiss Consistency Bound df In the sequel, bound ofth and rob( (t) respectively. First, the consistency error caused by inaccurate time series predictions can be bounded by
T Σ
T
con <
| smll
s w (CElIP10) 2 PF
Σ
2(-1)) 1
which can be further decomposed into a summation of component-wise latent variable time-series prediction emorr by denoting E(i) = ET- (r (r-()= (r) ())) where Ey/t (Erje(i)
26

---

k), 1.....
as defined in Section 2.2
Considering À = A(k)), it follows that
T-1
Sw (CF|1P110)
2(r-t)
|| o etlll
9t
PF
T
Sw (C=1IP118)
2(T-1) f) || |
PF
t=U
=w =w (Cr||P110)
(()))
(27)
Step 3: Robustness bound on g. Next, we consider deriving bound gl We obatin
2
((1s ((0) ))00)))
St Σ (F P
$ (Cr||P100) -N)))))))))))
Σ PF
I1
A) o)
which leads to
(C=|P10)) [111 - 2
Σ oF || ll1
(28)
by applying the Cauchy-Schwarz inequality where || 1|4 denotes the fn-norm. Denote by m (0) > 0
the smallest singular value of the full column rank mixing matrix 0. Noting that || |81| S
amin (0) [0s]| for all т, and [s; 0)
Os in this linear mixing setting, applying Lemma
1
t (CF||PI0) 1(1 A)
(29)
J
Coo
(1-0F)² min Amia (P), Amiu (R)/II Amin (Q)/ ma [|]]]]]
for constant 0 < Co <
Step 4: Competitive Analysis A-CON Putting together Lemma and 29 since J >
J (A-CoN) J $J (2 (7 (OpF) (00z))T))
8| J
T-1
(gt con + gi rohh)
(30)
where the last term can be further bounded from above by
8 H
T-1
J+ ((1))) + |1(1
2)113 -
Σ
o con + of rob)
S8| (Cr(IP110)
Coo (0)
(i) J (A(1))²
(1 A(i))² 2
<8| HII (Cr1PP100) 2
(31)
Coo2 min (B)
(FTRL) optimization with an lz-regularizer || \0 122 The following lemma can be proved based on the equivalent online convex optimization (OCO) form of the online mirror descent (OMD) (see the
Step 5: Online Learning of X via FTRL Consider haalowtth-reguulaizee---ee
implementation described in Section 5) above as shown in Lemma We reprise the OMD steps
below.
= TC-
VA (G),
(32a)
2
X, = (rojz(TT).
(32b)
Now, fix A = (De E
[7]) generated by DISC. Let us consider a pseudo-cost AJt(A) defined below, which is compared with the true objective AJ (A) of DIsC.
27

---

T-1 AJ(()) (rT))-TP (000 0, (A )/1=))] H [erTje-P (0sr 0r b61=)]
Ž=U T=E
T-1 AJ
ΣΣ 80 T=
Oss ( 0+(A o = H ( (00: 0r 301r))
The lemma in the following holds.
Lemma 5. The online mirror descent (OMD) (32a)- in DISC such that
32b
generates sequence (Nor...; Ar-1)
C5
AJ
min AJ - AET
H||
38
PI
VKT.
-
PF
Proof of Lemma
The gradient of the cost gap at time
[T] satisfies
T-1 t
V (G)) ΣΣ
(T,) I(T,1)
(T,1)
p (0sz - 0, (A (Ao Ao0))))
(33)
where in (33 we have used the fact that H = B(R+BTPB) PB) 'B" is symmetric. Moreover, noting
that 8// = (1) (1)....)
52 (1:).
in 33 above,
(1)((1)
tr,,t) = - t-T PQ,
/ (k)
denotes an n x & Jacobian matrix J (T, t) of g(T t) with respect to whose (i.j)-th entry is (g(r.)) Normalizing the predicted time series yields that
ax0)
C5
V (C))l < |||
30
PF
As have verified in Appendix D.I each g(T, t) (A H g( (T, t) is convex function of À I. There-
fore, based on the static regret analysis for OCO 23 76), FTRL with an Eg-regularizer guarantees (after optimizing B) that
())(2)
min AJ (A)
AEI
< (G:)
max
||
VollvT
λEZ
Cj
<8||
P1
VKT.
(34)
PF
Applying Assumption 2 and noting that X3 - A²_, I O(|ee - >e-r)) for all l [T], we conclude that AJ (A) AJ'(A) o(T), since for that is not constant, the exponent p² o(1).
Denote by X the optimal solution that minimizes (A-CON). Using 30 and 34 above, we conclude that
J(DISC) J
1 1+ ( J ((DISC) - J CAN-
J
J
Lomma
Lomma
28

---

whose right hand side can be bounded respectively via Lemma 4 and Lemma 5 Furthermore, noting that f();;0) # for all E [T', hence |0s, Therefore, using Lemma 1
0. As result, Lemma implies J Q(T)).
((Dsc) J 2 2|| (2lll)
Cy
17 (Op%) T)
1 PF
J
wE( (A(i))
(1 (1- (L-A(i))² A(1))² Coo min (0)
8| (C=1IP110)
min AET
+ H||
[PI|
C5
s0
2
"wKT
+ o(1)
PF
J
After optimizing over À E I,
J(DIsc) J
< 1 2| (3))))))
17 -0p%) r)
J
wE(i)
||| (C=11P100)
min
(O)Coue J
i=1
+ l| (IP1)
C5
"wVTT
80
J + o1).
PF
Proof of Corollary B.1
Under the classic assumptions such that [+) = 0, E [s](0] = 1, [ & M, and the fourth cumulant couma((x(1)) 2 for all E [T] and i 1....,k, k. the Recursive Fourier PCA
algorithm in 75 is polynomial-time and it guarantees (c.f. Theorem in (75) that for all ( E [T].
|nal = 100 0ell - 110 - 0ellr S 0s(log 0/(log"))//M/=>>2333
3
with high probability, which yields o (logg)) Everri :) O ((lgg 1/2 log T). Therefore, with high probability, the competitive ratio bound in the corollary holds.
G Proof of Theorem 4.1
Denote T minff + - 1,T 1). Similar to the analysis in the proof of Theorem 4.2 in Appendix
we obtain
(A-CON)- - J
2-1
Σ (FT) p (+=-
H
(35)
WLOG, we assume f is linear such that f(s; 0) = Os for some n x k matrix = (0.y) and the
estimated mixing matrix is accurate so that 77 Then it follows that
J (A-CON) - J 2 Amin (R+ B PB)
Σ E=0
B
where
T
Ver == Σ ( PO, (Aoεye- (14 - 2) 871e)
(36)
7=0
29

---

Since A-CON is (1 + o(1)) consistttt we must have J (A-CON) J < o{1)J (by our model assumption that s, # for fol E [T], 0). Combining this with (36) above, is necessary to have
T B
o(1)J* 2 Amin (R++ PB)
Σ
PO, ((1s - 2) s+1e)) (FT)*
(37)
T=t
The competitive ratio is the worst-case ratio for all A, B, Q. f,O, and (st : [2]), we must have 1; to guarantee (37) above.
Now, consider an adaptive offline adversary that generates EtiE = psyle for all & S S E. Since 1g. we obtain
T
J (A-CON) - J 2p² Amin (R+BTTB))
B
TF
T-P
8
t=
T=
Setting log I', above implies CR (A-CON) = w(1).
H Proof of Theorem 4.3
We first state detailed version of Theorem 4.2 below with explicit multiplicative constants.
Step 1: Consistency-Robustene Analysis of X-CON We first show the following lemma, which highlights a tradeoff between consistency and robustness.
Lemma 6. With a fixed trust parameter E I, the disentangled X-confident control (DIsC) has a worst-case dynamic regret at most
CF (,(PPl) 7 2| Cr (1-p
Cr 1-PF
2 )² PF 8|| CyllPll,
H|| 1 11-
(6) con + 9t
rob)
(38)
where H = B(R+ (T)++BPBB-112 fmax == fo + Cy3 with fo = (f(0)); ; P. CF,
PF, w are defined Cj denotes the Lipschitz constant of f. The terms gith and gt are functions of
in Section defined as
T Σ C,C00E
96 (A)
2
St job(() (A) =
Σ
Proof of Lemma 6
Similar to the analysis in the proof of Theorem 4.2 in Appendix we obtain
J (A-CoN) J
= Σ
T-1
Σ F (TT P P0--pppf---- (F--FaaC)))
T-1
(FT)--1 P (f^ Σ
(39)
t=1
where we simplify the latent perturbations and the corresponding predictions as Sr == ʃ (8+: 0) and Tr((A)= = / (2 (xprii00)) Especially, [A]((A) = when T. Thus, the total cost gap can be
30

---

bounded by
(A-CON) J
T-1
T
y P P(r - -F((N)))
<|
t=0
T=t
|2||
T-1
Σ
[F (FT)
( (
||
+
T-1 Σ (FT) r=1+-
Ps+
(40)
1=0
short term error gtne (general mixing)
long-tern h!me (general mixing)
Note that by Assumption T f is continuous with Lipschitz constant C₁ 0, hence, f is bounded, since || < Denote by fmx fo Cy3 the largest value of where fo [f(0)]. The second
term l eeror
(23 can bounded similarly to (24) in the proof of Theorem 4.2 (see Appendix D
(1 CF Cylllllsssssss
h1
Σ
OFPF CCylPln
(1-P-
PF
2ur
(41)
The first term St emoo in (23 characterizes total cost gap induced by inaccrurate mixing matrix estimation and time series prediction errors. It follows that
Ot
emtor
Σ
(FT)
p
((((sr))) f (st::1)
(sr:81)
(5 (1o 1
$2
Σ
1
CrC
8:1l +2 2 C A 3y111l (42)
where we have used the fact that || S Crpp for all L E [7] with a spectral radius pp E (0, 1).
Recall that 1/2 = Bg - A is the mixing parameter error in (4) defined Section 2.2 Furthermore, noting
Erje 84 s/e, continuing from 442 we get
9t
errer 2
Σ
+2
Σ
CFC || A 8
T
T Σ
< 2
Σ
CrCspF
+4
T=
r=m
consistency gin (general mixing)
2
+4
CFC IPI A) o st
(43)
robustness gt (general mixing)
The right hand side of 4 contains two types of errors the consistency error, denoted by gf that occurs due to the estimated time series; and the robustness eπror, denoted by (t) that arises when
becomes small. Therefore, rearranging the terms and combining (40). (41). and (43) imply the following upper bound on J (A-CoN) J
(A-CoNN J 2
(g eroo + h
CF 1-
2 CHIPPl)
CF
T-1
<4|
n + 21 H||
Cyl1Pl]smmx) 0Z" + H
Σ
9 + gE
(
1- PF
(44)
with 7 defined in
31

---

Furthermore, we can bound the terms gtmd and he separately as:
9t Sw (CrI (2(1))
(45)
and
J J 96
(CrC,1IPI)) I1 X)I
(46)
Co 11
where f is the inverse function of the mixing function f. Since by Assumption f is bijective, f-1 must exist. Thus, (45) and (46) together imply
T-1 J
k (2(1))
wE(i)
(1- x0))
Co 1- )
pon
St
mbh
S (CFC)11PP1)
Σ
(47)
Step 2: Online Learning of X via FTPL
Consider follow-theeperrtuheedeee (FTPL) opti-
mization 28
25
with:
Σ (e( +07x) [((A)) f
A E arg min
(FTPL FOR X-LEARNING)
(48)
AEZ
The following lemma a result of (25] by fixing each coordinate (i) (i = I,.... k) an IID random variable from an exponential distribution with some parameter >> 0.
Lemma 7. The FTPL procedure that
488
above generates sequence X = .Ao Ar-1) in DISC such
1 T
min AJ ((A))) 0 (ah5/2 1//2
E
AJT(A)
(49)
aT
Similarly to the proof of Theorem the optimal solution that minimizes
4.2 applying Assumption 2 J (DISC) - Jt (
Denote by X
(A-CON). Putting together
44)
(47 and 149
we conclude
J (DisC)
J
(Disc) - (A* -CNN
E
+
+
J
J
J
Lemma
Lemma
yielding the following after optimizing over a > O:
J (DIsc)
C3
wE(1)
- A(1))²
))
1+ J
(An + 2T (fmaax 8C4 min
Σ
(A(2))
J
XEZ
J.
Co
||
WVT
+0
(50)
< 1+0
PF
2w
+
π
+ 8C4
Σ
Co Co we J. ))
wE(
+
wh?
T
VT
i=1
(51)
where we denote C3 = ||| ( ( C and C. = ||I and have used J = !(T) in (50) (implied by Lemma 2 and the assumption that f is non-zero and Lipschitz continuous, similar the proof of Theorem 4.2 in Appendix E) derive (5T)
32

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The claims made in the abstract and introduction align with our theoretical and experimental results.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation long is clear that these goals are not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
The limitations of this work are discussed in our last concluding remark
section.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (c.g.. independence assumptions, noiseless settings, model sell-speciiiiatinn asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-tex, system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimesss.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
33

---

Answer: [Yes]
Justification:
We have clearly stated our model assumptions for the derived results to hold.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proof's can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided not)?
Question:
Answer: [Yes]
Justification: We have detailed the settings and parameters used in the experiments, and will further release our code after the anonymous review stage.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, descríbing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is new model (e-g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path reproducing verifying the results. 5. Open access to data and code
34

---

Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [NA]
Our contributions focus on the theoretical side. The experimental results shown in the submitted manuscript do not depend on private datasets and be reproduced using the provided settings and parameters.
Justification:
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips. cc/
wublitc/guudes//odeSSttiin oor.eeeeSuumiisiioooooooi..y. for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" an acceptable answer. Papers cannot be rejected simply for including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https
/mipss.cc/pulli/guidiee htt///CooSSSeiiaaeeeeeeee sisssssssiniyyyyyy for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data,
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permirted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: We have clearly stated our experimental settings and details to reproduce the results.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can provided either with the code, appendix, or as supplemental material.
7. Experiment Statistical Significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: When there is randomness in our experiments, we characterize the variance using shadow regions.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emoo bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
35

---

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative еπτог rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: Our contributions focus on the theory side, and the experimental setup is sufficiently basic that it does not require intense computing resources such as GPUs.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics
Answer: [Yes]
Justification:
The research conducted conform with the NeurlPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws regulations in their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: We have discussed both positive and negative societal impacts in our conclud- ing remark section.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
36

---

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
The paper poses no such risks.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g.. code, data, models), used in the paper, properly credited and the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [NA]
Justification:
The paper does not use existing assets.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URL
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
37

---

If assets are released, the license, copyright information, and terms of use in the thcode. has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
package should be provided. For popular datasets, paperswi
latasets.paperssi
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information not available online, the authors are encouraged to reach out to the asset's s creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: The paper does not release new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moode as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country institution) were obtained?
Answer: [NA]
Justification: The paper does not involve crowdsoureing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
38

---

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
39


Reviews:
[{"review_id": "UpzRWEHwO7", "reviewer": "Reviewer_F6Jr", "summary": "The paper investigates the setting of linear quadratic control with latent perturbations. In particular, the state evolution does not only depend on the current state and the control input, but also on various latent variables that are mixed through a linear or nonlinear mixing function of unknown parameterization. The authors attempt to address this problem by considering a $\\lambda$-confident policy that balances consistency and robustness. However, they shows that a fixed confidence value cannot achieve a good consistency and robustness tradeoff. Motivated by this, the authors suggest to use a disentangled confidence policy, where the algorithm adaptively learns the trust parameter in an online manner. The authors are able to prove that their scheme achieves near-optimal competitive ratio bounds for both linear and general mixing cases (under reasonable assumptions). The authors confirm the significance and superior performance of adaptive learning for the trust parameter using interesting use cases inspired by real-world applications.", "questions": "- In Equation (8), it seems to me that the matrix $H$ was never previously introduced.\n- In Equation (8), it seems to me that the $t$ index is not present in the right hand side of (8). Is it possible that the authors meant $\\psi_{l,t}(\\lambda)$ instead of $\\psi_{l,T}(\\lambda)$?\n- Similarly, in Equation (9), shouldn't the upper index on the summation operator be $\\min[l+w-1,t-1]$? At least, this is what the authors probably imply in Line 169.\n- In Line 164, it is explained that any $\\lambda_t$ takes values in $[0,1]^k$, but then in Line 168 $\\lambda$ appears to have $T-1$ components. Obviously, this cannot work for the Hadamard product $\\lambda \\circ \\tilde{s}\\_{\\tau \\mid l}$. Shouldn't any given $\\lambda$ have $k$ components in total? I believe the authors probably mean the sequence of $\\lambda$'s in Line 168, but I think this should be explained more clearly.\n- In Figure 2a, how exactly is the offline optimal policy generated? Do the authors rely on the prior literature for this purpose?\n- The experiments by the authors seem to rely on linear mixing matrices. As the authors explain, this is by far the simplest setting, unlike general nonlinear mixing functions. Have the authors experimented with nonlinear mixing functions, even with just 2 latent variables? It would be interesting to see how the whole framework (including the estimation of the latent variables) is affected in such a case. Furthermore, the paper's theory covers nonlinear/general mixing functions.\n- In Line 779, the authors make us of the $l_4$ norm. They claim that this is because of the Cauchy-Schwarz inequality. My understanding is that the Cauchy-Schwarz inequality $\\mid <u, v> \\mid \\geq \\mid\\mid u \\mid\\mid \\cdot \\mid\\mid v \\mid\\mid$ (where the norms are $l_2$-norms is only true for the $l_2$ norm. I was not clear how the $l_4$ norm suddenly appeared there. Could the authors detail their calculations?\n- Equation (42) in Line 850 also involves the $l_4$ norm, but I think it should be totally analogous to the point above.\n- In Equation (32), the authors define $\\psi_{t,T}$ but then in the right-hand side of (32) there is no $t$. Why not just use the same inner summation expression as in (31) instead of introducing new indexes? Furthermore, the inner summation expression in (31) has $-1$ as the upper index - why then in (32) this is changed to $\\min[l+w-1,T-1]$ (or, perhaps, $\\min[t+w-1,T-1]$)?\n- Line 813 makes sense mathematically if $(R+B^{\\top}PB)^{-1}$ is a positive definite matrix. Have the authors shown that formally? I was not so clear what is going on with matrix $P$ appearing inside it.\n- I am not sure I understood the claim in Line 801.The authors claim that Lemma 2 implies that $J^*=\\Omega(T)$ because $f(\\mathbf{s}_t,\\theta)$ is never 0 by assumption. However, in theory someone might claim that the claim could be violated if $f$ can become arbitrarily close to 0 but never 0 exactly. Obviously, this is not possible for continuous function, but might be true for strange functions. For general functions, one might instead wan to impose that $f$ is lower bounded by some positive constant. That said, I feel that this is not a problem in this work, because $f$ is assumed Lipschitz, hence it must also be continuous (and even uniformly continuous). \n- For a more rigorous exposition, could the authors discuss why $\\zeta_l(\\lambda)^{\\top}H\\zeta_l(\\lambda)$ are convex functions of $\\lambda$ $\\forall l$ in the linear mixing case or when $f$ is convex, so that OCO regret bounds are applicable in these cases?", "limitations": "No concerns.", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The problem statement is well-motivated and nice practical examples are provided.\n- The paper makes several interesting contributions. I particularly liked the fundamental gap in Theorem 4.1, which motivates that we need to learn the confidence parameter $\\lambda$ in an online manner. Furthermore, the authors provide competitive ratio results, not only for the linear case, but also for the general mixing case. The assumptions (e.g., Lipschitz continuity and bijectivity of $f$ are not so weak, but also not unreasonable).\n- On the theory side, the paper is quite rich and the proofs seem to work in general (even though I do have some concerns, as I explain in the following sections).\n- The experimental evaluation confirms the fact that online learning of the $\\lambda$-confidence parameter is beneficial in two interesting settings. Results look good overall.\n- The authors discuss several aspects of their work in the Appendix and most aspects are explained quite well.", "weaknesses": "- The paper contains several errors in various formulas. In the Questions section below, I have mentioned several problems I was able to identify, but there could be more. These errors do not seem to be critical, but the manuscript still needs a very careful proofreading to fix them.\n- I was not clear about certain theoretical claims, like the $l_4$ norm or certain assumptions. Please also see Questions section below. I believe the authors should be able to address all of them.\n- Even though the paper offers theoretical result on general mixing functions, all experiments seem to focus on linear mixing. The authors explain that the nonlinear mixing case is indeed much harder than the linear one (where Fast ICA is used). To me, the problem in the general nonlinear case is that the mixing parameter $\\theta$ can be much harder to estimate accurately, so the estimation of the latent variables will also be negatively affected. So, I am not sure how practical the proposed algorithms would be for the general nonlinear case. Perhaps simple experiments with just 2 latent variables (but nonlinearly mixed) could offer some preliminary insights.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you so much for your high-quality, very detailed, and insightful comments. They are very helpful and we sincerely appreciate a lot for what you dedicated in the reviewing process. Below are our responses to the questions:\n\n***Question 1***. Thank you for pointing out the typo. The matrix $ H \\coloneqq B(R+B^\\top P B)^{-1} B^\\top$ is indeed defined in Appendix E (see Theorem E.1). We will revise the manuscript to include a definition of $H$ before Eq. (8).\n\n***Question 2***. Thank you for catching this error. The correct notation indeed involves summation over $\\ell$ rather than $t$, aligning with the notation used in Lemma 1. The corrected equation should read:\n$$\nJ(\\pi(\\lambda)) - J^{\\star} = \\sum_{{\\color{red}\\ell=0}}^{T-1}\\psi_{\\ell,T}^{\\top}(\\lambda) H \\psi_{\\ell,T}(\\lambda).\n$$\nWe will update this in the manuscript.\n\n***Question 3***. In Equation (9), the upper index on the summation operator is indeed accurate. The clarification in Lines 168-169 highlights that the per-step cost $ \\psi_{\\ell,T}^\\top(\\lambda) H \\psi_{\\ell,T}(\\lambda)$ in Equation (8) is dependent on the time horizon $T$. Therefore, it cannot be solved as a canonical online optimization problem at each time $t$ since in general $\\psi_{\\ell,T}(\\lambda)\\neq \\psi_{\\ell,t}(\\lambda)$ for $t<T$ (see Figure 7 in Appendix D for a pictorial explanation). This necessitates a tailored approach for online learning enabled by Lemma 1, which addresses this challenge and derive an equivalent optimization. \n\n***Question 4***. Yes, in Equation (8), the total regret is defined as a function of the sequence $\\lambda = (\\lambda_0, \\ldots, \\lambda_{T-1})$, where each $\\lambda_t$ within the sequence, belonging to the interval $[0,1]^k$, is learned online. We will clarify in the revised manuscript. Thank you for the comment!\n\n***Question 5***. In Figure 2(a), the offline optimal controller is derived given all future perturbations $(Cw_t:t\\in [T])$ of the following discrete-time kinematic model (similar models can be found in [Li et al. 2019] and [Yu et al. 2020]):\n$$\\begin{bmatrix}\n    \\Delta x_{t+1}\n\\\\\\\\\n    v_{t+1}\n    \\end{bmatrix} = A    \\begin{bmatrix}\n   \\Delta  x_{t}\n\\\\\\\\\n    v_{t}\n    \\end{bmatrix} + B u_t + C w_t,\n$$\nfor some matrices $A\\in\\mathbb{R}^{4\\times 4}$, $B\\in\\mathbb{R}^{4\\times 2}$, $C\\in\\mathbb{R}^{4\\times 4}$. A detailed description of the drone piloting problem can be found in Example 1, Appendix B.2 and Appendix C.1. The offline optimal control is known as (see [Goel et al. 2022])\n\n$$u^*_t=-Kx_t-(R+B^{\\top}PB)^{-1}B^{\\top}\\psi^*_t$$\n\nwhere \n$\\\\psi_t^*=\\\\sum_{\\\\tau=t}^{T-1}(F^{\\\\top})^{\\\\tau-t} PCw_t$,\nassuming the terminal cost-to-go matrix is $P$. Note that this is equivalent to the MPC-type action in Eq. (5) with true predictions of latent variables and mixing parameters.\n\n***Question 6***. Thank you for the great suggestion. We have been working on examples for nonlinear mixing functions, and have discussed them in the **Global Rebuttal**.\n\n***Question 7***. Thank you for the question. The $\\\\ell_4$-norm appears because of the Hadamard product between $\\\\mathbf{1}\\_k-\\\\lambda$ and $s_{\\\\tau}$. The detailed derivation is as follows.\n\nApplying the Cauchy–Schwarz inequality,  the term in Line 778 satisfies \n\n$$||(\\\\mathbf{1}\\_k-\\\\lambda)\\\\circ {s_{\\\\tau}}||\\_{2}=\n\\left(\\sum_{i=1}^{k}(1-\\lambda(i))^{2}(s_{\\tau}(i))^{2}\\right)^{\\frac{1}{2}}\\leq \n\\left[\\left(\\sum_{i=1}^{k}(1-\\lambda(i))^4\\right)^{\\frac{1}{2}}\\left(\\sum_{i=1}^{k}(s_{\\tau}(i))^4\\right)^{\\frac{1}{2}}\\right]^{\\frac{1}{2}}$$\nwhere the RHS is \n$$\\left(\\sum_{i=1}^{k}(1-\\lambda(i))^4\\right)^{\\frac{1}{4}}\\left(\\sum_{i=1}^{k}(s_{\\tau}(i))^4\\right)^{\\frac{1}{4}}=||\\\\mathbf{1}\\_{k}-\\lambda||\\_{4}||s\\_{\\tau}||\\_{4}.$$\n\nNote that the Cauchy–Schwarz inequality applies to the inner product between the vectors $\\\\left((1-\\lambda(1))^{2},\\ldots,(1-\\lambda(k))^{2}\\\\right)$ and \n$\\\\left(s_{\\tau}(1))^{2},\\ldots,(s_{\\tau}(k))^{2}\\\\right)$ whose entries are already squared.\n\n***Question 8***. Yes, Eq. (42) can be derived similarly.\n\n***Question 9***. We apologize for the typo in Eq. (32). The original term $\\psi_{t,T}$ should be replaced by $\\psi_{\\ell,T}$. We changed the time index in the summation from $t$ to $\\ell$ to avoid abusing the online time index $t$ but didn't fix the index in $\\psi_{t,T}$. Given a fixed prediction window size $w$, the upper index would be $\\min\\\\{\\ell+w-1,T-1\\\\}$ in Eq. (31). Thank you again for correcting us. We have checked all the equations thoroughly in the revised manuscript to make them consistent. \n\n***Question 10***. The property of $P$ being positive definite is guaranteed as a solution of the Discrete-time Algebraic Riccati Equation (DARE). More specifically, given positive definite $Q$ and $R$, $P$ as a solution of the following DARE\n$$\n    P=Q+A^\\top P A - A^\\top PB (R+B^\\top P B)^{-1} B^\\top PA.\n$$\nis symmetric and positive definite. This is a well-known result in classic optimal control theory  (c.f. \ndiscussions in [Richardson 1986]), and $P$ determines the solution of LQR and LQG.\n\n***Question 11***. Yes, you have used the Lipschitz continuity of $f$. We will clarify in the revised proof.\n\n***Question 12***. Due to space limitations, we have appended a proof in **Global Rebuttal** above.\n\n```REFERENCES```\n\n1. [Li 2019] Yingying Li, Xin Chen, and Na Li. \"Online optimal control with linear dynamics and predictions: Algorithms and regret analysis.\" Advances in Neural Information Processing Systems 32 (2019).\n\n2. [Yu 2020] Chenkai Yu, Guanya Shi, Soon-Jo Chung, Yisong Yue, and Adam Wierman. \"The power of predictions in online control.\" Advances in Neural Information Processing Systems 33 (2020).\n\n3. [Goel 2022] Gautam Goel, and Babak Hassibi. \"The power of linear controllers in LQR control.\" 2022 IEEE 61st Conference on Decision and Control (CDC). IEEE, 2022.\n\n4. [Richardson 1986] Richardson TJ, Kwong R. On positive definite solutions to the algebraic Riccati equation. Systems & control letters. 1986 Apr 1;7(2):99-104.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "0cE1O9wbyV", "reviewer": "Reviewer_FjAh", "summary": "The paper considers the problem of LQR where the added perturbations may not be iid Gaussian noise but depend on some latent variables (which are themselves predicted using an ML model). In particular, the authors consider the dynamics given by \n\nx_{t+1} = A x_{t} + B u_t + f(s_t, \\theta) \n\nwhere s_t and \\theta are themselves predicted (up to some future window of length w) by an ML procedure. The goal is to provide an algorithm that (a) enjoys a good competitive ratio w.r.t. the optimal policy when the predictions are correct, and (b) is robust to the prediction of the latent variables. The learner knows A, B and f. \n\nThey provide theoretical results when the perturbations could be linear functions of the latent variables, as well as general functions, and provide experiments on drone navigation and power grid. The key idea in the algorithm is to first predict a \\lambda vector that decides how much to trust each latent vector, and then to run a \\lambda-confident policy. The authors use FTRL style online algorithm to predict \\lambda.", "questions": "Questions: \n1. Is there a straightforward way to incorporate unknown A and B matrices? \n2. Is there a way to use a fixed \\lambda but get o(T)-robustness while being O(1)-consistent, or vice-versa in Theorem 4.1?", "limitations": "NA", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The paper considers an important problem of using ML predictions in control and being robust to them. As far as I know, bounding competitive ratio for both consistency and robustness simultaneously has not been considered before in the control literature. \n\nThe lower bound in Theorem 4.1 that shows that a fixed \\lambda does not suffice is interesting. In hindsight, it makes sense since the ML predictions can change arbitrarily (but it is useful to present it). \n\nSufficient experimental evaluation.", "weaknesses": "The techniques are not fundamentally different from prior works, however, the fact that an online predictor of \\lambda parameter followed by plug-in control policy in eqn (5) suffices for robustness is interesting. I still support accepting the paper. \n\nMinor typos:\nLike 311 -- \"do not reply\" -> \"do not imply\" \nLine 21 -- \"adaptively\" -> \"can adapt\"", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for recognizing the novel aspects and strengths of our work, especially the dual focus on consistency and robustness in the use of machine learning predictions within control systems. We also appreciate your acknowledgment of the importance of the problem we are addressing and the novel contribution of Theorem 4.1, which highlights the limitations of the fixed \n$\\lambda$.\n\nRegarding the concern about the novelty of the techniques used in our research:\n\n***1. Technical Novelty*** While it is true that the foundational benchmarks such as competitive ratios [Goel 2022a, Sabag 2022],  and the basic idea of learning trust parameters online [Li 2022] we employed are established within the field, our work is still technically novel in the following aspects. \n\n***a. $k$ latent variables:*** We study a control problem \n $$x_{t+1} = Ax_{t}+ B u_t + f(s_t;\\theta), \\quad t\\in [T]$$\nwith perturbations depend on $k$ latent variables. Bounding the competitive ratio of the proposed control policy requires a significantly different technique (see Appendix E, the competitive analysis in Step 4 of the proof of Theorem 4.2) compared to those appeared in the previous literature. For example, special treatments of the total regret $J(\\pi(\\lambda)) - J^{\\star} = \\sum_{\\ell=0}^{T-1}\\psi_{\\ell,T}^{\\top}(\\lambda) H \\psi_{\\ell,T}(\\lambda)$ are necessary to decompose it as $O\\left( \\sum_{i=1}^{k}\\left(\\frac{w\\overline{\\varepsilon}(i) }{J^{\\star}}\\left(\\lambda(i)\\right)^2+\\frac{\\left(1-\\lambda(i)\\right)^2}{C_{0}\\left\\|f^{-1}\\right\\|^2} \\right)\\right)$ (see Eq. (43)) in the proof of Lemma 6, Appendix H.\n\n***b. Nonlinear $f$:*** Compared with the linear quadratic control models in [Goel 2022b] and [Li 2022], nonlinear perturbations lead to further technical challenges on learning $\\lambda$ since they form a Hadamard product inside the mixing function as in the $\\lambda$-CON policy (Eq. (5)):\n$$u_t =-Kx_t- Y\\sum_{\\tau=t}^{\\overline{t}}\\left(F^\\top\\right)^{\\tau-t} P f\\left(\\lambda\\circ\\widetilde{s}_{\\tau|t};\\widetilde{\\theta}_t\\right),$$\nwhere $Y:= (R+B^{\\top}PB)^{-1}B^{\\top}$. \n\n***c. Quadratic form transformation lemma:*** More importantly, we do not assume *bounded variations* of system perturbations and errors, which are often required in existing self-tuning policy [Li 2022], and robust control literature, such as robust model predictive control MPC [Berberich 2020] to ensure worst-case guarantees. For instance, the competitive ratio bound in [Li 2022] depends on \na term $O\\left(\\frac{1}{J^{\\star}}\\left(\\mu_{\\mathrm{VAR}}(\\mathbf{w})+\\mu_{\\mathrm{VAR}}(\\widehat{\\mathbf{w}})\\right)^2\\right)$\nwhere the self-variation $\\mu_{\\operatorname{VAR}}(\\mathrm{y})$ of a sequence $\\mathrm{y}:=\\left(y_0, \\ldots, y_{T-1}\\right)$ is defined as\n$$\n\\mu\\_{\\mathrm{VAR}}(\\mathrm{y}):=\\sum\\_{s=1}^{T-1} \\max _{\\tau=0, \\ldots, s-1}\\left\\|y\\_\\tau-y\\_{\\tau+T-s}\\right\\|.\n$$ The main reason of having such a term is because the self-tuning policy there basically updates $\\lambda$ through a FTL-type online optimization.\n\nIn contrast, proving our main result above is nontrivial due to the fact that despite it is known that an input-disturbed linear system can be reduced to an online convex optimization (OCO) with structured memory [Shi 2020], the connection between the problem with $\\lambda$-CON and a memoryless online optimization is not previously discovered. In the *quadratic form transformation lemma* (Lemma 1), we provide a result that decouples the loss terms in the total regret and future perturbations, thereby reducing the problem of choosing $\\lambda_t$ to an online optimization instance. Then we use a two-stage analysis that combines a dynamic regret bound for our control policy and static regret bounds induced by online learning algorithms to derive the main result.\n\nFurthermore, the perturbations are assumed to be bounded in [Li 2022]. Our technical achievement lies in offering the first bound of this nature, grounded in a term as a function of the prediction error, without restrictive assumptions on the errors $(\\overline{\\varepsilon}(1),\\ldots,\\overline{\\varepsilon}(k))$. \n\nBesides, the novelty of our work lies not only in the techniques themselves but also in how they are integrated together and applied to address a problem that has not been previously tackled in the learning and control community, as demonstrated by our practical examples.\n\nWe will add remarks to address these issues in the revised manuscript.\n\n***2. Typos*** Thank you so much for catching them. We have changed \"reply\" to \"rely\" and \"adaptively\" to \"can adapt\" in the revised manuscript.\n\n***3. Questions***  \n\nThank you for the great questions. Due to space limitations, we discuss them in the following comment.\n \n```REFERENCES```\n\n[Berberich 2020] Julian Berberich, Johannes Köhler, Matthias A. Müller, and Frank Allgöwer. \"Data-driven model predictive control with stability and robustness guarantees.\" IEEE Transactions on Automatic Control 66, no. 4 (2020): 1702-1717.\n\n[Shi 2020] Guanya Shi, Yiheng Lin, Soon-Jo Chung, Yisong Yue, and Adam Wierman. \"Online optimization with memory and competitive control.\" Advances in Neural Information Processing Systems 33 (2020): 20636-20647.\n\n[Goel 2022a] Gautam Goel, and Babak Hassibi. \"Competitive control.\" IEEE Transactions on Automatic Control 68.9 (2022): 5162-5173.\n\n[Goel 2022b] Gautam Goel, and Babak Hassibi. \"The power of linear controllers in LQR control.\" 2022 IEEE 61st Conference on Decision and Control (CDC). IEEE, 2022.\n\n[Li 2022] Tongxin Li, Ruixiao Yang, Guannan Qu, Guanya Shi, Chenkai Yu, Adam Wierman, and Steven Low. \"Robustness and Consistency in Linear Quadratic Control with Untrusted Predictions.\" ACM SIGMETRICS Performance Evaluation Review 50, no. 1 (2022): 107-108.\n\n[Sabag 2022] Oron Sabag, Sahin Lale, and Babak Hassibi. \"Competitive-ratio and regret-optimal control with general weights.\" 2022 IEEE 61st Conference on Decision and Control (CDC). IEEE, 2022.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "vah7TXsuyg", "reviewer": "Reviewer_4nx1", "summary": "The submission considers LQC with perturbations. The considered problem is interesting with potential applications on LQC with noisy/unreliable ML predictions. \n\nThe submission improves the existing method in Robustness and consistency in linear quadratic control with untrusted predictions, where there is a constant parameter determining the confidence of predictions. The submission is novel by learning the parameter in an online manner. \n\nBoth theoretical and empirical results are provided supporting the proposed method.", "questions": "The claim on line 157 is confusing. Why when $\\lambda=0$, the $f$ term will disappear? This statement requires further assumptions on $f$ which are currently missing. \n\nWhat is the update rule of $\\lambda_t$? \n\nAlso, I am wondering whether it might be a easy/doable direction to extend the proposed method to the case with a dynamics like $x_{t+1} = Ax_t+ B\\mu_t f(s_t)$, where the ML predictions also affect the effect of the actions.", "limitations": "NA", "rating": 5, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "The proposed method is novel by learning the confidence parameter in an online manner. Such a method can have many applications in control with noisy predictions. \n\nThe proposed method is analyzed both theoretically and empirically. Improved results are achieved.", "weaknesses": "The writing of the submission is really hard to follow. \n\nFirst of all, the submission mentions some concepts that are abstract and unprecise, like \"best-of-both-worlds\" and \"consistency and robustness\". \n\nFurther, the introduction has gets into technical details without proper definitions and explanations. Examples include \"(1 + o(1))-consistent\", \"ω(1)-robust\", CR(DISC), and \"Best-of-both-worlds utilization\".\n\nFinally, the update rule of $\\lambda_t$ in section 3.2 is missing. It is not self-contained to just refer to some other papers for such an important piece. Without a clear and explicit definition of the update rule of $\\lambda_t$, it is very hard to reproduce or evaluate the proposed method.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your review. We appreciate your comments on the strengths and acknowledge the concerns regarding the clarity of our writing and presentation. Please find our clarification below.\n\n1. ```Clarification of Basic Concepts in Abstract``` The detailed meanings of \"*best-of-both-worlds*\" and \"*consistency and robustness*\" in our abstract are explained below. FYI, the original sentence reads\n\n\"*Our results highlight a first-of-its-kind “best-of-both-worlds” integration of machine-learned predictions, thus lead to a near-optimal consistency and robustness tradeoff*\"\n\nIn our study, the predicted latent variable time series $(\\widetilde{s}_{\\tau|t} : t \\leq \\tau \\leq \\overline{t})$ and the mixing parameters $(\\widetilde{\\theta}_t : t \\in [T])$ can vary in accuracy, potentially being either precise or erroneous.\n The term “**best-of-both-worlds**” highlights that our method's ability to ensure reliable performance (a feature that we refer to as **robustness**) for worst-case scenarios. Mathematically, this corresponds to a bounded competitive ratio even if the prediction errors are significant. Conversely, if the predictions are near-optimal, our method demonstrates adaptability, and behaves similarly to an optimal controller (a.k.a **consistency**). This dual capability is encapsulated in our main result, which establishes a bound on the (expected) competitive ratio, illustrating our method's effectiveness across varying levels of prediction accuracy. Note that the control policy is not given the prediction accuracy beforehand.\n\nFor further context, this can be seen in our main result that bounds the (expected) competitive ratio:\n\n$$\\mathbb{E}\\left[\\mathsf{CR}(\\mathsf{DISC})\\right]\\leq 1+o(1)+ O\\left(\\rho^{2w}\\right) + \\underbrace{O\\left(\\sum_{i=1}^{k}\\frac{{\\color{blue}\\overline{\\varepsilon}(i)}}{\\Omega(T/w)+{\\color{blue}\\overline{\\varepsilon}(i)}}\\right)}_{\\text{\\textit{Best-of-both-worlds utilization}}},$$ \nwhere the $o(1)$ term hides quantities that vanish when the total number of steps $T$ increases; $k$ is the number of latent variables generating the perturbations; $\\rho\\in (0,1)$; $w$ denotes the prediction window size and each $\\overline{\\varepsilon}(i)$ (for $i=1,\\ldots,k$)  denotes the prediction error corresponding to each latent component. The expectation is over the randomness of the control policy if $f$ is a general mixing function (note that the control policy becomes deterministic for the linear mixing case).\n\nThe meaning of **Best-of-both-worlds utilization** in the last term reflects the following guarantees on robustness and consistency:\n\n**Robustness** The robustness can be inferred from the bound above that even if $\\overline{\\varepsilon}(i)$ becomes large, the last term is still bounded since $\\overline{\\varepsilon}(i)$ appears in both the numerator and the denominator. It's worth mentioning that such a bounded competitive ratio is not achievable by traditional control policies such as MPC.\n\n**Consistency** On the other hand, if $\\overline{\\varepsilon}(i)=0$, the resulted $i$-term becomes zero as well. This yields a smaller competitive ratio compared to traditional robust control policies that do not utilize disentangled predictions.\n\nWe hope the above explanations make sense. We will revise our abstract and introduction based on the concern and please let us know if there are further questions. Thank you again for the comment.\n\n2. ```Technical Details```\n\nThe formal definitions of **$(1+o(1))$-consistent** and **$\\omega(1)$-consistent** are provided in Definition 2.2 (Section 2). Let $\\mathsf{CR}(\\pi;\\varepsilon)$ denote the competitive ratio of a control policy $\\pi$ with a fixed prediction error $\\varepsilon$. Specifically, A policy $\\pi$ is *$\\gamma$-consistent* if its competitive ratio satisfies $\\mathsf{CR}(\\pi;\\varepsilon)\\leq \\gamma$ for $\\varepsilon=0$ and *$\\kappa$-robust* if $\\mathsf{CR}(\\pi;\\varepsilon)\\leq \\kappa$ for all $\\varepsilon$. \n\nWe will add a pointer to clarify in our introduction section. Thank you for pointing out this issue.\n\n3. ```Update Rule```\n\nIn fact, the update rules of $\\lambda_t$ are provided formally in later sections for both the linear mixing and general mixing cases correspondingly. Section 3.2 briefly overviews our control policy, without specifying the ONLINE-PROCEDURE. The main reason is because we use different update rules for linear mixing and general mixing settings, so we chose to specify them in later sections, together with the theoretical results.\n\n**Update Rule for Linear Mixing** In Section 4.2, Eq. (11), we have detailed the FTRL procedure for learning $\\lambda_t$ in our context:\n\n$$\\lambda\\_t \\in  \\text{argmin}\\_{\\lambda\\in\\mathcal{I}} \\\\left(\\sum_{\\ell=0}^{t-1}\\nabla_{\\lambda}^\\top\\left(\\zeta_\\ell(\\lambda)^\\top H \\zeta_\\ell(\\lambda)\n\\\\right)\\lambda + \\frac{1}{\\beta}\\|\\lambda-\\lambda_0\\|^2\\right)$$\nfor some $\\beta>0$ that can be optimized.\n\n**Update Rule for General Mixing** Similarly, in Section 4.2, Eq. (12), we have detailed the FTPL procedure for learning $\\lambda_t$. \n\nWe will revise Section 3.2 to incorporate the comment and improve our presentation.\n\n***Question 1.*** *The claim on line 157 ...* \nThank you so much for the comment. Our statement there is not accurate with the current assumption on $f$. We will revise the context correspondingly. Note that our proof for the general mixing case only requires the Lipschitz continuity and bijectivity of $f$.\n\n***Question 2.*** *Extend ... the ML predictions also affect the actions*. \n\nIn fact, in our $\\lambda$-CON policy, $u_t$ indeed depends on the predicted mixing parameters and latent variables, i.e.,\n$u_t =-Kx_t- Y\\sum_{\\tau=t}^{\\overline{t}}\\left(F^\\top\\right)^{\\tau-t} P f\\left(\\lambda\\circ\\widetilde{s}_{\\tau|t};\\widetilde{\\theta}_t\\right)$\nas specified in Eq. (5), Line 153. Going beyond our current setting, an interesting future direction is to consider the disentanglement of latent variables when they depend on current system states and actions.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "kTXlbOq6Sn", "reviewer": "Reviewer_6dfr", "summary": "This paper presents a novel policy, DISC, designed to manage uncertain perturbations in dynamical systems by learning a confidence parameter online. The focus is on integrating predictions from machine learning (ML) tools, which are often unreliable, into linear quadratic control (LQC) frameworks. The key innovation of DISC is its ability to harness accurate predictions when available and mitigate the impact of erroneous forecasts, achieving a balance between consistency and robustness.\n\nThe paper begins by discussing the challenges of using ML predictions in control systems, particularly the issue of reliability due to factors like high model variability and out-of-distribution generalization issues. The authors propose a policy, λ-CON, which extends existing methods by adapting a vectorized confidence parameter for each latent variable. However, λ-CON has limitations in achieving optimal consistency and robustness. DISC addresses these limitations by dynamically learning the confidence parameter through online learning, ensuring better performance regardless of prediction accuracy.\n\nTheoretical results provide competitive ratio bounds for DISC under both linear and general mixing functions, demonstrating its robustness and consistency. The paper also includes experiments in two real-world scenarios: drone navigation with mixed disturbances and voltage control in power grids. These experiments validate DISC's effectiveness, showing that it outperforms baseline policies in terms of cost and adaptability to rapid changes in the environment.", "questions": "- How does DISC scale with the increase in the number of latent variables and the complexity of the system? Are there any practical limits to the number of latent components DISC can handle efficiently?\n\n- What are the computational requirements for implementing DISC in real-time systems? Can the authors provide insights or benchmarks on the computational overhead introduced by the online learning of the confidence parameter?\n\n- Have the authors considered extending DISC to handle nonlinear dynamical systems? If so, what challenges do they anticipate, and are there any preliminary results or insights they can share?\n\n- Can the authors provide a more extensive comparison between DISC and other existing robust control methods? How does DISC's performance compare in terms of both consistency and robustness?", "limitations": "- The study mainly focuses on specific applications like drone navigation and voltage control, and its applicability to other fields remains uncertain.\n\n- The competitive ratio bounds depend on prediction errors, which might limit the policy's effectiveness in scenarios with high prediction inaccuracies.", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The introduction of a dynamic confidence parameter that adapts online is an important advancement in integrating ML predictions into control systems.\n\n- The paper provides theoretical guarantees for DISC, offering competitive ratio bounds that highlight its robustness and consistency.\n\n- The experimental validation in real-world scenarios, such as drone navigation and voltage control, demonstrates the practical relevance and effectiveness of DISC.\n\n- DISC's ability to adapt to varying levels of prediction accuracy and environmental changes is a valuable feature for real-world applications.", "weaknesses": "- The theoretical analysis relies on several assumptions, such as the continuity and invertibility of the mixing function, which may not hold in all real-world scenarios.\n\n- While the experiments support the theoretical findings, the datasets used might not fully capture the diversity and complexity of real-world applications.\n\n- The method may be complex to implement in practice, especially in real-time systems where computational resources are limited.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for all the comments and we appreciate your positive feedback on our work! \n\nRegarding the issues pointed out in the weakness part, here are our explanations and future plans:\n\n***Assumptions:*** We acknowledge the limitations of assuming the continuity and bijectivity of the mixing function $f$. It is worth highlighting that those assumptions are standard in nonlinear independent component analysis (ICA) models [Hyvarinen 2016, Khemakhem 2020, Yang 2022, Zheng 2022]. These advances leverage additional assumptions on the latent variables in $s_t$ and the mixing function $f$ to make the model identifiable. Below we summarize those common assumptions (besides the standard assumptions of mutual independence between latent variables and at most one latent variable is Gaussian) made in those models as examples. A similar table can be found in our Appendix B.1.\n\n*Standard Assumptions on $f$ and $s$ for a Subset of Nonlinear ICA Models*\n\n| **Nonlinear ICA Models**                  | **Key Assumptions on $f$**                                                                 |\n|--------------------------------------------|-------------------------------------------------------------------------------------------|\n| Identifiable VAE [Khemakhem 2020]   | Mixing function $f$ is bijective and smooth                                               |\n| Contrastive learning [Hyvarinen 2016] | Mixing function $f$ is bijective and smooth                                               |\n| Structural sparsity model [Zheng 2022] | Support of the Jacobian $\\mathsf{J}_f(s)$ of $f$ is sparse                                         |\n| Volume-preserving model [Yang 2022]      | Mixing function $f$ is bijective and $\\|\\mathrm{det}\\mathsf{J}\\_f(s)\\|=1$.   |\n\n| **Nonlinear ICA Models**                  | **Key Assumptions on $s$**                                                                 |\n|--------------------------------------------|-------------------------------------------------------------------------------------------|\n| Identifiable VAE [Khemakhem 2020]   | $(s_t(1),\\ldots,s_t(k))$ are conditionally independent given a variable $u$           |\n| Contrastive learning [Hyvarinen 2016] | $(s_t:t\\in [T])$ is non-stationary or has temporal dependencies                           |\n\n***Datasets:*** Thank you for the comment. In future work, we plan to evaluate DISC on a broader range of datasets and more diverse scenarios.\n\n***Implementation Complexity:*** We discuss the practicality of DISC in the following 1-1 responses to the questions raised.\n\n***Question 1.*** The core step in DISC is the computation of the action through the following $\\lambda$-CON policy:\n\n$$u_t =-Kx_t- Y\\sum_{\\tau=t}^{\\overline{t}}\\left(F^\\top\\right)^{\\tau-t} P f\\left(\\lambda\\circ\\widetilde{s}_{\\tau|t};\\widetilde{\\theta}_t\\right),$$\nwhere $Y:= (R+B^{\\top}PB)^{-1}B^{\\top}$; $\\\\widetilde{s}\\_{\\\\tau|t}$ and $\\\\widetilde{\\\\theta}\\_t$ denote the predicted latent variable and mixing parameter at time $t$. When the number of latent variables $k$ increases, the complexity comes from the computation of the Hadamard product $\\\\lambda\\\\circ\\\\widetilde{s}\\_{\\\\tau|t}$ ($k$ operations). The learning of $\\lambda\\in [0,1]^k$ involves solving an online optimization whose variable is in $[0,1]^k$. In practice, the state dimension $n$ and action dimension $m$ are often much larger than $k$ and the number of latent variables is not a bottleneck in all these key steps. For example, in our drone navigation example, external disturbances are caused by two latent variables, i.e., the wind speed and rain intensity. Similarly, in the second voltage control example, $k=3$ corresponding to PV integration, wind generation, and residential consumption.\n\n***Question 2.*** Due to space limitations, we provide a detailed discussion in the following official comment.\n\n***Question 3.*** This is a great question. Dealing with nonlinear dynamics is challenging, since there may not exist an explicit expression of the total regret $J(\\pi(\\lambda)) - J^{\\star} = \\sum_{\\ell=0}^{T-1}\\psi_{\\ell,T}^{\\top}(\\lambda) H \\psi_{\\ell,T}(\\lambda)$  as a function of some trust parameter $\\lambda$. It is therefore nontrivial if optimizing $\\lambda$ online is still possible. We have been thinking about this issue by constructing surrogate functions and would like share more insights if you are interested.\n\n***Question 4.*** We have compared DISC with vanilla MPC and LQR in our experiments. We will add more experiments that compare DISC with robust control methods such as the $\\mathcal{H}_{\\\\infty}$ controller to compare the consistency and robustness.\n\nThank you once again for your constructive feedback. We will incorporate all your suggestions to enhance the clarity and applicability of our work. Due to space limitations, we defer our responses to the remaining issues in the official comment.\n\n```REFERENCES```\n\n[Hyvarinen 2016] Aapo Hyvarinen, and Hiroshi Morioka. \"Unsupervised feature extraction by time-contrastive learning and nonlinear ica.\" Advances in neural information processing systems 29 (2016).\n\n[Khemakhem 2020] Ilyes Khemakhem, Diederik Kingma, Ricardo Monti, and Aapo Hyvarinen. \"Variational autoencoders and nonlinear ica: A unifying framework.\" In International conference on artificial intelligence and statistics, pp. 2207-2217. PMLR, 2020.\n\n[Yang 2022] Xiaojiang Yang, Yi Wang, Jiacheng Sun, Xing Zhang, Shifeng Zhang, Zhenguo Li, and Junchi Yan. \"Nonlinear ICA using volume-preserving transformations.\" In International Conference on Learning Representations. 2022.\n\n[Zheng 2022] Yujia Zheng, Ignavier Ng, and Kun Zhang. \"On the identifiability of nonlinear ICA: Sparsity and beyond.\" Advances in neural information processing systems 35 (2022): 16411-16422.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
We greatly appreciate all the questions received. In this global rebuttal, we provide more detailed discussions.

***Nonlinear Mixing Function:***

Regarding Question 6 from F6Jr, we provide a simple experiment involving two latent variables to offer preliminary insights. While our current experiments focus on the linear setting, DISC’s implementation can be efficiently generalized to nonlinear scenarios. 

Following the setting in our paper, we use the same $A$ and $B$ as in the paper's tracking example. The disturbance $w = [w\_1, w\_2, w\_3, w\_4]$ is generated nonlinearly from a two-dimensional latent variable $s = [s\_1, s\_2]$, where $s\_1$ records absolute value of a sinusoidal curve, and $s\_2$ is that of a Laplacian noise. We set $w\_1=w\_3= 2s\_1+3s\_2^2$, and $w\_2 = w\_4 = 4s\_1^2-3s\_2$. 

For disentanglement in this nonlinear setting, we employ Slow Feature Analysis (SFA) with nonlinear features. SFA is an unsupervised learning algorithm that extracts slowly varying features from quickly varying input signals. It is closely related to ICA and is a strong candidate for blind source separation problems. We extend the linear mappings of vanilla SFA to nonlinear by expansion into a nonlinear basis using scikit-learn’s *PolynomialFeatures*.

The online learning of confidence parameter $\lambda$ involves computing the gradient of $\zeta^{\top} H \zeta$ w.r.t. $\lambda$, which can be computed similarly to the linear mixing setting.  In this nonlinear example, we employ a neural network to approximate the mixing function, using its inference and the Jacobian computation to calculate the gradient efficiently (also see our response under 'Implementation complexity' to Reviewer 6dfr for more details).  In addition, DISC's performance naturally depends on the performance of the employed nonlinear disentanglement algorithm. Developing more efficient and stable disentanglement methods is an interesting direction for future research.

Experimental results show the baseline self-tuning policy achieves a cost ratio (CR) of **3.22**, the MPC with a CR of **3.30**, and the LQR with a CR of **8.28**. Nonlinear DISC achieves a CR of **2.22** in this example of nonlinear mixing functions, beating other baselines. All results are averaged over 5 runs. We also include nonlinear DISC's learning curves of each component of $\lambda$ in the attached **PDF**. We find that $\lambda_0$, which corresponds to the more predictable latent s1 (sinusoidal curve), quickly converges to the optimal value of 1. Conversely, $\lambda_1$ which corresponds to the more unpredictable latent $s\_2$ (Laplacian noise) stays below 0.5. We will add more examples in the revised manuscript and make the code public.


***Convexity of $\zeta\_{\ell}(\\lambda)^{\\top}H\zeta\_{\ell}(\\lambda)$:***

Regarding Question 12 from Reviewer F6Jr, below we verify that $\zeta\_{\ell}(\\lambda)^{\\top}H\zeta\_{\ell}(\\lambda)$ is a convex function of $\\lambda\\in [0,1]^{k}$.

By definition,

$$\zeta\_{\ell}(\\lambda)^{\\top}H\zeta\_{\ell}(\\lambda)=\\left[\\sum\_{\tau=\overline{\ell}}^{\ell}(F^{\top})^{\ell-\tau}P\\left(\theta s\_{\ell}-\widetilde{\theta}\_{\tau}(\lambda\circ \widetilde{s}\_{\ell|\tau})\\right)\\right]^{\top}H\\left[\\sum\_{\tau=\overline{\ell}}^{\ell}(F^{\top})^{\ell-\tau}P\\left(\theta s\_{\ell}-\widetilde{\theta}\_{\tau}(\lambda\circ \widetilde{s}\_{\ell|\tau})\\right)\\right],$$
which equals to (since $H$ is symmetric)
$$\\underbrace{\\left[\\sum\_{\tau=\overline{\ell}}^{\ell}(F^{\top})^{\ell-\tau}P\theta s\_{\ell}\\right]^{\top}H \\left[\\sum\_{\tau=\overline{\ell}}^{\ell}(F^{\top})^{\ell-\tau}P\theta s\_{\ell}\\right]}\_{\text{constant independent of } \lambda}+\\left[\\sum\_{\tau=\overline{\ell}}^{\ell}(F^{\top})^{\ell-\tau}P\widetilde{\theta}\_{\tau}(\lambda\circ \widetilde{s}\_{\ell|\tau})\\right]^{\top}H \\left[\\sum\_{\tau=\overline{\ell}}^{\ell}(F^{\top})^{\ell-\tau}P\widetilde{\theta}\_{\tau}(\lambda\circ \widetilde{s}\_{\ell|\tau})\\right]-2\\left[\\sum\_{\tau=\overline{\ell}}^{\ell}(F^{\top})^{\ell-\tau}P\theta s\_{\ell}\\right]^{\top}H \\left[\\sum\_{\tau=\overline{\ell}}^{\ell}(F^{\top})^{\ell-\tau}P\widetilde{\theta}\_{\tau}(\lambda\circ \widetilde{s}\_{\ell|\tau})\\right].$$

Now, we simplify the remaining two terms. Notice that $\\sum\_{\tau=\overline{\ell}}^{\ell}(F^{\top})^{\ell-\tau}P\widetilde{\theta}\_{\tau}(\lambda\circ \widetilde{s}\_{\ell|\tau}) = \Lambda \lambda$ for some matrix $\Lambda\in\mathbb{R}^{n\times k}$. To see this, denote $\digamma\_{\ell,\tau}:=(F^{\top})^{\ell-\tau}P\widetilde{\theta}\_{\tau}\in\mathbb{R}^{n\times k}.$ We get
$$\\sum\_{\tau=\overline{\ell}}^{\ell}(F^{\top})^{\ell-\tau}P\widetilde{\theta}\_{\tau}(\lambda\circ \widetilde{s}\_{\ell|\tau}) = \\sum\_{\tau=\overline{\ell}}^{\ell}\digamma\_{\ell,\tau}(\lambda\circ \widetilde{s}\_{\ell|\tau})=\begin{bmatrix}\vdots & & \vdots \\\\ \digamma\_{\ell,\tau}(1) & \cdots & \digamma\_{\ell,\tau}(k) \\\\ \vdots & & \vdots  \end{bmatrix} \begin{bmatrix} s_1\lambda_1 \\\\ \vdots \\\\ s_k \lambda_k \end{bmatrix} = \underbrace{\begin{bmatrix}\vdots & & \vdots \\\\ s_1\digamma\_{\ell,\tau}(1) & \cdots & s_k\digamma\_{\ell,\tau}(k) \\\\ \vdots & & \vdots  \end{bmatrix}}\_{=:\Lambda} \begin{bmatrix} \lambda_1 \\\\ \vdots \\\\ \lambda_k \end{bmatrix}.$$

Therefore, it suffices to validate the matrix $H$ is positive semi-definite to show $(\Lambda \lambda)^{\top} H (\Lambda \lambda)-2\phi^{\top} H (\Lambda \lambda)$ is convex where $\phi:= \\sum\_{\tau=\overline{\ell}}^{\ell}(F^{\top})^{\ell-\tau}P\theta s\_{\ell}\in\mathbb{R}^{ n}$. Note that $H=B(R+B^\top P B)^{-1} B^\top$. Since $R$ is positive definite by assumption and the DARE solution $P$ is also positive definite (see the response to Question 10), $(\Lambda \lambda)^{\top} H (\Lambda \lambda)-2\phi^{\top} H (\Lambda \lambda)$ is a summation of a quadratic form and a linear function of $\lambda$, validating the convexity of $\zeta\_{\ell}(\\lambda)^{\\top}H\zeta\_{\ell}(\\lambda)$.

Meta-review: This paper generated a fair body of discussion between the authors and the reviewers during which the authors were able to convince the reviewers regarding the merit of their approach; this is evident in the revised scores. The reviewers appreciated the introduction of prediction quality that goes beyond a scalar measure, and thorough experiments that accompany the theory. Glad to recommend this paper for acceptance.
Decision: Accept (poster)