Title: Disentangling Linear Quadratic Control with Untrusted ML Predictions
Venue: NeurIPS 2024
Accepted: yes
Keywords: Linear Quadratic Control

Disentanglement

Competitive Analysis
Abstract: Uncertain perturbations in dynamical systems often arise from diverse resources, represented by latent components. The predictions for these components, typically generated by "black-box" machine learning tools, are prone to inaccuracies. To tackle this challenge, we introduce DISC, a novel policy that learns a confidence parameter online to harness the potential of accurate predictions while also mitigating the impact of erroneous forecasts. When predictions are precise, DISC leverages this information to achieve near-optimal performance. Conversely, in the case of significant prediction errors, it still has a worst-case competitive ratio guarantee. We provide competitive ratio bounds for DISC under both linear mixing of latent variables as well as a broader class of mixing functions. Our results highlight a first-of-its-kind "best-of-both-worlds" integration of machine-learned predictions, thus lead to a near-optimal consistency and robustness tradeoff, which provably improves what can be obtained without learning the confidence parameter. We validate the applicability of DISC across a spectrum of practical scenarios.

Disentangling Linear Quadratic Control with Untrusted ML Predictions

Abstract

Uncertain perturbations in dynamical systems often arise from diverse resources, represented by latent components. The predictions for these components, typically generated by 'black-box' machine learning tools, are prone to inaccuracies. To tackle this challenge, we introduce DISC, a novel policy that learns a confidence parameter online to harness the potential of accurate predictions while also mitigating the impact of erroneous forecasts. When predictions are precise, DISC leverages this information to achieve near-optimal performance. Conversely, in the case of significant prediction errors, it still has a worst-case competitive ratio guarantee. We provide competitive ratio bounds for DISC under both linear mixing of latent variables as well as a broader class of mixing functions. Our results highlight a first-of-its-kind 'best-of-both-worlds' integration of machine-learned predictions, thus lead to a near-optimal consistency and robustness tradeoff, which provably improves what can be obtained without learning the confidence parameter. We validate the applicability of DISC across a spectrum of practical scenarios.

1 Introduction

We study the problem of online decision-making with predictions. Such settings are increasingly popular with the use of machine learning for predictive modeling, with applications to power grids and robotics [1, 2, 3, 4]. However, in many real-world settings, such black-box predictors can be unreliable due to practical problems such as high model variability [5, 6], and out-of-distribution generalization issues [7, 8]. In settings that require making reliable and high-quality decisions, recent research has been working on designing the decision-making policy that can be adaptively in response to receiving low-quality predictions [4].

In this paper, we consider the setting where there are different sources of disturbance, each representing distinct (and often independent) resources or factors. For instance, in electricity grids, machine learning (ML) forecasts of variables like power consumption/generation fluctuations, electricity prices, and battery states can facilitate near-optimal operational decisions on the one hand, but on the other hand, latent variables such as power injections on certain loads can be highly unpredictable due to their volatility [9]. Another example is the drone navigation task [10], where the challenge

lies in managing external perturbations caused by a mixture of predictable elements like air flows and less predictable factors like raindrops. When performing online decision-making in this setting, one important technical challenge is to be able to combine disentanglement with an adaptive online learning algorithm to result in a more exact estimation of the trustfulness of the disturbance. To address this challenge, in this work, we focus on a linear quadratic control problem where system perturbations originate from unidentified and possibly heterogeneous latent resources/components.

Contributions. We develop a novel policy DISC that does the disentanglement and learns the adaptive parameter online simultaneously. We first introduce a policy, λ -CON, which extends the λ -confident approach in [4] by adapting a vectorized confidence parameter λ ∈ [0 , 1] k , where each λ ( i ) represents the estimated trustworthiness of the ML prediction for the i -th latent variable in the dynamical system. We then show that a static λ cannot guarantee an optimal consistency and robustness tradeoff (see Definition 2.2 for a formal description) for λ -CON: if λ -CON is (1 + o (1)) -consistent , then it is at least ω (1) -robust .

To circumvent this limitation, we propose the dynamic policy DISC (Section 3.2). This algorithm leverages online learning to optimize the confidence parameter λ t at each time t . We establish competitive ratio guarantees for DISC in both linear and general mixing scenarios, as shown in Theorems 4.2 and 4.3 (Section 4) respectively. Under Assumption 1 and 2. The competitive ratio bound for DISC, outlined informally as follows, incorporates a term that embodies our ' best-of-bothworlds utilization ' of untrusted ML predictions:

where the o (1) term hides quantities that vanish when the total number of steps T increases; k is the number of latent variables generating the perturbations; ρ ∈ (0 , 1) ; w denotes the prediction window size and each ε ( i ) (for i = 1 , . . . , k ) denotes the prediction error corresponding to each latent component. The last term in this result highlights the desired performance guarantee: When a component-wise prediction error ε ( i ) is small, the individual term ε ( i ) / (Ω( T/w ) + ε ( i )) will be negligible; otherwise, it always holds that ε ( i ) / (Ω( T/w ) + ε ( i )) ≤ 1 , regardless of how high the prediction error becomes. Our proposed DISC is both (1 + o (1)) -consistent and O (1) -robust with a sufficiently large prediction window size. We offer the first bound of this nature, grounded in a term as a function of the prediction error, without restrictive assumptions on the errors ( ε (1) , . . . , ε ( k )) .

Proving our main result above is nontrivial due to the fact that despite it is known that an inputdisturbed linear system can be reduced to an online convex optimization (OCO) with structured memory [1], the connection between the problem with λ -CON and a memoryless online optimization is not previously discovered. In Lemma 1, we provide a result that decouples the loss terms in the total regret in (9) and future perturbations, thereby reducing the problem of choosing λ t to an online optimization instance. Then we use a two-stage analysis as depicted in Figure 7 that combines a dynamic regret bound for our control policy and static regret bounds induced by online learning algorithms to derive the main result.

We demonstrate the practicality of DISC through two real-world examples (Section 5): a drone navigation problem with mixed external disturbances and voltage control in a power grid with heterogeneous power injections. We demonstrate that DISC, when applied with disentangled ML predictions, outperforms baselines that do not distinguish between underlying latent variables. Additionally, DISC shows remarkable adaptability to rapid changes in various scenarios, such as those involving ML models trained with out-of-distribution data in a non-stationary environment.

Related Work. Our work contributes to the growing community of algorithms with predictions while also incorporating ideas from adaptive control, online learning, and disentangled representation learning. We summarize the detailed related work in Appendix A.

2 Problem Formulation

Notational conventions. Throughout this paper, ∥ · ∥ denotes the ℓ 2 -norm for vectors and the matrix norm induced by the ℓ 2 -norm. We use a subscript ( · ) t to represent a lengthk vector s t =

7

PreviousPerturbation

f(st-1;0)

k

ut

(S(i) : i = 1....,k)

Control Policy π

Xt+1

= Ax, + Bu,+ f(s;0)

UntrustedMLAgent

ObservedState

DynamicalSystems

( s t (1) , . . . , s t ( k )) at time t whose i -th coordinate is written as s t ( i ) . We use λ ◦ s := ( λ ( i ) s ( i ) , i = 1 , . . . , k ) ∈ R k to denote the Hadamard product between vectors λ, s ∈ R k .

2.1 Linear Quadratic Control with Latent Perturbations

Weconsider a finite-time linear dynamical system with latent perturbations . Let [ T ] := { 0 , . . . , T -1 } be the set of time steps. Denote by x t ∈ R n a system state and u t ∈ R m an action from a state feedback policy π t at time t ∈ [ T ] . The system update rule is given by

̸

where s t := ( s t (1) , . . . , s t ( k )) ∈ R k ( k ≤ n ) contains k latent variables (components) that together generate a perturbation at time t ∈ [ T ] via a mixing function f : R k × Θ → R n parameterized by θ ∈ Θ ⊆ R d . The states and the actions are observed after being generated, while the mixing function f transforms the unobservable latent variables in s t to an additive system perturbation. Throughout this paper, we focus on the nontrivial regime that f ( s t ; θ ) = 0 n for all t ∈ [ T ] .

In (2), A ∈ R n × n and B ∈ R n × m are system matrices. We consider the standard regime that the pair ( A,B ) is stabilizable [4, 11]. Without loss of generality, we also assume the system is initialized with some fixed x 0 ∈ R n . The goal of control is to minimize the following quadratic costs given matrices A,B,Q,R :

where π := ( π t : t ∈ [ T ]) ; Q ∈ R n × n , R ∈ R m × m are positive definite matrices, and P is a symmetric positive definite cost-to-go solution of the following discrete algebraic Riccati equation (DARE), which must exist because ( A,B ) is stabilizable and Q,R are positive definite [12]:

Given P , we define K := ( R + B ⊤ PB ) -1 B ⊤ PA , and u t = -Kx t is the feedback control law of a linear quadratic regulator (LQR), which is optimal in the case of zero disturbances ( w t = 0 for all t ∈ [ T ] ). Further, let F := A -BK be the closed-loop system matrix. The Gelfand's formula implies that there must exist a constant C F > 0 and a spectral radius ρ F ∈ (0 , 1) such that ∥ F t ∥ ≤ C F ρ t F for all t ∈ [ T ] .

In this work, we focus on designing a near-optimal policy π = ( π t : t ∈ [ T ]) that minimizes the total quadratic cost in (3) subject to the linear dynamics in (2). At time t ∈ [ T ] , each π t : R n × R k → R m is a state-feedback policy that produces an action u t ∈ R m with an observed state x t -1 , and ML predictions ( ˜ s t | t ( i ) : i = 1 , . . . , k ) of the latent variables s t ∈ R k for future w time steps with a prediction window size w > 0 (denoting t := min { t + w -1 , T -1 } ). Figure 1 above summarizes our system model. We defer a detailed introduction of the ML predictions in Section 2.2, together with the performance benchmarks considered in this paper. We relegate concrete latent perturbation modelling examples and real-world applications to Appendix B.

In light of the existing nonlinear ICA models summarized in Appendix B, throughout this paper, we assume the continuity and invertibility conditions hold for the mixing function f to guarantee

our theoretical results. This assumption is weaker compared to those required in the nonlinear ICA literature [13, 14, 15, 16, 17] to guarantee identifiability, as summarized in Table 1 (see Appendix B), noting that our goal is instead to provide a near-optimal competitive ratio bound regardless of the error of disentanglement and latent variable predictions (see our performance benchmarks defined in Section 2.2).

Assumption 1. The mixing function f : R k × Θ → R n is Lipschitz continuous and bijective with respect to s ∈ R k .

2.2 Performance Benchmark

Let ˜ s τ | t be the latent variable time series value at time τ , predicted by an untrusted ML agent at time t ≤ τ . The prediction window size is an integer w > 0 . We write t := min { t + w -1 , T -1 } . The estimated mixing parameter at each time t ∈ [ T ] is denoted by ˜ θ t . Define the following error terms 2 for all t ≤ τ ≤ t and t ∈ [ T ] :

To assess the cumulative impact of prediction error over all T time steps, we define the total prediction error corresponding to the component-wise latent variable time-series prediction and the mixing parameter estimation as follows:

where ρ F ∈ (0 , 1) represents the spectral radius of the matrix F . The term ρ τ -t F in (5) accounts for the exponential decay phenomenon in the impact of component-wise errors, implying that predictions further into the future contribute less to the total error. This approach to error measurement is analogous to methods employed in linear quadratic control models with adaptive offline adversarial perturbations, as discussed in [4], which provides a foundational understanding of error evaluation in our context.

Our performance benchmark is the competitive ratio for a given prediction error ε , defined as follows. To be more precise, we focus on the competitive ratio defined for deterministic online algorithms with an adaptive offline adversary that selects the system parameters A,B,Q,R,f,θ and latent time series ( s t : t ∈ [ T ]) . Write ε := ( ε ( i ) : i = 1 , . . . , k ) and denote by J ( π ; ε ) the total cost obtained by implementing π = ( π t : t ∈ [ T ]) with some fixed prediction error ε .

Definition 2.1. Fix some prediction error ε . The competitive ratio CR ( π ; ε ) is defined as the smallest constant C ≥ 1 such that J ( π ; ε ) ≤ C · J ⋆ for all A,B,Q,R,f,θ , and ( s t : t ∈ [ T ]) satisfying the model assumptions.

The following notions of consistency and robustness with respect to the competitive ratio offer a concise characterization to measure the algorithmic performance in the presence of prediction error ε . It aligns with the benchmarks used in the growing literature on algorithms with predictions [18, 19, 20, 21] (see further discussions provided in Section A).

Definition 2.2. A policy π is γ -consistent if its competitive ratio satisfies CR ( π ; ε ) ≤ γ for ε = 0 and κ -robust if CR ( π ; ε ) ≤ κ for all ε .

3 Disentangled Confident Policy

In this section, we present our policy, DISC, which takes disentangled and untrusted ML predictions and achieves near-optimal consistency and robustness tradeoff (see Definition 2.2).

3.1 Warm-Up: Disentangled λ -Confident Policy ( λ -CON)

Before proceeding to introduce DISC, we first consider a policy that balances consistency and robustness by combining an MPC policy that fully utilizes the untrusted ML predictions, and an LQR without considering any predictions.

Denote by t := min { t + w -1 , T -1 } and let Y := ( R + B ⊤ PB ) -1 B ⊤ . Recall that as defined in Section 2.2, at each time t ∈ [ T ] , ( ˜ s τ | t : t ≤ τ ≤ t ) represents a sequence of predictions for the future values of the latent variables ( s τ : t ≤ τ ≤ t ) , with these predictions spanning a predetermined window size w > 0 . Similarly, the sequence ( ˜ θ t : t ∈ [ T ]) corresponds to the estimated mixing parameters. At each time t ∈ [ T ] , an estimate ˜ θ t is obtained by applying some disentanglement algorithm (see those examples in Section B). With this setup, we proceed to define an MPC-type action as follows:

which specifies a disentangled λ -confident policy, denoted by λ -CON, where λ ∈ I := [0 , 1] k is a fixed trust parameter . The term λ ◦ ˜ s τ | t denotes the Hadamard product of λ and ˜ s τ | t . It is worth noting that it is well known that (6) is an optimal solution of the following MPC scheme [4, 2]:

In particular, if λ is an all-one vector 1 k in (7), it trusts the ML predictions. Note that it generalizes the λ -confident policy in [4], which linearly combines actions from an MPC policy and an LQR. In Section 4.1, we present a negative result for λ -CON, indicating that it cannot achieve the optimal consistency and robustness tradeoff (see Definition 2.2). This motivates the disentangled confident policy discussed in the next section.

3.2 Disentangled Confidence Policy (DISC)

At each time t ∈ [ T ] , the algorithm adaptively learns a trust parameter λ t ∈ I := [0 , 1] k , and uses a Hadamard product of λ t and the ML learned latent variable λ t ◦ ˜ s τ | t to estimate future perturbations. The update rule of the trust parameter λ t follows an ONLINE-PROCEDURE, which can be constructed by the following explicit form of the overall dynamic regret (see [22, 4]) with a fixed λ ∈ I :

where H := B ( R + B ⊤ PB ) -1 B ⊤ , and ψ ℓ,T ( λ ) is defined as

Consider the application of online optimization strategies to minimize total regret as depicted in (9). At each time t , we obtain ψ ℓ, 0 ( λ ) , . . . , ψ ℓ,t ( λ ) . For ψ ℓ,t ( λ ) , since the summation is over τ ∈ { ℓ, min { ℓ + w -1 , t -1 }} , ψ ℓ,t : I → R n is a function that depends on t ∈ [ T ] . Thus, the formulation of the offline optimization (9) does not conform to a canonical framework suitable for online optimization. This necessitates a tailored approach for online learning, which is facilitated by Lemma 1, allowing for the online learning of λ . Below we assume the following to regulate the learned sequence ( λ t : t ∈ [ T ]) , which holds for typical online learning algorithms with stationary environments [23, 24, 25]. Note that when t -τ < 0 , we consider λ t -τ = λ 0 .

Assumption 2. If τ > 0 is a constant, then ( λ t : t ∈ [ T ]) satisfies that ∑ T -1 t =0 | λ t -λ t -τ | = o ( T ) .

Define ℓ := max { ℓ -w +1 , 0 } . The following lemma helps convert (9) to a form that can be reduced to an online optimization problem. The proof can be found in Appendix D.3.

Note that each ζ ℓ ( λ ) defines a convex function of λ , as verified in Appendix D.1. Therefore, the selection of λ t at each time t ∈ [ T ] follows from an online learning procedure ONLINE-PROCEDURE, whose concrete realizations include the follow-the-regularized-leader (FTRL) approach with an ℓ 2 -norm regularizer [23, 24], which is equivalent to online mirror descent (OMD) when f is convex; and the follow-the-perturbed-leader (FTPL) [26, 27, 28, 25] for online non-convex learning, taken previous quantities ( ζ ℓ : ℓ ∈ [ t ]) as the input. Next, we will discuss theoretical guarantees for linear and general mixing models with specified online learning procedures, as detailed later in Section 4. Our policy is summarized in Algorithm 1.

4 Main Results

In this section, we present our technical results, proving that the scheme in DISC achieves nearoptimal competitive ratio bounds for both linear and general mixing cases. Proving our main result above is nontrivial due to the fact that despite it is known that an input-disturbed linear system can be reduced to an online convex optimization (OCO) with structured memory [1], the connection between the problem with λ -CON and a memoryless online optimization is not previously discovered. In Lemma 1, we provide a result that decouples the dependency between cost functions in our problem that depends on previous actions via a linear dynamical system (see the dynamical system defined in (2)), thereby reducing the problem of choosing λ t to an online optimization instance, then use a two-stage analysis (see Figure 7) that combines the dynamic regret analysis of λ -CON and static regret bounds corresponding to applying online optimization algorithms to learn a confidence parameter.

4.1 A Fundamental Gap

First, we motivate the necessity of learning ( λ t : t ∈ [ T ]) online. Based on Definition 2.2, the following theorem reveals a negative result of λ -CON, since for any fixed non-zero λ , there always exists predicted time series ( ˜ s τ | t : t ≤ τ ≤ t, t ∈ [ T ] ) such that the competitive ratio CR ( λ -CON ) for λ -CON can be unbounded (when T goes to infinity).

Theorem 4.1 (Consistency-Robustness Impossibility for CR ( λ -CON ) ) . If the λ -confident policy λ -CON is (1 + o (1)) -consistent, then it is at least ω (1) -robust, even if the mixing parameter estimate is perfect, i.e., η = 0 .

Here, o ( · ) and ω ( · ) characterize the asymptotic asymptotic growth rates with respect to the time horizon length T . Next, we show in both the linear and general mixing settings, CR ( DISC ) can be bounded as previewed in Section 1, thus the gap between CR ( λ -CON ) and CR ( DISC ) can be arbitrarily large as implied by Theorem 4.1 above. Especially, Corollary B.1 in Section 4.2 implies that DISC is both (1 + o (1)) -consistent and O (1) -robust, with a sufficiently large prediction window size w = ω (1) , revealing that online learning of the confidence parameter provides a significant improvement of the consistency and robustness tradeoff.

4.2 Linear Mixing

We consider a linear mixing setting, where the mixing function f is linear such that f ( s ; θ ) = θs for all s ∈ R k where θ = ( θ ij ) denotes an n × k full column rank mixing matrix. Suppose ( s t : t ∈ [ T ]) and θ are bounded such that ∥ s t ∥ ≤ s for all t ∈ [ T ] and ∥ θ ∥ ≤ θ . For this particular instance, we implement a (follow-the-regularized-leader) FTRL procedure that sets

at each time t ∈ [ T ] for some β > 0 that can be optimized. The DISC policy satisfies the following competitive ratio bound, whose proof can be found in Appendix E.

Theorem 4.2 (DISC for Linear Mixing) . With our model assumptions, the competitive ratio of DISC with a linear mixing function satisfies

where k is the number of latent variables; T is the time horizon length; ρ F ∈ (0 , 1) is the spectral radius of F ; ( ε ( i ) : i = 1 , . . . , k ) and η are defined in (5) ; O ( · ) and Ω( · ) hide multiplicative constants (see the details in Appendix E).

This result highlights a consistency and robustness tradeoff of DISC, as claimed in Section 1, which offers a dual advantage: it exploits accurate predictions when available, and safeguards against the ramifications of trusting inaccurate forecasts, thus ensuring system performance reliability. Besides, our analysis can be carried out to convex mixing functions, as the online learning of λ t in (12) via FTRL guarantees a sub-linear static regret as long as ζ ℓ ( λ ) is convex in λ for all ℓ ∈ [ T ] . We further provide a bound for sample efficient ICA in the appendix.

4.3 General Mixing

In this section, we extend the linear assumption on f by considering a general setting where the mixing function f is an arbitrary Lipschitz continuous function (see Assumption 1). To deal with the non-convexity of f , we revisit the (follow-the-perturbed-leader) FTPL approach (see [28, 25]) and consider the following online learning procedure for tuning ( λ t : t ∈ [ T ]) :

Here, σ t is a lengthk random vector with each coordinate σ t ( i ) ( i = 1 , . . . , k ) being an IID random variable from an exponential distribution. The DISC policy satisfies the following competitive ratio bound.

Theorem 4.3 (DISC for General Mixing) . With our model assumptions, the expected competitive ratio of DISC satisfies

where the parameters ρ F , w, k, T, η , and ( ε ( i ) : i = 1 , . . . , k ) are the same as in Theorem 4.2 and ∥ s t ∥ ≤ s for all t ∈ [ T ] . The expectation is taken over the randomly sampled ( σ t : t ∈ [ T ]) .

The notation O ( · ) and Ω( · ) above in Theorem 4.3 hide multiplicative constants such as the Lipschitz constant of the mixing function f (details are provided in Appendix H). The result above implies the best-of-both-worlds prediction utilization as previewed in Section 1, except that the online learning of λ t converges slower with a rate k 2 , causing the term O ( k 2 / √ T ) in the competitive ratio bound (14). Unlike the linear setting, to ensure a near-optimal expected competitive ratio, the latent time series ( s t : t ∈ [ T ]) is not necessarily bounded, as long as the prediction window size w is sufficiently large (e.g., w = Θ(log T ) ) so that the term sρ 2 w F vanishes. Finally, note that the online learning procedure of λ t is not limited to FTPL. For example, the non-convex online optimization algorithm in [29] can be used to optimize ( λ t : t ∈ [ T ]) online and leads to a similar guarantee. The proof of Theorem 4.3 can be found in Appendix H.

5 Experiments

Experimental Setup. To generate ML predictions, we use the FastICA method in [30] to decompose the mixed perturbations and train a multi-layer perceptron neural network with 4 hidden layers to predict the future latent variables. FastICA is a simple and efficient linear ICA method that aims to find an orthogonal rotation of the observation that maximizes the rotated components' non-Gaussianity using fixed-point iteration. Furthermore, in our experiments, we implement a follow-the-regularized-leader (FTRL) optimization with a ℓ 2 -regularizer ∥ λ -λ 0 ∥ 2 , which is equivalent to the following equivalent online mirror descent (OMD) implementation [24]: τ t = τ t -1 -β 2 ∇ λ ( ζ ⊤ t Hζ t ) , λ t = Proj I ( τ t ) . The detailed parameters and hyper-parameters used in our experiments can be found in Appendix C.3. The code is available at https://github.com/tspbfs/DisentangleControl.

5.1 Experiment A: Drone Navigation with Mixed Disturbances

16

Self-tuning policy

14

LQR

MPC (untrusted)

WindyandRainy

Rain intensity

Ratio of Costs J(π)/J*

12

DICS (this work)

(r, : t E [T])

10

Wind speed

(w, : t ∈ [T])

8

6

Offline optimal

Online tracking

4

·Unknown trajectory

2

0.25

0.4 0.5

0.751

2

4

56

8

We first consider the drone piloting task described in Example 1 (a detailed description is delegated to Appendix B) to track an unknown trajectory, shown on the left of Figure 2.

Competitive Ratio Analysis. Wepresent a comparative analysis of the ratio of costs achieved by our DISC policy (detailed in Algorithm 1 in Section 3) on the right of Figure 2, against several benchmarks: the Linear Quadratic Regulator (LQR), Model Predictive Control (MPC) using untrusted ML predictions (i.e., setting λ = 1 k in Equation (7)), and the self-tuning policy outlined in [4] and [31].

We examine the impact of varying a scaling factor ν in the range 0 . 25 to 8 , adjusting the additive perturbation r t c 2 in (16) to ( r t /ν ) c 2 . This scaling factor ν serves as a proxy for different rain intensities, influencing the level of environmental unpredictability-the larger ν is, the more predictable the

environment. With the theoretically guaranteed best-of-both-worlds utilization of disentangled and untrusted ML predictions, DISC consistently outperforms existing baselines in terms of cost ratios across varying levels of environmental predictability.

5.2 Example B: Voltage Control with Heterogeneous Power Injections

We consider the voltage control task outlined in Example 2 for a distribution power grid with 11 nodes, as depicted in Figure 3. Specifically, nodes 3 , 5 , and 8 are active, each supplying dynamic active power injections ( p t ( i ) : i = 3 , 5 , 8) at every time step t ∈ [ T ] . Collectively, these nodes induce the composite perturbation Gp t + v 0 1 n . The objective is to apply the DISC algorithm to determine near-optimal controllable reactive power injections that will effectively mitigate voltage fluctuations across the grid. More details of the problem setting can be found in Appendix C.4.

1.0

A(1) value

0.175

0.150

x(1)

0.8

0.125

0

Substation

0.6

0.100

0.075

0.4

- 050'0

0.2

0.025

0.000

2550

75

100125

150175200

0

50

100

150

200

250

4

0.5

A(2) value

0.10

s,(2)

8

2

W

0.4

0.08 -

ResidentialConsumption

Irregular Regular

E'0

0.06 -

Wind Generation

9

0.2

0.04 -

SolarIntegration

3

0.1

0.02

5

Irregular

www

0.5

50

75100125150175200

A(3) value

0.04 -

0

50

100

150

200

s,(3)

250

Regular

Irregular

6

0.4

0.02

- E0°0

10

0.3 -

0.01 -

0.00

0.2

- 10'0

0.1

0.03

0.02

255075100125150175200

50

100

150

200

250

We consider a realistic scenario when and use this to demonstrate the adaptivity of DISC in changing environments. Figure 3 shows the change of patterns corresponding to the solar integration and wind generation. We set T = 200 . At time t = 110 , the solar generation switches from a regular pattern to generating random Gaussian noise, representing miscommunication or system faults. Similarly, at time t = 100 , the wind generation is recovered from the irregular mode to a regular mode. For the regular solar and wind generation time series, we use the real solar PV generation time series data from the DTU-Data in 2021 [32] and wind generation from the U.S. Virgin Islands Wind Resources from the National Renewable Energy Lab (NREL) [33].

Adaptability Amidst Environmental Variability. Figure 3 exhibits the adaptability of our approach in response to environmental shifts in renewable energy generation and consumption patterns. Specifically, the figure presents the time series for solar generation ( s t (1) : t ∈ [ T ]) , wind generation ( s t (2) : t ∈ [ T ]) , and residential power consumption ( s t (3) : t ∈ [ T ]) . The transition points in the time series patterns are marked by dotted red lines in the figure.

A notable observation is the reactive behavior of the confidence parameters to the changes in predictability within each time series. For instance, as the solar generation transitions to an irregular Gaussian noise profile at t ≥ 110 , this unpredictability precipitates a decline in the corresponding confidence parameter λ t (1) . Conversely, the wind generation time series exhibits a move towards a more predictable pattern past t ≥ 100 , resulting in a progressive increase in the confidence parameter λ t (2) . In the case of residential consumption, which is consistently modeled as Gaussian

noise, the algorithm adaptively learns to reduce the confidence parameter λ (3) . The resilience of the DISC algorithm in dynamically adjusting confidence levels demonstrates its robustness against environmental shifts, a critical feature for real-world applications where conditions are subject to sudden and unpredictable changes. This further highlights the efficacy of DISC in maintaining system stability and performance through intelligent adaptability to the reliability of input data, as verified by the theoretical results in Section 4.

6 Concluding Remarks

Our work combines online control and the concept of learning disentangled predictions. In this paper, we have introduced a novel policy DISC that achieves the best-of-both-world utilization of untrusted ML predictions of latent variables for a linear control problem, where the advantage of disentanglement is theoretically validated. The practicality of our method has been validated through two real-world applications, which exemplify its relevance and adaptability to complex, real-world scenarios.

Limitations and Future Directions. Despite that our main results, i.e., the competitive ratio bounds in Theorem 4.2 and 4.3 do not rely on the assumption that the latent variables ( s t (1) , . . . , s t ( k )) are independent, most of the disentanglement methods do (like sample efficient ICA algorithms used to derive Corollary B.1 and the detailed assumptions are summarized in Table 1). It would be interesting to explore more carefully designed algorithms to guarantee both sample efficiency and identifiability based on our control model. Looking ahead, our results open several intriguing paths. An immediate extension of particular interest is adapting our methodology to nonlinear dynamical systems, which could strengthen our current results. Furthermore, our work contributes to the growing community of algorithms with predictions, a compelling question arises: can the best-of-both-worlds competitive ratio bounds we have achieved be replicated across different online decision-making problems? Obtaining either positive or negative results would potentially lead to broad implications for the field of robust control and online learning.

References

Guanya Shi, Yiheng Lin, Soon-Jo Chung, Yisong Yue, and Adam Wierman. Online optimization with memory and competitive control. Advances in Neural Information Processing Systems , 33:20636-20647, 2020.

Chenkai Yu, Guanya Shi, Soon-Jo Chung, Yisong Yue, and Adam Wierman. The power of predictions in online control. Advances in Neural Information Processing Systems , 33, 2020.

Tongxin Li, Yue Chen, Bo Sun, Adam Wierman, and Steven Low. Information aggregation for constrained online control. ACM SIGMETRICS Performance Evaluation Review , 49(1):7-8, 2021.

Tongxin Li, Ruixiao Yang, Guannan Qu, Guanya Shi, Chenkai Yu, Adam Wierman, and Steven Low. Robustness and consistency in linear quadratic control with untrusted predictions. ACM SIGMETRICS Performance Evaluation Review , 50(1):107-108, 2022.

Tobias Johannink, Shikhar Bahl, Ashvin Nair, Jianlan Luo, Avinash Kumar, Matthias Loskyll, Juan Aparicio Ojea, Eugen Solowjow, and Sergey Levine. Residual reinforcement learning for robot control. In 2019 International Conference on Robotics and Automation (ICRA) , pages 6023-6029. IEEE, 2019.

Benjamin Recht. A tour of reinforcement learning: The view from continuous control. Annual Review of Control, Robotics, and Autonomous Systems , 2:253-279, 2019.

Chaochao Lu, Yuhuai Wu, José Miguel Hernández-Lobato, and Bernhard Schölkopf. Invariant causal representation learning for out-of-distribution generalization. In International Conference on Learning Representations , 2021.

Florian Wenzel, Andrea Dittadi, Peter Gehler, Carl-Johann Simon-Gabriel, Max Horn, Dominik Zietlow, David Kernert, Chris Russell, Thomas Brox, Bernt Schiele, et al. Assaying out-ofdistribution generalization in transfer learning. Advances in Neural Information Processing Systems , 35:7181-7198, 2022.

Evan D Sherwin, Max Henrion, and Inês ML Azevedo. Estimation of the year-on-year volatility and the unpredictability of the united states energy system. Nature Energy , 3(4):341-346, 2018.

Tiago Oliveira, A Pedro Aguiar, and Pedro Encarnacao. Moving path following for unmanned aerial vehicles with applications to single and multiple target tracking problems. IEEE Transactions on Robotics , 32(5):1062-1078, 2016.

Sungho Shin, Yiheng Lin, Guannan Qu, Adam Wierman, and Mihai Anitescu. Near-optimal distributed linear-quadratic regulator for networked systems. SIAM Journal on Control and Optimization , 61(3):1113-1135, 2023.

Geir E Dullerud and Fernando Paganini. A course in robust control theory: a convex approach , volume 36. Springer Science & Business Media, 2013.

Aapo Hyvarinen and Hiroshi Morioka. Unsupervised feature extraction by time-contrastive learning and nonlinear ica. Advances in neural information processing systems , 29, 2016.

Aapo Hyvarinen and Hiroshi Morioka. Nonlinear ica of temporally dependent stationary sources. In Artificial Intelligence and Statistics , pages 460-469. PMLR, 2017.

Ilyes Khemakhem, Diederik Kingma, Ricardo Monti, and Aapo Hyvarinen. Variational autoencoders and nonlinear ica: A unifying framework. In International Conference on Artificial Intelligence and Statistics , pages 2207-2217. PMLR, 2020.

Xiaojiang Yang, Yi Wang, Jiacheng Sun, Xing Zhang, Shifeng Zhang, Zhenguo Li, and Junchi Yan. Nonlinear ica using volume-preserving transformations. In International Conference on Learning Representations , 2021.

Yujia Zheng, Ignavier Ng, and Kun Zhang. On the identifiability of nonlinear ica: Sparsity and beyond. Advances in Neural Information Processing Systems , 35:16411-16422, 2022.

Manish Purohit, Zoya Svitkina, and Ravi Kumar. Improving online algorithms via ml predictions. In Advances in Neural Information Processing Systems , pages 9661-9670, 2018.

Alexander Wei and Fred Zhang. Optimal robustness-consistency trade-offs for learningaugmented online algorithms. arXiv preprint arXiv:2010.11443 , 2020.

Shom Banerjee. Improving online rent-or-buy algorithms with sequential decision making and ml predictions. Advances in Neural Information Processing Systems , 33:21072-21080, 2020.

Nicolas Christianson, Tinashe Handina, and Adam Wierman. Chasing convex bodies and functions with black-box advice. In Conference on Learning Theory , pages 867-908. PMLR, 2022.

Chenkai Yu, Guanya Shi, Soon-Jo Chung, Yisong Yue, and Adam Wierman. Competitive control with delayed imperfect information. In 2022 American Control Conference (ACC) , pages 2604-2610. IEEE, 2022.

Adam Kalai and Santosh Vempala. Efficient algorithms for online decision problems. Journal of Computer and System Sciences , 71(3):291-307, 2005.

Elad Hazan and Satyen Kale. Extracting certainty from uncertainty: Regret bounded by variation in costs. Machine learning , 80:165-188, 2010.

Arun Sai Suggala and Praneeth Netrapalli. Online non-convex learning: Following the perturbed leader is optimal. In Algorithmic Learning Theory , pages 845-861. PMLR, 2020.

James Hannan. Approximation to bayes risk in repeated play. Contributions to the Theory of Games , 3:97-139, 1957.

Walid Krichene, Maximilian Balandat, Claire Tomlin, and Alexandre Bayen. The hedge algorithm on a continuum. In International Conference on Machine Learning , pages 824-832. PMLR, 2015.

Naman Agarwal, Alon Gonen, and Elad Hazan. Learning in non-convex games with an optimization oracle. In Conference on Learning Theory , pages 18-29. PMLR, 2019.

Lin Yang, Lei Deng, Mohammad H Hajiesmaili, Cheng Tan, and Wing Shing Wong. An optimal algorithm for online non-convex learning. Proceedings of the ACM on Measurement and Analysis of Computing Systems , 2(2):1-25, 2018.

Aapo Hyvärinen and Erkki Oja. Independent component analysis: algorithms and applications. Neural networks , 13(4-5):411-430, 2000.

Yiheng Lin, James A Preiss, Emile Timothy Anand, Yingying Li, Yisong Yue, and Adam Wierman. Online adaptive policy selection in time-varying systems: No-regret via contractive perturbations. In Thirty-seventh Conference on Neural Information Processing Systems , 2023.

Matti Juhani Koivisto and Juan Pablo Murcia Leon. Solar PV generation time series (PECD 2021 update). DTU-data , 5 2022.

Joseph Owen Roberts and Adam Warren. Us virgin islands wind resources update 2014. Technical report, National Renewable Energy Lab.(NREL), Golden, CO (United States), 2014.

Mohammad Mahdian, Hamid Nazerzadeh, and Amin Saberi. Online optimization with uncertain information. ACM Transactions on Algorithms (TALG) , 8(1):1-29, 2012.

Dhruv Rohatgi. Near-optimal bounds for online caching with machine learned advice. In Proceedings of the Fourteenth Annual ACM-SIAM Symposium on Discrete Algorithms , pages 1834-1845. SIAM, 2020.

Thodoris Lykouris and Sergei Vassilvitskii. Competitive caching with machine learned advice. Journal of the ACM (JACM) , 68(4):1-25, 2021.

Sungjin Im, Ravi Kumar, Aditya Petety, and Manish Purohit. Parsimonious learning-augmented caching. In International Conference on Machine Learning , pages 9588-9601. PMLR, 2022.

Antonios Antoniadis, Christian Coester, Marek Elias, Adam Polak, and Bertrand Simon. Online metric algorithms with untrusted predictions. In International Conference on Machine Learning , pages 345-355. PMLR, 2020.

Etienne Bamas, Andreas Maggiori, and Ola Svensson. The primal-dual method for learning augmented algorithms. arXiv preprint arXiv:2010.11632 , 2020.

Keerti Anand, Rong Ge, Amit Kumar, and Debmalya Panigrahi. Online algorithms with multiple predictions. In International Conference on Machine Learning , pages 582-598. PMLR, 2022.

Pengfei Li, Jianyi Yang, Adam Wierman, and Shaolei Ren. Robust learning for smoothed online convex optimization with feedback delay. Advances in Neural Information Processing Systems , 36, 2024.

Tongxin Li, Yiheng Lin, Shaolei Ren, and Adam Wierman. Beyond black-box advice: learningaugmented algorithms for mdps with q-value predictions. Advances in Neural Information Processing Systems , 36, 2024.

Jianyi Yang, Pengfei Li, Tongxin Li, Adam Wierman, and Shaolei Ren. Anytime-competitive reinforcement learning with policy prior. Advances in Neural Information Processing Systems , 36, 2024.

Yiheng Lin, Yang Hu, Guannan Qu, Tongxin Li, and Adam Wierman. Bounded-regret mpc via perturbation analysis: Prediction error, constraints, and nonlinearity. Advances in Neural Information Processing Systems , 35:36174-36187, 2022.

Tongxin Li, Ruixiao Yang, Guannan Qu, Yiheng Lin, Adam Wierman, and Steven H Low. Certifying black-box policies with stability for nonlinear control. IEEE Open Journal of Control Systems , 2:49-62, 2023.

Tongxin Li, Bo Sun, Yue Chen, Zixin Ye, Steven H Low, and Adam Wierman. Learningbased predictive control via real-time aggregate flexibility. IEEE Transactions on Smart Grid , 12(6):4897-4913, 2021.

Nicolas Christianson, Christopher Yeh, Tongxin Li, Mahdi Torabi Rad, Azarang Golmohammadi, and Adam Wierman. Robustifying machine-learned algorithms for efficient grid operation. In NeurIPS 2022 Workshop on Tackling Climate Change with Machine Learning , 2022.

Tongxin Li. Learning-Augmented Control and Decision-Making: Theory and Applications in Smart Grids . PhD thesis, California Institute of Technology, 2023.

Tongxin Li and Chenxi Sun. Out-of-distribution-aware electric vehicle charging. IEEE Transactions on Transportation Electrification , 2024.

Runyu Zhang, Yingying Li, and Na Li. On the regret analysis of online lqr control with predictions. arXiv preprint arXiv:2102.01309 , 2021.

Jean-Jacques E Slotine, Weiping Li, et al. Applied nonlinear control , volume 199. Prentice hall Englewood Cliffs, NJ, 1991.

Yasin Abbasi-Yadkori and Csaba Szepesvári. Regret bounds for the adaptive control of linear quadratic systems. In Proceedings of the 24th Annual Conference on Learning Theory , pages 1-26. JMLR Workshop and Conference Proceedings, 2011.

Max Simchowitz and Dylan Foster. Naive exploration is optimal for online lqr. In International Conference on Machine Learning , pages 8937-8948. PMLR, 2020.

Sarah Dean, Horia Mania, Nikolai Matni, Benjamin Recht, and Stephen Tu. On the sample complexity of the linear quadratic regulator. Foundations of Computational Mathematics , pages 1-47, 2019.

Naman Agarwal, Brian Bullins, Elad Hazan, Sham Kakade, and Karan Singh. Online control with adversarial disturbances. In International Conference on Machine Learning , pages 111-119. PMLR, 2019.

John Doyle, Keith Glover, Pramod Khargonekar, and Bruce Francis. State-space solutions to standard h 2 and h ∞ control problems. In 1988 American Control Conference , pages 1691-1696. IEEE, 1988.

Kemin Zhou and John Comstock Doyle. Essentials of robust control , volume 104. Prentice hall Upper Saddle River, NJ, 1998.

Alberto Bemporad and Manfred Morari. Robust model predictive control: A survey. In Robustness in identification and control , pages 207-226. Springer, 1999.

Pierre Comon. Independent component analysis, a new concept? Signal processing , 36(3):287314, 1994.

Aapo Hyvärinen and Petteri Pajunen. Nonlinear independent component analysis: Existence and uniqueness results. Neural networks , 12(3):429-439, 1999.

Sébastien Lachapelle, Pau Rodriguez, Yash Sharma, Katie E Everett, Rémi Le Priol, Alexandre Lacoste, and Simon Lacoste-Julien. Disentanglement via mechanism sparsity regularization: A new principle for nonlinear ica. In Conference on Causal Learning and Reasoning , pages 428-484. PMLR, 2022.

Aapo Hyvarinen, Hiroaki Sasaki, and Richard Turner. Nonlinear ica using auxiliary variables and generalized contrastive learning. In The 22nd International Conference on Artificial Intelligence and Statistics , pages 859-868. PMLR, 2019.

Aapo Hyvärinen, Ilyes Khemakhem, and Hiroshi Morioka. Nonlinear independent component analysis for principled disentanglement in unsupervised deep learning. Patterns , 4(10), 2023.

Yingying Li, Xin Chen, and Na Li. Online optimal control with linear dynamics and predictions: algorithms and regret analysis. In NeurIPS , pages 14858-14870, 2019.

Wen-Hua Chen, Donald J Ballance, Peter J Gawthrop, and John O'Reilly. A nonlinear disturbance observer for robotic manipulators. IEEE Transactions on industrial Electronics , 47(4):932-938, 2000.

Mesut E Baran and Felix F Wu. Network reconfiguration in distribution systems for loss reduction and load balancing. IEEE Transactions on Power delivery , 4(2):1401-1407, 1989.

Na Li, Guannan Qu, and Munther Dahleh. Real-time decentralized voltage control in distribution networks. In 2014 52nd Annual Allerton Conference on Communication, Control, and Computing (Allerton) , pages 582-588. IEEE, 2014.

Masoud Farivar, Lijun Chen, and Steven Low. Equilibrium and dynamics of local voltage control in distribution systems. In 52nd IEEE Conference on Decision and Control , pages 4329-4334. IEEE, 2013.

Navin Goyal, Santosh Vempala, and Ying Xiao. Fourier pca and robust tensor decomposition. In Proceedings of the forty-sixth annual ACM symposium on Theory of computing , pages 584-593, 2014.

Mikhail Belkin, Luis Rademacher, and James Voss. Blind signal separation in the presence of gaussian noise. In Conference on Learning Theory , pages 270-287. PMLR, 2013.

Santosh S Vempala and Ying Xiao. Max vs min: Tensor decomposition and ica with nearly linear sample complexity. In Conference on Learning Theory , pages 1710-1723. PMLR, 2015.

Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 , 2014.

Joshua S Richman and J Randall Moorman. Physiological time-series analysis using approximate entropy and sample entropy. American journal of physiology-heart and circulatory physiology , 278(6):H2039-H2049, 2000.

Peter Auer, Thomas Jaksch, and Ronald Ortner. Near-optimal regret bounds for reinforcement learning. Advances in neural information processing systems , 21, 2008.

Mohammad Gheshlaghi Azar, Ian Osband, and Rémi Munos. Minimax regret bounds for reinforcement learning. In International Conference on Machine Learning , pages 263-272. PMLR, 2017.

Elad Hazan. Introduction to online convex optimization . MIT Press, 2022.

Broader Impacts. As demonstrated in our experiments (see Appendix C), the ability to incorporate and correctly utilize latent variable predictions in control systems can drastically improve decisionmaking processes in critical areas such as autonomous driving, aerospace navigation, and healthcare systems, leading to more efficient and robust system performance under varying conditions. Despite that by leveraging disentangled predictions, our approach can potentially reduce the need for extensive data in making accurate predictions, concerns exist when implementing our method in real systems. For example, in the voltage control task, disentangling power generation resources including the residential consumption, solar injection, wind generation, etc may lead to discrimination among the system users.

A Related Work

Our work contributes to the growing community of algorithms with predictions, while also incorporating ideas from adaptive control, online learning, and disentangled representation learning.

Online Decision-Making with Predictions. The challenge posed by untrusted ML advice in online decision-making processes has attracted increasing attention, as evidenced by recent studies (e.g., [34, 18]) for various models such as ski rental [18, 19, 20], caching [35, 36, 37], bipartite matching [38], online covering [39, 40], online optimization [21, 41], reinforcement learning [42, 43], control [3, 4, 44, 45], and real-world applications [46, 47, 48, 49] with focuses on exploring the implications and potential solutions for handling unreliable ML predictions. The most relevant result to this work is [4], which establishes a consistency-robustness tradeoff for the linear quadratic control problem with untrusted predictions. However, the self-tuning algorithm proposed in [4] has a competitive ratio that suffers from high variability of system perturbations and predictions, yielding a sub-optimal consistency and robustness tradeoff provided there.

Linear Quadratic Control Our work intersects with and extends upon established research in the area of linear quadratic control (LQC) systems and the analysis of the linear quadratic regulator (LQR), particularly in the context of integrating predictions with control strategies. The existing literature primarily explores the balance between achieving optimal control and managing the uncertainties inherent in predictive models. Significant work has been conducted in the realm of LQR systems, especially regarding model predictive control (MPC). For instance, Yu et al. in [2] have analyzed LQR regret in MPC under the assumption of accurate perturbation predictions. The implications of imprecise predictions are explored in [22]. In addition, there are notable contributions to the discourse on regret and competitive ratio in MPC, as evidenced by works like [1, 50, 4]. Our work is closely related to [4], which reveals a consistency and robustness tradeoff in LQC, with a self-tuning policy that updates a 1 -dimensional confidence parameter based on past observations.

Adaptive and Robust Control Moreover, our approach also aligns with the adaptive control paradigm, especially in its recent intersection with learning theory to address non-asymptotic metrics. While the adaptive control community has traditionally concentrated on Lyapunov stability and asymptotic convergence [51], the newer trend represented in [52, 53, 54, 55] employs learningtheoretic measures such as regret and dynamic regret for finite-time horizon analysis. Our results contribute to this evolving field by presenting a adaptive control policy that achieves the first of-itskind near-optimal consistency and robustness tradeoff, as a novel endeavor in the context of LQR systems that also incorporates the unique challenge raised by untrusted predictions. Besides, robust control is a large area that concerns the design of controllers with performance guarantees that are robust against model uncertainty or adversarial disturbances [12]. Tools of robust control include H ∞ synthesis [56, 57] and robust MPC [58]. Our work diverges from these existing approaches by considering an LQC model that employs disentangled predictions. While it adheres to the robust control principle of resilience to uncertainties and perturbations, our work also aims to achieve near-optimal performance when predictions become accurate. This best-of-both-worlds strategy in utilizing untrusted predictions introduces a novel dimension to the robustness-consistency tradeoff, a concept not extensively explored in the existing robust control literature.

Learning Disentangled Latent Variables Disentanglement, a key goal of which is to separate the typically independent effects of latent variables on observations, is a well-established concept within the field of representation learning. For instance, when the underlying mixing function that

mixes latent independent signals to observations is linear with respect to s t , i.e., f ( s t ; θ ) = θs t , for a matrix θ ∈ R n × k , linear independent component analysis (ICA) [59, 30] can be effectively employed to identify the k latent components. This linear approach, while powerful in its simplicity, is often insufficient for more complex, real-world data structures. In contrast, advanced nonlinear ICA methods [60] have been developed to address these complexities, providing more nuanced tools for disentangling latent perturbations. These methods extend the traditional ICA framework to accommodate nonlinear mixing functions, thereby enhancing its applicability to a wider range of scenarios. State-of-the-art techniques in nonlinear ICA have introduced novel algorithms that can effectively manage the intricacies of nonlinear relationships between observed and latent variables. Furthermore, nonlinear ICA has been explored under various settings, with multiple assumptions ensuring the identifiability of the model, such as sparsity [61, 17], auxiliary information [62, 15], and others. Recent work by Hyvärinen et al. [63] has utilized contrastive learning to identify the nonlinear ICA model in a time-series context, demonstrating the evolving capabilities of these methods. Table 1 in Section B provides a summary of key assumptions of recent nonlinear ICA methods. Note that these methods often incorporate deep neural networks as critical black-box elements within their algorithmic structures. We focus on optimally harnessing the predictions of latent variables derived from those aforementioned disentanglement methods, regardless of their trustworthiness. Therefore, our work uniquely represents a strategic shift towards effectively utilizing uncertain or unverified predictions in downstream decision-making tasks, a challenge not directly addressed by existing representation learning techniques.

B Examples and Applications

B.1 Latent Perturbation Modelling

Understanding how the observed variables relate to a set of latent variables s t = ( s t (1) , . . . , s t ( k )) has been extensively studied in machine learning. When the mixing function f is linear, standard statistical assumptions (see Corollary B.1) on s t can guarantee the desired identifiability of f , up to multiplicative scaling and permutation. For the case when f is nonlinear, recent advances such as [13, 14, 15, 16, 17] in nonlinear independent component analysis (ICA) leverage additional assumptions on the latent variables in s t and the mixing function f , to make the model identifiable. Below in Table 1 we summarize the key assumptions (besides the common assumptions of mutual independence between latent variables and at most one latent variable is Gaussian) made in those models.

B.2 Real-World Applications

Our model represents a classical control framework with extensive applicability across a variety of real-world decision-making problems. In the following section, we illustrate the versatility of our model through a selection of examples that demonstrate our dynamical model in (2) (see Section 2.1), subject to latent perturbations. These applications not only exemplify our model, but also set the stage for the comprehensive experimental results we discuss later in Section C, where we validate the model's practical efficacy in real-world scenarios.

Example 1 (Piloting a Drone with mixed disturbances) . Imagine the challenge of piloting a drone to follow an unpredictable path ( y t ∈ R 2 : t ∈ [ T ]) on a windy and rainy day. This dynamic scenario

can be mathematically represented in a discrete-time counterpart of the kinematic model [64, 2]:

for some matrices A ∈ R 4 × 4 , B ∈ R 4 × 2 , C ∈ R 4 × 4 . The column vector w t = ( w t (1) , w t (2) , w t (3) , w t (4)) contains 4 latent variables where w t (1) := y t (1) -y t +1 (1) ; w t (2) := y t (2) -y t +1 (2) ; w t (3) is the environmental wind speed and w t (4) is the rainfall intensity at time t ∈ [ T ] . More details are discussed in Section 5.1.

Besides, it is well-known that any controllable system can be transformed into a canonical linear input-disturbed systems with dynamics described by x t +1 = Ax t + B ( u t + w t ) [65, 1] that can be used in many real-world applications with disturbances added to the control inputs. Now, we introduce another example in an electricity grid, with latent variables representing power injections from distinct buses of a distribution network.

Example 2 (Heterogeneous Voltage Control) . Consider a Volt/Var control task with the Simplified Distflow model in a distribution power network consisting of n branch buses and a substation bus [66, 67]:

wherein imaginary constants of the complex power injection at time t ∈ [ T ] are separated into two parts q c t ∈ R n and q e t ∈ R n , representing the controllable reactive power injection and the uncontrollable external injection (perturbation) respectively. The fixed and known matrices X,G ∈ R n × n are positive definite and positive (Lemma 1 in [68]) with their entries depend on the line impedance and the grid topology. Our goal is to design a Volt/Var controller to regulate squared voltage magnitudes v t +1 = ( v t +1 (1) , . . . , v t +1 ( n )) to their nominal values by provisioning the reactive power injection q c t for each time step t ∈ [ T ] .

For the above example, we focus on a scenario with a set of n heterogeneous users corresponding to n branch nodes that generate a sequence of time-varying active and reactive injections ( p t ( i ) : i = 1 , . . . , n ) and ( q c t ( i ) : i = 1 , . . . , n ) . The injection at the branch nodes i = 1 , . . . , n may be generated by various types of renewable resources and power consumption patterns, e.g., solar/wind generations, residential, and commercial activities, energy storage management, or charging EVs, etc. Together they form a set of latent variables s t in (2) with k = n and in this special case, the mixing function f ( s t ; θ ) = Gp t + Xq e t + v 0 1 n is affine.

In Section 5.1 and 5.2, we explore in greater detail the two examples previously discussed, and demonstrate the efficacy of our proposed method through case studies using real-world data.

B.3 Sample efficient ICA

Furthermore, there are sample efficient ICA algorithms for independent latent variables [69, 70, 71]. The following corollary can be derived from Theorem 4.2 and the sample complexity result in [71], which shows that η is sub-linear in T , thereby concluding the best-of-both-worlds prediction utilization claimed in Section 1.

Corollary B.1. Suppose the mixing matrix θ is full column rank, and the latent variables ( s t ( i ) : i = 1 , . . . , k ) are mutually independent for all t ∈ [ T ] . Let w = Θ(log T ) . Under the classic assumptions such that E [ s t ( i )] = 0 , E [ s 2 t ( i ) ] = 1 , E [∣ ∣ s 5 t ( i ) ∣ ∣ ] ≤ M , and the fourth cumulant | cum 4 ( s t ( i )) | ≥ ∆ for all t ∈ [ T ] and i = 1 , . . . , k , there exists a polynomial-time ICA algorithm that provides a sequence of mixing matrix estimates ( ˜ θ t : t ∈ [ T ]) of θ such that with high probability, the competitive ratio of DISC satisfies

Therefore, if for all i ∈ { 1 , . . . , k } , ε ( i ) is negligible, CR ( DISC ) will converge to the optimal competitive ratio 1 with an increasing time horizon length T and a sufficiently large prediction window w (e.g., w = Θ(log T ) ); otherwise, for any ε ( i ) > 0 , CR ( DISC ) will be asymptotically bounded from above by O ( k ) + 1 where k is the number of latent variables. It is worth noting that the statistical assumptions on the latent variables ( s t ( i ) : i = 1 , . . . , k ) are classic in the ICA analysis [69, 70, 71].

C Case Studies

In this section, we delve deeper into the practical applications of our main results by using two real-world examples (see Example 1 and 2) to demonstrate the efficacy of DISC. We first describe our experimental setup in Section 5. We then demonstrate the effectiveness of the proposed method in Section 5.1, and compare DISC with several baselines. In Section 5.2, we show the desired resilience with respect to a non-stationary environment in a voltage control task.

Online Learning of Confidence Parameters. We illustrate the convergence of the confidence parameter ( λ t : t ∈ [ T ]) with respect the latent dimensions. In Figure 4, we exemplify the convergence behavior of the confidence parameters associated with each of the four latent components shown in (16) on the left column, together the time series for the four latent components on the right.

Notably, the first three components display learnable patterns while the last component, representing rainfall intensity is some unpredictable Gaussian noise. This distinction is reflected in the convergence behaviors of the corresponding confidence parameters: λ (1) , λ (2) , and λ (3) , associated with the more predictable components, quickly converge to the optimal value of 1 . Conversely, the last parameter λ (4) associated with the erratic rainfall component exhibits a decreasing trend due to the unpredictability of the fourth component, highlighting the resilience of the proposed DISC policy against the prediction error.

C.1 Implementation Details of Experiment A

The drone's location at each moment, p t +1 ∈ R 2 , is determined by its current position and velocity v t ∈ R 2 , evolving as p t +1 = p t + c p v t . The drone controller can adjust its velocity at every time step through a control input u t , leading to the updated velocity v t +1 = v t + c v u t . The coefficients c p , c v > 0 are determined by control time intervals and physical parameters like the drone's weight and the drag coefficient. Defining ∆ x t := p t -y t the location mismatch at time t , this piloting task can be mathematically represented in a linear control framework as shown in (2):

The term w t c 1 + r t c 2 represents external perturbations from the environment. The column vectors c 1 , c 2 ∈ R 4 quantify the effects of the stochastic time-varying wind speed and rainfall intensity, represented by two time series ( w t : t ∈ [ T ]) and ( r t : t ∈ [ T ]) . We assume the wind speed time series forms a sinusoidal function and the rain intensity time series is Gaussian, which are visualized together with the ML predictions in Figure 4. Table 2 in Appendix C.3 presents detailed choices of the matrices A,B,Q,R and vectors c 1 , c 2 .

C.2 Additional Experimental Results

Weprovide further experimental results that complement and extend the results presented in Section C.

In Figure 5, we display the trajectories in the drone navigation task (Section 5.1) corresponding to DISC and the three baseline policies: the LQR, the self-tuning policy in [4], and the MPC with untrusted predictions (setting λ = 1 k in (7)). The trajectory for DISC closely aligns the best with the offline optimal trajectory upon convergence.

In Figure 6, for the voltage control task (see Section 5.2), we compare the average total costs J ( π ) (defined in (3)) corresponding to DISC and the three baseline policies. Note that DISC consistently achieves the lowest average cost with the smallest variance computed from 5 random tests. The scaling parameter ν of the Gaussian component at node 8 is set as 1 . 25 .

0.20

1.0

(1)value

0.15

0.10

0.8

0.05

000

0.6

-0.05

0.4

0.10

0.15

0.2

-0.20 -

WtC1

0

50

100

150

200

250

0

100

200

300

400

1.0 -

(2)value

60

0.2

0.8

0.1

0.7 -

0:0

0.6

0.5

0.1

0.4

0.2

[yt]

0.3

[02x1]

0

50

100

150

200

250

0

100

200

300

400

1.1

入(3)value

0.3

1.0 -

0.2

60

0.1

80

0.7

0.0

0.6

0.1

0.5

0.4

0.2

[y+1]

0.3

FE'0-

[02x1]

0

50

100

150

200

250

0

100

200

300

400

0.325

(4)value

0.4

0.300

0.275

0.2

0.250

0:0

0.225

0.200

0.2

0.175

0.4

0.150-

rtc2

-0.6

0

50

100

150

200

250

0

100

200

300

400

C.3 Detailed Setup and Hyper-Parameters

MLModel Setup We train an ML model online to generate predictions of ( s τ ( i ) θ ( i ) : t ≤ τ ≤ t ) at each time t ∈ [ T ] . Here, θ ( i ) denotes the i -th column of the true mixing parameter matrix θ . At each time step, the prediction model is trained and updated using previously collected disturbances. The warm-start buffer size is set as 100 and 50 respectively for the drone navigation and voltage control applications. In particular, we update the ML prediction model at each time step t ∈ [ T ] and use a lengthb subsequence of the collected time series ( s τ ( i ) ˜ θ t ( i ) : max { 0 , τ -b } ≤ τ < t ) as the input to predict the future w = 5 ( w is the prediction window size defined in Section 2) steps of perturbations, as the output. Note that ˜ θ t ( i ) is the i -th column of the estimated mixing parameter matrix ˜ θ t -1 provided by the FastICA method in [30] at time t ∈ [ T ] . All prediction models are formed via a fully-connected neural network with 4 hidden layers with a width 80 and LeakyReLU as the activation function except the final layer, and are trained using Adam [72] as the optimizer with a learning rate 1e -3 for 500 epochs.

Windy and Rainy

Rain intensity

(r: t∈[T])

Windspeed

Offline optimal

(w, : t ∈ [T])

LQR

MPC(untrusted)

Self-tuning

DISC

Unknown trajectory

Closertotheofflineoptimal

Disentanglement Implementation A wellrecognized issue in independent component analysis (ICA) and its nonlinear variants is the permutation indeterminacy of sources. This implies that the ordering of disentangled components can vary, such that the i -th component identified at one time step may not correspond to the i -th component in subsequent algorithm outputs. To mitigate this 'permutation unidentifiability', our methodology involves preprocessing the time series data by ranking the components according to their sample entropy, as suggested in [73]. This step precedes the application of the ML prediction model, thereby enhancing the consistency of component identification across time steps.

Table 2 below summarizes the detailed parameters used in Section C. In our experiments, the parameters are not subject to extensive optimization.

C.4 Detailed Setup in Section 5.2

The matrices X and G are generated from the following an impedance matrix Z ∈ C 11 × 11 with the following non-zero entries:

where i denotes the imaginary unit.

1100

LQR

1090

100

M

80

60

40

20

DISC

Self-tuning MPC (untrusted)

(b) Neural Network Hyper-parameters.

D Useful Lemmas

We first present some preliminary results that will be used when proving our main theorems.

D.1 Convexity of ζ ℓ

Below we verify that ζ ℓ ( λ ) is a convex function of λ ∈ [0 , 1] k . By definition, equals to (since H

Now, we simplify the remaining two terms. We first notice that ∑ ℓ τ = ℓ ( F ⊤ ) ℓ -τ P ˜ θ τ ( λ ◦ ˜ s ℓ | τ ) = Λ λ for some matrix Λ ∈ R n × k . To see this, denote ϕ ℓ,τ := ( F ⊤ ) ℓ -τ P ˜ θ τ ∈ R n × k . We get

Therefore, it suffices to validate that the matrix H is positive semi-definite to show (Λ λ ) ⊤ H (Λ λ ) -2 ϑ ⊤ H (Λ λ ) is convex, where we denote ϑ := ∑ ℓ τ = ℓ ( F ⊤ ) ℓ -τ Pθs ℓ ∈ R n . Note that H = B ( R + B ⊤ PB ) -1 B ⊤ . Since R is positive definite by assumption and the DARE solution P is also positive definite, (Λ λ ) ⊤ H (Λ λ ) -2 ϑ ⊤ H (Λ λ ) is a summation of a quadratic form and a linear function of λ , validating the convexity of the term in (17). For the same reason, ζ ℓ ( λ ) = ∑ ℓ τ = ℓ g ( τ, ℓ )( λ ) ⊤ Hg ( τ, ℓ )( λ ) is also convex.

D.2 A Generalized Lower Bound on J ⋆

The following lemma is a generalized result from [4], which provides a lower bound on the offline optimal cost J ⋆ , as a function of system perturbations.

Lemma 2 (Generalized Lower Bound on J ⋆ [4]) . Let λ min ( Q ) and λ min ( P ) denote the smallest eigenvalues of positive definite matrices Q and P , respectively. It follows that

for some constant 0 < C 0 ≤ (1 -ρ F ) 2 2 min { λ min ( P ) , λ min ( R ) / ∥ B ∥ , λ min ( Q ) / max { 2 , ∥ A ∥}} .

Next, we present the proof of Lemma 1.

D.3 Proof of Lemma 1

We restate the lemma below, which reduces the online learning objective ∑ t -1 ℓ =0 ψ ⊤ ℓ,t ( λ ) Hψ ℓ,t ( λ ) to a canonical online optimization formulation.

Lemma 3. Define ζ ℓ : I → R n as in (11) . Then for any t ∈ [ T ] , it follows that

Denote by g ( τ, ℓ ) := ( F ⊤ ) τ -ℓ P ( f ( s τ ; θ τ ) -f ( λ ◦ ˜ s τ | ℓ ; ˜ θ ℓ )) . We first note that

Since H = B ( R + B ⊤ PB ) -1 B ⊤ is positive semi-definite as we have shown in Appendix D.1, the function h ( x ) := x ⊤ Hx is convex. Applying the Jensen's inequality,

Summing over t = O,..., t - 1

0

t

t-1

Proceed from =t-1 to t= t

(2）(1)

M

=1

2

(FT)-P(f-fac)

W

Adding a new term &t

:= max{t - w + 1,0}

Need to modify previous We,'s

:= min{ +w - 1,}

Therefore,

where (20) follows from (19). The proof is illustrated in Figure 7. We now proceed to show our main results. The key structure of the lemmas and theorems is outlined in Figure 8.

E Proof of Theorem 4.2

We first state a detailed version of Theorem 4.2 below with explicit multiplicative constants.

Theorem E.1 (DISC for Linear Mixing) . Let H := B ( R + B ⊤ PB ) -1 B ⊤ . With our model assumptions, the competitive ratio of DISC with a linear mixing function satisfies

where σ min ( θ ) > 0 is the smallest singular value of θ ; s and θ are defined in Section 4.2; C 0 is defined in Lemma 2; C 1 := 2 ∥ H ∥ ( C F 1 -ρ F ∥ P ∥ s ) 2 , and C 2 := ∥ H ∥ ( C F ∥ P ∥ θ ) 2 .

Step 1: Consistency-Robustness Analysis of λ -CON We first show the following lemma, which highlights a tradeoff between consistency and robustness. The following definition of dynamic regret will be used as an alternative worst-case performance metric in our main results. Dynamic regret is a more general (and often more challenging to analyze) measure than classical static regret, which has been mostly used for stationary environments [74, 75].

Definition E.1 (Dynamic regret) . The dynamic regret of a policy π = ( π t : t ∈ [ T ]) is defined as the difference between the quadratic cost induced by the policy π , J ( π ) in (3), and the offline optimal cost J ⋆ := inf π J ( π ) subject to (2), i.e., DR ( π ) := J ( π ) -J ⋆ .

Lemma 4 (Consistency-Robustness Lemma) . With a fixed trust parameter λ ∈ I , the disentangled λ -confident policy in Section 3.1 has a dynamic regret of at most

where P, C F , η, ρ F are defined in Section 2; s and θ are defined in Section 4.2. The terms g con t and g rob t are functions of λ , defined as

Proof of Lemma 4. Denote by λ -CON the λ -confident policy, and J ( λ -CON ) the corresponding total cost induced by taking actions ( u 0 , . . . , u T -1 ) generated by λ -CON. Similarly, denote by J ⋆ the optimal total cost. Denote T := min { t + w -1 , T -1 } . Lemma 3 in [4] implies the following explicit form of J ( λ -CON ) -J ⋆ :

where κ ( τ ) ≡ 0 when τ > T . The sum of quadratic costs in (22) can be further bounded by

In above, the first term characterizes a total cost gap induced by inaccrurate ICA identifiability and predictions. The second term in (23) is the long-term error for disentangled times series predictions that are at least w steps away from the current time steps, denoted by h error t , which becomes 0 when t + w ≥ T -1 . It follows that

since by our assumption on F , the Gelfand's formula implies that there must exist a constant C F > 0 , ρ F ∈ (0 , 1) s.t. ∥ F t ∥ ≤ C F ρ t F for all t ≥ 0 . Moreover, for g error t , we can bound it from above by

where η t := ˜ θ t -θ is the mixing parameter error in (4) defined Section 2.2. Furthermore, noting ( ε τ | t (1) , . . . , ε τ | t ( k ) ) = ε τ | t := s τ -˜ s τ | t , we obtain

The right hand side of (26) contains two types of errors - the consistency error, denoted by g con t that occurs due to the estimated time series; and the robustness error, denoted by g rob ( t ) that arises when λ becomes small. Therefore, combining (23), (24), and (26) implies the following upper bound on J ( λ -CON ) -J ⋆ :

where η is defined in (4).

Step 2: Component-Wise Consistency Bound on g con t In the sequel, we bound g con t and g rob ( t ) respectively. First, the consistency error caused by inaccurate time series predictions can be bounded by

which can be further decomposed into a summation of component-wise latent variable time-series prediction errors by denoting ε ( i ) := ∑ T -1 t =0 ∑ T τ = t ( ρ ( τ -t ) F ε τ | t ( i )) 2 where ε τ | t = ( ε τ | t ( i ) : i =

1 , . . . , k ) , as defined in Section 2.2. Considering λ = ( λ (1) , . . . , λ ( k )) , it follows that

Step 3: Robustness bound on g rob t Next, we consider deriving a bound on g rob t . We obatin

which leads to

by applying the Cauchy-Schwarz inequality where ∥·∥ 4 denotes the ℓ 4 -norm. Denote by σ min ( θ ) > 0 the smallest singular value of the full column rank mixing matrix θ . Noting that ∥ s τ ∥ 4 ≤ ∥ s τ ∥ ≤ 1 σ min ( θ ) ∥ θs τ ∥ for all τ , and f ( s ; θ ) = θs in this linear mixing setting, applying Lemma 2,

for some constant 0 < C 0 ≤ (1 -ρ F ) 2 2 min { λ min ( P ) , λ min ( R ) / ∥ B ∥ , λ min ( Q ) / max { 2 , ∥ A ∥}} .

Step 4: Competitive Analysis of λ -CON Putting together Lemma 4, (27), and (29), since J ⋆ > 0 ,

where the last term can be further bounded from above by

Step 5: Online Learning of λ t via FTRL Consider a follow-the-regularized-leader (FTRL) optimization with an ℓ 2 -regularizer ∥ λ -λ 0 ∥ 2 . The following lemma can be proved based on the equivalent online convex optimization (OCO) form of the online mirror descent (OMD) (see the implementation described in Section 5) above as shown in Lemma 1. We reprise the OMD steps below.

Now, fix λ = ( λ t : t ∈ [ T ]) generated by DISC. Let us consider a pseudo-cost ∆ J † ( λ ) defined below, which is compared with the true objective ∆ J ( λ ) of DISC.

The lemma in the following holds.

Lemma 5. The online mirror descent (OMD) (32a) -(32b) generates a sequence λ = ( λ 0 , . . . , λ T -1 ) in DISC such that

Proof of Lemma 5. The gradient of the cost gap at time t ∈ [ T ] satisfies

where in (33) we have used the fact that H := B ( R + B ⊤ PB ) -1 B ⊤ is symmetric. Moreover, noting that ˜ s t | τ = ( ˜ s t | τ (1) , . . . , ˜ s t | τ ( k ) ) , in (33) above,

denotes an n × k Jacobian matrix J ( τ, t ) of g ( τ, t ) with respect to λ whose ( i, j ) -th entry is ∂g ( τ,t ) ( i ) ∂λ ( j ) . Normalizing the predicted time series yields that

As we have verified in Appendix D.1, each g ( τ, t ) ⊤ ( λ ) Hg ( τ, t ) is a convex function of λ ∈ I . Therefore, based on the static regret analysis for OCO [23, 76], FTRL with an ℓ 2 -regularizer guarantees (after optimizing β ) that

Applying Assumption 2 and noting that λ 2 ℓ -λ 2 ℓ -τ ≤ O ( | λ ℓ -λ ℓ -τ | ) for all ℓ ∈ [ T ] , we conclude that ∆ J ( λ ) -∆ J † ( λ ) = o ( T ) , since for τ that is not a constant, the exponent ρ τ = o (1) .

Denote by λ ∗ the optimal solution that minimizes J ( λ -CON ) . Using 30 and (34) above, we conclude that

̸

whose right hand side can be bounded respectively via Lemma 4 and Lemma 5. Furthermore, noting that f ( s t ; θ ) = 0 for all t ∈ [ T ] , hence ∥ θs t ∥ > 0 . As a result, Lemma 2 implies J ⋆ = Ω( T ) . Therefore, using Lemma 1,

After optimizing over λ ∈ I ,

F Proof of Corollary B.1

Under the classic assumptions such that E [ s t ( i )] = 0 , E [ s 2 t ( i ) ] = 1 , E [∣ ∣ s 5 t ( i ) ∣ ∣ ] ≤ M , and the fourth cumulant | cum 4 ( s t ( i )) | ≥ ∆ for all t ∈ [ T ] and i = 1 , . . . , k , the Recursive Fourier PCA algorithm in [75] is a polynomial-time and it guarantees (c.f. Theorem 1 in [75]) that for all t ∈ [ T ] ,

with high probability, which yields η = O ( (log n ) 7 / 2 k 1 / 2 ∑ t ∈ [ T ] 1 t ) = O ( (log n ) 7 / 2 k 1 / 2 log T ) . Therefore, with high probability, the competitive ratio bound in the corollary holds.

G Proof of Theorem 4.1

Denote T := min { ℓ + w -1 , T -1 } . Similar to the analysis in the proof of Theorem 4.2 in Appendix E, we obtain

WLOG, we assume f is linear such that f ( s ; θ ) = θs for some n × k matrix θ = ( θ ij ) and the estimated mixing matrix is accurate so that η = 0 . Then it follows that

where

̸

Since λ -CON is (1 + o (1)) -consistent, we must have J ( λ -CON ) -J ⋆ ≤ o (1) J ⋆ (by our model assumption that s t = 0 for all t ∈ [ T ] , J ⋆ > 0 ). Combining this with (36) above, it is necessary to have

The competitive ratio is the worst-case ratio for all A,B,Q,R,f,θ , and ( s t : t ∈ [ T ]) , we must have λ = 1 k to guarantee (37) above.

Now, consider an adaptive offline adversary that generates ε τ | ℓ = µs τ | ℓ for all ℓ ≤ τ ≤ ℓ . Since λ = 1 k , we obtain

Setting µ = log T , above implies CR ( λ -CON ) = ω (1) .

H Proof of Theorem 4.3

We first state a detailed version of Theorem 4.2 below with explicit multiplicative constants.

Step 1: Consistency-Robustness Analysis of λ -CON We first show the following lemma, which highlights a tradeoff between consistency and robustness.

Lemma 6. With a fixed trust parameter λ ∈ I , the disentangled λ -confident control ( DISC ) has a worst-case dynamic regret of at most

where H := B ( R + B ⊤ PB ) -1 B ⊤ ; f max := f 0 + C f s with f 0 := | f (0) | ; P, C F , ρ F , w are defined in Section 2; C f denotes the Lipschitz constant of f . The terms g con t and g rob t are functions of λ , defined as

Proof of Lemma 6. Similar to the analysis in the proof of Theorem 4.2 in Appendix E, we obtain

where we simplify the latent perturbations and the corresponding predictions as f τ := f ( s τ ; θ ) and ˜ f τ | t ( λ ) := f ( λ ◦ ˜ s τ | t ; ˜ θ t ) . Especially, ˜ f τ | t ( λ ) := 0 when τ > T . Thus, the total cost gap can be

bounded by

Note that by Assumption 1 f is continuous with a Lipschitz constant C f > 0 , hence, f is bounded, since ∥ s τ ∥ ≤ s . Denote by f max ≤ f 0 + C f s the largest value of f where f 0 := | f (0) | . The second term h error t in (23) can be bounded similarly to (24) in the proof of Theorem 4.2 (see Appendix E):

The first term g error t in (23) characterizes a total cost gap induced by inaccrurate mixing matrix estimation and time series prediction errors. It follows that

where we have used the fact that ∥ F t ∥ ≤ C F ρ t F for all t ∈ [ T ] with a spectral radius ρ F ∈ (0 , 1) .

Recall that η t := ˜ θ t -θ is the mixing parameter error in (4) defined Section 2.2. Furthermore, noting ( ε τ | t (1) , . . . , ε τ | t ( k ) ) = ε τ | t := s τ -˜ s τ | t , continuing from (42), we get

The right hand side of (43) contains two types of errors - the consistency error, denoted by g con t that occurs due to the estimated time series; and the robustness error, denoted by g rob ( t ) that arises when λ becomes small. Therefore, rearranging the terms and combining (40), (41), and (43) imply the following upper bound on J ( λ -CON ) -J ⋆ :

with η defined in (4).

Furthermore, we can bound the terms g error t and h error t separately as:

where f -1 is the inverse function of the mixing function f . Since by Assumption 1, f is bijective, f -1 must exist. Thus, (45) and (46) together imply

Step 2: Online Learning of λ t via FTPL Consider a follow-the-perturbed-leader (FTPL) optimization [28, 25], with:

The following lemma is a result of [25] by fixing each coordinate σ t ( i ) ( i = 1 , . . . , k ) an IID random variable from an exponential distribution with some parameter α > 0 .

Lemma 7. The FTPL procedure (48) above generates a sequence λ = ( λ 0 , . . . , λ T -1 ) in DISC such that

Similarly to the proof of Theorem 4.2, applying Assumption 2, J ( DISC ) -J † = o ( T ) . Denote by λ ∗ the optimal solution that minimizes J ( λ -CON ) . Putting together (44), (47), and (49), we conclude

yielding the following after optimizing over α > 0 :

where we denote C 3 := ∥ H ∥ ( C F 1 -ρ F C f ∥ P ∥ ) 2 and C 4 := ∥ H ∥ ( C F C f ∥ P ∥ ) 2 and have used J ⋆ = Ω( T ) in (50) (implied by Lemma 2 and the assumption that f is non-zero and Lipschitz continuous, similar to the proof of Theorem 4.2 in Appendix E) to derive (51).

and

NeurIPS Paper Checklist

1. Claims

Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?

Answer: [Yes]

Justification: The claims made in the abstract and introduction align with our theoretical and experimental results.

Guidelines:

The answer NA means that the abstract and introduction do not include the claims made in the paper.

The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.

The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.

It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.

2. Limitations

Question: Does the paper discuss the limitations of the work performed by the authors?

Answer: [Yes]

Justification: The limitations of this work are discussed in our last concluding remark section.

Guidelines:

The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.

The authors are encouraged to create a separate "Limitations" section in their paper.

The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.

The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.

The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.

The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.

If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.

While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.

3. Theory Assumptions and Proofs

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

Answer: [Yes]

Justification: We have clearly stated our model assumptions for the derived results to hold.

Guidelines:

The answer NA means that the paper does not include theoretical results.

All the theorems, formulas, and proofs in the paper should be numbered and crossreferenced.

All assumptions should be clearly stated or referenced in the statement of any theorems.

The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.

Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.

Theorems and Lemmas that the proof relies upon should be properly referenced.

4. Experimental Result Reproducibility

Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

Answer: [Yes]

Justification: We have detailed the settings and parameters used in the experiments, and we will further release our code after the anonymous review stage.

Guidelines:

The answer NA means that the paper does not include experiments.

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.

If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.

Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.

While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example

(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.

(b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.

(c) If the contribution is a new model (e.g., a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).

(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.

5. Open access to data and code

Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

Answer: [NA]

Justification: Our contributions focus on the theoretical side. The experimental results shown in the submitted manuscript do not depend on private datasets and can be reproduced using the provided settings and parameters.

Guidelines:

The answer NA means that paper does not include experiments requiring code.

Please see the NeurIPS code and data submission guidelines ( https://nips.cc/ public/guides/CodeSubmissionPolicy ) for more details.

While we encourage the release of code and data, we understand that this might not be possible, so 'No' is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).

The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines ( https: //nips.cc/public/guides/CodeSubmissionPolicy ) for more details.

The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.

The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).

Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.

6. Experimental Setting/Details

Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?

Answer: [Yes]

Justification: We have clearly stated our experimental settings and details to reproduce the results.

Guidelines:

The answer NA means that the paper does not include experiments.

The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.

The full details can be provided either with the code, in appendix, or as supplemental material.

7. Experiment Statistical Significance

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

Answer: [Yes]

Justification: When there is randomness in our experiments, we characterize the variance using shadow regions.

Guidelines:

The answer NA means that the paper does not include experiments.

The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)

The assumptions made should be given (e.g., Normally distributed errors).

It should be clear whether the error bar is the standard deviation or the standard error of the mean.

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.

For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).

If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.

8. Experiments Compute Resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: [Yes]

Justification: Our contributions focus on the theory side, and the experimental setup is sufficiently basic that it does not require intense computing resources such as GPUs.

Guidelines:

The answer NA means that the paper does not include experiments.

The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.

The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.

The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).

9. Code Of Ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines ?

Answer: [Yes]

Justification: The research conducted conform with the NeurIPS Code of Ethics.

Guidelines:

The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.

If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.

The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).

10. Broader Impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: [Yes]

Justification: We have discussed both positive and negative societal impacts in our concluding remark section.

Guidelines:

The answer NA means that there is no societal impact of the work performed.

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.

Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.

The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.

If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).

11. Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?

Answer: [NA]

Justification: The paper poses no such risks.

Guidelines:

The answer NA means that the paper poses no such risks.

Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.

Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.

We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.

12. Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer: [NA]

Justification: The paper does not use existing assets.

Guidelines:

The answer NA means that the paper does not use existing assets.

The authors should cite the original paper that produced the code package or dataset.

The authors should state which version of the asset is used and, if possible, include a URL.

The name of the license (e.g., CC-BY 4.0) should be included for each asset.

For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.

If this information is not available online, the authors are encouraged to reach out to the asset's creators.

13. New Assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: [NA]

Justification: The paper does not release new assets.

Guidelines:

The answer NA means that the paper does not release new assets.

Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.

The paper should discuss whether and how consent was obtained from people whose asset is used.

At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.

14. Crowdsourcing and Research with Human Subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

Answer: [NA]

Justification: The paper does not involve crowdsourcing nor research with human subjects. Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.

According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.

15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

Answer: [NA]

Justification: The paper does not involve crowdsourcing nor research with human subjects.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.

For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents a novel policy, DISC, designed to manage uncertain perturbations in dynamical systems by learning a confidence parameter online. The focus is on integrating predictions from machine learning (ML) tools, which are often unreliable, into linear quadratic control (LQC) frameworks. The key innovation of DISC is its ability to harness accurate predictions when available and mitigate the impact of erroneous forecasts, achieving a balance between consistency and robustness.\n\nThe paper begins by discussing the challenges of using ML predictions in control systems, particularly the issue of reliability due to factors like high model variability and out-of-distribution generalization issues. The authors propose a policy, λ-CON, which extends existing methods by adapting a vectorized confidence parameter for each latent variable. However, λ-CON has limitations in achieving optimal consistency and robustness. DISC addresses these limitations by dynamically learning the confidence parameter through online learning, ensuring better performance regardless of prediction accuracy.\n\nTheoretical results provide competitive ratio bounds for DISC under both linear and general mixing functions, demonstrating its robustness and consistency. The paper also includes experiments in two real-world scenarios: drone navigation with mixed disturbances and voltage control in power grids. These experiments validate DISC's effectiveness, showing that it outperforms baseline policies in terms of cost and adaptability to rapid changes in the environment.\n\nSTRENGTHS:\n- The introduction of a dynamic confidence parameter that adapts online is an important advancement in integrating ML predictions into control systems.\n\n- The paper provides theoretical guarantees for DISC, offering competitive ratio bounds that highlight its robustness and consistency.\n\n- The experimental validation in real-world scenarios, such as drone navigation and voltage control, demonstrates the practical relevance and effectiveness of DISC.\n\n- DISC's ability to adapt to varying levels of prediction accuracy and environmental changes is a valuable feature for real-world applications.\n\nWEAKNESSES:\n- The theoretical analysis relies on several assumptions, such as the continuity and invertibility of the mixing function, which may not hold in all real-world scenarios.\n\n- While the experiments support the theoretical findings, the datasets used might not fully capture the diversity and complexity of real-world applications.\n\n- The method may be complex to implement in practice, especially in real-time systems where computational resources are limited.\n\nQUESTIONS:\n- How does DISC scale with the increase in the number of latent variables and the complexity of the system? Are there any practical limits to the number of latent components DISC can handle efficiently?\n\n- What are the computational requirements for implementing DISC in real-time systems? Can the authors provide insights or benchmarks on the computational overhead introduced by the online learning of the confidence parameter?\n\n- Have the authors considered extending DISC to handle nonlinear dynamical systems? If so, what challenges do they anticipate, and are there any preliminary results or insights they can share?\n\n- Can the authors provide a more extensive comparison between DISC and other existing robust control methods? How does DISC's performance compare in terms of both consistency and robustness?\n\nLIMITATIONS:\n- The study mainly focuses on specific applications like drone navigation and voltage control, and its applicability to other fields remains uncertain.\n\n- The competitive ratio bounds depend on prediction errors, which might limit the policy's effectiveness in scenarios with high prediction inaccuracies.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-13"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe submission considers LQC with perturbations. The considered problem is interesting with potential applications on LQC with noisy/unreliable ML predictions. \n\nThe submission improves the existing method in Robustness and consistency in linear quadratic control with untrusted predictions, where there is a constant parameter determining the confidence of predictions. The submission is novel by learning the parameter in an online manner. \n\nBoth theoretical and empirical results are provided supporting the proposed method.\n\nSTRENGTHS:\nThe proposed method is novel by learning the confidence parameter in an online manner. Such a method can have many applications in control with noisy predictions. \n\nThe proposed method is analyzed both theoretically and empirically. Improved results are achieved.\n\nWEAKNESSES:\nThe writing of the submission is really hard to follow. \n\nFirst of all, the submission mentions some concepts that are abstract and unprecise, like \"best-of-both-worlds\" and \"consistency and robustness\". \n\nFurther, the introduction has gets into technical details without proper definitions and explanations. Examples include \"(1 + o(1))-consistent\", \"ω(1)-robust\", CR(DISC), and \"Best-of-both-worlds utilization\".\n\nFinally, the update rule of $\\lambda_t$ in section 3.2 is missing. It is not self-contained to just refer to some other papers for such an important piece. Without a clear and explicit definition of the update rule of $\\lambda_t$, it is very hard to reproduce or evaluate the proposed method.\n\nQUESTIONS:\nThe claim on line 157 is confusing. Why when $\\lambda=0$, the $f$ term will disappear? This statement requires further assumptions on $f$ which are currently missing. \n\nWhat is the update rule of $\\lambda_t$? \n\nAlso, I am wondering whether it might be a easy/doable direction to extend the proposed method to the case with a dynamics like $x_{t+1} = Ax_t+ B\\mu_t f(s_t)$, where the ML predictions also affect the effect of the actions.\n\nLIMITATIONS:\nNA", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-15"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper considers the problem of LQR where the added perturbations may not be iid Gaussian noise but depend on some latent variables (which are themselves predicted using an ML model). In particular, the authors consider the dynamics given by \n\nx_{t+1} = A x_{t} + B u_t + f(s_t, \\theta) \n\nwhere s_t and \\theta are themselves predicted (up to some future window of length w) by an ML procedure. The goal is to provide an algorithm that (a) enjoys a good competitive ratio w.r.t. the optimal policy when the predictions are correct, and (b) is robust to the prediction of the latent variables. The learner knows A, B and f. \n\nThey provide theoretical results when the perturbations could be linear functions of the latent variables, as well as general functions, and provide experiments on drone navigation and power grid. The key idea in the algorithm is to first predict a \\lambda vector that decides how much to trust each latent vector, and then to run a \\lambda-confident policy. The authors use FTRL style online algorithm to predict \\lambda.\n\nSTRENGTHS:\nThe paper considers an important problem of using ML predictions in control and being robust to them. As far as I know, bounding competitive ratio for both consistency and robustness simultaneously has not been considered before in the control literature. \n\nThe lower bound in Theorem 4.1 that shows that a fixed \\lambda does not suffice is interesting. In hindsight, it makes sense since the ML predictions can change arbitrarily (but it is useful to present it). \n\nSufficient experimental evaluation.\n\nWEAKNESSES:\nThe techniques are not fundamentally different from prior works, however, the fact that an online predictor of \\lambda parameter followed by plug-in control policy in eqn (5) suffices for robustness is interesting. I still support accepting the paper. \n\nMinor typos:\nLike 311 -- \"do not reply\" -> \"do not imply\" \nLine 21 -- \"adaptively\" -> \"can adapt\"\n\nQUESTIONS:\nQuestions: \n1. Is there a straightforward way to incorporate unknown A and B matrices? \n2. Is there a way to use a fixed \\lambda but get o(T)-robustness while being O(1)-consistent, or vice-versa in Theorem 4.1?\n\nLIMITATIONS:\nNA", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-15"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper investigates the setting of linear quadratic control with latent perturbations. In particular, the state evolution does not only depend on the current state and the control input, but also on various latent variables that are mixed through a linear or nonlinear mixing function of unknown parameterization. The authors attempt to address this problem by considering a $\\lambda$-confident policy that balances consistency and robustness. However, they shows that a fixed confidence value cannot achieve a good consistency and robustness tradeoff. Motivated by this, the authors suggest to use a disentangled confidence policy, where the algorithm adaptively learns the trust parameter in an online manner. The authors are able to prove that their scheme achieves near-optimal competitive ratio bounds for both linear and general mixing cases (under reasonable assumptions). The authors confirm the significance and superior performance of adaptive learning for the trust parameter using interesting use cases inspired by real-world applications.\n\nSTRENGTHS:\n- The problem statement is well-motivated and nice practical examples are provided.\n- The paper makes several interesting contributions. I particularly liked the fundamental gap in Theorem 4.1, which motivates that we need to learn the confidence parameter $\\lambda$ in an online manner. Furthermore, the authors provide competitive ratio results, not only for the linear case, but also for the general mixing case. The assumptions (e.g., Lipschitz continuity and bijectivity of $f$ are not so weak, but also not unreasonable).\n- On the theory side, the paper is quite rich and the proofs seem to work in general (even though I do have some concerns, as I explain in the following sections).\n- The experimental evaluation confirms the fact that online learning of the $\\lambda$-confidence parameter is beneficial in two interesting settings. Results look good overall.\n- The authors discuss several aspects of their work in the Appendix and most aspects are explained quite well.\n\nWEAKNESSES:\n- The paper contains several errors in various formulas. In the Questions section below, I have mentioned several problems I was able to identify, but there could be more. These errors do not seem to be critical, but the manuscript still needs a very careful proofreading to fix them.\n- I was not clear about certain theoretical claims, like the $l_4$ norm or certain assumptions. Please also see Questions section below. I believe the authors should be able to address all of them.\n- Even though the paper offers theoretical result on general mixing functions, all experiments seem to focus on linear mixing. The authors explain that the nonlinear mixing case is indeed much harder than the linear one (where Fast ICA is used). To me, the problem in the general nonlinear case is that the mixing parameter $\\theta$ can be much harder to estimate accurately, so the estimation of the latent variables will also be negatively affected. So, I am not sure how practical the proposed algorithms would be for the general nonlinear case. Perhaps simple experiments with just 2 latent variables (but nonlinearly mixed) could offer some preliminary insights.\n\nQUESTIONS:\n- In Equation (8), it seems to me that the matrix $H$ was never previously introduced.\n- In Equation (8), it seems to me that the $t$ index is not present in the right hand side of (8). Is it possible that the authors meant $\\psi_{l,t}(\\lambda)$ instead of $\\psi_{l,T}(\\lambda)$?\n- Similarly, in Equation (9), shouldn't the upper index on the summation operator be $\\min[l+w-1,t-1]$? At least, this is what the authors probably imply in Line 169.\n- In Line 164, it is explained that any $\\lambda_t$ takes values in $[0,1]^k$, but then in Line 168 $\\lambda$ appears to have $T-1$ components. Obviously, this cannot work for the Hadamard product $\\lambda \\circ \\tilde{s}\\_{\\tau \\mid l}$. Shouldn't any given $\\lambda$ have $k$ components in total? I believe the authors probably mean the sequence of $\\lambda$'s in Line 168, but I think this should be explained more clearly.\n- In Figure 2a, how exactly is the offline optimal policy generated? Do the authors rely on the prior literature for this purpose?\n- The experiments by the authors seem to rely on linear mixing matrices. As the authors explain, this is by far the simplest setting, unlike general nonlinear mixing functions. Have the authors experimented with nonlinear mixing functions, even with just 2 latent variables? It would be interesting to see how the whole framework (including the estimation of the latent variables) is affected in such a case. Furthermore, the paper's theory covers nonlinear/general mixing functions.\n- In Line 779, the authors make us of the $l_4$ norm. They claim that this is because of the Cauchy-Schwarz inequality. My understanding is that the Cauchy-Schwarz inequality $\\mid <u, v> \\mid \\geq \\mid\\mid u \\mid\\mid \\cdot \\mid\\mid v \\mid\\mid$ (where the norms are $l_2$-norms is only true for the $l_2$ norm. I was not clear how the $l_4$ norm suddenly appeared there. Could the authors detail their calculations?\n- Equation (42) in Line 850 also involves the $l_4$ norm, but I think it should be totally analogous to the point above.\n- In Equation (32), the authors define $\\psi_{t,T}$ but then in the right-hand side of (32) there is no $t$. Why not just use the same inner summation expression as in (31) instead of introducing new indexes? Furthermore, the inner summation expression in (31) has $-1$ as the upper index - why then in (32) this is changed to $\\min[l+w-1,T-1]$ (or, perhaps, $\\min[t+w-1,T-1]$)?\n- Line 813 makes sense mathematically if $(R+B^{\\top}PB)^{-1}$ is a positive definite matrix. Have the authors shown that formally? I was not so clear what is going on with matrix $P$ appearing inside it.\n- I am not sure I understood the claim in Line 801.The authors claim that Lemma 2 implies that $J^*=\\Omega(T)$ because $f(\\mathbf{s}_t,\\theta)$ is never 0 by assumption. However, in theory someone might claim that the claim could be violated if $f$ can become arbitrarily close to 0 but never 0 exactly. Obviously, this is not possible for continuous function, but might be true for strange functions. For general functions, one might instead wan to impose that $f$ is lower bounded by some positive constant. That said, I feel that this is not a problem in this work, because $f$ is assumed Lipschitz, hence it must also be continuous (and even uniformly continuous). \n- For a more rigorous exposition, could the authors discuss why $\\zeta_l(\\lambda)^{\\top}H\\zeta_l(\\lambda)$ are convex functions of $\\lambda$ $\\forall l$ in the linear mixing case or when $f$ is convex, so that OCO regret bounds are applicable in these cases?\n\nLIMITATIONS:\nNo concerns.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-17"}