Title: Efficient Spectral Control of Partially Observed Linear Dynamical Systems
Venue: NeurIPS 2025
Accepted: yes
Keywords: Online Convex Optimization

Online Control

Linear Dynamical Systems
Abstract: We propose a new method for the problem of controlling linear dynamical systems under partial observation and adversarial disturbances. Our new algorithm, Double Spectral Control (DSC),  matches the best known regret guarantees while exponentially improving runtime complexity over previous approaches in its dependence on the system's stability margin. Our key innovation is a two-level spectral approximation strategy, leveraging double convolution with a universal basis of spectral filters, enabling efficient and accurate learning of the best linear dynamical controllers.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis work studies the problem of online control of partially observed linear dynamical systems, where the cost function is convex and chosen at each time step by an adversary. In this adversarial setting, the authors propose an online optimization algorithm that matches the best known asymptotic regret, while considerably improving the dependency of the computational complexity with respect to the spectral radius. Theorem 4.1 provides the key theoretical results and leads to the claimed complexity bounds. Minimal experiments are included to empirically support the theoretical contributions.\n\nSTRENGTHS AND WEAKNESSES:\nOverall, the paper is clearly written and presents a significant improvement over prior work in terms of computational efficiency. However, my main concern is the lack of motivation and contextualization of this main contribution, particularly in the introduction and broader discussion.\nI would also suggest providing more interpretative insight into the proposed method and to better contextualize the theoretical results by discussing potential practical applications or scenarios where such improvements could be impactful.\n\n\n\n### Strengths\n\n- The paper is clearly written, and the mathematical framework is well introduced.\n- The related work is appropriately referenced and contextualized.\n- The results are clearly presented, with intuitive explanations accompanying the formal statements.\n- The theoretical contributions are rigorously derived, with relevant definitions, lemmas, and theorems.\n- The limited experiments suggest that for a fixed memory budget, the proposed algorithm outperforms LQG and GRC controllers.\n- While I am not a specialist in this subfield, the mathematical arguments appear sound.\n\n### Weaknesses\n\n\n**Presentation and motivation of the main contribution**\n\n- The main contribution of the paper lies in the improved runtime complexity of the proposed algorithm.\nWhile I recognize that the focus of this work is theoretical, I believe that a brief discussion of the potential implications or contexts where improved complexity would improve the practical relevance of the work. For instance, it remains unclear in which settings compute becomes a primary bottleneck, especially the spectral radius dependence, or how this improvement could concretely benefit applications. \n- As noted in the Related Work section, convex parameterizations have been explored in other approaches. However, the paper does not clearly explain how the defined memory-less convex loss compares to others such as GRC. An intuitive explanation of why the proposed relaxation leads to runtime improvements would be valuable.\n\n**Modeling choices**\n\n- The motivation behind the choice of state-space model and convex loss is not sufficiently grounded in real-world examples. Applications are only briefly mentioned in the introduction.  \n- For example, many classical control problems involve tracking a desired state, where the cost function is a convex function of the state-input pair, $c_t(x, u)$, not the observation-input pair, $c_t(y, u)$. While I understand that the latter allows for theoretical tractability and follows conventions in the literature, it would be helpful to provide some discussion or illustrative examples of how such time-varying convex costs on observations and inputs arise in real-world settings.\n\n**Experimental validation**\n\n- The experimental section is minimal and relegated to the appendix. Key details, such as the explicit form of the time-varying cost function, are missing.\n- Importantly, although the computational improvement is the main contribution of the paper, the runtime is not reported in the experiments.\n- Including more extensive experiments in the main paper would enhance the impact and help readers assess the practical relevance of the method.\n\n\n\n**Technical clarification**\n\n- I believe that there is an issue in the definition of $\\gamma$. Indeed, $1-\\gamma$ is defined as the spectral radius of the closed loop matrix, which is denoted as $C_\\pi$ in the submission, and depends on policy $\\pi$. Rather, $1-\\gamma$ should be defined as the spectral radius $A$ for the definition to be consistent with the results reported in Table 1.\n\nQUESTIONS:\n1. Comparing Equations (1) and (3), shouldn't Line 141 say: “the system would have produced had the learner applied zero disturbances”?\n\n2. What is meant in Line 144 by “natural observations are independent of the learner’s parameters”? How does this enable convex optimization?\n\n3. What does “learning” refer to in this context? Since the system matrices are known and the cost function is defined over known quantities, it is unclear in what sense this is an estimation or learning problem.\n\n4. What is the interpretation of the filter matrix $H$, and how does it help capture long-range dependencies?\n\n5. How could the proposed approach be adapted if the matrices $A$, $B$, and $C$ were unknown?\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 2, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-06-17"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a novel algorithm for controlling linear dynamical systems under partial observation of the latent states and adversarial disturbances. The new algorithm (called DSC) can achieve comparable regret with some prior methods, but has a significant advantage on runtime at eat iteration. The authors show that the per-step runtime of DSC is only $polylog(T/\\gamma)$, where $\\gamma$  is the stability margin of the system. This improves prior art significantly for small $\\gamma$.\n\nSTRENGTHS AND WEAKNESSES:\n### Strengths:\n- The writing of the paper is clear and easy to understand about the problem setting, formulation, and related results.\n- The proposed method seems to have a significant advantage on per-step runtime in term of stability margin, but maintain the competitive regret.\n\n### Weaknesses:\n- This paper is mostly theoretical with limited validation on practical settings. The authors did a simulation with a simple setup and then compared with two other related methods. \n- The main benefit of the proposed method mostly bases on the concept of stability margin. However, the authors have a limited discussion about this concept. This could pose a concern on how practically significant their method is. It could be useful for the readers if the authors can provide examples/situations in practical applications so that $\\gamma$ is (very) small. Otherwise, the proposed method may not be practically beneficial.\n- Many references lack information.\n\nQUESTIONS:\n- Can the authors provide examples/situations in practical applications so that $\\gamma$ is (very) small?\n- What is $poly(\\gamma^{-1})$ in each method in Table 1?\n- Can we trade off between the regret and per-step runtime?\n\nLIMITATIONS:\nThe authors already discussed some limitations and tried to resolved, e.g. providing some experiments. There seems to be some limitations as pointed out above.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 2, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-06-26"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposes a new method for online control of partially observed linear systems that is amenable to efficient online updates. The setting of the paper is that of online non-stochastic control where the costs and (potentially adversarial) disturbances are revealed sequentially.\nThe main challenge with previous approaches ([22]) is that the run-time per step (one online update) scales inversely proportially ($O(1/\\gamma)$) with the stability margin $\\gamma$ of the system, as the margin approaches zero. In particular, the number of tunable parameters scales polynomially with $1/\\gamma$.  By following the proposed approach the run-time scales with the improved rate of $O(\\log(1/\\gamma))$, which is an exponentially better for very small $\\gamma$. In the meantime, we preserve the standard $O(\\sqrt{T})$ regret guarantees of the convex cost setting (where $T$ is the total number of steps taken).\n\nThe main idea is to pre-filter (convolution operator) the observations twice with appropriate filters that are selected as the top eigenvectors of specific Hankel matrices. Then, the controller is obtained as a (convex) linear combination of the pre-filtered observations,where the linear coefficients are tuned online. Under certain conditions on the system and controller class, these specialized Hankel eigenvectors compress the information of the observations. As a result, it is sufficient that the number of tunable parameters scale only logarithmically with $1/\\gamma$.\n\nThese conditions require the system to be stable, diagonalizable, with positive real eigenvalues in both open-loop and closed-loop.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**\n\nThe idea of pre-filtering observations to improve dependencies on the stability margin is reasonable and interesting.  While I wouldn't rank it as the most important practical aspect it has theoretical interest. It can be impactful when the system exhibits long memory and slow attenuation. The analysis seems sound and correct. The strongest point of the paper is that the number of tunable controller parameters scales logarithmically with the inverse stability margin. The idea of pre-filtering has been studied before in prior work, e.g. in [7] for fully observed systems. Interestingly, here double filtering is used here for partially observed systems which is a non-trivial change. \n\n**Weaknesses**\nThe results are quite preliminary and lack generality. Moreover, little intuition is provided about why the method works. \nBased on the responses I am willing to improve my evaluation. To simplify the authors' response I included the actionable items in the questions section. \n\n**1.** Assumption 3.5 and Definition 3.4 seem extremely restrictive. They require the following: First, the controller itself is required to be implemented as a stable diagonalizable system with positive real eigenvalues. Second, the overall closed-loop dynamics should be stable diagonalizable with positive real eigenvalues. Third and perhaps more importantly, Assumption 3.5 requires the open-loop dynamical system itself to be stable, diagonalizable and to have positive real eigenvalues. \nThe limitations section should be expanded to discuss this in more detail. Currently, only issue 1 is discussed, while issue 3 is hidden; I only understood it while going through the proofs. \n\nAssuming stable and diagonalizable is ok. However, assuming systems with positive and real eigenvalues (both in open-loop and in closed-loop) is very peculiar. I have doubts that this can include meaningful systems. Almost all physical systems of interest exhibit oscillations, meaning they have complex eigenvalues. Even physical systems with positive real eigenvalues, like higher order integrators (which are excluded since they are not diagonalizable), will have complex eigenvalues in closed-loop. I couldn't find any evidence supporting footnote  1 \"This requirement is for tractability of analysis alone, and we show empricially that our method works regardless of it\".\n\n**2.** The paper does not compare the regret order (with respect to $\\gamma$) with that of prior work. It only compares run-times. \n\n**3.** The literature review is very biased with 16/24 references containing the same author. The paper lacks references to works from the control community. \n\n**4.** There is little intuition about how spectral filters work. \n\n**5.** The paper does not explain what the difference with [7] is and why we need double filtering instead of single.\n\nQUESTIONS:\n**Major comments**\n\nI am willing to improve my evaluation based on the authors response. To summarize, the limitations should be discussed more explicitly, comparison with prior work should be expanded, the literature review should be enhanced and some aspects of the analysis/contribution should be clarified.\n\n**1.** Dealing with weakness 1. In the limitations, it should be acknowledged that this assumption is unrealistic for actual physical systems. If the authors provide an example of a meaningful real-world physical system that satisfies the conditions I will improve my rating even further. The simulation should also include examples of systems that violate the conditions (in both open and closed loop) to support the claim of the footnote. In any case, there should be more discussion. Why are these assumptions needed? Is it an artifact of the proof? A fundamental limitation? What breaks in the case of complex or negative eigenvalues or in the case of Jordan blocks? Would we need a different Hankel matrix?\n\n**2.** Dealing with weakness 2. It should be clarified if the order of $1/\\gamma^{11}$ in Theorem 4.1 is worse/better/same compared to [22].  From this perspective, the claim \"optimal regret guarantees\" in the caption of Table 1 is currently unclear.\n\n**3.** Dealing with weakness 3. The paper would benefit from the addition of more papers from the control community. For example, the setting of Hinfinity control where the controller acts against adversarial disturbances seems quite relevant. Variations of this idea include mixed H2/Hinfty control, risk sensitive control, regret optimal control, minimax adaptive control. The following papers might be relevant and contain useful references\n\nT. Basar and P. Bernhard, $\\mathcal{H}_{\\infty}$, \"Optimal Control and Related Minimax\nDesign Problems — A dynamic Game Approach\", 2nd ed. Boston, MA,\nUSA: Birkhauser, 2008\n\nKaiqing Zhang, Bin Hu, Tamer Basar, \"Policy Optimization for $\\mathcal{H}_2$ Linear Control with $\\mathcal{H}\\_{\\infty}$ Robustness Guarantee: Implicit Regularization and Global Convergence\", SIAM, 2021 \n\nG. Goel and B. Hassibi, \"Regret-Optimal Estimation and Control,\" in IEEE Transactions on Automatic Control, vol. 68, no. 5, pp. 3041-3053, May 2023\n\nO. Kjellqvist and A. Rantzer, \"Output Feedback Minimax Adaptive Control\", arxiv, 2024\n\n**4.** Dealing with weakness 4. The paper could provide more intuition on how spectral filters work. I had to go to [15] to understand that the Hankel eigenvectors compress information. I recommend to include a brief discussion here as well. \n\nIt could also help to provide more intuition on what the main challenge in [22] is. Why does the run-time scale with $1/\\gamma$? It seems that a crucial component is studying the inequality $(1-\\gamma)^m\\le c$, where $m$ is the number of parameters and $c$ can be arbitrary. This inequality is equivalent to $m\\ge -\\log c/\\log(1-\\gamma)$. Although there is a logarithm, as the margin goes to $0$, we have $-1/\\log(1-\\gamma)\\sim 1/\\gamma$. I think the paper would benefit from such a discussion.\n\n**5.** Dealing with weakness 5. Please explain why we need double filtering in more detail. \n\n**6.** Convolution implementation. A naive implementation of the prefiltering steps would lead to a run-time that is polynomial with $1/\\gamma$. The reason is that naive convolutions involve $m,\\tilde{m}$ mutliplications. In turn, $m,\\tilde{m}$ scale polynomially with the inverse stability margin. The paper claims that this limitation can be lifted by implementing the convolutions efficiently.\nHowever, the paper does not provide enough details to understand how this is done. Unfortunately, the proof of Corollary 4.4 is informal and handwavy. Could you please provide more details? I suggest to make the proof of Corollary more precise by referring to specific results in [2] in a more principled way.\n\n**7.** Please provide more details about the simulations. Does matrix A satisfy all the restrictive stability assumptions including positive real eignevalues? Have you tried a matrix A that does not? What is the frequency of the sinusoid. Have you tried several frequencies? (Otherwise, it might be a matter of luck that the controller works well with a specific frequency.)\n\n**8.** In line 129 the following is stated \"– Appendix C reports preliminary empirical evaluations supporting our theoretical findings\". The theoretical results only claim that the proposed method has superior runtime. They do not claim anything regarding the performance or the regret order. The simulation results show that the proposed method is superior compared to [22] but they do not compare runtimes. Hence, I am quite confused since the theoretical claims and the simulations findings are about different things.\n\n**Other comments**\n**i)** Can we have measurement noise?\n\n**ii)** I couldn't find the definition of $\\mathcal{A}_{CL}$ (mentioned in line 194) anywhere. Could you clarify the definition?\n\n**iii)** What is the difference between frequency domain transformations and spectral filtering? Maybe the paper could include some discussion in the related work.\n\n**iv)**. I think $\\tau$ and $t$ mixed up in several steps in the proof of lemma A.2, e.g., between 746-747. \n\n**v)**. Please define $y_t(M)$ (line 216) formally. The current phrasing is ambiguous.\n\n**vi)**. Is $y^{nat}$ missing from $y_t(M^t)$, $y_t$ below line 882? In general the notation for the several versions of $y$ is quite confusing.\n\nLIMITATIONS:\nAs I describe in the weaknesses and questions the limitations should be further clarified as they are only partially addressed. (There are only technical limitations no societal issues.)", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 5, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-06-26"}