Title: Finite Sample Analyses for Continuous-time Linear Systems: System Identification and Online Control
Venue: NeurIPS 2025
Accepted: yes
Keywords: online control

system identification

continuous-time linear system
Abstract: Real world evolves in continuous time but computations are done from finite samples. Therefore, we study algorithms using finite observations in continuous-time linear dynamical systems. We first study the system identification problem, and propose a first non-asymptotic error analysis with finite observations. Our algorithm identifies system parameters without needing integrated observations over certain time intervals, making it more practical for real-world applications. Further we propose a lower bound result that shows our estimator is provably optimal up to constant factors. Moreover, we apply the above algorithm to online control regret analysis for continuous-time linear system. Our system identification method allows us \textcolor{blue}{to} explore more efficiently, enabling the swift detection of ineffective policies. We achieve a regret of $\mathcal{O}(\sqrt{T})$ over a single $T$-time horizon in a controllable system, requiring only $\mathcal{O}(T)$ observations of the system.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper studies continuous time linear dynamical systems and makes the following contributions. Firstly, for the system identification problem with time horizon $T$, and assuming that the system is stable, an algorithm is obtained that achieves an estimation error of  $O(1/\\sqrt{T})$ with only $O(T)$ samples. This method is also shown to be optimal up to constant factors. When a stabilizer is not known in advance, it is shown that $N$ independent short trajectories can be used  to obtain an estimate with error $O(1/\\sqrt{N})$. These results lead to an online continuous linear control algorithm which is shown to achieve $O(\\sqrt{T})$ regret.\n\nSTRENGTHS AND WEAKNESSES:\nPros\n---------\n\n1.  The paper is written well overall with a clearly defined problem, notation, and summary of existing works. The writing is clear for the most part, and makes it easy to follow the ideas. \n\n2.  The technique used for the system identification part (Alg. 1) seems quite natural, but it’s still non-trivial to carry out the full error analysis. The results are overall substantial, as upper and lower bounds are provided for system identification, along with an algorithm for online control. To my knowledge, these results appear to be novel as most recent works have addressed discrete-time linear systems. I think the results would be of interest for researchers working in this domain.\n\nCons\n---------\n\nIt would have been helpful to provide the full dependence on the problem parameters in the statements of the theorems/lemmas. For e.g., in Theorem 2, $T$ is required to be polynomial in the parameters, but it would be worth displaying the actual dependence (hiding only the absolute constants). This is the case in other statements as well.\n\nI also have some other questions which are separately outlined below.\n\nQUESTIONS:\n1. I do not see the dependence on the dimensions of $A,B$ appearing in the results, is this hidden inside the constant terms? If so, then I think it would be more clear if this could be explicitly shown. For discrete-time autonomous linear systems (with B = 0), we know that $T$ needs to be of the order $poly(n)$ where n is the dimension of $A$. Is it the same  for continuous time systems?\n\n2. The system identification results require A to be stable. For discrete time autonomous linear systems, recent results have shown that consistent identification is possible even when the spectral radius of A is only required to be less than or equal to 1. Is it possible to use those techniques here as well?\n\nLIMITATIONS:\nYes these have been discussed towards the end", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-30"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\n**Setting**. This paper studies the system identification and online control problems for continuous-time linear systems. Specifically, the system dynamics are given by: $d X_t = A X_t d t + B U_t d t + d W_t$, where $X_t$ and $U_t$ represent the state and action respectively, and $A, B$ are unknown system parameters. The control objective is to minimize the following continuous-time loss function:\n\n$$\nJ_T = \\mathbb{E}\\left[\\int_{t=0}^T \\left(X_t^{\\mathrm{T}} Q X_t + U_t^{\\mathrm{T}} R U_t\\right) d t\\right]\n$$\n\nwhere $Q, R$ are known. Since the system is unknown, the parameters $A, B$ must be estimated online from observed data, and actions must be selected based on these estimates to optimize the loss. The whole process is interactive and online, so the objective is rewritten as minimizing the regret $R_T = J_T - J_T^*$.\n\n**Methods**. The overall strategy consists of two parts. First, the continuous-time system is transformed into a discrete linear system:\n\n$x_{k+1} = A^{\\prime} x_k + B^{\\prime} u_k + w_k$.\n\nSecond, the remaining problem can be viewed as a bandit problem: on one hand, exploration is needed to estimate the parameters more accurately; on the other hand, we need to select optimal actions based on the estimated system.\n\nThus, an explore-then-commit strategy is adopted: during the initial phase, a known stable controller $K$ is used to sample data. Then, based on the collected data, the system is estimated, and the optimal actions are selected using the estimated system and the known loss function.\n\nSTRENGTHS AND WEAKNESSES:\n**Strength**\n\n- The theoretical development appears solid and the content is detailed.\n- It provides finite-time theoretical guarantees for continuous-time linear systems, while most prior work in control theory typically focuses on asymptotic convergence.\n\n**Weakness**\n\n- Assumption 3 seems a bit questionable: if the system dynamics are unknown, how can one find a stabilizing controller $K$? This implicitly requires Assumption 1 — namely, that the system matrix $A$ is already stable — otherwise finding such a controller would be very difficult. Therefore, this assumption feels rather strong.\n- Although Section 4.4 proposes a strategy to find a stabilizing controller, this procedure itself involves estimating $A, B$, which is also part of exploration. However, this is not clearly discussed or accounted for as part of the exploration strategy.\n- While the paper emphasizes its focus on continuous-time systems and providing theoretical guarantees in that setting, the core methodology still relies on first discretizing the system and then applying techniques from discrete-time analysis. The approach is not purely continuous in nature.\n\nQUESTIONS:\n- In the second figure of the Experiments section, the red line is flat by normalization, but was the regret also flat during the early exploratory phase? That seems a bit strange— the regret should increase rapidly at the beginning and then stabilize later, due to the exploration then commit.\n\nLIMITATIONS:\nAs mentioned in the weaknesses, I have concerns about the assumptions and setup of this work. The proposed method relies on a known stable controller for an unknown system, which seems contradictory. In an unknown environment, obtaining a stable controller typically requires some form of exploration or trial-and-error. This paper essentially ignores the impact of that exploration phase on the overall finite-sample analysis. A direct question would be: if we include the sampling used for exploration, is it still possible to achieve sub-linear finite-sample regret? Or does the analysis only hold in the asymptotic sense?", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-06-30"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper developed a system identification algorithm for continuous-time stochastic linear systems. The proposed method is based on time-discretization with a properly designed sampling interval. The estimation accuracy is guaranteed by an upper bound (Theorem 2). Moreover, based on the established result for the system identification, the authors devised an online control algorithm for continuous-time systems. Its performance is guaranteed by $O(\\sqrt{T})$ regret upper bound (Theorem 6). The effectiveness of the proposed methods is verified through numerical experiments.\n\nSTRENGTHS AND WEAKNESSES:\n### Strengths:\nThe proposed algorithm can attain an expected regret of $O(\\sqrt{T})$ with only $O(T)$ samples in time horizon $T$.\nEven when a stabilizing controller for an unstable, unknown system is not known in advance, the authors establish an upper bound of the estimation error by using multiple short-interval trajectories.\n\n\n### Weaknesses:\nI would like to point out that your paper adopts the pseudo regret in Theorem 6, while the existing studies [23,30,31] on continuous-time online control instead used the regret definition based on trajectory-wise comparisons, namely $$ \\int_0^T [c(x_t,K_t x_t) - c(x_t^{\\star}, K_{\\star} x_t^{\\star})] dt, $$\nwhere $x_t^{\\star}$ is the optimal trajectory and $c(x_t,u_t)$ denotes the cost function. The analysis of the regret is generally more challenging than that of the pseudo-regret as it requires accounting for trajectory deviations under the learned and optimal policies. Therefore, the fact that the proposed method achieves $O(\\sqrt{T})$ upper bound of the pseudo-regret, does not imply that it surpasses the performance guarantees of the existing studies [23,30,31]. The authors should clarify this distinction and carefully position their results in relation to those earlier studies.\n\nQUESTIONS:\n- In Theorem 4, the system parameters $A$ and $B$ are allowed to depend on the time horizon $T$. In fact, in the proof sketch, the authors consider stable matrices $A$ and $\\bar{A}$ satisfying $ |A - \\bar{A}| = 2c_1 / \\sqrt{T} $. However, the true system parameters are independent of $T$. Therefore, deriving a lower bound based on $A$ depending on $T$ appears to lack meaningful interpretation. If there is a valid reason why such a $T$-dependent construction of A is relevant or meaningful in the context of this work, it should be explained clearly and convincingly in the manuscript. Otherwise, a lower bound should be derived under the assumption that $A$ is independent of $T$.\n\n- In Section 4.4, the authors consider the situation where a stabilizer is not known in advance. Then, they propose to first find a stable controller. However, Theorem 5 only gives the estimation error of system parameters and does not establish a result about stability. Therefore, it is unclear how Theorem 5 is useful. I suggest using Theorem 5 for finding a stabilizer for online control of unstable systems and establishing $O(\\sqrt{T})$ regret bound without Assumption 3-1, which assumes knowledge of a stabilizer $K$. Then, the usefulness of Theorem 5 is clarified.\n\nThe usefulness of two theorems (Theorems 4, 5) is unclear in the current version.\nIf this issue is properly addressed, I would gladly raise my review score.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper develops non-asymptotic system-identification algorithms for continuous-time linear dynamical systems using only a finite number of noisy state observations. By discretizing at carefully chosen intervals and directly estimating the matrix exponential, the authors recover the continuous-time parameters without incurring discretization bias, achieving an estimation error of $O(1/\\sqrt{T})$. They also prove a matching lower bound showing this rate is optimal up to constants. They also apply the above algorithm to online control regret analysis for continuous-time linear system.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\n\nThe paper provides the provable error bound for continuous-time system identification, which also matches information-theoretic lower bound. It applies the identification scheme to online LQR, achieving $O(\\sqrt{T})$ regret. \n\nWeaknesses\n\nExperiments are restricted to small, simulated 3-dimensional systems; the performance and numerical stability of the method in higher dimensions or on real-world data remain untested. It would be helpful to comment/provide further simulations in higher dimensional setups. \n\nExtension to adversarial or non-Gaussian noise: As mentioned in the paper, the analysis assumes stochastic (Brownian) disturbances; it is unclear how the method performs under adversarial or heavy-tailed noise models.\n\nQUESTIONS:\nMy main concern is the linear model specification. As the authors note in Section 6, many practical models are nonlinear. To what extent can the results in this paper be extended to nonlinear settings? It would also be useful to run robustness checks to assess the proposed method’s performance under nonlinear models.\n\nThe simulation study is limited. Adding simulations for higher-dimensional systems would help evaluate the stability of the proposed method.\n\nHow should one choose the sampling interval $h$? In theory, it can be set according to Assumption 1, but $\\kappa_A$ is unknown in practice. Is there a data-driven procedure for selecting $h$? \n\nMinor typo\n\nPage 6, line 208: “we proof Theorem …”.\n\nLIMITATIONS:\nSee the weaknesses and questions for details.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 2, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-02"}