Title: Identification of Analytic Nonlinear Dynamical Systems with Non-asymptotic Guarantees
Venue: NeurIPS 2024
Accepted: yes
Keywords: set-membership

least-squares

nonlinear systems

non-asymptotic guarantees
Abstract: This paper focuses on the system identification of an important class of nonlinear systems: nonlinear systems that are linearly parameterized, which enjoy wide applications in robotics and other mechanical systems. We consider two system identification methods: least-squares estimation (LSE), which is a point estimation method; and set-membership estimation (SME), which estimates an uncertainty set that contains the true parameters. We provide non-asymptotic convergence rates for LSE and SME under  i.i.d. control inputs and control policies with i.i.d. random perturbations, both of which are considered as non-active-exploration inputs. Compared with the counter-example based on piecewise-affine systems in the literature, the success of non-active exploration in our setting relies on a key assumption about the system dynamics: we require the system functions to be real-analytic. Our results, together with the piecewise-affine counter-example, reveal the importance of differentiability in nonlinear system identification through non-active exploration. Lastly, we numerically compare our theoretical bounds with the empirical performance of LSE and SME on a pendulum example and a quadrotor example.

Identification of Analytic Nonlinear Dynamical Systems with Non-asymptotic Guarantees

Abstract

This paper focuses on the system identification of an important class of nonlinear systems: nonlinear systems that are linearly parameterized, which enjoy wide applications in robotics and other mechanical systems. We consider two system identification methods: least-squares estimation (LSE), which is a point estimation method; and set-membership estimation (SME), which estimates an uncertainty set that contains the true parameters. We provide non-asymptotic convergence rates for LSE and SME under i.i.d. control inputs and control policies with i.i.d. random perturbations, both of which are considered as non-active-exploration inputs. Compared with the counter-example based on piecewise-affine systems in the literature, the success of non-active exploration in our setting relies on a key assumption about the system dynamics: we require the system functions to be real-analytic. Our results, together with the piecewise-affine counter-example, reveal the importance of differentiability in nonlinear system identification through non-active exploration. Lastly, we numerically compare our theoretical bounds with the empirical performance of LSE and SME on a pendulum example and a quadrotor example.

1 Introduction

Learning control-dynamical systems with statistical methodology has received significant attention in the past decade (Sarker et al., 2023; Li et al., 2023b; Chen and Hazan, 2021; Simchowitz and Foster, 2020; Wagenmaker and Jamieson, 2020; Simchowitz et al., 2018; Dean et al., 2018; Abbasi-Yadkori and Szepesvári, 2011; Li et al., 2021b). In particular, the estimation of linear dynamical systems, e.g. x t +1 = A ∗ x t + B ∗ u t + w t , is relatively well-studied: it has been shown that non-active exploration by i.i.d. noises on control inputs u t and system disturbances w t are already enough for accurate system identification, and least square estimation (LSE) can achieve the optimal estimation convergence rate (Simchowitz and Foster, 2020; Simchowitz et al., 2018).

However, nonlinear control systems are ubiquitous in real-world applications, e.g. robotics (Siciliano et al., 2010; Alaimo et al., 2013), power systems (Simpson-Porco et al., 2016), transportation (Kong et al., 2015), etc. Motivated by this, there has been a lot of attention on learning nonlinear systems recently. One natural and popular direction to study nonlinear system identification is on learning linearly parameterized nonlinear systems as defined below, which is a straightforward extension from

the standard linear systems (Mania et al., 2022; Khosravi, 2023; Foster et al., 2020)

where θ ∗ is a vector of unknown parameters and ϕ ( x t , u t ) is a known vector of nonlinear features.

On the one hand, some classes of these systems are shown to enjoy similar benefits of linear systems. For example, bilinear systems can also be estimated by LSE under non-active exploration with i.i.d. noises (Sattar et al., 2022), as well as linear systems with randomly perturbed nonlinear policies (Li et al., 2023b).

On the other hand, it is also known that non-active exploration is insufficient for general linearly parameterized nonlinear systems. In particular, (Mania et al., 2022) provides a counter example showing that non-active exploration is insufficient to learn accurate models under piece-wise affine feature functions. This motivates a sequence of follow-up work on the design of active exploration for nonlinear system estimation, which is largely motivated by the non-smooth feature functions such as ReLu in neural networks (Mania et al., 2022; Kowshik et al., 2021; Khosravi, 2023).

However, there is a big gap between bilinear systems, which is infinitely differentiable, and the counter example by non-smooth systems. A natural question is: to what extent can non-active exploration still work for linearly parameterized nonlinear systems?

Contributions. One major contribution of this paper is showing that LSE with non-active i.i.d. noises can efficiently learn any linearly parameterized nonlinear systems with real-analytic feature functions and provide a non-asymptotic convergence rate. Notice that real-analytic feature functions are common in physical systems. For example, polynomial systems satisfy this requirement and have wide applications in power systems (Simpson-Porco et al., 2016), fluid dynamics (Noack et al., 2003), etc. Further, trigonometric functions also satisfy the real-analytic property so a large range of robotics and mechanical systems also satisfy this requirement (Siciliano et al., 2010; Alaimo et al., 2013).

A side product of our LSE convergence rate analysis is the convergence rate for another commonly used uncertainty quantification method in control: set membership estimation (SME).

Numerically, we test our theoretical results in pendulum and quadrotor systems. Simulations show that LSE and SME can indeed efficiently explore the system and converge to the true parameter under non-active exploration noises.

Technically, the key step in our proof is establishing the block-martingale-small-ball condition (BMSB) for general analytic feature functions, which greatly generalizes the bilinear feature function in Sattar et al. (2022). Our result is built on an intuition inspired by the counter example in (Mania et al., 2022): the counter example in (Mania et al., 2022) requires that some feature function is zero in a certain region, so nothing can be learned about its parameter if the states stay in this region. However, analytic functions cannot be a constant zero in a positive-measure region unless it is a constant zero everywhere. Therefore, the counter example does not work, and non-active exploration around any states can provide some useful information. Our proof formalizes this intuition by utilizing the Paley-Zygmund Petrov inequality (Petrov, 2007).

Related work. Inspired by neural network parameterization, nonlinear systems of the form x t +1 = ϕ ( A ∗ x t ) + w t is also studied in the literature, where ϕ ( · ) is a known nonlinear link function and A ∗ is unknown. The least square cost is no longer quadratic or even convex in this case and various optimization methods have been proposed to learn this type of systems (Kowshik et al., 2021; Sattar et al., 2022; Foster et al., 2020).

Another related line of research focuses on nonlinear regression with dependent data (Ziemann and Tu, 2022; Ziemann et al., 2023, 2024), 1 which can be applied to nonlinear system identification. The nonlinear regression in (Ziemann and Tu, 2022; Ziemann et al., 2023, 2024) is based on nonparametric LSE and its variants, and their convergence rates under different scenarios have been analyzed. It is interesting to note that this line of work usually assumes certain persistent excitation assumptions, 2 whereas our paper demonstrates that persistent excitation holds by establishing the BMSB condition for linearly parameterized and real-analytic nonlinear control systems.

Uncertainty set estimation is crucial for robust control under model uncertainties Lu and Cannon (2023); Lorenzen et al. (2019); Li et al. (2021a). SME is a widely adopted uncertainty set estimation method in robust adaptive control (Lorenzen et al., 2019; Lu and Cannon, 2023; Bertsekas, 1971; Bai et al., 1995). Recently, there is an emerging interest in analyzing SME's convergence and convergence rate for dynamical systems (Li et al., 2024; Lu et al., 2019; Xu and Li, 2024), because previous analysis focus more on the linear regression problem (e.g. (Akçay, 2004; Bai et al., 1998)). There are also recent applications of SME to online control Yu et al. (2023), power systems Yeh et al. (2024), and computer vision Gao et al. (2024); Tang et al. (2024).

Notation. The set of non-negative real numbers is denoted by R ≥ 0 . The notation ⌈·⌉ stands for the ceiling function. For a real vector z ∈ R n , ∥ z ∥ 2 represents its ℓ 2 norm, ∥ z ∥ ∞ represents its ℓ ∞ norm, and z i represents its i -th component with i = 1 · · · n . The set of real symmetric matrices is denoted by S n . For a real matrix Z , Z ⊺ represents its transpose, ∥ Z ∥ 2 its maximum singular value, ∥ Z ∥ F its Frobenius norm, σ min ( Z ) its minimum singular value, vec ( Z ) its vectorization obtained by stacking its columns, and for a real square matrix Z , tr ( Z ) represents its trace. For a real symmetric matrix Z , Z ≻ 0 and Z ⪰ 0 indicate that Z is positive definite and positive semi-definite, respectively. For a measurable set E ⊂ R n , λ n ( E ) represents its Lebesgue measure in R n and E c represents its complement in R n . The notation ∅ stands for the empty set. For a set T of matrices θ ∈ R n × m , diam ( T ) denotes its diameter and it is defined as diam ( T ) = sup θ,θ ′ ∈T ∥ θ -θ ′ ∥ F . For z i ∈ R with i = 1 , · · · , ℓ , the notation diag ( z 1 , · · · , z ℓ ) denotes a matrix in R ℓ × ℓ with diagonal entries of z i . This paper uses truncated-Gaussian (0 , σ w , [ -w max , w max ]) to refer to the truncated Gaussian distribution generated by Gaussian distribution with zero mean and σ 2 w variance with truncated range [ -w max , w max ] . The same applies to multi-variate truncated Gaussian distributions.

2 Problem Formulation and Preliminaries

This paper studies the system identification/estimation of linearly parameterized nonlinear systems:

where x t ∈ R n x , u t ∈ R n u , and w t ∈ R n x denote the state, control input, and system disturbance respectively; θ ∗ ∈ R n x × n ϕ denotes the unknown parameters to be estimated, and ϕ ( · ) denotes a vector of known nonlinear feature/basis functions, i.e., ϕ ( · ) = ( ϕ 1 ( · ) , · · · , ϕ n ϕ ( · )) ⊺ , where ϕ i ( · ) : R n x + n u → R . Without loss of generality, we consider zero initial condition, i.e. x 0 = 0 , and linearly independent feature functions, that is, ∑ n ϕ i =1 c i ϕ i ( x t , u t ) = 0 implies that c i = 0 for all i . 3

The linearly parameterized nonlinear system (1) is a natural generalization of linear control systems x t +1 = A ∗ x t + B ∗ u t + w t and has wide applications in, for example, robotics (Siciliano et al., 2010; Alaimo et al., 2013), power systems (Simpson-Porco et al., 2016), transportation (Kong et al., 2015), etc. Therefore, there has been a lot of research on learning this type of system (1) utilizing the methodology and insights from linear system estimation. For example, it is common to estimate a linearly parameterized nonlinear system by least squares estimation (LSE), which enjoys desirable performance in linear systems.

In particular, LSE for (1) is reviewed below

For linear systems, LSE enjoys the following good property: LSE can achieve the optimal rate of convergence with i.i.d. noises w t and i.i.d. control inputs u t under proper conditions ( (Simchowitz et al., 2018)). This good property has been generalized to some linearly parameterized nonlinear systems, such as bilinear systems, and linear systems with nonlinear control policies. Unfortunately, general linearly parameterized nonlinear systems do not enjoy this good property of linear systems, meaning i.i.d. random inputs may not provide enough exploration for non-smooth feature functions ϕ ( · ) . Therefore, a series of follow-up work focuses on the design of active exploration methods.

However, due to the simplicity of implementation, i.i.d. random inputs remain a popular method in empirical research of system identification and sometimes enjoy satisfactory performance, despite

the lack of theoretical guarantees. Therefore, this paper aims to establish more general conditions that allow provable convergence of nonlinear system estimation under i.i.d. random inputs.

In the rest of this paper, we will show that with certain smoothness and continuous conditions, i.i.d. random inputs are sufficient for estimation of (1), which recovers the good property of linear systems.

2.1 Assumptions

In the following, we formally describe the smoothness and continuity conditions that enables efficient exploration of (1) by i.i.d. random inputs.

Assumption 1 (Analytic feature functions) . All components of the feature vector ϕ ( · ) are real analytic functions in R n x + n u , 4 i.e., for every 1 ≤ i ≤ n ϕ , ϕ i ( x, u ) is an infinitely differentiable function such that the Taylor expansion in every (¯ x, ¯ u ) converges point-wise to ϕ i ( x, u ) in a neighborhood of (¯ x, ¯ u ) .

Analytic functions include polynomial functions and trigonometric functions, which are important components of many physical systems in real-world applications, e.g. power systems, robotics, transportation systems, etc. In particular, we provide two illustrative examples below.

Example 1 (Pendulum) . Many multilink robotic manipulators can be understood as interconnected pendulum dynamics. The motion equations of a single pendulum, consisting of a mass m suspended from a weightless rod of length l fixed at a pivot without friction, can be expressed as:

where α represents the angle of the rod relative to the vertical axis, g is the gravity constant, u is the torque input, and w is the disturbance applied to this system. After discretization the system dynamics can be rewritten in the structure of (1) with the feature vector consisting of expressions involving sin( α ) and u , all of which are analytic functions. The matrix of unknown parameters contains terms of the pendulum's mass and the rod's length.

Example 2 (Quadrotor (Alaimo et al., 2013)) . Let p ∈ R 3 and v ∈ R 3 represent the center of mass position and velocity of the quadrotor in the inertial frame, respectively; let ω ∈ R 3 denote its angular velocity in the body-fixed frame and q ∈ R 4 denote the quaternion vector. The quadrotor's equations of motion can then be expressed as:

where g is the gravity constant, m is its total mass, I = diag ( I xx , I yy , I zz ) its inertia matrix with respect to the body-fixed frame, f u ∈ R the total thrust, τ u ∈ R 3 the total moment in the body-fixed

Similar to the pendulum example, after discretization the system dynamics can be rewritten in the structure of (1) with the feature vector consisting of cubic polynomials in states and inputs, which are real-analytic. The unknown parameters contain terms of the mass and inertial moments of the quadrotor.

Next, we introduce the assumption on the random inputs, which relies on the following definition.

Definition 1 (Semi-continuous distribution) . We define a probability distribution P as semi-continuous if there does not exist a set E with Lebesgue measure zero such that P ( E ) = 1 .

The semi-continuous distribution is a weaker requirement than continuous distributions. In particular, any continuous distributions, or a mixture distribution with one component as a continuous distribution, can satisfy the requirement of semi-continuity. The semi-continuity can also be interpreted by the Lebesgue Decomposition Theorem (Chapter 6 of (Halmos, 2013)) as discussed below.

Remark 1 (Connection with Lebesgue Decomposition Theorem) . Definition 1 can be interpreted by the Lebesgue Decomposition Theorem, which suggests that any probability distribution can be decomposed into a purely atomic component and a non-atomic component (see more details in Halmos (2013)). A semi-continuous distribution as defined in Definition 1 requires the distribution's non-atomic component to be nonzero.

In the following, we provide the assumptions on w t and u t using the semi-continuity definition.

Assumption 2 (Bounded i.i.d. and semi-continuous disturbance) . w t is i.i.d. following a semicontinuous distribution with zero mean and a positive definite covariance matrix Σ w ⪰ σ 2 w I n x ≻ 0 and a bounded support, i.e. ∥ w t ∥ ∞ ≤ w max almost surely for all t .

The i.i.d. assumption is common in the literature of system identification for linear and nonlinear systems. As for the bounded assumption on w t , although it is stronger than the sub-Gaussian assumption on w t in the literature of linear system estimation, it is a common assumption in the literature of nonlinear system estimation (Mania et al., 2022; Shi et al., 2021; Kim and Lavaei, 2024). Further, in many physical applications, noises are usually bounded, e.g. the wind disturbances in quadrotor systems are bounded, the renewable energy injections in power systems are also bounded, etc.

The semi-continuity assumption may seem restrictive since it rules out the discrete distributions. However, the disturbances in many realistic systems can satisfy the semi-continuity because realistic noises are usually generated from a mixture distribution where at least one component is continuous, e.g. the wind disturbances and renewable generations are continuous.

For the control inputs u t , we first impose the same assumption as Assumption 2 for simplicity. Later in Section 3, we will also discuss the relaxation of this assumption to include control policies.

Assumption 3 (Bounded i.i.d. and semi-continuous inputs) . u t is i.i.d. following a semi-continuous distribution with zero mean and positive definite covariance Σ u ⪰ σ 2 u I n x ≻ 0 and bounded support, i.e. ∥ u t ∥ ∞ ≤ u max almost surely for all t .

Lastly, we introduce our stability assumption based on the input-to-state stability definition below.

Definition 2 (Locally input-to-state stability (LISS)) . Consider the general nonlinear system x t +1 = f ( x t , d t ) with x t ∈ R n x , d t ∈ R n d , f being a continuous function such that f (0 , 0) = 0 . This system is called locally input-to-state stable (LISS) if there exist constants ρ x > 0 , ρ > 0 and functions γ ∈ K , β ∈ KL such that for all x 0 ∈ { x 0 ∈ R n x : ∥ x 0 ∥ 2 ≤ ρ x } and any input d t ∈ { d ∈ R n d : sup t ∥ d t ∥ ∞ ≤ ρ } , it holds that ∥ x t ∥ 2 ≤ β ( ∥ x 0 ∥ 2 , t ) + γ ( sup t ∥ d t ∥ ∞ ) for all t ≥ 0 . 5

Assumption 4 (LISS system) . System (1) is LISS with parameters ρ x and ρ such that ρ x ≥ ∥ x 0 ∥ 2 and ρ ≥ max( w max , u max ) , respectively.

Assumption 4 is imposed, together with the bounded disturbances and inputs in Assumptions 2 and 3, to guarantee bounded states during the control dynamics (for instance, see the proof of Theorem 1in Appendix A). In particular, many studies on learning-based nonlinear control require a certain boundedness on the states for theoretical analysis Sattar and Oymak (2022); Foster et al. (2020); Li et al. (2023a).

In addition, it is interesting to note that this paper only requires local stability of the dynamics, whereas several learning-based nonlinear control papers assume certain global properties, such as global exponential stability in (Foster et al., 2020), global exponential incremental stability in (Sattar and Oymak, 2022; Li et al., 2023a; Lin et al., 2024), or global Lipschitz smoothness in (Lee et al., 2024). 6 This difference in the dynamics assumption reflects a trade-off with the disturbance assumptions:

we assume a stronger assumption on the boundedness of disturbances and a weaker assumption on local stability, whereas much of the literature considers (sub)Gaussian distributions (which can be unbounded) but requires stronger global properties for dynamics. Since this paper is largely motivated by physical systems, which typically encounter bounded disturbances/inputs and generally only satisfy local stability (Slotine and Li, 1991), we address this trade-off through our current set of assumptions, leaving it as an exciting future direction to consider relaxing these assumptions.

3 Main Results

In this section, we provide the estimation error bounds of LSE for linearly parameterized nonlinear systems under i.i.d. random inputs. The estimation error bounds rely on the establishment of probabilistic persistent excitation, which will be introduced in the first subsection. Later, we also generalize the results to include control policies and discuss the convergence rate of another popular uncertainty quantification method in the control literature, set membership estimation, whose formal definition is deferred to the corresponding subsection.

3.1 Probabilistic Persistent Excitation

It is well-known that persistent excitation (PE) is a crucial condition for successful system identification (Narendra and Annaswamy, 1987). In the following, we introduce the persistent excitation condition for our linearly parameterized nonlinear systems.

Definition 3 (Persistent excitation (Skantze et al., 2000; Sastry and Bodson, 2011)) . System (1) is persistently excited if there exist s > 0 and m ≥ 1 such that for any t 0 ≥ 0 , we have

In the stochastic setting, PE is closely related with a block-martingale small-ball (BMSB) condition proposed in Simchowitz et al. (2018), which can be viewed as a probabilistic version of PE.

Definition 4 (BMSB (Simchowitz et al., 2018)) . Let {F t } t ≥ 1 denote a filtration and let { y t } t ≥ 1 be an {F t } t ≥ 1 -adapted random process taking values in R n y . We say { y t } t ≥ 1 satisfies the ( k, Γ sb , p ) -block martingale small-ball (BMSB) condition for a positive integer k , a Γ sb ≻ 0 , and a p ∈ [0 , 1] , if for any fixed v ∈ R n y such that ∥ v ∥ 2 = 1 , the process { y t } t ≥ 1 satisfies 1 k ∑ k i =1 P ( | v ⊺ y t + i | ≥ √ v ⊺ Γ sb v | F t ) ≥ p almost surely for any t ≥ 1 .

One major contribution of this paper is formally establishing the BMSB condition for linearly parameterized nonlinear systems with real-analytic feature functions.

In the following, we first investigate the open-loop system with i.i.d. inputs and later extend the results to the closed-loop systems with inputs u t = π ( x t ) + η t , where η t represents the noise and π : R n x → R n u denotes a control policy. The following theorem considers the open-loop systems.

Theorem 1 (BMSB for open-loop systems) . Let u t = η t and consider the filtration F t = F ( w 0 , · · · , w t -1 , x 0 , · · · , x t , η 0 , · · · , η t ) . Suppose Assumptions 1, 2, 3, 4 hold, then there exist s ϕ > 0 and p ϕ ∈ (0 , 1) such that the {F t } t ≥ 1 -adapted process { ϕ ( x t , u t )} t ≥ 1 satisfies the ( 1 , s 2 ϕ I n ϕ , p ϕ ) -BMSB condition.

Proof Sketch. Intuitively, BMSB requires that any linear combination of feature functions remains positive with a non-vanishing probability. Notice that a linear combination of real-analytic functions is itself real-analytic, and the zeros of an analytic function have measure zero. These facts allow us to show that the probability of a linear combination of linearly independent feature functions equaling zero is less than one, as long as the noises follow semi-continuous distributions, by the connection of the Lebesgue measure and the probability measure in Definition 1.

In more detail, the proof leverages a variant of the Paley-Zygmund argument (Petrov, 2007), which provides a lower bound for the tail properties of positive random variables. Specifically, it states that the probability of a positive random variable being small depends on the ratio of its even moments.

We apply this result to the random variable | v T ϕ ( x t +1 , u t +1 ) | F t | with ∥ v ∥ 2 = 1 and aim to show that the lower bound is non-trivial for any direction v with ∥ v ∥ 2 = 1 and any filtration F t , t ≥ 0 . We then use results from measure theory to demonstrate the existence of such a non-trivial lower bound. This is done by showing that the Lebesgue measure of the set where | v T ϕ ( x t +1 , u t +1 ) | = 0 is zero, and thus the even moments of | v T ϕ ( x t +1 , u t +1 ) | F t | are non-zero, provided that the noise and disturbance distributions are semi-continuous. For further details, please refer to Appendix A.

It is worth pointing out that Theorem 1 only establishes the existence of the constants ( s ϕ , p ϕ ) , and deriving explicit formulas of these constants are left for future work. In particular, it can be challenging to derive a generic formula for all linearly parameterized nonlinear systems, but an exciting direction is to study reasonable sub-classes of systems and construct their corresponding formulas of the constants ( s ϕ , p ϕ ) .

3.2 Non-asymptotic Bounds for LSE

We are now prepared to present the non-asymptotic convergence rate for the LSE in learning the unknown parameters of the system (1).

Theorem 2 (LSE's convergence rate for open-loop systems) . Consider the dynamical system described in (1) with i.i.d. inputs u t = η t and assume that Assumptions 1, 2, 3, 4 are satisfied. Let s ϕ and p ϕ be as defined in Theorem 1, and define ¯ b ϕ = sup t ≥ 0 E [ ∥ ϕ ( z t ) ∥ 2 2 ] . For a fixed δ ∈ (0 , 1) and T ≥ 1 , if T satisfies the condition

then LSE's estimation ˆ θ T satisfies the following error bound with probability at least 1 -3 δ .

The proof relies on Theorem 1 and Theorem 2.4 in (Simchowitz et al., 2018). The complete proof is provided in Appendix B.1.

Theorem 2 demonstrates that LSE converges to the true parameters under random control inputs and random disturbances (non-active exploration) at a rate of 1 √ T for linearly parameterized nonlinear systems. This is consistent with the convergence rates of LSE for linear systems in terms of T .

Regarding the dependence of the convergence rate on dimension, the explicit dependence is √ n x + n ϕ , where n x and n ϕ refer to the dimensions of the state and the characteristic vector, respectively. In addition, it is worth mentioning that other parameters, such as s ϕ , p ϕ , ¯ b ϕ , may also implicitly depend on the dimensions. For some special systems, such as bilinear systems, it has been shown that these constants are independent of dimensions (Sattar et al., 2022). It is left as future work to explore other nonlinear systems' implicit dimension dependence.

Next, we can generalize the i.i.d. inputs u t to include control policies, i.e., u t = π ( x t ) + η t , where η t satisfies Assumption 3 and π ( x t ) is analytic.

Corollary 1 (LSE's convergence rate for closed-loop systems) . Consider inputs u t = π ( x t ) + η t , where π ( · ) is real-analytic, η t satisfies Assumption 3, and the closed-loop system x t +1 = θ ∗ ϕ ( x t , π ( x t ) + η t ) + w t satisfies Assumption 4 for both w t and η t . Then, the same convergence rate as in Theorem 2 holds.

The proof is provided in Appendix B.2.

3.3 Non-asymptotic Diameter Bounds for SME

Set membership estimation (SME) is another popular method for uncertainty quantification in control system estimation (Bertsekas, 1971; Fogel and Huang, 1982; Lu and Cannon, 2023; Li et al., 2024). Unlike LSE, SME is a set-estimator and directly estimates the uncertainty set. Since the analysis of SME also relies on the probabilistic persistent excitation analysis, we can also establish the

convergence rate of SME for linearly parameterized nonlinear systems under i.i.d. noises in the following. In particular, SME estimates the uncertainty set as

where W is a bounded set such that w t ∈ W for all t ≥ 0 .

The convergence of SME relies on an additional assumption as shown in the following: the tightness of the bound W on the support of w t '. This tightness assumption is commonly considered in SME's literature (Li et al., 2024; Lu et al., 2019; Akçay, 2004). In addition, (Li et al., 2024) discusses the relaxation of this assumption by learning a tight bound at the same time as learning the uncertainty set of θ ∗ for linear systems. Similar tricks can be applied to nonlinear systems, but this paper only considers the vanilla case of SME for simplicity.

Assumption 5 (Tight bound on disturbances) . Assume for any ϵ > 0 , there exists q w ( ϵ ) > 0 , such that for any 1 ≤ j ≤ n and t ≥ 0 , we have P ( w j t + w max ≤ ϵ ) ≥ q w ( ϵ ) > 0 , P ( w max -w j t ≤ ϵ ) ≥ q w ( ϵ ) > 0 .

Assumption 5 requires that w t can visit the boundary of the set W ' arbitrarily closely with a positive probability. For example, for a one-dimensional w t bounded by -w max ≤ w t ≤ w max , the assumption 5 requires that there is a positive probability that w t is close to w max and -w max , that is, for any ϵ > 0 , we have P ( w max -ϵ ≤ w t ≤ w max ) > 0 and P ( -w max ≤ w t ≤ -w max + ϵ ) > 0 .

Next, we state a non-asymptotic bound on the diameter of the uncertainty set estimated by SME.

Theorem 3 (SME's diameter bound for open-loop systems) . Consider the system (1) with i.i.d. inputs u t = η t . Suppose Assumptions 1, 2, 3, 4 are satisfied. Consider s ϕ and p ϕ defined in Theorem 1 and let b ϕ = sup t ≥ 0 ∥ ϕ ( z t ) ∥ 2 . For any m ≥ 0 and δ ∈ (0 , 1) , when T > m , we have

where a 1 = s ϕ p ϕ 4 , a 2 = 64 b 2 ϕ s 2 ϕ p 2 ϕ , a 3 = p 2 ϕ 8 , a 4 = 16 b ϕ √ n x s ϕ p ϕ . The constants hidden in ˜ O are provided in the Appendix C.1.

The proof of Theorem 3 is based on Theorem 1 in this paper and Theorem 1 from (Li et al., 2024). The detailed proof is provided in Appendix C.1.

Theorem 3 establishes an upper bound on the "failure" probability of SME, that is, the probability that the diameter of the uncertainty set exceeds δ . To ensure that the failure probability is less than 1 , one can select m = O (log( T )) and choose a sufficiently large T such that T ≥ m = O (log( T )) . If w t is more likely to visit the boundaries of the set W (meaning larger q ( ℓ ) ), the SME is less likely to estimate an uncertainty set with a diameter greater than δ .

To provide more intuitions on the diameter bound in Theorem 3, we consider q w ( ℓ ) = c w ℓ for some c w > 0 . Note that several common distributions, including the uniform distribution and the truncated Gaussian distribution, satisfy this property on q w ( ℓ ) (see Appendix C.2 for explicit formulas of c w ). With q w ( ℓ ) = c w ℓ , we can provide a convergence rate of the SME in terms of T in the following.

Corollary 2 (SME's convergence rate when q w ( ℓ ) = c w ℓ ) . For any ϵ > 0 , let

If w t 's distribution satisfies q w ( ℓ ) = c w ℓ for all ℓ > 0 , then when T > m , we have:

with probability at least 1 -2 ϵ . Here the constants hidden in O ( · ) are provided in Appendix C.

The proof of Corollary 2 is provided in Appendix C.2.

Finally, similar to LSE, we can extend SME's convergence rates from open-loop systems to closedloop systems with real-analytic control policies, i.e., u t = π ( x t ) + η t , where η t satisfies Assumption 3 and π ( x t ) is real-analytic.

Corollary 3 (SME's convergence rate for closed-loop systems) . Consider inputs u t = π ( x t ) + η t , where π ( · ) is real-analytic, η t satisfies Assumption 3, and the closed-loop system x t +1 = θ ∗ ϕ ( x t , π ( x t ) + η t ) + w t satisfies Assumption 4 in terms of both w t and η t . Then, the same convergence rates as in Theorem 3 and Corollary 2 still hold.

The proof of Corollary 3 is provided in Appendix C.3.

4 Numerical Experiments

In this section, we evaluate the performance of LSE in estimating the unknown parameter θ ∗ and SME in estimating the uncertainty set for the unknown parameters using the pendulum and quadrotor examples outlined in Section 2. We compare the empirical convergence rates of LSE and SME with the theoretical rates in Theorem 2 and Corollary 2. In each case, the input u t is composed of a control policy and i.i.d. noise, such that u t = π ( x t ) + η t . For our experiments, we employ noise and disturbances drawn from uniform and truncated-Gaussian distributions. To compute theoretical rates, we numerically estimate parameters such as s ϕ and p ϕ (see Appendix E). Further details can be found in our source code. 7

The details for these scenarios are outlined below:

Pendulum 1: In the pendulum example described in Section 2, the control input is u t = -k ˙ α t + η t . This scenario includes two unknown parameters: θ 1 = 1 l and θ 2 = 1 ml 2 .

Quadrotor 2: For the quadrotor example in Section 2, the control input is defined as u t = π ( x t ) + η t , where π ( x t ) follows the controller proposed by Alaimo et al. (2013). The quadrotor system involves 13 states and 4 inputs, with the unknown parameter matrix θ ∗ containing 7 parameters, including the mass m and specific elements of the inertia matrix I .

Further details on controller gains and unknown parameters are provided in Appendix D.

LSE Results: Figures 1a and 1b present a comparison between the LSE theoretical bound from Theorem 2 with its empirical estimation error of the unknown parameters θ ∗ versus trajectory length T for the pendulum example, with uniform and truncated-Gaussian noises and disturbances. Similarly, Figures 1c and 1d show this comparison for the quadrotor example. In each figure, both the theoretical bound and empirical error are normalized by the l 2 norm of the nominal parameter θ ∗ . The log-log plots for both scenarios demonstrate that the empirical error rate achieves O ( 1 √ T ) which in consistent with the theoretical rate in Theorem 2.

SME Results: Figures 2a and 2b show the empirical convergence rate of SME for the pendulum example, for uniform and truncated-Gaussian noises and disturbances, in comparison to the theoretical rate from Corollary 2. Both the theoretical bound and empirical error in the figures are normalized by the l 2 norm of the nominal parameter θ ∗ . The log-log plots indicate that the empirical rate achieves O ( 1 T ) , which is consistent with the results from Corollary 2 and with the related results for linear systems in (Li et al., 2024). A similar result can be observed for the quadrotor example, in Figures 2c and 2d. Additionally, Figure 3 shows the uncertainty sets estimated by SME for the two unknown parameters, labeled θ 1 and θ 2 , in the pendulum example, along with the diameters of these sets as trajectory length grows. We observe that these sets contract as trajectory length increases, with the true values of the unknown parameters lying within the estimated uncertainty sets. The illustration of uncertainty sets for the quadrotor example is provided in Appendix D.2.

(a) Uniform

(b) Truncated-Gaussian

(c) Uniform

(d) Truncated-Gaussian

pendulum

pendulum

quadrotor

quadrotor

2

10-1

10-2

2

10-2

2

10-3

2

10-3

2

￥

10-3

10-4

empr

2

10-3

10-4

empr

10-4

empr

*0

10-4

empr

10-51

10-51

103

104

103

104

104

104

T

T

T

T

(a) Uniform

(b) Truncated-Gaussian

(c) Uniform

(d) Truncated-Gaussian

normalized diamater

pendulum

normalizeddiamater

pendulum

normalizeddiamater

10-2

quadrotor

normalizeddiamater

quadrotor

10-1

10-2

empr

10-2

10-3

10-4

empr

10-3

empr

10-4

empr

10-4

103

104

103

104

104

103

104

T

T

T

T

5 Concluding Remarks

Conclusion. This study examines the probabilistic persistent excitation in a class of nonlinear systems influenced by i.i.d. noise and stochastic disturbances, with the stipulation that their distributions do not concentrate on sets of Lebesgue measure zero. Based on this we then present an explicit bound on the convergence rate of SME estimations and LSE estimations for this class of dynamical systems. Additionally, numerical experiments in the context of robotics are provided to illustrate both methods.

Limitations. One limitation of this work is that our analysis relies on a specific class of i.i.d. noises and stochastic disturbances, where the probability distribution is not concentrated on sets of Lebesgue measure zero. While this is a sufficient condition, it is possible that the BMSB conditions are satisfied under other circumstances. Another limitation is that, though we provide sufficient conditions for the existence of parameters satisfying the BMSB condition, the explicit dependence is not detailed here. Lastly, imperfect observations are not considered here.

Future Work. Our future work includes several promising directions, e.g., to explore cases that do not satisfy our semi-continuity assumption, such as discrete noises, and to investigate the explicit dependence of the BMSB parameter on system attributes, such as state, input, and feature dimensions, etc. Furthermore, extending this work to imperfect state observations is an important next step. Finally, a potential direction is to provide a non-asymptotic analysis of the volumes of uncertainty sets estimated by SME uncertainty sets, as opposed to the current focus on their diameters.

(a) Pendulum

(b) Uncertainty set diameter

(c) Uncertainty set

T=50

T=200

T=250

T=400

T=500

★

True

103

40.04

102

40.02

iameter

101

α

40.00

100

θ

2

n

39.98

10-1

39.96

10-2

39.94

10-3

0

250

500

750

1000

1.96

1.98

2.00

2.02

2.04

T

01

Broader Impact

This paper is a foundation research and develops theoretical insight to estimation of nonlinear control systems. We do not see a direct path to negative applications in general. But we want to mention that successful applications of our theoretical results rely on verifying the assumptions in this paper.

References

Yasin Abbasi-Yadkori and Csaba Szepesvári. Regret bounds for the adaptive control of linear quadratic systems. In Proceedings of the 24th Annual Conference on Learning Theory , pages 1-26. JMLR Workshop and Conference Proceedings, 2011.

Hüseyin Akçay. The size of the membership-set in a probabilistic framework. Automatica , 40(2): 253-260, 2004.

Andrea Alaimo, Valeria Artale, C Milazzo, Angela Ricciardello, and LUCA Trefiletti. Mathematical modeling and control of a hexacopter. In 2013 International conference on unmanned aircraft systems (ICUAS) , pages 1043-1050. IEEE, 2013.

Er-Wei Bai, Roberto Tempo, and Hyonyong Cho. Membership set estimators: size, optimal inputs, complexity and relations with least squares. IEEE Transactions on Circuits and Systems I: Fundamental Theory and Applications , 42(5):266-277, 1995.

Er-Wei Bai, Hyonyong Cho, and Roberto Tempo. Convergence properties of the membership set. Automatica , 34(10):1245-1249, 1998.

Dimitri P Bertsekas. Control of uncertain systems with a set-membership description of the uncertainty. PhD thesis, Massachusetts Institute of Technology, 1971.

Vladimir Igorevich Bogachev and Maria Aparecida Soares Ruas. Measure theory , volume 1. Springer, 2007.

Xinyi Chen and Elad Hazan. Black-box control for linear dynamical systems. In Conference on Learning Theory , pages 1114-1143. PMLR, 2021.

Alexandru Cr˘ aciun and Debarghya Ghoshdastidar. On the stability of gradient descent for large learning rate. arXiv preprint arXiv:2402.13108 , 2024.

Sarah Dean, Horia Mania, Nikolai Matni, Benjamin Recht, and Stephen Tu. Regret bounds for robust adaptive control of the linear quadratic regulator. Advances in Neural Information Processing Systems , 31:4188-4197, 2018.

Eli Fogel and Yih-Fang Huang. On the value of information in system identification-bounded noise case. Automatica , 18(2):229-238, 1982.

Dylan Foster, Tuhin Sarkar, and Alexander Rakhlin. Learning nonlinear dynamical systems from a single trajectory. In Learning for Dynamics and Control , pages 851-861. PMLR, 2020.

Yihuai Gao, Yukai Tang, Han Qi, and Heng Yang. Closure: Fast quantification of pose uncertainty sets. arXiv preprint arXiv:2403.09990 , 2024.

Paul R Halmos. Measure theory , volume 18. Springer, 2013.

Mohammad Khosravi. Representer theorem for learning koopman operators. IEEE Transactions on Automatic Control , 2023.

Jihun Kim and Javad Lavaei. Online bandit control with dynamic batch length and adaptive learning rate. 2024.

Jason Kong, Mark Pfeiffer, Georg Schildbach, and Francesco Borrelli. Kinematic and dynamic vehicle models for autonomous driving control design. In 2015 IEEE intelligent vehicles symposium (IV) , pages 1094-1099. IEEE, 2015.

Suhas Kowshik, Dheeraj Nagaraj, Prateek Jain, and Praneeth Netrapalli. Near-optimal offline and streaming algorithms for learning non-linear dynamical systems. Advances in Neural Information Processing Systems , 34:8518-8531, 2021.

Steven G Krantz and Harold R Parks. A primer of real analytic functions . Springer Science & Business Media, 2002.

Bruce D Lee, Ingvar Ziemann, George J Pappas, and Nikolai Matni. Active learning for controloriented identification of nonlinear systems. arXiv preprint arXiv:2404.09030 , 2024.

Yingying Li, Subhro Das, and Na Li. Online optimal control with affine constraints. In Proceedings of the AAAI Conference on Artificial Intelligence , volume 35, pages 8527-8537, 2021a.

Yingying Li, Yujie Tang, Runyu Zhang, and Na Li. Distributed reinforcement learning for decentralized linear quadratic control: A derivative-free policy optimization approach. IEEE Transactions on Automatic Control , 67(12):6429-6444, 2021b.

Yingying Li, James A Preiss, Na Li, Yiheng Lin, Adam Wierman, and Jeff S Shamma. Online switching control with stability and regret guarantees. In Learning for Dynamics and Control Conference , pages 1138-1151. PMLR, 2023a.

Yingying Li, Tianpeng Zhang, Subhro Das, Jeff Shamma, and Na Li. Non-asymptotic system identification for linear systems with nonlinear policies. IFAC-PapersOnLine , 56(2):1672-1679, 2023b.

Yingying Li, Jing Yu, Lauren Conger, Taylan Kargin, and Adam Wierman. Learning the uncertainty sets of linear control systems via set membership: A non-asymptotic analysis. In Proceedings of the 41st International Conference on Machine Learning , pages 29234-29265. PMLR, 2024. URL https://proceedings.mlr.press/v235/li24ci.html .

Yiheng Lin, James A Preiss, Emile Anand, Yingying Li, Yisong Yue, and Adam Wierman. Online adaptive policy selection in time-varying systems: No-regret via contractive perturbations. Advances in Neural Information Processing Systems , 36, 2024.

Matthias Lorenzen, Mark Cannon, and Frank Allgöwer. Robust mpc with recursive model update. Automatica , 103:461-471, 2019.

Xiaonan Lu and Mark Cannon. Robust adaptive model predictive control with persistent excitation conditions. Automatica , 152:110959, 2023.

Xiaonan Lu, Mark Cannon, and Denis Koksal-Rivet. Robust adaptive model predictive control: Performance and parameter estimation. International Journal of Robust and Nonlinear Control , 2019.

Horia Mania, Michael I Jordan, and Benjamin Recht. Active learning for nonlinear system identification with guarantees. Journal of Machine Learning Research , 23(32):1-30, 2022.

Kumpati S Narendra and Anuradha M Annaswamy. Persistent excitation in adaptive systems. International Journal of Control , 45(1):127-160, 1987.

Bernd R Noack, Konstantin Afanasiev, Marek Morzy´ nski, Gilead Tadmor, and Frank Thiele. A hierarchy of low-dimensional models for the transient and post-transient cylinder wake. Journal of Fluid Mechanics , 497:335-363, 2003.

Valentin V Petrov. On lower bounds for tail probabilities. Journal of statistical planning and inference , 137(8):2703-2705, 2007.

Arnab Sarker, Peter Fisher, Joseph E Gaudio, and Anuradha M Annaswamy. Accurate parameter estimation for safety-critical systems with unmodeled dynamics. Artificial Intelligence , page 103857, 2023.

Shankar Sastry and Marc Bodson. Adaptive control: stability, convergence and robustness . Courier Corporation, 2011.

Yahya Sattar and Samet Oymak. Non-asymptotic and accurate learning of nonlinear dynamical systems. Journal of Machine Learning Research , 23(140):1-49, 2022.

Yahya Sattar, Samet Oymak, and Necmiye Ozay. Finite sample identification of bilinear dynamical systems. In 2022 IEEE 61st Conference on Decision and Control (CDC) , pages 6705-6711. IEEE, 2022.

Guanya Shi, Kamyar Azizzadenesheli, Michael O'Connell, Soon-Jo Chung, and Yisong Yue. Metaadaptive nonlinear control: Theory and algorithms. Advances in Neural Information Processing Systems , 34:10013-10025, 2021.

Siciliano, L. Sciavicco, L. Villani, and G. Oriolo. Robotics: Modelling, Planning and Control . Advanced Textbooks in Control and Signal Processing. Springer London, 2010. ISBN 9781846286414. URL https://books.google.com/books?id=jPCAFmE-logC .

Max Simchowitz and Dylan Foster. Naive exploration is optimal for online lqr. In International Conference on Machine Learning , pages 8937-8948. PMLR, 2020.

Max Simchowitz, Horia Mania, Stephen Tu, Michael I Jordan, and Benjamin Recht. Learning without mixing: Towards a sharp analysis of linear system identification. In Conference On Learning Theory , pages 439-473. PMLR, 2018.

John W Simpson-Porco, Florian Dörfler, and Francesco Bullo. Voltage stabilization in microgrids via quadratic droop control. IEEE Transactions on Automatic Control , 62(3):1239-1253, 2016.

Fredrik P Skantze, A Koji´ c, A-P Loh, and Anuradha M Annaswamy. Adaptive estimation of discrete-time systems with nonlinear parameterization. Automatica , 36(12):1879-1887, 2000.

Jean-Jacques E Slotine and Weiping Li. Applied nonlinear control , volume 199. Prentice hall Englewood Cliffs, NJ, 1991.

Yukai Tang, Jean-Bernard Lasserre, and Heng Yang. Uncertainty quantification of set-membership estimation in control and perception: Revisiting the minimum enclosing ellipsoid. In 6th Annual Learning for Dynamics & Control Conference , pages 286-298. PMLR, 2024.

Andrew Wagenmaker and Kevin Jamieson. Active learning for identification of linear dynamical systems. In Conference on Learning Theory , pages 3487-3582. PMLR, 2020.

Haonan Xu and Yingying Li. On the convergence rates of set membership estimation of linear systems with disturbances bounded by general convex sets. arXiv preprint arXiv:2406.00574 , 2024.

Christopher Yeh, Jing Yu, Yuanyuan Shi, and Adam Wierman. Online learning for robust voltage control under uncertain grid topology. IEEE Transactions on Smart Grid , 2024.

Jing Yu, Dimitar Ho, and Adam Wierman. Online adversarial stabilization of unknown networked systems. Proceedings of the ACM on Measurement and Analysis of Computing Systems , 7(1):1-43, 2023.

Ingvar Ziemann and Stephen Tu. Learning with little mixing. Advances in Neural Information Processing Systems , 35:4626-4637, 2022.

Ingvar Ziemann, Anastasios Tsiamis, Bruce Lee, Yassir Jedra, Nikolai Matni, and George J Pappas. A tutorial on the non-asymptotic theory of system identification. In 2023 62nd IEEE Conference on Decision and Control (CDC) , pages 8921-8939. IEEE, 2023.

Ingvar Ziemann, Stephen Tu, George J Pappas, and Nikolai Matni. Sharp rates in dependent learning theory: Avoiding sample size deflation for the square loss. arXiv preprint arXiv:2402.05928 , 2024.

Roadmap

Appendix A provides a proof of Theorem 1.

Appendix B provides proofs of Theorem 2 and Corollary 1.

Appendix C presents a proof of Theorem 3 and Corollaries 2 and 3.

Appendix D provides more details of the simulation settings.

Appendix E discusses the numerical estimation of the BMSB parameters ( s ϕ , p ϕ ) in Theorem 1.

The NeurIPS Paper Checklist is provided after the appendices.

A Proof Theorem 1

Proof. Given that u t = η t and satisfies the conditions in Assumption 3, u t is bounded, meaning u t ∈ U , where U is a compact set. Moreover, since the system is LISS, there exist functions γ ∈ K and β ∈ KL , such that for all t ≥ 0 , the following holds:

with parameters ρ x ≥ ∥ x 0 ∥ 2 and ρ ≥ max( w max , u max ) . Let Z = X ×U , then z t ∈ Z for all t ≥ 0 . The set Z is a compact subset of R n x + n u .

To show that the {F t } t ≥ 1 -adapted process { ϕ ( z t ) } t ≥ 1 satisfies the BMSB condition, it is sufficient to demonstrate that there exist s ϕ > 0 and p ϕ ∈ (0 , 1) such that for all t ≥ 0 and for any v ∈ R n ϕ with ∥ v ∥ 2 = 1 , the following holds:

To establish this, we apply the Paley-Zygmund inequality, which gives a lower bound on the tail probability of a non-negative random variable:

Lemma 1. (Paley-Zygmund (Petrov, 2007)) Let x be a non-negative random variable. Then for any r ∈ (0 , 1) , the following holds:

Based on this result, for any r ∈ (0 , 1) , we have:

Let V = { v ∈ R n ϕ : ∥ v ∥ 2 = 1 } . To show that the BMSB condition holds, it is sufficient to establish the following two points:

inf F t , t ≥ 0 inf v ∈V E [( v ⊺ ϕ ( z t +1 ) ) 2 ∣ ∣ F t ] > 0 ,

Appendix

These conditions ensure that the {F t } t ≥ 1 -adapted process { ϕ ( z t ) } t ≥ 1 satisfies the BMSB condition with some constants s ϕ > 0 and p ϕ ∈ (0 , 1) . We will divide the proof into two parts:

We begin by noting the following:

Since z t ∈ F t while w t , u t +1 ̸∈ F t , we can treat z t as a constant and w t , u t +1 = η t +1 as random variables. From the continuity of features ϕ ( · ) , we can conclude that:

where w , η are independent random variables, as assumed in Assumptions 2 and 3. Now, let N z v = { ( w,η ) ∈ W × U : v ⊺ ϕ ( h ( z, w ) , η ) = 0 } , and we have:

̸

Therefore, we have:

It is evident that if N z v = ∅ , then

̸

̸

leading to inf z ∈Z inf v ∈V E [ ( v ⊺ ϕ ( h ( z, w ) , η )) 2 ] > 0 . Now we proceed with the case where N z v = ∅ .

For this, we can use the following lemma concerning the zero set of real-analytic functions in terms of Lebesgue measure.

Lemma 2 (The zero set of real-analytic functions (Cr˘ aciun and Ghoshdastidar, 2024)) . The set of zeros of a non-trivial real-analytic function f : R n → R has a Lebesgue measure zero in R n .

This is a known result and can be proved using the identity theorem along with Fubini's theorem. For further information on this topic, see sources such as (Krantz and Parks, 2002; Bogachev and Ruas, 2007).

Recall that we defined h ( z, w ) = θ ∗ ϕ ( z ) + w . Notice that h ( · , · ) is real-analytic. Now consider

where ϕ i ( h ( z, w ) , η ) are linearly independent. Hence, the sum ∑ n ϕ i =1 v i ϕ i ( h ( z, w ) , η ) ̸≡ 0 for any v ∈ V . This implies that v ⊺ ϕ ( h ( z, w ) , η ) is real-analytic and non-zero. Consequently, by Lemma 2, λ n x + n u ( N z v ) = 0 for any v ∈ V .

Under Assumptions 3 and 2, there cannot exist a set E ⊂ Z of Lebesgue measure zero in R n x + n u for which the P ( ( w,η ) ∈ E ) = 1 . Taking this into account, along with the fact that λ n x + n u ( N z v ) = 0 and that the sets V and Z are closed sets (implying they include all their limit points), we can conclude that

̸

̸

Moreover, since λ n x + n u ( N z v ) = 0 and λ n x + n u ( W × U ) = 0 , it follows that ( N z v ) c = ∅ . This implies

̸

̸

Substituting these results into (6), we obtain:

Since z t ∈ Z for t ≥ 0 , and considering that the noise and disturbances are bounded while the features are real-analytic, it follows that z t +1 |F t is a bounded random variable. Consequently, v ⊺ ϕ ( z t +1 ) |F t is also bounded. Given that both Z and V are compact sets-meaning they contain all their limit points-and that any random variable with bounded support has finite moments, we conclude that

We finalize the proof by combining the results from Step 1 and Step 2.

B Proofs for Theorem 2 and Corollary 1

B.1 Proof of Theorem 2

Proof. The proof hinges on the following key meta-theorem about the LSE convergence rate:

Theorem 4 (LSE meta-theorem (Simchowitz et al., 2018)) . Fix δ ∈ (0 , 1) , T ≥ 1 , and 0 ≺ Γ sb ≺ ¯ Γ . Consider a random process { ( y t , x t ) } t ≥ 1 ∈ ( R n y × R n x ) T , and a filtration {F t } t ≥ 1 . Suppose the following conditions hold:

x t = θ ∗ y t + w t , where w t |F t is a zero mean σ 2 w -sub-Gaussian,

{ y t } t ≥ 1 is an {F t } t ≥ 1 -adapted random process satisfying the ( k, Γ sb , p ) -BMSB condition,

P ( ∑ T t =1 y t y ⊺ t ̸⪯ T ¯ Γ ) ≤ δ .

If the trajectory length T satisfies

then with probability at least 1 -3 δ , LSE estimation error is bounded by:

Since w t satisfies the Assumption 2), and w t / ∈ F t , then σw |F t is sub-Gaussian with parameter σw . Additionally, system (1) is linear in unknown parameters θ ∗ . From Theorem 1, the {F t } t ≥ 1 -adapted process { ϕ ( z t ) } t ≥ 1 satisfies the (1 , s 2 ϕ I n ϕ , p ϕ ) -BMSB condition for some s ϕ > 0 and p ϕ ∈ (0 , 1] .

To complete the proof, it is left to show that for any δ ∈ (0 , 1) , there exists a ¯ Γ ≻ s 2 ϕ I n ϕ , such that

To see this, note that for ¯ b ϕ = sup t ≥ 0 E [ ∥ ϕ ( z t ) ∥ 2 2 ] , we have:

In addition, we have:

where ¯ Γ = ¯ b ϕ δ I n ϕ . since z t ∈ Z for all t ≥ 0 , with Z being a compact set (due to the system's LISS property and features ϕ ( · ) being real-analytic), such a bounded ¯ b ϕ exists, completing the proof.

B.2 Proof of Corollary 1

Proof. We start the proof by stating the following lemma which is extension of Theorem 1 to the case with u t = π ( x t ) + η t .

Lemma3. Let u t = π ( x t )+ η t , π ( · ) is real-analytic, η t satisfies Assumption 3. Consider the filtration F t = F ( w 0 , · · · , w t -1 , x 0 , · · · , x t , π ( x 0 ) , · · · , π ( x t ) , η 0 , · · · , η t ) . Suppose Assumptions 1, 2 hold and that the closed-loop system x t +1 = θ ∗ ϕ ( x t , π ( x t ) + η t ) + w t satisfies Assumption 4 for both w t and η t . Then there exist s ϕ > 0 , and p ϕ ∈ (0 , 1) such that the {F t } t ≥ 1 -adapted process { ϕ ( x t , u t )} t ≥ 1 satisfies the ( 1 , s 2 ϕ I n ϕ , p ϕ ) -BMSB condition.

Proof of Lemma 3. The proof closely follows the steps of Theorem 1. First, observe that u t = π ( x t ) + η t . Since η t satisfies the conditions outlined in Assumption 3, and the closed-loop system x t +1 = θ ∗ ϕ ( x t , π ( x t ) + η t ) + w t adheres to Assumption 4 with respect to both w t and η t , there exist functions γ ∈ K and β ∈ KL such that for all t ≥ 0 the following holds:

with parameters ρ x ≥ ∥ x 0 ∥ 2 and ρ ≥ max( w max , u max ) . Let Z = X ×U , where U is a compact set containing u t for all t ≥ 0 . Thus, z t ∈ Z for all t ≥ 0 . The set Z is a compact subset of R n x + n u . The remaining part of the proof, specifically to show that sup F t , t ≥ 0 sup v ∈V E [( v ⊺ ϕ ( z t +1 ) ) 4 ∣ ∣ F t ] < ∞

follows similarly to the proof in Theorem 1.

It remains to show that inf F t , t ≥ 0 inf v ∈V E [( v ⊺ ϕ ( z t +1 ) ) 2 ∣ ∣ F t ] > 0 . This can be shown as follows:

Since z t ∈ F t and w t , η t +1 ̸∈ F t then z t is treated as constant while w t , η t +1 are considered random variables. Then, by the continuity of ϕ we conclude that

where w and η are independent random variables constrained as described in Assumptions 2 and 3. By letting N z v = { ( w,η ) ∈ W × U : v ⊺ ϕ ( h ( z, w ) , π ( h ( z, w ) ) + η ) = 0 } , similar to (6) we have:

We aim to show that λ n x + n u ( N z v ) = 0 by applying Lemma 2. Note that ϕ ( · ) , h ( · ) , and π ( · ) are realanalytic. To use the results of Lemma 2, we need to establish that v ⊺ ϕ ( h ( z, w ) , π ( h ( z, w ) ) + η ) is non-zero for any v ∈ V . First, observe that:

Now consider two scenarios:

All components of π ( h ( z, w ) ) are linearly independent with any component of h ( z, w ) .

At least one component of π ( h ( z, w ) ) is linearly dependent with one or more components of h ( z, w ) .

In both cases, due to the additive nature of η , all the functions ϕ i ( h ( z, w ) , π ( h ( z, w )) + η ) ) with i = 1 , · · · , n ϕ are linearly independent, ensuring that v ⊺ ϕ ( h ( z, w ) , π ( h ( z, w )) + η ) ̸≡ 0 for any v ∈ V . The remainder of the proof follows similarly to the argument in Theorem 1.

Using this Lemma, Theorem 4, and reasoning similar to that in the proof of Theorem 2, the proof can be completed.

C Proofs for Theorem 3, Corollary 2, and Corollary 3

C.1 Proof of Theorem 3

Proof. The proof follows from applying the following meta-theorem on the convergence rate of SME.

Theorem 5 (SME meta-theorem (Li et al., 2024)) . Consider a general time series model with linear responses as follows:

Also, define the filtration F t = F ( w 0 , · · · , w t -1 , y 0 , · · · , y t ) . Assume the following conditions are met:

w t are i.i.d. with variance σ 2 w I n x , and box-constrained, i.e., w t ∈ W = { w ∈ R n x : ∥ w ∥ ∞ ≤ w max } for some w max > 0 .

{ y t } t ≥ 1 is an {F t } t ≥ 1 -adapted random process satisfying the ( k, s 2 y I n y , p y ) -BMSB condition.

There exists b y > 0 such that ∥ y t ∥ 2 ≤ b y almost surely for all t ≥ 0 .

For any ℓ > 0 , there exists q w ( ℓ ) > 0 , such that for any 1 ≤ j ≤ n and t ≥ 0 , we have P ( w j t + w max ≤ ℓ ) ≥ q w ( ℓ ) > 0 , P ( w max -w j t ≤ ℓ ) ≥ q w ( ℓ ) > 0 .

Then for any m ≥ 1 and any δ ∈ (0 , 1) , when T > m , the diameter of the uncertainty set

satisfies:

Observe that system (1) is linear in the unknown parameters θ ∗ , and we can prove Theorem 3 by showing that the {F t } t ≥ 1 -adapted process { ϕ ( z t ) } t ≥ 1 and w t meet the conditions of the metatheorem. By Assumptions 2 and 5, and since w t ̸∈ F t , the noise w t fulfills all the requirements of the meta-theorem. Moreover, according to Theorem 1, the {F t } t ≥ 1 -adapted process ϕ ( z t ) t ≥ 1 satisfies the (1 , s 2 ϕ I n ϕ , p ϕ ) -BMSB condition for some s ϕ > 0 and p ϕ ∈ (0 , 1] . Lastly, since the system is LISS, we have z t ∈ Z for all t ≥ 0 , where Z is the compact set defined in the proof of Theorem 1. Therefore, there exists a constant b ϕ > 0 such that sup t ≥ 0 ∥ ϕ ( z t ) ∥ 2 ≤ b ϕ , completing the proof of the theorem.

Explicitly, this means that for any m ≥ 1 , for any δ ∈ (0 , 1) , when T > m , we have:

C.2 Proof of Corollary 2

Let us provide two example distributions, truncated Gaussian and uniform, along with their corresponding q w ( · ) (from (Li et al., 2024)):

If w t follows a uniform distribution on [ -w max , w max ] n x , then q w ( ℓ ) = c w ℓ with c w = 1 2 w max .

If w t follows a truncated-Gaussian distribution on [ -w max , w max ] n x , generated by a Gaussian distribution with zero mean and covariance matrix σ 2 w I n x , then q w ( ℓ ) = c w ℓ with

Now, fix ϵ ∈ (0 , 1) . We want to show that if q ( a 1 δ 4 √ n ϕ ) = c w a 1 δ 4 √ n ϕ and we choose m such that

then for all T ≥ m , we have

with probability at least 1 -2 ϵ .

Let the two terms in the right-hand side of (8) be denoted by "term 1" and "term 2". We proceed with the proof in two steps as follows:

Step 1: showing that with the choice of m in (9) , term 1 ≤ ϵ :

With this choice of m , it is straightforward to see that

and thus term 1 ≤ ϵ .

Step 2: letting term 2 = ϵ and showing that δ satisfies the inequality in (10) :

Assuming without loss of generality that T m is an integer, note that term 2 = ϵ implies:

Combining these two steps, we conclude that, with probability at least 1 -2 ϵ ,

C.3 Proof of Corollary 3

This corollary's proof builds on Lemma 3 in Appendix B.2 and closely aligns with the proofs of Theorem 3 and Corollary 2.

D Numerical Experiments

This section provides details on the simulation experiments.

D.1 Pendulum

The ground truth for the unknown parameters for pendulum example in Example 1 is set to be

and discretization time step in our numerical experiments is dt = 0 . 01 (s). The control input is a simple feedback controller u t = -k ˙ α t + η t . In Figures 1a and 1b we choose k = 2 and in Figures 2a, 2b and 3 we choose k = 0 . 1 . Note there are two unknown parameters in this pendulum example as follows:

D.2 Quadrotor

The ground truth for the unknown parameters for quadrotor example in Example 2 is set to be

The discretization time step in our numerical experiments is dt = 0 . 01 (s). The control input is a control policy plus i.i.d. noise. The control policy on altitude and the three Euler angles is borrowed from (Alaimo et al., 2013). The controller gains in our numerical experiments are chosen as:

Note that there are seven unknown parameters in this quadrotor example, as follows:

Figure 4 displays the uncertainty set estimated by SME for the seven unknown parameters in the quadrotor example for various trajectory lengths, with η t and w t being i.i.d. samples from truncatedGaussian distributions. The uncertainty sets are observed to shrink as the trajectory length increases, consistent with our theoretical results. Note that the ground truth value is contained within all the uncertainty sets.

E Numerical Estimation of BMSB Parameters ( s ϕ , p ϕ )

We compare the empirical rates of LSE and SME with their theoretical counterparts in Section 4. The theoretical results presented in Theorem 2 and Corollary 2 rely on the parameters b ϕ , ¯ b ϕ , and the BMSBparameters ( s ϕ , p ϕ ) . However, the explicit relationship of these parameters with system, noise, and disturbance characteristics such as n x , n u , n ϕ , σ u , and σ w is not known and we will address this in our future work. Consequently, we estimate these parameters numerically and utilize these

T=200

T=500

T=2000

True

206.4

-0.8

0.10

-0.9

1.0

0.05

θ

2

206.2

-1.0

6

0.00

206.0

0.5

-0.05

1.1

205.8

-0.10L

2.13

2.14

2.15

205.8 206.0 206.2 206.4

205

206

112

113

114

01

02

04

06

estimates to calculate the theoretical rates discussed in Section 4. While b ϕ and ¯ b ϕ are straightforward to estimate, special attention is required to estimate the BMSB parameters. This section is dedicated to describing this estimation process.

For this, consider a system of the form (1). For this system, our goal is to estimate s ϕ and p ϕ , where

numerically. First, observe that ϕ ( z t +1 ) | F t = ϕ ( θ T ∗ ϕ ( z t ) + w t , u t +1 ) , where z t ∈ F t . This implies that ϕ ( z t +1 ) | F t is a random variable influenced by w t and u t +1 . We proceed by fixing s ϕ = ¯ s (for some ¯ s > 0 ) and empirically estimate p ϕ . To accomplish this, we first select a time horizon T and generate several trajectories of length T for the system. Let D T represent the set of these trajectories, while D t denotes a subset containing all trajectories up to t ≤ T . Additionally, we create multiple vectors v ∈ R n ϕ such that ∥ v ∥ 2 = 1 ; we refer to this collection as ¯ V . These vectors are randomly sampled from a Gaussian distribution and subsequently normalized.

We then estimate ¯ p as:

As T increases, along with the number of trajectories and vectors v , the minimum estimates will more accurately reflect the infimums. In this context, ¯ p represents the minimum of P ( | v T ϕ ( θ T ∗ ϕ ( z ) +

w t , u t +1 ) | ≥ ¯ s ) across all combinations in [ T ] ×D T × ¯ V . For each combination in this set, we estimate the probability P ( | v T ϕ ( θ T ∗ ϕ ( z ) + w t , u t +1 ) | ≥ ¯ s ) using Monte Carlo simulations. This

process entails generating multiple random samples based on the distributions of w t and u t +1 , verifying whether each sample satisfies the condition | ¯ v T ϕ ( z ) | ≥ ¯ s , and tallying how many samples meet this criterion. We repeat this procedure for various values of ¯ s until we identify a pair of (¯ s, ¯ p ) such that ¯ p ∈ (0 , 1) . The estimated probability is calculated as the ratio of the count of successful samples to the total number of samples. According to the law of large numbers, this ratio converges to the true probability as the number of samples increases. For our estimations, we select T = 50 , | ¯ V| = 1000 , and |D T | = 20 .

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper studies the problem of identifying an observable stochastic nonlinear dynamical system, in case the transition function is linearly parametrized and the noise is additive. The authors assume that the feature functions are analytic, and both the inputs and the noises are i.i.d., bounded, semi-continuous and have nonvanishing variances in each coordinate direction. It is also assumed that the system is locally input-to-state stable. Two kinds of estimation methods are studied: the classical least-squares estimate (LSE), which provides point estimates, and a set membership method, which provides region estimates. Finite sample bounds for the performance of LSE are proved, based on a block-martingale-small-ball condition, for both the open-loop and the close-loop cases. The sample complexity of the set membership method is studied under an additional assumption (tight bound on disturbance) and only for open-loop (i.i.d.) inputs. Finally, the authors present some numerical evaluations of LSE and set membership methods on pendulum and quadrotor examples to empirically illustrate their convergence rates.\n\nSTRENGTHS:\n- The presentation is clear, the paper is well-structured, the need assumptions are precisely stated.\n- The LSE are widely used, and the set membership approach is also reasonable and practically relevant.\n- Sample complexity analysis of system identification methods for nonlinear systems is an important problem. On the other hand, it is more relevant for control theory than to machine learning.\n- Finite sample error bounds are provided for both methods under the assumption that the features are analytic. This approach seems original and could be interesting for the community.\n- The examples illustrate well the theoretical viability of the analytic features assumption.\n\nWEAKNESSES:\n- Many of the assumptions are restrictive, such as bounded noises and inputs. Moreover, the inputs should also be i.i.d. (or should have an additive i.i.d. exploration noise, for the closed-loop case) which is unrealistic.\n- The error bounds contain terms which are unknown in practice, such as s_\\phi.\n- Corollary 1: the controller for the closed-loop case also contains an additive noise term that satisfies the assumptions for the open-loop inputs, and the system with the controller satisfies the stability assumption, in which case the statement becomes a simple consequence of Theorem 1. A case without additional noises on the inputs would have been much more interesting.\n- The figures do not show the actual bounds deducted by the paper, they only illustrate the empirical performance of LSE and set membership identification, which is a bit pointless, as they are classical, well-studied methods. I understand that the bounds of the paper are theoretical in nature, so they are conservative and mostly just give the convergence rate, still showing them (for example, on a logarithmic scale) would have been informative.\n- The title could be misleading, as the word \"analytic\" should refer to the feature vectors, but in the current title the term \"analytic system identification\" could also be understood in a way that the obtained solution is analytic.\n\nQUESTIONS:\n- What is the intuitive meaning of Assumption 5 (tight bound on disturbance)? This should be explained in the paper.\n\nLIMITATIONS:\nThere is a section dedicated to the limitations of the work, which is a good thing. On the other hand, this section did not mention some key limitations, such as bounded noises, bounded and i.i.d. inputs, as well was fully observable states.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-08"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe manuscript provides theoretical guarantees for nonlinear system identification using non-active i.i.d. noises, extending from linear systems to linearly parametrized nonlinear systems with analytic feature functions.  The findings demonstrate that non-active i.i.d. noises are capable of efficiently learning these systems with a non-asymptotic convergence rate.\n\nSTRENGTHS:\nTraining data selection is a crucial factor in ensuring the generalizability and robustness of identification algorithms. This is particularly important to linearly parametrized nonlinear systems, as there is still a theoretical gap regarding the effectiveness of non-active i.i.d. noise exploration. The study is well-motivated and has great potential for practical application. The authors establish conditions on noise that guarantee probabilistic persistent excitation for nonlinear dynamical systems, as defined by the BMSB condition. The findings in this study provide theoretical support for designing training data and identifying nonlinear systems.\n\nWEAKNESSES:\nAlthough the authors present an interesting idea, the manuscript could be improved by addressing the following concerns: 1. It is suggested to discuss the potential limitations of the non-asymptotic convergence rate in practical applications. 2. The analysis of the numerical experiments lacks depth. Could the authors further explore the convergence characteristics and the advantages and disadvantages of using i.i.d. noise for excitation? 3. The proof sketches could benefit from additional clarifications to enhance readability. For example, in line 234, the meaning of $\\delta$ and $\\bar{b}_\\phi$ could be re-mentioned.\n\nQUESTIONS:\nCan the authors please clarify the implications of the assumptions regarding bounded noises in more practical scenarios? It would be highly beneficial to delve deeper into the impact of these theoretical assumptions on real-world applications.\n\nLIMITATIONS:\nIt is recommended to further explore the potential limitations of this  theoretical guarantees in practical applications. For instance, whether non-asymptotic convergence implies reduced data collection efficiency for some high-dimensional systems, and whether it is still preferable to actively design experiments in certain scenarios.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-13"}

comments: SUMMARY:
The authors study the problem of system identification from a trajectory generated by an unknown linearly parametrized nonlinear system whose nonlinearity is an analytic function. Specifically, they theoretically analyze to estimators: the least squares estimator, and the set membership estimator. Both estimators, while being widely used in practice, do not have theoretical guarantees in many of the settings in which they are applied. The authors give such guarantees, and also conduct numerical experiments on certain nonlinear systems.

STRENGTHS:
The authors provide non asymptotic theoretical guarantees for common system identification estimators in settings more broad than those covered by existing theoretical results. As far as I can tell the results are new and the argumentation is sound. They also conduct experiments which verify that such estimators converge to the true system parameters in practice.

WEAKNESSES:
While the theoretical arguments presented seem sound, I am not sure that the extension of system identification results to the case of linearly parametrized smooth nonlinear systems is sufficiently significant from a theoreitcal perspective.

QUESTIONS:
If instead of assuming that the nonlinearity is analytic, one assumed directly that there does not exist an open set on which it vanishes identically, would the proof carry through?

LIMITATIONS:
The authors adequately adress the limitations of their work.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2024-07-15

comments: SUMMARY:
The authors extend the work of Simchowitz et al. (2018) [linear] and Sattar et al. (2022) [bi-linear] to linear in the parameters but analytic nonlinear features showing that PE of inputs still results in PE of the states which for general nonlinear systems is not true. From this result LSE results like those originally in Simchowitz et al. (2018) are recovered for this new class of systems and SME results like those in Li et al. (2023a) are recovered for this broader class of systems.

STRENGTHS:
- originality: original part is recognizing that analytics functions are only bad in finite number of spots and are otherwise "nice" functions 
- quality: yes rigorous math
- clarity: well written
- significance: greatly extends the original results to a much larger class of systems

WEAKNESSES:
- the biggest weakness of the paper is that very little time is spent on the analysis in the main text. yes their is a proof sketch for the main theorem, but it would be nice to get more intuition from the main text.

QUESTIONS:
- I would much prefer the simulations in the appendix and the inclusion of the entirety of the proof of the main theorem in the main text with clear exposition of how the properties of analytic functions can be exploited to prove PE of u results in PE of x for these systems. This paper is not as bad as others where the main text is only an advertisement of the main result, but it would be a much tighter and more useful paper if this was done. This reviewer may be an outlier though as it seems the trend in these conferences is to do just the opposite of what I have suggested.

LIMITATIONS:
limitations are discussed
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 4
DATE: 2024-07-17