Non-linear System Identification from Partial Observations via Iterative Smoothing and Learning

1 INTRODUCTION . The performance of controllers and state-estimators for non-linear systems depends heavily on the quality of the model of system dynamics ( Hou & Wang , 2013 ) . System-identification tackles the problem of learning or calibrating dynamics models from data ( Ljung , 1999 ) , which is often a timehistory of observations of the system and control inputs . In this work , we address the problem of learning dynamics models of partially observed , high-dimensional non-linear systems . That is , we consider situations in which the system ’ s state can not be inferred from a single observation , but instead requires inference over a time-series of observations . The problem of identifying systems from partial observations arises in many robotics domains ( Punjani & Abbeel , 2015 ; Cory & Tedrake , 2008 ; Ordonez et al. , 2017 ) . Though we often have direct measurements of a robot ’ s pose and velocity , in many cases we can not directly observe relevant quantities such as the temperature of actuators or the state the environment around the robot . Consider learning a dynamics model for an aerobatic helicopter . Abbeel et al . ( 2010 ) attempted to map only the helicopter ’ s pose and velocity to its acceleration and they found their model to be inaccurate when predicting aggressive maneuvers . They posited that the substantial airflow generated by the helicopter affected the dynamics . Since it is often impossible to directly measure the state of the airflow around a vehicle , identification must be performed in a partially observed setting . System-identification is a mature field with a rich history ( Ljung , 1999 ; 2010 ) . Various techniques can be classified by whether they apply to linear or non-linear systems , with partially or fully observed states . Additionally , techniques are applied in an online or batch-offline setting . This work presents an approach to offline identification of non-linear and partially observed systems . When a system is fully observed , i.e . its full state is observed but corrupted by noise , a set of techniques called equation-error methods are typically employed ( Åström & Eykhoff , 1971 ) . In such cases , we can consider observations as independent , and minimize the error between the observed statederivatives and those predicted by the model given the control input and observed states . In partially observed settings , merely knowing the current input is insufficient to accurately predict the observation . Several black-box approaches exist to predict observations from time-series of inputs . Autoregressive approaches directly map a time-history of past inputs to observations ( Billings , 2013 ) . Recurrent neural networks ( Bailer-Jones et al. , 1998 ; Zimmermann & Neuneier , 2000 ) and subspace-identification methods ( Van Overschee & De Moor , 1994 ) can also be used to learn blackbox dynamical systems from this data . However , in many cases prior knowledge can be used to specify structured , parameterized models of the system ( Gupta et al. , 2019 ) . Such models can be trained with less data and used with a wider array of control and state-estimation techniques than non-linear black-box models ( Gupta et al. , 2019 ; Lutter et al. , 2019b ; a ) . Techniques used to identify partially observed structured models are often based on Expectation-Maximization ( EM ) ( Dempster et al. , 1977 ; Schön et al. , 2011 ; Kantas et al. , 2015 ; Ghahramani & Roweis , 1999 ) . An alternating procedure is performed in which a smoothing step uses the current system dynamics estimate to infer the distribution over state-trajectories , and is followed by a learning step that uses this distribution to update the system dynamics estimate . In the non-linear or non-Gaussian case , it is typically not possible to analytically characterize the distribution over trajectories , and thus methods based on Sequential Monte-Carlo such as Particle Smoothing ( PS ) ( Schön et al. , 2011 ; Kantas et al. , 2015 ) , or Extended Kalman Smoothing ( EKS ) ( Ghahramani & Roweis , 1999 ) are employed in the E-step . Though considered state-of-theart for this problem , both methods become intractable in high-dimensional state spaces . PS suffers from the curse of dimensionality , requiring an intractably large number of particles if the state space is high-dimensional ( Snyder et al. , 2008 ; Kantas et al. , 2015 ) , and an M-step that can be quadratic in complexity with respect to the number of particles ( Schön et al. , 2011 ) . EKS-based methods are fast during the E-step , but the M-step requires approximations to integrate out state uncertainty , such as fitting non-linearities with Radial Basis Function approximators , and scales poorly with the dimension of the state-space ( Ghahramani & Roweis , 1999 ) . In this work , we present a system-identification algorithm that is suited for high-dimensional , nonlinear , and partially observed systems . By assuming that the systems are close to deterministic , as is often the case in robotics , we approximate the distribution over unobserved states using only their maximum-likelihood ( ML ) point-estimate . Our algorithm , called SISL ( System-identification via Iterative Smoothing and Learning ) performs the following two steps until convergence : • In the smoothing or E-step , we use non-linear programming to tractably find the ML pointestimate of the unobserved states . • In the learning or M-step , we use the estimate of unobserved states to improve the estimate of system parameters . The idea to use an ML point-estimate in lieu of the distribution over unobserved states in the EM procedure ’ s E-step is not new , and , in general , does not guarantee monotonic convergence to a local optimum ( Celeux & Govaert , 1992 ) . However , such an approximation is equivalent to regular EM if the ML point-estimate is the only instance of unobserved variables with non-negligible probability ( Celeux & Govaert , 1992 ; Neal & Hinton , 1998 ) . We apply this idea to the problem of system-identification for nearly deterministic systems , in which ML point-estimates can serve as surrogates for the true distribution over unobserved state-trajectories . The primary contribution of this work is an algorithm for identifying non-linear , partially observed systems that is able to scale to high-dimensional problems . In Section 2 , we specify the assumptions underpinning our algorithm and discuss the computational methodology for using it . In Section 3 , we empirically demonstrate that it is able to identify the parameters of a high-dimensional system of coupled Lorenz attractors , a problem that proves intractable for particle-based methods . We also demonstrate our algorithm on the problem of identifying the dynamics of an aerobatic helicopter , and compare against various approaches including the state-of-the-art approach ( Punjani & Abbeel , 2015 ) . 2 METHODOLOGY . 2.1 FORMAL PROBLEM STATEMENT . In this work , we assume that we are given a batch of trajectories containing observations y1 : T ∈ Rm×T of a dynamical system as it evolves over a time horizon T , possibly forced by some known input sequence u1 : T−1 . We assume that this dynamical system has a state x ∈ Rn that evolves and generates observations according to the following equations , xt+1 = f ( xt , ut , t ) + wt , wt ∼ pw ( · ) yt = g ( xt , ut , t ) + vt , vt ∼ pv ( · ) ( 1 ) where wt is referred to as the process noise and vt as the observation noise . wt and vt are both assumed to be additive for notational simplicity , but this is not a required assumption . Without loss of generality , we can drop the dependence on ut , absorbing it into the dependence on t. We further assume that we are provided a class of parameterized models fθ ( x , t ) and gθ ( x , t ) for θ ∈ Θ that approximate the dynamical system ’ s evolution and observation processes . The goal of our algorithm is to find the parameters θ that maximize the likelihood of the observations . That is , we seek to find : θML = arg max θ p ( y1 : T | θ ) = arg max θ ∫ x1 : T p ( y1 : T | x1 : T , θ ) p ( x1 : T | θ ) dx1 : T ( 2 ) Assuming , the system is Markovian in xt , we can factorize the distributions as : p ( x1 : T | θ ) = p ( x1 ) T−1∏ t=1 pw ( xt+1 − fθ ( xt , t ) ) , p ( y1 : T | x1 : T , θ ) = T∏ t=1 pv ( yt − gθ ( xt , t ) ) ( 3 ) In order to tractably solve the maximization problem in Equation ( 2 ) , particle-based EM techniques typically approximate the integral as an expectation over a particle set ( Schön et al. , 2011 ; Kantas et al. , 2015 ) . However , since particle-based methods can struggle with high-dimensional spaces ( Snyder et al. , 2008 ; Kantas et al. , 2015 ) , this work seeks point estimates for the likelihoodmaximizing state sequence , which can be found using non-linear programming . 2.2 SURROGATE OBJECTIVE FUNCTION . Instead of solving the full maximum likelihood problem shown in Equation ( 2 ) , we solve a surrogate problem : θSISL = arg max θ ( max x1 : T p ( y1 : T | x1 : T , θ ) p ( x1 : T | θ ) ) ( 4 ) The maximized surrogate objective is equivalent to the original objective if the distribution over x1 : T given y1 : T and θ is well-approximated by its ML point-estimate , i.e . the distribution is a Dirac delta function . We expect this assumption to hold if the trajectories are sufficiently long , the dynamics are close to deterministic , and all relevant dynamic modes are persistently excited . Making these assumptions , we now present an algorithm for solving Equation ( 4 ) . 2.3 SYSTEM-IDENTIFICATION VIA ITERATIVE SMOOTHING AND LEARNING . By taking the logarithm of the likelihood-objective in Equation ( 4 ) , and using the factorization from Equation ( 3 ) , we get the SISL objective J ( x1 : T , θ ) as follows : J ( x1 : T , θ ) = log p ( x1 ) + T∑ t=1 log pv ( yt − gθ ( xt , t ) ) + T−1∑ t=1 log pw ( xt+1 − fθ ( xt , t ) ) ( 5 ) Jointly maximizing this objective over x1 : T and θ yields θSISL . Though this objective can be optimized as is by a non-linear optimizer , it is not necessarily efficient to do so since θ and x1 : T are highly coupled , leading to inefficient and potentially unstable updates . For this reason , we take an iterative approach akin to EM , which performs block-coordinate ascent on the J ( x1 : T , θ ) . At iteration k , we first perform smoothing by holding θ constant , and finding the ML point-estimate for x1 : T as follows : x ( k ) 1 : T = arg max x1 : T J ( x1 : T , θ ( k−1 ) ) + ρx‖x1 : T − x ( k−1 ) 1 : T ‖22 ( 6 ) Here ρx scales a soft trust-region regularizer similar to damping terms found in LevenbergMarquardt methods ( Levenberg , 1944 ; Marquardt , 1963 ) . The smoothing problem can be solved in O ( n2T ) by taking advantage of sparsity in the smoothing objective ’ s Hessian matrix . In the learning step , we hold x1 : T constant and find : θ ( k ) = arg max θ J ( x ( k ) 1 : T , θ ) + ρθ‖θ − θ ( k−1 ) ‖22 + log p ( θ ) ( 7 ) Again , ρθ scales a soft trust-region regularizer , and specifying log p ( θ ) allows us to regularize θ toward a prior . The above optimization problem can be solved using any non-linear optimizer , such as a first or second-order gradient descent scheme . The SISL algorithm iterates between the smoothing and learning steps until convergence .

Summary: This paper proposes a novel way to address the well studied system identification problem. More specifically, this paper focuses on non linear systems with partial observations. To this aim, the authors propose the "System-indetification via Iterative Smoothing and Learning" (SISL) algorithm. The gist of the approach is to formulate an ML problem to estimate the system model parameters from a parametric family of models to estimate both the dynamical system evolution (state updates) and the observation process. The exact optimization for this ML problem is not tractable. This work proposes to solve a surrogate point-estimate  ML problem. This optimization is solve using an alternate approach (between smoothing and learning).