Subspace State-Space Identification and Model Predictive Control of Nonlinear Dynamical Systems Using Deep Neural Network with Bottleneck
Authors: Ichiro Maruta
Keito Yamada
Kenji Fujimoto
Venue: ICLR 2022 Submitted (2022)
Keywords: System identification
Model predictive control
Subspace state-space system identification

Abstract: A novel nonlinear system identification method that produces state estimator and predictor directly usable for model predictive control (MPC) is proposed in this paper. The main feature of the proposed method is that it uses a neural network with a bottleneck layer between the state estimator and predictor to represent the input-output dynamics, and it is proven that the state of the dynamical system can be extracted from the bottleneck layer based on the observability of the target system. The training of the network is shown to be a natural nonlinear extension of the subspace state-space system identification method established for linear dynamical systems. This correspondence gives interpretability to the resulting model based on linear control theory. The usefulness of the proposed method and the interpretability of the model are demonstrated through an illustrative example of MPC.

Under review as a conference paper at ICLR 2022
SUBSPACE STATE-SPACE IDENTIFICATION AND MODEL PREDICTIVE CONTROL OF NONLINEAR DY- NAMICAL SYSTEMS USING DEEP NEURAL NETWORK WITH BOTTLENECK
Anonymous authors Paper under double-blind review
ABSTRACT
A novel nonlinear system identification method that produces state estimator and predictor directly usable for model predictive control (MPC) is proposed in this paper. The main feature of the proposed method is that uses neural network with a bottleneck layer between the state estimator and predictor to represent the input-output dynamics, and it is proven that the state of the dynamical system can be extracted from the bottleneck layer based on the observability of the target system. The training of the network is shown to be a natural nonlinear extension of the subspace state-space system identification method established for linear dynamical systems. This correspondence gives interpretability to the resulting model based on linear control theory. The usefulness of the proposed method and the interpretability of the model are demonstrated through an illustrative example of MPC.
INTRODUCTION
Model predictive control (MPC) has been widely used as a practical method to control nonlinear dynamical systems (Qin & Badgwell 2000) Vazquez et al.) 2014)) Karamanakos al.] 2020)). As the name implies, MPC requires prediction model, and in most cases, state-space models are used. When we want to apply MPC complex system whose structure and state are unknown, model construction from input-output data, i.e., system identification, is required. For linear state-space system identification, the subspace methods (Favoreel et al. 2000 Katayama 2005) van der Veen et al. 2013) are widely used. Its fundamental idea is to reduce the dimensionality of the measured past and future input and output data, and it strongly depends on the linearity of the system. Although there have been continuous attempts specific structures such as Wiener and Hammerstein structures (e.g., Veehheeenn & Westwick 1996] Goethals et al. 2005; Katayama & Ase 2016) and related research has also been conducted in the field of machine learning as spectral learning of Hidden Markov Models (Kawahara et al. 2007 Hsu al. 2012 Kandasamy al., 2016), there is still lot of room for research extending the subspace method to general nonlinear dynamical systems.
extend the subspace methods nonlinear systems with
On the other hand, recent advances in machine learning techniques have had an impact on the re- search field of system identification (Ljung et al. 2020a|b) b). For example, deep state space models (Rangapuram et al., 2018), and Kalman variational auto entoodr (Fraccaro et al., 2017) apply deep learning to state space models. Among those technologies, autoencoder is notable dimensional reduction or feature extraction technique with deep neural networks, whose traits derives from topo- logical constraints of own bottleneck structure. Although several pieces of research apply it to state-space system identification (Masti Bemporad] (2018) Beintema et al. 2021), these works do not provide theoretical basis for obtaining the state sequence.
This paper aims to propose a theoretically supported and interpretable nonlinear dynamical sys- tem identification method by simply extending the subspace method with neural networks. For the proposed method, can be shown based on the observability of the target system that when a neu- ral network with appropriate bottleneck is trained, the state estimator and predictor are formed

---

Under review as a conference paper at ICLR 2022
across the bottleneck layer. The obtained model can be interpreted based the knowledge from linear system identification methods, and this aspect is explained through an illustrative example.
In this paper, the set of integers and the natural numbers denoted by Z and N, respectively. The set {Sa, sa+1y.... Sb- 8b3 is denoted {5*]* for conciseness. The m zero matrix is denoted as Onxms and On == Onx1- For matrix M, M is the Moore-Penrose inverse of M.
PROBLEM FORMULATION & NOTATION
We consider the following discrete-time nonlinear dynamical system:
1++1 [1x, U4),
Yr = h(x)))
(1)
with t E Z the time index, 2₁ R the state, y/ E R' the output, Ue E RR" the input, f Th= x RR= - RT= the state-transition map and RR, - R the output map. As the usual setting for system identification, In the following discussion, the fundamental properties of the proposed method for nonlinear target systems presented in the absence of measurement noise and disturbances. The effects of noise and disturbance are indirectly provided for linear target systems in section by showing that the proposed method is equivalent to the subspace identification method.
is a latent variable and and Yt are measurable with noise.
For convenience, We define the sequence input and output signals in the past and future from a certain time t as follows:
Ut-hp
Yp-hp Y We-ha hp
U
yt
Ut Me-hhrn +1
Ur+1
yt++
uP: ==
yp =
us
y! ==
(2)
Yt-1
where the superscripts and indicate past and future, and hp E N and he E N are horizons for corresponding directions, respectively. Using these notations, we formulate the problem as follows.
Problem 1. Given the measured input-output data {(u,, yo v.)1 T and the design parameters hp, hy, ng E N, construct a model that consists of a state estimator E,, which maps (u,, vt) + it. and a predictor P, which maps (it, u!) + by Here, &n E R is a state equivalent to I but in an arbitrary coordinate system and T E N indicates the size of the dataset.
Remark 1. In the usual state-space system identification, the goal is to construct a model of f and h, i.e., sate-space model. However, in many applications, including model predictive control, the state estimator and multi-step ahead predictor are often reconstructed from the resulting state-space model. In particular, for nonlinear systems that are difficult analyze, the merits of going through state-space models are questionable. Therefore, we focus here on the construction of the state estimator E and predictor P and only briefty discuss the identification of f and in Appendix A
To guarantee Problem to be solvable, we make some assumptions. With the vectors defined above we can write the system ! as
xG fbr (It-hi, u!!)
yf = hr (It-hip u!)
(3)
where
pk De, uT (I.,) f (It, Uz) ue+1) ... uc+k)
(4a)
h((x))
h f( us)
h (It. "..,(ul....
(4b)
ho pk-1 (t lut ..... uI uT(a_1l))
And define the uniform k-observability as follows. Definition (uniform k-observability (Moraal & Grizzle 1995)). If the mapping
Te (R=*)* (R**)* k (RR*))* by (x, u.) + (1" u), u.
(5)
injective, the system 7 is said to be uniformly k-observable (Moraal & Grizzle 1995
2

---

Under review as a conference paper at ICLR 2022
Then, make the following assumptions and theorem.
Assumption 1. The system is uniformly k-observable, where k: = min (bps hr). Remark 2. According to Stark et al. 1997), typical dynamical systems are k-observable for l:
2n, +
Assumption 2. The dimension of the state estimate is large enough, that ng 2 Ox-
Theorem 1. Under Assumptions and Problem has solution.
Proof. From Assumption 1 there exists the map
hhe { (u, hPp (xx, u²)) xP E R uº E (RR")hD} - R
(6)
that maps (u.: y() ht and the map (x",
V (u, (x.
, u²))
x E R"" uD )RR"))np} 12
that maps (u,, y + fhp (Tphp (u. yf), u²) also exists. Since the pair of maps and hh is pair of state estimator and predictor that is solution to Problem T it is guaranteed that there is solution when n2 2 Y1.
3 PROPOSED METHOD
This section presents the proposed nonlinear system identification approach. Also, nonlinear MPC based on the obtained model is described.
3.1 TRAINING STATE ESTIMATOR & PREDICTOR
In the proposed method, we train the neural network which has the structure of figure where the state estimator (RR")) by x (IR")) " + R8 maps the past input and output to the current state and the predictor P Rut Rhen, phrxny maps the current state and the future input to the future output.
State Estimator
Current State
Predictor
Past Input Sequence
uf E
Past Output Sequence y!
P
y Future Output Sequence
Future Input Sequence uf
Figure 1: Proposed network structure. The sequences of past inputs u. and outputs yf are once compressed into state variables T,, and then reflected the prediction the sequence of future outputs y!.
We set ni. hh for the bottleneck structure of the network to take a dower-dimensional state, and the training problem formulated as the following minimization problem
T
min E.P T t=1
vi
P(E u u:)
(8)
prediction of o/f
Since the state I, cannot measured, the state estimator E and predictor are coupled in series and trained in the same way the encoder and decoder in autoencoder. To solve (8). utilize modern batch optimization algorithms for deep learning (e.g. Adam by Kingma & Ba 2014).
Here, show that ah F. (uf. y.) is also state of the system in the ideal situation.

---

Under review as a conference paper at ICLR 2022
Theorem 2. Assume the training 88) is perfectly done, that is, for all x IR= up (R"")"r,
u E (Re" )o, the equation
((E 3²) u)) =y',
(9)
where y (x*,, u.,uu, ,u() := hhe (ax, w²). x (x,; uP) == fhv (xx,, u²), y (xP, uD) = n (xP; uP)
and arguments are omitted for conciseness, holds. Then, for all xp R and up e (IR".)
E (u,, hhv (x", u))) is a state equivalent to fhv
Proof. From obseervrvyyylityy of the system, the map
she {(u _ h" (xx (xx. u))) x E R"* ut (R"") R that maps (u',8" +
(10)
exists, and the map
R a x (R" R" Th that maps (á, u') + th (u', (
(11)
also exists. Then, map obtained by fixing the second argument of e.g., e.g.
To(á)) := r (á. Omen.)
(12)
maps & to æ, and proved be a state.
Remark 3. The existence of the map from I is ensured by the predictor P trained with common input Ouen. for all states as in (12). Although this is just one way of constructing a map that is possible under the strong assumption of Theorem this implies that obtain appropriate state estimator, the training data must be rich enough to include situations where common inputs are applied in diverse states.
Corollary 1. When the assumption of Theorem Cholds, the predictor that predicts the sequence of future output from state and sequence of future input.
3.2 MODEL PREDICTIVE CONTROL WITH ESTIMATOR & PREDICTOR
Next, the use the obtained model in MPC is briefly described.
Here we reuse the symbols for the past and future input/output sequences u., y." ui, yf defined in 2). but note that the MPC done after the training and these input-output signals are not the ones at the time of training.
MPC, a loss function 4 : I By + that evaluates the input and output at each time given. And, each time step the optimization problem
Σ leth-1 ((ul) (x (3') (k- 11myt1:kn) k=
minimize wf (((gm)) "
(13)
s.t.
y (E (u, yp)) u²)
(u', vf) satisfies constraints on the control system
is solved to obtain n which is candidate for the future input sequence uf. Then, the candidate for most immediate future input [I.., On,, (hr- u' actually applied to the system and moves to the next step + 1.
Usually, state-space models are used in MPC, and state estimator like extended Kalman filter and predictor are constructed from the models. On the other hand, the state estimator and predictor obtained by the proposed method can be directly used for MPC. The state estimator obtained by the proposed method is static mapping from the previous inputs and outputs on the finite horizon hp. and there is no risk of divergence, unlike commonly used filters with infinite impulse response (IIR) structures. This is important advantage when using the state estimator for MPC.
4 INTERPRETATION OF PROPOSED METHOD AND MODEL
The proposed method can be viewed as a natural extension of the subspace methods for linear systems, and the similarity with the subspace methods gives interpretability to the model obtained by the proposed method. In this section, we briefly explain the framework of the subspace method and describe how the model obtained by the proposed method can be interpreted based it.
4

---

Under review as a conference paper at ICLR 2022
4.1 RELATION TO SUBSPACE METHODS FOR LINEAR DYNAMICAL SYSTEMS
Here, we consider restricting the state estimator E and predictor P of the proposed method to linear maps and representing them by matrices E E [R (h, tny) and P E (
When the target system is linear, the optimal state estimator and predictor are known to be linear maps, and can be represented by these matrices if hp and hy are sufficiently large. In this case, the training of the state estimator and predictor 8 is reduced to the following nonlinear least-squares problem:
T
min BER Khpinutny) P. ER"1"XX0
yi
P
E
W
P
(14)
T
y
P. BuRhtnyy Khymu
UP:= where we split P [Ps. Pu] for convenience. Also, by defining the matrix UP := [up. u². ..... x and the matrices YP, U*, Y in the same manner, we can rewrite the objective function as follows
UP T -P-PPE 1x
min BER"s Kbplmu Ing) P. x. Pa Khymu
P.U
(15)
Y
On the other hand, while the actual computational procedure is cumbersome due to the of RQ decomposition for numerical benefits, the outline of the subspace methods can be summarized as follows (Peternell et al. 1996
(S1) Solve linear least squares problem
1 T
UP Yp
P.U
min P. Bey ing) Pa Eahhnyxkham
Y
P
E
(16)
to obtain P.E. E
R hen (h ) and P., E Rhhny Xhyna
(S2) Obtain Pa and E rank-n; approximation of P, via a 'weighted" SVD. (S3) Calculate state-space representation based on E.
As can be seen from the comparison of the two optimization problems (15) and (16), the subspace method is characterized by the fact that instead of solving the nonlinear least-squares problem in- volving the product of two matrix parameters P. and E, it solves the linear least-squares problem with the product E. E the parameter, and then obtain P. and E via the low-rank approximation in (S2).
The distinct advantage of the procedure (SI)-(S2) is that both linear least-squares and low-rank approximations by SVD yield globally optimal solutions. On the other hand, the solution obtained by (S1)-(S2) is different from the solution of the original optimization problem (15), and there many variants that have different statistical properties for this misalignment (see e.g., Van Overschee
& De Moor 1994 Peternell al. 1996. Favoreel al. 2200)..
Therefore, the proposed method can be regarded as a variant of the subspace methods that revert to the underlying nonlinear least-squares problem (5) and extend nonlinear systems by replacing linear mappings with neural networks.
Appendix B confirms that the proposed method indeed yields the same results as the ideal subspace identification method for linear system example. Next, we discuss the interpretation of the obtained model based on this agreement.
4.2 INTERPRETATION MODEL
It is also possible to interpret the obtained model based on the knowledge in linear system identifica- tion. Only in this section, we assume that the target system the following linear dynamical system with noise
E++1 Ax, + Bu, + Wt:
y/ Ca. + Ut.
(17)
5

---

Under review as a conference paper at ICLR 2022
where A R"xx" B R"*x" C E R" the system matrices; wa N (On.x)) x1 Q) and ~ .V (0,, are process noise and measurement noise, respectively. When the number of states in the model accurate (ng ng) and the number of data 7 and horizons hp hy are sufficiently large, the result of steps (SI) and (S2) is consistent with the solution of (15), and what
we get as a state estimator E is known to be the Kalman filter for the target system. And, the matrix E becomes the Markov parameters of the Kalman filter, that
E Eu
Eyl,
(18)
A E, =
[A
A AB, B
By = [A K,... A AK, K]
(19)
A = A KC,
K = AŠC (CEC + +R)
(20)
where K E XR"x"% " is the Kalman gain and Σ is the solution to the discrete-time algebraic Riccati equation
AEA - A (CECT R CÉA +Q.
(21)
From the Kalman filter designed for the target system, we can retrieve the frequency hands that are important for state estimation the noise characteristics in the target system. Conversely, also evaluate whether the obtained state estimator is reasonable or not, based on the knowledge of the target system. For the state estimator E implemented by neural network, E, and E, correspond at the operating point, respectively. And by calculating these, the characteristics of the obtained state estimator can be evaluated.
8E and aE Sy"
On the other hand, the predictor Pa and Pu are extended observability matrix and the Markov parameters of the target system, that is,
0
0
O
C
CB
0
CA
P. =
CAB
CB
On, Xnn. (22)
- CA
CB O
and characterize the unforced response and input-output characteristics the target system. For nonlinear predictors, Pa and P. correspond to dz and 0P respectively, from which the character- isties of the target system at the operating point can be illustrated.
@P
ILLUSTRATIVE EXAMPLE
Next, demonstrate the proposed method through toy example, where the identification and MPC of a cascaded tanks system (Wigren & Schoukens) 2013) is performed.
The state equation of the system is as follows:
I1+1,1
max (Ie,1 Ai / /It,1 12 (u, + we) 0)]
Et++1
12+1.2
max [1,,2 + ka /1,1 ks W2, 0)
31 x1.2 Uz,
(23)
where fg 2, nh Ny 1, Ki 0.5, k2 0.4, K3 0.2, K4 0.3, w, N (0, 0.35"). and
ve / 00.5.. The state variables J4,1 and 2t,2 are the levels of the upper and the lower tank, and denotes the applied voltage to the pump at t. To obtain the training dataset, the system is excited by a sequence of random input
min (max (u., 0),
oe ~
(24)
and a sequence of data {(4,, y1) y)) 1000019 is obtained.
On training the state estimator E and predictor P, We choose the hyperparameters of ng 2, h, = hy = 20. E and P are implemented by neural networks with two and three hidden layers, respectively, where each layer contains 64 ReLU neurons and every layers are fully connected. For

---

Under review as a conference paper at ICLR 2022
training. used the Adam optimizer in PyTorch (Paszke et al. 2019) with the default settings. To prevent overfitting, the 90 % the samples were used for training, and the training was continued until the fit to the remaining 10 % of the validation data did not improve for 1000 consecutive iterations.
Then, the MPC controller which solves the optimization problem 13 with the cost function
4 [u, 9/) = (y/ rz)² 0.05 (w+ u4-1)*
(25)
and the constraint S U I each time is implemented with the obtained state estimator and predictor. Here, E Rh is the reference signal. To solve the optimization problem, and we used Mevenberg-Marquardd Algorithm (Moré 1978).
The result of the tracking control up to f 400 is shown in figure 2 In the figure, the applied control input ue, the measured output y/. (red line), reference signal Tr (dotted black line), and true water levels in the tanks 1 (blue lines) are shown. In addition, the green lines in each plot show uf and y at t 400, respectively. As can be seen from the figure, appropriate MPC is performed using the obtained state estimator and predictor. Also, the state estimate é; and the state of the system 2t compared in Appendix C
v
mmmmmnmn
10
3611
3
50
100
150
200
250
300
350
380
400
t
Figure 2: Result of tracking control
Then, to support the interpretation discussed in section 4 figure 3 shows the visualization of the estimator and predictor at the operating point slow and nonlinearity is expected to be weak. The dashed lines in ((a)(c) and the bitmap on the right figure 4dd) show the corresponding properties of the linearized system of (23) the operating point. Since linear systems also have degrees of freedom in how they coordinate in state space, a transformation matrix
50, where the change in water level is
Pi dæ R==xn= is used align the coordinate system. From figures 3 a) and (b), the state estimator É obtained by the proposed method agrees well with the Kalman filter for the linearized system of the true system, and can be confirmed that the proposed method yields almost optimal state estimators expected for subspace methods in an ideal situation with sufficient data. It can also be seen that the predictor is consistent with the true system.
Moreover, figure 4 obtained at the operating point at t 380, where the nonlincarity is sidered to be more strongly observed. Although the results in this case are similar to those in the linearized system extent, differences in the characteristics be For example, in

---

Under review as a conference paper at ICLR 2022
-0.2
-0.1
-0.4
-0.2
5
10
15
20
15
20
i
Markov parameters state estimator for up
Predictor unforced response
solid:
2B OuF
dashed: ( -T Eu)..)
ar Solidd:
dashed (P, (P.))..)
BP the
(Puls
0.06
-0.5
0.04
0.02
-1
5
10
20
20 1
20
(b) Markov parameters of state estimator for y
(d) Predictor Markov parameters
solid:
dashed: (Φ Eyy... ($-TEy),.
Oyy :
Figure 3: Visualization of E and P at = 50
0
-0.2
-0.4
-0.2
5
10
15
20
15
20
Markov parameters of state estimator for a
Predictor unforced response
solid::
2F OuF
dashed: (Ø-TBL Em.)
'ollid (02) dashed (P. ))...
0
@P JPO
(Pu)s
0.2
0.06
-0.4
0.04
0.02
-0.6
0
10
20
20
1
20
(b) Markov parameters of state estimator for y
(d) Predictor Markov parameters
solid: dashed (E-TE,)),)
Pyp :.
Figure 4: Visualization E and P at 380
8

---

Under review as a conference paper at ICLR 2022
figure 4(d) the Toeplitz structure of linear systems is partially lost, indicating the existence of non- linearity. In figures a) and (b), can be seen that an estimator that is less dependent on the output and more focused on the recent imput than the Kalman filter for the linearized system is obtained.
To verify the validity of placing the bottleneck, we evaluated the performance with different widths of the bottleneck na- All settings except ng and the realized value of the stochastic noise Wa, et set to be identical to the case of nhg and 60 trials were conducted with different realizations of the stochastic noise.
The results are summarized in figure The loss for the validation dataset in figure 5 a) shows that
Bottleneck width ma
64
32
-0
16
+I+
+
+
+
-I-+
+
0.028
0.03
0.032
0.034
(a) Loss for Validation Set
(b) Cost in MPC
Figure 5: Width of bottleneck ny and performance
the performance does not degrade until the width of the bottleneck goes below the actual state dimension n 2. Also, the performance of the MPC calculated by Lt=h,+1 4e(u,, F1,2) does not show degradation until nhg < nhr shown in figure 5ib),, and these results support the of the bottleneck to provide interpretability.
CONCLUSION
In this paper, we proposed nonlinear subspace identification method that produces state esti- mator and predictor directly usable for nonlinear MPC. theoretically shown that by training a neural network with bottleneck that performs regression input-output sequences of length that provides observability in both past and future directions, a state estimator and a predictor are formed across the bottleneck layer. The relationship between this method and the subspace identification method, as well as the interpretation of the obtained model using this relationship, is also explained, and its use is demonstrated through an example.
This research is still in its early stages, and there is much work to be done in the future. In order to demonstrate the applicability to nigher-dimeesional problems, an example of identifying dynamics from simple video is included in Appendix [ However, verifying the performance in more prac- tical tasks is a subject for future work. Also, more precise condition for yielding a state estimator is desired. The current proof makes use of the assumption that zero-input responses from all states have been learned, and while the fact that zero-input responses constitute states often used in this type of research (Verdult & Scherpen] 22004), more precise conditions would be useful for experi- ment design. It also appears that the technique of inverse optimal control (Ab Azar et al.) (020)) can be applied to the interpretation of the obtained state estimator. Finally, since noise with high-order dynamics is ubiquitous (Bak et al. 1987) and the synthesis of state estimator for such a system is prone to failure, it important to develop mechanism to focus useful states.
REFERENCES
Nematollah Ab Azar, Aref Shahmansoorian, and Mohsen Davoudi. From inverse optimal control to inverse reinforcement learning: A historical review. Anmual Reviews in Control. 50: 1e-1119-18. 2020. ISSN 1367-5788. doi: http://///...11111..roooooooooo
URL https:
7/ 7/www. www.sciieencc. rect. www.ct..oosscciiieeeeooo htw.://cw....ccc.ooo..ceaaaaaaaa

---

Under review as a conference paper at ICLR 2022
Per Bak, Chao Tang, and Kurt Wiesenfeld. Self-organized criticality: An explanation of the 1/f noise. Phys. Rev. Len. 59:381 384, Jul 1987. doi: 10.100//PhyyRevectttt...... URL https:
7/link. aps.apps... otg. doi/10 103 wtt.s/yyseseeettttttt.oomo 388
Gerben I. Beintema, Roland Toth, and Maarten Schoukens. Non-liness state-space model iden- tification from video data using deep encoders. IFAC-PupersOnLLine; 54(7))697-701, 2021. ISSN 2405-8963. doi: hups://doi.org//10 1.1000000010)... ifacol/...cclll22220.... htt.a/ooo.22222.0c sciencedirect. com. www.///.....aae.. www./ceceecc/eee//e//ee articceeeee http///////.///////22222212 Symposium on System Identification SYSID 2021.
URL https: //ww:w
19th IFAC
Wouter Favoreel, Bart De Moor, and Peter Van Overschee, Subspace state space system iden- tification for industrial processes. Journal of Process Control, 10(2):149-155. 2000. ISSN 0959-1524. doi: https://oii.or/// 10..:00.000/001111..1.... 101111111///11111110 1529999900000099
URL https: //ww:.
sciencedirect www.com/ccc..oo sci wwwpeeeeneeeeiiieeeeeeeeeeeeeeee httpp/ii///9999122 http////a...../0.1155/2900000000
Marco Fraccaro, Simon Kamronn, UIrich Paquet, and Ole Winther. recognition and nonlinear dynamics model for unsupervised learning. In I. Guyon, U. V. Luxburg, S. Bengio, Н. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett (eds.), Advances in Neural Information Processing Systems, volume 30. Curran Asso- ciates, Inc., 2017. URL
disentangled
Tb7a53e239400a1 htt.e900oaaaaaeeeeeeeeecccee--ae pdf
I. Goethals, K. Pelckmans, L.. Hoegaerts, J. Suykens, and B. De Moor. Subspace intersection identi- fication of hammerstein-wieee systems. Proceedings the 44th IEEE Conference on Decision and Control, pp. 7108-7113, 2005. doi: 10. 10. 11110//DC..00..5553330
Daniel Hsu, Sham M. Kakade, and Tong Zhang. A spectral algorithm for learning hidden markov models. Journal of Computer and System Sciences, 1865):1460-1480) 2012. ISSN 0022-0000. doi: http://orrr/...0000///ssss....1. comm sccene/e/rrccee/////00000022022 puting 2011.
URL https:/ /wwp://wwww././../../////////w.. sci wwweeiieeeeeeee
JCSS Special Issue: Cloud Com-
Kirthevasan Kandasamy, Maruan Al-Shedivat, and Eric Xing. Learning hmms with nonpara- metric emissions via spectral decompositions of continuous matrices. In D. Lee, M. Sugiyama, U. Luxburg, I. Guyon, and R. Garnett (eds.), Advances in Neurul Information Processing Systems, volume 29. Curran Associates, Inc., 2016. URL https: http::::://///cceeee/e/e/ nwwrips....... ec/ pd
Petros Karamanakos, Eyke Liegmann, Tobias Geyer, and Ralph Kennel. Model predictive control of power electronic systems: Methods, results, and challenges. IEEE Open Journal of Industry Applications, 1:95-114, 2020. doi: 10.1 1009//JIA.2200.32200
Tohru Katayama. Subspuce methods for system identification. Springer, edition, 2005.
Tohru Katayama and Hajime Ase. Linear approximation and identification of mimo
wiener-hammerstein systems. Automatica, 71:118-124, 2016. ISSN 0005-1098. doi: https: //bb:.///00.000////autttttaaaao0 sssinne/aariccee//////0011111181
URL https: http://wwww..ceeeeeeeee.... com/
Yoshinobu Kawahara, Takehisa Yairi, and Kazuo Machida. A kernel subspace method by stochastic realization for learning nonlinear dynamical systems. In B. Schölkopf, J. Platt, and T. Hoffman (eds.), Advances in Neurul Information Processing Systems, volume MIT Press, 2007. URL https:://proceedinng htw.rooeeeednnn............... 96629f1 1alees/faaaedddd27777a2222353333 papee .pdf
cc//pappr//006//f1ee
Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint 1412 6980, 2014.
Lennart Ljung, Carl Andersson, Koen Tiels, and Thomas B. Schön. Deep learning and sys- tem identification. IFACC 13(2):115-11181 wcwencedirect.. com/
2020a. ISSN 2405-8963, doi: http:://ooogg///llloliiiaoooooon URL URR.E 21st IFAC World Congress.
10

---

Under review as a conference paper at ICLR 2022
Lennart Ljung. Tianshi Chen, and Biqiang Mu. A shift in paradigm for system identification. Inter- national Journal of Control, 93(2):173-1880 180, 2020b. doi: 10. 00.10000000011192219999187888889 URL 1080 001.000000000001...2000000
sttps:///ooi
Daniele Masti and Alberto Bemporad. learning nonlinear state-space models using deep autoen- coders. In Proc. of the 57th IEEE Conference on Decision and Control (CDC 2018), pp. 3862- 3867, 2018. doi: (0.1099CCDC2200888811977
P.E. Moraal and J.W. Grizzle. Observer design for nonlinear systems with discrete-time measure- ments. IEEE Transactions on Automatic Control, 40(3):395-404, 1995. doi: 10.1109/..36051.
Jorge J Moré. The Levenberg- analysis, pp. 105-116 116. Springer, 1978.
Levenberg-Marquarde Marquardt algorithm: implementation and theory. In Numerical
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf, Edward Yang, Zachary DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. PyTorch: An imperative style, nigh-performance deep learning library. In H. Wallach, H. Larochelle, A. Beygelzimer, F. d' d'Allcé-Bie Buc, E. Fox, and R. Garnett (eds.), Advances Neural Information Processing Systems 32, pp. 8024 8035. Curran Associates, Inc., 2019.
Klaus Peternell, Wolfgang Scherrer, and Manfred Deistler. Statistical analysis of novel subspace identification methods. Signal Processing. 52(2):161-177, 1996.
S. Joe Qin and Thomas A. Badgwell. An overview of nonlinear model predictive control applica- tions. In Frank Allgöwer and Alex Zheng (eds.), Nonlinear Model Predictive Control, volume 26 of Progress in Systems and Control Theory, pp. 369-392. Birkhäuser Basel, 2000. ISBN 978-3- 0348-8407-5.
Syama Sundar Rangapuram, Matthias Seeger, Jan Gasthaus, Lorenzo Stella, Yuyang Wang, and Tim Januschowski. Deep state space models for time series forecasting. In S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Gar- nett (eds.), Advances in Neural Information Processing Systems, volume 31. Curran Asso- ciates, Inc., 2018. URL https: Boff0933fo777466020000deeeeeeee
http:///o....aa.aoooo/o////////p cc/pppppr/211//11eeee. pdf
Stark, D.S. Broomhead, M.E. Davies, and J. Huke. Takens embedding theorems for forced and stochastic systems. Nonlinear Analysis: Theory, Methods & Applications,, 30 (8):5303-5314, 1997. ISSN 0362-546X. doi: htpp::/0o..arg//11066666666226 00149-6. URL https: ttmpp::/wwwzr www.wweeeeeiiieeeeoooo.oo com http://ctt.c.coocccce//////aaaaa
50362546X96001488 Proceedings the Second World Congress Nonlinear Analysts.
Gijs van der Veen, Jan- Willen van Wingerden, Marco Bergamasco, Marco Lovera, and Michel Verhaegen. Closed-loop subspace identification methods: an overview. IET Control Theory & Applications, 7(10):133911353 2013. doi: https:://oi http:o/ooooo...00///ett......... 0653. URL 1049/iet 2012. 0653
onlinelibrary. www.aa...wwlll.................e wtwpraa..wlllyy.cooo////////////
Peter Van Overschee and Bart De Moor. N4SID: Subspace algorithms for the identification of combined deterministic-stochae systems. Automatica, 30(1):75-93, 1994.
Sergio Vazquez, Jose Leon, Leopoldo G. Franquelo, Jose Rodriguez, Hector A. Young, Abraham Marquez, and Pericle Zanchetta. Model predictive control: A review of its applications in power electronics. IEEE Industrial Electronics Magazine, 8(1):16-31, 2014. doi: 10.1 1091109/ME.2013. 2290138.
Vincent Verdult and Jacquelien Scherpen. Identification of nonlinear stace systems using zero- input responses. Proc. of the 16th international symposium on mathematical theory of networks and systems (MTNS 2004), pp. 1-11, 2004.
11

---

Under review as a conference paper at ICLR 2022
Michel Verhaegen and David Westwick. Identifying mimo hammerstein systems in the context of subspace model identification methods. IFAC Proceedings Volumes, 29(1):4092 4097, 1996. ISSN 1474-6670. doi: http:///eeaaeeeeaaaaacc///////// rect cwm/sc
URL https:://... //www. 13th World
Congress of IFAC, 1996, San Francisco USA, 30 June 5 July.
Torbjörn Wigren and Johan Schoukens, Three free data sets for development and benchmarking in nonlinear system identification. In Proc. of 2013 European Control Conference (ECC), pp- 2933-2938. IEEE, 2013.
A IDENTIFIATIIO OF f AND h IN STATE-SPACE EQUATION
method of obtaining the state-space model from the state estimator and predictor is briefly ex- plained here.
The model of the output map h 2t + Yt, the map h R + R is obtained by extracting the first prediction of the predictor P as
17,, h()) = [ny On, (b; (b;-1)n,] P ((x O).
(26)
As for the model of the state-transition map f RR x Rnn + R by estimating the state sequence { J with E
f = E (u. yf),
(27)
the construction of f reduced to the fitting of static map
T
minimiie
ãi+1 - (zc, uz)
T
B VERIFICATION OF AGREEMENT WITH SUBSPACE METHODS FOR LINEAR SYSTEMS
In order validate the discussion in section system 17) with the following parameters:
we apply the proposed method the linear dynamical
0.7859
0.7358
A
B
C [0.3403 0.4834].
(28)
0.5
0
0.12
0
Q
R
0.1.2
(29)
0
0
The learning setup is the same as in section 5
except that the state estimator E' and predictor P linear maps which can be represented by matrices E and P, and the distribution of the input is Wdg ~ NN,, 1).
In figure 6 the E and P representing the obtained model are compared with the and P calculated by (1)-22 22). i.c., the results obtained by the ideal subspace method. Since the linear state-space systems have a degree of freedom in how to take the coordinate system, linear coordinate transfor- mation (such that P. is most consistent) is applied to the estimated model to facilitate comparison. As can be seen from the figure, the results are consistent except for a small deviation due to the finite sample size, indicating that the proposed method solves essentially the same problem as the subspace identification method for linear systems.
C COMPARISON OF STATE ESTIMATE é, AND ACTUAL STATE D3
We show the trajectories of the actual state and the state estimate of the cascaded tanks example in figure 7 Since the proposed method does not specify how to take the coordinates into the state space, they connected by nonlinear mapping that depends the initial weights of the neural network before training. For case comparison, the trajectory of &; shown over the trajectory of æ, (dotted line) in affine coordinates such that is closest to æ; in figure 7
12

---

Under review as a conference paper at ICLR 2022
0.5
0.5
-0.5
5
10
15
20
1
10
15
20
j
(a) Markov parameters of state estimator for input (Eu)..,
(c) Predictor unforced response (Pa)..:
(solid: estimate, dashed: idcal)
(solid: estimate, dashed: ideal)
Estimate
Ideal
0.4
0.1
0.2
20
20
5
10
15
20
(d) Predictor Markov parameters
(b) Markov parameters of state estimator for (Ey).. (solid: estimate, dashed: ideal)
Figure 6: Visualization of E and P for linear case
3
2
21.
52
0-
100
28.1
-1
-1
0
2
4
6
10
F1,1
(a) Trajectory of actual state æ.
(b) Trajectory of state estimate
Figure 7: Comparison of actual state and model state
D SUPPLEMENTARY EXAMPLE
To demonstrate the applicability high dimensional systems, the proposed method is applied to the problem of estimating the dynamics of video discussed in (Beintema et al.) (02T). Consider an environment where nonlinear force acts ball and we observe with camera 25 25 pixels. The dynamics of the ball is
1
P2 8
rp + kuz
(30a)
21-Pa)²
1
By 8
Yy. + kuy
(30b)
12
(1
pT)
where 1/200, 7 = 0.79, 1: = 1/4, and (pz. Py) the position of the ball, which is not measurable. (uz, uy) the external force whose sample value at time are given u, This system
13

---

Under review as a conference paper at ICLR 2022
has stable equilibrium point (pz. Pv) (0.5, 0.5) and simulate this system from this point with zero velocity. For observation, the pixel intensity at a position X, Y the video is calculated by nonlinear equation
pixel(X,,) max 0. (X x-P=)) (Y Py)2
+ +
(31)
y2
where r = 0.25 and v ~ W(0,0.204* is noise term. The video image is sampled from this continuous image with 25 25 grid from (0,0) (1, This system has 2(= nu)) inputs, 4(= nz) states, and 25 x 25 = (2 nyy) outputs.
We discretize this system by forward difference with the time step At 0.3 and simulate with random external force (Uz, Uy) uniformly distributed in [-1, to generate video data. On training, choose the hyperparameters of îg hp he = 20. E, P and f have three, three, and two layers respectively, where each layer contains 64 ReLU neurons, the images are flattened into vectors with 625 elements, and all layers are fully connected. The training dataset consists of sequence {u,, ) 100019 and other settings are the same as in section 5
In the following, describe the validation of the obtained model and its application to denoising. The data used in the following are obtained under the same settings as in training, except that the realizations of random input and noise different.
D.1 MODELING OF DYNAMICS
To verify that the resulting model correctly models the dynamics of the video, compare the video from the open-loop simulation of the obtained model with the video from the target system for the same input (external force).
Here, the output image from the model ÿ is computed recursively without using as
[ ue
(hr 1): (0.))r-)))
ÿt
Iny
(ng
1)]
Fl IP),
(32)
where of is a sequence of past model outputs defined as in (2) Note that in this test, the ball is stopped at the center (0.5, 0.5) at 1, and ÿt for state.
.p,...,. are set as the image at this
Figure [ shows the actual images without noise (above) and images from simulation with the model (below). As shown in the figure, the images obtained from the model simulation close to the
50
100
f 150
d 200
250
300
350
100
Figure 8: Validation of model dynamics (above: images from target system without noise, below: images from constructed model)
actual image without noise, indicating that the dynamics correctly learned. A video version of figure is provided in the supplementary material as actual -s simulated. mp4.
D.2 NOISE REDUCTION
Next, we show that the model obtained by the proposed method can be used for denoising the video. Here the denoised image ÿt is computed from the images yP and inputs uf of the past hp frames
14

---

Under review as a conference paper at ICLR 2022
and the current imput u,
ÿr
[n,
(br 1>] P Elu yp),
Ut On (nn((h--
(33)
Note that unlike the open-loop simulation 32 the state estimator E uses noisy output y instead of recursively using o!.
Figure shows the result of denoising, The bottom images are the denoised images obtained from the trained model, and the middle images are the images obtained from the actual system yt. From the figure, it be confirmed that the noise is removed by the model obtained by
1 50
100
1 150
300
350
400
Figure 9: (Top) noiseless images, (Middle) noisy images, (Bottom) denoised images
the proposed method. A video version of figure 9 is provided in the supplementary material as eetuallnnisy-denae mp4.
15


Reviews:
[{"review_id": "sQQmxXpYxUp", "reviewer": "Reviewer_dZQd", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "This paper proposes a non-linear extension of subspace identification and model predictive control. The idea of estimating the latent state vector $\\hat{\\boldsymbol{x}}_t$ using the bottleneck structure of estimator and predictor networks seems reasonable. The result of the experiment on a toy problem shows the proposed method works.\n\nI have several comments and questions as follows.\n\n1. While this study focuses on the system identification of non-linear systems based on the subspace identification, other recent works on learning non-linear dynamical systems using deep neural networks such as KVAE and Deep State Space Models should be referred as related works.\n\n2. The authors conducted a case study on cascaded tanks system which is a simple low-dimensional non-linear system. It is illustrative, but I think they should start with some linear systems and linear networks and show that the obtained result is equivalent to that of ordinary subspace identification and MPC. That would justify their claim in a straightforward manner.\n\n3. In the training process, the system is provided with random input sequence in Eq.(24). I wonder if it is possible in general. What if the system becomes unstable due to some inadequate inputs ? Is there a way of \"safe\" training ?\n\n4. In the case study, Levenberg-Marquardt algorithm is used solve the optimization in MPC. Is LM applicable to other systems in general ?\n\n5. In Figure 2, estimated state vector $\\hat{\\boldsymbol{x}}_t$ should be also shown and compared with actual state $\\boldsymbol{x}_t$. I am curious how $\\hat{\\boldsymbol{x}}_t$ looks like and different from $\\boldsymbol{x}_t$ due to the non-linear mapping.\n\n6. Is the proposed method applicable to more high-dimensional non-linear systems ?  For example, can it perform system identification and MPC of a non-linear system using a sequence of images as observation ?\n\n\n", "summary_of_the_paper": "This paper proposes a new system identification and model predictive control method based on neural networks for non-linear dynamical systems. The basic idea behind the proposed method comes from subspace identification for linear systems. In this sense, the proposed method can be regarded as a non-linear extension of subspace identification by deep neural networks.", "summary_of_the_review": "This paper proposes a non-linear extension of subspace identification and model predictive control. The idea is interesting and theoretically sound. I like this study. The case study using a simple non-linear system is not convincing enough. ", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "c5-vQpUHOn", "reviewer": "Reviewer_pfti", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths:\nI find the interpretation for the bottleneck layer interesting. I also find the connection to the subspace identification for linear systems interesting and insightful.\n\nWeaknesses:\nI am not convinced, based on what is provided, how efficient the methodology is. Maybe some convergence characterization for a class of simpler nonlinear dynamical systems would help illustrating the efficiency. The paper is not technically deep and in terms of dynamical systems not coherent. The example is only two dimensional and relatively simple.\n", "summary_of_the_paper": "The authors introduce a system identification methodology catered towards the model predictive approach. They claim that the main contribution is that the neural network, which is used, has a bottleneck layer and represents input-output dynamics. They also provide a connection to the subspace state-space system identification method for linear dynamical systems. ", "summary_of_the_review": "Overall, the paper has some interesting results yet lacks technical and experimental arguments to justify the benefits of the approach.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "NHdG8pzNJJG", "reviewer": "Reviewer_Sng6", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The idea of using a DNN to approximate state observers and predictors for general nonlinear systems is re-examined here.\nThe idea is not novel, since it has been used in Masti et al, CDC 2018. The  discussion added here about the linear case is trivial, and hardly counts as a theoretical analysis, as the authors are claiming. As for 'Theorems' 1, 2, they are trivial and cannot be claimed as theorem. I mean the statement ' if the system is observable, then an observer exists', is hardly a Theorem !\n\nI keep wondering how a dynamical system, i.e., and observer and predictor, can be approximated by a static mapping ? ! \nIn other words, can you prove that your observer converges asymptotically (over time beyond your training time span) to the true state, for a set of initial conditions ? this is a very basic property of any observer, in the sense on control theory, and if you do not have this 'robustness' w.r.t. state initial conditions, then this is merely and open loop identification problem, or model reduction, or give it any name you want, but do not call it state estimation problem.\n\nI think this is even clear from your toy example, where there are no tests showing that if you start the simulation from several different initial conditions for the states, you still converge to the true state trajectories.", "summary_of_the_paper": "The authors are proposing to use DNN to approximate observers and predictors for nonlinear systems. No convergence analysis of the 'said' observers are given. The empirical results are rather weak.", "summary_of_the_review": "It is not technically correct to assume obtaining an observer, i.e., a dynamical system, from a static DNN mapping. The technical point of robustness w.r.t. changes in the state initial conditions is at the core of observers theory and cannot be ignored. \nTheoretically the paper empty, and the empirical validation is too simple.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "1: The main claims of the paper are incorrect or not at all supported by theory or empirical results.", "technical_novelty_and_significance": "1: The contributions are neither significant nor novel.", "empirical_novelty_and_significance": "1: The contributions are neither significant nor novel."}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper uses neural networks for system identification.  The novelty of its contributions seems to be marginal, and the demonstration of its usefulness is not experimentally validated well enough.
Decision: Reject