S
UBSPACE
S
TATE -S
PACE
I DENTIFICATION
AND
M ODEL P REDICTIVE C ONTROL OF N ONLINEAR D YNAMICAL S YSTEMS U SING D EEP N EURAL N ETWORK WITH B OTTLENECK
Anonymous authors
Paper under double-blind review
A
BSTRACT
A novel nonlinear system identiﬁcation method that produces state estimator and predictor directly usable for model predictive control (MPC) is proposed in this paper.
The main feature of the proposed method is that it uses a neural network with a bottleneck layer between the state estimator and predictor to represent the input-output dynamics, and it is proven that the state of the dynamical system can be extracted from the bottleneck layer based on the observability of the target system.
The training of the network is shown to be a natural nonlinear extension of the subspace state-space system identiﬁcation method established for linear dynamical systems.
This correspondence gives interpretability to the resulting model based on linear control theory.
The usefulness of the proposed method and the interpretability of the model are demonstrated through an illustrative example of MPC.
I NTRODUCTION
Model predictive control (MPC) has been widely used as a practical method to control nonlinear dynamical systems (Qin & Badgwell, 2000;
Vazquez et al., 2014; Karamanakos et al., 2020).
As the name implies, MPC requires a prediction model, and in most cases, state-space models are used.
When we want to apply MPC to a complex system whose structure and state are unknown, model construction from input-output data, i.e., system identiﬁcation, is required.
For linear state-space system identiﬁcation, the subspace methods (Favoreel et al., 2000; Katayama, 2005; van der Veen et al., 2013) are widely used.
Its fundamental idea is to reduce the dimensionality of the measured past and future input and output data, and it strongly depends on the linearity of the system.
Although there have been continuous attempts to extend the subspace methods to nonlinear systems with speciﬁc structures such as Wiener and Hammerstein structures (e.g., Verhaegen & Westwick, 1996;
Goethals et al., 2005; Katayama & Ase, 2016) and related research has also been conducted in the ﬁeld of machine learning as spectral learning of Hidden Markov Models (Kawahara et al., 2007;
Hsu et al., 2012; Kandasamy et al., 2016), there is still a lot of room for research in extending the subspace method to general nonlinear dynamical systems.
On the other hand, recent advances in machine learning techniques have had an impact on the research ﬁeld of system identiﬁcation (Ljung et al., 2020a;b).
For example, deep state space models (Rangapuram et al., 2018), and Kalman variational auto-encoder (Fraccaro et al., 2017) apply deep learning to state space models.
Among those technologies, autoencoder is a notable dimensional reduction or feature extraction technique with deep neural networks, whose traits derives from topological constraints of own bottleneck structure.
Although several pieces of research apply it to state-space system identiﬁcation (Masti & Bemporad, 2018; Beintema et al., 2021), these works do not provide a theoretical basis for obtaining the state sequence.
This paper aims to propose a theoretically supported and interpretable nonlinear dynamical system identiﬁcation method by simply extending the subspace method with neural networks.
For the proposed method, it can be shown based on the observability of the target system that when a neural network with an appropriate bottleneck is trained, the state estimator and predictor are formed across the bottleneck layer.
The obtained model can be interpreted based on the knowledge from linear system identiﬁcation methods, and this aspect is explained through an illustrative example.
In this paper, the set of integers and the natural numbers are denoted by Z and N , respectively.
The set { s a , s a +1 , . . .
, s b − 1 , s b } is denoted as { s k } bk = a for conciseness.
The n × m zero matrix is denoted as 0 n × m , and 0 n := 0 n × 1 .
For a matrix M , M † is the Moore–
Penrose inverse of M .
P ROBLEM F ORMULATION & N OTATION
We consider the following discrete-time nonlinear dynamical system:
[Equation]
with t ∈ Z the time index, x t ∈ R n x the state, y t ∈ R n y the output, u t ∈ R n u the input, f : R n x × R n u → R n x the state-transition map and h : R n x → R n y the output map.
As in the usual setting for system identiﬁcation, x t is a latent variable and u t and y t are measurable with noise.
Although the noise is not considered explicitly in the theoretical discussion, its effect will be discussed in a later example.
For convenience, We deﬁne the sequence of input and output signals in the past and future from a certain time t as follows:
[Equation]
where the superscripts p and f indicate past and future, and h p ∈ N and h f ∈ N are horizons for corresponding directions, respectively.
Using these notations, we formulate the problem as follows.
Problem 1.
Given the measured input-output data { ( u t , y t ) } T +h f − 1 t = − h p +1 , and the design parameters h p , h f , n ˆ x ∈ N , construct a model that consists of a state estimator E , which maps ( u p t , y p t )
(cid:55)→ ˆ x t , and a predictor P , which maps (ˆ x t , u f t ) (cid:55)→ y f t .
Here, ˆ x t ∈ R n ˆ x is a state equivalent to x t but in an arbitrary coordinate system and T ∈ N indicates the size of the dataset.
Remark 1.
In the usual state-space system identiﬁcation, the goal is to construct a model of f and h , i.e., a state-space model.
However, in many applications, including model predictive control, the state estimator and multi-step ahead predictor are often reconstructed from the resulting state-space model.
In particular, for nonlinear systems that are difﬁcult to analyze, the merits of going through state-space models are questionable.
Therefore, we focus here on the construction of the state estimator E and predictor P and only brieﬂy discuss the identiﬁcation of f and h in Appendix A.
To guarantee Problem 1 to be solvable, we make some assumptions.
With the vectors deﬁned above (2), we can write the system (1) as
[Equation]
where
[Equation]
And deﬁne the uniform k -observability as follows.
Deﬁnition 1 (uniform k -observability (Moraal & Grizzle, 1995)) .
If the mapping
[Equation]
is injective, the system (1) is said to be uniformly k -observable
(Moraal & Grizzle, 1995).
Then, we make the following assumptions and theorem.
Assumption 1.
The system (1) is uniformly k -observable, where k = min (h p , h f ) .
Remark 2.
According to Stark et al. (1997), typical dynamical systems are k -observable for k ≥ 2 n x + 1 .
Assumption 2.
The dimension of the state estimate is large enough, that is, n ˆ x ≥ n x .
Theorem 1.
Under Assumptions 1 and 2, Problem 1 has a solution.
Proof.
From Assumption 1, there exists the map
[Equation]
that maps ( u p t , y p t )
(cid:55)→ x t − h p , and the map Ψ
[Equation]
p
[Equation]
p
[Equation] [Equation]
that maps ( u p t , y p t ) (cid:55)→ f h p (cid:0) Φ h p ( u p t , y p t ) , u p t (cid:1) also exists.
Since the pair of maps Ψ and h h f is a pair of state estimator and predictor that is a solution to Problem 1, it is guaranteed that there is a solution when n ˆ x ≥ n x .
P ROPOSED M ETHOD
This section presents the proposed nonlinear system identiﬁcation approach.
Also, nonlinear MPC based on the obtained model is described.
3. T RAINING S TATE E STIMATOR & P REDICTOR
In the proposed method, we train the neural network which has the structure of ﬁgure 1, where the state estimator E : ( [Equation]
x maps the past input and output to the current state and the predictor P : R n ˆ x × R h f n u
→
R h f × n y maps the current state and the future input to the future output.
[Figure]
We set n ˆ x < h p n y for the bottleneck structure of the network to take a lower-dimensional state, and the training problem is formulated as the following minimization problem
[Equation] [Equation]
Since the state x t cannot be measured, the state estimator E and predictor P are coupled in series and trained in the same way as the encoder and decoder in autoencoder.
To solve (8), we can utilize modern batch optimization algorithms for deep learning (e.g. Adam by Kingma & Ba, 2014).
Here, we show that ˆ x t := E ( u p t , y p t ) is also a state of the system in the ideal situation.
Theorem 2.
Assume the training (8) is perfectly done, that is, for all x p ∈ R n x , u p ∈ ( R n u ) h p , u f ∈ ( R n u ) h f , the equation
[Equation]
where y f (cid:0) x p , u p , u f (cid:1) :
=
h h f (cid:0) x , u f (cid:1) , x ( x p , u p ) :
= f h p ( x p , u p ) , y p ( x p , u p ) :
=
h h p ( x p , u p ) and arguments are omitted for conciseness, holds.
Then, for all x p ∈ R n x and u p ∈ ( R n u ) h p ,
E (cid:0) u p , h h p ( x p , u p ) (cid:1) is a state equivalent to f h p ( x p , u p ) .
Proof.
From h f -observability of the system, the map
[Equation]
exists, and the map
[Equation]
also exists.
Then, a map obtained by ﬁxing the second argument of Γ , e.g.,
[Equation]
maps ˆ x to x , and ˆ x is proved to be a state.
Remark 3.
The existence of the map from ˆ x to x is ensured by the predictor P trained with a common input
h f n u for all states as in (12).
Although this is just one way of constructing a map that is possible under the strong assumption of Theorem 2, this implies that to obtain an appropriate state estimator, the training data must be rich enough to include situations where common inputs are applied in diverse states.
Corollary 1.
When the assumption of Theorem 2 holds, P is the predictor that predicts the sequence of future output from a state and a sequence of future input.
3. M ODEL P REDICTIVE C ONTROL WITH E STIMATOR & P REDICTOR
Next, the use of the obtained model in MPC is brieﬂy described.
Here we reuse the symbols for the past and future input/output sequences u p t , y p t , u f t , y f t deﬁned in (2), but note that the MPC is done after the training and these input-output signals are not the ones at the time of training.
In MPC, a loss function l t : R n u × R n y
→ R that evaluates the input and output at each time t is given.
And, at each time step t , the optimization problem
[Equation] [Equation]
constraints on the control system is solved to obtain u f , which is a candidate for the future input sequence u f t .
Then, the candidate for most immediate future input (cid:2)
I
[Equation] [Equation]
u f is actually applied to the system and moves to the next step t + 1 .
Usually, state-space models are used in MPC, and a state estimator like extended Kalman ﬁlter and a predictor are constructed from the models.
On the other hand, the state estimator and predictor obtained by the proposed method can be directly used for MPC.
The state estimator obtained by the proposed method is a static mapping from the previous inputs and outputs on the ﬁnite horizon h p , and there is no risk of divergence, unlike commonly used ﬁlters with inﬁnite impulse response (IIR) structures.
This is an important advantage when using the state estimator for MPC.
I NTERPRETATION OF P ROPOSED M ETHOD AND M ODEL
The proposed method can be viewed as a natural extension of the subspace methods for linear systems, and the similarity with the subspace methods gives interpretability to the model obtained by the proposed method.
In this section, we brieﬂy explain the framework of the subspace method and describe how the model obtained by the proposed method can be interpreted based on it.
4. R ELATION TO S UBSPACE M ETHODS FOR L INEAR D YNAMICAL S YSTEMS
Here, we consider restricting the state estimator E and predictor P of the proposed method to linear maps and representing them by matrices E ∈ R n ˆ x × h p ( n u + n y ) and P ∈ R h f n y × ( n x +h f n u ) .
When the target system is linear, the optimal state estimator and predictor are known to be linear maps, and can be represented by these matrices if h p and h f are sufﬁciently large.
In this case, the training of the state estimator and predictor (8) is reduced to the following nonlinear least-squares problem:
[Equation]
where we split P
=
[ P x , P u ] for convenience.
Also, by deﬁning the matrix U p :=
[ u p1 , u p2 , . . .
, u p T ] and the matrices Y p , U f , Y f in the same manner, we can rewrite the objective function as follows
[Equation]
On the other hand, while the actual computational procedure is more cumbersome due to the use of
RQ decomposition for numerical beneﬁts, the outline of the subspace methods can be summarized as follows (Peternell et al., 1996):
(S1)
Solve linear least squares problem
[Equation]
to
obtain
(cid:91) P x E ∈ R h
f n y × h p ( n u + n y ) and
P
u ∈ R h f n y ×
h
f n u
[Equation]
(S2)
Obtain P x and E rank-
n ˆ x approximation of (cid:91) P x E via a ‘weighted’ SVD.
(S3)
Calculate state-space representation based on E .
As can be seen from the comparison of the two optimization problems (15) and (16), the subspace method is characterized by the fact that instead of solving the nonlinear least-squares problem involving the product of two matrix parameters P x and E , it solves the linear least-squares problem with the product (cid:91) P x E as the parameter, and then obtain P x and E via the low-rank approximation in (S2).
The distinct advantage of the procedure (S1)-(S2) is that both linear least-squares and low-rank approximations by SVD yield globally optimal solutions.
On the other hand, the solution obtained by (S1)-(S2) is different from the solution of the original optimization problem (15), and there are many variants that have different statistical properties for this misalignment (see e.g., Van Overschee & De Moor, 1994;
Peternell et al., 1996; Favoreel et al., 2000).
Therefore, the proposed method can be regarded as a variant of the subspace methods that revert to the underlying nonlinear least-squares problem (15) and extend it to nonlinear systems by replacing linear mappings with neural networks. Appendix B conﬁrms that the proposed method indeed yields the same results as the ideal subspace identiﬁcation method for a linear system example. Next, we discuss the interpretation of the obtained model based on this agreement.
4. I NTERPRETATION OF M ODEL
It is also possible to interpret the obtained model based on the knowledge in linear system identiﬁcation.
Only in this section, we assume that the target system is the following linear dynamical system with noise
[Equation]
where A ∈ R n x × n x , B ∈ R n x × n u , C ∈ R n y × n x are the system matrices; w t ∼ N ( 0 n u × 1 , Q ) and v t ∼ N (cid:0) 0 n y × 1 , R (cid:1) are process noise and measurement noise, respectively.
When the number of states in the model is accurate ( n ˆ x = n x ) and the number of data T and horizons h p , h f are sufﬁciently large, the result of steps (S1) and (S2) is consistent with the solution of (15), and what we get as a state estimator E is known to be the Kalman ﬁlter for the target system.
And, the matrix
E becomes the Markov parameters of the Kalman ﬁlter, that is,
[Equation]
where K ∈ R n x × n y is the Kalman gain and ˆ Σ is the solution to the discrete-time algebraic Riccati equation
[Equation]
From the Kalman ﬁlter designed for the target system, we can retrieve the frequency bands that are important for state estimation or the noise characteristics in the target system.
Conversely, we can also evaluate whether the obtained state estimator is reasonable or not, based on the knowledge of the target system.
For the state estimator E implemented by a neural network, E u and E y correspond to ∂E∂ u p and ∂E∂ y p at the operating point, respectively.
And by calculating these, the characteristics of the obtained state estimator can be evaluated.
On the other hand, the predictor P x and P u are an extended observability matrix and the Markov parameters of the target system, that is,
[Equation]
and characterize the unforced response and input-output characteristics of the target system.
For nonlinear predictors, P x and P u correspond to ∂P∂ ˆ x and ∂P∂ y p , respectively, from which the characteristics of the target system at the operating point can be illustrated.
I LLUSTRATIVE E XAMPLE
Next, we demonstrate the proposed method through a toy example, where the identiﬁcation and
MPC of a cascaded tanks system (Wigren & Schoukens, 2013) is performed.
The state equation of the system is as follows:
[Equation]
where n x = 2 , n u = n y = 1 , k 1 = 0 .
5 , k 2 = 0 .
4 , k 3 = [Equation]
[Equation] , k 4 = 0 .
[Equation] N (cid:0) 0 , 0 .
35 2 (cid:1) , and v t ∼ N (cid:0) 0 , 0 .
15 2 (cid:1) .
The state variables x t, 1 and x t, 2 are the levels of the upper and the lower tank, and u t denotes the applied voltage to the pump at t .
To obtain the training dataset, the system is excited by a sequence of random input
[Equation]
and a sequence of data { ( u t , y t ) } 1000019 t = − 19 is obtained.
On training the state estimator E and predictor P , We choose the hyperparameters of n ˆ x = 2 , h p =
h f = 20 .
E and P are implemented by neural networks with two and three hidden layers, respectively, where each layer contains 64 ReLU neurons and every layers are fully connected.
For training, we used the Adam optimizer in PyTorch (Paszke et al., 2019) with the default settings.
To prevent overﬁtting, the 90 % of the samples were used for training, and the training was continued until the ﬁt to the remaining 10 % of the validation data did not improve for 1000 consecutive iterations.
Then, the MPC controller which solves the optimization problem (13) with the cost function
[Equation]
and the constraint 0 ≤ u t ≤ 5 at each time t is implemented with the obtained state estimator and predictor.
Here, r t ∈ R n y is the reference signal.
To solve the optimization problem, and we used
Levenberg-Marquardt Algorithm (Mor´e, 1978).
The result of the tracking control up to t = 400 is shown in ﬁgure 2.
In the ﬁgure, the applied control input u t , the measured output y t (red line), reference signal r t (dotted black line), and the true water levels in the tanks x t, 1 , x t, 2 (blue lines) are shown.
In addition, the green lines in each plot show u f t and y f t at t = 400 , respectively.
As can be seen from the ﬁgure, appropriate MPC is performed using the obtained state estimator and predictor.
Also, the state estimate ˆ x t and the state of the system x t are compared in Appendix C.
[Figure]
Then, to support the interpretation discussed in section 4, ﬁgure 3 shows the visualization of the state estimator and predictor at the operating point at t = 50 , where the change in water level is slow and nonlinearity is expected to be weak.
The dashed lines in ﬁgures 3(a)-(c) and the bitmap on the right in ﬁgure 4(d)
show the corresponding properties of the linearized system of (23) at the operating point.
Since linear systems also have degrees of freedom in how they coordinate in state space, a transformation matrix Φ :
= P † x ∂P∂ ˆ x ∈ R n x × n x is used to align the coordinate system.
From ﬁgures 3(a) and (b), the state estimator E obtained by the proposed method agrees well with the
Kalman ﬁlter for the linearized system of the true system, and it can be conﬁrmed that the proposed method yields almost optimal state estimators as expected for subspace methods in an ideal situation with sufﬁcient data.
It can also be seen that the predictor is consistent with the true system.
Moreover, ﬁgure 4 was obtained at the operating point at t = 380 , where the nonlinearity is considered to be more strongly observed.
Although the results in this case are similar to those in the linearized system to some extent, differences in the characteristics can be seen.
For example, in [Figure] [Figure]
[Figure]
[Figure] [Figure]
[Figure] [Figure]
ﬁgure 4(d), the Toeplitz structure of linear systems is partially lost, indicating the existence of nonlinearity. In ﬁgures 4(a) and (b), it can be seen that an estimator that is less dependent on the output and more focused on the recent input than the Kalman ﬁlter for the linearized system is obtained.
To verify the validity of placing the bottleneck, we evaluated the performance with different widths of the bottleneck n ˆ x .
All settings except n ˆ x and the realized value of the stochastic noise w t , v t are set to be identical to the case of n ˆ x = 2 and 60 trials were conducted with different realizations of the stochastic noise.
The results are summarized in ﬁgure 5.
The loss for the validation dataset in ﬁgure 5(a) shows that [Figure]
[Figure]
for Validation Set
[Figure]
[Figure] w i d t h [Figure]
[Figure]
the performance does not degrade until the width of the bottleneck n ˆ x goes below the actual state dimension n x = 2 .
Also, the performance of the MPC calculated by (cid:80) 400 t =h p +1 l t ( u t , x t, 2 ) does not show degradation until n ˆ x < n x = 2 as shown in ﬁgure 5(b), and these results support the use of the bottleneck to provide interpretability.
C ONCLUSION
In this paper, we proposed a new nonlinear subspace identiﬁcation method that produces state estimator and predictor directly usable for nonlinear MPC.
It is theoretically shown that by training a neural network with a bottleneck that performs regression on input-output sequences of a length that provides observability in both past and future directions, a state estimator and a predictor are formed across the bottleneck layer.
The relationship between this method and the subspace identiﬁcation method, as well as the interpretation of the obtained model using this relationship, is also explained, and its use is demonstrated through an example.
This research is still in its early stages, and there is much work to be done in the future.
In order to demonstrate the applicability to higher-dimensional problems, an example of identifying dynamics from a simple video is included in Appendix D.
However, verifying the performance in more practical tasks is a subject for future work.
Also, a more precise condition for yielding a state estimator is desired.
The current proof makes use of the assumption that zero-input responses from all states have been learned, and while the fact that zero-input responses constitute states is often used in this type of research (Verdult & Scherpen, 2004), more precise conditions would be useful for experiment design.
It also appears that the technique of inverse optimal control (Ab Azar et al., 2020) can be applied to the interpretation of the obtained state estimator.
Finally, since noise with high-order dynamics is ubiquitous (Bak et al., 1987) and the synthesis of state estimator for such a system is prone to failure, it is important to develop a mechanism to focus on useful states.
A I DENTIFICATION OF f AND h IN S TATE -S PACE E QUATION
A method of obtaining the state-space model from the state estimator and predictor is brieﬂy explained here.
The model of the output map h : x t (cid:55)→ y t , the map ˆ h : R n ˆ x → R n y is obtained by extracting the ﬁrst prediction of the predictor P as
[Equation]
As for the model of the state-transition map ˆ f : R n ˆ x × R n u → R n ˆ x , by estimating the state sequence { ˆ x t } Tt =1 with E as [Equation]
[Equation]
(27) the construction of ˆ f is reduced to the ﬁtting of a static map as minimize ˆ f T [Equation] t
[Equation]
(cid:13)(cid:13)(cid:13)
[Equation]
B V ERIFICATION OF A GREEMENT WITH S UBSPACE M ETHODS FOR L INEAR S YSTEMS
In order to validate the discussion in section 4, we apply the proposed method to the linear dynamical system (17) with the following parameters:
[Equation]
The learning setup is the same as in section 5, except that the state estimator E and predictor P are linear maps which can be represented by matrices E and P , and the distribution of the input is u t ∼ N (0 , 1) .
In ﬁgure 6, the E and P representing the obtained model are compared with the E and P calculated by (18)-(22), i.e., the results obtained by the ideal subspace method.
Since the linear state-space systems have a degree of freedom in how to take the coordinate system, a linear coordinate transformation (such that P x is most consistent) is applied to the estimated model to facilitate comparison.
As can be seen from the ﬁgure, the results are consistent except for a small deviation due to the ﬁnite sample size, indicating that the proposed method solves essentially the same problem as the subspace identiﬁcation method for linear systems.
C C OMPARISON OF S TATE E STIMATE ˆ x t AND A CTUAL S TATE x t
We show the trajectories of the actual state x t and the state estimate x t of the cascaded tanks example in ﬁgure 7.
Since the proposed method does not specify how to take the coordinates into the state space, they are connected by a nonlinear mapping that depends on the initial weights of the neural network before training.
For ease of comparison, the trajectory of ˆ x t is shown over the trajectory of x t (dotted line) in afﬁne coordinates such that ˆ x t is closest
[Figure]
20 i
[Figure]
[Figure]
[Figure]
i,
[Figure]
[Figure]
[Figure]
[Figure]
[Figure]
D S UPPLEMENTARY E XAMPLE
To demonstrate the applicability to high dimensional systems
[Figure]
the proposed method is applied to the problem of estimating the dynamics of video discussed in (Beintema et al., 2021).
Consider an environment where a nonlinear force acts a ball and we observe it with a camera of 25 × 25 pixels
[Figure]
The dynamics of the ball is
[Equation]
(30a)
[Equation]
(30b) where β = 1 / 200 , γ = 0 .
79 , k = 1 / 4 , and ( p x , p y ) is the position of the ball, which is not measurable.
( u x , u y ) is the external force whose sample value at time t are given as u t .
This system has a stable equilibrium point ( p x , p y )
= (0 . 5 , 0 . 5)
and we simulate this system from this point with zero velocity.
For observation, the pixel intensity at a position X, Y of the video is calculated by nonlinear equation
[Equation]
where r = 0 .
25 and v ∼ N (0 , 0 .
204 2 ) is noise term.
The video image y t is sampled from this continuous image with 25 × 25 grid from (0 , 0) to (1 , 1) .
This system has 2(= n u ) inputs, 4(= n x ) states, and 25 × 25 = 625(= n y ) outputs.
We discretize this system by forward difference with the time step ∆ t = 0 .
3 and simulate with random external force ( u x , u y ) uniformly distributed in [ − 1 , 1] to generate video data.
On training, we choose the hyperparameters of ˆ n x = 4 , h p =
h f = 20 .
E, P and ˆ f have three, three, and two layers respectively, where each layer contains 64 ReLU neurons, the images are ﬂattened into vectors with 625 elements, and all layers are fully connected.
The training dataset consists of a sequence { ( u t , y t ) } 100019 t = − 19 , and other settings are the same as in section 5.
In the following, we describe the validation of the obtained model and its application to denoising.
The data used in the following are obtained under the same settings as in training, except that the realizations of random input and noise are different.
D. M ODELING OF D YNAMICS
To verify that the resulting model correctly models the dynamics of the video, we compare the video from the open-loop simulation of the obtained model with the video from the target system for the same input (external force).
Here, the output image from the model ˆ y t is computed recursively without using y t as ˆ y t = (cid:2)
I n y
[Equation]
(32) where ˆ y p t is a sequence of past model outputs deﬁned as in (2).
Note that in this test, the ball is stopped at the center (0 . 5 , 0 . 5) at t = 1 , and ˆ y t for t = 1 − h p , . .
, 0 are set as the image at this state.
Figure 8 shows the actual images without noise (above) and images from simulation with the model (below).
As shown in the ﬁgure, the images obtained from the model simulation are close to the t
=t =t =t =
t =t =
t =t = actual image without noise, indicating that the dynamics are correctly learned.
A video version of ﬁgure 8 is provided in the supplementary material as actual-simulated.mp4 .
D. N OISE R EDUCTION
Next, we show that the model obtained by the proposed method can be used for denoising the video.
Here the denoised image ˆ y t is computed from the images y p t and inputs u p t of the past h p frames and the current input u t as
[Equation]
Note that unlike the open-loop simulation (32), the state estimator E uses a noisy output y p t instead of recursively using ˆ y p t .
Figure 9 shows the result of denoising, The bottom images are the denoised images ˆ y t obtained from the trained model, and the middle images are the images obtained from the actual system y t .
From the ﬁgure, it can be conﬁrmed that the noise is removed by the model obtained by
[Equation]
[Equation]
the proposed method.
A video version of ﬁgure 9 is provided in the supplementary material as