Learning Deep Input-Output Stable Dynamics
Anonymous Author(s)
AfﬁliationAddress email
Abstract
Learning stable dynamics from observed time-series data is an essential problemin robotics, physical modeling, and systems biology.
Many of these dynamics arerepresented as an inputs-output system to communicate with the external environment.
In this study, we focus on input-output stable systems, exhibiting robustnessagainst unexpected stimuli and noise.
We propose a method to learn nonlinearsystems guaranteeing the input-output stability.
Our proposed method utilizes thedifferentiable projection onto the space satisfying the Hamilton-Jacobi inequalityto realize the input-output stability.
The problem of ﬁnding this projection canbe formulated as a quadratic constraint quadratic programming problem, and wederive the particular solution analytically.
Also, we apply our method to a toybistable model and the task of training a benchmark generated from a glucoseinsulin simulator.
The results show that the nonlinear system with neural networksby our method achieves the input-output stability, unlike naive neural networks.
Introduction
Learning dynamics from time-series data has many applications such as industrial robot systems[1], physical systems[2], and biological systems [3, 4].
Many of these real-world systems equippedwith inputs and outputs to connect for each other, which are called input-output systems [5].
Forexample, biological systems sustain life by obtaining energy from the external environment throughtheir inputs.
Such real-world systems have various properties such as stability, controllability, andobservability, which provide clues to analyze the complex systems.
Our purpose is to learn a complex system with “desired properties” from a dataset consisting ofpairs of input and output signals.
To represent the target system, this paper considers the followingnonlinear dynamics:
[Equation]
where the inner state x , the input u , and the output y belong to a signal space that maps time interval[0 , 1 ) to the Euclidean space.
We denote the dimension of x , u , and y as n , m , and l , respectively
Recently, with the development of deep learning, many methods to learn systems from time-series data using neural networks have been proposed [6, 7, 8].
By representing the maps ( f, G, h ) in Eq.
(1) as neural networks, complex systems can be modeled and trained from a given dataset.
However, guaranteeing that a trained system has the desired properties is challenging.
A naively trained system ﬁts the input signals contained in the training dataset, but does not always ﬁt for new input signals.
For example, Figure 1 shows our preliminary experiments where we naively learned neural networks ( f, G, h ) in Eq.
(1) from input and output signals.
The trained neural networks provided small predictive errors for an input signal in the training dataset (Figure 1 (A) )
Contrastingly, the unbounded output signals were computed by the trained system with new step
[Figure]
input signals (Figure 1 (B) ).
The reason can be expected that the magnitude of this input signal waslarger than that of the input signals in the training dataset.
The internal stability is known as one of the attractive properties that should be often satisﬁed inreal-world dynamical systems.
The conventional methods to train internal stable systems consistingof neural networks adopt the Lyapunov-based approaches [9, 10, 11].
These methods focus on theinternal system, ˙ x
= f ( x ) in the input-output system (1).
Thus, how to learn the entire system (1)with the desired property related to the inﬂuence of the input signal is still challenging.
We propose a novel method to learn a dynamical system consisting of neural networks consideringthe input-output stability.
The notion of the input-output stability is often used together with the
Hamilton-Jacobi inequality for controller design of the target system in the ﬁeld of control theory.
The Hamilton-Jacobi inequality is one of the sufﬁcient conditions for the input-output stability.
Thefeature of this condition is that the variable for an input signal u does not appear in the expression,i.e., we do not need to evaluate the condition for unknown inputs [12].
To the best of our knowledge,this is the ﬁrst work that establishes a learning method for the dynamical systems consisting of neuralnetworks using the Hamilton-Jacobi inequality.
The contributions of this paper are as follows:
• This paper derives differentiable projections to the space satisfying the Hamilton-Jacobiinequality.
• This paper presents the learning method for the input-output system proven to always satisfythe Hamilton-Jacobi inequality.•
This paper also provides a loss function derived from the Hamilton-Jacobi inequality.
By
combining this loss function with the projection described above, efﬁcient learning can be
expected.
• This paper presents experiments using two types of benchmarks to evaluate our method.
Background
This section describes the L 2 stability, a standard deﬁnition of the input-output stability, and
the
Hamilton-Jacobi inequality.
First, we deﬁne the L 2 stability of the nonlinear dynamical system (1).
If the norm ratio of the outputsignal to the input signal is bounded, the system is L 2 stable and this norm ratio is called the L 2 gain.
The L 2 norm on the input and output signal space is used for the deﬁnition of the L 2 stability.
Todeal with the L 2 stability on the nonlinear system (1), we assume that the origin x ⌘ 0 of the internalsystem ˙ x = f ( x ) is an asymptotically stable equilibrium point and h (0)
= 0 .
Since the translation ofany asymptotically stable equilibrium points is possible, this assumption can be satisﬁed without lossof generality.
Deﬁnition 1.
The L 2 norm is deﬁned as k x k L 2 :=
qR 1
k x ( t ) k 2 d t .
If there exists   0 and afunction  on a domain D ⇢ R n such that
[Equation]
then the system (1) is L 2 stable, where the function  ( · ) is non-negative and  (0) = 0 .
Furthermore,the minimum  satisfying (2) is called the L 2 gain of the nonlinear system (1).
Next, we describe a sufﬁcient condition of the L 2 stability using the Lyapunov function V : D !
R ,where the function V is positive deﬁnite, i.e., V ( x )  0 and V (0) = 0 .
The Hamilton-Jacobiinequality is an input-independent sufﬁcient condition.
Proposition 1.
If there exist a constant  > 0 and a smooth positive deﬁnite function V : D ⇢ R n !
R such that
[Equation]
then the system (1) is L 2 stable and the L 2 gain is less than or equal to  .
This condition is calledthe Hamilton-Jacobi inequality.
The above proposition can be generalized to allow the more complicated situations such as limit-cycleand bistable cases, where the domain D contains multiple asymptotically stable equilibrium points.
The equilibrium point assumed in this proposition can be replaced with positive invariant sets byextending the L 2 norm [13].
Furthermore, by mixing multiple Lyapunov functions, this propositioncan be generalized around multiple isolated equilibrium points.
Method
The goal of this paper is to learn the L 2 stable system represented by using neural networks ( f, G, h ) .
The Hamiltonian-Jacobi inequality, which implies the L 2 stability, is expressed by ( f, G, h ) .
Wepresent a method to project ( f, G, h ) onto the space where the Hamilton-Jacobi inequality holds.
3.1 Modiﬁcation of nonlinear systems
Supposing ˆ f : R n !
R n , ˆ G : R n !
R n ⇥ m , and ˆ h : R n !
R l , a triplet map ( ˆ f, ˆ G, ˆ h ) denoteas nominal dynamics.
Introducing a distance on the triplet maps, the nearest triplet satisfyingthe Hamilton-Jacobi inequality from the nominal dynamics ( ˆ f, ˆ G, ˆ h ) is called modiﬁed dynamics( f, G, h ) .
The purpose of this section is to describe the modiﬁed dynamics ( f, G, h ) associated withthe nominal dynamics ( ˆ f, ˆ G, ˆ h ) by analytically deriving a projection onto the space satisfying the Hamilton-Jacobi inequality.
The problem of ﬁnding the modiﬁed dynamics ( f, G, h ) is written as a quadratic constraint quadraticprogramming (QCQP) problem for the nominal dynamics ( ˆ f, ˆ G, ˆ h ) .
Since there is generally noanalytical solution for QCQP problems, we aim to ﬁnd the particular solution by adjusting the distanceon the triplets.
To prepare for the following theorem, we deﬁne the ramp and the clamp functions as
[Equation]
and deﬁne the Hamilton-Jacobi function as
[Equation]
(4) where V is a given positive deﬁnite function.
The function V can be designed as a trainable neuralnetwork [14] or as a ﬁxed function that achieves the desired properties of the nonlinear dynamics[15].
Theorem 1.
Consider that the following optimal problem:minimize
[Equation]
to HJ
[Equation]
5b
[Equation]
where k 2 [0 , 1] .
The solution of (5) is given by
[Equation]
where
[Equation]
Proof: See Appendix A.
The objective function of (5a) is a new distance between the nominal dynamics ( ˆ f, ˆ G, ˆ h ) and themodiﬁed dynamics ( f, G, h ) .
This new distance allows the derivation of analytical solutions bycombining three distances of f , G , and h .
Focusing on the objective function (5a), the constant k represents the ratio of the distance scale of fto g and h .
When k = 0 , the problem (5) is consistent with the optimization problem for the internalstability [9].
As the constant k converges 1 , the modiﬁcation method of Theorem 1 approaches Gand h to ˆ G and ˆ h , respectively.
In this case, the objective function (5a) becomes the distance betweenf to ˆ f .
Therefore, the following corollary is satisﬁed.
Corollary 1.
The solution of
[Equation]
6b) is given by
[Equation]
Proof: This solution is easily derived from Theorem 1.
Corollary 1 derives a solution to a linear programming problem rather than QCQP problems.
Replacing the Hamilton-Jacobi function HJ( f, ˆ G, ˆ h ) with the time derivative of a positive deﬁnite functionr V f , this corollary matches the result of the conventional study [9].
When the map h is ﬁxed as ˆ h , a similar solution as Theorem 1 is derived.
Although Corollary 1is proved by changing k , the modiﬁed dynamics with the ﬁxed h are not derived.
We reprove thismodiﬁed dynamics in a similar way to Theorem 1.
Corollary 2.
Consider the following problem:minimize
[Equation]
subject to HJ
[Equation]
f, G
[Equation]
7b
[Equation]
where k 2 [0 , 1] .
The solution of (7) is given by
[Equation]
[Figure]
where
[Equation]
Proof: See Appendix A.
3.2 Loss function
We represent ( ˆ f, ˆ G, ˆ h ) as neural networks and denote ( f, G, h ) as the L 2 stable dynamics modiﬁedby Theorem 1, Corollary 1, or 2.
Note that the modiﬁcation depends on the candidate of L 2 gain  .
Figure 2 (A) shows the sketch of this modiﬁcation, where the blue region satisﬁes the Hamilton-Jacobiinequality.
Since the nonlinear system of the modiﬁed dynamics ( f, G, h ) is represented as ordinary differentialequations (ODEs) consisting of the differentiable functions ˆ f , ˆ G , and ˆ h , the techniques of trainingneural ODEs can be applied [16].
Once a loss function is designed, the parameters of neural networksin the modiﬁed system can be learned from given data.
This paper presents the following loss function:
[Equation]
(9) where  and ↵ are non-negative coefﬁcients.
The ﬁrst term shows the prediction error of the output signal
y
( Figure 2 (A)).
A dataset D consists oftuples ( x 0 , u, y ) where the initial value x 0 , the input signal u , and the output signal y .
The predictedoutput ˆ y is calculated from x 0 , u , and the modiﬁed dynamics ( f, G, h ) .
The second term aims to improve the nominal dynamics ( ˆ f, ˆ G, ˆ h ) closer to the modiﬁed dynamics( f, G, h ) and is deﬁned as
[Equation]
where " is a positive constant ( Figure 2 (B)).
Since this term is a form of the hinge loss, " representsthe magnitude of the penalties for the Hamilton-Jacobi inequality.
To evaluate this inequality for anyx 2 D , we introduce a distribution of x over the domain D .
In our experiments, this distribution isdecided as a Gaussian distribution N ( µ,  2 ) where the mean µ is placed at the asymptotically stablepoint and the variance  2 is an experimental parameter.
The modifying parameter  can be manually designed for the application or automatically trainedfrom data by introducing the last term.
This training explores smaller  while keeping the same levelof the prediction error (Figure 2 (C)).  Related work
Estimating parameters of a given system is traditionally studied as system identiﬁcation.
In the ﬁeldof system identiﬁcation, much research on the identiﬁcation of linear systems have been done, wherethe maps ( f, G, h ) in Eq.
(1) assumes to be linear [17].
The nonlinear models such as the nonlinear
ARX model [18] and the Hammerstein-Wiener model [19] also have been developed and often trainedby error minimization.
For example, the system identiﬁcation method using a piece-wise ARX modelallows more complicated functions by relaxing the assumption of linearity [18].
These traditional methods often concern gray-box systems, where the maps ( f, G, h ) in Eq.
(1) arepartially known [20].
This paper deals with a case of black-box systems, where f , G , and h in thesystem (1) are represented by neural networks.
Our method can be used regardless of whether all f ,
G , and h functions are parameterized using neural networks.
So, our method can be easily appliedto application-dependent gray-box systems when the functions f , G , and h are differentiable withrespect to the parameters.
Because the input-output system (1) can be regarded as a differential equation, our study is closelyrelated to the method of combining neural networks and ODEs [7, 16].
These techniques have beenimproved in recent years, including discretization errors and computational complexity.
Although weused an Euler method for simplicity, we can expect that learning efﬁciency would be further improvedby using these techniques.
The internal stability is a fundamental property in ODEs, and learning a system with this propertyusing neural networks plays an important role in the identiﬁcation of real-world systems [21].
Inparticular, the ﬁrst method to guarantee the internal stability of the trained system has been proposedin [9].
Furthermore, another method [15] extends this method to apply positive invariant sets, e.g.limit cycles and line attractors.
Encouraged by these methods based on the Lyapunov function, ourmethod further generalizes these methods using the Hamilton-Jacobi inequality to guarantee theinput-output stability.
Although this paper deals with deterministic systems, neural networks for stochastic dynamics withthe variational inference is well studied [6, 22, 23, 24].
Guaranteeing the internal stability of trainedstochastic dynamics is important for noise ﬁltering and robust controller design [10].
Experiments
We conduct two experiments to evaluate our proposed method.
The ﬁrst experiment uses a benchmarkdataset generated from a nonlinear model with multiple asymptotically equilibrium points.
In the nextexperiment, we applied our method to a biological system using a simulator of the glucose-insulinsystem.
5.1 Experimental setting
Before describing the results of the experiments, this section explains the evaluation metrics and ourexperimental setting.
For evaluation metrics, we deﬁne the root mean square error (RMSE) and the average L 2 gain(GainIO) for the given input and output signals as follows:
[Equation]
where N is the number of signals in the dataset.
u i ( · ) and y i ( · ) are the input and output signal at thei -th index, respectively.
The prediction signal ˆ y i ( · ) is computed from u i ( · ) , the trained dynamics,and the initial state.
Note that the integral contained in the L 2 norm is approximated by a ﬁnitesummation.
The RMSE is a metric of the prediction errors related to the output signal, and the
GainIO is a metric of the property of the L 2 stability.
Whether the target system satisﬁes or does notsatisfy the L 2 stability, the GainIO with a given ﬁnite-size dataset can be calculated.
The GainIOerror is deﬁned by the absolute error between the GainIO of the test dataset and that of the prediction.
In our experiments, 90 % of the dataset is used for training and the
[Figure]
For simplicity in our experiments, the sampling step  t for the output y is set as constant and the
Euler method is used to solve ODEs.
x 0 is put at an asymptotically stable point for each benchmarkand is known.
In this experiment, to prevent the state from diverging during learning of dynamics,the clipping operation is used so that the absolute values of the states are less than ten.
For comparative methods, we use vanilla neural networks, ARX, ORT [25], MOESP [26], andpiece-wise ARX[18].
In the method of vanilla neural networks, the maps ( f, G, h ) in the nonlinearsystem (1) is represented by using three neural networks, i.e., this method is consistent with a methodused in Figure 1.
To determine the hyperparameters of comparative methods except for neuralnetworks, the grid search is used.
Note that these comparative methods only consider the predictionerrors.
For training each method with neural networks, an NVIDIA Tesla T4 GPU was used.
Our experimentsare totally run on 20 GPUs over about three days.
5.2 Bistable model benchmark
The ﬁrst experiment is carried out using a bistable model, which is known as a bounded system withmultiple asymptotically equilibrium points.
This bistable model is deﬁned as
[Equation]
The internal system of this model has two asymptotically stable equilibrium points x ⌘ 1 ,  1 .
We generate 1000 input and output signals for this experiment.
To construct this dataset, we prepareinput signals using positive and negative pulse wave signals whose pulse width is changed at random.
The input and output signals on the period [0 , 10] are sampled with an interval  t = 0 .
In this [Figure] benchmark, we set the number of dimensions of the internal system as one and use a ﬁxed function
V ( x ) = min(( x  1) 2 , ( x + 1) 2 ) , a mixture of the two positive deﬁnite functions.
In the result of our experiments, we name the proposed methods modiﬁed by Theorem 1, Corollary 1,and 2 as DIOS-fgh, DIOS-f, and DIOS-fg, respectively.
In these methods, the parameters of our lossfunction are set as  = 0 and ↵ = 0 .
Also, DIOS-fgh+ uses  = 0 . 01 and ↵ = 0 .
01 under thesame conditions as DIOS-fgh.
[-For this example (1000 input signal), it took about 1 hour using 
-]
The results of the RMSE and the GainIO error in this experiment are shown in Figure 3.
Figures 3 (A)and (B) demonstrate that a piece-wise ARX model (PWARX) gives very low RMSE but its GainIOerror is high.
Our proposed methods achieve a small GainIO error while keeping the RMSE.
Notethat linear models such as MOESP and ORT only approach one point although the bistable model hastwo asymptotically stable equilibrium points.
Figures 3 (C) and (D) display the effect of dataset sizes.
When the dataset size was varied to 100, 1000, and 3000, the result of the larger dataset providedthe smaller RMSE for all methods.
The vanilla neural networks do not consider the L 2 gain, so
the
GainIO error was large in the case of the low RMSE.
Figure 4 shows the relationship between x and f ( x ) in the trained system in (1) to compare thevanilla neural networks, DIOS-fgh and DIOS-fgh+.
DIOS-fgh and DIOS-fgh+ successfully ﬁnd twostable points, i.e., the trained function f ( x ) had roots of two stable points x = ± 1 and an unstablepoint x = 0 .
Especially, these stable points were robustly estimated in the trained system usingour presented loss function (DIOS-fgh+) even when the dataset size was small.
The vanilla neuralnetworks failed to obtain the two stable points in all cases.
5.3 Glucose-insulin benchmark
This section addresses an example of the identiﬁcation of biological systems.
We consider the task oflearning the glucose-insulin system using a simulator [27] to construct responses for various inputsand evaluate the robustness of the proposed method for unexpected inputs.
This simulator outputs theconcentrations of plasma glucose y 1 and insulin y 2 for the appearance of plasma glucose per minuteu .
To determine the realistic input u , we adopt another model, an oral glucose absorption model [28].
Using this simulator, 1000 input and output signals are synthesized for this experiment.
The inputand output signals are sampled with a sampling interval  t = 1 and 1000 steps for each sequence.
Inthis benchmark, we set the number of dimensions of the internal system as six, ﬁx a positive deﬁnitefunction V ( x ) = x 2 , and use our loss function with  = 0 . 001 and ↵ = 0 .
The hyperparametersincluding the number of layers in the neural networks, the learning rate, optimizer, and the weighteddecay are determined using the tree-structured Parzen estimator (TPE) implemented in Optuna [29].
The RMSE of vanilla and our proposed method are 0.0103 and 0.0050, respectively.
So, fromthe perspective of RMSE, these methods achieved almost the same performance.
Figure 5 showsinput and output signals in the test dataset and the predicted output by vanilla neural networks andour method (DIOS-fgh+).
The L 2 stability of the system using the vanilla neural networks is notguaranteed.
Since the proposed method guarantees the L 2 stability, the output signals of DIOS-fgh [Figure]
To demonstrate this, we conducted an additional experiment using the trained system.
Figure 6 showsthe transition of output behavior caused by the magnitude of the input signal changing from 2 to 10.
Note that the maximum magnitude in the training dataset is one.
In this experiment,  t was changedto 0 .
and the clipping operation was removed to deal with the large values of the state.
This result shows the output of the vanilla neural networks quickly diverged with an unexpectedlylarge input.
Contrastingly, the output behavior of our proposed method is always bounded.
Therefore,we actually conﬁrmed that our proposed method satisﬁes the L 2 stability.
Conclusion
This paper proposed a learning method for nonlinear dynamical system guaranteeing the L 2 stability.
By theoretically deriving the projection of a triplet ( f, G, h ) to the space satisfying the HamiltonJacobi inequality, our proposed method realized the L 2 stability of trained systems.
Also, weintroduced a loss function to empirically achieve a smaller L 2 gain while reducing prediction errors.
We conducted two experiments to learn dynamical systems consisting of neural networks.
The ﬁrstexperiment used a nonlinear model with multiple asymptotically equilibrium points.
The result ofthis experiment showed that our proposed method can robustly estimate a system with multiple stablepoints.
In the next experiment, we applied our method to a biological system using a simulator of theglucose-insulin system.
It was conﬁrmed that the proposed method can successfully learn a propersystem that works well under unexpectedly large inputs due to the L 2 stability.
There is a limitationthat our method cannot apply the system without the L 2 stability.
Future work will expand the L 2stability-based method to a more generalized learning method by dissipativity and apply our approachof this study to stochastic systems.
Checklist
The checklist follows the references.
Please read the checklist guidelines carefully for information onhow to answer these questions.
For each question, change the default [TODO] to [Yes] [No] or[N/A] .
You are strongly encouraged to include a justiﬁcation to your answer , either by referencingthe appropriate section of your paper or providing a brief inline description.
For example:
• Did you include the license to the code and datasets?
[Yes] See Section 2.
• Did you include the license to the code and datasets?
[No] The code and the data are proprietary.
• Did you include the license to the code and datasets?
[N/A]
Please do not modify the questions and only use the provided macros for your answers.
Note that the
Checklist section does not count towards the page limit.
In your paper, please delete this instructionsblock and only keep the Checklist section heading above along with the questions/answers below.
For all authors...
(a) Do the main claims made in the abstract and introduction accurately reﬂect the paper’scontributions and scope?
[Yes] See Section
(b) Did you describe the limitations of your work?
[Yes] See Section
(c) Did you discuss any potential negative societal impacts of your work?
[No]
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them?
[Yes]
If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results?
[Yes] See Section
(b) Did you include complete proofs of all theoretical results?
[Yes] See the supplemental material (Appendix.pdf)
If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main ex-
perimental results (either in the supplemental material or as a URL)?
[Yes] See thesupplemental material.
Our codes contains README.md to reproduce the our experi-
mental results.
(b) Did you specify all the training details (e g., data splits, hyperparameters, how theywere chosen)?
[Yes] See Section 5(c)
Did you report error bars (e
g., with respect to the random seed after running experi- ments multiple times)?
[Yes] See Section
(d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)?
[Yes] See Section 5.
If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators?
[N/A]
(b) Did you mention the license of the assets?
[Yes] See the supplemental materials.
Ourcodes contains LICENSE.txt.
(c) Did you include any new assets either in the supplemental material or as a URL?
[Yes]
[Table]
See the supplemental materials containing our source codes
(d) Did you discuss whether and how consent was obtained from people whose data you’reusing/curating?
[N/A]
Our dataset was reproduced by ourselves based on the literature
(e) Did you discuss whether the data you are using/curating contains personally identiﬁable
[Table]
information or offensive content?
[Yes]
If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, if [Table] applicable?
[N/A]
(b) Did you describe any potential participant risks, with links to Institutional Review
Board (IRB) approvals, if applicable?
[N/A]
[Table]
(c) Did you include the estimated hourly wage paid to participants and the total amountspent on participant compensation?
[N/A]
[Table]