Learning Deep Input-Output Stable Dynamics
Abstract
Learning stable dynamics from observed time-series data is an essential problem in robotics, physical modeling, and systems biology.
Many of these dynamics are represented as an inputs-output system to communicate with the external environment.
In this study, we focus on input-output stable systems, exhibiting robustness against unexpected stimuli and noise.
We propose a method to learn nonlinear systems guaranteeing the input-output stability.
Our proposed method utilizes the differentiable projection onto the space satisfying the Hamilton-Jacobi inequality to realize the input-output stability.
The problem of finding this projection can be formulated as a quadratic constraint quadratic programming problem, and we derive the particular solution analytically.
Also, we apply our method to a toy bistable model and the task of training a benchmark generated from a glucoseinsulin simulator.
The results show that the nonlinear system with neural networks by our method achieves the input-output stability, unlike naive neural networks. Our code is available at https://github.com/clinfo/DeepIOStability .
Introduction
Learning dynamics from time-series data has many applications such as industrial robot systems [1], physical systems[2] and biological systems [3, 4].
Many of these real-world systems equipped with inputs and outputs to connect for each other, which are called input-output systems [5].
For example, biological systems sustain life by obtaining energy from the external environment through their inputs.
Such real-world systems have various properties such as stability, controllability and observability, which provide clues to analyze the complex systems.
Our purpose is to learn a complex system with “desired properties” from a dataset consisting of pairs of input and output signals.
To represent the target system, this paper considers the following nonlinear dynamics:
[Equation]
where the inner state x , the input u , and the output y belong to a signal space that maps time interval [0 , ∞ ) to the Euclidean space.
We denote the dimension of x , u , and y as n , m , and l , respectively.
Recently, with the development of deep learning, many methods to learn systems from time-series data using neural networks have been proposed [6, 7, 8].
By representing the maps ( f, G, h ) in Eq.
(1) as neural networks, complex systems can be modeled and trained from a given dataset.
However, guaranteeing that a trained system has the desired properties is challenging.
A naively trained system fits the input signals contained in the training dataset, but does not always fit for new input signals. For example, Figure 1 shows our preliminary experiments where we y : ou t pu t training data predict    1000 t : time 0. u : i npu t 0.
y :
ou t pu t 0.
1.5 t : time 0. u : i npu t [Figure] naively learned neural networks ( f, G, h ) in Eq.
(1) from input and output signals.
The trained neural networks provided small predictive errors for an input signal in the training dataset (Figure 1 (A) ).
Contrastingly, the unbounded output signals were computed by the trained system with new step input signals (Figure 1 (B) ).
The reason can be expected that the magnitude (integral value) of this input signal was larger than that of the input signals in the training dataset.
The internal stability is known as one of the attractive properties that should be often satisfied in real-world dynamical systems.
The conventional methods to train internal stable systems consisting of neural networks adopt the Lyapunov-based approaches [9, 10, 11].
These methods focus on the internal system, ˙ x
= f ( x ) in the input-output system (1).
Thus, how to learn the entire system (1) with the desired property related to the influence of the input signal is still challenging.
We propose a novel method to learn a dynamical system consisting of neural networks considering the input-output stability.
The notion of the input-output stability is often used together with the
Hamilton-Jacobi inequality for controller design of the target system in the field of control theory.
The Hamilton-Jacobi inequality is one of the sufficient conditions for the input-output stability.
The feature of this condition is that the variable for an input signal u does not appear in the expression, i.e., we do not need to evaluate the condition for unknown inputs [12].
To the best of our knowledge, this is the first work that establishes a learning method for the dynamical systems consisting of neural networks using the Hamilton-Jacobi inequality.
The contributions of this paper are as follows:
• This paper derives differentiable projections to the space satisfying the Hamilton-Jacobi inequality.
• This paper presents the learning method for the input-output system proven to always satisfy the Hamilton-Jacobi inequality.
• This paper also provides a loss function derived from the Hamilton-Jacobi inequality.
By combining this loss function with the projection described above, efficient learning can be expected.
• This paper presents experiments using two types of benchmarks to evaluate our method.
Background
This section describes the L 2 stability, a standard definition of the input-output stability, and
the
Hamilton-Jacobi inequality.
First, we define the L 2 stability of the nonlinear dynamical system (1).
If the norm ratio of the output signal to the input signal is bounded, the system is L 2 stable and this norm ratio is called the L 2 gain.
The L 2 norm on the input and output signal space is used for the definition of the L 2 stability.
To deal with the L 2 stability on the nonlinear system (1), we assume that the origin x ≡ 0 of the internal system ˙ x = f ( x ) is an asymptotically stable equilibrium point and h (0)
= 0 .
Since the translation of any asymptotically stable equilibrium points is possible, this assumption can be satisfied without loss of generality.
Definition 1.
The L 2 norm is defined as ∥ x ∥ L 2 := (cid:113)(cid:82) ∞
∥ x ( t ) ∥ 2 d t .
If there exists γ ≥ 0 and a function β on a domain D ⊂ R n such that
[Equation]
then the system (1) is L 2 stable, where the function β ( · ) is non-negative and β (0) = 0 .
Furthermore, the minimum γ satisfying (2) is called the L 2 gain of the nonlinear system (1).
Next, we describe a sufficient condition of the L 2 stability using the Lyapunov function V : D → R , where the function V is positive definite, i.e., V ( x ) ≥ 0 and V (0) = 0 .
The Hamilton-Jacobi inequality is an input-independent sufficient condition.
Proposition 1 ([5, Theorem 5.5]) .
Let f be locally Lipschitz and let G and h be continuous.
If there exist a constant γ > 0 and a continuously differentiable positive definite function V : D ⊂ R n → R such that
[Equation]
then the system (1) is L 2 stable and the L 2 gain is less than or equal to γ .
This condition is called the Hamilton-Jacobi inequality.
The above proposition can be generalized to allow the more complicated situations such as limit-cycle and bistable cases, where the domain D contains multiple asymptotically stable equilibrium points.
The equilibrium point assumed in this proposition can be replaced with positive invariant sets by extending the L 2 norm [13].
Furthermore, by mixing multiple Lyapunov functions, this proposition can be generalized around multiple isolated equilibrium points.
Method
The goal of this paper is to learn the L 2 stable system represented by using neural networks ( f, G, h ) .
The Hamiltonian-Jacobi inequality, which implies the L 2 stability, is expressed by ( f, G, h ) .
We present a method to project ( f, G, h ) onto the space where the Hamilton-Jacobi inequality holds.
3.1 Modification of nonlinear systems
Supposing f n : R n → R n , G n : R n → R n × m , and h n : R n → R l , a triplet map ( f n , G n , h n ) denote as nominal dynamics.
Introducing a distance on the triplet maps, the nearest triplet satisfying the Hamilton-Jacobi inequality from the nominal dynamics ( f n , G n , h n ) is called modified dynamics ( f m , G m , h m ) .
The purpose of this section is to describe the modified dynamics ( f m , G m , h m ) associated with the nominal dynamics ( f n , G n , h n ) by analytically deriving a projection onto the space satisfying the Hamilton-Jacobi inequality.
The problem of finding the modified dynamics ( f m , G m , h m ) is written as a quadratic constraint quadratic programming (QCQP) problem for the nominal dynamics ( f n , G n , h n ) .
Since there is generally no analytical solution for QCQP problems, we aim to find the particular solution by adjusting the distance on the triplets.
To prepare for the following theorem, we define the ramp and the clamp functions as
[Equation]
and define the Hamilton-Jacobi function as
[Equation]
where V is a given positive definite function.
A way to design V is to determine the desired positive invariant set and design the increasing function around this set.
For example, if the target system has a unique stable point, x = 0 can be regarded as the positive invariant set and the increasing function V can be designed as V ( x ) = 12 x 2 .
Theorem 1.
Consider that the following optimal problem:
[Equation]
subject to HJ( f m , G m , h m [Equation] ≤ 0 ,
[Equation]
5b
[Equation]
where k ∈ [0 , 1] .
The solution of (5) is given by
[Equation]
where
[Equation]
Proof: See Appendix A.
The objective function of (5a) is a new distance between the nominal dynamics ( f n , G n , h n ) and the modified dynamics ( f m , G m , h m ) .
This new distance allows the derivation of analytical solutions by combining three distances of f , G , and h .
Focusing on the objective function (5a), the constant k represents the ratio of the distance scale of f to g and h .
When k = 0 , the problem (5) is consistent with the optimization problem for the internal stability [9].
As the constant k converges 1 , the modification method of Theorem 1 approaches G m and h m to G n and h n , respectively.
In this case, the objective function (5a) becomes the distance between f m to f n .
Therefore, the following corollary is satisfied.
Corollary 1.
The solution of
[Equation]
subject to
[Equation]
is given by
[Equation]
Proof: This solution is easily derived from Theorem 1.
Corollary 1 derives a solution to a linear programming problem rather than QCQP problems.
Replacing the Hamilton-Jacobi function HJ( f m , G n , h n ) with the time derivative of a positive definite function ∇ V f , this corollary matches the result of the conventional study [9].
When the map h m is fixed as h n , a similar solution as Theorem 1 is derived.
Although Corollaryis proved by changing k , the modified dynamics with the fixed h n are not derived.
We reprove this modified dynamics in a similar way to Theorem 1.
Corollary 2.
Consider the following problem:
[Equation]
subject to HJ
[Equation]
f
[Equation]
7b
[Equation]
where k ∈ [0 , 1] .
The solution of (7) is given by
[Equation]
( f , G , h ) ( f , G , h )
Modification E [|| y y || 2 ] Contour line of E [|| y y
|| 2 ] ( f , G , h ) ( f , G , h ) L HJ HJ ( f , G , h ) < ( f , G , h ) ( f , G , h )
Reduced region
[Figure]
(C) where
[Equation]
Proof: See Appendix A.
3.2 Loss function
We represent ( f n , G n , h n ) as neural networks and denote ( f m , G m , h m ) as the L 2 stable dynamics modified by Theorem 1, Corollary 1, or 2.
Note that the modification depends on the candidate of
L 2 gain γ .
Figure 2 (A) shows the sketch of this modification, where the blue region satisfies the
Hamilton-Jacobi inequality.
Since the nonlinear system of the modified dynamics ( f m , G m , h m ) is represented as ordinary differential equations (ODEs) consisting of the differentiable functions f n , G n , and h n , the techniques of training neural ODEs can be applied [14].
Once a loss function is designed, the parameters of neural networks in the modified system can be learned from given data.
[Equation]
where λ and α are non-negative coefficients.
The first term shows the prediction error of the output signal
y
( Figure 2 (A)).
A dataset D consists of tuples ( x 0 , u, y ) where the initial value x 0 , the input signal u , and the output signal y .
The predicted output ˆ y is calculated from x 0 , u , and the modified dynamics ( f m , G m , h m ) .
The second term aims to improve the nominal dynamics ( f n , G n , h n ) closer to the modified dynamics ( f m , G m , h m ) and is defined as
[Equation]
where ε is a positive constant ( Figure 2 (B)).
Since this term is a form of the hinge loss, ε represents the magnitude of the penalties for the Hamilton-Jacobi inequality.
To evaluate this inequality for any x ∈ D , we introduce a distribution of x over the domain D .
In our experiments, this distribution is decided as a Gaussian distribution N ( µ, σ 2 ) where the mean µ is placed at the asymptotically stable point and the variance σ 2 is an experimental parameter.
The modifying parameter γ can be manually designed for the application or automatically trained from data by introducing the last term.
This training explores smaller γ while keeping the same level of the prediction error (Figure 2 (C)).  Related work
Estimating parameters of a given system is traditionally studied as system identification.
In the field of system identification, much research on the identification of linear systems have been done, where the maps ( f, G, h ) in Eq.
(1) assumes to be linear [15].
Linear state-space models include identification methods by impulse response like the Eigensystem Realization Algorithm (ERA)[16], by the state-space representation like Multivariable Output Error State sPace (MOESP)[17] and
ORThogonal decomposition method (ORT)[18].
System identification methods for non-state-space models, unlike our target system, contain Dynamic Mode Decomposition with control (DMDc)[19] and its nonlinear version, Sparse Identification of Nonlinear DYnamics with control (SINDYc)[20].
Also, the nonlinear models such as the nonlinear ARX model [21] and the Hammerstein-Wiener model [22] have been developed and often trained by error minimization.
For example, the system identification method using a piece-wise ARX model allows more complicated functions by relaxing the assumption of linearity [21].
These traditional methods often concern gray-box systems, where the maps ( f, G, h ) in Eq.
(1) are partially known [23].
This paper deals with a case of black-box systems, where f , G , and h in the system (1) are represented by neural networks.
Our method can be used regardless of whether all f ,
G , and h functions are parameterized using neural networks.
So, our method can be easily applied to application-dependent gray-box systems when the functions f , G , and h are differentiable with respect to the parameters.
Because the input-output system (1) can be regarded as a differential equation, our study is closely related to the method of combining neural networks and ODEs [7, 14].
These techniques have been improved in recent years, including discretization errors and computational complexity.
Although we used an Euler method for simplicity, we can expect that learning efficiency would be further improved by using these techniques.
The internal stability is a fundamental property in ODEs, and learning a system with this property using neural networks plays an important role in the identification of real-world systems [24].
In particular, the first method to guarantee the internal stability of the trained system has been proposed in [9].
Furthermore, another method [25] extends this method to apply positive invariant sets, e.g. limit cycles and line attractors.
Encouraged by these methods based on the Lyapunov function, our method further generalizes these methods using the Hamilton-Jacobi inequality to guarantee the input-output stability.
In this paper, the Lyapunov function V is considered to be given, but this function can be learned from data.
Since a pair of dynamics (1) and V has redundant degrees of freedom, additional assumptions are required to determine V uniquely.
In [9], it is realized by limiting the dynamics to the internal system and restricting V to a convex function [26].
Lyapunov functions are also used to design controllers, where the whole system should satisfy the stability condition.
A method for learning such a controller using neural nets has been proposed [27].
This method deals with optimization problems over the space that satisfies the stability condition, which is similar to our method.
Whereas we solve the QCQP problem to derive the projection onto the space, this method uses a Satisfiability Modulo Theories (SMT) solver to satisfy this condition.
The method has also been extended to apply unknown systems [28].
Although this paper deals with deterministic systems, neural networks for stochastic dynamics with the variational inference is well studied [6, 29, 30, 31].
Guaranteeing the internal stability of trained stochastic dynamics is important for noise filtering and robust controller design [10].
Experiments
We conduct two experiments to evaluate our proposed method.
The first experiment uses a benchmark dataset generated from a nonlinear model with multiple asymptotically equilibrium points.
In the next experiment, we applied our method to a biological system using a simulator of the glucose-insulin system.
5.1 Experimental setting
Before describing the results of the experiments, this section explains the evaluation metrics and our experimental setting.
For evaluation metrics, we define the root mean square error (RMSE) and the average L 2 gain (GainIO) for the given input and output signals as follows:
[Equation]
where N is the number of signals in the dataset.
u i ( · ) and y i ( · ) are the input and output signal at the i -th index, respectively.
The prediction signal ˆ y i ( · ) is computed from u i ( · ) , the trained dynamics, and the initial state.
Note that the integral contained in the L 2 norm is approximated by a finite summation.
The RMSE is a metric of the prediction errors related to the output signal, and the
GainIO is a metric of the property of the L 2 stability.
Whether the target system satisfies or does not satisfy the L 2 stability, the GainIO with a given finite-size dataset can be calculated.
The GainIO error is defined by the absolute error between the GainIO of the test dataset and that of the prediction.
In our experiments, 90 % of the dataset is used for training and the remaining 10 % is used for testing.
We retry five times for all experiments and show the mean and standard deviations of the metrics.
For simplicity in our experiments, the sampling step ∆ t for the output y is set as constant and the
Euler method is used to solve ODEs.
x 0 is put at an asymptotically stable point for each benchmark and is known.
In this experiment, to prevent the state from diverging during learning of dynamics, the clipping operation is used so that the absolute values of the states are less than ten.
For comparative methods, we use vanilla neural networks, ARX, ORT [18], MOESP [17], and piece-wise ARX[21].
In the method of vanilla neural networks, the maps ( f, G, h ) in the nonlinear system (1) is represented by using three neural networks, i.e., this method is consistent with a method used in Figure 1.
To determine the hyperparameters of comparative methods except for neural networks, the grid search is used.
Note that these comparative methods only consider the prediction errors.
For training each method with neural networks, an NVIDIA Tesla T4 GPU was used.
Our experiments are totally run on 20 GPUs over about three days.
5.2 Bistable model benchmark
The first experiment is carried out using a bistable model, which is known as a bounded system with multiple asymptotically equilibrium points.
This bistable model is defined as
[Equation]
The internal system of this model has two asymptotically stable equilibrium points x ≡ 1 , − 1 .
We generate 1000 input and output signals for this experiment.
To construct this dataset, we prepare input signals using positive and negative pulse wave signals whose pulse width is changed at random.
The input and output signals on the period [0 , 10] are sampled with an interval ∆ t = 0 .
In this benchmark, we set the number of dimensions of the internal system as one and use a fixed function
V ( x ) = min(( x − 1) 2 , ( x + 1) 2 ) , a mixture of the two positive definite functions.
In the result of our experiments, we name the proposed methods modified by Theorem 1, Corollary 1, and 2 as DIOS-fgh, DIOS-f, and DIOS-fg, respectively.
In these methods, the parameters of our loss function are set as λ = 0 and α = 0 .
Also, DIOS-fgh+ uses λ = 0 . 01 and α = 0 .
under the same conditions as DIOS-fgh.
For this example (1000 input signal), it took about 1 hour using[-
-]
GPU to learn one model training.
The results of the RMSE and the GainIO error in this experiment are shown in Figure 3.
Figures 3 (A) and (B) demonstrate that a piece-wise ARX model (PWARX) gives very low RMSE but its GainIO error is high.
Our proposed methods achieve a small GainIO error while keeping the RMSE.
Note that linear models such as MOESP and ORT only approach one point although the bistable model has two asymptotically stable equilibrium points.
Figures 3 (C) and (D) display the effect of dataset sizes.
When the dataset size was varied to 100, 1000, and 3000, the result of the larger dataset provided
[Figure]
e rr o
[Figure]
+ [Equation] - - True [Equation]
[Figure]
the smaller RMSE for all methods.
The vanilla neural networks do not consider the L 2 gain, so
the
GainIO error was large in the case of the low RMSE.
Figure 4 shows the relationship between x and f ( x ) in the trained system in (1) to compare the vanilla neural networks, DIOS-fgh and DIOS-fgh+.
DIOS-fgh and DIOS-fgh+ successfully find two stable points, i.e., the trained function f ( x ) had roots of two stable points x = ± 1 and an unstable point x = 0 .
Especially, these stable points were robustly estimated in the trained system using our presented loss function (DIOS-fgh+) even when the dataset size was small.
The vanilla neural networks failed to obtain the two stable points in all cases. y 1 : G l u c o s e Vanilla  1000 t : Time 0. y 2 : I n s u li n
Our method (DIOS-fgh+)  1000 0.
u : I npu t
True [Figure]
5.3 Glucose-insulin benchmark
This section addresses an example of the identification of biological systems.
We consider the task of learning the glucose-insulin system using a simulator [32] to construct responses for various inputs and evaluate the robustness of the proposed method for unexpected inputs.
This simulator outputs the concentrations of plasma glucose y 1 and insulin y 2 for the appearance of plasma glucose per minute u .
To determine the realistic input u , we adopt another model, an oral glucose absorption model [33].
Using this simulator, 1000 input and output signals are synthesized for this experiment.
The input and output signals are sampled with a sampling interval ∆ t = 1 and 1000 steps for each sequence.
In this benchmark, we set the number of dimensions of the internal system as six, fix a positive definite function V ( x ) = x 2 , and use our loss function with λ = 0 . 001 and α = 0 .
Training one model for this examples (1000 input signal) tooks about 7.5hours using 1GPU.
The hyperparameters including the number of layers in the neural networks, the learning rate, optimizer, and the weighted decay are determined using the tree-structured Parzen estimator (TPE) implemented in Optuna [34].
The RMSE of vanilla and our proposed method are 0.0103 and 0.0050, respectively.
So, from the perspective of RMSE, these methods achieved almost the same performance.
Figure 5 shows input and output signals in the test dataset and the predicted output by vanilla neural networks and our method (DIOS-fgh+).
The L 2 stability of the system using the vanilla neural networks is not guaranteed.
Since the proposed method guarantees the L 2 stability, the output signals of DIOS-fgh+ are bounded even if the input signals are unexpectedly large.
To demonstrate this, we conducted an additional experiment using the trained system.
Figure 6 shows the transition of output behavior caused by the magnitude of the input signal changing from 2 to 10.
Note that the maximum magnitude in the training dataset is one.
In this experiment, ∆ t was changed to 0 .
and the clipping operation was removed to deal with the large values of the state.
This result shows the output of the vanilla neural networks quickly diverged with an unexpectedly large input.
Contrastingly, the output behavior of our proposed method is always bounded.
Therefore, we actually confirmed that our proposed method satisfies the L 2 stability.
Conclusion
This paper proposed a learning method for nonlinear dynamical system guaranteeing the L 2 stability.
By theoretically deriving the projection of a triplet ( f, G, h ) to the space satisfying the HamiltonJacobi inequality, our proposed method realized the L 2 stability of trained systems.
Also, we introduced a loss function to empirically achieve a smaller L 2 gain while reducing prediction errors.
We conducted two experiments to learn dynamical systems consisting of neural networks.
The first experiment used a nonlinear model with multiple asymptotically equilibrium points.
The result of this experiment showed that our proposed method can robustly estimate a system with multiple stable points.
In the next experiment, we applied our method to a biological system using a simulator of the glucose-insulin system.
It was confirmed that the proposed method can successfully learn a proper system that works well under unexpectedly large inputs due to the L 2 stability.
There is a limitation that our method cannot apply the system without the L 2 stability.
Future work will expand the Lstability-based method to a more generalized learning method by dissipativity and apply our approach of this study to stochastic systems. Acknowledgements This research was supported by JST Moonshot R&D Grant Number JPMJMS2021 and JPMJMS2024.
This work was also supported by JSPS KAKENHI Grant No.21H04905.
This paper was also based on a part of results obtained from a project commissioned by the New Energy and Industrial Technology
Development Organization (NEDO).
Checklist
The checklist follows the references.
Please read the checklist guidelines carefully for information on how to answer these questions.
For each question, change the default [TODO] to [Yes] , [No] , or [N/A] .
You are strongly encouraged to include a justification to your answer , either by referencing the appropriate section of your paper or providing a brief inline description.
For example:
• Did you include the license to the code and datasets?
[Yes] See Section 2.
• Did you include the license to the code and datasets?
[No] The code and the data are proprietary.
• Did you include the license to the code and datasets?
[N/A]
Please do not modify the questions and only use the provided macros for your answers.
Note that the
Checklist section does not count towards the page limit.
In your paper, please delete this instructions block and only keep the Checklist section heading above along with the questions/answers below.
For all authors... (a) Do the main claims made in the abstract and introduction accurately reflect the paper’s contributions and scope?
[Yes] See Section(b)
Did you describe the limitations of your work?
[Yes] See Section(c)
Did you discuss any potential negative societal impacts of your work?
[No] (d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes] The paper conforms with the provided guidelines.
If you are including theoretical results... (a) Did you state the full set of assumptions of all theoretical results?
[Yes] See Section(b)
Did you include complete proofs of all theoretical results?
[Yes] See the supplemental material (Appendix.pdf) 3.
If you ran experiments... (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?
[Yes] See the supplemental material.
Our codes contains README.md to reproduce the our experimental results.
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)?
[Yes] See Section(c)
Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?
[Yes] See Section(d)
Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)?
[Yes] See Section 5. 4.
If you are using existing assets (e.g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the creators?
[N/A] (b) Did you mention the license of the assets?
[Yes] See the supplemental materials.
Our codes contains LICENSE.txt.
(c) Did you include any new assets either in the supplemental material or as a URL?
[Yes] See the supplemental materials containing our source codes (d) Did you discuss whether and how consent was obtained from people whose data you’re using/curating?
[N/A]
Our dataset was reproduced by ourselves based on the literature (e) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content?
[Yes] 5.
If you used crowdsourcing or conducted research with human subjects... (a) Did you include the full text of instructions given to participants and screenshots, if applicable?
[N/A] (b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable? [N/A] (c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?
[N/A]
To prove Theorem 1 and Corollary 2, we use the following particular solution of QCQP problems. Lemma 1. Suppose that A ∈ R n × n is a positive definite matrix, a vector b ∈ R n , a scalar c , and positive constants k x , k y .
The solution of this problem minimize k x x T Ax + k y | y | subject to y ≥ x T Ax − 2 b T x + c. is given by [Equation] where [Equation]
Proof: See Appendix B.
To keep the notation short in the following proofs, we define
[Equation]
Proof of Theorem 2: For the Hamilton-Jacobi function HJ( f m , G m , h m ) of the modified dynamics ( f m , G m , h m ) , the map f m and each row of the map G m don’t depend on the orthogonal vector of β .
Hence, f m and G m are written as
[Equation]
where p is scalar and q ∈ R m .
h m depends only on the norm in HJ( f m , G m , h m ) ,
i. e., h m = (1 − r ) hn, where r is scalar.
The optimal function (5a) is rewritten as the p , q , and r , i.e.,
[Equation]
Also, the condition of constraint (5b) is rewritten as
HJ β
[Equation]
f m , G m , h m )
[Equation]
≤ 0 .
If x ≜ [ q T , r ] T , y ≜ p , and
{+A ≜ (cid:34) ∥ β ∥ 2 2 γ 
+}
I m
[Equation]
then this optimal problem (5) of this theorem becomes a problem used in Lemma 1.
The optimal point of p , q and r is given by
[Equation]
Therefore, Theorem 1 is derived.
Proof of Theorem 2:
Optimal maps f m and G m are also written as
[Equation]
Hence, the objective function (7a) is rewritten as p and q function,
i
[Equation]
e.,
[Equation]
Also the condition of constraint (7b) is written
[Equation]
≤ 0 .
If x ≜ q , y ≜ p , and
[Equation]
then this optimal problem (7) becomes the problem used in Lemma 1.
The optimal points of p and q are given by
[Equation]
Therefore, the solution of problem (7) becomes Eqs.
B Proof of the particular solution of QCQP
This section presents the proof of the following QCQP problem.
[Equation]
where A is a positive definite matrix.
We classify this problem according to the parameter A, b and c .
First, the solution of this optimal problem is switched depending on the positive or negative value of c .
Lemma 2.
If c ≤ 0 then the solution of Eq.
(10) is ( x, y )
=
If c > 0 , the solution x ∗ of
Eq.
(10) equals the solution of the following problem:
[Equation]
Furthermore, the solution y ∗ is given by
[Equation]
Proof: The objective function (10a) is strictly convex and the minimum point is ( x, y ) =
If c ≤ 0 , the origin ( x, y ) = (0 , 0) satisfies the condition of constraint (10b).
Therefore, the solution of
Eq.
(10) is ( x, y )
=
If c > 0 , ( x, y ) = (0 , 0) does not satisfies the constraint condition (10b), and the optimal solution belong the boundary of the region satisfying (10b).
Therefore, the solution point ( x ∗ , y ∗ ) satisfies
Eq.
Also, the solution of the new optimal problem (11) is switched by the ratio between c and b T A − 1 b .
Lemma 3.
If ˜ c > 1 − ˜ k 2 x then the solution x ∗ of the optimal problem (11) is given by
[Equation]
where
[Equation]
If ˜ c ≤ 1 − ˜ k 2 x , the solution x ∗ equals the solution of the following QCQP problem, such that
[Equation]
14a
[Equation]
14b
[Equation]
Proof: The optimal problem (11) is split on the sign of x ∗ T
Ax ∗ − 2 b T x ∗ + c .
[Equation]
The optimal problem of this case is written as
[Equation]
If k x − k y ≤ 0 , there is no optimal point.
Otherwise k x − k y > 0 , the optimal point is written as
[Equation]
The optimal point does not satisfies the condition x ∗ T
Ax ∗ − 2 b T x ∗ + c < 0 , because
[Equation]
where A − 1 is a positive definite matrix and c > 0 .
Case x ∗ T Ax ∗ − 2 b T x ∗ + c > 0 :
The problem (11) is written as minimize
[Equation]
Hence, this optimal point is written as
[Equation]
where √ A is a square root of the positive definite matrix A .
The condition x ∗ T Ax ∗ − 2 b T x ∗ + c >is rewritten by the previous optimal solution, such that
[Equation]
Case x T Ax ∗ − 2 b T x ∗ + c = 0 :
The problem (11) is written as the QCQP problem (14).
The solution of the simple QCQP problem (14) is easily derived using the method of Lagrange multiplier.
Lemma 4.
The solution of the simple QCQP problem (14) is given by
[Equation]
Proof: Supposing a Lagrange multiplier λ > 0 , the Lagrange function is written as
[Equation]
The KKT condition is given by
[Equation]
[Equation]
Eq.
(15) is written as
[Equation]
and the Eq.
(16) is given by
[Equation]
As λ > 0 and ˜ c > 0 , the Lagrange multiplier is written as
[Equation]
Therefore, the optimal point x ∗ is given by
[Equation]
Finally
[Equation]
we summarize three Lemmas 2-4 and solve the QCQP problem (10)
[Equation]
Lemma 1.
The solution of the problem (10) is given by
[Equation]
Proof: Lemmas 2-4 split the solution x ∗ as three cases:
[Equation]
Furthermore, the solution of y ∗ is written as,
[Equation]
Therefore, the solution become Eq.
C Overall schematic of the learning process
Algorithm 1 shows the overall schematic of the learning process.
The first line defines the modified dynamics ( f m , G m , h m ) from the nominal dynamics ( f n , G n , h n ) , defined by the neural network, where ϕ is a set of parameters of the nominal dynamics.
The 2-7 line represents a training loop, where the gradient-based optimization methods can be used by using the forward and backward calculation.
Note that an ODE solver is used for forward calculation, and Algorithm 2 shows the forward calculation when the Euler method is used.
For simplicity, mini-batch computation omitted in this schematic.
D Neural network architecture and hyper parameters
This section details how to determine the neural network architecture.
The architecture and hyper parameters of the neural networks were basically determined by Bayesian optimizing using the validation dataset.
Table 1 shows the search space of Bayesian optimization.
The first three parameters: learning rate, weight decay, and batch size are parameters for training the neural networks.
Also, an optimizer is selected from AdamW, Adam, and RMSProp.
The structure of neural network is determined from
Output output the number of intermediate layers and dimensions for each layer.
One layer in our setting consists of a fully connected layer with a ReLU activation.
The last three rows represent parameters related to our proposed methods.
ϵ is a parameter of the loss function L HJ .
Initial scale parameter is multiplied with the output of f n to prevent the value of f n ( x ) from becoming large in the initial stages of learning.
When f n ( x ) , which determine the behavior of the internal system, outputs a large value, it diverges due to time evolution, and the learning of the entire system may not progress.
Therefore, it is empirically preferable to start with a small value for f n ( x ) at the initial stage of learning.
When the flag of ‘stop gradient for projection’ is false, backward computation related to the second term of modification of f m and G m is disabled.
Note that modification related to f m and G m consists of two terms (see Theorem 1).
Setting this parameter to false resulted in better performance in our all experiments.
We ran 300 trials using the Bayesian optimization for the bistable model benchmark and the glucose insulin benchmark with the above settings, and the hyper parameters obtained are shown in Table 2, where the number of dimensions for each hidden layer is shown in tuple from the order closest to the input layer.
Hyperparameters of comparative methods were determined by grid search using the validation dataset.
ARX and PWARX have an order parameter n of the autoregressive model, and this parameter is searched in the range of 1 − 5 .
The number of iterations was set to 10000 so that the optimization of
PWARX converges sufficiently.
MOESP and ORT have an internal dimension n ( 1 ≤ n ≤ 20 ) and the number of subsequences used for estimation k ( 2 n < k ≤ 20 ).
[Table]
gradient for projection
[Table]
[Table]
for f n  # layer for G n  # layer for h n  # dim.
for a hidden layer of f n (17,10,22) (8) # dim.
for a hidden layer of G n (34) (27,29) #
[Table]
for
[Table]
layer of h n (
[Table]
,62,58) (35,18) ϵ
[Table]
E Glucose-Insulin system
Glucose concentration in the blood is modeled as a time-delay system regulated by insulin concentration (See Fig.
7 (A))
Suppose that G
I
[Table]
and X are the glucose, insulin, and accumulated glucose plasma concentration ([mg/100ml],[ µ UI/ml]
[Table]
and [min mg/100ml], respectively) and u is the amount of ingested glucose per minute [min − 1 mg/100ml].
The dynamics of each concentration is given by
[Equation]
where k 1 is a spontaneous glucose disappearance rate, k 2 is an insulin-dependent glucose disappearance rate, g 0 is a constant increase in plasma glucose concentration, k 3 is an insulin disappearance rate, k 4 is an insulin release rate per the average glucose concentration within the last τ minute.
[Table]
[Figure]
This system has a unique asymptotically stable equilibrium point ( G, I, X ) ≡ ( G ∗ , I ∗ , X ∗ ) on the nonlinear plain such that
[Equation]
Here we set the initial state of this model as G (0) =
G ∗ , I (0)
=
I ∗ , X (0) =
X ∗ and set the model parameters as shown in the following Table 3.
Furthermore, we adopt the output of the previous oral glucose absorption system [33] as u (See Fig.
7 (B)), and the glucose absorption amount u is normalized based on human blood volume per body weight (0.80[100 ml/kg]).