Machine Learning Modeling of Dynamical Systems
by using Biased Training Data
Yoshitaka Saiki (Hitotsubashi University)

composition of the input data u by using nonlinear function tanh:

1 Introduction.
Reservoir

computing,abrain-inspired

machine-learning technique that employs ar(t + ∆t) = (1 − α)r(t) + α tanh(Ar(t) + Win u(t)).
A, Win : sparse random matrix, whose maximal
eigenvalue is controlled.

data-driven dynamical system, is eﬀective in
Fixed

predicting time series and frequency spectra

The extent to which a data-driven model usingu(t)ditions. Zhu et al. [7] identified some unstable-30t∀ < T030 -30

30
15-30 -15

0015-1530 -30

Figure 3: Examples of three periodic trajectories
used as training data20∥(Wout r(l∆t)) − s(l∆t)∥1515 -30-1545
30
150Wout r(t + ∆t) ≈ s(t + ∆t).

200-15M<<N⇒ Determine them s.t. the form is minimized:
L∑

15-15

0-15

⃝ 2nd step (determination of output layer)
We determine Wout s.t.

parameters. Nakai and Saiki [3] confirmed that aries of chaotic fluid flow from various initial con-RR

30

15

Figure 1: Schematic picture of a reservoir
computing (training phase)original system under an appropriate choice of
single data-driven model could infer the time se-MN

R

properties of original systems should be determodel has an attractor similar to that of thet)

45
30
15030s(t+t)
outputdatareservoir

Mreservoir computing can capture the dynamical
mined. Lu et al. [1] reported that a data-driven

Wout , cr(t+input
data

45
30
150Training

Winin chaotic behaviors, including fluid flow andglobal atmospheric dynamics [1, 2, 4, 5, 6].

r(t)Ais much bigger than that of Rs) can be used as abiased training data. In this study we evaluate eachof the models trained from diﬀerent biased trainingdata.

T+ β[T r(Wout Wout )].

reservoir
actual10l=10periodic orbits of a data-driven model through-10

delayed feedback control. They suggested that adata-driven model could reconstruct the attractor of the original dynamical system.
This study clarifies that a data-driven modelusing reservoir computing has richer information

2.2 Procedure of inference.
∗ŝ(t) = Wout r(t)r(t + ∆t) = (1 − α)r(t) + α tanh(Ar(t) + Win ŝ(t)).
determined*Wout, c*Winu(t)r(t+2468t)s(t+t)

10

Figure 4: Time development of x of a model
trajectory in compared to that of an actual
trajectory
reservoirr(t)Acially from dynamical system point of view, sugunknown dynamical system can be estimated by0∗
Using the Wout, we infer the time-series s.

than that obtained from a training data, espegesting that dynamical properties of the original-20

actual604530150604530
15030-30 -15

150015-1530 -30

30-30 -15

150015-1530 -30

reservoir computing from a relatively short timeseries. The dynamical properties, such as Lyapunov exponents and manifold structures between stable and unstable manifolds, can be reconstructed by the data-driven model throughFigure 2: Schematic picture of a reservoir
computing (prediction phase)This reservoir system corresponds to the data-driven
model of u.

Referencereservoir computing.

3 Our project.
2 Reservoir computation.
What’s Reservoir computation?
• a relatively high-dimentional fixed neuralnetwork composed of simple nonlinear dynamical systems• determination of output layer• ForLorenzsystem

andKuramoto–Sivashinsky system, inference [1, 5, 6]

2.1 Procedure of training.
⃝ 1st step (making a reservoir vector)
Making a reservoir vector r(t) correspond to de-Figure 5: Projection of a long model trajectory andthat of a long actual trajectory

In our project by employing a reservoir computing we evaluate a dynamical model constructed froma biased training data. It is well known that aninfinitely many periodic orbits are embedded in achaotic attractor. We create training data by choosing a set of periodic orbits. In our preliminary study
we have confirmed that a model trained from a setof relatively low periods periodic orbits reconstructs
a trajectory which approximates the actual one fora certain amout of time. For the Lorenz systemwith classical parameter values, each periodic orbit
is coded by the symbol sequence L or R dependingon the sign of the x coordinate along a periodic orbit. A typical long trajectory has similar numberof Ls and Rs. Therefore, a periodic orbit with asymbol sequence such as L10 R1 (the number of Ls[1] Z. Lu, B. R. Hunt, and E. Ott, Chaos 28, 061104(2018).
[2] K. Nakai and Y. Saiki, Physical Review E 98,023111 (2018).
[3] K. Nakai and Y. Saiki, Discrete Contin. Dyn.
Syst. S 14, 3641079 (2021).
[4] M. U. Kobayashi, K. Nakai, Y. Saiki and N.
Tsutsumi, Physical Review E 104 (4), 044215(2021).
[5] J. Pathak, Z. Lu, B. Hunt, M. Girvan, and E.
Ott, Chaos 27, 121102 (2017).
[6] J. Pathak, B. Hunt, M. Girvan, Z. Lu, and E.
Ott, Phys. Rev. Lett. 120, 024102 (2018).
[7] Q. Zhu, H. Ma, and W. Lin, Chaos 29, 093125(2019).