[jh240051] Analysis of Mathematical Structure ofReservoir Computing Model
Yoshitaka Saiki (Hitotsubashi University)
108r(t)AFixedTraining

Wincomputing,abrain-inspired

machine-learning technique that employs adata-driven dynamical system, is effective inu(t)the original system under an appropriate choiceof parameters. Nakai and Saiki [3] confirmed

RR∀t <T-2-4-6

Wout r(t + ∆t) ≈ s(t + ∆t).

0driven model could reconstruct the attractor of

10t-T⇒ Determine them s.t. the form is minimized:
L∑

0.80.6T∥(Wout r(l∆t)) − s(l∆t)∥2 + β[T r(Wout Wout)].

l=10.40.2
02.2 Procedure of inference.

-0.2-0.4∗, we infer the time-series s.
Using the Wout

0510

constructed using reservoir computing from a∗r(t)ŝ(t) = Woutr(t + ∆t) = (1 − α)r(t) + α tanh(Ar(t) + Win ŝ(t)).

In particu-r(t)Alar, we focus on reconstructing the geometric
spectrum of the actual dynamical system un-u(t)r(t+t)

s(t+tive exponents, is reproduced within a relatively
small subspace in the machine learning modelobtained through reservoir computing.

10-10(i)-30λFigure 2: Schematic picture of a reservoir
computing (prediction phase)This reservoir system corresponds to the data-driven
model of u.

-50-70-900

2000

40006000i

2 Reservoir computation.
3 Our project.

What’s Reservoir computation?
• a relatively high-dimentional fixed neuralnetwork composed of simple nonlinear dynamical systems

The first example is the data-driven model of theLorenz [7], a dissipative N -dimensional ODE which
is a model of some oscillating scalar atmosphericquantity described as• determination of output layerand25t)

derlying the training data, including the nega-system

20

Figure 4: Autocorrelation functions for thedata-driven model and the actual system of theLorenz-96 system.

determined*Wout, c*Winstructure. We investigate that the Lyapunov

Lorenz15t

In our project, we analyze a data-driven model• For

20

reservoir output
actualthe original dynamical system.

dynamical system point of view.

15

Figure 3: Short time inference of a variable of theLorenz-96 systemtime series of chaotic fluid flow from various iniThey suggested that a data-

51

that a single data-driven model could infer thetial conditions.

20M<<N⃝ 2nd step (determination of output layer)
We determine Wout s.t.

ical properties of original systems should be dedriven model has an attractor similar to that ofM

Figure 1: Schematic picture of a reservoir
computing (training phase)

ing reservoir computing can capture the dynamtermined. Lu et al. [1] reported that a data-t)NR

4s(t+t)outputdata

reservoir

Min chaotic behaviors, including fluid flow andThe extent to which a data-driven model us-r(t+input
datapredicting time series and frequency spectra
global atmospheric dynamics [1, 2, 4, 5, 6].

Wout , ccorrelation

Reservoir

6x

1 Introduction.

reservoir output
actualKuramoto–

Sivashinsky system, inference [1, 5, 6]

2.1 Procedure of training.
⃝ 1st step (making a reservoir vector)
Making a reservoir vector r(t) correspond to decomposition of the input data u by using nonlinear function tanh:r(t + ∆t) = (1 − α)r(t) + α tanh(Ar(t) + Win u(t)).
A, Win : sparse random matrix, whose maximal
eigenvalue is controlled.

dxk= xk−1 (xk+1 −xk−2 )−xk +f, for k = 1, . . . , N,dt
(1)
where the system has cyclical symmetry so xN +k =xk for all k = 1, . . . , N , and where f is a forcing
parameter.
We analyze machine learning models obtained
through reservoir computing using examples such as
high-dimensional Lorenz systems, which are often
used in the field of meteorology. There are some parameters that need to be artificially set in machinelearning (so-called hyperparameters), and we aim toclarify the reproducibility of these settings and thestructures.

Figure 4: Lyapunov spectrum of the Lorenz-96
system.

Reference[1] Z. Lu, B. R. Hunt, and E. Ott, Chaos 28,061104 (2018).
[2] K. Nakai and Y. Saiki, Physical Review E 98,023111 (2018).
[3] K. Nakai and Y. Saiki, Discrete Contin. Dyn.
Syst. S 14, 3641079 (2021).
[4] M. U. Kobayashi, K. Nakai, Y. Saiki and N.
Tsutsumi, Physical Review E 104 (4), 044215(2021).
[5] J. Pathak, Z. Lu, B. Hunt, M. Girvan, and E.
Ott, Chaos 27, 121102 (2017).
[6] J. Pathak, B. Hunt, M. Girvan, Z. Lu, and E.
Ott, Phys. Rev. Lett. 120, 024102 (2018).
[7] E.N. Lorenz and K.A. Emanuel, J. Atmos. Sci.
45, 399–414 (1998).