S PARSE R EPRESENTATION OF KOOPMAN O PERATOR

Yuya Note
Department of Electrical and Electronic Engineering
Kobe University
Kobe, Japan

Takaharu Yaguchi
Department of Mathematics
Kobe University
Kobe, Japanyaguchi@pearl.kobe-u.ac.jp

Toshiaki Omori
Department of Electrical and Electronic Engineering
Kobe University
Kobe, Japanomori@eedept.kobe-u.ac.jp

A BSTRACTIn this paper we propose a method for estimating dynamical systems using a sparse representation of
the Koopman operator. The Koopman operator is inherently considered with infinite dimensional
observables. The proposed method imposes a sparse constraint on the elements of the matrix thatapproximates the Koopman operator. Experiments on nonlinear dynamical systems showed that the
proposed method was more accurate than the unconstrained method when the number of observables
was large. It is also shown that the Koopman operator is sparsely estimated with more zero elements.
Keywords Sparse Modeling, Koopman Operator, Nonlinear Dynamical System

1

Introduction

In recent years, advances in measurement and computational techniques have improved the quality of data, and also
the quantity of data has become enormous. These developments have led to a focus on data-driven approaches toextracting the structure and properties of systems from data. Data-driven approaches are important approaches for avariety of disciplines [1, 2], including natural sciences such as earth sciences [3, 4, 5, 6, 7] and neuroscience [8, 9], andfor engineering disciplines such as fluid engineering [10, 11, 12, 13] and thermal engineering [14, 15].
Nonlinear dynamical systems are often used to describe natural phenomena and engineering systems due to their ability
to model complex or highly interrelated behavior that is difficult to represent in linear dynamics. Because nonlinear
dynamical systems are represented by nonlinear functions, the correspondence between inputs and outputs becomescomplex, making it difficult to predict and analyze their behavior.
In the analysis of nonlinear dynamical systems, the Koopman operator has attracted much attention [16, 17, 18, 19, 20,
21, 22, 23]. The Koopman operator is a linear operator introduced for nonlinear dynamics. The Koopman operator
generally requires consideration of an infinite-dimensional function space, including a function called an observable,
but for actual calculations the Koopman operator must be approximated in a finite dimension. Therefore, when using
the Koopman operator, the choice of functions as observables is important.
In this paper we propose a method for estimating dynamical systems using a sparse representation of the Koopman
operator. This method imposes a regularization on the magnitude of the corresponding coefficient for each observableand performs sparse estimation. The results demonstrated that this method can achieve accurate estimation in long-term
forecasts even when a high-dimensional observable space is assumed, and that the Koopman operator can be represented
by a sparse matrix.
International Conference on Scientific Computing and Machine Learning 2024 (SCML2024)

Sparse Representation of Koopman Operator

2

Methodology

2.1

Koopman Operator

The Koopman operator allows linearization of nonlinear dynamical systems. In this case, the dynamics which isnonlinear in a finite dimension, is transformed into an infinite dimensional linear space by introducing observables. Fora state variable y(t), consider the following mapping that transfers one observation value to another:
KF (y(t)) = F (A(y(t)))(1)

where y ∈ Y is state variable, Y is finite-dimensional state space, A : Y → Y is nonlinear mapping in state space
Y , F : Y → R is nonlinear mapping in the space F of observables, and K : F → F is mapping from the space ofobservables to the space of different observables.
With respect to Eq. (1), the operator that specifically takes into account the time evolution of the state variable is called
the Koopman operator. Using the Koopman operator, the time evolution of the time-variant state variable y(t) can be
expressed as follows:
Ks F (y(t)) = F (y(t + s))(2)
where t and s are real numbers representing time, and Ks is the Koopman operator, which is the operator that leads tothe s-step forward time evolution. Since the Koopman operator is a linear operator, the following properties of linear
operators can be used:
Ks {c1 f1 (y(t)) + c2 f2 (y(t))} = c1 Ks f1 (y(t)) + c2 Ks f2 (y(t))(3)

where c1 , c2 ∈ R and f1 , f2 ∈ F.
When performing calculations using the Koopman operator, it is necessary to approximate an infinite-dimensional
operator with a finite-dimensional one due to the limitation of computational resources. Therefore it is important tointroduce a function space that is compatible with the characteristics of nonlinear dynamical systems for estimation
dynamical systems using the Koopman operator.
2.2

Proposed Method

In this paper we propose a method to select a function from a large number of candidate functions by using sparsity,
aiming to achieve accuracy in dynamical system estimation with a better approximation of the Koopman operator in
finite dimensions. Since the Koopman operator is a linear operator, it can be approximated using matrices as follows:
Ks ≈ [k1k2⊤

kp ] ∈ Rp×p

···

(4)

p

where p ∈ Z is the number of observables to define and k1 , · · · , kp ∈ R are elements of the Koopman operator ineach row.
Let the observables be f1 (y(t)), · · · , fp (y(t)), and focusing on fi (y(t)), it can be expressed as follows using theapproximation in Eq. (4):pXfi (y(t + s)) ≈ki,j fj (y(t))(5)
j=1

We estimate to be sparse with respect to this coefficient ki . We consider the following loss function with respect to thecoefficients ki corresponding to all observables(i = 1, ..., p):L(ki ) = fi (y(t + s)) −

p
X
j=1

2

+ λ∥ki ∥1

ki,j fj (y(t))(6)

2The sparse coefficients can be estimated by minimizing Eq. (6). By performing this minimization for each observable,
the Koopman operator is reconstructed as follows:⊤Ks⋆ ≈ k1⋆ k2⋆ · · · kp⋆ ∈ Rp×p
(7)
Estimated Koopman operator Ks⋆ has smaller values of elements, especially eliminating the effects of unnecessarycoefficients. The reproduction of the behavior of the state variable y(t) can be calculated by including y(t) as theobservable. Set f1 (y(t)) = y(t) and repeat the following lines of calculation:[y(t + s) f2 (y(t + s) · · ·⊤fp (t + s))] = Ks⋆ [y(t) f2 (y(t))···⊤

fp (y(t))](8)

By performing this calculation, the observable f1 (y(t)) = y(t) is the value corresponding to the behavior of the system.
2

Sparse Representation of Koopman Operator

3

Results

To evaluate the performance of the proposed method, experimental results are presented for two nonlinear dynamicalsystems: a single pendulum and a Duffing oscillator. The evaluation criterion is the long-term prediction accuracy when
only initial values are given, and the sparsity of the Koopman operator is also evaluated. To demonstrate the superiorityof the proposed method, we compare our method with the least squares method without regularization (LSM). The
hyperparameter λ of the proposed method was selected to provide the highest long-term forecast accuracy.
3.1

Single Pendulum

In this experiment, we consider the behavior of the state variable of a single pendulum, which is formulated by thefollowing simultaneous differential equations:dy1 (t)y2 (t)=dtmdy2 (t)= −mg sin(y1 (t))
dt

(9)
(10)

where y1 (t) and y2 (t) are the state variables and m and g are the parameters of the system. For training, data
points of time span 0 ≤ t ≤ 2 at
0.05 intervals were used. Polynomials up to the 10th order of the statevariables y1 (t) and y2 (t) were prepared in advance as the observed values.
The results of the long-term forecasts
given only the initial values are shownin Figure 1. LSM shows divergence as
early as in the prediction of the next
step of the initial value. On the other
hand, the proposed method reproduces
the behavior quite accurately over aspan up to t = 50, which is longer
than the training data; root mean squareerror (RMSE) is 2.69 × 10−5 , whichquantitatively guarantees the forecast’s
accuracy.
Next, Figure 2 shows a visualizationof the elements of the estimated Koopman operator. It can be seen that theKoopman operator estimated by LSM
has large absolute values of its elements. On the other hand, the operator estimated by the proposed method
has smaller absolute values than does
the operator estimated by LSM. In
particular, there are more zero elements: 0.02% (1 element) for LSM and52.07% (2200 elements) for the proposed method. In the method without
regularization, the fitting becomes complicated and positive and negative values tend to cancel each other out. On
the other hand, it can be seen that the
proposed method achieves the selection

Ours

0.6
0.4
0.2
y1 0.0
0.2
0.4
0.6

LSM

1
y2 0
1
0

10

20

t

30

40
exact

50 0
train

10

20
test

t

30

40

50

Figure 1: Comparison of long-term forecasts in a single pendulum.
True trajectories (black dashed lines) and predictions for the training
period (blue dots) and the test period (red crosses) are shown. The
results of long-term forecasts by LSM deviate from the true trajectories
after the next point of the initial values.

Ours

1

LSM
3

11

2

21

1

k31

0

41

1

51

2

61

3

Figure 2: Comparison of estimated Koopman operators in a single
pendulum. The operator estimated by the proposed method (left figure)and one estimated by LSM (right figure) are visualized in the form of Eq.
(4). As the element increases in the positive direction, the red becomesdarker, and as the element moves toward the negative direction, the blue
becomes darker. As the element approaches zero, it becomes whiter,and a completely zero element is represented by a black dot.

3

Sparse Representation of Koopman Operatorof observables by imparting sparsity and suppresses the appearance of elements that are unnecessary for reproducing
the behavior.
3.2

Duffing Oscillator

The Duffing oscillator formulation is asfollows:
dy1 (t)= y2 (t)dtdy2 (t)= −ay1 (t) − by13 (t)
dt

(11)
(12)

where y1 (t), y2 (t) are the state variables and a, b are the parameters of thesystem. For training, data points oftime span 0 ≤ t ≤ 4 at 0.05 intervals
were used. Polynomials up to the 10thorder of the state variables y1 (t) andy2 (t) were prepared in advance as theobserved values.
Figure 3 shows the results of the longterm forecasts given only the initial values As in the case of the single pendulum, the proposed method reproduces
the behavior quite accurately for spansup to t = 50, which is longer than
the training data, while LSM shows divergence at the prediction of the next
step of the initial value. RMSE of
the long-term forecasts of the proposed
method is 8.82 × 10−5 , which assuresthat the accuracy is quantitatively high.

Ours

2
1
y1 0
1
2
4
2
y2 0
2
4

0

10

20

t

30

LSM

40
exact

50 0

10
train

20
test

t

30

40

50

Figure 3: Comparison of long-term forecasts in Duffing oscillator. See
the caption for Figure 1.
1

Ours

LSM
15000

11

10000

21

5000

k31

0

41

5000

51

10000
15000

61

A visualization of the estimated KoopFigure 4: Comparison of estimated Koopman operators in Duffing
man operators is shown in Figure 4.
oscillator. See the caption for Figure 2.
The Koopman operator estimated by
LSM has large absolute values of theirelements, while the operator estimated by the proposed method has smaller absolute values than does the operator
estimated by LSM. Zero elements also occur more frequently: 0.07% (3 elements) for LSM and 46.84% (1979 elements)
for the proposed method. The matrix with more zero elements derived by the proposed method can be applied toalgorithms such as sparse matrices, which may lead to a reduction in the computational complexity of the simulation.

4

Conclusion

In this paper we propose a method for estimating dynamical systems using a sparse representation of a Koopman
operator. This is a data-driven estimation of the Koopman operator that imposes a constraint on the magnitude of
the absolute values of the elements, aiming to appropriately approximate a Koopman operator, which is originallyinfinite-dimensional, in finite dimensions. In the dynamical system estimation using a Koopman operator, when many
functions are assumed as observables, it is shown that the proposed method is considerably superior to the unconstrained
method in terms of long-term forecasts. It was also shown that Koopman operators with many zero elements were
estimated, and the possibility of applying the method to sparse matrices was also demonstrated.

Acknowledgments
This work was partially supported by Grant-in-Aid for Scientific Research (B) (No. JP21H03509), and a Fund for the
Promotion of Joint International Research (International Collaborative Research) (No. JP23KK0184), MEXT, Japan,
CREST (No. JPMJCR1914), JST, Japan.
4

Sparse Representation of Koopman Operator

References[1] Amin G. and Bogdan I. G. Data-driven prediction in dynamical systems: recent developments. Phil. Trans. R.
Soc. A., 380:20210213, 2022.
[2] Course K. and Nair P. B. State estimation of a physical system with unknown governing equations. Nature,622(7982):261–267, 2023.
[3] Filoche A., Béréziat D., and Charantonis A. Deep prior in variational assimilation to estimate an ocean circulation
without explicit regularization. Environmental Data Science, 2:e2, 2023.
[4] Yadav N., Ravela S., and Ganguly A. R. Machine Learning for Robust Identification of Complex Nonlinear
Dynamical Systems: Applications to Earth Systems Modeling. ArXiv, abs/2008.05590, 2020.
[5] Chattopadhyay A., Hassanzadeh P., and Subramanian D. Data-driven predictions of a multiscale Lorenz 96 chaotic
system using machine-learning methods: reservoir computing, artificial neural network, and long short-term
memory network. Nonlinear Processes in Geophysics, 27(3):373–389, 2020.
[6] Maulik R. Hamzi B. and Owhadi H. Simple, low-cost and accurate data-driven geophysical forecasting withlearned kernels. Proc. R. Soc. A., 477:20210326, 2021.
[7] Tandeo P., Aillio P., and Sévellec F. Data-driven reconstruction of partially observed dynamical systems. Nonlinear
Processes in Geophysics, 30(2):129–137, 2023.
[8] Omori T., Kuwatani T., Okamoto A., and Hukushima K. Bayesian inversion analysis of nonlinear dynamics insurface heterogeneous reactions. Phys. Rev. E, 94:033305, 2016.
[9] Oyanagi R., Kuwatani T., and Omori T. Exploration of nonlinear parallel heterogeneous reaction pathways through
Bayesian variable selection. Eur. Phys. J. B, 94:42, 2021.
[10] Agostini L. Exploration and prediction of fluid dynamical systems using auto-encoder technology. Physics of
Fluids, 32(6):067103, 2020.
[11] Cheng S., Lucor D., and Argaud J. P. Observation data compression for variational assimilation of dynamicalsystems. Journal of Computational Science, 53:101405, 2021.
[12] Yu C., Bi X., and Fan Y. Deep learning for fluid velocity field estimation: A review. Ocean Engineering,
271:113693, 2023.
[13] Jayaraman B. and Mamun S M A. A. On Data-Driven Sparse Sensing and Linear Estimation of Fluid Flows.
Sensors, 20(13), 2020.
[14] Sholahudin, Giannetti N., Yamaguchi S., Saito K., Miyaoka Y., Tanaka K., and Ogami H. Experimental
implementation of artificial neural network for cost effective and non-intrusive performance estimation of air
conditioning systems. Applied Thermal Engineering, 181:115985, 2020.
[15] Alonso-Bastida A., Franco-Nava M. A., Adam-Medina M., López-Zapata B., Álvarez Gutiérrez P. E., and GarcíaMorales J. Mathematical modeling of thermal interactions in a self-cooling pilot-scale photobioreactor. Case
Studies in Thermal Engineering, 31:101825, 2022.
[16] Brunton S. L., Budišić M., Kaiser E., and Nathan Kutz J. Modern Koopman Theory for Dynamical Systems.
SIAM Rev., 64:229–340, 2021.
[17] Otto S. E. and Rowley C. W. Koopman Operators for Estimation and Control of Dynamical Systems. Annual
Review of Control, Robotics, and Autonomous Systems, 4(1):59–87, 2021.
[18] Kostic V., Novelli P., Maurer A., Ciliberto C., Rosasco L., and Pontil M. Learning Dynamical Systems via
Koopman Operator Regression in Reproducing Kernel Hilbert Spaces. In S. Koyejo, S. Mohamed, A. Agarwal,
D. Belgrave, K. Cho, and A. Oh, editors, Advances in Neural Information Processing Systems, volume 35, pages
4017–4031. Curran Associates, Inc., 2022.
[19] Narendra K. S. and Parthasarathy K. Neural networks and dynamical systems. International Journal of Approximate Reasoning, 6(2):109–131, 1992.
[20] Bevanda P., Sosnowski S., and Hirche S. Koopman operator dynamical models: Learning, analysis and control.
Annual Reviews in Control, 52:197–212, 2021.
[21] Kaiser E., Kutz J. N., and Brunton S. L. Data-driven discovery of Koopman eigenfunctions for control. Machine
Learning: Science and Technology, 2(3):035023, 2021.
[22] Li M. and Jiang L. Deep learning nonlinear multiscale dynamic problems using Koopman operator. Journal ofComputational Physics, 446:110660, 2021.
[23] Korda M., Putinar M., and Mezić I. Data-driven spectral analysis of the Koopman operator. Applied andComputational Harmonic Analysis, 48(2):599–629, 2020.
5