Sharp Spectral Rates for Koopman Operator Learning
Authors: Vladimir R Kostic
Karim Lounici
Pietro Novelli
Massimiliano Pontil
Venue: NeurIPS 2023 spotlight (2023)
Keywords: Statistical Learning Theory
Dynamical Systems
TL;DR: We derive minimax optimal statistical rates for learning Koopman operator and combine them with spectral perturbation theory to provide strong guarantees for learning Koopman eigenvalues and eigenfunctions.

Abstract: Non-linear dynamical systems can be handily described by the associated Koopman operator, whose action evolves every observable of the system forward in time. Learning the Koopman operator and its spectral decomposition from data is enabled by a number of algorithms. In this work we present for the first time non-asymptotic learning bounds for the Koopman eigenvalues and eigenfunctions. 
We focus on time-reversal-invariant stochastic dynamical systems, including the important example of Langevin dynamics. 
We analyze two popular estimators: Extended Dynamic Mode Decomposition (EDMD) and Reduced Rank Regression (RRR). Our results critically hinge on novel {minimax} estimation bounds for the operator norm error, that may be of independent interest. Our spectral learning bounds are driven by the simultaneous control of the operator norm error and a novel metric distortion functional of the estimated eigenfunctions. The bounds indicates that both EDMD and RRR have similar variance, but EDMD suffers from a larger bias which might be detrimental to its learning rate. Our results shed new light on the emergence of spurious eigenvalues, an issue which is well known empirically. Numerical experiments illustrate the implications of the bounds in practice.

Sharp Spectral Rates for Koopman Operator Learning
Vladimir R. Kostic Istituto Italiano di Tecnologia University of Novi Sad kladimir.kostic@iit....
Karim Lounici CMAP-Ecole Polytechnique
karim. 1ounici@@oolyyyecciiige edu
Pietro Novelli
Massimiliano Pontil
Istituto Italiano di Tecnologia pietro novelli@iit
Istituto Italiano di Tecnologia
University College London
massimiliano httpllaannnooonneei11111 pontileiit.......
Abstract
Nonlinear dynamical systems can be handily described by the associated Koopman operator, whose action evolves every observable of the system forward in time. Learning the Koopman operator and its spectral decomposition from data is enabled by number of algorithms. In this work we present for the first time non-asymptotic learning bounds for the Koopman eigenvalues and eigenfunctions. We focus on www--reeeeal-ivaiiaaaa stochastic dynamical systems, including the important example of Langevin dynamics. We analyze two popular estimators: Extended Dynamic Mode Decomposition (EDMD) and Reduced Rank Regression (RRR). Our results critically hinge on novel minimax estimation bounds for the operator norm error, that may be of independent interest. Our spectral learning bounds are driven by the simultaneous control of the operator norm error and a novel metric distortion functional of the estimated eigenfunctions. The bounds indicates that both EDMD and RRR have similar variance, but EDMD suffers from larger bias which might be detrimental to its learning rate. Our results shed new light on the emergence of spurious eigenvalues, an issue which is well known empirically. Numerical experiments illustrate the implications of the bounds in practice.
Introduction
Recently, rescarchers have emphasized the utmost importance of developing moysically-informe machine learning models that prioritize interpretability and foster physical insight and intuition, see for example and references therein. One technique highlighted in these works is the Koopman operator regression framework to learn and interpret nonlinear dynamical systems see, e.g. M 20 and references therein. A key component this approach the Koopman Mode Decomposition (KMD), which decomposes complex dynamical systems into simpler, coherent structures. When ordinary least squares are used to learn Koopman operator from data, estimated KMD is known as the Dynamic Mode Decomposition (DMD) [36] Koopman operator estimators and their modal decomposition find many applications, including fluid dynamics, molecular kinetics and robotics 7 19
The Koopman operator returns the expected value of observables of the system in the future given the present, and one relies on estimators of this operator to in turn estimate its spectral decomposition that leads to the estimation of KMD. Our goal is to study the statistical properties of the eigenvalues and eigenfunctions of the Koopman operator estimators via two mainstream algorithms: Principal Component Regression (PCR) and Reduced Rank Regression (RRR) studied in [21 24 PCR en-
compasses as particular cases the popular Extended Dynamic Mode Decomposition (EDMD), which is the de- facto estimator in the data-driven dynamical system literature [see 26 45 and references
37th Conference on Neural Information Processing Systems (NcurIPS 2023).

---

therein]. Both PCR and RRR are kernel-based algorithms that, given dataset of observations of the dynamical system, implement strategy approximate the action of the Koopman operator on a reproducing kernel Hilbert space (RKHS)
3 38
We present for the first time non-asymptotic learning bounds on the distance between the Koopman cigenvalues and cigenfunctions and those estimated by either PCR or RRR. We show that the eigenvalues produced by such algorithms are biased estimators of the true Koopman eigenvalues, with PCR incurring larger bias. Our results critically hinge on novel estimation bounds for the operator norm error, that may be of independent interest, leading to minimax optimal bounds for finite-rank Koopman operators. Moreover, we introduce the novel notion of metric distortion, which characterize how the norm of eigenfunctions vary when moving from the RKHS in which learning takes place to the underlying ambient space where the Koopman operator is properly defined. We show that both the operator norm error and metric distortion are needed in order to estimate the operator spectra and our bounds can be used to explain the well-known spuriousness phenomena in eigenvalue estimation [4. namely, the scenario in which the estimated eigenvalues are not related the true ones, despite small operator norm error.
Contributions and Organization. We make the following contributions: i) We introduce the notion of metric distortion and show that it has to be used alongside the operator norm error to derive Koopman spectra estimation error bounds (Theorem I ii) We establish the first sharp estimation bound for the operator norm error (Theorem 2: iii) We establish spectral learning rates (Thms. 3 and for both PCR and RRR; iv) We propose how use the results entailed by Theorem to detect the presence of spurious eigenvalues from data.
The paper is organized follows. In Section 2 recall the notion of Koopman operator, its spectral decomposition, and review PCR and RRR estimators. Section 3 describes the estimation problem and outline our main results. Section A
presents our approach to bound eigenvalue and eigenvector estimation errors. Section 5 gives sharp upper bounds for the operator norm error. Section O presents our spectral learning bounds. Finally, Section illlsttrae the implications of the bounds in practice, and is designed to provide practitioners with the tools to benchmark the performance of algorithms in real scenarios.
7
2 Background
Dynamical Systems and Koopman Operator. In this work we study Markovian dynamical systems, that is collections of random variables (X,: L E IN], where X, represents the state at time I, taking values in some space X. We focus time- hime-homogencous (i.e. autonomous) systems hosting an
invariant measure π for which the Koopman operator 244 27
(A+f) = EIS(Xx+ =
= x]) E
is a well defined bounded linear operator on I.3 (X)), the space of square integrable functions on X relative to measure т. In the field of stochastic processes, ① is also known as the transfer operator and returns the expected value of f in the future given the present. This operator is is self-adjoint (i.e. As = AA) whenever dynamics is time-reversal invariant w.r.t. т, which is satisfied by many stochastic processes in the physical sciences.
Example 1 (Langevin Dynamics), Let X' = Rd and let 3 > O. The (overdamped) Langevin equation driven by potential U Rd + R is given by d.X. -U + /23-idW,, where We is a Wiener process. The invariant measure of this process is the Boltzman distribution n (dx) ox (0)) dr, and the associated Koopman operator self-adjoint
The Langevin equation models a wealth of phenomena, such the evolution of chemical and biological systems at thermal equilibrium [6], the mechanism regulating cell size in bacteria M. chemical reactions 251, the dynamics of synapses [I 42] stock market fluctuations [6] and many more. Furthermore, when Uxx) = 0|| | /22 (# > O), the Langevin equation reduces to the celebrated Ornstein- Ornstein-Uhlenbeek process t4 Chapter 6].
The operator ① evolves every observable of the system forward time. Since bounded and linear, it admits speciral decomposition, which plays central role in the analysis and interpretation of the dynamical system 26 as well as (nonlinear) control [2]. As [6), to study the spectral
decomposition we further assume that Aq is compact operator, which rules out the presence of

---

continuous and residual spectrum components and leads to
A = Eien Pi fi @ fi.
(2)
where (ni, Ji)iEN S R x (x) are Koopman eigenpairs, i.e. Ax = p4 St. Moreover, lims-poc (4 O and (ei form a complete orthonormal system of (X*). In the context of molecular dynamics, the leading eigenvalues and their eigenfunctions are key in the study of long- term dynamics and so-called meta-stable states [see, e.g..A...
Koopman Operator Regression in RKHS. Throughout the paper we let H. be an RKHS and let
k X X R be the associated kernel function. We let ó : X H be a feature map 38 such that k(x,x') = (o(x), o(x') for all xz x' E X. We consider RKHSs satisfying H. C (X)
BN Chapter
4.3], so that PCR and RRR approximate A (X) +
(X) with an operator G 7 - H. Notice that despite H. c L (2)), the two spaces have different metric structures, that is for all f.ge H C L2 one in general has (f.g) 9) x * (f.9)13(x)) In order to handle this ambiguity,
we introduce the injection operator S H + LA (2)) such that for all f E H. the object Sf is the element L (X)) which is pointwise equal to f E H. but endowed with the appropriate L (2) norm. With this in mind, the Koopman operator restricted to H is simply A,S, which is then estimated by SG for some G E HS (H). We will measure the operator norm error, , - SGI| This is in contrast to the more frequently used Filleer-Schhiie (HS) norm.
Koopman operator regression estimators are supervised learning algorithms learn the Koopman operator, in which input and output data are consecutive states of the system (X,, Xx+1) for some t E N. Since the Markov process is homooeneoosuu and stationary, the joint probability distribution of (Xe, X is the same for every t E N and we denote it by p. Furthermore, stationarity also implies that X₁ ~ т for all E N. Given a datase Dr, == (æi, yi |== of consecutive states, PCR and RRR are two different strategies to minimize, under fixed-rank constraint, the mean square error
RG) == 1
n Eiety ( - G ((::)l?
where G E HS (H), the space of Hilbert-Schhiid operator acting on H. PCR and RRR estimators are expressed as functions of the input and cross empirical covariances, defined respectively as
- (())) and T= Eien) ((x)))
Likewise, the population risk R(G) = F (ox.)wlllbb))) - G*)(X)1 and the population co- variance and cross convariance are C = E Ex (X), and T' (X))) tively. We note that by the reproducing kernel property one finds that C = S see e.g. 38
respec-
Two Important Estimators. We next briefly recall two operator regression estimators that we study in this paper. The Principal Component Regression (PCR) estimator works by first projecting the input data into the r-dimensional principal subspace of the covariance matrix Č. and then ordinary least squares are solved for such projected data, yielding the estimator [see e.g. 24
G = 1.
(4)
Here Ō₁ --:=0 7Iu and [-, denotes the r-truncated SVD. The population counterpart is GPCR IC,1,,1 where C2 ;= C' + 7/4- Note, however, that the empirical PCR estimator does not minimize the empirical risk 3 under the low-rank constraint.
The Reduced Rank Regression (RRR) algorithm, in contrast, is the exact minimizer of 3 under fixed (H)),
rank constraint. Specifically, RRR is defined as GRRR == arg min (R() + + G where the regularization term >||G lās is added to ensure stability, and (,H( denotes the set of bounded operators on H that have rank at most r. The closed form solution of the empirical RRR
estimator is 24
GARR 1,,1/10,,1/27
(5)
while the population counterpart is given by GRRR C, 1/2 11
Once either the PCR or RRR estimators are fitted, their spectral decomposition is a proxy for the spectral dccomposition of the Koopman operator Aç. Theorem 2 in [24] shows how such a decomposition can be calculated via the kernel trick for both ČPCR and ČRRR
For simplicity we consider the i.i.d. setting, however our forthcoming analysis is directly applicable to sample trajectories following 2
3

---

Good kernel
Bad kornel
Ugly kernel
c
Reduced Rank (RRR)
Principal Components (PCR) a.k.a. EDMD
Figure 1: PCR vs. RRR in estimating the largest eigenvalues of the 1D Crmstein-Uhlentlee process with three different kemnels over 50 independent trials. Vertical lines correspond to Koopman eigenvalues. The good kernel is such that its H. corresponds to the leading eigenspace of the Koopman operator, while the other two are spans of scaled and permuted eigenfunctions for which the distortion with respect to the original metric structure of A introduce slow (bad kernel) and fast (ugly kernel) spectral decay of the covariance.
The Problem and Main Result in a Nutshell
In this section introduce the spectral estimation problem, outline main results in distilled form, and discuss some important implications. Recall the definition of Koopman operator ① and its spectral decomposition 2 Given a rank estimator G E B,H of As, we let (A,, ) be its spectral decomposition, satisfying Gip, = X, $5. We aim to study how well nonzero eigenvalue A of G estimates its closest Koopman eigenvalue Py((). where
j(i) = argmin 2 -yl-
(6)
Moreover we wish compare v with the corresponding true Koopman eigenfunction. To this end, we embed di in (2*) by means of the operator and define the normalized estimated eigenfunction
= S / ||
(7)
One of the key quantities studied in this work is the eigenvalue estimation error [X, Recalling that A, is compact and self-adjoint, the classical Davis-Kahai result 17 implies that the eigenvalue estimation error [2, M)(6)) also bounds the quality of the eigenfunction approximation as
1309-= i E [r].
2/²x M569]
I f0
ggap ((0) (Ax)
1X; M500
where =Ain
is the distance between 24 and its closest Koopman eigenvalue.
Let (.) denotes the j-th singular value of an operator. To give flavour of our results, here we report spectral bounds for the Gaussian kernel. this case, Theorem below gives a high probability bound on the estimation error [7, M. that of order
o ( ( or (A-S))
Vn) for GRRR and O ( ( (A+ - (S+1 + +/m for GPCR
(S)
If the Koopman operator has finite rank then 0r (A, =0 the RRR estimator is unbiased, and its error goes to zero at the rate 1/ Otherwise, recalling that (S) is the square root of the (r 1)- th eigenvalue of the kernel operator 38 Chapter 4.5], if 76 is infinite dimensional oy+1 (S) > 0, i.c. PCR has a strictly positive bias. In general, the presence a bias in the estimated eigenvalues may result in the appearance of spurious eigenvalues. This phenomenon for PCR is well documented in practice, see e.g. 1K] 14 28 In Figure T we illustrate such an effect on a simple dynamical
system discussed both in Example B and in Scction 7
4 Approach
The core of our analysis Theorem It reveals that in order to derive spectral estimation bounds for the Koopman operator, it is not enough study the excess risk in the HS norm. Indeed, spectral

---

bounds are determined by both the operator norm error of the Koopman estimator
(() =
- -A,5-5G], & E HS
(9)
and the metric distortion between H and (2)),
T/hh= = /|S|| h e H.
(10)
Note that since Sh (2') is just an equivalence class of function h, ||Sh|] is simply L (x)) norm of h, and hence the metric distortion can be written, with a slight abuse of notation, as m(h) := ||A]]] b (x). While the (HS norm) error was studied before [see 2y] and references therein], little is know about operator norm error bounds. Moreover, the metric distortion is, to the best of our knowledge, novel quantity in the spectral analysis of Koopman operator.
Theorem 1. Let Ax be a self-adjoint compact operator and let E N. Then, for every empirical estimator G E (H) and every i E [r]
2n(45s) E(G)
15 Pstol s (())) E(G), and II fooll gap (()) (Ax) - n(0r) E(G)]+
(11)
Proof Sketch. First, note that for compact self-adjoint operators 1A 19(0 S - So, following the reasoning of [24] Theorem 1] and observing that [IA,5- - l1 =
((G)) gives the right hand side of the first equation in (11. Next, since additionally 11(A,5 &
|| s (G)), we can apply the Davis-Kahan spectral perturbation result for
compact self-adjoint operators (Proposition Appendix C to bound sin(O), where 0, == < f5a)) 2 The claim then follows since llf: 1/ < cos(O.)) s 2 (.,)) The full proof can be found in Appendix C
Note that the error P at least for universal kernels, can be made arbitrary small, see 124 Proposi- tion 1]. Still, the metric distortion may dominate the error and, since the bound [T] is tight, one may have that the operator is well estimated in norm, but the estimated eigenpairs are far from the true ones. This phenomenon is at the origin of spurious eigenvalues. 'he proposed way to detect them for deterministic systems in [14] is to check if eigenvalue equations are satisfied empirically, which, however, is not useful for stochastic systems, see Rem. of Appendix C
Spuriousness may also originate from poor conditioning of the true eigenvalues, i.c. when the angle between true left and right eigenfunctions is small. Here, however, we assume A, = Ar, so that we restrict ourselves to the case in which the only source of spuriousness is due to the learning method.
While we defer the discussion of the operator norm error to the next section, the following result bound the metric distortion; the proof can be found in Appendix C
Proposition 1. Let G E B,H.). For all E [r] the metric distortion of 6r can be lightly bounded as
1 1/VIICI S vV(i)) I min([Sal (2s) [|ā]]) / o (SG).
(12)
where cond (x)) 1/ Ei) is the condition number of XE and & is its left eigenfunction.
The upper bound (22) depends on the estimator's eigenvalues and their conditioning. Notice that while the true eigenvalues of A, have condition number one, the conditioning of the estimated ones depends on the choice of the kernel. Morcover, the upper bound can be controlled by tuning the estimator rank. Since the bound is tight (see Rem. in Appendix C). the metric distortion can grow with the rank of the estimator, further motivating the use of low-rank estimators of A, in practice, (26].
We end this section by introducing an empirical estimator of the metric distortion n/(5s,, given by
n/ = 1 V(Oe, V.)
(13)
Proposition 4 of Appendix C shows that 7JS can be efficiently computed and report upper bounds for concentration around its mean. The empirical metric distortion (3), used in conjunction with the spectral bounds in Theorem g below, provides proxy assess the reliability of the PCR and RRR estimators and can be successtully used as novel model selection criterion. We refer the reader to the second and third experiment in Section 7 for concrete use-cases.
5

---

5
Controlling the Operator Norm Error
The HS norm error of the PCR estimator was already studied, either in the "well-specified" setting. i.c. when there exists Gz E HS (H) such that A,S = SGH, i.e. Gv is æ-a.e. Koopman operator [12 Theorem B. B.10]. On the other hand, KRR estimator is studied also in the "misspecified setting" 129
But, up to our knowledge, the operator norm error has not yet been studied. To analyse these learning rates, we make the following assumptions:
(RC) Regularity of Aq. For some a E (0,2) there exists a > such that TT 1 t2:Cto;;
(BK) Boundedness. There exists eg 0 such that ess sup [x]]) S CH i.c. L (X, H));
1/8 (SD) Spectral Decay. There exists 8 E (0, 1] and > such that ²y(C) S j 1- for all j E J.
While we keep assumptions ( [BK) and (SD) as in 18 29 assumption (RC) is. up to our knowledge, novel. The rationale behind it is that for
equivalent Im S) C Im((), in which case there exists a bounded æ-..e. Koopman operator GH: H H 124.. On the other hand, as
(RC)
a- (RC) becomes closer S) C
(S)) which always satisfied for universal kernels
since cl cl(m( (S)) = L3 (X') 33 Chapter 4]. Importantly, as the next example shows, RCC) is weaker condition than the usual regularity conditions; see Appendix D.] a detailed discussion.
Example 2. Let X be X-valued random variable with law x. Consider the Markov chain (Xi)teN such that X₁ = X for all t E N. Then n is an invariant measure and A = Ix2( is the identity map on L (2). Clearly. (RC) holds for all a E (0. On the other hand, since S ffr bounded operator GH 1x & HS (H). HS-norm learning rates derived in [29] do not apply.
In order to study the error of any empirical finite rank estimator G we rely on the error decomposition
E(G) < ||A,S - - ||S(G,
GYلl + ||S(G ر|-
(14)
regularization bias
rank reductionn bias
estimatio's variance
where G=== =,:= 1 is the minimizer of the full (i.e. without rank constraint), Tikhonov regularized, HS norm error, and G is the population version of the empirical estimator G.
While the last two terms in the r.h.s. of [44 depend of the estimator of choice, the first term depends only on the choice of H and the regularity of Ag w.r.t. H. In this work we focus on the classical where chooses universal kernel [3 Chapter 4] for which Im(A,)) C (I()) and controls the regularization bias with a regularity condition. For details see Rem. of Appendix D.2.
kemel-bbaeed learning of the Koopman operator 2 24 29
The second source of bias and the estimator's variance in our error decomposition depends on the choice of the low rank estimator. While throughout this section we consider (RC) for E [1. 2]. we discuss extensions of our results to a < 1 in Appendix D.5]
Theorem 2. Assume the operator Ag satisfies o (A, > o (A,) 2 0 for some r E N. Let
(SD) and (RC) hold for some 3 E (0, 1] and a E [1,2], respectively, and let cl(Im(S)) = L (x). Let
7 at and e == 26ो...
(15)
Let ó E (0. 1). Then, there exists a constant e> 0, depending only on H. such that for large enough r, with probability least 1 8 in the i...d draw Dr, from p
o a (A, (A, In5-1
if G - =GRRR T.y7"
(16a)
E(G) <
a (S) + Caa Ind-1 if G = GPCR and o,( > (5)1 n.
=GGRR
(16b)
Proof Sketch. The regularization bias is bounded by a 7 by Proposition 5 of Appendix D.2 For the RRR estimator, the rank reduction bias is upper bounded by 1 while for PCR by (++1(()) The bounds on the variance terms critically rely on the well-known perturbation result for spectral projectors reported in Proposition 3 Appendix A This result is then chained to two versions of the Bernstein inequality in separable Hilbert spaces. The first one is
inequality and the second is Minsker's inequality extended to self-adjoint HS-operators, Props. 2 and M in Appendix D.3.I respectively. These inequalities provide high probability bounds for the norms

---

C)C, as well as CT and ( TTC of C. CC-))(C--) and C -C- Combining the bias due to regularization and variance terms, for both estimators we obtain the balancing equation y 2 n Ind which yields the optimal choice of y and the rates. D
We stress that the number of samples in the previous theorem depends on the problem's 's complexity, expressed in the constants CRRR <?(A, (=(A=5)===_1AA55)) and CPCR = --(5)-0-=1($) Namely, the better the separation of singular values, the smaller number of needed samples. Furthermore, analyzing the bounds (T6a) and (T6b), we see that faster spectral decay is, in general, preferable. For example, for the Gaussian kernel 3 can be chosen arbitrarily small, yielding the rate 1/22, On the other hand, kernels with slow spectral decay for which 3 = 1 can give slower rates between n 11/4 and n -1133, Finally, from the variance bounds for RRR and PCR, c.f. Appendix D.3 one can specify constants. Namely, in the slower regime when e > n 1/22 we have that c a 7.2 log((1) (200) 1 + acz
v1=z); while in fastest regime ea n-11/2 there is significant difference between RRR and PCR since should be multiplied with the constants CRRR and CPCR, respectively.
(a- (oe-))/3( ( (VCH A
6a/2
As argued in Section 3 the bound (6a)) indicates that for rank Koopman operators the error converges to zero w.r.t. the number of training samples, while the bias PCR is strictly positive. Hence, order to ensure small error for PCR. high values of the rank parameter might be necessary. To theoretically explain this effect, in Theorem o of Appendix D.4 we give also lower bounds of operator norm error for the RRR and PCR estimators showing that E(GRRR) always concentrates
around -,- 1 (A, while the concentration of E risk of the learning problem. To illustrate the tightness of the error concentration bounds present Example 3 (see also Appendix D.4]
(PCR) around 0 (5) depends on the irreducible
Example 3. Let X =R. Consider the ID equidistant sampling of the Ornstein-Uhlenbee process, obtained by integrating the Langevin equation of Example ① with 3 1 and Uxx) x² /2, given by x, vi e 6t where (41)12 are i.i.d. standard Gaussians. For this process it is well-known (34] that π is N ( 1) and that A₂ admits a spectral decomposition (1is, J KeN n in terms of Hermite polynomials. We study the family of kernel functions ( 22) Σien ( fi where II a permutation of the indices of the eigenvalues and v is scaling factor. The rationale behind this class of kernels is that by varying Il and v one morphs the original metric structure of An way which is harder and harder to revert when learning from finite sets of data. In particular, for any target rank r, setting == 1/22 and II to the permutation such that i + 2r - i rr), i Hii r (r + S is 2r) and +> i elsewhere, elementary algebra and our concentration bounds give
EE [(GPCR)
- e-1/1 1/22
|E(GRRR)
-e-
We refer the reader to Figure 7 Mand Section 7 for numerical implementation of this example.
We conclude this section with remarks on the tightness of our statistical analysis of operator norm error. Since discussed results are not the main focus of the paper, we present them in Appendix D.5
Remark 1 (Lower bound). The rate e = 2(a+ guaranteed by (6a) matches the minimax lower bound for the operator norm error when learning finite rank Az. Formal statement and its proof is given in Theorem 7 of Appendix D.5
Remark 2 (Extension to misspecified setting). The optimal rates for HS-norm error of the KRR estimator are developed in (29) under stronger condition than [RC] In Theorem of Appendix D.6 we extended this analysis to PCR and RRR estimators, deriving the optimal operator norm rates that
also cover cases when Koopman operator cannot be properly defined as bounded operator on the chosen RKHS space H.
6 Spectral Learning Rates
Collecting all the previous results, we are now ready to present our spectral learning rates for the two estimators in general form. For brevity, we focus two different type of bounds in which (i) we analyse the uniform bound for the whole estimated spectra, and (ii) we express the estimators' bias in empirical form to provide insight into spuriousness of eigenvalues. Moreover, present only
7

---

eigenvalue estimation bounds, noting that the eigenfunction estimation bounds readily follow from The complete results are presented detail in Appendix E
Theorem 3. Let Ar be a compact self-adjoint operator. Under the assumptions of Theorem there exists a constant e > O, depending only on H., such that for every 8 E (0, 1), for every large enough and every i [r] with probability at least 1 - 8 in the i.i.d. draw of Dn from p
(A-41(A-S) (A + CEE 1n
if G
RRRR r:7
[X Pyt0l S
(17)
20-11(9) (o (5) CE Ind if G -=
GPCR f;
The uniform eigenvalue learning rates for RRR and PCR estimators, differ in the estimator's bias. While the PCR bias has a factor 1 (S) in the numerator, RRR has 0r+1 (A,5) S (5)) The striking difference happens when the Koopman operator is of finite rank. Then, assuming that is properly chosen, RRR estimator has no bias, and CuRR is typically moderate. On the other hand, PCR's 's bias can be potentially large, depending of the choice of the kernel, and choosing higher rank increases CPCR, thus requiring larger sample sizes. Therefore, even in well-conditioned problems (self-adjoint operator) the spurious eigenvalues may arise purely from the learning method. To facilitate detection of such occurrences, we further provide an empirical estimator of the bias of both methods and illustrate their experimentally in Section 7
Theorem 4. Under the assumptions of Theorems 2 anddy
3 there exists constant e > O, depending
only on H. such that for large enough n r and every [r] with probability at least 1 in the ii.dd draw of D from
M Mor+i (C-1/27) + ce Ind-1,
G
CRRR G
A - 103000 S
(18)
T1 V 0r+1((C) c In8-1,
GPCR
We remark that when Ag of finite rank r, the bound above for the RRR estimator reduces to
2 2e In gap, (GRRR) - 3 3cet Ino-1]+
A p4l S Ce In8 and Il fill?
Cor. in Appendix Hence, in this case RRR algorithm can learn all the eigenvalues and eigenfunctions of A, with rate E 20+3) On the other hand, even in this case, the bounds for the PCR estimator do not guarantee unbiased estimation of Koopman eigenvalues and cigenfunctions.
Choosing y and r. The bias term 0++1 (A=S)/o, aa/ (A,,S) appearing in represents the theoretical limit when estimating eigenvalues using RRR. It reflects the capacity of the RKHS to detect the separation of the leading Koopman eigenvalues from the rest of its spectra. If A, has infinite rank and slowly decaying cigenvalues, estimating the leading ones becomes challenging, since increasing r leads to smaller operator norm error, but larger bias. Luckily, in many practical problems there is a separation of time-scales in the dynamics and the above ratio can be controlled by choosing appropriately. While we do not have access to Sz we can still choose r via the empirical operator C-1/27, see Proposition 20 of Appendix D.4 Note also that the optimal y depends on a which is typically unknown. In practice, one implement standard grid-search CV procedure for time series to tune this parameter.
Spectral Bias as Tool for Model Selection. In equation S: (GRRR) == ar- (r-1(C-1/22) and 8, (GPCR) n or+ (C) represent the empirical spectral biases of RRR and PCR estimators of the Koopman operator, respectively. When they are small enough, the spectral estimation error is dominated by the same variance term, which decreases the number of samples grows. Therefore, given a number of different kernels, we propose to select the best one (w.r.t. spectral estimation) by choosing the smallest spectral bias. This is illustrated in the Alenine Dipeptide example of following section.
18 the data dependent quantities
Normal operators. Since Kahan theorem 17 also holds for normal operators, the results in this section apply whenever A, A Az. While in this case Koopman eigenfunctions remain orthogonal in (X'), the eigenvalues are in general complex. On the other hand, extension beyond normal compact operators asks for involved spectral perturbation analysis and new statistical learning theory.
8

---

V/2
4/4
0.0
0.002
0.004
0.010
0.018
0.5
Ground truth
RRR
-1.0
PCR
-1.5
Figure 2: Estimated eigenfunctions Vi of a Langevin dynamics vs. ground truth. The average empirical biases 5; i [4] are discussed at the end of Section The results correspond to 50 independent estimations on 2000 training points each. PCR and RRR estimators were fitted with the same parameters: Gaussian kernel of length scale 0.175, y 10-5 and 4.
7
Experiments
We illustrate various aspects of our theory with simple experiments. They have been imple- mented in Python using the library Kooplearn (available at atlttts:///itthu. www.coM.mme....... -IT-LLII kooplearn) to fit the PCR and RRR estimators. Full details are in Appendix
Learning the Spectrum of the Orastein-Uhlenben different kernel functions (the "good", the "bad" and the "ugly") to illustrate how an unseemly kernel choice can induce catastrophic biases in the estimation of Koopman eigenvalues. We focus on the uniformly sampled OrnsteinUUhlenbee Uhlenbeck (OU) process, discussed in Example 3 relying on the spectral decomposition of its Koopman operator (pi ft) to design the three kernel functions. The good kernel is just the sum of the leading T = 53 terms of the spectral decomposition of A,, i.e. kgood (ar, y) == E (z)(()))(())) The associated RKHS coincides with the leading eigenspace of A,, and no deformation of the metric structure takes place, so that the injection map S: H - 12 a partial isometry. The bad kernel defined according to the construction presented in Example for 1/r² where is the rank of the estimator. For this kernel, the introduced bias is innocuous for RRR, but lethal for PCR. Finally, the ugly kernel corresponds to v = ,² introducing large quotients 0r41 (A,S)/o, (A, and 0 (S) (S)). and, hence, an irreparable bias in both estimators.
Process. In this experiment designed three
Figure I depicts the distribution the eigenvalues estimated by PCR and RRR over 50 independent simulations, against the ground truth. For both algorithms each simulation is comprised of 20000 training points, the regularization y 10 and the rank is r 3. The three largest eigenvalues of A, are correctly estimated by both algorithms for Kgood and by RRR for kbad- On the contrary, the distribution of the eigenvalues for kugly (and kbad for PCR) does not concentrate around any true eigenvalue of A,, signaling the presence of spurious eigenvalues in the estimation.
Model selection for Alanine Dipeptide
estimator
RMSE:
10
Forecast borizon [us]
Figure 3: Forecasting RMSE on the Alanine Dipeptide dataset for 19 different RRR estimators, each corresponding to different kernel, which show how the best model, according to the empirical spectral bias metric, also attains the best forecasting performances by a large margin.

---

A Realistic Example: Langevin Dynamics, Because of its ubiquitous use in modelling real systems, we now study a numerical implementation of the Langevin dynamics Example D with 3 = 1 and a potential U() = 4(x +0.8e 80x² +0.2c 50(=-0.5)* + 0.5e (0(e-0000000) that is a mixture of three Gaussians barriers
E (-0.5,0,0000 0.5.) and smooth "bounding" term x constraining most of the equilibrium distribution in the interval 1,, see 37. In Figure 2 for i E [r], we compare the
fr estimated by PCR and RRR against the ground truth ft. The visible difficulty of PCR compared to RRR in estimating eigenfunctions is nicely explained by larger values of the empirical bias for PCR, which we report in the upper part of the figure. The reference eigenpairs of Arr have been obtained by diagonalizing a finely discretized approximation of the infinitesimal generator (see Appendix (A].
Spectral Bias and Model Selection: the Case of Alanine Dipeptide. In this example we show that minimizing the first term on the r.h.s. of (18) over a validation dataset, is also good criterion for Koopman model selection. We use realistic simulation of the small molecule Alanine Dipeptide already discussed in
different kernel and then we evaluated the forecasting RMSE on 2000 initial conditions drawn from test dataset. In Figure we report these errors, highlighting the model with the smallest average empirical spectral bias 18 evaluated on 5000 validation points.
2 4
We trained 19 RRR estimators each corresponding
8 Conclusion
We established minimax optimal rates for the operator norm error in the Koopman regression problem, which we then used to derive sharp estimation bounds for eigenvalues and eigenfunctions of the Koopman operator associated with a time-invariant Markov chain. We considered two important estimators that implement either principal component regression (PCR) or reduced rank regression (RRR) to learn linear operator on a reproducing kernel Hilbert space. Our bounds indicate that RRR may be advantageous over PCR (also known as EDMD, the de-facto estimator in the data-driven dynamical system literature) which may exhibit a larger estimation bias. This ultimately depends on the choice of the kernel, which significantly impacts the rate. A bad choice of the kernel could also introduce spurious cigenvalues, a phenomena which has been observed in the literature and which is now explained by our theory. Finally, we proposed a method to detect spuriousness in practice, which can be used also kernel selection tool. A limitation of this work is that it applies to compact normal operators only. While many real dynamical systems involve such operators, in the future our analysis may be extended using more sophisticated spectral perturbation theory.
Ackmowledgements. This work was supported in part from the PNRR MUR Project PE000013 CUP J532220000000000 "Future Artificial Intelligence Research (FAIR)", funded by the European Union - NexIGenerationUU, and EU Project ELIAS under grant agreement No. 101120237.
References
[1] Amir, A. (2014). Cell size regulation in bacteria. Physical Review Letters, 112(20).
[2] Arbabi, H. and Mezić, (2017). Ergodic theory, dynamic mode decomposition, and computation of spectral properties of the koopman operator. SIAM Journal Applied Dynamical Systems, 16(4):2096-2122
[3] Aronszajn, N. (1950). Theory of reproducing kernels. Transactions of the American Mathematical Sociery, 68(3):337-404
[4] Bakry, D., Gentil, I., and Ledoux, M. (2014). Analysis and Geometry of Markov Diffusion Operators. Springer.
[5] Bonati, L., Piccini, G., and Parrinello, M. (2021). Deep learning the slow modes for rare events sampling. Proceedings of the National Academy of Sciences, 118(44).
[6] Bouchaud, J.-P. and Cont, R. (1998). A langevin approach to stock market fluctuations and crashes. The European Physical Journal B, 6(4):543-550.
[7] Bruder, D., Fu, X., Gillespie, R. B., Remy, C. D., and Vasudevan, R. (2021). Data-Driven Control of Soft Robots Using Koopman Operator Theory. IEEE Transactions on Robotics, 37(3):948 91.161.
10

---

[8] Brunton, S. L., Budišić, M., Kaiser, E., and Kutz, J. N. (2022). Modern Koopman Theory for Dynamical Systems. SIAM Review, 64(2):229-344
[9] Budišić, M., Mohr, R., and Mezić, I. (2012). Applied Koopmanism. Chaos: An Interdisciplinary Journal of Nonlinear Science, 22(4):047510.
[10] Caponnetto, A. and De Vito, E. (2007). Optimal rates for the regularized least-squares algorithm. Foundations of Computational Mathematics, 7(3):331-368.
[11] Choquet, D. and Triller, A. (2013). The dynamic synapse. Neuron, 80(3):691-703.
[12] Ciliberto, C., Rosasco, L., and Rudi, (2020). general framework for consistent structured prediction with implicit loss embeddings. Journal of Machine Learning Research, 21(98):
[13] Colbrook, M. J., Roman, B., and Hansen, A. C. (2019). How to compute spectra with error control. Physical Review Letters, 122(25).
[14] Colbrook, M. J. and Townsend, A. (2021). Rigorous data-driven computation of spectral properties of koopman operators for dynamical systems.
[15] Da Prato, G. and Zabczyk, J. (1996). Ergodicity for Infinite Dimensional Systems. London Mathematical Society Lecture Note Series. Cambridge University Press.
[16] Davidchack, R. L., Ouldridge, E., and Tretyakov, M. V. (2015). New langevin and gradient thermostats for rigid body dynamics. The Journal of Chemical Physics, 142(14):144114,
[17] Davis, C. and Kahan, W. M. (1970). The rotation of eigenvectors by perturbation. iii. SIAM Journal on Numerical Analysis, 7(1):1-66.
[18] Fischer, S. and Steinwart, I. (2020). Sobolev norm learning rates for regularized least-squares algorithms. Journal of Machine Learning Research, 21(255):
[19] Folkestad, C., Chen, Y., Ames, A. D., and Burdick, J. W. (2021). Data-Driven Safety-Crition Control: Synthesizing Control Barrier Functions with Koopman Operators. IEEE Control Systems Letters, 20):2011-
[20] Grünewälder, S., Lever, G., Gretton, A., Baldassarre, L.., Patterson, S., and Pontil, M. (2012). Conditional mean embeddings as regressors. In ICML. icml.ec Omnipress.
[21] Héas, P. and Herzet, C. (2021). Low-rank dynamic mode decomposition: An exact and tractable solution. Journal of Nonlinear Science, 32(1).
[22] Karniadakis, G. E., Kevrekidis, I. G., Lu, L., and et al. (2021). Physics-informed machine learning, Nature Reviews Physics, 3(6):422-440.
[23] Klus, S., Nüske, F., Koltai, P., Wu, H., Kevrekidis, I., Schütte, C., and Noé, (2018). Data- driven model reduction and transfer operator approximation. Journal of Nonlinear Science, 2803):955-1010
[24] Kostic, V., Novelli, P., Maurer, A., Ciliberto, C., Rosasco, L., and Pontil, M. (2022). Learning dynamical systems via Koopman operator regression in reproducing kernel hilbert spaces. In Advances in Neural Information Processing Systems.
[25] Kramers, H. (1940). Brownian motion in field of force and the diffusion model of chemical reactions. Physica, 7(4):284-304.
[26] Kutz, J. N., Brunton, S. L., Brunton, B. W., and Proctor, L. (2016). Dynamic Mode Decompo- sition. Society for Industrial and Applied Mathematics.
[27] Lasota, A. and Mackey, M. (1994). Chaos, Fractals, and Noise, volume 97 of Applied Mathematical Sciences. Springer New York.
[28] Lewin, M. and Séré, é. (2009). Spectral pollution and how to avoid Proceedings of the London Mathematical Society, 100(3):864-000

---

[29] Li, Z., Meunier, D.. Mollenhauer, M., and Gretton, A. (2022). Optimal rates for regularized conditional mean embedding learning. In Advances in Neural Information Processing Systems.
[30] Mardt, A., Pasquali, L., Wu, H., and Noé, (2018). VAMPnets for deep learning molecular kinetics. Nature Communications, 9(1).
[31] Mauroy, A., Mezić, I., and Susuki, Y., editors (2020). The Koopman Operator in Systems and Control. Springer.
[32] Meyn, S. P. and Tweedie, R. L.. (1993). Markov Chains and Stochastic Stability. Communica- tions and Control Engineering. Springer London.
[33] Minsker, S. (2017). On some extensions of Bernstein's 's inequality for self-adjoint operators. Statistics & Probability Letters, 127:111-119.
[34] Pavliotis, G. A. (2014). Stochastic Processes and Applications. Springer New York.
[35] Reed, M. and Simon, B. (1980). I: Functional Analysis. Academic Press.
[36] Rowley, C. W., Mezić, Bagheri, S., Schlatter, P., and Henningson, D. S. (2009). Spectral analysis of nonlinear flows. J. Fluid Mech., 641:115-127.
[37] Schwantes, C. R. and Pande, V. S. (2015). Modeling Molecular Kinetics with tCCA and the Kerne Trick. Journal of Chemical Theory and Computation, 11(2):600-60) 608.
[38]
Steinwart, I. and Christmann, (2008). Support Vector Machines. Springer New York.
[39] Tropp, J. A. (2012). User-friendly tail bounds for sums random matrices. Technical report.
[40] Tuckerman, M. (2010). Staristical Mechanics: Theory and Molecular Simulation. Oxford university press.
[41] Varadhan, S. R. (2007). Additional remarks. In Courant Lecture Notes, pages 121-121. American Mathematical Society, Providence, Rhode Island.
[42] Ventriglia, F. and Maio, V. D. (2000). A brownian model of glutamate diffusion in excitatory synapses of hippocampus. Biosystems, 58(1-3):67-74.
[43] Wang, Z., Luo, Y., Li, Y., Zhu, J., and Schölkopf, B. (2022). Spectral representation learning for conditional moment models.
[44] Wehmeyer, C. and Noé, (2018). Time-lagged autoencoders: Deep learning of slow collective variables for molecular kinetics. The Journal of Chemical Physics, 148(24):241003.
[45] Williams, M. O., Rowley, C. W., and Kevrekidis, I. G. (2015). A kernel-based method for data-driven Koopman spectral analysis. Journal of Computational Dynamics, 2(2):247-265.
[46] Wu, H. and Noé, F. (2019). Variational Approach for Learning Markov Processes from Time Series Data. Journal of Nonlinear Science, 30(1):03
[47] Zabezyk, J. (2020). Mathematical Control Theory: An Introduction. Systems & Control: Foundations & Applications. Springer International Publishing.
[48] Zwald, L. and Blanchard, G. (2005). On the Convergence of Eigenspaces in Kernel Principal Component Analysis. In Advances in Neural Information Processing Systems.
12


Reviews:
[{"review_id": "6R8CTEtqtn", "reviewer": "Reviewer_1FxQ", "summary": "This paper proves sharp upper bounds for eigenvalue estimation of a Koopman operator for a time-homogeneous Markovian dynamical system using either reduced rank regression or principal component regression. The bounds include operator norm error and metric distortion. These results are illustrated on simple models and some discussion of a higher dimensional molecular example is included. The error estimate also yields a design principle for kernels, based on spectral bias. ", "questions": "The estimator of metric distortion is introduced in the main text and some reference to its bearing on the experiments is made, but it is not clear from the experiments what role it plays. Can this be better explained?\n\nIf I understand correctly, the \"ugly\" kernel is chosen to make large the bias. Is there a way of making the deformation of the metric structure large in this example to illustrate the contribution of that term in the error?\n\nIn the appendix, it would be better to say in the alanine dipeptide example more explicitly how the RMSE is estimated. I assume the authors are forecasting the structure, but it could be some observable. ", "limitations": "Yes, the conclusion captures limitations well. Negative impacts are not very relevant to this paper.", "rating": 7, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "The paper is clearly written and articulates the both the theoretical results and their consequences on practical examples in a lucid manner. The results on metric distortion are novel for this problem. The paper synthesizes a number of existing arguments in a compelling way to generate a clear estimate on the spectral learning rate. ", "weaknesses": "My impression is that the argument in Sec. 5 is not particularly new in the case that the HS norm error is used. But it is not very clear why to perfer the operator norm error. ", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate the reviewer's insightful evaluation and valuable comments. In what follows, we aim to address the highlighted weaknesses and respond to the reviewer's questions.\n\n__Weaknesses:__\n\nIndeed, the tools used to establish variance bounds in Section 5 are not particularly novel, and their fundamental concepts can be found in several papers on kernel-based learning, e.g., Ref. [18], [29]. The truly innovative aspects of our work in Section 5 are:\n1. We derived the first operator norm error bounds, which usually pose greater challenges in achieving optimal rates. We have justified the necessity of using this norm in the global response to the reviewers.\n2. In contrast to classical regression problems in reproducing kernel Hilbert spaces (RKHS) where the well-specified assumption implies that the regression function (conditional mean) lies in the RKHS, operator learning requires the much stronger assumption that the Koopman operator is a HS operator on the RKHS itself. This implies that both the mean and the law of the noise need to align with the RKHS, and that noise distribution needs to be sufficiently smooth. To bypass such strong assumptions we introduced a weaker regularity condition based on the cross-covariance operators that is applicable to a wider class of problems, e.g. practical systems like Langevin dynamics. \n3. We establish that the RRR estimator, for which no error bounds were previously proven (neither in HS-norm), achieves the optimal learning rate. \n\nFinally, we remark that, while the state-of-the-art optimal rates presented by Li et al. (2022), Ref. [29], for HS-norm errors are provided under assumptions that do not guarantee the learning of a self-adjoint Koopman operator with a Gaussian kernel, our work demonstrates that the optimal operator norm rate is at least of the order $1/\\sqrt{n}$.    \n\n__Questions:__\n\n- The empirical estimator of the metric distortion plays a crucial role as the component of the empirical spectral bias $s_i$ in Theorem 4 (lines 291-29). This quantity is utilized in the experiments on Langevin dynamics to demonstrate its capability to predict eigenfunction estimation quality, as shown in Fig. 2, and for model selection in the Alanine-Dipeptide, as depicted in Fig. 3. We will provide further clarification on this matter in the revised version.  \n- Indeed, you are correct. The choice of the \"ugly\" kernel is intended to amplify the operator norm error bias. However, since the family of kernels in this example is designed via Koopman eigenfunctions, we obtain that the eigenvalue decomposition of $C$ and $A_\\pi$ differ only in eigenvalues and not eigenfunctions. Consequently, all three kernels induce no metric distortion on the estimator's eigenfunctions ($\\eta(\\psi_i)=1$).  So, to distort the metric, a different construction would be needed. \n- Indeed, we forecasted the 30 = 3*10 positions of the 10 atoms of the system. As you suggest, we will clarify this in the appendix.\n"}, {"review_id": "SoKqnE2PD6", "reviewer": "Reviewer_GGn4", "summary": "This paper analyses two approximation techniques for self-adjoint compact Koopman operators. Both estimators are constructed based on regularized least-squares with a certain rank condition in mind. The Principal Component regression (PCR) performs unconstrained Tikhonov   estimation of data projected on a low-dimensional subspace. The Reduced Rank Regression (RRR) minimizes the Tikhonov objective function subject to a rank constraint on the optimizer. Both estimators admit closed form solutions. The paper study spectral properties of these estimators, specifically deviation of eigenvalues and eigenvectors. The novelty of this study resides on the error measures: operator norm error of the Koopman operator estimation (instead of Hilbert-Schmidt) and the \"metric distortion\" that compares the two Hilbert space, the RKHS that approximates the process, and the ambient L^2 Hilbert space where the Koopman operator is initially defined. \nThe conclusion of this study is that both estimators have a similar variance, but the PCR may have a potentially larger bias, particularly for badly chosen kernels.\n ", "questions": "1. The paper is well-written. Just a few typos that can be easily fixed. I suppose in Example 3, the definition of the permutation Pi, the first case is i<= r, right?\n\n2. I understand that condition (RC) is weaker than Im (A_pi S)\\subset Im(S). Is it weaker also than Im (A_pi S)\\subset closure(Im(S)), with closure w.r.t. L^2-norm ?\n\n3. It would be useful to indicate the Hilbert space norms throughout the paper. Particularly in equation (10), but also elsewhere.  ", "limitations": "Nothing to be reported here.", "rating": 7, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "The authors employ existing state-of-the-art bounds in spectral theory of compact operators. The study sheds light on the phenomenon of \"spurious eigenvalues\". The asymptotic rates of convergence are shown to be optimal. \nOverall it seems a solid paper.", "weaknesses": "The rates of convergence and error bounds are tight, but only asymptotically. Since both PCR and RRR have similar asymptotic rates (for variance), the constants become important.  A more careful analysis of the constants would strengthen the paper. However it is understandable that such a study might be analytically too complex.\n\nThe authors mention that results are limited to self-adjoint Koopman operators. This is true, however, compact operators admit a SVD factorization with similar spectral properties (control of singular values) as self-adjoint operators. At a cursory reading, the results obtained in this study seem extendable to non-self-adjoint compact Koopman operators.  \n", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate the reviewer's insightful evaluation and valuable comments. In what follows, we aim to address the highlighted weaknesses and respond to the reviewer's questions.\n\n__Weaknesses:__\n- Regarding the variance bounds' constants, we carefully derived the proofs in the Appendix so that they can be easily analytically derived and compared using the right-hand side (r.h.s.) of Equation (56) for RRR and Equation (57) for PCR. Indeed, since both equations are combined with the same regularization bias term containing $\\gamma$, they are sufficient to understand differences in constants. As suggested, in the revised version we will address the constants, too. For the reviewer’s convenience, here we report the most important findings. Interestingly, one can see that when $\\varepsilon_n^1(n^{-\\frac{1}{\\alpha+\\beta}}, \\delta/5) < 0.1$, for notation see (46) in Apendix D.3.1, we have that both estimators have the same constant in front of the leading term $n^{-\\frac{\\alpha}{2(\\alpha+\\beta)}} \\log(1/ \\delta)$ which is equal to $$ a+7.2 \\log(10)\\sqrt{2 c_{\\mathcal{H}}} (1+ a c_{\\mathcal{H}}^{(\\alpha-1)/2}) \\,b^{\\beta/2} / \\sqrt{1-\\beta},$$ where all appearing constants are defined in assumptions (BD), (SD) and (RC). However, It is worth noting that in front of the (in general) slower term of the order $n^{-1/2}$ there is an important difference in the constants expressed as $c_{\\rm PCR}$ vs $c_{\\rm RRR}$ (both defined in line 226). For example, for the Gaussian kernel and self-adjoint Koopman operator such that $\\alpha>1$, the leading constant becomes simply $\\approx 48$, while for maximal $\\alpha$ being equal to 1, we have that constant for PCR becomes $const. c_{PCR}  \\sqrt{\\log(tr(C) / \\Vert C \\Vert},$ _const._ being a universal, while the one for RRR is obtained by replacing $c_{PCR}$ by $c_{RRR}$. \n \n- In the reply to all the reviewers we tried to clarify this aspect. As long as one does not analyze the spectral rates, compactness and self-adjointness are not really needed and the boundedness of the Koopman operator suffices as long as (BD), (SD) and (RC) hold. Concerning the SVD, we recall that, in general, singular value decomposition is related to the notion of norms, while eigenvalue decomposition is related to powering of operators (hence, dynamics). Consequently, the singular values of non-normal compact Koopman operators do not provide useful information about the system's evolution. That being said, we have already established the rates for estimating the singular values of $A_\\pi S$ (the restriction of the Koopman operator to the RKHS) via estimator $\\hat{B} :=\\hat{C}_\\gamma^{-1/2} \\hat{T}$. This result is presented in Propositions 6 and 17 of the Appendix D and serves as a key element in deriving bounds for the RRR estimator. As the reviewer suggests, it might be interesting to further analyze the estimation of the SVD of compact Koopman operators. We leave this as a direction of future work as it looks like a nontrivial open question.  \n\n\n__Questions:__\n1. Yes, $i\\leq r$, thanks for spotting it. \n2. (RC) condition is not weaker than $Im(A_\\pi S)\\subseteq cl(Im(S))$ which is always assured when the kernel is universal. Let's call the latter (UC). In a certain sense, letting the $\\alpha\\to0$ (RC) condition “gets closer” to (UC) $\\implies$ (RC). To be precise, we refer to Appendix D.1 that contains details on the relationships between the assumptions. There we see that using Sobolev norms we can define interpolation spaces $[\\mathcal{H}]\\_{\\alpha}$ that progressively cover $L^2_\\pi(\\mathcal{X})= [\\mathcal{H}]\\_0$, as $\\alpha\\to 0$. Thus, since  $Im(A_\\pi S)\\subseteq [\\mathcal{H}]\\_{\\alpha}$, and implies (RC), we understand that the smaller $\\alpha$ gets, the “closer” we are getting to (UC) being weaker than (RC). \n3. We understand the difficulty, having in mind the recurrent message of the paper that metric distortion, and change of geometry between RKHS and L2 spaces in general, transmits. So, when writing this paper we have decided to explicitly use injection $S$ to clarify the change of spaces, and, for easier readability, omit putting each time the space subscript in the vector norms. However, we agree that Equation (10) merits extra clarity,  and we will clarify the meaning of the norms employed therein. We can also make a general comment that the $L^2_\\pi(\\mathcal{X})$-norm of a function $h\\in\\mathcal{H}$ is denoted as $\\Vert Sh \\Vert$.\n"}, {"review_id": "v6sgbv5Ezk", "reviewer": "Reviewer_nHa6", "summary": "The authors provide bounds for the spectral decomposition of the estimated Koopman operators. The bounds are given for self-adjoint time-reversible operators in terms of two new metrics. Compared to estimation guarantees given in terms of the Hilbert Space distance norm, the proposed bounds require less restrictive assumptions. The theoretical results are specialized for two existing estimation algorithms.\n", "questions": "- I do not fully understand this sentence, \"The Koopman operator [...], and DMD relies [...] on to, in turn, estimate its spectral decomposition.\"\n\n-The Koppman operator is a linear HS operator. What is peculiar about bounding the Koopman operator compared to other HS operators?\n\n- How common and restrictive are the assumptions that the operator is time-reversible and compact?\n\n- Why do you say that \"the empirical PCR estimator does not minimize the empirical risk (3) under the low-rank constraint.\"? Is this because of the projection? Or because the optimization is unconstrained?\n\n- How can the learning method generate spurious eigenvalues? Is the standard operator norm insensitive to eigenvalue spuriousness?\n\n- Is the regularity condition called RC on page 5 used in the bounds? I am confused by the sentence on the following page, \"where one chooses a universal kernel for which ...\".\n\n- Intuitively, bounding the error on the estimated eigenvalue can be easier than on the estimated error functions or eigenvector. I understand the final formula may be too intricate for the main text. But it would be helpful for the reader to see an intuitive explanation of the claim, \"the eigenfunction estimation bounds readily follow from (8)\". \n", "limitations": "The authors say that restricting the analysis to self-adjoint operators is the main limitation of their work. This is mentioned in the very last lines of the paper. It would be better to elaborate on this limitation in the introduction, where the self-adjoint assumption is made. ", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "Estimating the eigenvalue of the Koopaman operator is a widespread problem. And the work seems to extend to general self-adjoint operators. The proposed new evaluation metrics are interesting, especially if they produce theoretical bounds that hold under less restrictive assumptions.", "weaknesses": "It is unclear how the bounds depend on the sample size and why it is interesting to specialize them for existing estimators. The experiments show the performance of two existing estimators but have no straightforward link with the theoretical part of the paper. It\nwould be more interesting to plot, for a given estimator, the wideness of the proposed bounds versus the (a posteriori) empirical estimation error.\n\nAfter assuming that the process is time-homogeneous and stationary, the learning task looks similar to standard non-parametric regression. The authors should specify what are the challenging aspects of the dynamical setup. Otherwise, the contribution of the paper is unclear. The core part of this work seems to be applying classical spectral bounds to a finite-dimensional approximation of HS operators. If the novelty is to use new metrics, the paper should focus more on explaining why these new metrics are better than the HS norm.\n\nThe paper's conclusion is somehow expected. Direct learning of low-rank representations is better than projecting the data and solving an unconstrained problem. The latter option may have computational advantages. But the authors do not comment on it.\n", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate the reviewer's insightful evaluation and valuable comments. In what follows, we first aim to address the weaknesses and respond to the reviewer's questions.\n\n__Weaknesses:__\n\n>_It is unclear..._\n \nWe are not sure to understand the reviewer's remark. In the appendix all the bounds are expressed explicitly, even including all relevant constants. In the main text, for easier readability, they are expressed as learning rates, that is in the standard form commonly used in statistical learning papers where the asymptotic dependence on the number of samples and the probability is shown. \n\n>_The experiments [...] no straightforward link..._\n\nWe respectfully disagree. Each experiment is directly linked to theoretical results. Please note that:\n- The experiment on the Ornstein–Uhlenbeck Process is an illustration of Example 3 that answers why the bias term in Thm. 2 is tightly estimated. This is remarked throughout the paper, e.g. in lines 232-238\n- The experiment on Langevin Dynamics is used to show two aspects. First, in the main text we show that the novel object, the empirical spectral bias $s_i$ introduced in Thm. 4 and lines 291-29, predicts well the quality of the estimated eigenfunctions. Second, in the App. F, we empirically verify (on a validation set) the spectral learning rates for eigenvalues and eigenfunctions. \n- The experiment on Alanine Dipeptide demonstrates how our theoretical results on the empirical spectral bias can be used as a tool for model selection, an important practical task within kernel-based Koopman methods for dynamical systems. \n\n>_...empirical estimation error_\n\nFirst, we would like to stress that we have derived minimax optimal learning rates which offer a framework for objectively comparing different methods. In addition to the theoretical analysis, we have properly verified the bounds on a validation set, see Appendix F.\n\n>_After assuming [...] the learning task…_\n\nAs introduced in [24], the learning task we consider involves estimating spectral decomposition of an unknown linear operator on an unknown domain, leading to novel challenges. See, please also the global response to all reviewers, where we elaborate on different challenges and aspects of our contributions.  \n\n>_The paper's conclusion..._\n \nWe agree, the RRR estimator proposed in [24], was shown to be superior to the standard PCR in terms of uniform learning bounds, and computational complexity was studied, too. Still, we deem our detailed analysis essential to fully comprehend the non-trivial phenomena arising when learning the spectral decomposition of Koopman operators, crucially important in dynamical systems applications, as stated in e.g. [8, 9, 26, 36].\n\n__Questions:__\n\n- The interest in the Koopman operator stems from its ability to globally linearize nonlinear dynamical systems in a (typically infinite dimensional) function space, usually called space of observables. In this new space one can solve a linear dynamical system via spectral decomposition, and the result in the original (state) space is known as Koopman mode decomposition (KMD). KMD is an efficient formula for evolving general observables (measurements) of nonlinear dynamical systems, while Dynamic Mode Decomposition (DMD) is a method to approximate KMD from the recorded trajectory using a space of linear functions as observables.\n- We are not sure to fully understand the reviewer’s question. Should they please elaborate on it? Please note that the Koopman operator is in general only bounded. In order for it to also be HS, the transition kernel needs to have square integrable density w.r.t. the invariant distribution. On the other hand, the restriction of the Koopman operator onto RKHS, i.e. $A_\\pi S$, is an HS-operator. Concerning the utility of the Koopman operator please see the discussion in e.g. [7, 8, 9, 22, 23, 26, 31].\n- We tried to explain better the assumptions and contributions in the global reply to all reviewers. Compactness of $A_\\pi$ is guaranteed, e.g., whenever the transition kernel is absolutely continuous w.r.t. the invariant measure, i.e. $p(x,\\cdot)<< \\pi$ for $\\pi$-a.e. $x\\in\\mathcal{X}$. This is a reasonable assumption implying that the system cannot transition into sets that at equilibrium have measure zero. If additionally, this density $f(x,y):= \\frac{dp(x,\\cdot)} { d\\pi}(y)$, $x,y\\in\\mathcal{X}$ is symmetric, i.e. $f(x,y)=f(y,x)$ $\\pi$-a.e., then the Koopman operator $A_\\pi$ is self-adjoint. Many physical systems satisfy these conditions, see lines 74-77.\n- PCR estimator $\\hat{G}$ is not a minimizer of objective (3) under the rank constraint $rank(\\hat{G})\\leq r$, instead it is a spectral filter applied to the OLS solution.\n- In general, spuriousness of eigenvalues is a perturbation phenomena happening when $|\\lambda_i(A) - \\lambda_i(A+\\Delta)|>>\\Vert \\Delta \\Vert$. The norm used to properly address this phenomena is the operator norm, under which the equality holds for normal operators. Hence, in our case the spuriousness stems only from the learning method, since we study the _operator norm_ error in the estimation of normal $A_\\pi$. Since the estimator is built from a subspace $\\mathcal{H}$ of a function space $L^2_\\pi(\\mathcal{X})$ there is a metric distortion that comes into play. This doesn’t occur in the classical estimation of spectra in (finite-dimensional) matrix problems. \n- Yes, all the results in the bounds use (RC) condition. The sentence means that when the kernel is universal, the only contribution to the regularization bias is due to $\\gamma$ thanks to (RC). For non-universal kernels, an additional term is present, see Prop 5 of App D.2.\n- Thanks! Indeed, in general, bounding errors for eigenvectors is more difficult than for eigenvalues. However, for normal linear operators, the situation is easier due to Davis-Kahan theorem (Prop 2 of App A.1). As you have suggested we will elaborate better on this in the revised manuscript."}, {"review_id": "R60Xm9RHV3", "reviewer": "Reviewer_B3rh", "summary": "This paper studies the approximation and learning of the Koopman operator. Koopman operator is helpful in modeling a broad class of Markovian dynamical systems. In this paper, two types of approximation strategies are studied, namely, \"Principal Component Regression (PCR)\" and \"Reduced Rank Regression (RRR).\" Both types are based on general reproducing kernel Hilbert spaces. With the assumptions that the population covariance operator bounds the population cross covariance operator, the RKHS feature map being L-infinity, and the eigenvalues of the population covariance operator decay as O(i^(-1/beta)) with 0<beta<=1, the operator norm error, the eigenvalue estimation error, and the eigenfunction approximation error are bounded.", "questions": "1. Lines 116--117: What is the definition of rank-r operator? If the dimension of the image of one operator is finite, the operator is automatically Hilbert-Schmidt. If this is the definition, I guess using the notion of \"rank-r Hilbert-Schmidt operator\" is confusing because it implies the possibility of rank-r operators that are not Hilbert-Schmidt.\n2. Which theorem does it refer to for the claim in Line 335?\n3. In Line 68, the Koopman operator A_pi is assumed self-adjoint, and in Line 81, A_pi is further assumed compact. Is the second assumption made globally for the whole paper? In Theorem 3, A_pi is again assumed compact and self-adjoint, but this assumption does not appear in Theorem 4. This could not be very clear, leaving the reader wondering whether this assumption is adopted for Theorem 4.", "limitations": "No limitation issues were found.", "rating": 9, "confidence": 3, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "The paper's originality is high, because of the new error estimation provided. This paper represents solid research results with high quality. The writing is clear and easy to follow. The paper's significance is guaranteed by the broad scope of applications of the Markovian dynamical systems, including Langevin dynamics.", "weaknesses": "We do not have significant concerns about this paper. No weakness is identified.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate the reviewer's insightful evaluation and the positive comments. In what follows, we aim to respond to the reviewer's questions. \n\n__Questions:__\n\n1. We agree, the notation is not the best. We will follow the advice and denote by ${\\rm B}_r(\\mathcal{H})$ finite rank operators $\\mathcal{H}\\to\\mathcal{H}$.\n2. It is Theorem 7 in the appendix that is discussed in lines 252-254. Should we include this Theorem on the lower bound in the main body?\n3. Thank you for spotting this. It is a typo, the statement of Theorem 4 should start as “Under the assumptions of Theorems 2 and 3…”\n"}, {"review_id": "T38NMPjsFr", "reviewer": "Reviewer_ZSn2", "summary": "This paper develops bounds on how close the approximated Koopman modes and eigenvalues are to the true eigenvalues and modes, for two important classes of methods used to compute the Koopman mode decomposition. The authors find that one class, Principal Component Regression, which included extended dynamic mode decomposition (EDMD), can suffer more from poorly chosen kernels and can have larger bias than Reduced Rank Regression (RRR). They additionally provide an empirical method for determining spurious eigenvalues, which can be used for model selection.", "questions": "1. How does this work compare to previous work studying the convergence of EDMD (e.g., Klus et al., 2016; Korda and Mezic, 2018)? \n\n2. What exactly is Fig. 2 showing (in terms of details)?", "limitations": "The authors did a good job being clear that their work was limited to self-adjoint operators. ", "rating": 7, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "1. This paper is well written and easy to follow. \n\n2. This paper provides new techniques for computing bounds on the approximated Koopman spectral objects, and the discovery that PCR can have larger bias than RRR, are important ones for the field. \n\n3. This paper provides a new empirical method for identifying spurious eigenvalues and making model selection. Again, both of these are important topics for the field and for the application of numerical methods in applied settings. \n\n4. The numerical examples provided in Figs. 1-3 are helpful for understanding the theory developed, and provide support for the developed claims. \n\n", "weaknesses": "1. Klus et al., 2016 and Korda and Mezic, 2018, as examples, proved the convergence of EDMD to the true Koopman operator, when $M \\rightarrow \\infty$, where $M$ is the number of data points. This work seems to be missing from the paper, as does discussion surrounding how the paper differs from that work. I assume the primary difference is that this paper's results are not in the asymptotic limit (although, they are in the sense that the bounds for RRR, for example, in the Gaussian case converge as $1/\\sqrt{n}$, which approaches $0$ as $n\\rightarrow \\infty$). Additionally, the results for PCR obtained by this paper would suggest that in the asymptotic limit EDMD does not converge, since it has a bias. Discussion on how this is reconciled with the work of Klus et al., 2016 and Korda and Mezic, 2018, is necessary. \n\n2. Fig. 3 was confusing. Was the best estimator found on the test data set, and then the red line in Fig. 3 the result of applying it to the validation data? A secondary panel in that figure describing what was being done would be helpful. \n\nMINOR COMMENTS: \n\n1. It was unclear to me how $| \\lambda_i - \\mu_{j(i)} | \\leq || (A_\\pi - \\lambda_i I)^{-1}||^{-1}$ leads to observing that $||(A_\\pi  - S\\hat{G})\\hat{\\psi}_i|| \\leq \\mathcal{E}(\\hat{G})\\eta(\\hat{\\psi})$ (lines 154-155). Adding a little more detail/comment on this would be helpful.\n\n2. \"left hand side\" (line 156) should be \"right hand side\" no?\n\n3. The connection between DMD and KMD should be discussed (lines 22-25) (Rowley et al., 2009). \n\n4. The original EDMD paper (Williams et al., 2015) should be cited when discussing EDMD for the first time (line 31). \n\n5. Very minor but both \"non-linear\" and \"nonlinear\" are written. ", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate the reviewer's insightful evaluation and valuable comments. In what follows, we aim to address the highlighted weaknesses and respond to the reviewer's questions.\n\n__Weakness 1 & Question 1:__\n\nFirst, we would like to stress out that [Korda and Mezic 2018] prove the consistency (infinitely many samples $n\\to\\infty$) of EDMD with $L^2_\\pi$ orthogonal projection of the Koopman operator onto the finite $r$-dimensional subspace spanned by the dictionary of functions. Note that this _projection is not the same as the Koopman operator_ (acting on the whole $L^2_\\pi$), see pg. 50 of [8]. Additionally, [Korda and Mezic 2018]  show that EDMD is consistent with the Koopman operator on $L^2_\\pi$, only when, under certain assumptions,  $r\\to\\infty$. Moreover, these results are limited to deterministic dynamical systems. On the other hand, [Klus et al. 2016] prove the consistency of EDMD for stochastic processes with Galerkin projection on the same subspace (finite $r$), while our results also imply consistency of EDMD when $r\\to\\infty$. Note that our work is concerned with the convergence of the spectral decomposition, and that the eigenvalues of (Galerkin) projections, depending on the choice of the subspace, may be spurious for finite $r$. In light of these considerations we do not see any contrast between our results and the existing literature on the subject.  \n\nNext, we would like to recall that proving consistency of estimators (an infinite number of data points) is typically easier than proving non-asymptotic generalization bounds for a finite number of data points, especially when the goal is to establish the matching information theoretic lower bound. This provides deeper insight into the learning problem. When one deals with eigenvalues, this becomes even more so, since, in general, the eigenvalue estimation hides the effect of sensitivity to perturbations that cannot be seen from the asymptotic results, but needs more sophisticated perturbation analysis. Namely, even though the estimator is consistent (hence its eigenvalues converge to the true ones) spurious eigenvalues may arise for finite sample size. This happens when the norm of the estimator error is small but the estimator’s eigenvalues may be very far from the true ones.\n\n__Weakness 2:__\n\nFigure three has been produced as follows: we trained every one of the 19 estimators considered on the same train dataset of 2000 points. Then, for each of the fitted models we have evaluated the (data-dependent) spectral bias introduced in lines 291-295 related to the r.h.s. of Eq. (18). The evaluation of the spectral bias has been done on a validation set comprising 5000 points. From this evaluation procedure we were able to compare the 19 models, rank them, and select the “best” one as the one minimizing the spectral bias. Finally, we have selected 2000 trajectories out of a third (independent) test set, and evaluated the RMSE attained by each model in forecasting the state. We will clarify this procedure either in the text or with an additional panel as suggested.\n\n__Question 2:__\n\nFor 50 trials and n=2000 training points both PCR and RRR estimators were fitted with the same parameters, in particular with rank r=4. Then, the estimated eigenfunctions using PCR (blue) and RRR (orange) were plotted, comparing them to the true eigenfunctions in black (one per plot). Additionally, average (across 50 independent trials) empirical spectral biases $\\hat{s}_1,...,\\hat{s}_4$ were computed. Comparing the values of $\\hat{s}_i$ with the eigenfunction estimation error, we observe that, as indicated by our theory, spectral biases predicts the performance of the estimators (PCR vs RRR) on recovering Koopman eigenfunctions. We will revise the caption to help the reader better understand the Figure.   \n\n__Minor comments:__\n\n1. We are sorry for the confusion, lines 155-156 should read “So, following the reasoning of [24 , Thm. 1] and observing that $\\Vert(A_\\pi S- S\\hat{G})\\hat{\\psi}\\_i\\Vert / \\Vert S\\hat{\\psi}\\_i \\Vert \\leq \\mathcal{E}(\\hat{G})\\eta(\\hat{\\psi}\\_i)$ gives the left hand side of (11) .” Namely, recalling that $\\Vert (A_\\pi-\\hat{\\lambda}\\_i I)\\^{-1}\\Vert\\^{-1}= \\min_{\\Vert f\\Vert =1} \\Vert A_\\pi f-\\hat{\\lambda}_i f \\Vert$ and plugging $\\hat{f_i}$ instead of $f$, we obtain (11).\n2. Sorry for the confusion, it should state: “r.h.s. of the first equation in (11)”\n3. Indeed, we will revise the lines 22-26, citing  [Rowley et al. 2009], to clarify that DMD is an algorithm to estimate the Koopman mode decomposition (KMD).\n4. Yes, thank you! \n5. Thanks for spotting it, we will fix it.\n"}]

Author rebuttal:
We wish to thank all reviewers for their insightful evaluation of our paper. We appreciate all their comments and remarks, which we will incorporate in our revision. Before addressing each review in detail, we would like to point out some general remarks that apply to all of them.

__Assumptions & contributions:__ We thank the reviewers for the comments on the global assumptions of the paper. This made us notice that we should improve our discussion on self-adjointness and compactness in lines 67-69 and line 81, which will be rectified in the revision as follows:

Our main results come under different assumptions on the Koopman operator $A_\pi$:
- We prove _minimax optimal learning rates_ for operator norm error (Section 5). For these results to hold, $A_\pi$ just needs to be _bounded_. 

- We derive _spectral learning rates_ (Section 6), further requiring $A_\pi$ to be _compact and self-adjoint_. 

The additional assumptions in item 2 serve two purposes. Firstly, since there are currently no finite sample error bounds for eigenvalues and eigenfunctions of the Koopman operator, our results provide a first important step in this direction. Secondly, we emphasize that due to the peculiarities of the operator learning problem (in comparison to the simpler spectral estimation of matrices) even in the “well-conditioned” scenario of self-adjoint operators, bounding the operator norm error is insufficient to yield accurate estimates of the eigenvalues, leading to spurious phenomena. 

__Operator norm instead of HS-norm:__ Koopman operators play a crucial role in science and engineering, offering valuable diagnostics and insights into dynamical systems' properties. Accurate estimation of their eigenvalues and eigenfunctions using the operator norm is essential for these applications, making spectral generalization bounds of paramount importance. However, existing theoretical guarantees with the Hilbert-Schmidt norm are often unsatisfactory, as they typically result in suboptimal bounds. In more detail, Section 5 is, in flavor, similar to existing recent work on optimal rates for Hilbert-Schmidt norm error in the estimation of the conditional expectation operator by Li et al. 2022, Ref. [29]. The main difference is that our bounds are derived for the operator norm (with a further novelty that two additional estimators are considered and under weaker assumptions). The significance of the operator norm stems from its critical property that, for every normal operator $A$ (even non-compact), $$dist(z,Sp(A)) :=\min_{\mu\in Sp(A)} | z-\mu | = \Vert (A-z)^{-1}\Vert^{-1}_{op}.$$ In contrast, using the Hilbert-Schmidt norm often leads to loose bounds for this distance. Employing the operator norm is crucial in obtaining sharp bounds for the spectral estimation of bounded operators and is a necessary step in addressing the problem of spurious eigenvalues, as is customary in spectral perturbation theory.

We will clarify the above points  in the revised version, providing better explanations of the limitations and highlighting essential directions for future research. 

Once again we wish to thank the reviewers for their valuable feedback!
The Authors

Meta-review: The authors tackle the important problem of learning nonlinear dynamical systems through learning the Koopman operator. This is an established approach but one that is lacking in theoretical guarantees. The authors give both strong/extensive non-asymptotic theoretical results (significant over previous work, with the first analysis of popular estimators), and provide accompanying experiments; the theory results seem to be informative for empirical phenomena (e.g., spurious eigenvalues). The paper received excellent scores, and the authors clarified with technical interpretations of the results. I recommend acceptance.
Decision: Accept (spotlight)