SlowGuess/NeurIPS_2023_OCR huggingface dataset:
CMAP-Ecole Polytechnique
Nonlinear dynamical systems can be handily described by the associated Koopman operator, whose action evolves every observable of the system forward in time. Learning the Koopman operator and its spectral decomposition from data is enabled by a number of algorithms. In this work we present for the first time non-asymptotic learning bounds for the Koopman eigenvalues and eigenfunctions. We focus on time-reversal-invariant stochastic dynamical systems, including the important example of Langevin dynamics. We analyze two popular estimators: Extended Dynamic Mode Decomposition (EDMD) and Reduced Rank Regression (RRR). Our results critically hinge on novel minimax estimation bounds for the operator norm error, that may be of independent interest. Our spectral learning bounds are driven by the simultaneous control of the operator norm error and a novel metric distortion functional of the estimated eigenfunctions. The bounds indicate that both EDMD and RRR have similar variance, but EDMD suffers from a larger bias which might be detrimental to its learning rate. Our results shed new light on the emergence of spurious eigenvalues, an issue which is well known empirically. Numerical experiments illustrate the implications of the bounds in practice.
Recently, researchers have emphasized the utmost importance of developing physically-informed machine learning models that prioritize interpretability and foster physical insight and intuition, see for example [22] and references therein. One technique highlighted in these works is the Koopman operator regression framework to learn and interpret nonlinear dynamical systems see, e.g. [8, 26] and references therein. A key component of this approach is the Koopman Mode Decomposition (KMD), which decomposes complex dynamical systems into simpler, coherent structures. When ordinary least squares are used to learn Koopman operator from data, estimated KMD is known as the Dynamic Mode Decomposition (DMD) [36]. Koopman operator estimators and their modal decomposition find many applications, including fluid dynamics, molecular kinetics and robotics [7, 19].
The Koopman operator returns the expected value of observables of the system in the future given the present, and one relies on estimators of this operator to in turn estimate its spectral decomposition that leads to the estimation of KMD. Our goal is to study the statistical properties of the eigenvalues and eigenfunctions of the Koopman operator estimators via two mainstream algorithms: Principal Component Regression (PCR) and Reduced Rank Regression (RRR) studied in [21, 24]. PCR encompasses as particular cases the popular Extended Dynamic Mode Decomposition (EDMD), which is the de-facto estimator in the data-driven dynamical system literature [see 26, 45] and references
therein]. Both PCR and RRR are kernel-based algorithms that, given a dataset of observations of the dynamical system, implement a strategy to approximate the action of the Koopman operator on a reproducing kernel Hilbert space (RKHS) [3, 38].
We present for the first time non-asymptotic learning bounds on the distance between the Koopman eigenvalues and eigenfunctions and those estimated by either PCR or RRR. We show that the eigenvalues produced by such algorithms are biased estimators of the true Koopman eigenvalues, with PCR incurring larger bias. Our results critically hinge on novel estimation bounds for the operator norm error, that may be of independent interest, leading to minimax optimal bounds for finite-rank Koopman operators. Moreover, we introduce the novel notion of metric distortion, which characterizes how the norm of eigenfunctions vary when moving from the RKHS in which learning takes place to the underlying ambient space where the Koopman operator is properly defined. We show that both the operator norm error and metric distortion are needed in order to estimate the operator spectra and our bounds can be used to explain the well-known spuriousness phenomena in eigenvalue estimation [14], namely, the scenario in which the estimated eigenvalues are not related to the true ones, despite small operator norm error.
Contributions and Organization. We make the following contributions: i) We introduce the notion of metric distortion and show that it has to be used alongside the operator norm error to derive Koopman spectra estimation error bounds (Theorem 1); ii) We establish the first sharp estimation bound for the operator norm error (Theorem 2); iii) We establish spectral learning rates (Thms. 3 and 4) for both PCR and RRR; iv) We propose how to use the results entailed by Theorem 4 to detect the presence of spurious eigenvalues from data.
The paper is organized as follows. In Section 2 we recall the notion of Koopman operator, its spectral decomposition, and review PCR and RRR estimators. Section 3 describes the estimation problem and outline our main results. Section 4 presents our approach to bound eigenvalue and eigenvector estimation errors. Section 5 gives sharp upper bounds for the operator norm error. Section 6 presents our spectral learning bounds. Finally, Section 7 illustrates the implications of the bounds in practice, and is designed to provide practitioners with the tools to benchmark the performance of algorithms in real scenarios.
Dynamical Systems and Koopman Operator. In this work we study Markovian dynamical systems, that is collections of random variables  $\{X_{t}\colon t\in \mathbb{N}\}$ , where  $X_{t}$  represents the state at time  $t$ , taking values in some space  $\mathcal{X}$ . We focus on time-homogeneous (i.e. autonomous) systems hosting an invariant measure  $\pi$  for which the Koopman operator [24, 27]
$$
(A _ {\pi} f) (x) := \mathbb {E} [ f (X _ {t + 1}) | X _ {t} = x ], \quad x \in \mathcal {X} \tag {1}
$$
is a well defined bounded linear operator on  $L_{\pi}^{2}(\mathcal{X})$ , the space of square integrable functions on  $\mathcal{X}$  relative to measure  $\pi$ . In the field of stochastic processes,  $(\mathbb{1})$  is also known as the transfer operator and returns the expected value of  $f$  in the future given the present. This operator is self-adjoint (i.e.  $A_{\pi} = A_{\pi}^{*}$ ) whenever dynamics is time-reversal invariant w.r.t.  $\pi$ , which is satisfied by many stochastic processes in the physical sciences.
Example 1 (Langevin Dynamics). Let  $\mathcal{X} = \mathbb{R}^d$  and let  $\beta >0$ . The (overdamped) Langevin equation driven by a potential  $U:\mathbb{R}^d\to \mathbb{R}$  is given by  $dX_{t} = -\nabla U(X_{t})dt + \sqrt{2\beta^{-1}} dW_{t}$ , where  $W_{t}$  is a Wiener process. The invariant measure of this process is the Boltzman distribution  $\pi (dx)\propto e^{-\beta U(x)}dx$ , and the associated Koopman operator is self-adjoint.
The Langevin equation models a wealth of phenomena, such as the evolution of chemical and biological systems at thermal equilibrium [16], the mechanism regulating cell size in bacteria [11], chemical reactions [25], the dynamics of synapses [11, 42], stock market fluctuations [6] and many more. Furthermore, when  $U(x) = \theta \| x\|^2 / 2$  ( $\theta > 0$ ), the Langevin equation reduces to the celebrated Ornstein-Uhlenbeck process [34, Chapter 6].
The operator  $(1)$  evolves every observable of the system forward in time. Since it is bounded and linear, it admits a spectral decomposition, which plays a central role in the analysis and interpretation of the dynamical system [26], as well as (nonlinear) control [2]. As in [46], to study the spectral decomposition we further assume that  $A_{\pi}$  is a compact operator, which rules out the presence of
continuous and residual spectrum components and leads to
$$
A _ {\pi} = \sum_ {i \in \mathbb {N}} \mu_ {i} f _ {i} \otimes f _ {i}, \tag {2}
$$
where  $(\mu_i, f_i)_{i \in \mathbb{N}} \subseteq \mathbb{R} \times L_\pi^2(\mathcal{X})$  are Koopman eigenpairs, i.e.  $A_\pi f_i = \mu_i f_i$ . Moreover,  $\lim_{i \to \infty} \mu_i = 0$  and  $\{f_i\}_{i \in \mathbb{N}}$  form a complete orthonormal system of  $L_\pi^2(\mathcal{X})$ . In the context of molecular dynamics, the leading eigenvalues and their eigenfunctions are key in the study of long-term dynamics and so-called meta-stable states [see, e.g., [40]].
Koopman Operator Regression in RKHS. Throughout the paper we let  $\mathcal{H}$  be an RKHS and let  $k:\mathcal{X}\times \mathcal{X}\to \mathbb{R}$  be the associated kernel function. We let  $\phi :\mathcal{X}\rightarrow \mathcal{H}$  be a feature map [38] such that  $k(x,x^{\prime}) = \langle \phi (x),\phi (x^{\prime})\rangle$  for all  $x,x^{\prime}\in \mathcal{X}$ . We consider RKHSs satisfying  $\mathcal{H}\subset L_{\pi}^{2}(\mathcal{X})$  [38] Chapter 4.3], so that PCR and RRR approximate  $A_{\pi}:L_{\pi}^{2}(\mathcal{X})\to L_{\pi}^{2}(\mathcal{X})$  with an operator  $G:\mathcal{H}\to \mathcal{H}$ . Notice that despite  $\mathcal{H}\subset L_{\pi}^{2}(\mathcal{X})$ , the two spaces have different metric structures, that is for all  $f,g\in \mathcal{H}\subset L_{\pi}^{2}(\mathcal{X})$ , one in general has  $\langle f,g\rangle_{\mathcal{H}}\neq \langle f,g\rangle_{L_{\pi}^{2}(\mathcal{X})}$ . In order to handle this ambiguity, we introduce the injection operator  $S:\mathcal{H}\to L_{\pi}^{2}(\mathcal{X})$  such that for all  $f\in \mathcal{H}$ , the object  $Sf$  is the element of  $L_{\pi}^{2}(\mathcal{X})$  which is pointwise equal to  $f\in \mathcal{H}$ , but endowed with the appropriate  $L_{\pi}^{2}(\mathcal{X})$  norm. With this in mind, the Koopman operator restricted to  $\mathcal{H}$  is simply  $A_{\pi}S$ , which is then estimated by  $SG$  for some  $G\in \mathrm{HS}(\mathcal{H})$ . We will measure the operator norm error,  $\| A_{\pi}S - SG\|$ . This is in contrast to the more frequently used Hilbert-Schmidt (HS) norm.
Koopman operator regression estimators are supervised learning algorithms to learn the Koopman operator, in which input and output data are consecutive states of the system  $(X_{t},X_{t + 1})$  for some  $t\in \mathbb{N}$ . Since the Markov process is time-homogeneous and stationary, the joint probability distribution of  $(X_{t},X_{t + 1})$  is the same for every  $t\in \mathbb{N}$  and we denote it by  $\rho$ . Furthermore, stationarity also implies that  $X_{t}\sim \pi$  for all  $t\in \mathbb{N}$ . Given a dataset  $\mathcal{D}_n\coloneqq (x_i,y_i)_{i = 1}^n$  of consecutive states, PCR and RRR are two different strategies to minimize, under a fixed-rank constraint, the mean square error
$$
\widehat {\mathcal {R}} (G) := \frac {1}{n} \sum_ {i \in [ n ]} \| \phi \left(y _ {i}\right) - G ^ {*} \phi \left(x _ {i}\right) \| ^ {2}, \tag {3}
$$
where  $G \in \mathrm{HS}(\mathcal{H})$ , the space of Hilbert-Schmidt operator acting on  $\mathcal{H}$ . PCR and RRR estimators are expressed as functions of the input and cross empirical covariances, defined respectively as
$$
\widehat {C} = \frac {1}{n} \sum_ {i \in [ n ]} \phi (x _ {i}) \otimes \phi (x _ {i}), \text {a n d} \widehat {T} = \frac {1}{n} \sum_ {i \in [ n ]} \phi (x _ {i}) \otimes \phi (y _ {i}).
$$
Likewise, the population risk is  $\mathcal{R}(G) = \mathbb{E}_{(X,Y)\sim \rho}\| \phi (Y) - G^{*}\phi (X)\|^{2}$ , and the population covariance and cross covariance are  $C = \mathbb{E}_{X\sim \pi}\phi (X)\otimes \phi (X)$ , and  $T = \mathbb{E}_{(X,Y)\sim \rho}\phi (X)\otimes \phi (Y)$ , respectively. We note that by the reproducing kernel property one finds that  $C = S^{*}S$ ; see e.g. [38].
Two Important Estimators. We next briefly recall two operator regression estimators that we study in this paper. The Principal Component Regression (PCR) estimator works by first projecting the input data into the  $r$ -dimensional principal subspace of the covariance matrix  $\widehat{C}$ , and then ordinary least squares are solved for such projected data, yielding the estimator [see e.g. 24]
$$
\widehat {G} _ {r, \gamma} ^ {\mathrm {P C R}} = \llbracket \widehat {C} _ {\gamma} ^ {- 1} \rrbracket_ {r} \widehat {T}. \tag {4}
$$
Here  $\widehat{C}_{\gamma} := \widehat{C} + \gamma I_{\mathcal{H}}$  and  $[\cdot]_r$  denotes the  $r$ -truncated SVD. The population counterpart is  $G_{r,\gamma}^{\mathrm{PCR}} = [C_{\gamma}^{-1}]_r T$ , where  $C_{\gamma} := C + \gamma I_{\mathcal{H}}$ . Note, however, that the empirical PCR estimator does not minimize the empirical risk (3) under the low-rank constraint.
The Reduced Rank Regression (RRR) algorithm, in contrast, is the exact minimizer of (3) under fixed rank constraint. Specifically, RRR is defined as  $\widehat{G}_{r,\gamma}^{\mathrm{RRR}}\coloneqq \arg \min \{\hat{\mathcal{R}} (G) + \gamma \| G\|_{\mathrm{HS}}^2:G\in \mathrm{B}_r(\mathcal{H})\}$ , where the regularization term  $\gamma \| G\|_{\mathrm{HS}}^2$  is added to ensure stability, and  $\mathrm{B}_r(\mathcal{H})$  denotes the set of bounded operators on  $\mathcal{H}$  that have rank at most  $r$ . The closed form solution of the empirical RRR estimator is [24]
$$
\widehat {G} _ {r, \gamma} ^ {\mathrm {R R R}} = \widehat {C} _ {\gamma} ^ {- 1 / 2} \llbracket \widehat {C} _ {\gamma} ^ {- 1 / 2} \widehat {T} \rrbracket_ {r}, \tag {5}
$$
while the population counterpart is given by  $G_{r,\gamma}^{\mathrm{RRR}} = C_{\gamma}^{-1/2}[[C_{\gamma}^{-1/2}T]]_r$ .
Once either the PCR or RRR estimators are fitted, their spectral decomposition is a proxy for the spectral decomposition of the Koopman operator  $A_{\pi}$ . Theorem 2 in [24] shows how such a decomposition can be calculated via the kernel trick for both  $\widehat{G}_{r,\gamma}^{\mathrm{PCR}}$  and  $\widehat{G}_{r,\gamma}^{\mathrm{RRR}}$ .
For simplicity we consider the i.i.d. setting, however our forthcoming analysis is directly applicable to sample trajectories following [24].
Figure 1: PCR vs. RRR in estimating the largest eigenvalues of the 1D Ornstein-Uhlenbeck process with three different kernels over 50 independent trials. Vertical lines correspond to Koopman eigenvalues. The good kernel is such that its  $\mathcal{H}$  corresponds to the leading eigenspace of the Koopman operator, while the other two are spans of scaled and permuted eigenfunctions for which the distortion with respect to the original metric structure of  $A_{\pi}$  introduces slow (bad kernel) and fast (ugly kernel) spectral decay of the covariance.
3 The Problem and Main Result in a Nutshell
In this section we introduce the spectral estimation problem, outline our main results in a distilled form, and discuss some important implications. Recall the definition of Koopman operator (1) and its spectral decomposition (2). Given a rank  $r$  estimator  $\widehat{G} \in \mathrm{B}_r(\mathcal{H})$  of  $A_{\pi}$ , we let  $(\widehat{\lambda}_i,\widehat{\psi}_i)_{i = 1}^r$  be its spectral decomposition, satisfying  $\widehat{G}\widehat{\psi}_i = \widehat{\lambda}_i\widehat{\psi}_i$ . We aim to study how well a nonzero eigenvalue  $\widehat{\lambda}_i$  of  $\widehat{G}$  estimates its closest Koopman eigenvalue  $\mu_{j(i)}$ , where
$$
j (i) = \operatorname {a r g m i n} _ {j \in \mathbb {N}} | \widehat {\lambda} _ {i} - \mu_ {j} |. \tag {6}
$$
Moreover we wish to compare  $\widehat{\psi}_i$  with the corresponding true Koopman eigenfunction. To this end, we embed  $\widehat{\psi}_i$  in  $L_{\pi}^{2}(\mathcal{X})$  by means of the operator  $S$  and define the normalized estimated eigenfunction
$$
\widehat {f _ {i}} = S \widehat {\psi_ {i}} / \| S \widehat {\psi_ {i}} \|. \tag {7}
$$
One of the key quantities studied in this work is the eigenvalue estimation error  $|\widehat{\lambda}_i - \mu_{j(i)}|, i \in [r]$ . Recalling that  $A_{\pi}$  is compact and self-adjoint, the classical Davis-Kahan result [17] implies that the eigenvalue estimation error  $|\widehat{\lambda}_i - \mu_{j(i)}|$  also bounds the quality of the eigenfunction approximation as
$$
\left\| \widehat {f} _ {i} - f _ {j (i)} \right\| ^ {2} \leq \frac {2 \left| \widehat {\lambda} _ {i} - \mu_ {j (i)} \right|}{\left[ \operatorname {g a p} _ {j (i)} \left(A _ {\pi}\right) - \left| \widehat {\lambda} _ {i} - \mu_ {j (i)} \right| \right] +} \tag {8}
$$
where  $\mathrm{gap}_j(A_\pi) = \min_{\ell \neq j}|\mu_\ell -\mu_j|$  is the distance between  $\mu_{j}$  and its closest Koopman eigenvalue.
Let  $\sigma_{j}(\cdot)$  denotes the  $j$ -th singular value of an operator. To give a flavour of our results, here we report spectral bounds for the Gaussian kernel. In this case, Theorem below gives a high probability bound on the estimation error  $|\widehat{\lambda}_i - \mu_{j(i)}|$ , that is of order
$$
\mathcal {O} \left(\frac {\sigma_ {r + 1} (A _ {\pi} S)}{\sigma_ {r} (A _ {\pi} S)} + \frac {1}{\sqrt {n}}\right) \text {f o r} \widehat {G} _ {r, \gamma} ^ {\text {R R R}}, \text {a n d} \mathcal {O} \left(\frac {\sigma_ {r + 1} (S)}{\left[ \sigma_ {r} (A _ {\pi} S) - \sigma_ {r + 1} (S) \right] _ {+}} + \frac {1}{\sqrt {n}}\right) \text {f o r} \widehat {G} _ {r, \gamma} ^ {\text {P C R}}.
$$
If the Koopman operator has finite rank then  $\sigma_{r+1}(A_{\pi}S) = 0$ , the RRR estimator is unbiased, and its error goes to zero at the rate  $1/\sqrt{n}$ . Otherwise, recalling that  $\sigma_{r+1}(S)$  is the square root of the  $(r+1)$ -th eigenvalue of the kernel operator [38, Chapter 4.5], if  $\mathcal{H}$  is infinite dimensional  $\sigma_{r+1}(S) > 0$ , i.e. PCR has a strictly positive bias. In general, the presence of a bias in the estimated eigenvalues may result in the appearance of spurious eigenvalues. This phenomenon for PCR is well documented in practice, see e.g. [13, 14, 26, 28]. In Figure I we illustrate such an effect on a simple dynamical system discussed both in Example 3 and in Section 7.
The core of our analysis is Theorem 1. It reveals that in order to derive spectral estimation bounds for the Koopman operator, it is not enough to study the excess risk in the HS norm. Indeed, our spectral
bounds are determined by both the operator norm error of the Koopman estimator
$$
\mathcal {E} (\widehat {G}) := \left\| A _ {\pi} S - S \widehat {G} \right\|, \widehat {G} \in \operatorname {H S} (\mathcal {H}) \tag {9}
$$
and the metric distortion between  $\mathcal{H}$  and  $L_{\pi}^{2}(\mathcal{X})$
$$
\eta (h) := \| h \| / \| S h \|, \quad h \in \mathcal {H}. \tag {10}
$$
Note that since  $Sh \in L_{\pi}^{2}(\mathcal{X})$  is just an equivalence class of a function  $h$ ,  $\| Sh \|$  is simply  $L_{\pi}^{2}(\mathcal{X})$ -norm of  $h$ , and hence the metric distortion can be written, with a slight abuse of notation, as  $\eta(h) \coloneqq \| h \|_{\mathcal{H}} / \| h \|_{L_{\pi}^{2}(\mathcal{X})}$ . While the (HS norm) error was studied before [see 29 and references therein], little is known about operator norm error bounds. Moreover, the metric distortion is, to the best of our knowledge, a novel quantity in the spectral analysis of Koopman operator.
Theorem 1. Let  $A_{\pi}$  be a self-adjoint compact operator and let  $r \in \mathbb{N}$ . Then, for every empirical estimator  $\widehat{G} \in \mathrm{B}_r(\mathcal{H})$  and every  $i \in [r]$
$$
\left| \widehat {\lambda} _ {i} - \mu_ {j (i)} \right| \leq \eta (\widehat {\psi} _ {i}) \mathcal {E} (\widehat {G}), \quad \text {a n d} \quad \| \widehat {f} _ {i} - f _ {j (i)} \| ^ {2} \leq \frac {2 \eta (\widehat {\psi} _ {i}) \mathcal {E} (\widehat {G})}{[ \operatorname {g a p} _ {j (i)} (A _ {\pi}) - \eta (\widehat {\psi} _ {i}) \mathcal {E} (\widehat {G}) ] _ {+}}. \tag {11}
$$
Proof Sketch. First, note that for compact self-adjoint operators  $|\widehat{\lambda}_i - \mu_{j(i)}| \leq \| (A_\pi - \widehat{\lambda}_i I)^{-1} \|^-$ . So, following the reasoning of [24] Theorem 1] and observing that  $\|(A_\pi S - SG) \widehat{\psi}_i\| / \| S \widehat{\psi}_i\| \leq \mathcal{E}(\widehat{G}) \eta(\widehat{\psi}_i)$  gives the right hand side of the first equation in [11]. Next, since additionally  $\|(A_\pi S - SG) \widehat{\psi}_i\| / \| S \widehat{\psi}_i\| \leq \mathcal{E}(\widehat{G}) \eta(\widehat{\psi}_i)$ , we can apply the Davis-Kahan spectral perturbation result for compact self-adjoint operators (Proposition 2, Appendix C) to bound  $\sin(\widehat{\theta})$ , where  $\widehat{\theta}_i := \triangleleft(f_i, f_{j(i)})$ . The claim then follows since  $\| \widehat{f}_i - f_{j(i)} \|^2 \leq 2(1 - \cos(\widehat{\theta}_i)) \leq 2\sin(\widehat{\theta}_i)$ . The full proof can be found in Appendix C.
Note that the error (9), at least for universal kernels, can be made arbitrary small, see [24, Proposition 1]. Still, the metric distortion may dominate the error and, since the bound (11) is tight, one may have that the operator is well estimated in norm, but the estimated eigenpairs are far from the true ones. This phenomenon is at the origin of spurious eigenvalues. The proposed way to detect them for deterministic systems in [14] is to check if eigenvalue equations are satisfied empirically, which, however, is not useful for stochastic systems, see Rem. of Appendix C.
Spuriousness may also originate from poor conditioning of the true eigenvalues, i.e. when the angle between true left and right eigenfunctions is small. Here, however, we assume  $A_{\pi} = A_{\pi}^{*}$ , so that we restrict ourselves to the case in which the only source of spuriousness is due to the learning method.
While we defer the discussion of the operator norm error to the next section, the following result bound the metric distortion; the proof can be found in Appendix C.
Proposition 1. Let  $\widehat{G} \in \mathrm{B}_r(\mathcal{H})$ . For all  $i \in [r]$  the metric distortion of  $\widehat{\psi}_i$  can be tightly bounded as
$$
1 / \sqrt {\| C \|} \leq \eta (\widehat {\psi_ {i}}) \leq \min  (| \widehat {\lambda_ {i}} | \operatorname {c o n d} (\widehat {\lambda_ {i}}), \| \widehat {G} \|) / \sigma_ {\min } ^ {+} (S \widehat {G}), \tag {12}
$$
where  $\operatorname{cond}(\widehat{\lambda}_i) \coloneqq \|\widehat{\xi}_i\| |\widehat{\psi}_i\| / |\langle \widehat{\psi}_i, \widehat{\xi}_i \rangle|$  is the condition number of  $\widehat{\lambda}_i$ , and  $\widehat{\xi}_i$  is its left eigenfunction.
The upper bound (12) depends on the estimator's eigenvalues and their conditioning. Notice that while the true eigenvalues of  $A_{\pi}$  have condition number one, the conditioning of the estimated ones depends on the choice of the kernel. Moreover, the upper bound can be controlled by tuning the estimator rank. Since the bound is tight (see Rem.3 in Appendix C), the metric distortion can grow with the rank of the estimator, further motivating the use of low-rank estimators of  $A_{\pi}$  in practice, see [26].
We end this section by introducing an empirical estimator of the metric distortion  $\eta (\hat{\psi}_i)$ , given by
$$
\widehat {\eta} _ {i} := \left\| \widehat {\psi} _ {i} \right\| / \sqrt {\langle \widehat {C} \widehat {\psi} _ {i} , \widehat {\psi} _ {i} \rangle}. \tag {13}
$$
Proposition 4 of Appendix C shows that  $\widehat{\eta}_i$  can be efficiently computed and report upper bounds for concentration around its mean. The empirical metric distortion (13), used in conjunction with the spectral bounds in Theorem 4 below, provides a proxy to assess the reliability of the PCR and RRR estimators and can be successfully used as novel model selection criterion. We refer the reader to the second and third experiment in Section 7 for concrete use-cases.
5 Controlling the Operator Norm Error
The HS norm error of the PCR estimator was already studied, either in the "well-specified" setting, i.e. when there exists  $G_{\mathcal{H}} \in \mathrm{HS}(\mathcal{H})$  such that  $A_{\pi}S = SG_{\mathcal{H}}$ , i.e.  $G_{\mathcal{H}}$  is  $\pi$ -a.e. Koopman operator [12]. Theorem B.10]. On the other hand, KRR estimator is studied also in the "misspecified setting" [29]. But, up to our knowledge, the operator norm error has not yet been studied. To analyse these learning rates, we make the following assumptions:
(RC) Regularity of  $A_{\pi}$ . For some  $\alpha \in (0,2]$  there exists  $a > 0$  such that  $TT^{*} \preceq a^{2}C^{1 + \alpha}$ ;
(BK) Boundedness. There exists  $c_{\mathcal{H}} > 0$  such that  $\operatorname{ess}\sup \| \phi(x)\|^2 \leq c_{\mathcal{H}}$ , i.e.  $\phi \in L_{\pi}^{\infty}(\mathcal{X},\mathcal{H})$ ;
(SD) Spectral Decay. There exists  $\beta \in (0,1]$  and  $b > 0$  such that  $\lambda_{j}(C)\leq bj^{-1 / \beta}$ , for all  $j\in J$ .