SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Giacomo Meanti*
giacomo.meanti@iit.it
Antoine Chatalic*
Universita di Genova
antoine.chatalic@dibris.unige.it
Vladimir R. Kostic
University of Novi Sad
vladimir.kostic@iit.it
Pietro Novelli
pietro.novelli@iit.it
Massimiliano Pontil
massimiliano.pontil@iit.it
lrosasco@mit.edu
The theory of Koopman operators allows to deploy non-parametric machine learning algorithms to predict and analyze complex dynamical systems. Estimators such as principal component regression (PCR) or reduced rank regression (RRR) in kernel spaces can be shown to provably learn Koopman operators from finite empirical observations of the system's time evolution. Scaling these approaches to very long trajectories is a challenge and requires introducing suitable approximations to make computations feasible. In this paper, we boost the efficiency of different kernel-based Koopman operator estimators using random projections (sketching). We derive, implement and test the new "sketched" estimators with extensive experiments on synthetic and large-scale molecular dynamics datasets. Further, we establish non-asymptotic error bounds giving a sharp characterization of the trade-offs between statistical learning rates and computational efficiency. Our empirical and theoretical analysis shows that the proposed estimators provide a sound and efficient way to learn large scale dynamical systems. In particular our experiments indicate that the proposed estimators retain the same accuracy of PCR or RRR, while being much faster. Code is available at https://github.com/Giodiro/NystromKoopman.
In the physical world, temporally varying phenomena are everywhere, from biological processes in the cell to fluid dynamics to electrical fields. Correspondingly, they generate large amounts of data both through experiments and simulations. This data is often analyzed in the framework of dynamical systems, where the state of a system  $\pmb{x}$  is observed at a certain time  $t$ , and the dynamics is described by a function  $f$  which captures its evolution in time
$$
\boldsymbol {x} _ {t + 1} = f (\boldsymbol {x} _ {t}).
$$
The function  $f$  must capture the whole dynamics, and as such it may be non-linear and even stochastic for instance when modeling stochastic differential equations, or simply noisy processes. Applications of this general formulation arise in fields ranging from robotics, atomistic simulations, epidemiology, and many more. Along with a recent increase in the availability of simulated data, data-driven techniques for learning the dynamics underlying physical systems have become commonplace. The typical approach of such techniques is to acquire a dataset of training pairs  $(x_{t}, y_{t} = x_{t + 1})$  sampled in time, and use them to learn a model for  $f$  which minimizes a forecasting error. Since dynamical systems stem from real physical processes, forecasting is not the only goal and the ability to interpret the dynamics is paramount. One particularly important dimension for interpretation is the separation of dynamics into multiple temporal scales: fast fluctuations can e.g. be due to thermodynamical noise or electrical components in the system, while slow dynamics describe important conformational changes in molecules or mechanical effects.
Koopman operator theory [27, 28] provides an elegant framework in which the potentially non-linear dynamics of the system can be studied via the Koopman operator
$$
(\mathcal {K} \psi) (\boldsymbol {x}) = \mathbf {E} [ \psi (f (\boldsymbol {x})) ], \tag {1}
$$
which has the main advantage of being linear but is defined on a typically infinite-dimensional set of observable functions. The expectation in (1) is taken with respect to the potential stochasticity of  $f$ . Thanks to its linearity, the operator  $\mathcal{K}$  can e.g. be applied twice to get two-steps-ahead forecasts, and one can compute its spectrum (beware however that  $\mathcal{K}$  is not self-adjoint, unless the dynamical process is time-reversible). Accurately approximating the Koopman operator and its spectral properties is of high interest for the practical analysis of dynamical systems. However doing so efficiently for long temporal trajectories remains challenging. In this paper we are interested in designing estimators which are both theoretically accurate and computationally efficient.
Related works Learning the spectral properties of the Koopman operator directly from data has been considered for at least three decades [39, 40], resulting in a large body of previous work. Among the different approaches proposed over time (see Mezić [41] for a recent review) it is most common to search for finite dimensional approximations to the operator, from which part of the spectrum and the Koopman modes [8] can be obtained. Dynamic mode decomposition (DMD) [56, 63], time-lagged independent component analysis (tICA) [42, 49] and many subsequent extensions [31] for example can be seen as minimizers of the forecasting error when  $\psi$  is restricted to be a linear function of the states [52]. Extended DMD (eDMD) [66, 25] and the variational approach for conformation dynamics (VAC) [45, 46] instead allow for a (potentially learnable, as in recent deep learning algorithms [32, 37, 69, 62]) dictionary of non-linear functions  $\psi$ . Kernel DMD [67, 26] and kernel tICA [57] are further generalizations which again approximate the Koopman operator but using an infinite dimensional space of features  $\psi$ , encoded by the feature map of a reproducing kernel. While often slow from a computational point of view, kernel methods are highly expressive and can be analyzed theoretically, to prove convergence and derive learning rates of the resulting estimators [29, 30]. Approximate kernel methods which are much faster to run have been recently used for Koopman operator learning by Baddoo et al. [6] where an iterative procedure is used to identify the best approximation to the full kernel, but no formal learning rates are demonstrated, and by Ahmad et al. [3] who derive learning rates in Hilbert-Schmidt norm (while we consider operator norm) for the Nyström KRR estimator (one of the three considered in this paper).
Contributions In this paper we adopt the kernel learning approach. Starting from the problem of approximating the Koopman operator in a reproducing kernel Hilbert space, we derive three different estimators based on different inductive biases: kernel ridge regression (KRR) which comes from Tikhonov regularization, principal component regression (PCR) which is equivalent to DMD and its extensions, and reduced rank regression (RRR) which comes from a constraint on the maximum rank of the estimator [24]. We show how to overcome the computational scalability problems inherent in full kernel methods using an approximation based on random projections which is known as the Nyström method [58, 65]. The approximate learning algorithms scale very easily to the largest datasets, with a computational complexity which goes from  $O(n^{3})$  for the exact algorithm to  $O(n^{2})$  for the approximate one. We can further show that the Nyström KRR, PCR and RRR estimators have the same convergence rates as their exact, slow counterparts – which are known to be optimal under our assumptions. We provide learning bounds in operator norm, which are known to translate to bounds for dynamic mode decomposition and are thus of paramount importance for applications. Finally, we thoroughly validate the approximate PCR and RRR estimators on synthetic dynamical systems, comparing efficiency and accuracy against their exact counterparts [29], as well as recently
proposed fast Koopman estimator streaming KAF [20]. To showcase a realistic scenario, we train on a molecular dynamics simulation of the fast-folding Trp-cage protein [35].
Structure of the paper We introduce the setting in Section 2, and define our three estimators in Section 3. In Section 4 we provide bounds on the excess risk of our estimators, and extensive experiments on synthetic as well as large-scale molecular dynamics datasets in Section 5.
Notation We consider a measurable space  $(\mathcal{X},\mathcal{B})$  where  $\mathcal{X}$  corresponds to the state space, and denote  $L_{\pi}^{2} \coloneqq L^{2}(\mathcal{X},\mathcal{B},\pi)$  the  $L^2$  space of functions on  $\mathcal{X}$  w.r.t. to a probability measure  $\pi$ , and  $L_{\pi}^{\infty}$  the space of measurable functions bounded almost everywhere. We denote  $\mathrm{HS}(\mathcal{H})$  the space of Hilbert-Schmidt operators on a space  $\mathcal{H}$ .
Setting The setting we will consider is that of Markovian, time-homogeneous stochastic process  $\{X_{t}\}_{t\in \mathbb{N}}$  on  $\mathcal{X}$ . By definition of a Markov process,  $X_{t}$  only depends on  $X_{t - 1}$  and not on any previous states. Time-homogeneity ensures that the transition probability  $\mathbb{P}\big[X_{t + 1}\in B|X_t = x\big]$  for any measurable set  $B$  does not depend on  $t$ , and can be denoted with  $p(\boldsymbol {x},B)$ . This implies in particular that the distribution of  $(X_{t},X_{t + 1})$  does not depend on  $t$ , and we denote it  $\rho$  in the following. We further assume the existence of the invariant density  $\pi$  which satisfies  $\pi (B) = \int_{\mathcal{X}}\pi (\pmb {x})p(\pmb {x},B)\mathrm{d}\pmb{x}$ . This classical assumption allows one to study a large class of stochastic dynamical systems, but also deterministic systems on the attractor, see e.g. [14]. The Koopman operator  $\kappa_{\pi}:L_{\pi}^{2}(\mathcal{X})\to L_{\pi}^{2}(\mathcal{X})$  is a bounded linear operator, defined by
$$
\left(\mathcal {K} _ {\pi} g\right) (\boldsymbol {x}) = \int_ {\mathcal {X}} p (\boldsymbol {x}, \boldsymbol {y}) g (\boldsymbol {y}) \mathrm {d} \boldsymbol {y} = \mathbf {E} \left[ g \left(X _ {t + 1}\right) \mid X _ {t} = \boldsymbol {x} \right], \quad g \in L _ {\pi} ^ {2} (\mathcal {X}), \boldsymbol {x} \in \mathcal {X}. \tag {2}
$$
We are in particular interested in the eigenpairs  $(\lambda_i, \varphi_i) \in \mathbb{C} \times L_\pi^2$ , that satisfy
$$
\mathcal {K} _ {\pi} \varphi_ {i} = \lambda_ {i} \varphi_ {i}. \tag {3}
$$
Through this decomposition it is possible to interpret the system by separating fast and slow processes, or projecting the states onto fewer dimensions [15, 19, 7]. In particular, the Koopman mode decomposition (KMD) allows to propagate the system state in time. Given an observable  $g: \mathcal{X} \to \mathbb{R}^d$  such that  $g \in \operatorname{span}\{\varphi_i | i \in \mathbb{N}\}$ , the modes allow to reconstruct  $g(\boldsymbol{x})$  with a Koopman eigenfunction basis. The modes  $\boldsymbol{\eta}_i^g \in \mathbb{C}^d$  are the coefficients of this basis expansion:
$$
\left(\mathcal {K} _ {\pi} g\right) (\boldsymbol {x}) = \mathbf {E} \left[ g \left(X _ {t}\right) \mid X _ {0} = \boldsymbol {x} \right] = \sum_ {i} \lambda_ {i} \varphi_ {i} (\boldsymbol {x}) \boldsymbol {\eta} _ {i} ^ {g}. \tag {4}
$$
This decomposition describes the system's dynamics in terms of a stationary component (the Koopman modes), a temporal component (the eigenvalues  $\lambda_{i}$ ) and a spatial component (eigenfunctions  $\varphi_{i}$ ).
Kernel-based learning In this paper we approximate  $\mathcal{K}_{\pi}$  with kernel-based algorithms, using operators in reproducing kernel Hilbert spaces (RKHS)  $\mathcal{H}$  associated with kernel  $k: \mathcal{X} \times \mathcal{X} \to \mathbb{R}$  and feature map  $\phi: \mathcal{X} \to \mathcal{H}$ . We wish to find an operator  $A: \mathcal{H} \to \mathcal{H}$  which minimizes the risk
$$
\mathcal {R} _ {\mathrm {H S}} (A) = \mathbf {E} _ {\rho} [ \ell (A, (\boldsymbol {x}, \boldsymbol {y})) ] \quad \text {w h e r e} \quad \ell (A, (\boldsymbol {x}, \boldsymbol {y})) := \| \phi (\boldsymbol {y}) - A \phi (\boldsymbol {x}) \| ^ {2}. \tag {5}
$$
The adjoint of  $A$ , denoted by  $A^{*}$ , should thus be understood as an estimator of the Koopman operator  $\mathcal{K}_{\pi}$  in  $\mathcal{H}$  as will be clarified in (15). In practice  $\pi$  and  $\rho$  are unknown, and one typically has access to a dataset  $\{(\pmb{x}_i,\pmb{y}_i)\}_{i = 1}^n$  sampled from  $\rho$ , where each pair  $(\pmb{x}_i,\pmb{y}_i = f(\pmb{x}_i))$  may equivalently come from a single long trajectory or multiple shorter ones concatenated together. We thus use the empirical risk
$$
\hat {\mathcal {R}} _ {\mathrm {H S}} (A) = \frac {1}{n} \sum_ {i = 1} ^ {n} \ell \left(A, \left(\boldsymbol {x} _ {i}, \boldsymbol {y} _ {i}\right)\right) \tag {6}
$$
as a proxy for (5). Since minimizing eq. (6) may require finding the solution to a very badly conditioned linear system, different regularization methods (such as Tikhonov or truncated SVD) can be applied on top of the empirical risk.
Remark 2.1 (Connections to other learning problems): The problem of minimizing eqs. (5) and (6) has strong connections to learning conditional mean embeddings [59, 22, 44, 33] where the predictors and targets are embedded in different RKHSs, and to structured prediction [12, 13] which is an even more general framework. On the other hand, the most substantial difference from the usual kernel regression setting [9] is the embedding of both targets and predictors into a RKHS, instead of just targets.
We denote the input and cross covariance  $C = \mathbf{E}_{\pi}[\phi(\boldsymbol{x}) \otimes \phi(\boldsymbol{x})]$  and  $C_{YX} = \mathbf{E}_{\rho}[\phi(\boldsymbol{y}) \otimes \phi(\boldsymbol{x})]$ , and their empirical counterparts as  $\hat{C} = \frac{1}{n}\sum_{i=1}^{n}[\phi(\boldsymbol{x}_i) \otimes \phi(\boldsymbol{x}_i)]$  and  $\hat{C}_{YX} = \frac{1}{n}\sum_{i=1}^{n}\phi(\boldsymbol{y}_i) \otimes \phi(\boldsymbol{x}_i)]$ . We also use the abbreviation  $C_{\lambda} := C + \lambda I$ . Minimizing the empirical risk (6) with Tikhonov regularization [9] yields the following KRR estimator
$$
\hat {A} _ {\lambda} = \underset {A \in \mathrm {H S} (\mathcal {H})} {\arg \min } \hat {\mathcal {R}} _ {\mathrm {H S}} (A) + \lambda \| A \| _ {\mathrm {H S}} ^ {2} = \hat {C} _ {Y X} (\hat {C} + \lambda I) ^ {- 1}. \tag {7}
$$
Eq. (7) can be computed by transforming its expression with the kernel trick [23], to arrive at a form where one must invert the kernel matrix - a  $n \times n$  matrix whose  $i, j$ -th entry is  $k(\pmb{x}_i, \pmb{x}_j)$ . This operation requires  $O(n^3)$  time and  $O(n^2)$  memory, severely limiting the scalability of KRR to  $n \lesssim 100000$  points. Improving the scalability of kernel methods is a well-researched topic, with the most important solutions being random features [50, 51, 68, 21] and random projections [58, 65, 21]. In this paper we use the latter approach, whereby the kernel matrix is assumed to be approximately low-rank and is sketched to a lower dimensionality. In particular we will use the Nyström method to approximate the kernel matrix projecting it onto a small set of inducing points, chosen among the training set. The sketched estimators are much more efficient than the exact ones, increasingly so as the training trajectories become longer. For example, the state of the art complexity for solving (non vector valued) approximate kernel ridge regression is  $O(n\sqrt{n})$  time instead of  $O(n^3)$  [38, 1, 10]. Furthermore, when enough inducing points are used (typically on the order of  $\sqrt{n}$ ), the learning rates of the exact and approximate estimators are the same, and optimal [5, 53]. Hence it is possible - and in this paper we show it for learning the Koopman operator - to obtain large efficiency gains, without losing anything in terms of theoretical guarantees of convergence.
3 Nyström estimators for Koopman operator regression
In this section, we introduce three efficient approximations of the KRR, PCR and RRR estimators of the Koopman operator. Our estimators rely on the Nyström approximation, i.e. on random projections onto low-dimensional subspaces of  $\mathcal{H}$  spanned by the feature-embeddings of subsets of the data. We thus consider two sets of  $m \ll n$  inducing points  $\{\tilde{\pmb{x}}_j\}_{j=1}^m \subset \{\pmb{x}_t\}_{t=1}^n$  and  $\{\tilde{\pmb{y}}_j\}_{j=1}^m \subset \{\pmb{y}_t\}_{t=1}^n$  sampled respectively from the input and output data. The choice of these inducing points (also sometimes called Nyström centers) is important to obtain a good approximation. Common choices include uniform sampling, leverage score sampling [17, 55], and iterative procedures such as the one used in [6] to identify the most relevant centers. In this paper we focus on uniform sampling for simplicity, but we stress that our theoretical results in Section 4 can easily be extended to leverage scores sampling by means of [53, Lemma 7]. To formalize the Nyström estimators, we define operators  $\widetilde{\Phi}_X, \widetilde{\Phi}_Y: \mathbb{R}^m \to \mathcal{H}$  as  $\widetilde{\Phi}_X w = \sum_{j=1}^m w_j \phi(\tilde{\pmb{x}}_j)$  and  $\widetilde{\Phi}_Y w = \sum_{j=1}^m w_j \phi(\tilde{\pmb{y}}_j)$ , and denote  $P_X$  and  $P_Y$  the orthogonal projections onto span  $\widetilde{\Phi}_X$  and span  $\widetilde{\Phi}_Y$  respectively.
In the following paragraphs we apply the projection operators to three estimators corresponding to different choices of regularization. For each of them a specific proposition (proven in Section C) states an efficient way of computing it based on the kernel trick. For this purpose we introduce the kernel matrices  $K_{\tilde{X},X}$ ,  $K_{\tilde{Y},Y} \in \mathbb{R}^{m \times n}$  between training set and inducing points with entries  $(K_{\tilde{X},X})_{ji} = k(\tilde{\boldsymbol{x}}_j, \boldsymbol{x}_i)$ ,  $(K_{\tilde{Y},Y})_{ji} = k(\tilde{\boldsymbol{y}}_j, \boldsymbol{y}_i)$ , and the kernel matrices of the inducing points  $K_{\tilde{X},\tilde{X}}$ ,  $K_{\tilde{Y},\tilde{Y}} \in \mathbb{R}^{m \times m}$  with entries  $(K_{\tilde{X},X})_{jk} = k(\tilde{\boldsymbol{x}}_j, \tilde{\boldsymbol{x}}_k)$  and  $(K_{\tilde{X},X})_{jk} = k(\tilde{\boldsymbol{y}}_j, \tilde{\boldsymbol{y}}_k)$ .
Kernel Ridge Regression (KRR) The cost of computing  $\hat{A}_{\lambda}$  defined in Eq. (7) is  $O(n^{3})$  [29] which is prohibitive for datasets containing long trajectories. However, applying the projection operators to each side of the empirical covariance operators, we obtain an estimator which additionally depends on the  $m$  inducing points:
$$
\hat {A} _ {m, \lambda} ^ {\mathrm {K R R}} := P _ {Y} \hat {C} _ {Y X} P _ {X} \left(P _ {X} \hat {C} P _ {X} + \lambda I\right) ^ {- 1}: \mathcal {H} \rightarrow \mathcal {H}. \tag {8}
$$
If  $\mathcal{H}$  is infinite dimensional, Eq. (8) cannot be computed directly. Proposition 3.1 (proven in Section C) provides a computable version of the estimator.
Proposition 3.1 (Nyström KRR): The Nyström KRR estimator (8) can be expressed as
$$
\hat {A} _ {m, \lambda} ^ {K R R} = \widetilde {\Phi} _ {Y} K _ {\tilde {Y}, \tilde {Y}} ^ {\dagger} K _ {\tilde {Y}, Y} K _ {X, \tilde {X}} \left(K _ {\tilde {X}, X} K _ {X, \tilde {X}} + n \lambda K _ {\tilde {X}, \tilde {X}}\right) ^ {\dagger} \widetilde {\Phi} _ {X} ^ {*}. \tag {9}
$$
The computational bottlenecks are the inversion of an  $m \times m$  matrix and a large matrix multiplication, which overall need  $O(2m^3 + 2m^2 n)$  operations. In particular, in Section 4 we will show that  $m \asymp \sqrt{n}$  is sufficient to guarantee optimal rates even with minimal assumptions, leading to a final cost of  $O(n^2)$ . Note that a similar estimator was derived in [3].
Please note that the  $O(n^2)$  cost is for a straightforward implementation, and can indeed be reduced via iterative linear solvers (possibly preconditioned, to further reduce the practical running time), and randomized linear algebra techniques. In particular, we could leverage results from Rudi et al. [54] to reduce the computational cost to  $O(n\sqrt{n})$ .
Principal Component Regression (PCR) Typical settings in which Koopman operator theory is used focus on the decomposition of a dynamical system into a small set of components, obtained from the eigendecomposition of the operator itself. For this reason, a good prior on the Koopman estimator is for it to be low rank. The kernel PCR estimator  $\hat{A}^{\mathrm{PCR}} = \hat{C}_{YX}[[\hat{C}]]_r^\dagger$  formalizes this concept [29, 67], where here  $[\cdot ]_r$  denotes the truncation to the first  $r$  components of the spectrum. Again this is expensive to compute when  $n$  is large, but the estimator can be sketched as follows:
$$
\hat {A} _ {m} ^ {\mathrm {P C R}} = P _ {Y} \hat {C} _ {Y X} \llbracket P _ {X} \hat {C} P _ {X} \rrbracket_ {r} ^ {\dagger}. \tag {10}
$$
The next proposition provides an efficiently implementable version of this estimator.
Proposition 3.2 (Nyström PCR): The sketched PCR estimator (10) satisfies
$$
\hat {A} _ {m} ^ {P C R} = \widetilde {\Phi} _ {Y} K _ {\tilde {Y}, \tilde {Y}} ^ {\dagger} K _ {\tilde {Y}, Y} K _ {X, \tilde {X}} [   [ K _ {\tilde {X}, \tilde {X}} ^ {\dagger} K _ {\tilde {X}, X} K _ {X, \tilde {X}} ]   ] _ {r} \widetilde {\Phi} _ {X} ^ {*} \tag {11}
$$
requiring  $O(2m^3 + 2m^2 n)$  operations, i.e. optimal rates can again be obtained at a cost of at most  $O(n^2)$  operations.
Note that with  $m = n$ ,  $\hat{A}_m^{\mathrm{PCR}}$  is equivalent to the kernel DMD estimator [67], also known as kernel analog forecasting (KAF) [4]. The sketched estimator of Proposition 3.2 was also recently derived in [6], albeit without providing theoretical guarantees.
Reduced Rank Regression (RRR) Another way to promote low-rank estimators is to add an explicit rank constraint when minimizing the empirical risk. Combining such a constraint with Tikhonov regularization corresponds to the reduced rank regression [24, 29] estimator:
$$
A _ {\lambda} ^ {\text {R R R}} = \underset {A \in \mathrm {H S}: \operatorname {r k} (A) \leq r} {\arg \min } \hat {\mathcal {R}} _ {\mathrm {H S}} (A) + \lambda \| A \| _ {\mathrm {H S}} ^ {2}. \tag {12}
$$
Minimizing Eq. (12) requires solving a  $n \times n$  generalized eigenvalue problem. The following proposition introduces the sketched version of this estimator, along with a procedure to compute it which instead requires the solution of a  $m \times m$  eigenvalue problem. For  $m \asymp \sqrt{n}$ , which is enough to guarantee optimal learning rates with minimal assumptions (see Section 4), this represents a reduction from  $O(n^3)$  to  $O(n\sqrt{n})$  time.
Proposition 3.3 (Nyström RRR): The Nyström RRR estimator can be written as
$$
\hat {A} _ {m, \lambda} ^ {R R R} = \llbracket P _ {Y} \hat {C} _ {Y X} P _ {X} \left(P _ {X} \hat {C} P _ {X} + \lambda I\right) ^ {- 1 / 2} \rrbracket_ {r} \left(P _ {X} \hat {C} P _ {X} + \lambda I\right) ^ {- 1 / 2}. \tag {13}
$$
To compute it, solve the  $m \times m$  eigenvalue problem
$$
\left(K _ {\tilde {X}, X} K _ {X, \tilde {X}} + n \lambda K _ {\tilde {X}, \tilde {X}}\right) ^ {\dagger} K _ {\tilde {X}, X} K _ {Y, \tilde {Y}} K _ {\tilde {Y}, \tilde {Y}} ^ {\dagger} K _ {\tilde {Y}, Y} K _ {X, \tilde {X}} w _ {i} = \sigma_ {i} ^ {2} w _ {i}
$$
for the first  $r$  eigenvectors  $W_{r} = [w_{1},\ldots ,w_{r}]$ , appropriately normalized. Then denoting  $D_r\coloneqq K_{\tilde{Y},\tilde{Y}}^\dagger K_{\tilde{Y},Y}K_{X,\tilde{X}}W_r$  and  $E_{r}\coloneqq (K_{\tilde{X},X}K_{X,\tilde{X}} + n\lambda K_{\tilde{X},\tilde{X}})^{\dagger}K_{\tilde{X},X}K_{Y,\tilde{Y}}D_r$  it holds
$$
\hat {A} _ {m, \lambda} ^ {R R R} = \widetilde {\Phi} _ {Y} D _ {r} E _ {r} ^ {*} \widetilde {\Phi} _ {X} ^ {*}. \tag {14}
$$
4 Learning bounds in operator norm for the sketched estimators
In this section, we state the main theoretical results showing that optimal rates for operator learning with KRR, PCR and RRR can be reached with Nyström estimators.
Assumptions We first make two assumptions on the space  $\mathcal{H}$  used for the approximation, via its reproducing kernel  $k$ .
Assumption 4.1 (Bounded kernel): There exists  $K < \infty$  such that  $\operatorname{ess}\sup_{\boldsymbol{x} \sim \pi} \| \phi(\boldsymbol{x}) \| \leq K$ .
Assumption 4.1 ensures that  $\mathcal{H}$  is compactly embedded in  $L_{\pi}^{2}$  [61, Lemma 2.3], and we denote  $\Phi_X^*:\mathcal{H}\to L_\pi^2$  the embedding operator which maps any function in  $\mathcal{H}$  to its equivalence class  $\pi$ -almost everywhere in  $L_{\pi}^{2}$ .