SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Petar Bevanda
petar.bevanda@tum.de
Max Beier
max.beier@tum.de
Armin Lederer
armin.lederer@tum.de
Stefan Sosnowski
sosnowski@tum.de
Eyke Hüllermeier
eyke@ifi.lmu.de
Sandra Hirche
hirche@tum.de
Many machine learning approaches for decision making, such as reinforcement learning, rely on simulators or predictive models to forecast the time-evolution of quantities of interest, e.g., the state of an agent or the reward of a policy. Forecasts of such complex phenomena are commonly described by highly nonlinear dynamical systems, making their use in optimization-based decision-making challenging. Koopman operator theory offers a beneficial paradigm for addressing this problem by characterizing forecasts via linear time-invariant (LTI) ODEs, turning multi-step forecasts into sparse matrix multiplication. Though there exists a variety of learning approaches, they usually lack crucial learning-theoretic guarantees, making the behavior of the obtained models with increasing data and dimensionality unclear. We address the aforementioned by deriving a universal Koopman-invariant reproducing kernel Hilbert space (RKHS) that solely spans transformations into LTI dynamical systems. The resulting Koopman Kernel Regression (KKR) framework enables the use of statistical learning tools from function approximation for novel convergence results and generalization error bounds under weaker assumptions than existing work. Our experiments demonstrate superior forecasting performance compared to Koopman operator and sequential data predictors in RKHS.
Dynamical systems theory is a fundamental paradigm for understanding and modeling the time evolution of a phenomenon governed by certain underlying laws. Such a perspective has been successful in describing countless real-world phenomena, ranging from engineering mechanics [1] and human movement modeling [2] to molecular and quantum systems [3, 4]. However, as the laws governing dynamical systems are often unknown, modeling and understanding the underlying phenomena may have to rely on data rather than first principles. In this regard, machine learning methods, which have shown immense potential in tackling complex tasks in domains such as language models [5] and computer vision [6], are coming to the fore. Though powerful, state-of-the-art neural vector fields [7] or flows [8] commonly compose highly nonlinear maps for forecast, i.e. computing
$$
x (t) = x (0) + \int_ {0} ^ {t} f (x (t)) \mathrm {d} t \tag {1}
$$
for, e.g. a scalar ODE  $\dot{x} = f(x)$ . Hence, it is often challenging to use such models in optimization-based decision making that relies on simulators or predictive models, e.g., reinforcement learning [9-11]. A particularly beneficial perspective for dealing with the aforementioned comes from Koopman operator theory [12-15]. Through a point-spectral decomposition of Koopman operators, forecasts become superpositions of solution curves of a set of linear ODEs  $\{\dot{z}_j = \lambda_j z_j\}_{j=1}^D$
$$
x (t) = \sum_ {k = 1} ^ {D} \mathrm {e} ^ {\lambda_ {j} t} z _ {j} (0), \quad \{x \stackrel {\text {l i f t}} {\longmapsto} g _ {j} z _ {j} \} _ {j = 1} ^ {D} \tag {2}
$$
where a vector valued function  $\mathrm{span}(\{g_j\}_{j=1}^D)$  "lifts"  $x$  onto a manifold  $\mathbb{Z} := \mathrm{span}\left(\{z_j\}_{j=1}^D\right)$ . Throughout, we refer to these models as linear time-invariant (LTI) predictors. The learning objective of such representations is twofold: spanning system trajectories by the learned manifold  $\mathbb{Z}$  and constraining the LTI dynamics to it. The latter is a long-standing challenge of Koopmanism [16-20], as manifold dynamics of existing approaches "leak-out" [21] and limit predictive performance.
To tackle the aforesaid, we connect the representation theories of reproducing kernel Hilbert spaces (RKHS) and Koopman operators. As a first in the literature, we derive a universal kernel whose RKHS exclusively spans manifolds invariant under the dynamics, as depicted in Figure 1. A key corollary of
Figure 1: Illustration of on-manifold dynamics of LTI predictors.
unconstrained manifold dynamics is the lack of essential learning-theoretic guarantees, making the behavior of existing learned models unclear for increasing data and dimensionality. To address this, we utilize equivalences to function regression in RKHS to formalize a statistical learning framework for learning LTI predictors from sample trajectories of a dynamical system. This, in turn, enables the use of statistical learning tools from function approximation for novel convergence results and generalization error bounds under weaker assumptions than before [22-24]. Thus, we believe that our Koopman Kernel Regression (KKR) framework takes the best of both RKHS and Koopmanism by leveraging modular kernel learning tools to build provably effective LTI predictors.
The remainder of this paper is structured as follows: We briefly introduce LTI predictors and discuss related work in Section 2. The derivation of the KKR framework, including the novel Koopman RKHS, is presented in Section 3. In Section 4, we show the novel learning guarantees in terms of convergence and generalization error bounds. They are validated in comparison to the state-of-the-art through numerical experiments in Section 5.
Notation Lower/upper case bold symbols  $\boldsymbol{x} / \boldsymbol{X}$  denote spatial vector/matrix-valued quantities. A trajectory defines a curve  $\boldsymbol{x}_T \subset \mathbb{X}$  traced out by the flow over time  $\mathbb{T} = [0, T]$  from any  $(\tau, \boldsymbol{x}) \in \mathbb{T} \times \mathbb{X}$ . In discretizing  $\mathbb{T}$ , collection of points  $\boldsymbol{x}_{\mathrm{H}} \subset \mathbb{X}$  from discrete time steps  $\mathbb{H} = \{t_0, \dots, t_{\mathrm{H}}\}$  is considered. The state/output trajectory spaces are denoted as  $\mathbb{X}_T \subseteq L^2(\mathbb{T}, \mathbb{X}) / \mathbb{Y}_T \subseteq L^2(\mathbb{T}, \mathbb{Y})$ , with discrete-time analogues  $\mathbb{X}_{\mathrm{H}} \subseteq \ell^2(\mathbb{H}, \mathbb{X}) / \mathbb{Y}_{\mathrm{H}} \subseteq \ell^2(\mathbb{H}, \mathbb{Y})$  with domain and co-domain separated by “,”. The vector space of continuous functions on  $\mathbb{X}_T$  endowed with the topology of uniform convergence on compact domain subsets is denoted  $C(\mathbb{X}_T)$ . The collection of bounded linear operators from  $\mathbb{Y}_T$  to  $\mathbb{Y}_T$  is denoted as  $\mathcal{B}(\mathbb{Y}_T)$ . The adjoint of  $\mathcal{A} \in \mathcal{B}(\cdot)$  is  $\mathcal{A}^*$ . Discrete-time eigenvalues read  $\mu := e^{\lambda \Delta t}, \lambda \in \mathbb{C}$ . A random variable  $X$  defined on a probability space  $(\Omega, \mathcal{A}, \rho)$  has expectation  $\mathbb{E}[X] = \int_{\Omega} X(\omega) \rho(\omega)$ .
2 Problem Statement and Related Work
To begin, we formalize our problem statement and put our work into into context with existing work.
2.1 Problem Statement
Consider a forward-complete system comprising a nonlinear state-space model
$$
\dot {\boldsymbol {x}} = \boldsymbol {f} (\boldsymbol {x}), \boldsymbol {x} _ {0} = \boldsymbol {x} (0), \tag {3a}
$$
$$
y = q (\boldsymbol {x}), \tag {3b}
$$
on a compact domain  $\mathbb{X} \subset \mathbb{R}^d$  with a quantity of interest  $q \in C(\mathbb{X})$ . The above system class includes all systems with Lipschitz flow  $\pmb{F}^{t}(\pmb{x}_{0}) \coloneqq \int_{0}^{t} \pmb{f}(\pmb{x}(\tau)) d\tau$ , e.g., mechanical systems [27].
Inspired by the spectral decomposition of Koopman operators, we look to replace the nonlinear state-space model (3) by an LTI predictor
$$
\dot {z} = \boldsymbol {A} z, \quad z _ {0} = \boldsymbol {g} \left(\boldsymbol {x} _ {0}\right), \tag {4a}
$$
$$
y = \boldsymbol {c} ^ {\top} z, \tag {4b}
$$
<sup>1</sup>Although we outline the scalar output case for ease of exposition, expanding to a vector-valued case is possible w.l.o.g. If required, forward completeness can be relaxed to unboundedness observability [25, 26].
with  $\bar{D} \in \mathbb{N}$  and  $g$  a  $\bar{D}$ -dimensional function approximator dense in  $C(\mathbb{X})$ . Then, from initial conditions  $\mathbb{X}_0 \subseteq \mathbb{X}$  that form a non-recurrent domain  $\mathbb{X}_T$ , (4) admits a universal approximation of the flow of (3) such that  $\forall \varepsilon > 0$ ,  $\exists \bar{D}$  so that  $\sup_{\boldsymbol{x} \in \mathbb{X}_0} |y_T(t) - \boldsymbol{c}^\top \mathrm{e}^{\boldsymbol{A}t} g(\boldsymbol{x})| < \varepsilon$ ,  $\forall t \in [0,T]$  [28]^2. In this work, we aim to find a solution to the following constrained, functional optimization problem (OR) Output reconstruction:
$$
\min  _ {\boldsymbol {c}, \boldsymbol {g}, \boldsymbol {A}} \| y _ {T} - \boldsymbol {c} ^ {\top} \boldsymbol {g} _ {T} \| _ {\mathbb {Y} _ {T}}, \tag {5a}
$$
(KI) Koopman-invariance:
$$
\text {s u c h} \quad \boldsymbol {g} (\boldsymbol {x} (t)) = \mathrm {e} ^ {\boldsymbol {A} t} \boldsymbol {g} (\boldsymbol {x} (0)), \quad \forall t \in [ 0, T ]. \tag {5b}
$$
Although the sought-out model (4) is simple, the above problem is non-trivial and much of the existing body of work utilizes different simplifications that often lead to undesirable properties. In the following, we elaborate on these properties and motivate our novel sample-based solution to (5), which remains relatively simple but nonetheless ensures a well-defined solution with strong learning guarantees.
Koopman operator regression in RKHS Equipped with a rich set of estimators, operator regression in RKHS seeks a sampled-data solution to
$$
\min  _ {\boldsymbol {A}} \| \boldsymbol {g} (\boldsymbol {x} (t)) - \boldsymbol {A} ^ {*} \boldsymbol {g} (\boldsymbol {x} (0)) \| _ {L ^ {2}}, \tag {6}
$$
with  $A$  a Hilbert-Schmid operator [29] — commonly known as KRR, and EDMD (PCR) or RRR when under different fixed-rank constraints [23, 24]. The choice of RKHS  $g$  is commonly one that is dense in a suitable  $L^2$  space, e.g. that of the RBF kernel. By an additional projection, a quantity of interest can be predicted via a mode decomposition of the estimated operator, leading to a model akin to (4). In the light of (5), the feature map  $g$  is predetermined while violating (KI) is merely minimized for a single time-instant  $t$ . As a consequence, such approaches are oblivious to the time-series structure — offering limited predictive power over the time interval  $[0, T]$  of a trajectory as displayed in Figure 1. The extent to which (KI) is violated due to spectral properties [30] or estimator bias [24] is known as spectral pollution [31]. The strong implications of this phenomena, motivate regularization [32] and spectral bias measures [24] to reduce its effects. Due to the above challenges, guarantees for Koopman operator regression (KOR) have only recently gained increased attention. Often, however, existing theoretical results [29, 33] are generally not applicable to nonlinear dynamics [34] due to the commonly unavoidable misspecification [35] of the problem (6) incurred by neglecting (KI). The first more general statistical learning results [23, 24] are derived in a stochastic setting under the assumption that the underlying operator is compact and self-adjoint. In stark contrast, the same set of assumptions is restrictive for the deterministic setting [35]: compactness only holds for affine deterministic dynamics [36, 37] while self-adjointness is known to generally not hold for Koopman operators [13, 38, 39]. Regardless of the setting, however, the state-of-the-art exhibits alarming properties: forecasting error not necessarily vanishing with LTI predictor (4) rank [23, Theorem 1] and risk based on a single time-instant.
Learning via Koopman eigenspaces Geared towards LTI predictors and closer to our own problem setting (2), another distinct family of approaches aims to directly learn the operator's invariant subspaces [28, 40-42]. The goal is to fit  $g(\cdot)$  based on approximate Koopman operator eigenfunctions that still fit the output of interest (OR). However, existing data-driven approaches in this line of work rely on ad-hoc choices and lack essential learning-theoretic properties such as feasibility and uniqueness of solutions — prohibiting provably accurate and automated LTI predictor learning.
Kernels for sequential data Motivated by the lack of priors that naturally incorporate streaming and sequential data, there is an increasing interest in signatures [43]. They draw from the rich theory of controlled differential equations (CDEs) [44, 45] and build models that depend on a time-varying observation history. An RKHS suitable for sequence modeling is induced by a signature transformation of a base/static RKHS. Generally, if the latter is universal, so are the signature kernels [46]. While arguably more general and well-versed for discriminative and generative tasks [47], forecasting using signature kernels [48] comes at a price, as their nonlinear dependence on observation streams leads to a significant complexity increase compared to LTI predictors.
2 Background on prerequisite Koopman operator theory can be found in the supplementary material.
Motivated by the restrictions of existing Koopman-based predictors, we propose a function approximation approach that exploits exploits time-series data and Koopman operator theory to provably learn LTI predictors. Through a novel invariance transform we can satisfy (KI) by construction and directly minimize the forecasting risk over an entire time-interval (OR). In simple terms: Koopman operator regression fixes  $g(\cdot)$  and regresses  $A$  and  $c$  in (5), whereas our KKR approach selects  $A$  to jointly regress  $c$  and  $g(\cdot)$ . Similar in spirit to generalized Laplace analysis [21, 49], our approach allows the construction of eigenmodes from data without inferring the operator itself. Crucially, we demonstrate that selecting  $A$  requires no prior knowledge as confirmed by our theoretical results and experiments. To facilitate learning LTI predictors, we derive universal RKHSs that are guaranteed to satisfy (KI) over trajectories — a first in the literature. The resulting equivalences to function regression in RKHS allow for more general and complete learning guarantees in terms of consistency and risk bounds that are free of restrictive operator-theoretic assumptions.
3 Koopman Kernel Regression
With the optimization (5) being prohibitively hard due to nonlinear and possibly high dimensional constraints, we eliminate the constraints (5b) by enforcing the feature map  $g(\cdot)$  to have the dynamics of intrinsic LTI coordinates associated with Koopman operators, i.e., their (open) eigenfunctions [21].
Definition 1. A Koopman eigenfunction  $\phi_{\lambda \in \mathbb{C}} \in C(\mathbb{X})$  satisfies  $\phi_{\lambda}(\pmb{x}) = \mathrm{e}^{-\lambda t}\phi_{\lambda}(\pmb{F}^t (\pmb{x}))$ ,  $\forall t \in [0, T]$ .
It is proven that Koopman eigenfunctions from Definition 1 are universal approximators of continuous functions [28] — making them a viable replacement for the feature map  $g(\cdot)$  in (4). However, following their definition, it is evident that Koopman eigenfunctions are by no means arbitrary due to their inherent dependence on the dynamics' flow. Using the well-established fact that Koopman operators compose a function with the flow, i.e.,  $\mathcal{K}^t g(\cdot) = g(\pmb{F}^t(\cdot))$ , it becomes evident the eigenfunctions from Definition 1 are (semi)group invariants, as they remain unchanged after applying  $\{\mathrm{e}^{-\lambda t}\mathcal{K}^t\}_{t=0}^T$ . Thus, inspired by the seminal work of Hurwitz on constructing invariants [50], we can equivalently reformulate (5) as an unconstrained problem and jointly optimize over eigenfunctions<sup>3</sup>.
Lemma 1 (Invariance transform). Consider a function  $g \in C(\mathbb{X}_0)$  over a set of initial conditions  $\mathbb{X}_0 \subseteq \mathbb{X}$  that form a non-recurrent domain  $\mathbb{X}_T$ . The invariance transform  $\mathcal{I}_{\lambda}^{T}$  transforms  $g$  into an Koopman eigenfunction  $\phi_{\lambda} \in C(\mathbb{X}_T)$  for (3a) with LTI dynamics described by  $\lambda \in \mathbb{C}$
$$
\phi_ {\lambda} \left(\boldsymbol {x} _ {T}\right) = \mathcal {I} _ {\lambda} ^ {T} g \left(\boldsymbol {x} _ {0}\right) := \int_ {\tau = 0} ^ {T} \mathrm {e} ^ {- \lambda (\tau - t)} g \left(\boldsymbol {F} ^ {\tau} \left(\boldsymbol {x} _ {0}\right)\right) d \tau . \tag {7}
$$
The above Lemma 1 is a key stepping stone towards deriving a representative theorem for LTI predictors. However, it is also interesting in its own right as it provides an explicit expression for the flow of an eigenfunction from any point in the state space. Thus, it provides a recipe to obtain a function space that fulfills (KI) by construction. As we show in the following, a sufficiently rich set of eigenvalues [28] and Lemma 1 will allow for a reformulation of (5) into an unconstrained problem
$$
\min  _ {M} \| y _ {T} - M \left(\boldsymbol {x} _ {T}\right) \| _ {\mathbb {Y} _ {T}}. \tag {8}
$$
where the operator  $M(\cdot) \coloneqq \mathbf{1}^{\top}[\phi_{\lambda_1}(\cdot)\dots \phi_{\lambda_{\bar{D}}}(\cdot)]^{\top}$  is universal and consisting of Koopman-invariant functions.
3.1 Functional Regression Problem
Notice that the problem reformulation (8) is still intractable, as a closed-form expression for the flow map is generally unavailable even for known ODEs. This requires integration schemes that can introduce inaccuracies over a time interval  $[0,T]$ . Thus, to make the above optimization problem tractable, data samples are used — ubiquitous in learning dynamical systems.
Assumption 1. A collection of  $N$  pairs of trajectories  $\mathbb{D}_N = \{\pmb{x}_T^{(i)},y_T^{(i)}\}_{i = 1}^N\in (\mathbb{X}_T\times \mathbb{Y}_T)^N$  is available.
By aggregating different invariance transformations (7) into the mode decomposition operator
$$
M (\cdot) \equiv \sum_ {j = 1} ^ {\bar {D}} \phi_ {\lambda_ {j}} (\cdot): \mathbb {X} _ {T} \mapsto \mathbb {Y} _ {T}, \tag {9}
$$
<sup>3</sup>Proofs for all theoretical results can be found in the supplementary material.
we can formulate a supervised learning approach in the following.
Learning Problem With Assumption 1 and Lemma 1, the sample-based approximation of problem (8) reduces to solving
$$
\min  _ {M} \sum_ {i = 1} ^ {N} \| y _ {T} ^ {(i)} - M \left(\boldsymbol {x} _ {T} ^ {(i)}\right) \| _ {\mathbb {Y} _ {T}}. \tag {10}
$$
while preserving the mode decomposition structure (9). To realize the above learning problem, we resort to the theory of reproducing kernels [51, 52] and look for an operator  $\hat{M} \in \mathcal{H}$ , where  $\mathcal{H}$  is an RKHS. A well-established approach using RKHS theory is to select  $\hat{M}$  as a solution to the regularized least squares problem
$$
\hat {M} = \underset {M \in \mathcal {H}} {\arg \min } \sum_ {i = 1} ^ {N} \| y _ {T} ^ {(i)} - M \left(\boldsymbol {x} _ {T} ^ {(i)}\right) \| _ {\mathbb {Y} _ {T}} ^ {2} + \gamma \| M \| _ {\mathcal {H}} ^ {2}, \tag {11}
$$
with  $\gamma \in \mathbb{R}_+$  and  $\| \cdot \|_{\mathcal{H}}$  a corresponding RKHS norm. As our target is a function-valued mapping  $M(\cdot)$  - an operator  $-\| \cdot \|_{\mathcal{H}}$  is induced by an operator-valued kernel  $K\colon \mathbb{X}_T\times \mathbb{X}_T\mapsto \mathcal{B}(\mathbb{Y}_T)$  mapping to the space of bounded operators over the output space [53]. The salient feature of the above formulation (11) is its well-posedness: its solution exists and is unique for any  $\mathcal{H}$ , expressed as
$$
\hat {M} (\cdot) = \sum_ {i = 1} ^ {N} K (\cdot , \boldsymbol {x} _ {T} ^ {(i)}) \beta_ {i}, \quad \beta_ {i} \in \mathbb {Y} _ {T} \tag {12}
$$
through a representative theorem [54]. Still, due to the Koopman-invariant structure (9) from Lemma 1, the choice of the RKHS  $\mathcal{H}$  for  $\hat{M}$  is not arbitrary. Thus, the question is how to craft  $\mathcal{H}$  so the solution  $\hat{M}$  is decomposable into Koopman operator eigenfunctions (9), forming an LTI predictor.
Firstly, it is obvious that (9) consists of summands that may lie in different RKHS, denoted as  $\{\mathcal{H}^{\lambda_j}\}_{j = 1}^D$ . Then,  $\mathcal{H}$  is constructed from the following direct sum of Hilbert spaces [55]:
$$
\tilde {\mathcal {H}} = \mathcal {H} ^ {\lambda_ {1}} \oplus \dots \oplus \mathcal {H} ^ {\lambda_ {\bar {D}}} \text {s o t h a t} \mathcal {H} = \operatorname {r a n g e} (\mathcal {S}) := \left\{f _ {1} + \dots + f _ {\bar {D}}: f _ {1} \in \mathcal {H} ^ {\lambda_ {1}}, \dots , f _ {\bar {D}} \in \mathcal {H} ^ {\lambda_ {\bar {D}}} \right\} \tag {13}
$$
with  $\mathcal{S}:\tilde{\mathcal{H}}\to \mathcal{H},(f_1\dots f_{\bar{D}})\mapsto f_1 + \ldots +f_{\bar{D}}$  the summation operator [56]. Thus, to construct  $\mathcal{H}$ , a specification of the RKHS collection  $\{\mathcal{H}^{\lambda_j}\}_{j = 1}^{\bar{D}}$  is required, so that it represents Koopman eigenfunctions from (9).
Theorem 1 (Koopman eigenfunction kernel). Consider trajectory data  $\{\pmb{x}_T^{(i)}\}_{i=1}^N$  from Assumption 1,  $a \lambda \in \mathbb{C}$  and a universal (base) kernel  $k: \mathbb{X} \times \mathbb{X} \mapsto \mathbb{R}$ . Then, the kernel  $K^\lambda: \mathbb{X}_T \times \mathbb{X}_T \mapsto \mathcal{B}(\mathbb{Y}_T)$
$$
K ^ {\lambda} \left(\boldsymbol {x} _ {T}, \boldsymbol {x} _ {T} ^ {\prime}\right) = \int_ {\tau = 0} ^ {T} \int_ {\tau^ {\prime} = 0} ^ {T} \mathrm {e} ^ {- \lambda (\tau - t)} k \left(\boldsymbol {x} _ {T} (\tau), \boldsymbol {x} _ {T} ^ {\prime} \left(\tau^ {\prime}\right)\right) \mathrm {e} ^ {- \lambda^ {*} \left(\tau^ {\prime} - t\right)} d \tau d \tau^ {\prime}, \tag {14}
$$
(i) defines an RKHS  $\mathcal{H}^{\lambda}$
(ii) is universal for every eigenfunction of Definition 1 corresponding to  $\lambda$ ,
(iii) induces a data-dependent function space span  $\{K^{\lambda}(\cdot ,\pmb{x}_T^{(1)}),\dots ,K^{\lambda}(\cdot ,\pmb{x}_T^{(N)})\}$  that is Koopman-invariant over trajectory-data  $\{\pmb{x}_T^{(i)}\}_{i = 1}^N$
In Theorem 1, we derive an eigenfunction RKHS by defining its corresponding kernel that embeds the invariance transformation (7) over data samples. Also, we would like to highlight that the above result addresses a long-standing open challenge in the Koopman operator community [19-21], i.e., defining universal function spaces that are guaranteed to be Koopman-invariant. Now, we are ready to introduce the Koopman kernel as the kernel obtained by combining "eigen-RKHS" as described in (13).
Proposition 1 (Koopman kernel). Consider trajectory data  $\mathbb{D}_N$  of Assumption 1 and a set of kernels  $\{K^{\lambda_j}\}_{j = 0}^D$  from Theorem 1. Then, the kernel  $K:\mathbb{X}_T\times \mathbb{X}_T\mapsto \mathcal{B}(\mathbb{Y}_T)$  given by
$$
K \left(\boldsymbol {x} _ {T}, \boldsymbol {x} _ {T} ^ {\prime}\right) = \sum_ {j = 1} ^ {\bar {D}} K ^ {\lambda_ {j}} \left(\boldsymbol {x} _ {T}, \boldsymbol {x} _ {T} ^ {\prime}\right) \tag {15}
$$
(i) defines an RKHS  $\mathcal{H} \coloneqq \mathcal{S}(\mathcal{H}^{\lambda_1} \oplus \dots \oplus \mathcal{H}^{\lambda_{\bar{D}}})$
(ii) is universal for any output (3b), provided a sufficient amount<sup>4</sup> of eigenspaces  $\bar{D}$ .
Above, we have derived the "Koopman-RKHS"  $\mathcal{H}$  for solving the problem (11) with a universal RKHS spanning Koopman eigenfunctions. Thus, the sample-data solution for an eigenfunction flow follows from the functional regression problem (11) and takes the form  $\phi_{\lambda_j}(\cdot) = \sum_{i=1}^{N} K^{\lambda_j}(\cdot, \boldsymbol{x}_T^{(i)}) \beta_i$ ,  $\beta_i \in \mathbb{Y}_T$  — providing a basis for the LTI predictor.
4 Sufficient amount is a rich enough set of eigenvalues  $\{\mathrm{e}^{\lambda_j[0,T]}\}_{j = 1}^D$  from  $\overline{\mathbb{B}_1(\mathbf{0})}$  in  $\mathbb{C}$  [57, Theorem 3.0.2].
3.2 Practicable LTI Predictor Regression