SlowGuess/NeurIPS_2023_OCR huggingface dataset:
jwhiteho@andrew.cmu.edu
zstevenwu@cmu.edu
In the kernelized bandit problem, a learner aims to sequentially compute the optimum of a function lying in a reproducing kernel Hilbert space given only noisy evaluations at sequentially chosen points. In particular, the learner aims to minimize regret, which is a measure of the suboptimality of the choices made. Arguably the most popular algorithm is the Gaussian Process Upper Confidence Bound (GP-UCB) algorithm, which involves acting based on a simple linear estimator of the unknown function. Despite its popularity, existing analyses of GP-UCB give a suboptimal regret rate, which fails to be sublinear for many commonly used kernels such as the Matérn kernel. This has led to a longstanding open question: are existing regret analyses for GP-UCB tight, or can bounds be improved by using more sophisticated analytical techniques? In this work, we resolve this open question and show that GP-UCB enjoys nearly optimal regret. In particular, our results yield sublinear regret rates for the Matérn kernel, improving over the state-of-the-art analyses and partially resolving a COLT open problem posed by Vakili et al. Our improvements rely on a key technical contribution — regularizing kernel ridge estimators in proportion to the smoothness of the underlying kernel  $k$ . Applying this key idea together with a largely overlooked concentration result in separable Hilbert spaces (for which we provide an independent, simplified derivation), we are able to provide a tighter analysis of the GP-UCB algorithm.
An essential problem in areas such as econometrics [12, 13], medicine [22, 23], optimal control [4, 3], and advertising [21] is to optimize an unknown function given bandit feedback, in which algorithms only get to observe the outcomes for the chosen actions. Due to the bandit feedback, there is a fundamental tradeoff between exploiting what has been observed about the local behavior of the function and exploring to learn more about the function's global behavior. There has been a long line of work on bandit learning that investigates this tradeoff across different settings, including multi-armed bandits [29, 19, 37], linear bandits [2, 30], and kernelized bandits [5, 26, 32].
In this work, we focus on the kernelized bandit framework, which can be viewed as an extension of the well-studied linear bandit setting to an infinite-dimensional reproducing kernel Hilbert space (or RKHS)  $(H,\langle \cdot ,\cdot \rangle_{H})$ . In this problem, there is some unknown function  $f^{*}:\mathcal{X}\to \mathbb{R}$  of bounded norm in  $H$ , where  $\mathcal{X}\subset \mathbb{R}^d$  is a bounded set. In each round  $t\in [T]$ , the learner uses previous observations to select an action  $X_{t}\in \mathcal{X}$ , and then observes feedback  $Y_{t}\coloneqq f^{*}(X_{t}) + \epsilon_{t}$ , where  $\epsilon_{t}$  is a zero-mean noise variable. The learner aims to minimize (with high probability) the regret at time
$T$ , which is defined as
$$
R _ {T} := \sum_ {t = 1} ^ {T} f ^ {*} (x ^ {*}) - f ^ {*} (X _ {t})
$$
where  $x^{*} \coloneqq \arg \max_{x \in \mathcal{X}} f^{*}(x)$ . The goal is to develop simple, efficient algorithms for the kernelized bandit problem that minimize regret  $R_{T}$ . We make the following standard assumption. We also make assumptions on the underlying kernel  $k$ , which we discuss in Section 2.
Assumption 1. We assume that (a) there is some constant  $D > 0$  known to the learner such that  $\| f^{*}\|_{H}\leq D$  and (b) for every  $t\geq 1$ ,  $\epsilon_t$  is  $\sigma$ -subGaussian conditioned on  $\sigma(Y_{1:t - 1},X_{1:t})$ .
Arguably the simplest algorithm for the kernelized bandit problem is GP-UCB (Gaussian process upper confidence bound) [31, 5]. GP-UCB works by maintaining a kernel ridge regression estimator of the unknown function  $f^{*}$  alongside a confidence ellipsoid, optimistically selecting in each round the action that provides the maximal payoff over all feasible functions. Not only is GP-UCB efficiently computable thanks to the kernel trick, but it also offers strong empirical guarantees [5]. The only seeming deficit of GP-UCB is its regret guarantee, as existing analyses only show that, with high probability,  $R_{T} = \widetilde{O} (\gamma_{T}\sqrt{T})$ , where  $\gamma_{T}$  is a kernel-dependent measure of complexity known as the maximum information gain [31, 6]. In contrast, more complicated, less computationally efficient algorithms such as SupKernelUCB [34, 25] have been shown to obtain regret bounds of  $\widetilde{O} (\sqrt{\gamma_T T})$ , improving over the analysis of GP-UCB by a multiplicative factor of  $\sqrt{\gamma_T}$ . This gap is stark as the bound  $\widetilde{O} (\gamma_{T}\sqrt{T})$  fails, in general, to be sub-linear for the practically relevant Matérn kernel, whereas  $\widetilde{O} (\sqrt{\gamma_T T})$  is sublinear for any kernel experiencing polynomial eigendecay [32].
This discrepancy has prompted the development of many variants of GP-UCB that, while less computationally efficient, offer better regret guarantees in some situations [17, 27, 28]. (See a detailed discussion of these algorithms along with other related work in Appendix A.) However, the following question remains an open problem in online learning [33]: are existing analyses of vanilla GP-UCB tight, or can an improved analysis show GP-UCB enjoys sublinear regret?
In this work, we show that GP-UCB obtains almost optimal, sublinear regret for any kernel experiencing polynomial eigendecay. This, in particular, implies that GP-UCB obtains sublinear regret for the commonly used Matérn family of kernels. We provide a brief roadmap of our paper below.
1. In Section 3, we provide background into self-normalized concentration in Hilbert spaces. In particular, in Theorem 1, we provide an independent, simplified derivation of a bound due to Abbasi-Yadmori [1], which concerns to self-normalized concentration of certain process in separable Hilbert spaces. This bound has been largely overlooked in the kernel bandit literature, so we draw attention to it in hopes it can be leveraged in solving further kernel-based learning problems. As opposed to the existing bound of Chowdhury and Gopalan [5], which involves employing a complicated "double mixture" argument, the bound we present follows directly from applying the well-studied finite-dimensional method of mixtures alongside a simple truncation argument [7-9, 2]. These bounds are clean and show simple dependence on the regularization parameter.
2. In Section 4, we use leverage the self-normalized concentration detailed in Theorem 1 to provide an improved regret analysis for GP-UCB. By carefully choosing regularization parameters based on the smoothness of the underlying kernel, we demonstrate that GP-UCB enjoys sublinear regret of  $\widetilde{O}\left(T^{\frac{3 + \beta}{2 + 2\beta}}\right)$  for any kernel experiencing  $(C,\beta)$ -polynomial eigendecay. As a special case of this result, we obtain regret bounds of  $\widetilde{O}\left(T^{\frac{\nu + 2d}{2\nu + 2d}}\right)$  for the commonly used Matérn kernel with smoothness  $\nu$  in dimension  $d$ . Our new analysis improves over existing state-of-the-art analysis for GP-UCB, which fails to guarantee sublinear regret in general for the Matérn kernel family [5], and thus partially resolves an open problem posed by [33] on the suboptimality of GP-UCB.
In sum, our results show that GP-UCB, the go-to algorithm for the kernelized bandit problem, is nearly optimal, coming close to the algorithm-independent lower bounds of Scarlett et al. [25]. Our
work thus can be seen as providing theoretical justification for the strong empirical performance of GP-UCB [31]. Perhaps the most important message of our work is the importance of careful regularization in online learning problems. While many existing bandit works treat the regularization parameter as a small, kernel-independent constant, we are able to obtain significant improvements by carefully selecting the regularization parameter. We hope our work will encourage others to pay close attention to the selection of regularization parameters in future works.
Notation. We briefly touch on basic definitions and notational conveniences that will be used throughout our work. If  $a_1, \ldots, a_t \in \mathbb{R}$ , we let  $a_{1:t} \coloneqq (a_1, \ldots, a_t)^\top$ . Let  $(H, \langle \cdot, \cdot \rangle_H)$  be a reproducing kernel Hilbert space associated with a kernel  $k: \mathcal{X} \times \mathcal{X} \to \mathbb{R}$ . We refer to the identity operator on  $H$  as  $\mathrm{id}_H$ . This is distinct from the identity mapping on  $\mathbb{R}^d$ , which we will refer to as  $I_d$ . For elements  $f, g \in H$ , we define their outer product as  $fg^\top \coloneqq f\langle g, \cdot \rangle_H$  and inner product as  $f^\top g \coloneqq \langle f, g \rangle_H$ . For any  $t \geq 1$  and sequence of points  $x_1, \ldots, x_t \in \mathcal{X}$  (which will typically be understood from context), let  $\Phi_t \coloneqq (k(\cdot, x_1), \ldots, k(\cdot, x_t))^\top$ . We can respectively define the Gram matrix  $K_t: \mathbb{R}^t \to \mathbb{R}^t$  and covariance operator  $V_t: H \to H$  as  $K_t \coloneqq (k(x_i, x_j))_{i,j \in [t]} = \Phi_t \Phi_t^\top$  and  $V_t \coloneqq \sum_{s=1}^{t} k(\cdot, x_s) k(\cdot, x_s)^\top = \Phi_t^\top \Phi_t$ . These two operators essentially encode the same information about the observed data points, the former being easier to work with when actually performing computations (by use of the well known kernel trick) and latter being easier to algebraically manipulate.
Suppose  $A:H\to H$  is a Hermitian operator of finite rank; enumerate its non-zero eigenvalues as  $\lambda_1(A),\ldots ,\lambda_k(A)$ . We can define the Fredholm determinant of  $I + A$  as  $\operatorname *{det}(I + A)\coloneqq \prod_{m = 1}^{k}(1+$ $\lambda_i(A))$  [20]. For any  $t\geq 1,\rho >0$  and  $x_{1},\ldots ,x_{t}\in \mathcal{X}$ , one can check via a straightforward computation that  $\operatorname *{det}(I_t + \rho^{-1}K_t) = \operatorname *{det}(\mathrm{id}_H + \rho^{-1}V_t)$ , where  $K_{t}$  and  $V_{t}$  are the Gram matrix and covariance operator defined above. We, again, will use these two quantities interchangeably in the sequel, but will typically prefer the latter in our proofs.
If  $(H, \langle \cdot, \cdot \rangle_H)$  is a (now general) separable Hilbert space and  $(\varphi_n)_{n \geq 1}$  is an orthonormal basis for  $H$ , for any  $N \geq 1$  we can define the orthogonal projection operator  $\pi_N: H \to \operatorname{span}\{\varphi_1, \ldots, \varphi_N\} \subset H$  by  $\pi_N f := \sum_{n=1}^{N} \langle f, \varphi_n \rangle_H \varphi_n$ . We can correspondingly define the projection onto the remaining basis functions to be the map  $\pi_N^\perp: H \to \operatorname{span}\{\varphi_1, \ldots, \varphi_N\}^\perp$  given by  $\pi_N^\perp f := f - \pi_N f$ . Lastly, if  $A: H \to H$  is a symmetric, bounded linear operator, we let  $\lambda_{\max}(A)$  denote the maximal eigenvalue of  $A$ , when such a value exists. In particular,  $\lambda_{\max}(A)$  will exist whenever  $A$  has a finite rank, as will typically be the case considered in this paper.
Basics on RKHSs. Let  $\mathcal{X} \subset \mathbb{R}^d$  be some domain. A kernel is a positive semidefinite map  $k: \mathcal{X} \times \mathcal{X} \to \mathbb{R}$  that is square-integrable, i.e.  $\int_{\mathcal{X}} \int_{\mathcal{X}} |k(x,y)|^2 dx dy < \infty$ . Any kernel  $k$  has an associated reproducing kernel Hilbert space or RKHS  $(H, \langle \cdot, \cdot \rangle_H)$  containing the closed span of all partial kernel evaluations  $k(\cdot, x), x \in \mathcal{X}$ . In particular, the inner product  $\langle \cdot, \cdot \rangle_H$  on  $H$  satisfies the reproducing relationship  $f(x) = \langle f, k(\cdot, x) \rangle_H$  for all  $x \in \mathcal{X}$ .
A kernel  $k$  can be associated with a corresponding Hilbert-Schmidt operator, which is the Hermitian operator  $T_{k}: L^{2}(\mathcal{X}) \to L^{2}(\mathcal{X})$  given by  $(T_{k}f)(x) \coloneqq \int_{\mathcal{X}} f(y) k(x,y) dy$  for any  $x \in \mathcal{X}$ . In short,  $T_{k}$  can be thought of as "smoothing out" or "mollifying" a function  $f$  according to the similarity metric induced by  $k$ .  $T_{k}$  plays a key role in kernelized learning through Mercer's Theorem, which gives an explicit representation for  $H$  in terms of the eigenvalues and eigenfunctions of  $T_{k}$ .
Fact 1 (Mercer's Theorem). Let  $(H, \langle \cdot, \cdot \rangle_{H})$  be the RKHS associated with kernel  $k$ , and let  $(\mu_n)_{n \geq 1}$  and  $(\phi_n)_{n \geq 1}$  be the sequence of non-increasing eigenvalues and corresponding eigenfunctions for  $T_k$ . Let  $(\varphi_n)_{n \geq 1}$  be the sequence of rescaled functions  $\varphi_n \coloneqq \sqrt{\mu}_n \phi_n$ . Then,
$$
H = \left\{\sum_ {n = 1} ^ {\infty} \theta_ {n} \varphi_ {n}: \sum_ {n = 1} ^ {\infty} \theta_ {n} ^ {2} <   \infty \right\},
$$
and  $(\varphi_{n})_{n\geq 1}$  forms an orthonormal basis for  $(H,\langle \cdot ,\cdot \rangle_H)$
We make the following assumption throughout the remainder of our work, which is standard and comes from Vakili et al. [32].
Assumption 2 (Assumption on kernel  $k$ ). The kernel  $k: \mathcal{X} \times \mathcal{X} \to \mathbb{R}$  satisfies (a)  $|k(x,y)| \leq L$  for all  $x,y \in \mathcal{X}$ , for some constant  $L > 0$  and (b)  $|\phi_n(x)| \leq B$  for all  $x \in \mathcal{X}$ , for some  $B > 0$ .
"Complexity" of RKHS's. By the eigendecay of a kernel  $k$ , we really mean the rate of decay of the sequence of eigenvalues  $(\mu_n)_{n\geq 1}$ . In the literature, there are two common paradigms for studying the eigendecay of  $k$ :  $(C_1,C_2,\beta)$ -exponential eigendecay, under which  $\forall n\geq 1,\mu_n\leq C_1\exp (-C_2n^\beta)$ , and  $(C,\beta)$ -polynomial eigendecay, under which  $\forall n\geq 1,\mu_n\leq Cn^{-\beta}$ . For kernels experiencing exponential eigendecay, of which the squared exponential is the most important example, GP-UCB is known to be optimal up to poly-logarithmic factors. However, for kernels experiencing polynomial eigendecay, of which the Matérn family is a common example, existing analyses of GP-UCB fail to yield sublinear regret. It is this latter case we focus on in this work.
Given the above representation in Fact 1, it is clear that the eigendecay of the kernel  $k$  governs the "complexity" or "size" of the RKHS  $H$ . We make this notion of complexity precise by discussing maximum information gain, a sequential, kernel-dependent quantity governing concentration and hardness of learning in RKHS's [6, 31, 32].
Let  $t \geq 1$  and  $\rho > 0$  be arbitrary. The maximum information gain at time  $t$  with regularization  $\rho$  is the scalar  $\gamma_t(\rho)$  given by
$$
\gamma_ {t} (\rho) := \sup  _ {x _ {1}, \dots , x _ {t} \in \mathcal {X}} \frac {1}{2} \log \det  \left(\operatorname {i d} _ {H} + \rho^ {- 1} V _ {t}\right) = \sup  _ {x _ {1}, \dots , x _ {t} \in \mathcal {X}} \frac {1}{2} \log \det  \left(I _ {t} + \rho^ {- 1} K _ {t}\right).
$$
Our presentation of maximum information gain differs from some previous works in that we encode the regularization parameter  $\rho$  into our notation. This inclusion is key for our results, as we obtain improvements by carefully selecting  $\rho$ . Vakili et al. [32] bound the rate of growth of  $\gamma_t(\rho)$  in terms of the rate of eigendecay of the kernel  $k$ . We leverage the following fact in our main results.
Fact 2 (Corollary 1 in Vakili et al. [32]). Suppose that kernel  $k$  satisfies Assumption 2 and experiences  $(C, \beta)$ -polynomial eigendecay. Then, for any  $t \geq 1$ , we have
$$
\gamma_ {t} (\rho) \leq \left(\left(\frac {C B ^ {2} t}{\rho}\right) ^ {1 / \beta} \log^ {- 1 / \beta} \left(1 + \frac {L t}{\rho}\right) + 1\right) \log \left(1 + \frac {L t}{\rho}\right).
$$
We last define the practically relevant Matérn kernel and discuss its eigendecay.
Definition/Fact 3. The Matérn kernel with bandwidth  $\sigma >0$  and smoothness  $\nu >1 / 2$  is given by
$$
k _ {\nu , \sigma} (x, y) := \frac {1}{\Gamma (\nu) 2 ^ {\nu - 1}} \left(\frac {\sqrt {2 \nu} \| x - y \| _ {2}}{\sigma}\right) ^ {\nu} B _ {\nu} \left(\frac {\sqrt {2 \nu} \| x - y \| _ {2}}{\sigma}\right),
$$
where  $\Gamma$  is the gamma function and  $B_{\nu}$  is the modified Bessel function of the second kind. It is known that there is some constant  $C > 0$  that may depend on  $\sigma$  but not on  $d$  or  $\nu$  such that  $k_{\nu,\sigma}$  experiences  $\left(C, \frac{2\nu + d}{d}\right)$ -eigendecay [24, 32].
Basics on martingale concentration: A filtration  $(\mathcal{F}_t)_{t\geq 0}$  is a sequence of  $\sigma$ -algebras satisfying  $\mathcal{F}_t\subset \mathcal{F}_{t + 1}$  for all  $t\geq 1$ . If  $(M_t)_{t\geq 0}$  is a  $H$ -valued process, we say  $(M_t)_{t\geq 0}$  is a martingale with respect to  $(\mathcal{F}_t)_{t\geq 0}$  if (a)  $(M_t)_{t\geq 0}$  is  $(\mathcal{F}_t)_{t\geq 0}$ -adapted, and (b)  $\mathbb{E}(M_t\mid \mathcal{F}_{t - 1}) = M_{t - 1}$  for all  $t\geq 1$ . An  $\mathbb{R}$ -valued process is called a supermartingale if the equality in (b) is replaced with “ $\leq$ ”, i.e. supermartingales tend to decrease. Martingales are useful in many statistical applications due to their strong concentration of measure properties [15, 36]. The following fact can be leveraged to provide time-uniform bounds on the growth of any non-negative supermartingale.
Fact 4 (Ville's Inequality). Let  $(M_t)_{t \geq 0}$  be a non-negative supermartingale with respect to some filtration. Suppose  $\mathbb{E}M_0 = 1$ . Then, for any  $\delta \in (0,1)$ , we have
$$
\mathbb {P} \left(\exists t \geq 0: M _ {t} \geq \frac {1}{\delta}\right) \leq \delta .
$$
See Howard et al. [14] for a self-contained proof of Ville's inequality, and many applications.
If  $\mathcal{F}$  is a  $\sigma$ -algebra, and  $\epsilon$  is an  $\mathbb{R}$ -valued random variable, we say  $\epsilon$  is  $\sigma$ -subGaussian conditioned on  $\mathcal{F}$  if, for any  $\lambda \in \mathbb{R}$ , we have  $\log \mathbb{E}\left(e^{\lambda \epsilon} \mid \mathcal{F}\right) \leq \frac{\lambda^2 \sigma^2}{2}$ ; in particular this condition implies
that  $\epsilon$  is mean zero. With this, we state the following result on self-normalized processes. To our understanding, the following result was first presented in some form as Example 4.2 of de la Peña et al. [8] (in the setting of continuous local martingales), and can be derived leveraging the argument of Theorem 1 in de la Peña et al. [9]. The exact form below was established (in the setting of discrete-time processes) in Theorem 1 of Abbasi-Yadkori et al. [2], which is commonly leveraged to construct confidence ellipsoids in the linear bandit setting.
Fact 5 (Example 4.2 from [8], Theorem 1 from [2]). Let  $(\mathcal{F}_t)_{t\geq 0}$  be a filtration, let  $(X_{t})_{t\geq 1}$  be an  $(\mathcal{F}_t)_{t\geq 0}$ -predictable sequence in  $\mathbb{R}^d$ , and let  $(\epsilon_t)_{t\geq 1}$  be a real-valued  $(\mathcal{F}_t)_{t\geq 1}$ -adapted sequence such that conditional on  $\mathcal{F}_{t - 1}$ ,  $\epsilon_t$  is mean zero and  $\sigma$ -subGaussian. Then, for any  $\rho >0$ , the process  $(M_t)_{t\geq 0}$  given by
$$
M _ {t} := \frac {1}{\sqrt {\operatorname * {d e t} (I _ {d} + \rho^ {- 1} V _ {t})}} \exp \left\{\frac {1}{2} \left\| (\rho I _ {d} + V _ {t}) ^ {- 1 / 2} S _ {t} / \sigma \right\| _ {2} ^ {2} \right\}
$$
is a non-negative supermartingale with respect to  $(\mathcal{F}_t)_{t\geq 0}$ , where  $S_{t}:= \sum_{s = 1}^{t}\epsilon_{s}X_{s}$  and  $V_{t}:= \sum_{s = 1}^{t}X_{s}X_{s}^{\top}$ . Consequently, by Fact 4, for any confidence  $\delta \in (0,1)$ , the following holds: with probability at least  $1 - \delta$ , simultaneously for all  $t\geq 1$ , we have
$$
\left\| \left(V _ {t} + \rho I _ {d}\right) ^ {- 1 / 2} S _ {t} \right\| _ {2} \leq \sigma \sqrt {2 \log \left(\frac {1}{\delta} \sqrt {\det \left(I _ {d} + \rho^ {- 1} V _ {t}\right)}\right)}.
$$
Note the simple dependence on the regularization parameter  $\rho > 0$  in the above bound. While the regularization parameter  $\rho$  doesn't prove important in regret analysis for linear bandits (where  $\rho$  is treated as constant), the choice for  $\rho$  will be critical in our setting. In the following section, we will discuss how Fact 5 can be extended to the setting of separable Hilbert spaces essentially verbatim (an observation first noticed by Abbasi-Yadkori [1]).
3 A Remark on Self-Normalized Concentration in Hillbert Spaces
We begin by discussing a key, self-normalized concentration inequality for martingales. We use this bound in the sequel to construct simpler, more flexible confidence ellipsoids than currently exist for GP-UCB. The bound we present (in Theorem 1 below) is, more or less, equivalent to Corollary 3.5 in the thesis of Abbasi-Yadkori [1]. Our result is mildly more general in the sense that it directly argues that a target mixture process is a nonnegative supermartingale. The result in Abbasi-Yadkori [1] is more general in the sense it allows the regularization (or shift) matrix to be non-diagonal. Either concentration result is sufficient for the regret bounds obtained in the sequel.
The aforementioned corollary in [1], quite surprisingly, has not been referenced in central works on the kernelized bandit problem, namely Chowdhury and Gopalan [5] and Vakili et al. [32, 33]. In fact, strictly weaker versions of the conclusion have been independently rediscovered in the context of kernel regression [10]. We emphasize that this result of Abbasi-Yadkori [1] (and the surrounding technical conclusions) are very general and may allow for further improvements in problems related to kernelized learning.
We now present Theorem 1, providing a brief sketch and a full proof in Appendix B. We believe our proof, which directly shows a target process is a nonnegative supermartingale, is of independent interest when compared to that of Abbasi-Yadkori [1] due to its simplicity. In particular, our proof follows from first principles, avoiding advanced topological notions of convergence (e.g. in the weak operator topology) and existence of certain Gaussian measures on separable Hilbert spaces, which were heavily utilized in the proof of Corollary 3.5 in Abbasi-Yadkori [1].
Theorem 1 (Self-normalized concentration in Hilbert spaces). Let  $(\mathcal{F}_t)_{t\geq 0}$  be a filtration,  $(f_t)_{t\geq 1}$  be an  $(\mathcal{F}_t)_{t\geq 0}$ -predictable sequence in a separable Hilbert space  $H$  such that  $\| f_t\| _H < \infty$  a.s. for all  $t\geq 0$ , and  $(\epsilon_{t})_{t\geq 1}$  be an  $(\mathcal{F}_t)_{t\geq 1}$ -adapted sequence in  $\mathbb{R}$  such that conditioned on  $\mathcal{F}_{t - 1}$ ,  $\epsilon_{t}$  is mean zero and  $\sigma$ -subGaussian. Defining  $S_{t}:= \sum_{s = 1}^{t}\epsilon_{s}f_{s}$  and  $V_{t}:= \sum_{s = 1}^{t}f_{s}f_{s}^{\top}$ , we have that for any  $\rho >0$ , the process  $(M_t)_{t\geq 0}$  defined by
$$
M _ {t} := \frac {1}{\sqrt {\operatorname* {d e t} (\mathrm {i d} _ {H} + \rho^ {- 1} V _ {t})}} \exp \left\{\frac {1}{2} \left\| (\rho \mathrm {i d} _ {H} + V _ {t}) ^ {- 1 / 2} S _ {t} / \sigma \right\| _ {H} ^ {2} \right\}
$$
<sup>1</sup>A space is separable if it has a countable, dense set. Separability is key, because it means we have a countable basis, whose first  $N$  elements we project onto.
is a nonnegative supermartingale with respect to  $(\mathcal{F}_t)_{t\geq 0}$ . Consequently, by Fact 4, for any  $\delta \in (0,1)$ , with probability at least  $1 - \delta$ , simultaneously for all  $t\geq 1$ , we have
$$
\left\| \left(V _ {t} + \rho I _ {d}\right) ^ {- 1 / 2} S _ {t} \right\| _ {H} \leq \sigma \sqrt {2 \log \left(\frac {1}{\delta} \sqrt {\det  (\mathrm {i d} _ {H} + \rho^ {- 1} V _ {t})}\right)}.
$$
We can summarize our independent proof in two simple steps. First, following from Fact 5, the bound in Theorem 1 holds when we project  $S_{t}$  and  $V_{t}$  onto a finite number  $N$  of coordinates, defining a "truncated" nonnegative supermartingale  $M_t^{(N)}$ . Secondly, we can make a limiting arugment, showing  $M_t^{(N)}$  is "essentially"  $M_{t}$  for large values of  $N$ .