SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Qian Yu
qy4628@princeton.edu
Yining Wang
yining.wang@utdallas.edu
Baihe Huang
University of California, Berkeley baihe_huang@berkeley.edu
New York University q1518@nyu.edu
Princeton University Jasondl@princeton.edu
In stochastic zeroth-order optimization, a problem of practical relevance is understanding how to fully exploit the local geometry of the underlying objective function. We consider a fundamental setting in which the objective function is quadratic, and provide the first tight characterization of the optimal Hessian-dependent sample complexity. Our contribution is twofold. First, from an information-theoretic point of view, we prove tight lower bounds on Hessian-dependent complexities by introducing a concept called energy allocation, which captures the interaction between the searching algorithm and the geometry of objective functions. A matching upper bound is obtained by solving the optimal energy spectrum. Then, algorithmically, we show the existence of a Hessian-independent algorithm that universally achieves the asymptotic optimal sample complexities for all Hessian instances. The optimal sample complexities achieved by our algorithm remain valid for heavy-tailed noise distributions, which are enabled by a truncation method.
Stochastic optimization in the zeroth order (gradient-free) setting has attracted significant attention in recent decades. It naturally arises in various applications such as autonomous driving [12], AI gaming [22], robotics [14], healthcare [23], and education [13]. This setting is particularly important when the gradient of the objective function cannot be directly evaluated or is expensive.
An important case of interest in the zeroth order setting is the bandit optimization of smooth and strongly-convex functions. Although there are ample results regarding the minimax rates of this problem [1, 8, 21, 6, 3, 24], little is known about how its complexity depends on the geometry of the objective function  $f$  near the global optimum  $x^{*}$ , as specified by the quadratic approximation  $\frac{1}{2}(x - x^{*})^{\top}\nabla^{2}f(x^{*})(x - x^{*})$ . As an initial step, we investigate the following natural questions:
- For zeorth-order bandit optimization problems of quadratic functions of the form  $\frac{1}{2} (x - x_0)^\top A(x - x_0)$ , what is the optimal instance-dependent upper bound with respect to  $A$ ?
- Is there an algorithm that universally achieves the optimal instance-dependent bounds for all quadratic functions, but without the knowledge of Hessian?
As our main contributions, we fully addressed the above questions as follows. First, we established the tight Hessian-dependent upper and lower bounds of the simple regret. Our bounds indicate asymptotic sample complexity bounds of  $\mathrm{Tr}^2 (A^{-\frac{1}{2}}) / (2\epsilon)$  to achieve  $\epsilon$  accuracy. This covers the
minimax lower bound of  $\Omega (d^2 /\epsilon)$  in [21]. Second, we prove the existence of a Hessian-independent algorithm with a matching upper bound of  $O\left(\mathrm{Tr}^2 (A^{-\frac{1}{2}}) / \epsilon\right)$ . Thus, we complete the theory of zeroth-order bandit optimization on quadratic functions.
Related works beyond linear or convex bandit optimization. Compared to their linear/convex counterparts, much less is known for finding optimal bandit under nonlinear/non-convex reward function. A natural next step beyond linear bandits is to look at quadratic reward functions, as studied in this paper and some prior work (see reference on bandit PCA [15, 4], its rank-1 special cases [18, 11], bilinear [10], low-rank linear [19] and some other settings [9, 5, 16]).
When getting beyond quadratic losses, prior work on non-convex settings mainly focuses on finding achieving  $\epsilon$ -stationary points instead of  $\epsilon$ -optimal reward [17], except for certain specific settings (see, e.g., [7, 25, 26, 20, 26, 20]).
2 Problem Formulation and Main Results
We first present a rigorous formulation for the stochastic zeroth-order optimization problem studied in this paper. Given a fixed dimension parameter  $d$ , let  $f: \mathcal{B} \to \mathbb{R}$  be an unknown objective function defined on the closed unit  $L_{2}$ -ball  $\mathcal{B}$  of the Euclidean space  $\mathbb{R}^d$ , which takes the following form for some positive-semidefinite (PSD) matrix  $A$ .
$$
f (\boldsymbol {x}) = \frac {1}{2} \left(\boldsymbol {x} - \boldsymbol {x} _ {0}\right) ^ {\intercal} A \left(\boldsymbol {x} - \boldsymbol {x} _ {0}\right). \tag {1}
$$
For each time  $t \in [T]$ , an optimization algorithm  $\mathcal{A}$  produces a query point  $\boldsymbol{x}_t \in \mathcal{B}$ , and receives
$$
y _ {t} = f \left(\boldsymbol {x} _ {t}\right) + w _ {t}. \tag {2}
$$
The algorithm can be adaptive, so that  $\mathcal{A}$  is described by a sequence of conditional distributions for choosing each  $\pmb{x}_t$  based on all prior observations  $\{\pmb{x}_{\tau},y_{\tau}\}_{\tau < t}$ . Then we only assume that the noises  $\{w_t\}_{t = 1}^{T - 1}$  are independent random variables with zero mean and unit variance, i.e.,  $\mathbb{E}[w_t|\pmb{x}_t] = 0$  and  $\mathbb{E}[w_t^2 |\pmb{x}_t]\leq 1$ .
For brevity, we use  $\mathcal{F}(A)$  to denote all functions  $f$  satisfying equation (1) for some  $\pmb{x}_0\in \mathcal{B}$ . We are interested in the following minimax simple regret for any PSD matrix  $A$ .
$$
\Re (T; A) := \inf  _ {\mathcal {A}} \sup  _ {f \in \mathcal {F} (A)} \mathbb {E} [ f (\boldsymbol {x} _ {T}) ]. \tag {3}
$$
The above quantity characterizes the simple regrets achievable by algorithms with perfect Hessian information. We also aim to identify the existence of algorithms that universally achieves the minimax regret for all  $A$ , without having access to that knowledge.
Our first result provides a tight characterization for the asymptotics of the minimax regret.
Theorem 2.1. For any PSD matrix  $A$ , we have
$$
\lim  _ {T \rightarrow \infty} \sup  _ {\mathbf {A}} \Re (T; A) \cdot T \leq \frac {1}{2} \left(\operatorname {T r} \left(A ^ {- \frac {1}{2}}\right)\right) ^ {2}, \tag {4}
$$
where  $A^{-\frac{1}{2}}$  denotes the pseudo inverse of  $A^{\frac{1}{2}}$ . Moreover, if the distributions of  $w_{t}$  are i.i.d. standard Gaussian, the above bound provides a tight characterization, i.e., there is a matching lower bound that the following equality is implied.
$$
\lim  _ {T \to \infty} \Re (T; A) \cdot T = \frac {1}{2} \left(\operatorname {T r} \left(A ^ {- \frac {1}{2}}\right)\right) ^ {2}.
$$
We prove Theorem 2.1 in Section 3. More generally, we also provide a full characterization of  $\Re (T;A)$  for the non-asymptotic regime, stated as follows and proved in Appendix C.
Theorem 2.2. For any PSD matrix  $A$  and  $T > 3\dim A$ , where  $\dim A$  denotes the rank of  $A$ , we have
$$
\Re (T; A) = \left\{ \begin{array}{l} \Theta \left(\frac {\left(\sum_ {k = 1} ^ {k ^ {*}} \lambda_ {k} ^ {- \frac {1}{2}}\right) ^ {2}}{T} + \lambda_ {k ^ {*} + 1}\right) \text {i f} k ^ {*} <   \dim A \\ \Theta \left(\frac {(\operatorname {T r} (A ^ {- \frac {1}{2}})) ^ {2}}{T}\right) \text {o t h e r w i s e} \end{array} \right.
$$
where  $\lambda_{j}$  is the jth smallest eigenvalue of  $A$  and  $k^{*}$  is the largest integer in  $\{0,1,\dots,\dim A\}$  satisfying  $T\geq \left(\sum_{k = 1}^{k^{*}}\lambda_{k}^{-\frac{1}{2}}\right)\left(\sum_{k = 1}^{k^{*}}\lambda_{k}^{-\frac{3}{2}}\right)$ .
Remark 2.3. While our formulation requires the algorithm to return estimates from the bounded domain, i.e.,  $\mathbf{x}_T \in \mathcal{B}$ , our lower bound applies to algorithms without this constraint as well. This can be proved by showing that the worst-case simple regret over all  $f \in \mathcal{F}(A)$  is always achievable by estimators satisfying  $\mathbf{x}_T \in \mathcal{B}$ . Their construction can be obtained using the projection step in Algorithm 1, and proof details can be found in Appendix A.
Finally, we provide a positive answer to the existence of universally optimal algorithms, stated in the following Theorem. We present the algorithm and prove its achievability guarantee in Section 4.
Theorem 2.4. There exists an algorithm  $\mathcal{A}$ , which does not depend on the Hessian parameter  $A$ , such that for  $A$  being any PSD matrix, the achieved minimax simple regret satisfies
$$
\limsup_{T\to \infty}\sup_{f\in \mathcal{F}(A)}\mathbb{E}\left[f(\boldsymbol{x}_{T})\right]\cdot T = O\left((\operatorname {Tr}(A^{-\frac{1}{2}}))^{2}\right).
$$
3 Proof of Theorem 2.1
To motivate the main construction ideas, we start by proving a weaker version of the lower bound in Theorem 2.1. We use the provided intuition to construct a matching upper bound. Then we complete the proof by strengthening the lower bound through a Bayes analysis and the uncertainty principle.
3.1 Proof of a Weakened Lower Bound
Assuming  $A$  being positive definite, we prove the following inequality when the additive noises  $w_{t}$ 's are i.i.d. standard Gaussian.
$$
\lim  _ {T \rightarrow \infty} \inf  \Re (T; A) \cdot T \geq \Omega \left(\left(\operatorname {T r} \left(A ^ {- \frac {1}{2}}\right)\right) ^ {2}\right). \tag {5}
$$
Throughout this section, we fix any basis such that  $A = \mathrm{diag}(\lambda_1,\dots,\lambda_d)$ . We construct a class of hard instances by letting
$$
\boldsymbol {x} _ {0} \in \mathcal {X} _ {\mathrm {H}} \triangleq \left\{(x _ {1}, x _ {2}, \dots , x _ {d}) \Bigg |   x _ {k} = \pm \sqrt {\frac {\lambda_ {k} ^ {- \frac {3}{2}} \left(\sum_ {j = 1} ^ {d} \lambda_ {j} ^ {- \frac {1}{2}}\right)}{2 T}}, \forall k \in [ d ] \right\}.
$$
We investigate a Bayes setting where the objective function is defined by equation (1) and  $\pmb{x}_0$  is uniformly random on the above set. For convenience, let  $\pmb{x}_t = (x_{1,t},\dots,x_{d,t})$  be the action of the algorithm at time  $t$ . We define the energy spectrum to be a vector  $\pmb{R} = (R_1,\dots,R_d)$  with each entry given by
$$
R _ {k} = \mathbb {E} \left[ \sum_ {t = 1} ^ {T} x _ {k, t} ^ {2} \right].
$$
Intuitively, our proof is to show that an allocation of at least  $R_{k} \geq \Omega (\lambda_{k}^{-2}x_{k}^{-2})$  energy is required to correctly estimate each  $x_{k}$  with  $\Theta (1)$  probability. Note that for any entry that is incorrectly estimated, a penalty of  $\Omega (\lambda_k x_k^2)$  is applied to the simple regret. This penalty is proportional to the required energy, which is due to the design of each  $x_{k}$ . Meanwhile, the total expected energy is upper bounded by  $T$ , which is no greater than the summation of the individual requirements. Therefore, an  $\Omega (1)$  fraction of the penalty is guaranteed, resulting in an overall effect of  $\Omega \left(\sum_{k}\lambda_{k}x_{k}^{2}\right) = \Omega \left((\mathrm{Tr}(A^{-\frac{1}{2}}))^{2} / T\right)$  on the simple regret.
Rigorously, consider any fixed algorithm, we define the estimation cost of each entry as follows.
$$
E _ {k} = \mathbb {E} \left[ \frac {1}{2} \lambda_ {k} \left(x _ {k, T} - x _ {k}\right) ^ {2} \right].
$$
For brevity, let  $s_k = \mathrm{sign}(x_k)$ . The above function is minimized by the minimum mean square error (MMSE) estimator (e.g., see [2, Sec. 4.6]), which depends on the following log-likelihood ratio (LLR) function.
$$
L _ {k} \triangleq \log \frac {\mathbb {P} [ s _ {k} = 1 | \{\boldsymbol {x} _ {\tau} , y _ {\tau} \} _ {\tau <   T} ]}{\mathbb {P} [ s _ {k} = - 1 | \{\boldsymbol {x} _ {\tau} , y _ {\tau} \} _ {\tau <   T} ]}.
$$
Specifically, the MMSE estimator is given by  $\widehat{x}_{k,T} \triangleq |x_k| \tanh \frac{L_k}{2}$ , and the resulting error conditioned on any fixed  $\{x_{\tau}, y_{\tau}\}_{\tau < T}$  has an expectation of  $x_k^2 \cdot \mathrm{sech}^2 \frac{L_k}{2}$ . Hence, we have the following lower bound for each cost entry.
$$
E _ {k} \geq \mathbb {E} \left[ \frac {1}{2} \lambda_ {k} \left(\widehat {x} _ {k, T} - x _ {k}\right) ^ {2} \right] = \frac {1}{2} \lambda_ {k} x _ {k} ^ {2} \cdot \mathbb {E} \left[ \operatorname {s e c h} ^ {2} \frac {L _ {k}}{2} \right]. \tag {6}
$$
By the Gaussian noise assumption, the conditional expectation of LLR can be written as follows.
$$
\begin{array}{l} \mathbb {E} \left[ L _ {k} | s _ {k} \right] = \mathbb {E} \left[ \sum_ {t} - \frac {\left(y _ {t} - \frac {1}{2} \lambda_ {k} (x _ {k , t} - x _ {k}) ^ {2}\right) ^ {2}}{2} + \frac {\left(y _ {t} - \frac {1}{2} \lambda_ {k} (x _ {k , t} + x _ {k}) ^ {2}\right) ^ {2}}{2} \Bigg | s _ {k} \right] \\ = 2 s _ {k} \left(\lambda_ {k} x _ {k}\right) ^ {2} \mathbb {E} \left[ \sum_ {t} x _ {k, t} ^ {2} \Big | s _ {k} \right]. \\ \end{array}
$$
The above quantity equals the KL divergence between the distributions of action-reward sequences generated by  $s_k = \pm 1$ . Clearly, it has the following connection to the energy spectrum.
$$
\mathbb {E} \left[ L _ {k} s _ {k} \right] = 2 \left(\lambda_ {k} x _ {k}\right) ^ {2} \mathbb {E} \left[ \sum_ {t} x _ {k, t} ^ {2} \right] = 2 \left(\lambda_ {k} x _ {k}\right) ^ {2} R _ {k}. \tag {7}
$$
Recall inequality (6), it also implies the following lower bound of the mean-squared error.
$$
\mathbb {E} \left[ L _ {k} s _ {k} \right] = \mathbb {E} \left[ L _ {k} \mathbb {E} \left[ s _ {k} \mid L _ {k} \right] \right] = \mathbb {E} \left[ L _ {k} \tanh  \frac {L _ {k}}{2} \right] \geq 2 - 2 \mathbb {E} \left[ \operatorname {s e c h} ^ {2} \frac {L _ {k}}{2} \right] \geq 2 - \frac {4 E _ {k}}{\lambda_ {k} x _ {k} ^ {2}}. \tag {8}
$$
Combine inequality (7) and (8), the overall simple regret can be bounded as follows.
$$
\mathbb {E} \left[ f \left(\boldsymbol {x} _ {T}\right) \right] = \sum_ {k} E _ {k} \geq \frac {1}{2} \sum_ {k} \left(1 - \lambda_ {k} ^ {2} x _ {k} ^ {2} R _ {k}\right) \lambda_ {k} x _ {k} ^ {2}. \tag {9}
$$
Note that the definition of  $\mathcal{X}_H$  implies that the value of each  $x_{k}^{2}$  is fixed. Hence,
$$
\mathbb {E} \left[ f (\boldsymbol {x} _ {T}) \right] \geq \frac {\left(\sum_ {j} \lambda_ {j} ^ {- \frac {1}{2}}\right) ^ {2}}{4 T} - \frac {\left(\sum_ {j} \lambda_ {j} ^ {- \frac {1}{2}}\right) ^ {2}}{8 T ^ {2}} \sum_ {k} R _ {k} \geq \frac {\left(\sum_ {j} \lambda_ {j} ^ {- \frac {1}{2}}\right) ^ {2}}{8 T} = \frac {\left(\operatorname {T r} (A ^ {- \frac {1}{2}})\right) ^ {2}}{8 T},
$$
where the last inequality uses the fact that the total energy is upper bounded by the number of samples.
Finally, when  $T$  is sufficiently large, we have  $||\pmb{x}_0||_2 \leq 1$  almost surely, which implies that our hard-instance functions belong to the set of objective functions  $\mathcal{F}(A)$ . Therefore,  $\mathbb{E}[f(\pmb{x}_T)]$  provides a lower bound of the asymptotic minimax regret, and we can conclude that
$$
\liminf_{T\to \infty}\Re (T;A)\cdot T\geq \frac{1}{8}\left(\operatorname {Tr}(A^{-\frac{1}{2}})\right)^{2}.
$$
Remark 3.1. The validity of the hard-instance functions requires  $T = \Omega\left(\left(\sum_{k}\lambda_{k}^{-\frac{1}{2}}\right)\left(\sum_{k}\lambda_{k}^{-\frac{3}{2}}\right)\right)$ , which is consistent with the transition threshold in Theorem 2.2 to the non-asymptotic regimes.
3.2 Proof of the Upper Bound
Now we provide a proof for equation (4), which is implied by the following result.
Proposition 3.2. For any PSD matrix  $A$ , let  $k^*$  be the rank,  $\lambda_1, \dots, \lambda_{k^*}$  be the non-zero eigenvalues, and  $e_1, e_2, \dots, e_{k^*}$  be the associated orthonormal eigenvectors. Then the expected simple regret achieved by algorithm 1 satisfies
$$
\lim  _ {T \rightarrow \infty} \sup  _ {f \in \mathcal {F} (A)} \mathbb {E} [ f (\boldsymbol {x} _ {T}) ] \cdot T \leq \frac {1}{2} \left(\operatorname {T r} \left(A ^ {- \frac {1}{2}}\right)\right) ^ {2}. \tag {10}
$$
Algorithm 1 Hessian Dependent Algorithm
procedure HESSIAN DEPENDENT ESTIMATION  $(\lambda_{1},\lambda_{2},\dots,\lambda_{k^{*}},e_{1},\dots,e_{k^{*}},T)$
for  $k\gets 1$  to  $k^{*}$  do
$$
\text {L e t} R _ {k} = \frac {\lambda_ {k} ^ {- \frac {1}{2}}}{\sum_ {j = 1} ^ {k ^ {*}} \lambda_ {j} ^ {- \frac {1}{2}}} \cdot (T - 2 d - 1), t _ {k} = \lceil R _ {k} / 2 \rceil .
$$
Let  $\alpha_{k} = -\frac{1}{2\lambda_{k}}$  (Sample  $(e_k, t_k) - \text{Sample}(-e_k, t_k)$ ). ▷ Obtain an unbounded estimator for
return  $x_{T} = \operatorname{argmin}_{\boldsymbol{x}\in \mathcal{B}}\sum_{k = 1}^{k^{*}}\lambda_{k}\left(\alpha_{k} - \boldsymbol {x}\cdot \boldsymbol{e}_{k}\right)^{2}$  Projection to B end procedure
procedure SAMPLE  $(x,t)$
return the average of  $t$  samples of  $f$  at  $\pmb{x}$
Proof. Consider an eigenbasis of  $A$  with  $e_1, \dots, e_{k^*}$  being the first  $k^*$  vectors. For each  $k \in [k^*]$ , the algorithm allocates  $R_k$  energy to estimate the  $k$ th entry of  $\mathbf{x}_0$ . The value of  $R_k$  is chosen such that the hard instances in the earlier subsection maximizes (modulo a constant factor) the lower bound in inequality (9) (i.e.,  $R_k$  satisfies  $x_k^2 \asymp 1 / (\lambda_k^2 R_k)$ ) while ensuring the total number of samples does not exceed  $T - 1$ . By the zero-mean and unit-variance assumptions of the noise distribution, we have
$$
\mathbb {E} [ (\alpha_ {k} - \pmb {x} _ {0} \cdot \pmb {e} _ {k}) ^ {2} ] = \frac {1}{2 \lambda_ {k} ^ {2} t _ {k}} \leq \frac {\lambda_ {k} ^ {- \frac {3}{2}} \cdot \sum_ {j = 1} ^ {k ^ {*}} \lambda_ {j} ^ {- \frac {1}{2}}}{T - 2 d - 1}.
$$
This essentially provides an unbounded estimator  $\widehat{\pmb{x}}\triangleq \sum_{k = 1}^{k^*}\alpha_k\pmb {e}_k$  that satisfies
$$
\begin{array}{l} \limsup_{T\to \infty}\sup_{f\in \mathcal{F}(A)}\mathbb{E}[f(\widehat{\boldsymbol{x}})]\cdot T = \limsup_{T\to \infty}\sup_{\boldsymbol {x}_{0}\in \mathcal{B}}\frac{1}{2}\sum_{k = 1}^{k^{*}}\lambda_{k}\mathbb{E}[(\alpha_{k} - \boldsymbol {x}_{0}\cdot \boldsymbol {e}_{k})^{2}]\cdot T \\ \leq \frac {\left(\sum_ {j = 1} ^ {k ^ {*}} \lambda_ {j} ^ {- \frac {1}{2}}\right) ^ {2}}{2} \cdot \lim _ {T \rightarrow \infty} \frac {T}{T - 2 d - 1} \\ = \frac {1}{2} \left(\operatorname {T r} \left(A ^ {- \frac {1}{2}}\right)\right) ^ {2}. \tag {11} \\ \end{array}
$$
Then, to obtain  $\pmb{x}_T$  within the bounded constraint set,  $\widehat{\pmb{x}}$  is projected to  $\mathcal{B}$  under a pseudometric defined by matrix  $A$ . Due to the convexity of set  $\mathcal{B}$ , we have  $f(\widehat{\pmb{x}}) \geq f(\pmb{x}_T)$  with probability 1 (see Appendix A for a proof). Therefore, inequality (10) is implied by inequality (11).
3.3 Proof of the Lower Bound with Tight Constant Factors
To complete the proof of Theorem 2.1, it remains to show that for standard Gaussian noise, we have
$$
\lim  _ {T \rightarrow \infty} \inf  _ {T \rightarrow \infty} \Re (T; A) \cdot T \geq \frac {1}{2} \left(\operatorname {T r} \left(A ^ {- \frac {1}{2}}\right)\right) ^ {2}. \tag {12}
$$
Notice that the object function and reward feedbacks depend only on the projections of  $\pmb{x}_0$  and query points onto the column (or row) space of  $A$ . Hence, it suffices to focus on algorithms with actions that are constrained on this subspace. This reduces the original problem to an equivalent instance that is defined on a (possibly) lower dimensional space and by a non-singular  $A$ . Therefore, we only need to prove inequality (12) for those reduced cases (i.e., when  $A$  is full-rank).
We lower bound the minimax regret by comparing them with a Bayes estimation error, where  $\pmb{x}_0$  has a prior distribution that violates the bounded-norm constraint. Formally, for any fixed algorithm  $\mathcal{A}$ , we can consider its expected simple regret  $\mathbb{E}[f(\pmb{x}_T)]$  over an extended class of objective functions where  $f$  is defined by equation (1) but  $\pmb{x}_0$  is chosen from the entire Euclidean space. Then for any distribution of  $\pmb{x}_0$ , the overall expectation is upper bounded by the following inequality.
$$
\mathbb {E} _ {\boldsymbol {x} _ {0}} \left[ \mathbb {E} [ f (\boldsymbol {x} _ {T}) ] \right] \leq \mathbb {P} [ \boldsymbol {x} _ {0} \in \mathcal {B} ] \cdot \sup  _ {f \in \mathcal {F} (A)} \mathbb {E} [ f (\boldsymbol {x} _ {T}) ] + \mathbb {E} \left[ \mathbb {1} (\boldsymbol {x} _ {0} \notin \mathcal {B}) \cdot \sup  _ {\boldsymbol {x} \in \mathcal {B}} f (\boldsymbol {x}) \right], \tag {13}
$$
where the first term on the RHS above is obtained by taking the supremum over all objective functions that satisfies  $\pmb{x}_0 \in \mathcal{B}$ , then the second term is obtained from the adversarial choice over all estimators for  $\pmb{x}_0 \notin \mathcal{B}$ . Compare the RHS of inequality (13) with equation (3), a lower bound of the minimax simple regret  $\Re(T; A)$  can be obtained by taking the infimum over algorithm  $\mathcal{A}$  on both sides. Hence, it remains to characterize the optimal Bayes estimation error on the LHS.
To provide a concrete analysis, let  $\pmb{x}_0$  be a Gaussian vector with zero mean and a covariance of  $T^{-\frac{2}{3}} \cdot I_d$ , where  $I_d$  denotes the identity matrix. Under this setting, conditioned on any realization of queries  $x_1, \ldots, x_{T-1}$  and feedbacks  $y_1, \ldots, y_{T-1}$ , the posterior distribution of  $\pmb{x}_0$  is proportional to
$$
\exp \left(- \frac {\left| \left| \boldsymbol {x} _ {0} \right| \right| _ {2} ^ {2} \cdot T ^ {\frac {2}{3}}}{2} - \sum_ {t = 1} ^ {T - 1} \frac {\left(y _ {t} - \frac {1}{2} (\boldsymbol {x} _ {t} - \boldsymbol {x} _ {0}) ^ {\intercal} A (\boldsymbol {x} _ {t} - \boldsymbol {x} _ {0})\right) ^ {2}}{2}\right).
$$
Clearly, the Bayesian error can be lower bounded by the expectation of the conditional covariance, which is further characterized by the following principle (see Appendix B for a proof).
Proposition 3.3 (Uncertainty Principle). Let  $\mathbf{Z}$  be a random variable on any measurable space and  $\theta$  be a real-valued random variable dependent of  $\mathbf{Z}$ . If the conditional distribution of  $\theta$  given  $\mathbf{Z}$  has a density function  $f_{\mathbf{Z}}(\theta)$ , and  $\ln f_{\mathbf{Z}}(\theta)$  has a second derivative that is integrable over  $f_{\mathbf{Z}}$ , then
$$
\mathbb {E} \left[ \mathrm {V a r} [ \theta | \boldsymbol {Z} ] \right] \geq \frac {1}{\mathbb {E} \left[ - \frac {\partial^ {2}}{\partial \theta^ {2}} \ln f _ {\boldsymbol {Z}} (\theta) \right]}.
$$
Hence, by taking the second derivative, the squared estimation error for the  $k$ th entry of  $\pmb{x}_0$  is lower bounded by the inverse of the following expectation.
$$
\begin{array}{l} \mathbb {E} \left[ T ^ {\frac {2}{3}} + \left(\sum_ {t = 1} ^ {T - 1} A (\boldsymbol {x} _ {t} - \boldsymbol {x} _ {0}) (\boldsymbol {x} _ {t} - \boldsymbol {x} _ {0}) ^ {\intercal} A ^ {\intercal} - A w _ {t}\right) _ {k k} \right] \\ = T ^ {\frac {2}{3}} + \mathbb {E} \left[ \left(\sum_ {t = 1} ^ {T - 1} A \left(\boldsymbol {x} _ {t} - \boldsymbol {x} _ {0}\right) \left(\boldsymbol {x} _ {t} - \boldsymbol {x} _ {0}\right) ^ {\intercal} A ^ {\intercal}\right) _ {k k} \right], \\ \end{array}
$$
where  $(\cdot)_{kk}$  denotes the  $k$ th diagonal entry of the given matrix. Recall that we chose a basis where  $A$  is diagonal. The overall Bayes error is bounded as follows
$$
\mathbb {E} _ {\boldsymbol {x} _ {0}} \left[ \mathbb {E} \left[ f (\boldsymbol {x} _ {T}) \right] \right] \geq \sum_ {k} \frac {\lambda_ {k}}{2} / \left(T ^ {\frac {2}{3}} + \lambda_ {k} ^ {2} \mathbb {E} \left[ \left(\sum_ {t = 1} ^ {T - 1} (\boldsymbol {x} _ {t} - \boldsymbol {x} _ {0}) (\boldsymbol {x} _ {t} - \boldsymbol {x} _ {0}) ^ {\intercal}\right) _ {k k} \right]\right),
$$
where  $\lambda_{k} = (A)_{kk}$  is the  $k$ th eigenvalue of  $A$ . By Cauchy's inequality, the RHS above can be further bounded by
$$
\frac {\frac {1}{2} \left(\sum_ {k} \lambda_ {k} ^ {- \frac {1}{2}}\right) ^ {2}}{\sum_ {k} \left(T ^ {\frac {2}{3}} \lambda_ {k} ^ {- 2} + \mathbb {E} \left[ \left(\sum_ {t = 1} ^ {T - 1} (\boldsymbol {x} _ {t} - \boldsymbol {x} _ {0}) (\boldsymbol {x} _ {t} - \boldsymbol {x} _ {0}) ^ {\intercal}\right) _ {k k} \right]\right)} = \frac {\frac {1}{2} \left(\operatorname {T r} \left(A ^ {- \frac {1}{2}}\right)\right) ^ {2}}{T ^ {\frac {2}{3}} \operatorname {T r} (A ^ {- 2}) + \sum_ {t = 1} ^ {T - 1} \mathbb {E} \left[ | | \boldsymbol {x} _ {t} - \boldsymbol {x} _ {0} | | _ {2} ^ {2} \right]}.
$$
Note that by triangle inequality