SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Case 2. When  $a = 2$ , the infinite sum  $\sum_{t=1}^{T} t^{-1} \log t$  diverges. Its growth speed is on the order of  $\mathcal{O}(\log^2 T)$ . The total number of samples is  $n = d \sum_{t=1}^{T} t^2 = \mathcal{O}(dT^3)$ , and thus  $T = \mathcal{O}(d^{-\frac{1}{3}}n^{\frac{1}{3}})$ . Then the rate is
$$
\begin{array}{l} \min _ {1 \leq t \leq T} \| \nabla f (\mathbf {x} _ {t}) \| ^ {2} \lesssim \frac {1}{T} + \frac {\sigma d \log^ {2} T}{T} \\ \lesssim \sigma d T ^ {- 1} \log^ {2} T \\ \lesssim \sigma d ^ {\frac {4}{3}} n ^ {- \frac {1}{3}} \log^ {2} n. \\ \end{array}
$$
Case 3. When  $a > 2$ , the infinite sum  $\sum_{t=1}^{T} t^{-\frac{1}{2}a} \log t$  converges to a constant when  $T \to \infty$ . The total number of samples  $n = \mathcal{O}(dT^{a+1})$ . Thus, the rate is
$$
\begin{array}{l} \min  _ {1 \leq t \leq T} \| \nabla f (\mathbf {x} _ {t}) \| ^ {2} \lesssim T ^ {- 1} + \sigma d T ^ {- 1} \\ \lesssim \sigma d ^ {\frac {a + 2}{a + 1}} n ^ {- \frac {1}{a + 1}}. \\ \end{array}
$$
Note that  $a = 2$  achieves the fastest rate  $\mathcal{O}(n^{-\frac{1}{3}})$  in terms of samples  $n$ .
Now we discuss a batch size with logarithmic growth  $b_{t} = d\log^{2}t$ . We have
$$
\begin{array}{l} \min  _ {1 \leq t \leq T} \| \nabla f (\mathbf {x} _ {t}) \| ^ {2} \lesssim \frac {1}{T} + \frac {\sigma d}{T} \sum_ {t = 1} ^ {T} \mathcal {O} (1) \\ \lesssim \frac {1}{T} + \sigma d. \\ \end{array}
$$
This section presents additional experimental details and additional numerical simulations.
Details of Figure 2. In Figure 2a, we plot the error function starting from  $b = 20$  to make sure  $b \geq 2d$  so that Lemma 5 indeed applies. The decay rate  $\mathcal{O}(\sigma d^{\frac{3}{2}}b^{-\frac{1}{2}})$  has a (leading) hidden constant of  $\frac{15\sqrt{2}}{2}$  inside the big  $\mathcal{O}$  notation (see the proof of Lemma 5), and thus the bounds plotted in Figure 2 are multiplied by this constant. Otherwise, the expression  $\sigma d^{\frac{3}{2}}b^{-\frac{1}{2}}$  alone is not a valid upper bound.
ReLU. The ReLU function  $\max \{0, x\}$  is non-differentiable at  $x = 0$ . Nevertheless, thanks to convexity the subdifferential at  $x = 0$  is defined as  $[0,1]$ . We estimate the "derivative" at  $x = 0$  by minimizing the acquisition function. The estimate  $\mu_{\mathcal{D}}'(0)$  and queries are plotted in Figure 5. The estimated derivative  $\mu_{\mathcal{D}}'(0)$  is always in  $[0,1]$ . Thus, the posterior mean gradient  $\mu_{\mathcal{D}}'(0)$  produces a subgradient in this case.
(a)  $n = 2$ $\mu_{\mathcal{D}}^{\prime}(0) = 2.06\times 10^{-5}$
(b)  $n = 4$ $\mu_{\mathcal{D}}^{\prime}(0) = 0.49$
Figure 5: Estimating the "derivative" of ReLU at  $x = 0$  with noisy observations  $(\sigma = 0.01)$ .