SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
\begin{array}{l} \hat {\kappa} _ {\tau , t} (x) - \kappa_ {\tau , t} (x) \leq \frac {\tau - \left\langle \bar {w} _ {x} ^ {t} , \mathbf {u c b} _ {x} ^ {t} \right\rangle}{\| \bar {w} _ {x} ^ {t} - w _ {t} \| _ {M}} - \frac {\tau - \left\langle \bar {\bar {w}} _ {x} ^ {t} , f _ {x} \right\rangle}{\| \bar {\bar {w}} _ {x} ^ {t} - w _ {t} \| _ {M}} \\ \leq \frac {\tau - \langle \bar {w} _ {x} ^ {t} , \mathbf {u c b} _ {x} ^ {t} \rangle}{\| \bar {w} _ {x} ^ {t} - w _ {t} \| _ {M}} - \frac {\tau - \langle \bar {w} _ {x} ^ {t} , f _ {x} \rangle}{\| \bar {w} _ {x} ^ {t} - w _ {t} \| _ {M}} \\ = \frac {\left\langle \bar {w} _ {x} ^ {t} , f _ {x} - \mathbf {u c b} _ {x} ^ {t} \right\rangle}{\left\| \bar {w} _ {x} ^ {t} - w _ {t} \right\| _ {M}} \\ \leq 0. \\ \end{array}
$$
C.2 An Auxiliary Concentration Lemma
Lemma 6. ([10, Lemma 7]) Let  $S_{t} \geq 0$  be a non-negative stochastic process with filtration  $\mathcal{F}_t$ , and define  $m_t = \mathbb{E}[S_t|\mathcal{F}_{t - 1}]$ . Further assume that  $S_{t} \leq B$  for  $B \geq 1$ . Then for any  $T \geq 1$ , with probability at least  $1 - \delta$  it holds that
$$
\begin{array}{l} \sum_ {t = 1} ^ {T} m _ {t} \leq 2 \sum_ {t = 1} ^ {T} S _ {t} + 4 B \log \frac {1}{\delta} + 8 B \log (4 B) + 1 \\ \leq 2 \sum_ {t = 1} ^ {T} S _ {t} + 8 B \log \frac {6 B}{\delta}. \\ \end{array}
$$
The robust satisficing regret is upper bounded by the sum of instantaneous regrets  $r_t^{rs} \coloneqq \tau - \kappa_{\tau, t} \| w_t^* - w_t \|_M - \langle w_t^*, f_{x_t} \rangle$  as follows:
$$
\begin{array}{l} R _ {T} ^ {r s} = \sum_ {t = 1} ^ {T} \left(\tau - \kappa_ {\tau , t} \Delta \left(P _ {t} ^ {*}, P _ {t}\right) - \mathbb {E} _ {c \sim P _ {t} ^ {*}} [ f (x _ {t}, c) ]\right) ^ {+} \\ = \sum_ {t = 1} ^ {T} \left(\tau - \kappa_ {\tau , t} \| w _ {t} ^ {*} - w _ {t} \| _ {M} - \langle w _ {t} ^ {*}, f _ {x _ {t}} \rangle\right) ^ {+} \\ = \sum_ {t = 1} ^ {T} r _ {t} ^ {r s} \mathbb {I} \left(r _ {t} ^ {r s} \geq 0\right). \tag {16} \\ \end{array}
$$
Assume that the confidence intervals in Lemma 1 hold (an event that happens with probability at least  $1 - \delta /2$ ). For the instantaneous regret we have
$$
\begin{array}{l} r _ {t} ^ {r s} = \tau - \kappa_ {\tau , t} \| w _ {t} ^ {*} - w _ {t} \| _ {M} - \left\langle w _ {t} ^ {*}, \mathbf {u c b} _ {x _ {t}} ^ {t} \right\rangle + \left\langle w _ {t} ^ {*}, \mathbf {u c b} _ {x _ {t}} ^ {t} - f _ {x _ {t}} \right\rangle \\ \leq \tau - \kappa_ {\tau , t} \| w _ {t} ^ {*} - w _ {t} \| _ {M} - \left\langle w _ {t} ^ {*}, \mathbf {u c b} _ {x _ {t}} ^ {t} \right\rangle + 2 \beta_ {t} \left\langle w _ {t} ^ {*}, \sigma_ {t} \left(x _ {t}, \cdot\right) \right\rangle (17) \\ \leq \tau - \kappa_ {\tau , t} \| w _ {t} ^ {*} - w _ {t} \| _ {M} - (\tau - \hat {\kappa} _ {\tau , t} \| w _ {t} ^ {*} - w _ {t} \| _ {M}) + 2 \beta_ {t} \left\langle w _ {t} ^ {*}, \sigma_ {t} \left(x _ {t}, \cdot\right) \right\rangle (18) \\ = \| w _ {t} ^ {*} - w _ {t} \| _ {M} \left(\hat {\kappa} _ {\tau , t} - \kappa_ {\tau , t}\right) + 2 \beta_ {t} \left\langle w _ {t} ^ {*}, \sigma_ {t} \left(x _ {t}, \cdot\right) \right\rangle \\ \leq 2 \beta_ {t} \left\langle w _ {t} ^ {*}, \sigma_ {t} \left(x _ {t}, \cdot\right) \right\rangle . (19) \\ \end{array}
$$
In the derivation above, (17) follows from the confidence bounds given in Lemma 1; (18) follows from  $\langle w_t^*,\mathsf{ucb}_{x_t}^t\rangle \geq \tau -\hat{\kappa}_{\tau ,t}(x_t)||w_t^* -w_t||_M$  and  $\hat{\kappa}_{\tau ,t} = \hat{\kappa}_{\tau ,t}(x_t)$ ; (19) is due to Lemma 2.
From this point on, we bound the robust satisficing regret by following standard steps for bounding regret of GP bandits (see e.g., [10]). First, by plugging the upper bound on  $r_t^{rs}$  obtained in (19) to
(16), and using monotonicity of  $\beta_{t}$  , we obtain
$$
\begin{array}{l} R _ {T} ^ {r s} \leq 2 \beta_ {T} \sum_ {t = 1} ^ {T} \left\langle w _ {t} ^ {*}, \sigma_ {t} \left(x _ {t}, \cdot\right) \right\rangle (20) \\ \leq 2 \beta_ {T} \sqrt {T \sum_ {t = 1} ^ {T} \left\langle w _ {t} ^ {*}, \sigma_ {t} \left(x _ {t} , \cdot\right) \right\rangle^ {2}} (21) \\ \leq 2 \beta_ {T} \sqrt {T \sum_ {t = 1} ^ {T} \left\langle w _ {t} ^ {*}, \sigma_ {t} \left(x _ {t} , \cdot\right) ^ {2} \right\rangle}, (22) \\ \end{array}
$$
where (21) uses the Cauchy-Schwarz inequality and (22) uses the Jensen inequality.
To complete the proof we use Lemma 6 to relate the expectation of the posterior variance in (22) to the posterior variance of the observations. Namely, we have with probability at least  $1 - \delta /2$
$$
\sum_ {t = 1} ^ {T} \left\langle w _ {t} ^ {*}, \sigma_ {t} \left(x _ {t}, \cdot\right) ^ {2} \right\rangle \leq 2 \sum_ {t = 1} ^ {T} \sigma_ {t} \left(x _ {t}, c _ {t}\right) ^ {2} + 8 \log \frac {1 2}{\delta}. \tag {23}
$$
Next, to relate the sum of variances that appears in (23) to the maximum information gain. We use  $x \leq 2\log(1 + x)$  for all  $x \in [0,1]$  to obtain
$$
\begin{array}{l} \sum_ {t = 1} ^ {T} \sigma_ {t} \left(x _ {t}, c _ {t}\right) ^ {2} \leq \sum_ {t = 1} ^ {T} 2 \log \left(1 + \sigma_ {t} \left(x _ {t}, c _ {t}\right) ^ {2}\right) \\ \leq 4 \gamma_ {T}, \tag {24} \\ \end{array}
$$
where (24) follows from [35, Lemma 3].
Finally, to bound the robust satisficing regret, we plug the upper bound in (24) to the r.h.s. of (23), and use it to upper bound (22) as follows
Let  $A$  and  $B$  be the events that Lemma 1 and Lemma 6 hold. Setting the confidence of each event to  $1 - \delta /2$ , we can bound from below the probability that both events hold
$$
P (A \cap B) = 1 - P (\bar {A} \cup \bar {B}) \geq 1 - P (\bar {A}) - P (\bar {B}) = 1 - \delta ,
$$
completing the proof of Theorem 3.
C.4 Proof of Theorem 4
Define the instantaneous regret as  $r_t^l \coloneqq \tau - \langle w_t^*, f_{x_t} \rangle$ . We have
$$
R _ {T} ^ {l} := \sum_ {t = 1} ^ {T} \left(\tau - \mathbb {E} _ {c \sim P _ {t} ^ {*}} [ f (x _ {t}, c) ]\right) ^ {+} = \sum_ {t = 1} ^ {T} r _ {t} ^ {l} \mathbb {I} \left(r _ {t} ^ {l} \geq 0\right). \tag {25}
$$
Assume that the confidence intervals in Lemma 1 hold (an event that happens with probability at least  $1 - \delta /2$ ). Then, for the instantaneous regret we have
$$
\begin{array}{l} r _ {t} ^ {l} = \tau - \left\langle w _ {t} ^ {*}, \mathbf {u c b} _ {x _ {t}} ^ {t} \right\rangle + \left\langle w _ {t} ^ {*}, \mathbf {u c b} _ {x _ {t}} ^ {t} - f _ {x _ {t}} \right\rangle \\ \leq \tau - \left\langle w _ {t} ^ {*}, \operatorname {u c b} _ {x _ {t}} ^ {t} \right\rangle + 2 \beta_ {t} \left\langle w _ {t} ^ {*}, \sigma_ {t} \left(x _ {t}, \cdot\right) \right\rangle . \tag {26} \\ \end{array}
$$
When,  $w_{t}^{*} = w_{t}$ , by Assumption 1 and the selection rule of RoBOS (i.e.,  $\langle w_t,\mathrm{ucb}_{x_t}^t\rangle \geq \tau$ ), we obtain
$$
r _ {t} ^ {l} \leq \tau - \left\langle w _ {t}, \mathbf {u c b} _ {x _ {t}} ^ {t} \right\rangle + 2 \beta_ {t} \left\langle w _ {t} ^ {*}, \sigma_ {t} (x _ {t}, \cdot) \right\rangle \leq 2 \beta_ {t} \left\langle w _ {t} ^ {*}, \sigma_ {t} (x _ {t}, \cdot) \right\rangle .
$$
When  $w_{t}^{*} \neq w_{t}$ , continuing from (26), we obtain
$$
\begin{array}{l} r _ {t} ^ {l} \leq \| w _ {t} ^ {*} - w _ {t} \| _ {M} \frac {\tau - \langle w _ {t} ^ {*} , \mathbf {u c b} _ {x _ {t}} ^ {t} \rangle}{\| w _ {t} ^ {*} - w _ {t} \| _ {M}} + 2 \beta_ {t} \langle w _ {t} ^ {*}, \sigma_ {t} (x _ {t}, \cdot) \rangle \\ \leq \left\| w _ {t} ^ {*} - w _ {t} \right\| _ {M} \frac {\tau - \left\langle \bar {w} _ {x _ {t}} ^ {t} , \mathbf {u c b} _ {x _ {t}} ^ {t} \right\rangle}{\left\| \bar {w} _ {x _ {t}} ^ {t} - w _ {t} \right\| _ {M}} + 2 \beta_ {t} \left\langle w _ {t} ^ {*}, \sigma_ {t} \left(x _ {t}, \cdot\right) \right\rangle (27) \\ \leq \left\| w _ {t} ^ {*} - w _ {t} \right\| _ {M} \frac {\tau - \langle \bar {w} _ {\hat {x} _ {t}} ^ {t} , \mathbf {u c b} _ {\hat {x} _ {t}} ^ {t} \rangle}{\left\| \bar {w} _ {\hat {x} _ {t}} ^ {t} - w _ {t} \right\| _ {M}} + 2 \beta_ {t} \left\langle w _ {t} ^ {*}, \sigma_ {t} (x _ {t}, \cdot) \right\rangle (28) \\ \leq \left\| w _ {t} ^ {*} - w _ {t} \right\| _ {M} \frac {\left\langle w _ {t} , f _ {\hat {x} _ {t}} \right\rangle - \left\langle \bar {w} _ {\hat {x} _ {t}} ^ {t} , \mathbf {u c b} _ {\hat {x} _ {t}} ^ {t} \right\rangle}{\left\| \bar {w} _ {\hat {x} _ {t}} ^ {t} - w _ {t} \right\| _ {M}} + 2 \beta_ {t} \left\langle w _ {t} ^ {*}, \sigma_ {t} (x _ {t}, \cdot) \right\rangle (29) \\ \leq \left\| w _ {t} ^ {*} - w _ {t} \right\| _ {M} \frac {\left\langle w _ {t} - \bar {w} _ {\hat {x} _ {t}} ^ {t} , f _ {\hat {x} _ {t}} \right\rangle}{\left\| \bar {w} _ {\hat {x} _ {t}} ^ {t} - w _ {t} \right\| _ {M}} + 2 \beta_ {t} \left\langle w _ {t} ^ {*}, \sigma_ {t} (x _ {t}, \cdot) \right\rangle (30) \\ \leq \left\| w _ {t} ^ {*} - w _ {t} \right\| _ {M} \frac {\left\| \bar {w} _ {\hat {x} _ {t}} ^ {t} - w _ {t} \right\| _ {M} \left\| f _ {\hat {x} _ {t}} \right\| _ {M ^ {- 1}}}{\left\| \bar {w} _ {\hat {x} _ {t}} ^ {t} - w _ {t} \right\| _ {M}} + 2 \beta_ {t} \langle w _ {t} ^ {*}, \sigma_ {t} (x _ {t}, \cdot) \rangle (31) \\ = \| w _ {t} ^ {*} - w _ {t} \| _ {M} \| f _ {\hat {x} _ {t}} \| _ {M ^ {- 1}} + 2 \beta_ {t} \langle w _ {t} ^ {*}, \sigma_ {t} (x _ {t}, \cdot) \rangle \\ \leq 2 \beta_ {t} \left\langle w _ {t} ^ {*}, \sigma_ {t} \left(x _ {t}, \cdot\right) \right\rangle + \epsilon_ {t} B ^ {\prime}. (32) \\ \end{array}
$$
Since the final bound is non-negative, it can be used to bound all terms  $r_t^l \mathbb{I}(r_t^l \geq 0)$  in  $R_T^l$ . In the derivation above, (26) follows from Lemma 1; (27) uses the fact that  $\bar{w}_{x_t}^t$  is the distribution that achieves  $\hat{\kappa}_{\tau,t}(x_t)$ ; (28) uses the fact that  $x_t$  minimizes  $\hat{\kappa}_{\tau,t}(x)$ , (29) uses Assumption 1, (30) again uses Lemma 1, (31) follows from the Cauchy-Schwarz inequality; and (32) follows from the facts  $\| w_t^* - w_t \|_M = \epsilon_t$  and  $B' = \max_x \| f_x \|_{M^{-1}}$ . From this point on, the regret bound follows the same arguments as in the proof of Theorem 3. In particular, using (32), we write
$$
R _ {T} ^ {l} \leq 2 \beta_ {T} \sum_ {t = 1} ^ {T} \left\langle w _ {t} ^ {*}, \sigma_ {t} \left(x _ {t}, \cdot\right) \right\rangle + B ^ {\prime} \sum_ {t = 1} ^ {T} \epsilon_ {t}. \tag {33}
$$
We complete the proof by continuing from (20), by using the same steps as in the proof of Theorem 3 starting from (20), which leads to the regret bound in the statement of Theorem 4.
The first term of Lemma 5 directly follows from Theorem 4. For the second term, we use the result of [38, Theorem 3.4], which is restated below.
[38, Theorem 3.4] Assume  $k(c_{i},c_{j})\leq 1$  for all  $c_{i},c_{j}\in \mathcal{C}$ . Then, with probability at least  $1 - \delta$
$$
d \left(P ^ {*}, \hat {P} _ {t}\right) \leq \frac {1}{\sqrt {t - 1}} \left(2 + \sqrt {2 \log (1 / \delta)}\right).
$$
To use the lemma above, let  $\delta_t = \frac{2\delta}{\pi^2L^2}$ . Then,
$$
\mathbb {P} \left(d \left(P ^ {*}, \hat {P} _ {t}\right) > \frac {1}{\sqrt {t - 1}} \left(2 + \sqrt {2 \log \left(\frac {\pi^ {2} t ^ {2}}{2 \delta}\right)}\right)\right) \leq \frac {2 \delta}{\pi^ {2} t ^ {2}}.
$$
Using a union bound, we obtain
$$
\mathbb {P} \left(\exists t \in [ T ]: d (P ^ {*}, \hat {P} _ {t}) > \frac {1}{\sqrt {t - 1}} \left(2 + \sqrt {2 \log \left(\frac {\pi^ {2} t ^ {2}}{2 \delta}\right)}\right)\right) \leq \sum_ {t = 1} ^ {T} \frac {2 \delta}{\pi^ {2} t ^ {2}} = \frac {2 \delta}{\pi^ {2}} \frac {\pi^ {2}}{6} = \frac {\delta}{3}.
$$
The inequality above implies that
$$
\mathbb {P} \left(\forall t \in [ T ]: d (P ^ {*}, \hat {P} _ {t}) > \frac {1}{\sqrt {t - 1}} \left(2 + \sqrt {2 \log \left(\frac {\pi^ {2} t ^ {2}}{2 \delta}\right)}\right)\right) \leq 1 - \frac {\delta}{3}.
$$
Since  $\epsilon_t \leq \frac{1}{\sqrt{t - 1}}\left(2 + \sqrt{2\log\left(\frac{\pi^2t^2}{2\delta}\right)}\right)$ , we have with probability at least  $1 - \delta$
$$
\begin{array}{l} R _ {T} ^ {l} \leq 4 \beta_ {T} \sqrt {T \left(2 \gamma_ {T} + 2 \log \left(\frac {1 2}{\delta}\right)\right)} + B ^ {\prime} \sum_ {t = 1} ^ {T} \epsilon_ {t}. \\ \leq 4 \beta_ {T} \sqrt {T \left(2 \gamma_ {T} + 2 \log \left(\frac {1 2}{\delta}\right)\right)} + B ^ {\prime} \epsilon_ {1} + B ^ {\prime} \sum_ {t = 2} ^ {T} \frac {1}{\sqrt {t - 1}} \left(2 + \sqrt {2 \log \left(\frac {\pi^ {2} t ^ {2}}{2 \delta}\right)}\right) \\ \leq 4 \beta_ {T} \sqrt {T \left(2 \gamma_ {T} + 2 \log \left(\frac {1 2}{\delta}\right)\right)} + B ^ {\prime} \epsilon_ {1} + B ^ {\prime} \sum_ {t = 2} ^ {T} \frac {1}{\sqrt {t - 1}} \left(2 + \sqrt {2 \log \left(\frac {\pi^ {2} T ^ {2}}{2 \delta}\right)}\right) \\ = 4 \beta_ {T} \sqrt {T \left(2 \gamma_ {T} + 2 \log \left(\frac {1 2}{\delta}\right)\right)} + B ^ {\prime} \epsilon_ {1} + B ^ {\prime} \sum_ {t = 1} ^ {T - 1} \frac {1}{\sqrt {t}} \left(2 + \sqrt {2 \log \left(\frac {\pi^ {2} T ^ {2}}{2 \delta}\right)}\right) \\ \leq 4 \beta_ {T} \sqrt {T \left(2 \gamma_ {T} + 2 \log \left(\frac {1 2}{\delta}\right)\right)} + B ^ {\prime} \epsilon_ {1} + B ^ {\prime} 2 \sqrt {T} \left(2 + \sqrt {2 \log \left(\frac {\pi^ {2} T ^ {2}}{2 \delta}\right)}\right). \\ \end{array}
$$
$$
\epsilon_ {1} = \| w ^ {*} - w _ {1} \| _ {M} = \| w ^ {*} - \frac {1}{n} \| _ {M} \leq \sup _ {\{w: \| w \| _ {1} = 1 \}} \| w - \frac {1}{n} \| _ {M} = O (1).
$$