SlowGuess/NeurIPS_2023_OCR huggingface dataset:
- Let the input metric space  $(\mathcal{X},d)$  be constructed in the following way. Consider the feature space  $\mathcal{X} = \{\mathbf{e}_1,\ldots ,\mathbf{e}_n\} \cup X_0$ , where  $X_0 = \{\frac{\sigma(0,1,\dots,n - 1)}{z} |\sigma \in S_n\}$  with  $z = \sqrt{1^2 + \dots + (n - 1)^2}$  for some small  $\alpha = 0.1$ . Here  $\mathcal{S}_n$  is the set of all permutations over  $n$  elements. So  $X_0$  is the set of points whose coordinates are a permutation of  $\{0,1 / z,\dots ,(n - 1) / z\}$  and all points in  $X_0$  have the  $\ell_2$  norm equal to  $\alpha$ . We define the metric  $d$  by restricting  $\ell_2$  distance to  $\mathcal{X}$ , i.e.,  $d(x_{1},x_{2}) = \| x_{1} - x_{2}\|_{2}$  for all  $x_{1},x_{2}\in \mathcal{X}$ . Then we have that for any  $x\in X_0$  and  $i\in [n]$ , the distance between  $x$  and  $\mathbf{e}_i$  is
$$
d (x, \mathbf {e} _ {i}) = \| x - \mathbf {e} _ {i} \| _ {2} = \sqrt {(x _ {i} - 1) ^ {2} + \sum_ {j \neq i} x _ {j} ^ {2}} = \sqrt {1 + \sum_ {j = 1} ^ {n} x _ {j} ^ {2} - 2 x _ {i}} = \sqrt {1 + \alpha^ {2} - 2 x _ {i}},
$$
which is greater than  $\sqrt{1 + \alpha^2 - 2\alpha} > 0.8 > 2\alpha$ . For any two points  $x, x' \in X_0$ ,  $d(x, x') \leq 2\alpha$  by triangle inequality.
- Let the hypothesis class be a set of singletons over  $\{\mathbf{e}_i | i \in [n]\}$ , i.e.,  $\mathcal{H} = \{2\mathbb{1}_{\{\mathbf{e}_i\}} - 1 | i \in [n]\}$ .
- We now define a collection of distributions  $\{\mathcal{D}_i | i \in [n]\}$  in which  $\mathcal{D}_i$  is realized by  $2\mathbb{1}_{\{\mathbf{e}_i\}} - 1$ . For any  $i \in [n]$ , we define  $\mathcal{D}_i$  in the following way. Let the marginal distribution  $\mathcal{D}_{\mathcal{X}}$  over  $\mathcal{X}$  be uniform over  $X_0$ . For any  $x$ , the label  $y$  is + with probability  $1 - 6\varepsilon$  and - with probability  $6\varepsilon$ , i.e.,  $\mathcal{D}(y|x) = \mathrm{Rad}(1 - 6\varepsilon)$ . Note that the marginal distribution  $\mathcal{D}_{\mathcal{X} \times \mathcal{Y}} = \mathrm{Unif}(X_0) \times \mathrm{Rad}(1 - 6\varepsilon)$  is identical for any distribution in  $\{\mathcal{D}_i | i \in [n]\}$  and does not depend on  $i$ .
If the label is positive  $y = +$ , then let the radius  $r = 2$ . If the label is negative  $y = -$ , then let  $r = \sqrt{1 + \alpha^2 - 2(x_i + \frac{1}{z})}$ , which guarantees that  $x$  can be manipulated to  $\mathbf{e}_j$  iff  $d(x, \mathbf{e}_j) < d(x, \mathbf{e}_i)$  for all  $j \in [n]$ . Since  $x_i \leq \alpha$  and  $\frac{1}{z} \leq \alpha$ , we have  $\sqrt{1 + \alpha^2 - 2(x_i + \frac{1}{z})} \geq \sqrt{1 - 4\alpha} > 2\alpha$ . Therefore, for both positive and negative examples, we have radius  $r$  strictly greater than  $2\alpha$  in both cases.
Randomization and improperness of the output  $f_{\mathrm{out}}$  do not help Note that algorithms are allowed to output a randomized  $f_{\mathrm{out}}$  and to output  $f_{\mathrm{out}} \notin \mathcal{H}$ . We will show that randomization and improperness of  $f_{\mathrm{out}}$  don't make the problem easier. That is, supposing that the data distribution is  $\mathcal{D}_{i^*}$  for some  $i^* \in [n]$ , finding a (possibly randomized and improper)  $f_{\mathrm{out}}$  is not easier than identifying  $i^*$ . Since our feature space  $\mathcal{X}$  is finite, we can enumerate all hypotheses not equal to  $21_{\{\mathbf{e}_{i^*}\}} - 1$  and calculate their strategic population loss as follows.
-  $21_{\emptyset} - 1$  predicts all negative and thus  $\mathcal{L}^{\mathrm{str}}(21_{\emptyset} - 1) = 1 - 6\varepsilon$ ;
- For any  $a \subset \mathcal{X}$  s.t.  $a \cap X_0 \neq \emptyset$ ,  $2\mathbb{1}_a - 1$  will predict any point drawn from  $\mathcal{D}_{i^*}$  as positive (since all points have radius greater than  $2\alpha$  and the distance between any two points in  $X_0$  is smaller than  $2\alpha$ ) and thus  $\mathcal{L}^{\mathrm{str}}(2\mathbb{1}_a - 1) = 6\varepsilon$ ;
- For any  $a \subset \{\mathbf{e}_1, \ldots, \mathbf{e}_n\}$  satisfying that  $\exists i \neq i^*$ ,  $\mathbf{e}_i \in a$ , we have  $\mathcal{L}^{\mathrm{str}}(2\mathbb{1}_a - 1) \geq 3\varepsilon$ . This is due to that when  $y = -x$  is chosen from  $\mathrm{Unif}(X_0)$  and the probability of  $d(x, \mathbf{e}_i) < d(x, \mathbf{e}_{i*})$  is  $\frac{1}{2}$ . When  $d(x, \mathbf{e}_i) < d(x, \mathbf{e}_{i*})$ ,  $2\mathbb{1}_a - 1$  will predict  $x$  as positive.
Under distribution  $\mathcal{D}_{i^*}$ , if we are able to find a (possibly randomized)  $f_{\mathrm{out}}$  with strategic loss of  $\mathcal{L}^{\mathrm{str}}(f_{\mathrm{out}}) \leq \varepsilon$ , then we have  $\mathcal{L}^{\mathrm{str}}(f_{\mathrm{out}}) = \mathbb{E}_{h \sim f_{\mathrm{out}}}[\mathcal{L}^{\mathrm{str}}(h)] \geq \operatorname*{Pr}_{h \sim f_{\mathrm{out}}} (h \neq 21_{\{\mathbf{e}_{i^*}\}} - 1) \cdot 3\varepsilon$ . Thus,  $\operatorname*{Pr}_{h \sim f_{\mathrm{out}}} (h = 21_{\{\mathbf{e}_{i^*}\}} - 1) \geq \frac{2}{3}$ . Hence, if we are able to find a (possibly randomized)  $f_{\mathrm{out}}$  with  $\varepsilon$  error, then we are able to identify  $i^*$  by checking which realization of  $f_{\mathrm{out}}$  has probability greater than  $\frac{2}{3}$ . In the following, we will focus on the sample complexity to identify  $i^*$ . Let  $i_{\mathrm{out}}$  denote the algorithm's answer to question "what is  $i^*$ ?".
Conservative algorithms When running a conservative algorithm, the rule of choosing  $f_{t}$  at round  $t$  and choosing the final output  $f_{\mathrm{out}}$  does not depend on the correct rounds, i.e.  $\{\tau \in [T] | \widehat{y}_{\tau} = y_{\tau}\}$ . Let's define
$$
\Delta_ {t} ^ {\prime} = \left\{ \begin{array}{l l} \Delta_ {t} & \text {i f} \hat {y} _ {t} \neq y _ {t} \\ \perp & \text {i f} \hat {y} _ {t} = y _ {t}, \end{array} \right. \tag {11}
$$
where  $\bot$  is just a symbol representing "no information". Then for any conservative algorithm, the selected predictor  $f_{t}$  is determined by  $(f_{\tau},\widehat{y}_{\tau},y_{\tau},\Delta_{\tau}^{\prime})$  for  $\tau < t$  and the final output  $f_{\mathrm{out}}$  is determined by  $(f_t,\widehat{y}_t,y_t,\Delta_t')_{t = 1}^T$ . From now on, we consider  $\Delta_t^\prime$  as the feedback in the learning process of a conservative algorithm since it makes no difference from running the same algorithm with feedback  $\Delta_t$ .
Smooth the data distribution For technical reasons (appearing later in the analysis), we don't want to analyze distribution  $\{\mathcal{D}_i|i\in [n]\}$  directly as the probability of  $\Delta_t = \mathbf{e}_i$  is 0 when  $f_{t}(\mathbf{e}_{i}) = +1$  under distribution  $\mathcal{D}_i$ . Instead, we consider the mixture of  $\mathcal{D}_i$  and another distribution  $\mathcal{D}_i''$ , which is identical to  $\mathcal{D}_i$  except that  $r(x) = d(x,\mathbf{e}_i)$  when  $y = -$ . More specifically, let  $\mathcal{D}_i' = (1 - p)\mathcal{D}_i + p\mathcal{D}_i''$  with some extremely small  $p$ , where  $\mathcal{D}_i''$ s marginal distribution over  $\mathcal{X}\times \mathcal{Y}$  is still  $\mathrm{Unif}(X_0)\times \mathrm{Rad}(1 - 6\varepsilon)$ ; the radius is  $r = 2$  when  $y = +,;$  and the radius is  $r = d(x,\mathbf{e}_i)$  when  $y = -$ . For any data distribution  $\mathcal{D}$ , let  $\mathbf{P}_{\mathcal{D}}$  be the dynamics of  $(f_{1},y_{1},\widehat{y}_{1},\Delta_{1}',\ldots ,f_{T},y_{T},\widehat{y}_{T},\Delta_{T}')$  under  $\mathcal{D}$ . According to Lemma 4, by setting  $p = \frac{\varepsilon}{16n^2}$ , when  $T\leq \frac{n}{\varepsilon}$ , with high probability we never sample from  $\mathcal{D}_i''$  and have that for any  $i,j\in [n]$
$$
\left| \mathbf {P} _ {\mathcal {D} _ {i}} \left(i _ {\text {o u t}} = j\right) - \mathbf {P} _ {\mathcal {D} _ {i} ^ {\prime}} \left(i _ {\text {o u t}} = j\right) \right| \leq \frac {1}{8}. \tag {12}
$$
From now on, we only consider distribution  $\mathcal{D}_i^\prime$  instead of  $\mathcal{D}_i$ . The readers might have the question that why not using  $\mathcal{D}_i^\prime$  for construction directly. This is because  $\mathcal{D}_i^\prime$  does not satisfy realizability and no hypothesis has zero loss under  $\mathcal{D}_i^\prime$ .
Information gain from different choices of  $f_{t}$  In each round of interaction, the learner picks a predictor  $f_{t}$ , which can be out of  $\mathcal{H}$ . Here we enumerate all choices of  $f_{t}$ .
-  $f_{t}(\cdot) = 2\mathbb{1}_{\emptyset} - 1$  predicts all points in  $\mathcal{X}$  by negative. No matter what  $i^{*}$  is, we will observe  $(\Delta_{t} = x_{t},y_{t}) \sim \mathrm{Unif}(X_{0}) \times \mathrm{Rad}(1 - 6\varepsilon)$  and  $\widehat{y}_t = -$ . They are identically distributed for all  $i^{*} \in [n]$ , and thus,  $\Delta_t^\prime$  is also identically distributed. We cannot tell any information of  $i^{*}$  from this round.
-  $f_{t} = 2\mathbb{1}_{a_{t}} - 1$  for some  $a_{t} \subset \mathcal{X}$  s.t.  $a \cap X_0 \neq \emptyset$ . Then  $\Delta_t = \Delta (x_t, f_t, r_t) = \Delta (x_t, f_t, 2\alpha)$  since  $r_t > 2\alpha$  and  $d(x_{t},f_{t}) \leq 2\alpha$ ,  $\widehat{y}_t = +$ ,  $y_{t} \sim \mathrm{Rad}(1 - 6\varepsilon)$ . None of these depends on  $i^{*}$  and again, the distribution of  $(\widehat{y}_t, y_t, \Delta_t')$  is identical for all  $i^{*}$  and we cannot tell any information of  $i^{*}$  from this round.
-  $f_{t} = 2\mathbb{1}_{a_{t}} - 1$  for some non-empty  $a_{t} \subset \{\mathbf{e}_{1}, \ldots, \mathbf{e}_{n}\}$ . For rounds with  $y_{t} = +$ , we have  $\widehat{y}_{t} = +$  and  $\Delta_{t} = \Delta(x_{t}, f_{t}, 2)$ , which still not depend on  $i^{*}$ . Thus we cannot learn any information about  $i^{*}$ . But we can learn when  $y_{t} = -$ . For rounds with  $y_{t} = -$ , if  $\Delta_{t} \in a_{t}$ , then we could observe  $\widehat{y}_{t} = +$  and  $\Delta_{t}' = \Delta_{t}$ , which at least tells that  $2\mathbb{1}_{\{\Delta_{t}\}} - 1$  is not the target function (with high probability); if  $\Delta_{t} \notin a_{t}$ , then  $\widehat{y}_{t} = -$  and we observe  $\Delta_{t}' = \bot$ .
Therefore, we only need to focus on the rounds with  $f_{t} = 2\mathbb{1}_{a_{t}} - 1$  for some non-empty  $a_{t} \subset \{\mathbf{e}_{1}, \ldots, \mathbf{e}_{n}\}$  and  $y_{t} = -$ . It is worth noting that drawing an example  $x$  from  $X_{0}$  uniformly, it is equivalent to uniformly drawing a permutation of  $\mathcal{H}$  such that the distances between  $x$  and  $h$  over all  $h \in \mathcal{H}$  are permuted according to it. Then  $\Delta_{t} = \mathbf{e}_{j}$  iff  $\mathbf{e}_{j} \in a_{t}$ ,  $d(x, \mathbf{e}_{j}) \leq d(x, \mathbf{e}_{i^{*}})$  and  $d(x, \mathbf{e}_{j}) \leq d(x, \mathbf{e}_{l})$  for all  $\mathbf{e}_{l} \in a_{t}$ . Let  $k_{t} = |a_{t}|$  denote the cardinality of  $a_{t}$ . In such rounds, under distribution  $D_{i^{*}}^{\prime}$ , the distribution of  $\Delta_{t}^{\prime}$  are described as follows.
1. The case of  $\mathbf{e}_{i^*} \in a_t$ : For all  $j \in a_t \setminus \{i^*\}$ , with probability  $\frac{1}{k_t}$ ,  $d(x_t, \mathbf{e}_j) = \min_{\mathbf{e}_l \in a_t} d(x_t, \mathbf{e}_l)$  and thus,  $\Delta_t' = \Delta_t = \mathbf{e}_j$  and  $\widehat{y}_t = +$  (mistake round). With probability  $\frac{1}{k_t}$ , we have  $d(x_t, \mathbf{e}_{i^*}) = \min_{\mathbf{e}_l \in a_t} d(x_t, \mathbf{e}_l)$ . If the example is drawn from  $\mathcal{D}_{i^*}$ , we have  $\Delta_t = x_t$  and  $y_t = -$  (correct round), thus  $\Delta_t' = \bot$ . If the example is drawn from  $\mathcal{D}_{i^*}'$ , we have we have  $\Delta_t' = \Delta_t = \mathbf{e}_{i^*}$  and  $y_t = +$  (mistake round). Therefore, according to the definition of  $\Delta_t'$  (Eq (11)), we have
$$
\Delta_ {t} ^ {\prime} = \left\{ \begin{array}{l l} \mathbf {e} _ {j} & \text {w . p .} \frac {1}{k _ {t}} \text {f o r} \mathbf {e} _ {j} \in a _ {t}, j \neq i ^ {*} \\ \mathbf {e} _ {i ^ {*}} & \text {w . p .} \frac {1}{k _ {t}} p \\ \perp & \text {w . p .} \frac {1}{k _ {t}} (1 - p). \end{array} \right.
$$
We denote this distribution by  $P_{\in}(a_t,i^*)$
2. The case of  $\mathbf{e}_{i^*} \notin a_t$ : For all  $j \in a_t$ , with probability  $\frac{1}{k_t + 1}$ , then  $d(x_t, \mathbf{e}_j) = \min_{\mathbf{e}_l \in a_t \cup \{\mathbf{e}_{i^*}\}} d(x_t, \mathbf{e}_l)$  and thus,  $\Delta_t = \mathbf{e}_j$  and  $\widehat{y}_t = +$  (mistake round). With probability  $\frac{1}{k_t + 1}$ , we have  $d(x, \mathbf{e}_{i^*}) < \min_{\mathbf{e}_l \in a_t} d(x_t, \mathbf{e}_l)$  and thus,  $\Delta_t = x_t$ ,  $\widehat{y}_t = -$  (correct round), and  $\Delta'_t = \bot$ . Therefore, the distribution of  $\Delta'_t$  is
$$
\Delta_ {t} ^ {\prime} = \left\{ \begin{array}{l l} \mathbf {e} _ {j} & \text {w . p .} \frac {1}{k _ {t} + 1} \text {f o r} \mathbf {e} _ {j} \in a _ {t} \\ \perp & \text {w . p .} \frac {1}{k _ {t} + 1}. \end{array} \right.
$$
We denote this distribution by  $P_{\notin}(a_t)$ .
To measure the information obtained from  $\Delta_t^\prime$ , we will utilize the KL divergence of the distribution of  $\Delta_t^\prime$  under the data distribution  $\mathcal{D}_{i^*}$  from that under a benchmark distribution. Let  $\overline{\mathcal{D}} = \frac{1}{n}\sum_{i\in n}\mathcal{D}_i^\prime$  denote the average distribution. The process of sampling from  $\overline{\mathcal{D}}$  is equivalent to sampling  $i^*$  uniformly at random from  $[n]$  first and drawing a sample from  $\mathcal{D}_{i^*}$ . Then under  $\overline{\mathcal{D}}$ , for any  $\mathbf{e}_j\in a_t$ , we have
$$
\begin{array}{l} \Pr \left(\Delta_ {t} ^ {\prime} = \mathbf {e} _ {j}\right) = \Pr \left(i ^ {*} = j\right) \Pr \left(\Delta_ {t} ^ {\prime} = \mathbf {e} _ {j} \mid i ^ {*} = j\right) + \Pr \left(i ^ {*} \in a _ {t} \setminus \{j \}\right) \Pr \left(\Delta_ {t} ^ {\prime} = \mathbf {e} _ {j} \mid i ^ {*} \in a _ {t} \setminus \{j \}\right) \\ + \Pr (i ^ {*} \notin a _ {t}) \Pr (\Delta_ {t} ^ {\prime} = \mathbf {e} _ {j} | i ^ {*} \notin a _ {t}) \\ \end{array}
$$
$$
= \frac {1}{n} \cdot \frac {p}{k _ {t}} + \frac {k _ {t} - 1}{n} \cdot \frac {1}{k _ {t}} + \frac {n - k _ {t}}{n} \cdot \frac {1}{k _ {t} + 1} = \frac {n k _ {t} - 1 + p (k _ {t} + 1)}{n k _ {t} (k _ {t} + 1)},
$$
$$
\begin{array}{l} \Pr \left(\Delta_ {t} ^ {\prime} = \bot\right) = \Pr \left(i ^ {*} \in a _ {t}\right) \Pr \left(\Delta_ {t} ^ {\prime} = \bot \mid i ^ {*} \in a _ {t}\right) + \Pr \left(i ^ {*} \notin a _ {t}\right) \Pr \left(\Delta_ {t} ^ {\prime} = \bot \mid i ^ {*} \notin a _ {t}\right) \\ = \frac {k _ {t}}{n} \cdot \frac {1 - p}{k _ {t}} + \frac {n - k _ {t}}{n} \cdot \frac {1}{k _ {t} + 1} = \frac {n + 1 - p (k _ {t} + 1)}{n (k _ {t} + 1)}. \\ \end{array}
$$
Thus, the distribution of  $\Delta_t^\prime$  under  $\overline{\mathcal{D}}$  is
$$
\Delta_ {t} ^ {\prime} = \left\{ \begin{array}{l l} \mathbf {e} _ {j} & \text {w . p .} \frac {n k _ {t} - 1 + p (k _ {t} + 1)}{n k _ {t} (k _ {t} + 1)} \text {f o r} \mathbf {e} _ {j} \in a _ {t} \\ \perp & \text {w . p .} \frac {n + 1 - p (k _ {t} + 1)}{n (k _ {t} + 1)}. \end{array} \right.
$$
We denote this distribution by  $\overline{P}(a_t)$ . Next we will compute the KL divergences of  $P_{\in}(a_t, i^*)$  and  $P_{\notin}(a_t)$  from  $\overline{P}(a_t)$ . We will use the inequality  $\log(1 + x) \leq x$  for  $x \geq 0$  in the following calculation. For any  $i^*$  s.t.  $\mathbf{e}_{i^*} \in a_t$ , we have
$$
\begin{array}{l} \mathrm {D} _ {\mathrm {K L}} \left(\bar {P} \left(a _ {t}\right) \| P _ {\in} \left(a _ {t}, i ^ {*}\right)\right) \\ = \left(k _ {t} - 1\right) \frac {n k _ {t} - 1 + p \left(k _ {t} + 1\right)}{n k _ {t} \left(k _ {t} + 1\right)} \log \left(\frac {n k _ {t} - 1 + p \left(k _ {t} + 1\right)}{n k _ {t} \left(k _ {t} + 1\right)} k _ {t}\right) \\ + \frac {n k _ {t} - 1 + p \left(k _ {t} + 1\right)}{n k _ {t} \left(k _ {t} + 1\right)} \log \left(\frac {n k _ {t} - 1 + p \left(k _ {t} + 1\right)}{n k _ {t} \left(k _ {t} + 1\right)} \cdot \frac {k _ {t}}{p}\right) \\ + \frac {n + 1 - p \left(k _ {t} + 1\right)}{n \left(k _ {t} + 1\right)} \log \left(\frac {n + 1 - p \left(k _ {t} + 1\right)}{n \left(k _ {t} + 1\right)} \cdot \frac {k _ {t}}{1 - p}\right) \\ \leq 0 + \frac {1}{k _ {t} + 1} \log \left(\frac {1}{p}\right) + \frac {2 p}{k _ {t} + 1} = \frac {1}{k _ {t} + 1} \log \left(\frac {1}{p}\right) + \frac {2 p}{k _ {t} + 1}, \tag {13} \\ \end{array}
$$
$$
\begin{array}{l} \mathrm {D} _ {\mathrm {K L}} \left(\bar {P} (a _ {t}) \| P _ {\notin} (a _ {t})\right) \\ = k _ {t} \frac {n k _ {t} - 1 + p \left(k _ {t} + 1\right)}{n k _ {t} \left(k _ {t} + 1\right)} \log \left(\frac {n k _ {t} - 1 + p \left(k _ {t} + 1\right)}{n k _ {t} \left(k _ {t} + 1\right)} \left(k _ {t} + 1\right)\right) \\ + \frac {n + 1 - p \left(k _ {t} + 1\right)}{n \left(k _ {t} + 1\right)} \log \left(\frac {n + 1 - p \left(k _ {t} + 1\right)}{n \left(k _ {t} + 1\right)} \left(k _ {t} + 1\right)\right) \\ \leq 0 + \frac {n + 1}{n ^ {2} \left(k _ {t} + 1\right)} = \frac {n + 1}{n ^ {2} \left(k _ {t} + 1\right)}. \tag {14} \\ \end{array}
$$
Lower bound of the information We utilize the information theoretical framework of proving lower bounds for linear bandits (Theorem 11 by Rajaraman et al. (2023)) here. For notation simplicity, for all  $i\in [n]$ , let  $\mathbf{P}_i$  denote the dynamics of  $(f_{1},\Delta_{1}^{\prime},y_{1},\widehat{y}_{1},\ldots ,f_{T},\Delta_{T}^{\prime},y_{T},\widehat{y}_{T})$  under  $\mathcal{D}_i^\prime$  and  $\overline{\mathbf{P}}$  denote the dynamics under  $\overline{\mathcal{D}}$ . Let  $B_{t}$  denote the event of  $\{f_t = 2\mathbb{1}_{a_t} - 1$  for some non-empty  $a_{t}\subset \{\mathbf{e}_{1},\dots ,\mathbf{e}_{n}\}\}$ . As discussed before, for any  $a_{t}$ , conditional on  $\neg B_{t}$  or  $y_{t} = +1$ ,  $(\Delta_t^\prime ,y_t,\widehat{y}_t)$  are identical in all  $\{\mathcal{D}_i^\prime |i\in [n]\}$ , and therefore, also identical in  $\overline{\mathcal{D}}$ . We can only obtain information at rounds when  $B_{t}\wedge (y_{t} = -1)$  occurs. In such rounds, we know that  $f_{t}$  is fully determined by history (possibly with external randomness, which does not depend on data distribution),  $y_{t} = -1$  and  $\widehat{y}_t$  is fully determined by  $\Delta_t^\prime (\widehat{y}_t = +1$  iff.  $\Delta_t^\prime \in a_t)$ .
Therefore, conditional the history  $H_{t - 1} = (f_1,\Delta_1',y_1,\widehat{y}_1,\ldots ,f_{t - 1},\Delta_{t - 1}'y_{t - 1},\widehat{y}_{t - 1})$  before time  $t$ , we have
$$
\begin{array}{l} \mathrm {D} _ {\mathrm {K L}} \left(\overline {{\mathbf {P}}} \left(f _ {t}, \Delta_ {t} ^ {\prime}, y _ {t}, \widehat {y} _ {t} \mid H _ {t - 1}\right) \| \mathbf {P} _ {i} \left(f _ {t}, \Delta_ {t} ^ {\prime}, y _ {t}, \widehat {y} _ {t} \mid H _ {t - 1}\right)\right) \\ = \overline {{\mathbf {P}}} (B _ {t} \wedge (y _ {t} = - 1)) \mathrm {D} _ {\mathrm {K L}} (\overline {{\mathbf {P}}} (\Delta_ {t} ^ {\prime} | H _ {t - 1}, B _ {t} \wedge (y _ {t} = - 1)) \| \mathbf {P} _ {i} (\Delta_ {t} ^ {\prime} | H _ {t - 1}, B _ {t} \wedge (y _ {t} = - 1))) \\ = 6 \varepsilon \overline {{\mathbf {P}}} \left(B _ {t}\right) \mathrm {D} _ {\mathrm {K L}} \left(\overline {{\mathbf {P}}} \left(\Delta_ {t} ^ {\prime} \mid H _ {t - 1}, B _ {t} \wedge (y _ {t} = - 1)\right) \| \mathbf {P} _ {i} \left(\Delta_ {t} ^ {\prime} \mid H _ {t - 1}, B _ {t} \wedge (y _ {t} = - 1)\right)\right), \tag {15} \\ \end{array}
$$
where the last equality holds due to that  $y_{t}\sim \mathrm{Rad}(1 - 6\varepsilon)$  and does not depend on  $B_{t}$ .
For any algorithm that can successfully identify  $i$  under the data distribution  $\mathcal{D}_i$  with probability  $\frac{3}{4}$  for all  $i\in [n]$ , then  $\mathbf{P}_{\mathcal{D}_i}(i_{\mathrm{out}} = i)\geq \frac{3}{4}$  and  $\mathbf{P}_{\mathcal{D}_j}(i_{\mathrm{out}} = i)\leq \frac{1}{4}$  for all  $j\neq i$ . Recall that  $\mathcal{D}_i$  and  $\mathcal{D}_i^\prime$  are very close when the mixture parameter  $p$  is small. Combining with Eq (12), we have
$$
\begin{array}{l} \left| \mathbf {P} _ {i} \left(i _ {\text {o u t}} = i\right) - \mathbf {P} _ {j} \left(i _ {\text {o u t}} = i\right) \right| \\ \geq \left| \mathbf {P} _ {\mathcal {D} _ {i}} \left(i _ {\text {o u t}} = i\right) - \mathbf {P} _ {\mathcal {D} _ {j}} \left(i _ {\text {o u t}} = i\right) \right| - \left| \mathbf {P} _ {\mathcal {D} _ {i}} \left(i _ {\text {o u t}} = i\right) - \mathbf {P} _ {i} \left(i _ {\text {o u t}} = i\right) \right| - \left| \mathbf {P} _ {\mathcal {D} _ {j}} \left(i _ {\text {o u t}} = i\right) - \mathbf {P} _ {j} \left(i _ {\text {o u t}} = i\right) \right| \\ \geq \frac {1}{2} - \frac {1}{4} = \frac {1}{4}. \\ \end{array}
$$
Then we have the total variation distance between  $\mathbf{P}_i$  and  $\mathbf{P}_j$
$$
\operatorname {T V} \left(\mathbf {P} _ {i}, \mathbf {P} _ {j}\right) \geq \left| \mathbf {P} _ {i} \left(i _ {\text {o u t}} = i\right) - \mathbf {P} _ {j} \left(i _ {\text {o u t}} = i\right) \right| \geq \frac {1}{4}. \tag {16}
$$
$$
\begin{array}{l} \mathbb {E} _ {i \sim \operatorname {U n i f} ([ n ])} \left[ \mathrm {T V} ^ {2} \left(\mathbf {P} _ {i}, \mathbf {P} _ {(i + 1) \bmod n}\right) \right] \leq 4 \mathbb {E} _ {i \sim \operatorname {U n i f} ([ n ])} \left[ \mathrm {T V} ^ {2} \left(\mathbf {P} _ {i}, \overline {{\mathbf {P}}}\right) \right] \\ \leq 2 \mathbb {E} _ {i} \left[ \mathrm {D} _ {\mathrm {K L}} (\overline {{\mathbf {P}}} \| \mathbf {P} _ {i}) \right] \quad \text {(P i n s k e r ' s i n e q)} \\ = 2 \mathbb {E} _ {i} \left[ \sum_ {t = 1} ^ {T} \mathrm {D} _ {\mathrm {K L}} \left(\overline {{\mathbf {P}}} \left(f _ {t}, \Delta_ {t} ^ {\prime}, y _ {t}, \widehat {y} _ {t} \mid H _ {t - 1}\right) \| \mathbf {P} _ {i} \left(f _ {t}, \Delta_ {t} ^ {\prime}, y _ {t}, \widehat {y} _ {t} \mid H _ {t - 1}\right)\right) \right] (Chain rule) \\ = 1 2 \varepsilon \mathbb {E} _ {i} \left[ \sum_ {t = 1} ^ {T} \overline {{\mathbf {P}}} \left(B _ {t}\right) \mathrm {D} _ {\mathrm {K L}} \left(\overline {{\mathbf {P}}} \left(\Delta_ {t} ^ {\prime} \mid H _ {t - 1}, B _ {t} \wedge (y _ {t} = - 1)\right) \| \mathbf {P} _ {i} \left(\Delta_ {t} ^ {\prime} \mid H _ {t - 1}, B _ {t} \wedge (y _ {t} = - 1)\right)\right) \right] (ApplyEq(15)) \\ = \frac {1 2 \varepsilon}{n} \sum_ {t = 1} ^ {T} \overline {{\mathbf {P}}} (B _ {t}) \sum_ {i = 1} ^ {n} \mathrm {D} _ {\mathrm {K L}} (\overline {{\mathbf {P}}} (\Delta_ {t} ^ {\prime} | H _ {t - 1}, B _ {t} \wedge (y _ {t} = - 1)) \| \mathbf {P} _ {i} (\Delta_ {t} ^ {\prime} | H _ {t - 1}, B _ {t} \wedge (y _ {t} = - 1))) \\ = \frac {1 2 \varepsilon}{n} \mathbb {E} _ {f _ {1: T} \sim \overline {{\mathbf {P}}}} \left[ \sum_ {t = 1} ^ {T} \mathbb {1} (B _ {t}) \left(\sum_ {i: i \in a _ {t}} \mathrm {D} _ {\mathrm {K L}} (\bar {P} (a _ {t}) \| P _ {\in} (a _ {t}, i)) + \sum_ {i: i \notin a _ {t}} \mathrm {D} _ {\mathrm {K L}} (\bar {P} (a _ {t}) \| P _ {\notin} (a _ {t}))\right) \right] \\ \leq \frac {1 2 \varepsilon}{n} \sum_ {t = 1} ^ {T} \mathbb {E} _ {f _ {1: T} \sim \overline {{\mathbf {P}}}} \left[ \sum_ {i: i \in a _ {t}} \left(\frac {1}{k _ {t} + 1} \log \left(\frac {1}{p}\right) + \frac {2 p}{k _ {t} + 1}\right) + \sum_ {i: i \notin a _ {t}} \frac {n + 1}{n ^ {2} (k _ {t} + 1)} \right] (ApplyEq(13),(14)) \\ \leq \frac {1 2 \varepsilon}{n} \sum_ {t = 1} ^ {T} (\log (\frac {1}{p}) + 2 p + 1) \\ \leq \frac {1 2 T \varepsilon (\log (1 6 n ^ {2} / \varepsilon) + 2)}{n}. \\ \end{array}
$$
Combining with Eq (16), we have that there exists a universal constant  $c$  such that  $T \geq \frac{cn}{\varepsilon(\log(n / \varepsilon) + 1)}$ .
I Proof of Theorem 8
Proof. We will prove Theorem 8 by constructing an instance of  $\mathcal{Q}$  and  $\mathcal{H}$  and then reduce it to a linear stochastic bandit problem.
- Consider the input metric space in the shape of a star, where  $\mathcal{X} = \{0,1,\dots,n\}$  and the distance function of  $d(0,i) = 1$  and  $d(i,j) = 2$  for all  $i \neq j \in [n]$ .
- Let the hypothesis class be a set of singletons over  $[n]$ , i.e.,  $\mathcal{H} = \{2\mathbb{1}_{\{i\}} - 1 | i \in [n]\}$ .