SlowGuess/NeurIPS_2023_OCR huggingface dataset:
By taking logarithm of both sides, we have
$$
0 \leq \ln (n _ {T}) = \ln (n) + \sum_ {t = 1} ^ {T} \ln (1 - \mathbb {1} (E _ {t}) \mathbb {1} (B _ {t}) \frac {p _ {t}}{2}) \leq \ln (n) - \sum_ {t = 1} ^ {T} \mathbb {1} (E _ {t}) \mathbb {1} (B _ {t}) \frac {p _ {t}}{2},
$$
where the last inequality adopts  $\ln (1 - x) \leq -x$  for  $x \in [0, 1)$ . Then by taking expectation of both sides, we have
$$
0 \leq \ln (n) - \sum_ {t = 1} ^ {T} \Pr \left(E _ {t} \wedge B _ {t}\right) \frac {p _ {t}}{2}.
$$
Since  $\operatorname{Pr}(E_t) = p_t$  and  $\operatorname{Pr}(B_t|E_t) \geq \frac{1}{2}$ , then we have
$$
\frac {1}{4} \sum_ {t = 1} ^ {T} p _ {t} ^ {2} \leq \ln (n).
$$
Then we have the expected number of mistakes  $\mathbb{E}\left[\mathcal{M}_{\mathrm{MWMR}}(T)\right]$  as
$$
\mathbb {E} \left[ \mathcal {M} _ {\mathrm {M W M R}} (T) \right] = \sum_ {t = 1} ^ {T} p _ {t} \leq \sqrt {\sum_ {t = 1} ^ {T} p _ {t} ^ {2}} \cdot \sqrt {T} \leq \sqrt {4 \ln (n) T},
$$
where the first inequality applies Cauchy-Schwarz inequality.
F Proof of Theorem 5
Proof. Construction of  $\mathcal{Q},\mathcal{H}$  and a set of realizable distributions
- Let feature space  $\mathcal{X} = \{\mathbf{0},\mathbf{e}_1,\dots ,\mathbf{e}_n\} \cup X_0$  where  $X_0 = \{\frac{\sigma(0,1,\ldots,n - 1)}{z} |\sigma \in S_n\}$  with  $z = \frac{\sqrt{1^2 + \ldots + (n - 1)^2}}{\alpha}$  for some small  $\alpha = 0.1$ . Here  $S_{n}$  is the set of all permutations over  $n$  elements. So  $X_0$  is the set of points whose coordinates are a permutation of  $\{0,1 / z,\dots ,(n - 1) / z\}$  and all points in  $X_0$  have the  $\ell_2$  norm equal to  $\alpha$ . Define a metric  $d$  by letting  $d(x_{1},x_{2}) = \| x_{1} - x_{2}\|_{2}$  for all  $x_{1},x_{2}\in \mathcal{X}$ . Then for any  $x\in X_0$  and  $i\in [n]$ ,  $d(x,\mathbf{e}_i) = \| x - \mathbf{e}_i\| _2 = \sqrt{(x_i - 1)^2 + \sum_{j\neq i}x_j^2} = \sqrt{1 + \sum_{j = 1}^n x_j^2 - 2x_i} = \sqrt{1 + \alpha^2 - 2x_i}$ . Note that we consider space  $(\mathcal{X},d)$  rather than  $(\mathbb{R}^n,\| \cdot \| _2)$ .
- Let the hypothesis class be a set of singletons over  $\{\mathbf{e}_i | i \in [n]\}$ , i.e.,  $\mathcal{H} = \{2\mathbb{1}_{\{\mathbf{e}_i\}} - 1 | i \in [n]\}$ .
- We now define a collection of distributions  $\{\mathcal{D}_i | i \in [n]\}$  in which  $\mathcal{D}_i$  is realized by  $2\mathbb{1}_{\{\mathbf{e}_i\}} - 1$ . For any  $i \in [n]$ ,  $\mathcal{D}_i$  puts probability mass  $1 - 3n\varepsilon$  on  $(0, 0, -1)$ . For the remaining  $3n\varepsilon$  probability mass,  $\mathcal{D}_i$  picks  $x$  uniformly at random from  $X_0$  and label it as positive. If  $x_i = 0$ , set radius  $r(x) = r_u \coloneqq \sqrt{1 + \alpha^2}$ ; otherwise, set radius  $r(x) = r_l \coloneqq \sqrt{1 + \alpha^2 - 2 \cdot \frac{1}{z}}$ . Hence,  $X_0$  are all labeled as positive. For  $j \neq i$ ,  $h_j = 2\mathbb{1}_{\{\mathbf{e}_j\}} - 1$  labels  $\{x \in X_0 | x_j = 0\}$  negative since  $r(x) = r_l$  and  $d(x, h_j) = r_u > r(x)$ . Therefore,  $\mathcal{L}^{\mathrm{str}}(h_j) = \frac{1}{n} \cdot 3n\varepsilon = 3\varepsilon$ . To output  $f_{\mathrm{out}} \in \mathcal{H}$ , we must identify the true target function.
Information gain from different choices of  $f_{t}$  Let  $h^{*} = 21_{\{\mathbf{e}_{i^{*}}\}} - 1$  denote the target function. Since  $(0,0,-1)$  is realized by all hypotheses, we can only gain information about the target function when  $x_{t} \in X_{0}$ . For any  $x_{t} \in X_{0}$ , if  $d(x_{t},f_{t}) \leq r_{l}$  or  $d(x_{t},f_{t}) > r_{u}$ , we cannot learn anything about the target function. In particular, if  $d(x_{t},f_{t}) \leq r_{l}$ , the learner will observe  $x_{t} \sim \mathrm{Unif}(X_{0}), y_{t} = +1$ ,  $\widehat{y}_{t} = +1$  in all  $\{\mathcal{D}_i|i \in [n]\}$ . If  $d(x_{t},f_{t}) > r_{u}$ , the learner will observe  $x_{t} \sim \mathrm{Unif}(X_{0}), y_{t} = +1$ ,  $\widehat{y}_{t} = -1$  in all  $\{\mathcal{D}_i|i \in [n]\}$ . Therefore, we cannot obtain any information about the target function.
Now for any  $x_{t} \in X_{0}$ , with the  $i_{t}$ -th coordinate being 0, we enumerate the distance between  $x$  and  $x'$  for all  $x' \in \mathcal{X}$ .
- For all  $x' \in X_0$ ,  $d(x, x') \leq \|x\| + \|x'\| \leq 2\alpha < r_l$ ;
- For all  $j \neq i_{t}$ ,  $d(x, \mathbf{e}_{j}) = \sqrt{1 + \alpha^{2} - 2x_{j}} \leq r_{l}$ ;
$d(x,\mathbf{e}_{i_t}) = r_u$
$d(x,\mathbf{0}) = \alpha < r_l.$
Only  $f_{t} = 2\mathbb{1}_{\{\mathbf{e}_{i_{t}}\}} - 1$  satisfies that  $r_{l} < d(x_{t},f_{t})\leq r_{u}$  and thus, we can only obtain information when  $f_{t} = 2\mathbb{1}_{\{\mathbf{e}_{i_{t}}\}} - 1$ . And the only information we learn is whether  $i_{t} = i^{*}$  because if  $i_{t}\neq i^{*}$ , no matter which  $i^{*}$  is, our observation is identical. If  $i_{t}\neq i^{*}$ , we can eliminate  $2\mathbb{1}_{\{\mathbf{e}_{i_{t}}\}} - 1$ .
Sample size analysis For any algorithm  $\mathcal{A}$ , his predictions are identical in all environments  $\{\mathcal{D}_i|i\in [n]\}$  before a round  $t$  in which  $f_{t} = 2\mathbb{1}_{\{\mathbf{e}_{i_{t}}\}} - 1$ . Then either he learns  $i_{t}$  in  $\mathcal{D}_{i_t}$  or he eliminates  $2\mathbb{1}_{\{\mathbf{e}_{i_t}\}} - 1$  and continues to perform the same in the other environments  $\{\mathcal{D}_i|i\neq i_t\}$ . Suppose that we run  $\mathcal{A}$  in all stochastic environments  $\{\mathcal{D}_i|i\in [n]\}$  simultaneously. When we identify  $i_{t}$  in environment  $\mathcal{D}_{i_t}$ , we terminate  $\mathcal{A}$  in  $\mathcal{D}_{i_t}$ . Consider a good algorithm  $\mathcal{A}$  which can identify  $i$  in  $\mathcal{D}_i$  with probability  $\frac{7}{8}$  after  $T$  rounds of interaction for each  $i\in [n]$ , that is,
$$
\Pr_ {\mathcal {D} _ {i}, \mathcal {A}} \left(i _ {\text {o u t}} \neq i\right) \leq \frac {1}{8}, \forall i \in [ n ]. \tag {3}
$$
$$
\sum_ {i \in [ n ]} \Pr_ {\mathcal {D} _ {i}, \mathcal {A}} \left(i _ {\text {o u t}} \neq i\right) \leq \frac {n}{8}. \tag {4}
$$
Let  $n_T$  denote the number of environments that have been terminated by the end of round  $T$ . Let  $B_t$  denote the event of  $x_t$  being in  $X_0$  and  $C_t$  denote the event of  $f_t = 2\mathbb{1}_{\{\mathbf{e}_{i_t}\}} - 1$ . Then we have  $\operatorname*{Pr}(B_t) = 3n\varepsilon$  and  $\operatorname*{Pr}(C_t|B_t) = \frac{1}{n}$ , and thus  $\operatorname*{Pr}(B_t \wedge C_t) = 3n\varepsilon \cdot \frac{1}{n}$ . Since at each round, we can eliminate one environment only when  $B_t \wedge C_t$  is true, then we have
$$
\mathbb {E} \left[ n _ {T} \right] \leq \mathbb {E} \left[ \sum_ {t = 1} ^ {T} \mathbb {1} \left(B _ {t} \wedge C _ {t}\right) \right] = T \cdot 3 n \varepsilon \cdot \frac {1}{n} = 3 \varepsilon T.
$$
Therefore, by setting  $T = \frac{\left\lfloor \frac{n}{2} \right\rfloor - 1}{6\varepsilon}$  and Markov's inequality, we have
$$
\Pr (n _ {T} \geq \left\lfloor \frac {n}{2} \right\rfloor - 1) \leq \frac {3 \varepsilon T}{\left\lfloor \frac {n}{2} \right\rfloor - 1} = \frac {1}{2}.
$$
When there are  $\left\lceil \frac{n}{2} \right\rceil + 1$  environments remaining, the algorithm has to pick one  $i_{\mathrm{out}}$ , which fails in at least  $\left\lceil \frac{n}{2} \right\rceil$  of the environments. Then we have
$$
\sum_ {i \in [ n ]} \operatorname * {P r} _ {\mathcal {D} _ {i}, \mathcal {A}} (i _ {\mathrm {o u t}} \neq i) \geq \left\lceil \frac {n}{2} \right\rceil \Pr (n _ {T} \leq \left\lfloor \frac {n}{2} \right\rfloor - 1) \geq \frac {n}{4},
$$
which conflicts with Eq (4). Therefore, for any algorithm  $\mathcal{A}$ , to achieve Eq (3), it requires  $T \geq \frac{\left\lfloor \frac{n}{2} \right\rfloor - 1}{6\varepsilon}$ .
G Proof of Theorem 6
Given Lemma 1, we can upper bound the expected strategic loss, then we can boost the confidence of the algorithm through the scheme in Section A.1. Theorem 6 follows by combining Lemma 1 and Lemma 2. Now we only need to prove Lemma 1.
Proof of Lemma 1. For any set of hypotheses  $H$ , for every  $z = (x, r, y)$ , we define
$$
\kappa_ {p} (H, z) := \left\{ \begin{array}{l l} | \{h \in H | h (\Delta (x, h, r)) = - \} | & \text {i f y = + ,} \\ 0 & \text {o t h e r w i s e .} \end{array} \right.
$$
So  $\kappa_{p}(H,z)$  is the number of hypotheses mislabeling  $z$  for positive  $z$ 's and 0 for negative  $z$ 's. Similarly, we define  $\kappa_{n}$  as follows,
$$
\kappa_ {n} (H, z) := \left\{ \begin{array}{l l} | \{h \in H | h (\Delta (x, h, r)) = + \} | & \text {i f} y = -, \\ 0 & \text {o t h e r w i s e .} \end{array} \right.
$$
So  $\kappa_{n}(H,z)$  is the number of hypotheses mislabeling  $z$  for negative  $z$ 's and 0 for positive  $z$ 's.
In the following, we divide the proof into two parts. First, recall that in Algorithm 2, the output is constructed by randomly sampling two hypotheses with replacement and taking the union of them. We represent the loss of such a random predictor using  $\kappa_p(H,z)$  and  $\kappa_{n}(H,z)$  defined above. Then we show that whenever the algorithm makes a mistake, with some probability, we can reduce  $\frac{\kappa_p(\mathrm{VS}_{t - 1},z_t)}{2}$  or  $\frac{\kappa_n(\mathrm{VS}_{t - 1},z_t)}{2}$  hypotheses and utilize this to provide a guarantee on the loss of the final output.
Upper bounds on the strategic loss For any hypothesis  $h$ , let  $\mathrm{fpr}(h)$  and  $\mathrm{fnr}(h)$  denote the false positive rate and false negative rate of  $h$  respectively. Let  $p_{+}$  denote the probability of drawing a positive sample from  $\mathcal{D}$ , i.e.,  $\operatorname*{Pr}_{(x,r,y)}\sim \mathcal{D}(y = +)$  and  $p_{-}$  denote the probability of drawing a negative sample from  $\mathcal{D}$ . Let  $\mathcal{D}_{+}$  and  $\mathcal{D}_{-}$  denote the data distribution conditional on that the label is positive and that the label is negative respectively. Given any set of hypotheses  $H$ , we define a random predictor  $R2(H) = h_1\lor h_2$  with  $h_1,h_2$  randomly picked from  $H$  with replacement. For a true positive  $z$ ,  $R2(H)$  will misclassify it with probability  $\frac{\kappa_p(H,z)^2}{|H|^2}$ . Then we can find that the false negative rate of  $R2(H)$  is
$$
\operatorname {f n r} (R 2 (H)) = \mathbb {E} _ {z = (x, r, +) \sim \mathcal {D} _ {+}} [ \Pr (R 2 (H) (x) = -) ] = \mathbb {E} _ {z = (x, r, +) \sim \mathcal {D} _ {+}} \left[ \frac {\kappa_ {p} (H , z) ^ {2}}{| H | ^ {2}} \right].
$$
Similarly, for a true negative  $z$ ,  $R2(H)$  will misclassify it with probability  $1 - (1 - \frac{\kappa_n(H,z)}{|H|})^2 \leq \frac{2\kappa_n(H,z)}{|H|}$ . Then the false positive rate of  $R2(H)$  is
$$
\operatorname {f p r} (R 2 (H)) = \mathbb {E} _ {z = (x, r, -) \sim \mathcal {D} _ {-}} [ \Pr (R 2 (H) (x) = +) ] \leq \mathbb {E} _ {z = (x, r, -) \sim \mathcal {D} _ {+}} \left[ \frac {2 \kappa_ {n} (H , z)}{| H |} \right].
$$
Hence the loss of  $R2(H)$  is
$$
\begin{array}{l} \mathcal {L} ^ {\mathrm {s t r}} (R 2 (H)) \leq p _ {+} \mathbb {E} _ {z \sim \mathcal {D} _ {+}} \left[ \frac {\kappa_ {p} (H , z) ^ {2}}{| H | ^ {2}} \right] + p _ {-} \mathbb {E} _ {z \sim \mathcal {D} _ {+}} \left[ \frac {2 \kappa_ {n} (H , z)}{| H |} \right] \\ = \mathbb {E} _ {z \sim \mathcal {D}} \left[ \frac {\kappa_ {p} (H , z) ^ {2}}{| H | ^ {2}} + 2 \frac {\kappa_ {n} (H , z)}{| H |} \right], \tag {5} \\ \end{array}
$$
where the last equality holds since  $\kappa_p(H,z) = 0$  for true negatives and  $\kappa_n(H,z) = 0$  for true positives.
Loss analysis In each round, the data  $z_{t} = (x_{t}, r_{t}, y_{t})$  is sampled from  $\mathcal{D}$ . When the label  $y_{t}$  is positive, if the drawn  $f_{t}$  satisfying that 1)  $f_{t}(\Delta(x_{t}, f_{t}, r_{t})) = -\alpha$  and 2)  $d(x_{t}, f_{t}) \leq \text{median}(\{d(x_{t}, h) | h \in \mathrm{VS}_{t-1}, h(\Delta(x_{t}, h, r_{t})) = -\}$  ), then we are able to remove  $\frac{\kappa_{p}(\mathrm{VS}_{t-1}, z_{t})}{2}$  hypotheses from the version
space. Let  $E_{p,t}$  denote the event of  $f_t$  satisfying the conditions 1) and 2). With probability  $\frac{1}{[\log_2(n_t)]}$ , we sample  $k_t = 1$ . Then we sample an  $f_t \sim \mathrm{Unif}(\mathrm{VS}_{t-1})$ . With probability  $\frac{\kappa_p(\mathrm{VS}_{t-1}, z_t)}{2n_t}$ , the sampled  $f_t$  satisfies the two conditions. So we have
$$
\Pr \left(E _ {p, t} \mid z _ {t}, \mathrm {V S} _ {t - 1}\right) \geq \frac {1}{\log_ {2} \left(n _ {t}\right)} \frac {\kappa_ {p} \left(\mathrm {V S} _ {t - 1} , z _ {t}\right)}{2 n _ {t}}. \tag {6}
$$
The case of  $y_{t}$  being negative is similar to the positive case. Let  $E_{n,t}$  denote the event of  $f_{t}$  satisfying that 1)  $f_{t}(\Delta (x_{t},f_{t},r_{t})) = +$  and 2)  $d(x_{t},f_{t})\geq \mathrm{median}(\{d(x_{t},h)|h\in \mathrm{VS}_{t - 1},h(\Delta (x_{t},h,r_{t})) = +\})$ . If  $\kappa_{n}(\mathrm{VS}_{t - 1},z_{t})\geq \frac{n_{t}}{2}$ , then with probability  $\frac{1}{[\log_2(n_t)]}$ , we sample  $k_{t} = 1$ . Then with probability greater than  $\frac{1}{4}$  we will sample an  $f_{t}$  satisfying that 1)  $f_{t}(\Delta (x_{t},f_{t},r_{t})) = +$  and 2)  $d(x_{t},f_{t})\geq \mathrm{median}(\{d(x_{t},h)|h\in \mathrm{VS}_{t - 1},h(\Delta (x_{t},h,r_{t})) = +\})$ . It follows that  $\kappa_{n}(\mathrm{VS}_{t - 1},z_{t}) < \frac{n_{t}}{2}$ , then with probability  $\frac{1}{[\log_2(n_t)]}$ , we sampled a  $k_{t}$  satisfying
$$
\frac {n _ {t}}{4 \kappa_ {n} (\mathrm {V S} _ {t - 1} , z _ {t})} <   k _ {t} \leq \frac {n _ {t}}{2 \kappa_ {n} (\mathrm {V S} _ {t - 1} , z _ {t})}.
$$
Then we randomly sample  $k_{t}$  hypotheses and the expected number of sampled hypotheses which mislabel  $z_{t}$  is  $k_{t} \cdot \frac{\kappa_{n}(\mathrm{VS}_{t-1}, z_{t})}{n_{t}} \in \left( \frac{1}{4}, \frac{1}{2} \right]$ . Let  $g_{t}$  (given the above fixed  $k_{t}$ ) denote the number of sampled hypotheses which mislabel  $x_{t}$  and we have  $\mathbb{E}[g_{t}] \in \left( \frac{1}{4}, \frac{1}{2} \right]$ . When  $g_{t} > 0$ ,  $f_{t}$  will misclassify  $z_{t}$  by positive. We have
$$
\operatorname * {P r} (g _ {t} = 0) = \left(1 - \frac {\kappa_ {n} (\mathrm {V S} _ {t - 1} , z _ {t})}{n _ {t}}\right) ^ {k _ {t}} <   \left(1 - \frac {\kappa_ {n} (\mathrm {V S} _ {t - 1} , z _ {t})}{n _ {t}}\right) ^ {\frac {n _ {t}}{4 \kappa_ {n} (\mathrm {V S} _ {t - 1} , z _ {t})}} \leq e ^ {- 1 / 4} \leq 0. 7 8
$$
and by Markov's inequality, we have
$$
\Pr (g _ {t} \geq 3) \leq \frac {\mathbb {E} [ g _ {t} ]}{3} \leq \frac {1}{6} \leq 0. 1 7.
$$
Thus  $\operatorname*{Pr}(g_t\in \{1,2\})\geq 0.05$ . Conditional on  $g_{t}$  is either 1 or 2, with probability  $\geq \frac{1}{4}$ , all of these  $g_{t}$  hypotheses  $h^\prime$  satisfies  $d(x_{t},h^{\prime})\geq$  median  $(\{d(x_{t},h)|h\in \mathrm{VS}_{t - 1},h(\Delta (x_{t},h,r_{t})) = +\})$ , which implies that  $d(x_{t},f_{t})\geq$  median  $(\{d(x_{t},h)|h\in \mathrm{VS}_{t - 1},h(\Delta (x_{t},h,r_{t})) = +\})$ . Therefore, we have
$$
\Pr \left(E _ {n, t} \mid z _ {t},, \mathrm {V S} _ {t - 1}\right) \geq \frac {1}{8 0 \log_ {2} \left(n _ {t}\right)}. \tag {7}
$$
Let  $v_{t}$  denote the fraction of hypotheses we eliminated at round  $t$ , i.e.,  $v_{t} = 1 - \frac{n_{t + 1}}{n_{t}}$ . Then we have
$$
v _ {t} \geq \mathbb {1} \left(E _ {p, t}\right) \frac {\kappa_ {p} \left(\mathrm {V S} _ {t - 1} , z _ {t}\right)}{2 n _ {t}} + \mathbb {1} \left(E _ {n, t}\right) \frac {\kappa_ {n} \left(\mathrm {V S} _ {t - 1} , z _ {t}\right)}{2 n _ {t}}. \tag {8}
$$
Since  $n_{t + 1} = n_t(1 - v_t)$ , we have
$$
1 \leq n _ {T + 1} = n \prod_ {t = 1} ^ {T} \left(1 - v _ {t}\right).
$$
$$
0 \leq \ln n _ {T + 1} = \ln n + \sum_ {t = 1} ^ {T} \ln (1 - v _ {t}) \leq \ln n - \sum_ {t = 1} ^ {T} v _ {t},
$$
where we use  $\ln (1 - x) \leq -x$  for  $x \in [0, 1)$  in the last inequality. By re-arranging terms, we have
$$
\sum_ {t = 1} ^ {T} v _ {t} \leq \ln n.
$$
Combined with Eq (8), we have
$$
\sum_ {t = 1} ^ {T} \mathbb {1} (E _ {p, t}) \frac {\kappa_ {p} (\mathrm {V S} _ {t - 1} , z _ {t})}{2 n _ {t}} + \mathbb {1} (E _ {n, t}) \frac {\kappa_ {n} (\mathrm {V S} _ {t - 1} , z _ {t})}{2 n _ {t}} \leq \ln n.
$$
By taking expectation w.r.t. the randomness of  $f_{1:T}$  and dataset  $S = z_{1:T}$  on both sides, we have
$$
\sum_ {t = 1} ^ {T} \mathbb {E} _ {f _ {1: T}, z _ {1: T}} \left[ \mathbb {1} (E _ {p, t}) \frac {\kappa_ {p} (\mathrm {V S} _ {t - 1} , z _ {t})}{2 n _ {t}} + \mathbb {1} (E _ {n, t}) \frac {\kappa_ {n} (\mathrm {V S} _ {t - 1} , z _ {t})}{2 n _ {t}} \right] \leq \ln n.
$$
Since the  $t$ -th term does not depend on  $f_{t+1:T}, z_{t+1:T}$  and  $\mathrm{VS}_{t-1}$  is determined by  $z_{1:t-1}$  and  $f_{1:t-1}$ , the  $t$ -th term becomes
$$
\mathbb {E} _ {f _ {1: t}, z _ {1: t}} \left[ \mathbb {1} (E _ {p, t}) \frac {\kappa_ {p} (\mathrm {V S} _ {t - 1} , z _ {t})}{2 n _ {t}} + \mathbb {1} (E _ {n, t}) \frac {\kappa_ {n} (\mathrm {V S} _ {t - 1} , z _ {t})}{2 n _ {t}} \right]
$$
$$
= \mathbb {E} _ {f _ {1: t - 1}, z _ {1: t}} \left[ \mathbb {E} _ {f _ {t}} \left[ \mathbb {1} (E _ {p, t}) \frac {\kappa_ {p} (\mathrm {V S} _ {t - 1} , z _ {t})}{2 n _ {t}} + \mathbb {1} (E _ {n, t}) \frac {\kappa_ {n} (\mathrm {V S} _ {t - 1} , z _ {t})}{2 n _ {t}} | f _ {1: t - 1}, z _ {1: t} \right] \right]
$$
$$
= \mathbb {E} _ {f _ {1: t - 1}, z _ {1: t}} \left[ \mathbb {E} _ {f _ {t}} \left[ \mathbb {1} \left(E _ {p, t}\right) \mid f _ {1: t - 1}, z _ {1: t} \right] \frac {\kappa_ {p} \left(\mathrm {V S} _ {t - 1} , z _ {t}\right)}{2 n _ {t}} + \mathbb {E} _ {f _ {t}} \left[ \mathbb {1} \left(E _ {n, t}\right) \mid f _ {1: t - 1}, z _ {1: t} \right] \frac {\kappa_ {n} \left(\mathrm {V S} _ {t - 1} , z _ {t}\right)}{2 n _ {t}} \right] \tag {9}
$$
$$
\geq \mathbb {E} _ {f _ {1: t - 1}, z _ {1: t}} \left[ \frac {1}{\log_ {2} (n _ {t})} \frac {\kappa_ {p} ^ {2} \left(\mathrm {V S} _ {t - 1} , z _ {t}\right)}{4 n _ {t} ^ {2}} + \frac {1}{8 0 \log_ {2} (n _ {t})} \frac {\kappa_ {n} \left(\mathrm {V S} _ {t - 1} , z _ {t}\right)}{2 n _ {t}} \right], \tag {10}
$$
where Eq (9) holds due to that  $\mathrm{VS}_{t-1}$  is determined by  $f_{1:t-1}, z_{1:t-1}$  and does not depend on  $f_t$  and Eq (10) holds since  $\operatorname{Pr}_{f_t}(E_{p,t}|f_{1:t-1}, z_{1:t}) = \operatorname{Pr}_{f_t}(E_{p,t}|\mathrm{VS}_{t-1}, z_t) \geq \frac{1}{\log_2(n_t)} \frac{\kappa_p(\mathrm{VS}_{t-1}, z_t)}{2n_t}$  by Eq (6) and  $\operatorname{Pr}_{f_t}(E_{n,t}|f_{1:t-1}, z_{1:t}) = \operatorname{Pr}_{f_t}(E_{n,t}|\mathrm{VS}_{t-1}, z_t) \geq \frac{1}{80\log_2(n_t)}$  by Eq (7). Thus, we have
$$
\sum_ {t = 1} ^ {T} \mathbb {E} _ {f _ {1: t - 1}, z _ {1: t}} \left[ \frac {1}{\log_ {2} (n _ {t})} \frac {\kappa_ {p} ^ {2} (\mathrm {V S} _ {t - 1} , z _ {t})}{4 n _ {t} ^ {2}} + \frac {1}{8 0 \log_ {2} (n _ {t})} \frac {\kappa_ {n} (\mathrm {V S} _ {t - 1} , z _ {t})}{2 n _ {t}} \right] \leq \ln n.
$$
Since  $z_{t} \sim \mathcal{D}$  and  $z_{t}$  is independent of  $z_{1:t-1}$  and  $f_{1:t-1}$ , thus, we have the  $t$ -th term on the LHS being
$$
\begin{array}{l} \mathbb {E} _ {f _ {1: t - 1}, z _ {1: t}} \left[ \frac {1}{\log_ {2} (n _ {t})} \frac {\kappa_ {p} ^ {2} (\mathrm {V S} _ {t - 1} , z _ {t})}{4 n _ {t} ^ {2}} + \frac {1}{8 0 \log_ {2} (n _ {t})} \frac {\kappa_ {n} (\mathrm {V S} _ {t - 1} , z _ {t})}{2 n _ {t}} \right] \\ = \mathbb {E} _ {f _ {1: t - 1}, z _ {1: t - 1}} \left[ \mathbb {E} _ {z _ {t} \sim \mathcal {D}} \left[ \frac {1}{\log_ {2} (n _ {t})} \frac {\kappa_ {p} ^ {2} (\mathrm {V S} _ {t - 1} , z _ {t})}{4 n _ {t} ^ {2}} + \frac {1}{8 0 \log_ {2} (n _ {t})} \frac {\kappa_ {n} (\mathrm {V S} _ {t - 1} , z _ {t})}{2 n _ {t}} \right] \right] \\ \geq \frac {1}{3 2 0 \log_ {2} (n)} \mathbb {E} _ {f _ {1: t - 1}, z _ {1: t - 1}} \left[ \mathbb {E} _ {z \sim \mathcal {D}} \left[ \frac {\kappa_ {p} ^ {2} (\mathrm {V S} _ {t - 1} , z)}{n _ {t} ^ {2}} + \frac {2 \kappa_ {n} (\mathrm {V S} _ {t - 1} , z)}{n _ {t}} \right] \right] \\ \geq \frac {1}{3 2 0 \log_ {2} (n)} \mathbb {E} _ {f _ {1: t - 1}, z _ {1: t - 1}} \left[ \mathcal {L} ^ {\mathrm {s t r}} \left(R 2 \left(\mathrm {V S} _ {t - 1}\right)\right) \right], \\ \end{array}
$$
where the last inequality adopts Eq (5). By summing them up and re-arranging terms, we have
$$
\mathbb {E} _ {f _ {1: T}, z _ {1: T}} \left[ \frac {1}{T} \sum_ {t = 1} ^ {T} \mathcal {L} ^ {\mathrm {s t r}} (R 2 (\mathrm {V S} _ {t - 1})) \right] = \frac {1}{T} \sum_ {t = 1} ^ {T} \mathbb {E} _ {f _ {1: t - 1}, z _ {1: t - 1}} \left[ \mathcal {L} ^ {\mathrm {s t r}} (R 2 (\mathrm {V S} _ {t - 1})) \right] \leq \frac {3 2 0 \log_ {2} (n) \ln (n)}{T}.
$$
For the output of Algorithm 2, which randomly picks  $\tau$  from  $[T]$ , randomly samples  $h_1, h_2$  from  $\mathrm{VS}_{\tau - 1}$  with replacement and outputs  $h_1 \vee h_2$ , the expected loss is
$$
\begin{array}{l} \mathbb {E} \left[ \mathcal {L} ^ {\mathrm {s t r}} (\mathcal {A} (S)) \right] = \mathbb {E} _ {S, f _ {1: T}} \left[ \frac {1}{T} \sum_ {t = 1} ^ {T} \mathbb {E} _ {h _ {1}, h _ {2} \sim \mathrm {U n i f} \left(\mathrm {V S} _ {t - 1}\right)} \left[ \mathcal {L} ^ {\mathrm {s t r}} \left(h _ {1} \vee h _ {2}\right) \right] \right] \\ = \mathbb {E} _ {S, f _ {1: T}} \left[ \frac {1}{T} \sum_ {t = 1} ^ {T} \mathcal {L} ^ {\mathrm {s t r}} (R 2 (\mathrm {V S} _ {t - 1})) \right] \\ \leq \frac {3 2 0 \log_ {2} (n) \ln (n)}{T} \leq \varepsilon , \\ \end{array}
$$
when  $T \geq \frac{320\log_2(n)\ln(n)}{\varepsilon}$ .
Post proof discussion of Lemma 1
- Upon first inspection, readers might perceive a resemblance between the proof of the loss analysis section and the standard proof of converting regret bound to error bound. This standard proof converts a regret guarantee on  $f_{1:T}$  to an error guarantee of  $\frac{1}{T}\sum_{t=1}^{T}f_t$ . However, in this proof, the predictor employed in each round is  $f_t$ , while the output is an average over  $R2(\mathrm{VS}_{t-1})$  for all  $t \in [T]$ . Our algorithm does not provide a regret guarantee on  $f_{1:T}$ .
- Please note that our analysis exhibits asymmetry regarding losses on true positives and true negatives. Specifically, the probability of identifying and reducing half of the misclassifying hypotheses on true positives, denoted as  $\operatorname{Pr}(E_{p,t}|z_t,\mathrm{VS}_{t - 1})$  (Eq (6)), is lower than the corresponding probability for true negatives,  $\operatorname{Pr}(E_{n,t}|z_t,\mathrm{VS}_{t - 1})$  (Eq (7)). This discrepancy arises due to the different levels of difficulty in detecting misclassifying hypotheses. For example, if there is exactly one hypothesis  $h$  misclassifying a true positive  $z_{t} = (x_{t},r_{t},y_{t})$ , it is very hard to detect this  $h$ . We must select an  $f_{t}$  satisfying that  $d(x_{t},f_{t}) > d(x_{t},h^{\prime})$  for all  $h^\prime \in \mathcal{H}\setminus \{h\}$  (hence  $f_{t}$  will make a mistake), and that  $d(x_{t},f_{t})\leq d(x_{t},h)$  (so that we will know  $h$  misclassify  $z_{t}$ ). Algorithm 2 controls the distance  $d(x_{t},f_{t})$  through  $k_{t}$ , which is the number of hypotheses in the union. In this case, we can only detect  $h$  when  $k_{t} = 1$  and  $f_{t} = h$ , which occurs with probability  $\frac{1}{n_t\log(n_t)}$ .
However, if there is exactly one hypothesis  $h$  misclassifying a true negative  $z_{t} = (x_{t}, r_{t}, y_{t})$ , we have that  $d(x_{t}, h) = \min_{h' \in \mathcal{H}} d(x_{t}, h')$ . Then by setting  $f_{t} = \vee_{h \in \mathcal{H}} h$ , which will make a mistake and tells us  $h$  is a misclassifying hypothesis. Our algorithm will pick such an  $f_{t}$  with probability  $\frac{1}{\log(n_{t})}$ .
H Proof of Theorem 7
Proof. We will prove Theorem 7 by constructing an instance of  $\mathcal{Q}$  and  $\mathcal{H}$  and showing that for any conservative learning algorithm, there exists a realizable data distribution s.t. achieving  $\varepsilon$  loss requires at least  $\widetilde{\Omega} \left( \frac{|H|}{\varepsilon} \right)$  samples.
Construction of  $\mathcal{Q},\mathcal{H}$  and a set of realizable distributions