SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
\bar {L} _ {n} ^ {(N)} = 1 + q _ {1} \bar {L} _ {2: n} ^ {(N A _ {1, 1})} + (1 - q _ {1}) \bar {L} _ {2: n} ^ {(N)},
$$
and, using the same reasoning as before, it suffices to take
$$
q _ {1} \geq - \frac {1}{\log_ {4} A _ {1 , 1}}.
$$
An analogous reasoning applies when  $A_{1,1} > 3/4$ , so that for this case, it suffices to take
$$
q _ {1} \geq - \frac {1}{\log_ {4} (1 - A _ {1 , 1})}.
$$
Now, consider the case  $1 / 4 < A_{1,1} \leq 1 / 2$ . If  $y_{1} = 0$ , then
$$
\begin{array}{l} \bar {L} _ {n} ^ {(N)} = 1 + \log_ {4} A _ {1, 1} + q _ {1} \bar {L} _ {2: n} ^ {(N (1 - A _ {1, 1}))} + (1 - q _ {1}) \bar {L} _ {2: n} ^ {(N)} \\ \leq 1 + \log_ {4} A _ {1, 1} + q _ {1} \log_ {4} (N (1 - A _ {1, 1})) + (1 - q _ {1}) \log_ {4} N, \\ \end{array}
$$
$$
q _ {1} \geq - \frac {1 + \log_ {4} A _ {1 , 1}}{\log_ {4} (1 - A _ {1 , 1})}.
$$
Similarly, if  $y_{1} = 1$ , we must have
$$
q _ {1} \geq - \frac {- \log_ {4} A _ {1 , 1}}{\log_ {4} A _ {1 , 1}} = 1.
$$
Since we do not know  $y_{1}$  before the decision, the only possibility is to take  $q_{1} = 1$ .
A similar reasoning applies to the case  $1/2 < A_{1,1} \leq 3/4$ . Therefore, the above relations determine the expression for  $q_t$  in the theorem, while enforcing the desired regret bound.
C Proof of Theorem 2
As mentioned after Theorem 2, an analogous result holds for non-uniform weight initializations. We will state and prove this more general result below, from which Theorem 2 as stated in the main text follows as a special case.
Theorem 2 (with non-uniform weight initialization). Consider an exponentially weighted forecaster with initial weight vector  $\pmb{w}_{\cdot,0} = (w_{1,0}, \dots, w_{N,0})$  such that  $\sum_{i \in [N]} w_{i,0} = 1$ ,  $p_t = A_{1,t}$  and
$$
\mathbb {E} \left[ L _ {n} ^ {\left(\boldsymbol {w} _ {i, 0}\right)} \right] \leq \mathbb {E} \left[ \min  _ {i \in [ N ]} \left(L _ {i, n} - \frac {\ln w _ {i , 0}}{\eta}\right) \right] + \frac {n \eta}{8}. \tag {12}
$$
where the superscript in  $L_{n}^{(\boldsymbol{w},,0)} \coloneqq \sum_{t=1}^{n} \ell(\hat{y}_{t}, y_{t})$  makes the dependence on the initial weights explicit. In particular, for the choice of initial weights  $w_{i,0} \coloneqq 1/N$  for all  $i \in [N]$ , we have
$$
\bar {R} _ {n} = \mathbb {E} \left[ L _ {n} \right] - \mathbb {E} \left[ \min  _ {i \in [ N ]} L _ {i, n} \right] \leq \frac {\ln N}{\eta} + \frac {n \eta}{8}.
$$
Proof. The proof strategy is similar to that used in Theorem 1. We begin by noting that at time  $t$ , the internal state of the label-efficient exponentially weighted forecaster is determined by the weight vector  $\boldsymbol{w}_{.,t-1} = (w_{1,t-1},\dots,w_{N,t-1})$ . Therefore, it suffices to focus on the requirements for  $q_1$  for an arbitrary weight vector. As in Theorem 1, we proceed by induction on  $n$ .
For  $n = 1$ , the theorem statement is trivially true, as this algorithm coincides with the ordinary exponentially weighted forecaster (also, the right-hand-side of (12) is bounded from below by  $1/2$ ). Proceeding by induction in  $n$ , suppose (12) holds for  $1, \ldots, n - 1$  outcomes. Let  $\bar{L}_{n|1}^{(\boldsymbol{w},0)}$  denote the expected cumulative loss of the forecaster given  $(y_1, \{f_{i,1}\}_{i=1}^N)$ , i.e., the true label and the expert predictions for time  $t = 1$ :
$$
\bar {L} _ {n | 1} ^ {\left(\boldsymbol {w} _ {., 0}\right)} = \mathbb {E} \left[ L _ {n} ^ {\left(\boldsymbol {w} _ {., 0}\right)} \mid y _ {1}, \left\{f _ {i, 1} \right\} _ {i = 1} ^ {N} \right].
$$
$$
\begin{array}{l} \bar {L} _ {n | 1} ^ {\left(\boldsymbol {w} _ {., 0}\right)} = \mathbb {E} \left[ \ell (\hat {y} _ {1}, y _ {1}) \mid y _ {1}, \left\{f _ {i, 1} \right\} _ {i = 1} ^ {N} \right] + \mathbb {E} \left[ \sum_ {t = 2} ^ {n} \ell (\hat {y} _ {t}, y _ {t}) \mid y _ {1}, \left\{f _ {i, 1} \right\} _ {i = 1} ^ {N} \right] \\ = A _ {1, 1} + \left(1 - 2 A _ {1, 1}\right) y _ {1} + q _ {1} \mathbb {E} \left[ \sum_ {t = 2} ^ {n} \ell \left(\hat {y} _ {t}, y _ {t}\right) \mid Z _ {1} = 1, y _ {1}, \{f _ {i, 1} \} _ {i = 1} ^ {N} \right] \\ + (1 - q _ {1}) \mathbb {E} \left[ \sum_ {t = 2} ^ {n} \ell (\hat {y} _ {t}, y _ {t}) \mid Z _ {1} = 0, y _ {1}, \{f _ {i, 1} \} _ {i = 1} ^ {N} \right] \\ \leq A _ {1, 1} + (1 - 2 A _ {1, 1}) y _ {1} + q _ {1} \bar {L} _ {n - 1} ^ {\left(\boldsymbol {w} _ {., 1}\right)} + (1 - q _ {1}) \bar {L} _ {n - 1} ^ {\left(\boldsymbol {w} _ {., 0}\right)}. \\ \end{array}
$$
In order to prove the desired result, it is enough to show that
$$
\bar {L} _ {n | 1} ^ {\left(\boldsymbol {w} _ {\cdot , 0}\right)} \leq \mathbb {E} \left[ \min  _ {i \in [ N ]} \left(\ell_ {i, 1} + L _ {i, 2: n} - \frac {\ln w _ {i , 0}}{\eta}\right) \right] + \frac {n \eta}{8} = \mathbb {E} \left[ \left(\ell_ {i ^ {\prime}, 1} + L _ {i ^ {\prime}, 2: n} - \frac {\ln w _ {i ^ {\prime} , 0}}{\eta}\right) \right] + \frac {n \eta}{8}.
$$
Here, we let  $L_{i,2:n} \coloneqq \sum_{t=2}^{n} \ell_{i,t}$  and let  $i'$  denote the arg min of the right-hand side. Now, using the induction hypothesis, we obtain
$$
\begin{array}{l} \bar {L} _ {n | 1} ^ {\left(\boldsymbol {w},. 0\right)} \leq A _ {1, 1} + (1 - 2 A _ {1, 1}) y _ {1} + \frac {(n - 1) \eta}{8} \\ + \mathbb {E} \left[ \min  _ {i \in [ N ]} \left(L _ {i, 2: n} + q _ {1} \frac {- \ln w _ {i , 1}}{\eta} + (1 - q _ {1}) \frac {- \ln w _ {i , 0}}{\eta}\right) \right] \\ \leq A _ {1, 1} + (1 - 2 A _ {1, 1}) y _ {1} + \frac {(n - 1) \eta}{8} \\ + \mathbb {E} \left[ \left(L _ {i ^ {\prime}, 2: n} + q _ {1} \frac {- \ln w _ {i ^ {\prime} , 1}}{\eta} + (1 - q _ {1}) \frac {- \ln w _ {i ^ {\prime} , 0}}{\eta}\right) \right]. \\ \end{array}
$$
In the last step, we used the fact that since the upper bound holds for the minimum  $i$ , it holds for  $i'$  in particular. To ensure that the bound in the theorem holds, it is thus sufficient to select  $A_{1,1}$  such that it satisfies
$$
A _ {1, 1} + (1 - 2 A _ {1, 1}) y _ {1} + \left(\frac {q _ {1}}{\eta} (\ln w _ {i ^ {\prime}, 0} - \ln w _ {i ^ {\prime}, 1}) - \ell_ {i ^ {\prime}, 1}\right) \leq \frac {\eta}{8}.
$$
Notice that, after the first observation, we are back in a situation similar to that at time  $t = 1$ , but possibly with a different weight vector. Specifically, for  $i \in [N]$
$$
w _ {i, 1} = \frac {w _ {i , 0} e ^ {- \eta \ell_ {i , 1} / q _ {1}}}{\sum_ {i = 1} ^ {N} w _ {i , 0} e ^ {- \eta \ell_ {i , 1} / q _ {1}}}
$$
It is important at this point that  $w_{i,1}$  depends on the choice  $q_{1}$ , so we cannot simply solve the above equation for  $q_{1}$ . To proceed, it is easier to consider the two possible values of  $y_{1}$  separately.
Case  $y_{1} = 0$ : Note that
$$
w _ {i, 1} = \frac {w _ {i , 0} e ^ {- \eta \ell \left(f _ {i , 1} , y _ {1}\right) / q _ {1}}}{1 - A _ {1 , 1} + A _ {1 , 1} e ^ {- \eta / q _ {1}}} .
$$
Therefore, it suffices to have
$$
A _ {1, 1} + \frac {q _ {1}}{\eta} \ln \left(1 - A _ {1, 1} + A _ {1, 1} e ^ {- \eta / q _ {1}}\right) \leq \frac {\eta}{8}. \tag {13}
$$
Case  $y_{1} = 1$ : Similarly as above,
$$
w _ {i, 1} = \frac {w _ {i , 0} e ^ {- \eta \ell \left(f _ {i , 1} , y _ {1}\right) / q _ {1}}}{A _ {1 , 1} + \left(1 - A _ {1 , 1}\right) e ^ {- \eta / q _ {1}}} .
$$
$$
1 - A _ {1, 1} + \frac {q _ {1}}{\eta} \ln \left(A _ {1, 1} + (1 - A _ {1, 1}) e ^ {- \eta / q _ {1}}\right) \leq \frac {\eta}{8}. \tag {14}
$$
As we do not know the values of  $y_{1}$  when computing  $q_{1}$ , we must simultaneously satisfy (13) and (14). Nevertheless, these two conditions involve only  $\eta$  and  $A_{1,1}$ . Thus, we can identify the range of values that  $q_{1}$  can take, as a function of  $\eta$  and  $A_{1,1}$ , while still ensuring the desired regret bound. Specifically, we require that  $q_{1} \geq q_{1}^{*}$ , where
$$
\begin{array}{l} q _ {1} ^ {*} := q _ {1} ^ {*} (A _ {1, 1}, \eta) = \inf  \left\{q \in (0, 1 ]: A _ {1, 1} + \frac {q}{\eta} \ln \left(1 - A _ {1, 1} + A _ {1, 1} e ^ {- \eta / q}\right) \leq \frac {\eta}{8}, \right. \\ \left. 1 - A _ {1, 1} + \frac {q}{\eta} \ln \left(A _ {1, 1} + (1 - A _ {1, 1}) e ^ {- \eta / q}\right) \leq \frac {\eta}{8} \right\}. \\ \end{array}
$$
At this point, it might be unclear if  $q_1^*$  is well defined, namely, if there always exists  $q \in [0,1]$  satisfying both (13) and (14). This is indeed the case, as shown in Lemma 1. By noting that  $\mathbb{E}[\bar{L}_{i,n|1}^{(\boldsymbol{w}_{\cdot,0})}] = \bar{L}_{i,n}^{(\boldsymbol{w}_{\cdot,0})}$ , we have completed the induction step.
As stated at the beginning of the proof, looking at  $q_{1}$  suffices to determine the general requirements for  $q_{t}$ , concluding the proof. The statement given in (7) follows from instantiating the general result with uniform initial weights.
D Proof of Lemma 1
Lemma 1. For all  $\eta >0$  and  $x\in [0,1]$  we have
Proof. Let  $B \sim \operatorname{Ber}(x)$ , with  $x \in [0,1]$ . Note that  $\mathbb{E}[e^{-\eta B}] = (1 - x) + xe^{-\eta}$ . Note also that, by [3, Lem. A.1] we have  $\ln \mathbb{E}[e^{-\eta B}] \leq -\eta x + \eta^2 / 8$ . Putting these two facts together we conclude that
$$
x + \frac {1}{\eta} \ln \left(1 - x + x e ^ {- \eta}\right) \leq x - x + \eta / 8 = \eta / 8.
$$
Therefore, the point 1 is always contained in Definition 1, concluding the proof.
E Proof of Lemma 2
$$
q ^ {*} (x, \eta) \leq \min  (4 x (1 - x) + \eta / 3, 1).
$$
Proof. As already shown in Lemma 1, we know that  $q^{*}(x,\eta)\leq 1$ . Let  $\eta >0$  be arbitrary. Note that  $q^{*}(x,\eta)$  needs to be a solution in  $q$  of the following equation:
$$
\underbrace {\eta x + q \ln \left(1 - x + x e ^ {- \eta / q}\right)} _ {:= g (\eta)} \leq \frac {\eta^ {2}}{8}. \tag {15}
$$
Note that  $\lim_{\eta \to 0}g(\eta) = 0$  , so we can extend the definition of  $g$  to 0 by continuity. Specifically,
$$
g (\eta) := \left\{ \begin{array}{l l} \eta x + q \ln (1 - x + x e ^ {- \eta / q}) & \text {i f} \eta > 0 \\ 0 & \text {i f} \eta = 0 \end{array} \right..
$$
We proceed by using a Taylor expansion of  $g(\eta)$  around 0. Tenuous, but straightforward computations yields
$$
g ^ {\prime} (\eta) = \frac {\partial}{\partial \eta} g (\eta) = x \left(1 - \frac {1}{x + (1 - x) e ^ {\eta / q}}\right); g ^ {\prime \prime} (\eta) = \frac {1}{q} x (1 - x) \frac {e ^ {\eta / q}}{(x + (1 - x) e ^ {\eta / q}) ^ {2}},
$$
$$
g ^ {\prime \prime \prime} (\xi) = \frac {1}{q ^ {2}} \left(- \tau + 3 \tau^ {2} - 2 \tau^ {3}\right), \text {w i t h} \tau = \frac {x e ^ {- \xi / q}}{1 - x + x e ^ {- \xi / q}},
$$
where  $\xi > 0$ . In conclusion,
$$
\begin{array}{l} g (\eta) = g (0) + g ^ {\prime} (0) \eta + g ^ {\prime \prime} (0) \frac {\eta^ {2}}{2} + g ^ {\prime \prime \prime} (\xi) \frac {\eta^ {3}}{6} \\ = \frac {1}{q} x (1 - x) \frac {\eta^ {2}}{2} + g ^ {\prime \prime \prime} (\xi) \frac {\eta^ {3}}{6}. \\ \end{array}
$$
where  $\xi \in [0,\eta ]$ . At this point we can examine the structure of the solution of (15) when  $\eta \rightarrow 0$ . Note that  $q^{*}(x,\eta)$  necessarily satisfies
$$
g (\eta) = \frac {1}{q ^ {*} (x , \eta)} x (1 - x) \frac {\eta^ {2}}{2} + o (\eta^ {2}) = \frac {\eta^ {2}}{8},
$$
implying that  $q^{*}(x,\eta)\to 4x(1 - x)$  as  $\eta \rightarrow 0$ , proving the first statement in the lemma.
For the second statement in the lemma, one needs to more carefully control the error term  $g^{\prime \prime \prime}(\xi)$ . We begin by noting that  $\tau \in [0,x]$ . This implies that  $-\tau +3\tau^{2} - 2\tau^{3}\leq x(1 - x)$  (this can be checked by algebraic manipulation<sup>1</sup>). Therefore,  $q^{*}(x,\eta)\leq q$ , where  $q$  is the solution of
$$
\frac {1}{q} x (1 - x) \frac {\eta^ {2}}{2} + \frac {1}{q ^ {2}} x (1 - x) \frac {\eta^ {3}}{6} = \frac {\eta^ {2}}{8}.
$$
This is a simple quadratic equation in  $q$ , yielding the solution
$$
q = 2 x (1 - x) + \sqrt {(2 x (1 - x)) ^ {2} + \frac {4}{3} x (1 - x) \eta}.
$$
Although the above expression is a valid upper bound on  $q^{*}(x,\eta)$ , it is not a very convenient one. A more convenient upper bound can be obtained by noting that
$$
\begin{array}{l} 2 x (1 - x) + \sqrt {(2 x (1 - x)) ^ {2} + \frac {4}{3} x (1 - x) \eta} \leq 4 x (1 - x) + \eta / 3 \\ \Longleftrightarrow (2 x (1 - x)) ^ {2} + \frac {4}{3} x (1 - x) \eta \leq (2 x (1 - x) + \eta / 3) ^ {2} \\ \Longleftrightarrow \frac {4}{3} x (1 - x) \eta \leq \frac {4}{3} x (1 - x) + \frac {\eta^ {2}}{9}. \\ \end{array}
$$
Thus, we have  $q \leq 4x(1 - x) + \eta / 3$ , concluding the proof.
It suffices to check that the solutions in  $\tau$  of  $-1 + 3\tau -2\tau^2\leq 1 - x$  , if they exist, satisfy  $\tau \in [0,x]$
F Proof of Theorem 3
In the proof of Theorem 3, we will require the following anytime version of Freedman's inequality:
Lemma 3. Let  $X_{1},\ldots ,X_{n}$  be a martingale difference sequence with respect to some filtration  $\mathcal{F}_1\subset \dots \subset \mathcal{F}_n$  and with  $|X_t|\leq b$  for all  $t$  almost surely. Let  $\Sigma_t^2 = \sum_{j = 1}^t\mathbb{E}[X_j^2 |\mathcal{F}_{j - 1}]$ . Then, for any  $\delta < 1 / e$  and  $n\geq 4$ ,
$$
\mathbb {P} \bigg (\exists t \in [ n ]: \sum_ {j = 1} ^ {t} X _ {j} > 2 \max \left\{2 \sqrt {\Sigma_ {t} ^ {2} \ln (1 / \delta)}, b \ln (1 / \delta) \right\} \bigg) \leq \ln (n) \delta .
$$
A proof of this result for the special case that  $\mathcal{F}_t = \sigma(X_1, \ldots, X_t)$  can be found in [37, Lemma 3], which is an extended version of [38]. Their proof goes through unchanged for general filtrations.
We are now ready to prove Theorem 3. As aforementioned, we will prove the result under a more general condition than given in the theorem statement. Specifically, instead of assuming that the best expert in expectation is apparent from the first round, we only require the best expert to emerge after a time  $\tau^{*}$ . This condition is given in a precise form in (17). Clearly, the assumption stated in the main text implies that the condition in (17) holds with  $\tau^{*} = 0$ .
Theorem 3 (with the best expert emerging after a time  $\tau^{*}$ ). Consider the label-efficient exponentially weighted forecaster from Theorem 2 with  $q_{t} = \min(4A_{1,t}(1 - A_{1,t}) + \eta/3, 1)$  and any  $\eta > 0$ . Define  $\tau$  as
$$
\tau = \left\lceil \frac {4 8 \ln \left(1 / \delta_ {2}\right)}{\eta \Delta^ {2}} + \frac {2}{\eta \Delta} \ln \left(\frac {N}{\eta}\right)\right\rceil . \tag {16}
$$
Suppose that there exists a single best expert  $i^*$  and a time  $\tau^* \leq \tau$  such that, for all  $i \neq i^*$ ,
$$
\frac {1}{t} \sum_ {j = 1} ^ {t} \mathbb {E} _ {t} \left[ \ell_ {i, j} - \ell_ {i ^ {*}, j} \right] \geq \Delta > 0 \quad \text {a l m o s t s u r e l y f o r} t \geq \tau^ {*}. \tag {17}
$$
$$
\mathbb {E} [ S _ {n} ] \leq \frac {5 0}{\eta \Delta^ {2}} \ln \left(\frac {N \ln n}{\eta}\right) + 3 \eta n + 1.
$$
Proof. Recall that the estimated losses are  $\tilde{\ell}_{i,t} = \ell_{i,t}Z_t / q_t$ . Let  $\tilde{\Lambda}_t^i = \sum_{j=1}^t \tilde{l}_{i,j} - \tilde{l}_{i^*,j}$  denote the cumulative estimated loss relative to that of the best expert, and let  $\tilde{\Lambda}_t^{\min} = \min_{i \neq i^*} \tilde{\Lambda}_t^i$ .
Our argument separates the analysis in two regimes: Regime 1, where  $t \leq \tau$ , and Regime 2, where  $t > \tau$ . The specific choice of  $\tau$  given above arises naturally later in the analysis. Regime 1, in which labels will be collected frequently, is expected to be relatively short, so we simply upper-bound  $S_{\tau} \leq \tau$ . It then remains to bound the number of collected labels in Regime 2. To this end, we need  $\tau$  to be chosen such that
$$
q _ {t} \leq \frac {4 \eta}{3} \quad \text {f o r a l l} t > \tau \tag {18}
$$
with probability at least  $1 - \delta_{1}$ , where  $\delta_1\in (0,1]$  will be fixed later. Let  $\mathcal{E}$  denote the event that (18) holds. It then follows that the expected number of labels collected in Regime 2 is at most
$$
\mathbb {E} \left[ S _ {n} - S _ {\tau} \right] = \mathbb {E} \left[ \sum_ {t = \tau + 1} ^ {n} q _ {t} \mathbb {1} \left\{\mathcal {E} \right\} \right] + \mathbb {E} \left[ \sum_ {t = \tau + 1} ^ {n} q _ {t} \mathbb {1} \left\{\bar {\mathcal {E}} \right\} \right] \leq \frac {4}{3} \eta n \Pr (\mathcal {E}) + n \Pr (\bar {\mathcal {E}}) \leq \frac {4}{3} \eta n + n \delta_ {1}.
$$
All together, we arrive at the following bound on the expected label complexity:
$$
\mathbb {E} \left[ S _ {n} \right] \leq \tau + \frac {4}{3} \eta n + n \delta_ {1}. \tag {19}
$$
It remains to verify that the choice of  $\tau$  in (16) leads to  $\delta_{1}$  being sufficiently small. To this end, let
$$
A _ {t} ^ {*} := \frac {\sum_ {i : f _ {i , t} = f _ {i ^ {*} , t}} w _ {i , t - 1}}{\sum_ {i = 1} ^ {N} w _ {i , t - 1}}
$$
denote the weighted proportion of experts that agrees with the best expert at time  $t$ . Then
$$
A _ {t + 1} ^ {*} \geq \frac {w _ {i ^ {*} , t}}{\sum_ {i = 1} ^ {N} w _ {i , t}} = \frac {1}{1 + \sum_ {i \neq i ^ {*}} e ^ {- \eta \tilde {\Lambda} _ {t} ^ {i}}} \geq \frac {1}{1 + N e ^ {- \eta \tilde {\Lambda} _ {t} ^ {\mathrm {m i n}}}}.
$$
Note that  $A_{1,t}\in \{A_{t + 1}^{*},1 - A_{t + 1}^{*}\}$  . Consequently,
$$
q _ {t + 1} \leq 4 A _ {t + 1} ^ {*} (1 - A _ {t + 1} ^ {*}) + \frac {\eta}{3} \leq 4 (1 - A _ {t + 1} ^ {*}) + \frac {\eta}{3} \leq \frac {4}{1 + e ^ {\eta \tilde {\Lambda} _ {t} ^ {\mathrm {m i n}}} / N} + \frac {\eta}{3}.
$$
The desired condition from (18) is therefore satisfied if, for all  $i \neq i^*$ ,
$$
\tilde {\Lambda} _ {t} ^ {i} \geq \frac {1}{\eta} \ln \left(\frac {N}{\eta}\right) \quad \text {f o r a l l} t = \tau , \dots , n - 1. \tag {20}
$$
To study the evolution of  $\tilde{\Lambda}_t^i$ , we use a martingale argument. Consider any fixed  $i \neq i^*$ , and define the martingale difference sequence
$$
X _ {t} = - (\tilde {\ell} _ {i, t} - \tilde {\ell} _ {i ^ {*}, t}) + \mathbb {E} _ {t} [ \tilde {\ell} _ {i, t} - \tilde {\ell} _ {i ^ {*}, t} ] = - \frac {(\ell_ {i , t} - \ell_ {i ^ {*} , t}) Z _ {t}}{q _ {t}} + \mathbb {E} _ {t} [ \ell_ {i, t} - \ell_ {i ^ {*}, t} ].
$$
Without loss of generality, we may assume that  $\eta \leq 3$ , because otherwise (19) holds trivially for any pair of  $\tau$  and  $\delta_{1}$ . Then  $q_{t} \geq \eta / 3$ ,  $|X_{t}| \leq 2 / q_{t} \leq 6 / \eta$ , and
$$
\mathbb {E} [ X _ {t} ^ {2} | \mathcal {F} _ {t - 1} ] \leq \mathbb {E} \Big [ (\tilde {\ell} _ {i, t} - \tilde {\ell} _ {i ^ {*}, t}) ^ {2} | \mathcal {F} _ {t - 1} \Big ] = \mathbb {E} \Big [ \frac {(\ell_ {i , t} - \ell_ {i ^ {*} , t}) ^ {2}}{q _ {t}} | \mathcal {F} _ {t - 1} \Big ] \leq \frac {3}{\eta}.
$$
Hence, by Lemma 3, we have
$$
\tilde {\Lambda} _ {t} ^ {i} \geq \sum_ {j = 1} ^ {t} \mathbb {E} _ {t} \left[ \ell_ {i, t} - \ell_ {i ^ {*}, t} \right] - \max  \left\{4 \sqrt {\frac {3 t}{\eta} \ln \left(1 / \delta_ {2}\right)}, \frac {1 2}{\eta} \ln \left(1 / \delta_ {2}\right) \right\} \quad \text {f o r a l l} t \in [ n ]. \tag {21}
$$
Using Assumption (17), it follows that
$$
\tilde {\Lambda} _ {t} ^ {i} \geq t \Delta - \max  \left\{4 \sqrt {\frac {3 t}{\eta} \ln \left(1 / \delta_ {2}\right)}, \frac {1 2}{\eta} \ln \left(1 / \delta_ {2}\right) \right\} \quad \text {f o r a l l} t \geq \tau^ {*} \tag {22}
$$
with probability at least  $1 - (\ln (n)\delta_2)$  for any  $\delta_2\in (0,1 / e]$ . By taking  $\delta_{2} = \min \{\delta_{1} / (N\ln n),1 / e\}$  and applying the union bound, we can make (22) hold for all  $i\neq i^{*}$  simultaneously with probability at least  $1 - \delta_{1}$ . A sufficient condition for (22) to imply (20) is then to take
$$
\tau = \left\lceil \frac {4 8 \ln (1 / \delta_ {2})}{\eta \Delta^ {2}} + \frac {2}{\eta \Delta} \ln \left(\frac {N}{\eta}\right)\right\rceil ,
$$
matching the definition in (16). We prove this claim in Lemma 4 below. Note that if our choice of  $\tau > n$ , then (19) still holds trivially, because  $S_{n} \leq n$ . Evaluating (19) with the given choice of  $\tau$  and taking  $\delta_{1} = \eta$  (assuming  $\eta < 1$ , since (19) holds trivially if  $\delta_{1} \geq 1$ ), we arrive at the following bound:
$$
\begin{array}{l} \mathbb {E} [ S _ {n} ] \leq \left\lceil \frac {4 8 \ln (1 / \delta_ {2})}{\eta \Delta^ {2}} + \frac {2}{\eta \Delta} \ln \left(\frac {N}{\eta}\right)\right\rceil + \frac {4}{3} \eta n + n \delta_ {1} \\ \leq \frac {4 8 \ln (N \ln (n) / \delta_ {1})}{\eta \Delta^ {2}} + \frac {2}{\eta \Delta} \ln \left(\frac {N}{\eta}\right) + \frac {4}{3} \eta n + n \delta_ {1} + 1 \\ = \frac {4 8 \ln (N \ln (n) / \eta)}{\eta \Delta^ {2}} + \frac {2}{\eta \Delta} \ln \left(\frac {N}{\eta}\right) + \frac {4}{3} \eta n + \eta n + 1 \\ \leq \frac {5 0 \ln (N \ln (n) / \eta)}{\eta \Delta^ {2}} + 3 \eta n + 1, \\ \end{array}
$$
thus completing the proof.
Lemma 4. Assume that
$$
\tilde {\Lambda} _ {t} ^ {i} \geq t \Delta - \max  \left\{4 \sqrt {\frac {3 t}{\eta} \ln \left(1 / \delta_ {2}\right)}, \frac {1 2}{\eta} \ln \left(1 / \delta_ {2}\right) \right\} \quad f o r a l l \quad t \leq n.
$$
Then, with
$$
\tilde {\Lambda} _ {t} ^ {i} \geq \frac {1}{\eta} \ln \left(\frac {N}{\eta}\right) \quad f o r a l l \quad t = \tau , \dots , n - 1.
$$
Proof. We will consider the two possible outcomes of the maximum separately. First, we consider the case where the first term is the maximum. Then, we need to find  $\tau$  such that for  $t \geq \tau$ ,
$$
\tilde {\Lambda} _ {t} ^ {i} \geq t \Delta - 4 \sqrt {\frac {3 t}{\eta} \ln (1 / \delta_ {2})} - \frac {1}{\eta} \ln \left(\frac {N}{\eta}\right) \geq 0.
$$
A straightforward calculation shows that this is satisfied for
$$
t \geq \frac {\left(\sqrt {4 8 \ln (1 / \delta_ {2}) + 4 \Delta \ln (N / \eta)} + 4 \sqrt {3 \Delta \ln (1 / \delta_ {2})}\right) ^ {2}}{4 \eta \Delta^ {2}}.
$$
Since  $(a + b)^2 \leq 2a^2 + 2b^2$  for  $a, b > 0$ , this is satisfied given the simpler condition
$$
t \geq \left\lceil \frac {4 8}{\eta \Delta^ {2}} \ln \left(1 / \delta_ {2}\right) + \frac {2}{\eta \Delta} \ln \left(\frac {N}{\eta}\right)\right\rceil = \tau .
$$
Next, we assume that the second term is the maximum. Then, we have
$$
\begin{array}{l} \tilde {\Lambda} _ {t} ^ {i} \geq t \Delta - \frac {1 2}{\eta} \ln (1 / \delta_ {2}) \\ \geq \tau \Delta - \frac {1 2}{\eta} \ln (1 / \delta_ {2}) \\ = \left\lceil \frac {4 8 \ln \left(1 / \delta_ {2}\right)}{\eta \Delta^ {2}} + \frac {2}{\eta \Delta} \ln \left(\frac {N}{\eta}\right)\right\rceil \Delta - \frac {1 2}{\eta} \ln \left(1 / \delta_ {2}\right) \\ \geq \frac {4 8 \ln \left(1 / \delta_ {2}\right)}{\eta \Delta} + \frac {2}{\eta} \ln \left(\frac {N}{\eta}\right) - \frac {1 2 \ln \left(1 / \delta_ {2}\right)}{\eta \Delta} \\ \geq \frac {2}{\eta} \ln \left(\frac {N}{\eta}\right), \\ \end{array}
$$
where we used the assumption that  $t \geq \tau$ . Thus, for this case, the desired condition holds with the specified  $\tau$ . Therefore, with the specified  $\tau$ , the desired statement holds for  $t = \tau, \dots, n - 1$ .
G Experimental details
In this section, we describe the simulation study in Section 6 in detail. The full code, which can be executed in less than one hour on an M1 processor, is provided in the supplementary material.
We consider a sequential prediction problem with  $n = 50000$  total rounds and  $N = 225$  experts. The number of experts is chosen to be higher than  $\sqrt{n}$  and odd. Then, we repeat the following simulation for 500 independent runs. First, we generate  $n$  independent features  $\{X_{i}\}_{i\in [n]}$  from the uniform distribution on  $[0,1]$ . Then, for each feature  $X_{i}$ , we randomly generate a label  $Y_{i}$ , where the probability  $\mathbb{P}(Y_i = 1|X_i = x) = \zeta (x)$ , where
$$
\zeta (x) = \frac {1}{2} + \frac {1}{2} \operatorname {s i g n} (x - 1 / 2) | x - 1 / 2 | ^ {\kappa - 1}.
$$
Note that the optimal prediction of the label is simply  $\mathbb{1}\{x\geq 1 / 2\}$ . We run simulations both for  $\kappa = 1.5$  and  $\kappa = 2$ .
We set the experts to be threshold classifiers, with thresholds uniformly spaced across  $[0,1]$ . Specifically, for all  $i\in [N]$  and  $t\in [n]$ ,
$$
f _ {i, t} = \mathbb {1} \left\{X _ {t} \geq \frac {i - 1}{N - 1} \right\}.
$$