SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Proof. We only used the calibration property in the proofs of Equations (2) and (3). In the proof of Equation (2), we used the fact that  $\mathbb{P}\left[J = j\mid p_j^i\leq e^{-\zeta}\right]\leq e^{-\zeta}$ ; the right-hand side now becomes  $\tau e^{-\zeta}$ , and so the right-hand side of Equation (2) changes to  $\tau n e^{-\zeta}$ . Similarly, in the proof of Equation (3), we use the calibration property in the proof of Lemma A.4; the right-hand side of the lemma changes to  $\tau mnq$ , and correspondingly Equation (3) changes to  $\tau mn^{2}e^{-\zeta /n}$ .
Lemma 4.8 is only used in the proof of Corollary 4.9, where  $2m^{2}n^{2}T$  is replaced by  $2\tau m^{2}n^{2}T$ . Since  $\tau$  is a constant, Corollary 4.9 holds verbatim.
B  $\Omega (\sqrt{T})$  Lower bound
We show that no OMD algorithm with a constant step size<sup>11</sup> substantially outperforms Algorithm 1.
Theorem B.1. For every strictly convex function  $R:\Delta^m\to \mathbb{R}$  that is continuously twice differentiable at its minimum, and  $\eta \geq 0$ , online mirror descent with regularizer  $R$  and constant step size  $\eta$  incurs  $\Omega (\sqrt{T})$  expected regret.
Proof. Our examples will have  $m = n = 2$ . The space of weights is one-dimensional; let us call  $w$  the weight of the first expert. We may treat  $R$  as a (convex) function of  $w$ , and similarly for the losses at each time step. We assume that  $R'(0.5) = 0$ ; this allows us to assume that  $w_{1} = 0.5$  and does not affect the proof idea.
It is straightforward to check that if Experts 1 and 2 assign probabilities  $p$  and  $\frac{1}{2}$ , respectively, to the correct outcome, then
$$
L ^ {\prime} (w) = \frac {(1 - p) ^ {w}}{p ^ {w} + (1 - p) ^ {w}} \ln \frac {1 - p}{p}.
$$
If roles are reversed (they say  $\frac{1}{2}$  and  $p$  respectively) then
$$
L ^ {\prime} (w) = - \frac {(1 - p) ^ {1 - w}}{p ^ {1 - w} + (1 - p) ^ {1 - w}} \ln \frac {1 - p}{p}.
$$
We first prove the regret bound if  $\eta$  is small ( $\eta \leq T^{-1/2}$ ). Consider the following setting: Expert 1 always reports  $(50\%, 50\%)$ ; Expert 2 always reports  $(90\%, 10\%)$ ; and Outcome 1 happens with probability  $90\%$  at each time step. It is a matter of simple computation that:
-  $L^{\prime}(w) \leq 2$  no matter the outcome or the value of  $w$ .
- If  $w \geq 0.4$ , then  $p_1^* (w) \leq 0.8$ .
The first point implies that  $R'(w_t) \geq -2\eta t$  for all  $t$ . It follows from the second point that the algorithm will output weights that will result in an aggregate probability of less than  $80\%$  for values of  $t$  such that  $-2\eta t \geq R'(0.4)$ , i.e. for  $t \leq \frac{-R'(0.4)}{2\eta}$ . Each of these time steps accumulates constant regret compared to the optimal weight vector in hindsight (which with high probability will be near 1). Therefore, the expected total regret accumulated during these time steps is  $\Omega(1/\eta) = \Omega(\sqrt{T})$ .
$^{10}$ Fun fact: the technical term for a corollary to a proof is a porism.
11While Algorithm 1 does not always have a constant step size, it does so with high probability. The examples that prove Theorem B.1 cause  $\Omega (\sqrt{T})$  regret in the typical case, rather than causing unusually large regret in an atypical case. This makes our comparison of Algorithm 1 to this class fair.
Now we consider the case in which  $\eta$  is large ( $\eta \geq \sqrt{T}$ ). In this case our example is the same as before, except we change which expert is "ignorant" (reports  $(50\%, 50\%)$  and which is "informed" (reports  $(90\%, 10\%)$ ). Specifically the informed expert will be the one with a lower weight (breaking ties arbitrarily).
We will show that our algorithm incurs  $\Omega(\eta)$  regret compared to always choosing weight 0.5. Suppose without loss of generality that at a given time step  $t$ , Expert 1 is informed (so  $w^t \leq 0.5$ ). Observe that
$$
\begin{array}{l} L (w ^ {t}) - L (0. 5) = - (0. 5 - w ^ {t}) L ^ {\prime} (0. 5) + O ((0. 5 - w) ^ {2}) \\ = - (0. 5 - w ^ {t}) \frac {\sqrt {1 - p}}{\sqrt {p} + \sqrt {1 - p}} \ln \frac {1 - p}{p} + O ((0. 5 - w) ^ {2}), \\ \end{array}
$$
where  $p$  is the probability that Expert 1 assigns to the event that happens (so  $p = 0.9$  with probability 0.9 and  $p = 0.1$  with probability 0.1). This expression is (up to lower order terms) equal to  $c(0.5 - w^t)$  if  $p = 0.9$  and  $-3c(0.5 - w^t)$  if  $p = 0.1$ , where  $c \approx 0.55$ . This means that an expected regret (relative to  $w = 0.5$ ) of  $0.6c(0.5 - w^t)$  (up to lower order terms) is incurred.
Let  $D$  be such that  $R''(w) \leq D$  for all  $w$  such that  $|w - 0.5| \leq \frac{\sqrt{T}}{4D}$ . (Such a  $D$  exists because  $R$  is continuously twice differentiable at 0.5.) If  $|w^t - 0.5| \geq \frac{\sqrt{T}}{4D}$ , we just showed that an expected regret (relative to  $w = 0.5$ ) of  $\Omega\left(\frac{\sqrt{T}}{4D}\right)$  is incurred. On the other hand, suppose that  $|w^t - 0.5| \leq \frac{\sqrt{T}}{4D}$ . We show that  $|w^{t+1} - 0.5| \geq \frac{\sqrt{T}}{4D}$ .
To see this, note that  $|L'(w^t)| \geq 0.5$ , we have that  $\left| R'(w^{t+1}) - R'(w^t) \right| \geq 0.5\eta$ . We also have that  $D\left| w^{t+1} - w^t \right| \geq \left| R'(w^{t+1}) - R'(w^t) \right|$ , so  $D\left| w^{t+1} - w^t \right| \geq 0.5\eta$ . Therefore,  $\left| w^{t+1} - w^t \right| \geq \frac{\eta}{2D} \geq \frac{\sqrt{T}}{2D}$ , which means that  $\left| w^{t+1} - 0.5 \right| \geq \frac{\sqrt{T}}{4D}$ .
This means that an expected regret (relative to  $w = 0.5$ ) of  $\Omega\left(\frac{\sqrt{T}}{4D}\right)$  is incurred on at least half of time steps. Since  $D$  is a constant, it follows that a total regret of at least  $\Omega(\sqrt{T})$  is incurred, as desired.