SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Let us first assume that  $(B_{a} - \ell_{a})\left(e_{B_{a}}^{\alpha / B_{a}} - 1\right) \geq \frac{1}{\alpha_{B}} e_{B_{a}}^{-\alpha \ell_{a} / B_{a}}$  such that  $\phi_{i}$  is decreasing in  $i$  which helps us to bound the linear combinations in Lemma 7 over  $\{I_{a} - B_{a} + 1, \dots, I_{a} - \ell_{a}\}$  and  $\{I_{a} - \ell_{a} + 1, \dots, T\}$  by the average values  $\bar{w}_{\Phi} \coloneqq \frac{1}{B_{a} - \ell_{a}} \sum_{i = I_{a} - B_{a} + 1}^{I_{a} - \ell_{a}} w_{at_{i}}$  and  $\bar{w}_{\Psi}$ , respectively. We further use that  $w_{atI_a - B_a} \leq \bar{w}_{\Phi}$  to charge mass from  $\Omega_{a}$  to  $\Phi_{a}$  and obtain due to (8) that
$$
\begin{array}{l} \sum_ {i = I _ {a} - B _ {a} + 1} ^ {I _ {a} - \ell_ {a}} w _ {a t _ {i}} \phi_ {i} + \sum_ {i = I _ {a} - \ell_ {a} + 1} ^ {I _ {a}} w _ {a t _ {i}} \psi_ {i} + w _ {a t I _ {a} - B _ {a}} \Omega_ {a} \\ \leq \bar {w} _ {\Phi} \Phi_ {a} + \bar {w} _ {\Psi} \Psi_ {a} + w _ {a t I _ {a} - B _ {a}} \Omega_ {a} (9) \\ \leq \bar {w} _ {\Phi} \left(\Phi_ {a} + \Omega_ {a}\right) + \bar {w} _ {\Psi} \Psi_ {a} \\ = \bar {w} _ {\Phi} \left(B _ {a} - \ell_ {a}\right) \tau_ {a} + \bar {w} _ {\Phi} \left(\Phi_ {a} + \Omega_ {a} - \left(B _ {a} - \ell_ {a}\right) \tau_ {a}\right) + \bar {w} _ {\Psi} \Psi_ {a} \\ = \sum_ {i = I _ {a} - B _ {a} + 1} ^ {I _ {a} - \ell_ {a}} \tau_ {a} w _ {a t _ {i}} + \bar {w} _ {\Phi} \left(\Phi_ {a} + \Omega_ {a} - \left(B _ {a} - \ell_ {a}\right) \tau_ {a}\right) + \bar {w} _ {\Psi} \Psi_ {a} (10) \\ \end{array}
$$
On the other hand, if  $(B_{a} - \ell_{a})\left(e_{B_{a}}^{\alpha / B_{a}} - 1\right) \leq \frac{1}{\alpha_{B}} e_{B_{a}}^{-\alpha \ell_{a} / B_{a}}$  we can no longer bound the values over  $\{I_{a} - B_{a} + 1, \ldots, I_{a} - \ell_{a}\}$  by the average value  $\bar{w}_{\Phi}$ . However, each factor  $\phi_{i}$  is less than  $\tau_{a}$  which can be seen by rearranging
$$
\begin{array}{l} \phi_ {i} = \frac {1}{e _ {B _ {a}} ^ {\alpha} - 1} \left((B _ {a} - \ell_ {a}) (e _ {B _ {a}} ^ {\alpha / B _ {a}} - 1) - \frac {1}{\alpha_ {B}} e _ {B _ {a}} ^ {- \alpha \ell_ {a} / B _ {a}}\right) e _ {B _ {a}} ^ {\alpha (I _ {a} - i) / B _ {a}} + \frac {1}{\alpha_ {B}} \frac {e _ {B _ {a}} ^ {\alpha}}{e _ {B _ {a}} ^ {\alpha} - 1} \\ \leq 1 + \frac {1}{e _ {B _ {a}} ^ {\alpha} - 1} \frac {1}{\alpha_ {B}} \left(e _ {B _ {a}} ^ {\alpha} - \frac {e _ {B _ {a}} ^ {\alpha} - 1}{\alpha_ {B _ {a}}}\right) = \tau_ {a} \\ \end{array}
$$
to the equivalent expression
$$
\begin{array}{l} \underbrace {\left((B _ {a} - \ell_ {a}) \left(e _ {B _ {a}} ^ {\alpha / B _ {a}} - 1\right) - \frac {1}{\alpha_ {B}} e _ {B _ {a}} ^ {- \alpha \ell_ {a} / B _ {a}}\right)} _ {<   0} \underbrace {e _ {B _ {a}} ^ {\alpha (I _ {a} - i) / B _ {a}}} _ {\geq 0} \\ \leq e _ {B _ {a}} ^ {\alpha} - 1 - \frac {1}{\alpha_ {B}} \frac {e _ {B _ {a}} ^ {\alpha} - 1}{\alpha_ {B _ {a}}} = \underbrace {\left(1 - \frac {1}{\alpha_ {B} \cdot a _ {B _ {a}}}\right)} _ {\geq 0} \underbrace {\left(e _ {B _ {a}} ^ {\alpha} - 1\right)} _ {\geq 0} \\ \end{array}
$$
which is true since the LHS is  $\leq 0$  and the RHS  $\geq 0$ . We can thus charge  $\tau_{a} - \phi_{i}$  of mass from  $\Omega_{a}$  to the coefficients  $\phi_{i}$  for each  $i \in \{I_{a} - B_{a} + 1, \dots, I_{a} - \ell_{a}\}$  which yields
$$
\begin{array}{l} \sum_ {i = I _ {a} - B _ {a} + 1} ^ {I _ {a} - \ell_ {a}} w _ {a t _ {i}} \phi_ {i} + \sum_ {i = I _ {a} - \ell_ {a} + 1} ^ {I _ {a}} w _ {a t _ {i}} \psi_ {i} + w _ {a t _ {I _ {a} - B _ {a}}} \Omega_ {a} \\ \leq \sum_ {i = I _ {a} - B _ {a} + 1} ^ {I _ {a} - \ell_ {a}} w _ {a t _ {i}} \phi_ {i} + \bar {w} _ {\Psi} \Psi_ {a} + w _ {a t _ {I _ {a} - B _ {a}}} \Omega_ {a} \\ = \sum_ {i = I _ {a} - B _ {a} + 1} ^ {I _ {a} - \ell_ {a}} \tau_ {a} w _ {a t _ {i}} - \sum_ {i = I _ {a} - B _ {a} + 1} ^ {I _ {a} - \ell_ {a}} w _ {a t _ {i}} \underbrace {\left(\tau_ {a} - \phi_ {i}\right)} _ {\geq 0} + \bar {w} _ {\Psi} \Psi_ {a} + w _ {a t _ {I _ {a} - B _ {a}}} \Omega_ {a} \\ \leq \sum_ {i = I _ {a} - B _ {a} + 1} ^ {I _ {a} - \ell_ {a}} \tau_ {a} w _ {a t _ {i}} - \sum_ {i = I _ {a} - B _ {a} + 1} ^ {I _ {a} - \ell_ {a}} w _ {a t _ {I _ {a} - B _ {a}}} \left(\tau_ {a} - \phi_ {i}\right) + \bar {w} _ {\Psi} \Psi_ {a} + w _ {a t _ {I _ {a} - B _ {a}}} \Omega_ {a} \\ = \sum_ {i = I _ {a} - B _ {a} + 1} ^ {I _ {a} - \ell_ {a}} \tau_ {a} w _ {a t _ {i}} + w _ {a t _ {I _ {a} - B _ {a}}} \left(\Phi_ {a} + \Omega_ {a} - \left(B _ {a} - \ell_ {a}\right) \tau_ {a}\right) + \bar {w} _ {\Psi} \Psi_ {a} \tag {11} \\ \end{array}
$$
In both cases (10) and (11), we have shown that
$$
\begin{array}{l} \sum_ {i = I _ {a} - B _ {a} + 1} ^ {I _ {a} - \ell_ {a}} w _ {a t _ {i}} \phi_ {i} + \sum_ {i = I _ {a} - \ell_ {a} + 1} ^ {I _ {a}} w _ {a t _ {i}} \psi_ {i} + w _ {a t _ {I _ {a} - B _ {a}}} \Omega_ {a} \\ \leq \sum_ {i = I _ {a} - B _ {a} + 1} ^ {I _ {a} - \ell_ {a}} \tau_ {a} w _ {a t _ {i}} + v \left(\Phi_ {a} + \Omega_ {a} - \left(B _ {a} - \ell_ {a}\right) \tau_ {a}\right) + \bar {w} _ {\Psi} \Psi_ {a} \\ \end{array}
$$
for a  $v \leq \bar{w}_{\Psi}$ . If  $\ell_a > 0$ , we can use  $v \leq \bar{w}_{\Psi}$  to charge the remaining mass to  $\Psi_a$  if the factors over  $\{I_a - \ell_a + 1, \dots, T\}$  leave enough space. In symbols, this means
$$
\begin{array}{l} \sum_ {i = T - B _ {a} + 1} ^ {I _ {a} - \ell_ {a}} \tau_ {a} w _ {a t _ {i}} + v \left(\Phi_ {a} + \Omega_ {a} - \left(B _ {a} - \ell_ {a}\right) \tau_ {a}\right) + \bar {w} _ {\Psi} \Psi_ {a} \\ \leq \sum_ {i = T - B _ {a} + 1} ^ {I _ {a} - \ell_ {a}} \tau_ {a} w _ {a t _ {i}} + \bar {w} _ {\Psi} \max  \left\{\Phi_ {a} + \Omega_ {a} - \left(B _ {a} - \ell_ {a}\right) \tau_ {a}, 0 \right\} + \bar {w} _ {\Psi} \Psi_ {a} \\ = \sum_ {i = T - B _ {a} + 1} ^ {I _ {a} - \ell_ {a}} \tau_ {a} w _ {a t _ {i}} + \bar {w} _ {\Psi} \max  \left\{\Phi_ {a} + \Psi_ {a} + \Omega_ {a} - \left(B _ {a} - \ell_ {a}\right) \tau_ {a}, \Psi_ {a} \right\} \\ = \sum_ {i = T - B _ {a} + 1} ^ {I _ {a} - \ell_ {a}} \tau_ {a} w _ {a t _ {i}} + \bar {w} _ {\Psi} \max  \left\{\ell_ {a} \tau_ {a}, \Psi_ {a} \right\} \\ \end{array}
$$
$$
\begin{array}{l} \leq \tau_ {a} \sum_ {i = T - B _ {a} + 1} ^ {I _ {a} - \ell_ {a}} w _ {a t _ {i}} + \max  \left\{\tau_ {a}, \frac {\Psi_ {a}}{\ell_ {a}} \right\} \sum_ {i = T - \ell_ {a} + 1} ^ {I _ {a}} w _ {a t _ {i}} \\ \leq \max  \left\{\tau_ {a}, \frac {\Psi_ {a}}{\ell_ {a}} \right\} \sum_ {t \in S _ {a}} w _ {a t}. \\ \end{array}
$$
If  $\ell_{a} = 0$ , we have  $\Psi_{a} = 0$  and immediately obtain by definition of  $\tau_{a}$  that
$$
\sum_ {i = I _ {a} - B _ {a} + 1} ^ {I _ {a} - \ell_ {a}} \tau_ {a} w _ {a t _ {i}} + v \left(\Phi_ {a} + \Omega_ {a} - \left(B _ {a} - \ell_ {a}\right) \tau_ {a}\right) + \bar {w} _ {\Psi} \Psi_ {a} = \sum_ {i = I _ {a} - B _ {a} + 1} ^ {I _ {a} - \ell_ {a}} \tau_ {a} w _ {a t _ {i}}.
$$
Lemma 10. We have
$$
\frac {\Psi_ {a}}{\ell_ {a}} = 1 + \left(\frac {B _ {a}}{\ell_ {a}} - 1\right) \frac {e _ {B _ {a}} ^ {\alpha \ell_ {a} / B _ {a}} - 1}{e _ {B _ {a}} ^ {\alpha} - 1}
$$
$$
\tau_ {a} = 1 + \frac {1}{e _ {B _ {a}} ^ {\alpha} - 1} \frac {1}{\alpha_ {B}} \left(e _ {B _ {a}} ^ {\alpha} - \frac {e _ {B _ {a}} ^ {\alpha} - 1}{\alpha_ {B _ {a}}}\right).
$$
Proof. We compute
$$
\begin{array}{l} \Phi_ {a} = \sum_ {i = T - B _ {a} + 1} ^ {T - \ell_ {a}} \phi_ {i} \\ = \sum_ {i = T - B _ {a} + 1} ^ {T - \ell_ {a}} \left(\frac {1}{e _ {B _ {a}} ^ {\alpha} - 1} \left((B _ {a} - \ell_ {a}) \left(e _ {B _ {a}} ^ {\alpha / B _ {a}} - 1\right) - \frac {1}{\alpha_ {B}} e _ {B _ {a}} ^ {- \alpha \ell_ {a} / B _ {a}}\right) e _ {B _ {a}} ^ {\alpha (T - i) / B _ {a}} + \frac {1}{\alpha_ {B}} \frac {e _ {B _ {a}} ^ {\alpha}}{e _ {B _ {a}} ^ {\alpha} - 1}\right) \\ = \frac {1}{e _ {B _ {a}} ^ {\alpha} - 1} \left((B _ {a} - \ell_ {a}) \left(e _ {B _ {a}} ^ {\alpha / B _ {a}} - 1\right) - \frac {1}{\alpha_ {B}} e _ {B _ {a}} ^ {- \alpha \ell_ {a} / B _ {a}}\right) \frac {e _ {B _ {a}} ^ {\alpha} - e _ {B _ {a}} ^ {\alpha \ell_ {a} / B _ {a}}}{e _ {B _ {a}} ^ {\alpha / B _ {a}} - 1} + (B _ {a} - \ell_ {a}) \frac {1}{\alpha_ {B}} \frac {e _ {B _ {a}} ^ {\alpha}}{e _ {B _ {a}} ^ {\alpha} - 1} \\ = \frac {1}{e _ {B _ {a}} ^ {\alpha} - 1} \left(B _ {a} - \ell_ {a}\right) \left(e _ {B _ {a}} ^ {\alpha} - e _ {B _ {a}} ^ {\alpha \ell_ {a} / B _ {a}} + \frac {1}{\alpha B} e _ {B _ {a}} ^ {\alpha}\right) - \frac {1}{e _ {B _ {a}} ^ {\alpha} - 1} \frac {1}{\alpha B} \frac {e _ {B _ {a}} ^ {\alpha - \alpha \ell_ {a} / B _ {a}} - 1}{e _ {B _ {a}} ^ {\alpha / B _ {a}} - 1} \\ \end{array}
$$
$$
\begin{array}{l} \Psi_ {a} = \sum_ {i = T - \ell_ {a} + 1} ^ {T} \psi_ {i} \\ = \sum_ {i = T - \ell_ {a} + 1} ^ {T} \left(1 + \left(B _ {a} - \ell_ {a}\right) \frac {e _ {B _ {a}} ^ {\alpha / B _ {a}} - 1}{e _ {B _ {a}} ^ {\alpha} - 1} e _ {B _ {a}} ^ {\alpha (T - i) / B _ {a}}\right) \\ = \ell_ {a} + (B _ {a} - \ell_ {a}) \frac {e _ {B _ {a}} ^ {\alpha / B _ {a}} - 1}{e _ {B _ {a}} ^ {\alpha} - 1} \frac {e _ {B _ {a}} ^ {\alpha \ell_ {a} / B _ {a}} - 1}{e _ {B _ {a}} ^ {\alpha / B _ {a}} - 1} \\ = \ell_ {a} + (B _ {a} - \ell_ {a}) \frac {e _ {B _ {a}} ^ {\alpha \ell_ {a} / B _ {a}} - 1}{e _ {B _ {a}} ^ {\alpha} - 1}. \\ \end{array}
$$
Summing up,
$$
\begin{array}{l} \Phi_ {a} + \Psi_ {a} + \Omega_ {a} \\ = \frac {1}{e ^ {\alpha} - 1} \left(B _ {a} - \ell_ {a}\right) \left(e _ {B _ {a}} ^ {\alpha} - e _ {B _ {a}} ^ {\alpha \ell_ {a} / B _ {a}} + \frac {1}{\alpha_ {B}} e _ {B _ {a}} ^ {\alpha}\right) - \frac {1}{e _ {B _ {a}} ^ {\alpha} - 1} \frac {1}{\alpha_ {B}} \frac {e _ {B _ {a}} ^ {\alpha - \alpha \ell_ {a} / B _ {a}} - 1}{e _ {B _ {a}} ^ {\alpha / B _ {a}} - 1} \\ + \ell_ {a} + (B _ {a} - \ell_ {a}) \frac {e _ {B _ {a}} ^ {\alpha \ell_ {a} / B _ {a}} - 1}{e _ {B _ {a}} ^ {\alpha} - 1} + \frac {1}{e _ {B _ {a}} ^ {\alpha} - 1} \frac {1}{\alpha_ {B}} \left(\ell_ {a} e _ {B _ {a}} ^ {\alpha} - \frac {e _ {B _ {a}} ^ {\alpha} - e _ {B _ {a}} ^ {\alpha (B _ {a} - \ell_ {a}) / B _ {a}}}{e _ {B _ {a}} ^ {\alpha / B _ {a}} - 1}\right) \\ \end{array}
$$
$$
\begin{array}{l} = \frac {1}{e _ {B _ {a}} ^ {\alpha} - 1} (B _ {a} - \ell_ {a}) \left(e _ {B _ {a}} ^ {\alpha} - 1 + \frac {1}{\alpha_ {B}} e _ {B _ {a}} ^ {\alpha}\right) - \frac {1}{e _ {B _ {a}} ^ {\alpha} - 1} \frac {1}{\alpha_ {B}} \frac {e _ {B _ {a}} ^ {\alpha} - 1}{e _ {B _ {a}} ^ {\alpha / B _ {a}} - 1} \\ + \ell_ {a} + \frac {1}{e _ {B _ {a}} ^ {\alpha} - 1} \frac {1}{\alpha_ {B}} \ell_ {a} e _ {B _ {a}} ^ {\alpha} \\ = B _ {a} + \frac {1}{e _ {B _ {a}} ^ {\alpha} - 1} \frac {1}{\alpha_ {B}} (B _ {a} - \ell_ {a}) e _ {B _ {a}} ^ {\alpha} - \frac {1}{e _ {B _ {a}} ^ {\alpha} - 1} \frac {1}{\alpha_ {B}} \frac {e _ {B _ {a}} ^ {\alpha} - 1}{e _ {B _ {a}} ^ {\alpha / B _ {a}} - 1} + \frac {1}{e _ {B _ {a}} ^ {\alpha} - 1} \frac {1}{\alpha_ {B}} \ell_ {a} e _ {B _ {a}} ^ {\alpha} \\ = B _ {a} + \frac {1}{e _ {B _ {a}} ^ {\alpha} - 1} \frac {1}{\alpha_ {B}} B _ {a} e _ {B _ {a}} ^ {\alpha} - \frac {1}{e _ {B _ {a}} ^ {\alpha} - 1} \frac {1}{\alpha_ {B}} \frac {e _ {B _ {a}} ^ {\alpha} - 1}{e _ {B _ {a}} ^ {\alpha / B _ {a}} - 1} \\ = B _ {a} + \frac {1}{e _ {B _ {a}} ^ {\alpha} - 1} \frac {1}{\alpha_ {B}} \left(B _ {a} e _ {B _ {a}} ^ {\alpha} - \frac {e _ {B _ {a}} ^ {\alpha} - 1}{e _ {B _ {a}} ^ {\alpha / B _ {a}} - 1}\right) \\ \end{array}
$$
which does no longer depend on  $\ell_{a}$ . Dividing  $\Psi_{a}$  by  $\ell_{a}$  and  $\Phi_{a} + \Psi_{a} + \Omega_{a}$  by  $B_{a}$  yields the result.
Putting everything together, we have PRD/ALG  $\leq \max_{a}\max \left\{\tau_{a},\max_{\ell_{a}\in \{1,\dots,B_{a}\}}\Psi_{a} / \ell_{a}\right\}$  as  $\tau_{a}$  does not depend on  $\ell_{a}$ . The reader can refer back to Figure 2 for an illustration of this upper bound. In the following lemma, we further analyze analytically  $\max_{\ell_a\in \{1,\ldots ,B_a\}}\Psi_a / \ell_a$  and compare it with  $\tau_{a}$  to obtain the upper bound:
Lemma 11. The consistency of Algorithm 1 is given by
$$
\mathrm {P R D} / \mathrm {A L G} \leq \left(1 + \frac {1}{e _ {B} ^ {\alpha} - 1} \max  \left\{\frac {1}{\alpha_ {B}} \left(e _ {B} ^ {\alpha} - \frac {e _ {B} ^ {\alpha} - 1}{\alpha_ {B}}\right), \ln \left(e _ {B} ^ {\alpha}\right) \right\}\right).
$$
Proof. Due to Lemma 9, it is sufficient to show
$$
1 + \frac {1}{e _ {B} ^ {\alpha} - 1} \max  \left\{\frac {1}{\alpha_ {B}} \left(e _ {B} ^ {\alpha} - \frac {e _ {B} ^ {\alpha} - 1}{\alpha_ {B}}\right), \ln \left(e _ {B} ^ {\alpha}\right) \right\} \geq \left\{ \begin{array}{l l} \max  \left\{\tau_ {a}, \Psi_ {a} / \ell_ {a} \right\} & \text {i f} \ell_ {a} > 0 \\ \tau_ {a} & \text {o t h e r w i s e}. \end{array} \right.
$$
By Lemma 10, we know for the first term in the maximum that
This term is maximized for  $B_{a} = B$  since
$$
\begin{array}{l} 1 + \frac {1}{e _ {B _ {a}} ^ {\alpha} - 1} \frac {1}{\alpha_ {B}} \left(e _ {B _ {a}} ^ {\alpha} - \frac {e _ {B _ {a}} ^ {\alpha} - 1}{B _ {a} \left(e _ {B _ {a}} ^ {\alpha / B _ {a}} - 1\right)}\right) = 1 + \frac {1}{\alpha_ {B}} \left(\frac {e _ {B _ {a}} ^ {\alpha}}{e _ {B _ {a}} ^ {\alpha} - 1} - \frac {1}{\alpha_ {B _ {a}}}\right) \\ \leq 1 + \frac {1}{\alpha_ {B}} \left(\frac {e _ {B} ^ {\alpha}}{e _ {B} ^ {\alpha} - 1} - \frac {1}{\alpha_ {B}}\right) = 1 + \frac {1}{e _ {B} ^ {\alpha} - 1} \underbrace {\frac {1}{\alpha_ {B}} \left(e _ {B} ^ {\alpha} - \frac {e _ {B} ^ {\alpha} - 1}{\alpha_ {B}}\right)} _ {=: p (\alpha)} \\ \end{array}
$$
due to Lemma 2. The lemma statement therefore follows immediately if  $\ell_{a} = 0$ . We may thus assume that  $\ell_{a} > 0$  and use Lemma 10 to determine the second term in the maximum as
$$
\frac {\Phi_ {a}}{\ell_ {a}} = 1 + \frac {1}{e _ {B _ {a}} ^ {\alpha} - 1} \left(\frac {1}{x} - 1\right) \left(e _ {B _ {a}} ^ {\alpha x} - 1\right)
$$
where  $x \eqqcolon \ell_a / B_a$ . The second term behaves similarly to the first as
$$
\frac {\Phi_ {a}}{\ell_ {a}} = 1 + \frac {1}{e _ {B _ {a}} ^ {\alpha} - 1} \left(\frac {1}{x} - 1\right) \left(e _ {B _ {a}} ^ {\alpha x} - 1\right) \leq 1 + \frac {1}{e _ {B} ^ {\alpha} - 1} \left(\frac {1}{x} - 1\right) \left(e _ {B} ^ {\alpha x} - 1\right)
$$
since  $\left(e_{B_a}^{\alpha x} - 1\right) / \left(e_{B_a}^{\alpha}\right)\leq \left(e_B^{\alpha x} - 1\right) / \left(e_B^{\alpha} - 1\right)$ . We define  $g(\alpha ,x)\coloneqq \left(\frac{1}{x} -1\right)(e_B^{\alpha x} - 1)$  such that we can write
$$
\max  \left\{\frac {\Phi_ {a} + \Psi_ {a} + \Omega_ {a}}{B _ {a}}, \frac {\Psi_ {a}}{\ell_ {a}} \right\} \leq 1 + \frac {1}{e _ {B} ^ {\alpha} - 1} \max  \left\{p (\alpha), g (\alpha , x) \right\}.
$$
We want to remove the dependency on  $x$  in  $g$  by maximizing  $g$  over  $x \in [0,1]$  for a fixed  $\alpha$ . As  $g(\alpha, x)$  is continuous, it suffices to evaluate  $g$  in both endpoints and find the stationary points. We have
$$
\begin{array}{l} g (\alpha , 0) = \lim  _ {x \rightarrow 0} \left(\frac {1}{x} - 1\right)\left(e _ {B} ^ {\alpha x} - 1\right) = \lim  _ {x \rightarrow 0} \frac {(1 - x) \left(e _ {B} ^ {\alpha x} - 1\right)}{x} \\ = \lim _ {x \to 0} - (e _ {B} ^ {\alpha x} - 1) + (1 - x) \ln (e _ {B} ^ {\alpha}) e _ {B} ^ {\alpha x} = \ln (e _ {B} ^ {\alpha}) \\ \end{array}
$$
by L'Hoptial. Further,  $g(\alpha, 1) = 0$ . Next, we find the stationary points  $x^{*} \in [0,1]$  as solutions to the equation
$$
\frac {\partial}{\partial x} g (\alpha , x ^ {*}) = \ln (e _ {B} ^ {\alpha}) \left(\frac {1}{x ^ {*}} - 1\right) e _ {B} ^ {\alpha x ^ {*}} - \frac {e _ {B} ^ {\alpha x ^ {*}} - 1}{(x ^ {*}) ^ {2}} = 0
$$
$$
e _ {B} ^ {\alpha x ^ {*}} - 1 = \ln \left(e _ {B} ^ {\alpha}\right) \left(x ^ {*}\right) ^ {2} \left(\frac {1}{x ^ {*}} - 1\right) e _ {B} ^ {\alpha x ^ {*}}.
$$
There is no closed form solution for  $x^{*}$ , but we can replace  $e_{B}^{\alpha x} - 1$  in  $g$  with the RHS of the above. This yields a new function
$$
h (\alpha , y) = \left(\frac {1}{y} - 1\right) \ln (e _ {B} ^ {\alpha}) y ^ {2} \left(\frac {1}{y} - 1\right) e _ {B} ^ {\alpha y} = \ln (e _ {B} ^ {\alpha}) \left(1 - y\right) ^ {2} e _ {B} ^ {\alpha y}
$$
with  $h(\alpha, x^*) = g(\alpha, x^*)$ . We can thus maximize  $h$  over  $y \in [0,1]$  to obtain an upper bound on  $g(x^*)$ . Note that  $h(\alpha, 0) = \ln(e_B^\alpha) = g(\alpha, 0)$  and  $h(\alpha, 1) = 0 = g(\alpha, 1)$ . To this end, let  $y^*$  be such that
$$
\frac {\partial}{\partial y ^ {*}} h (\alpha , y ^ {*}) = \ln (e _ {B} ^ {\alpha}) ^ {2} (1 - y ^ {*}) ^ {2} e _ {B} ^ {\alpha y ^ {*}} - 2 \ln (e _ {B} ^ {\alpha}) (1 - y ^ {*}) e _ {B} ^ {\alpha y ^ {*}} = 0
$$
which is equivalent to  $\ln (e_B^\alpha)(1 - y^*) - 2 = 0$  or  $y^{*} = 1 - \frac{2}{\ln(e_{B}^{\alpha})}$ . We evaluate  $h$  in  $y^{*}$  and obtain
$$
h (\alpha , y ^ {*}) = \alpha \ln (e _ {B}) \left(\frac {2}{\alpha \ln (e _ {B})}\right) ^ {2} e _ {B} ^ {\alpha - \frac {2}{\ln (e _ {B})}} = \frac {4}{\alpha \ln (e _ {B}) e ^ {2}} e _ {B} ^ {\alpha}.
$$
Note that  $y^{*}\geq 0\iff \alpha \geq 2 / \ln (e_{B})$  . Furthermore,  $h^{\ast}(\alpha)$  always exceeds the endpoint  $g(\alpha ,0)$  We calculate
$$
\begin{array}{l} h ^ {*} (\alpha) := h (\alpha , y ^ {*}) = \frac {4}{\ln (e _ {B} ^ {\alpha}) e ^ {2}} e _ {B} ^ {\alpha} \\ \geq \frac {4}{\ln (e _ {B} ^ {\alpha}) e ^ {2}} e ^ {2} \ln (e _ {B} ^ {\alpha / 2}) ^ {2} \\ = \ln (e _ {B} ^ {\alpha}) \\ \end{array}
$$
where the inequality is due to  $e^z \geq ez$  for  $z = \ln \left(e_B^{\alpha /2}\right) \geq 0$ . Therefore, for all  $x \in [0,1]$ ,
$$
g (\alpha , x) \leq \left\{ \begin{array}{l l} \ln (e _ {B} ^ {\alpha}) & \text {i f} \alpha \leq \frac {2}{\ln (e _ {B})} \\ h ^ {*} (\alpha) & \text {o t h e r w i s e .} \end{array} \right.
$$
We consider both intervals separately. Let us first consider the case when  $\alpha \in \left[0, \frac{2}{\ln(e_B)}\right]$ . If  $B < \infty$ , there could be multiple intersection points between  $p(\alpha)$  and  $\alpha \ln(e_B)$ . However, the situation is easier if  $B \to \infty$  as the intersection points given by
$$
p (\alpha) = \frac {1}{\alpha} \left(e ^ {\alpha} - \frac {e ^ {\alpha} - 1}{\alpha}\right) = \alpha \iff \alpha e ^ {\alpha} - e ^ {\alpha} + 1 = \alpha^ {3}
$$
are at  $\alpha = 1$  and  $\alpha^{*}\approx 1.79$ , whereas  $\alpha \ln (e_B)$  dominates  $p(\alpha)$  between 1 and  $\alpha^{*}$ .
It remains to consider the case  $\alpha \geq \frac{2}{\ln(e_{B_a})}$ . Again, there can be many intersection points of  $p(\alpha)$  with  $\alpha \ln (e_{B_a})$  and  $h^* (\alpha)$ . However, if  $B\to \infty$ , then  $p(\alpha)$  already dominates  $h^{*}(\alpha)$  for  $\alpha >2$  which we can see as follows. First,
$$
h ^ {*} (\alpha) = \frac {4}{\alpha} e ^ {\alpha - 2} \leq \frac {1}{\alpha} \left(e ^ {\alpha} - \frac {e ^ {\alpha} - 1}{\alpha}\right) = p (\alpha)
$$
$$
\Longleftrightarrow 4 e ^ {- 2} \leq 1 - \frac {1 - e ^ {- \alpha}}{\alpha}.
$$
We can see that  $\frac{1 - e^{-\alpha}}{\alpha}$  is decreasing in  $\alpha$  as
$$
\frac {\partial}{\partial \alpha} \frac {1 - e ^ {- \alpha}}{\alpha} = \frac {e ^ {- \alpha} (\alpha - e ^ {\alpha} + 1)}{\alpha^ {2}} \leq 0
$$
which holds as  $1 + \alpha \leq e^{\alpha}$ . Finally, we check that  $h^{*}(2) = 2\leq 2.10\approx p(2)$ .
B Generalized Assignment Problem
The generalized assignment problem (GAP) is a generalization of Display Ads where impressions  $t$  can take up any size  $u_{at}$  in the budget constraint of advertiser  $a$ . This formulation encompasses both Display Ads and Ad Words, and we empirically compare it to the Ad Words algorithm with predictions due to Mahdian et al. (2007) in Section C.3. For simplicity of presentation, we assume that budgets are all 1 and instead,  $u_{at} \rightarrow 0$ . However, as before it is possible to adapt the algorithm to work with large sizes  $u_{at}$ . We state the LP below.
GAP Primal
$$
\forall a \colon \sum_ {t} u _ {a t} x _ {a t} \leq 1
$$
GAP Dual
$$
\min  \sum_ {a} \beta_ {a} + \sum_ {t} z _ {t}
$$
$$
\forall a, t: z _ {t} \geq w _ {a t} - u _ {a t} \beta_ {a}
$$
Algorithm 2 is a generalization of Algorithm 1 to GAP. An immediate difference is that the discounted gain  $w_{at} - u_{at}\beta_{a}$  respects the impression size  $u_{at}$  in accordance with the changed dual constraint. We still follow the predicted advertiser if its discounted gain still is a sufficiently high fraction of the maximum discounted gain. However, we might now have to remove multiple impressions with least value-size ratio to accommodate the new impression. The update for  $\beta_{a}$  also differs and is based on value-size ratios of impressions allocated to  $a$ : For a fixed advertiser  $a$  let  $U_{a} = \sum_{t\in \mathbf{X}_{a}}u_{at}$ , be the total size of all impressions ever allocated to  $a$ . For any  $x\in (0,U_a]$  define  $\frac{w_x}{u_x}$  as the minimal ratio such that
$$
\sum_ {t \in \mathbf {X} _ {a}: \frac {w _ {a t}}{u _ {a t}} \leq \frac {w _ {x}}{u _ {x}}} u _ {a t} > x. \tag {12}
$$
Then, we can naturally define  $\beta_{a}$  as the exponential average over ratios  $\frac{w_x}{u_x}$ . As before, we also assume that there exists a dummy advertiser that only receives impressions of zero value-size ratio and that all advertisers are initially filled up with impressions of zero value.
B.1 Robustness
Theorem 12. Algorithm 1 has a robustness of
$$
\frac {\mathrm {A L G}}{\mathrm {O P T}} \geq \frac {e ^ {\alpha} - 1}{\alpha e ^ {\alpha}}
$$
Proof. Assume we assign impression  $t$  to advertiser  $a$  while disposing of some impressions to make space. We will bound the dual increase as a multiple of the primal increase. We now assume that after allocating  $t$  to  $a$ , it becomes the impression with highest value-size ratio (a general proof follows analogously to the proof of robustness for Display Ads in Section A.1). The primal increase is simply
$$
\Delta P = \int_ {U _ {a} - u _ {a t}} ^ {U _ {a}} \frac {w _ {x}}{u _ {x}} d x - \int_ {U _ {a} - 1 - u _ {a t}} ^ {U _ {a} - 1} \frac {w _ {x}}{u _ {x}} d x = w _ {a t} - \int_ {U _ {a} - 1 - u _ {a t}} ^ {U _ {a} - 1} \frac {w _ {x}}{u _ {x}} d x.
$$
Algorithm 2 Exponential Averaging with Predictions for GAP
1: Input: Robustness-consistency trade-off parameter  $\alpha \in [1,\infty)$
2: For each advertiser  $a$ , initialize  $\beta_{a} \gets 0$  and fill up  $a$  with zero-value impressions
3: for all arriving impressions  $t$  do
4:  $a_{\mathrm{(PRD)}} \gets \mathrm{PRD}(t)$
5:  $a_{(\mathrm{EXP})}\gets \arg \max_{a}\{w_{at} - u_{at}\beta_{a}\}$
6: if  $\alpha\left(w_{a_{(\mathrm{PRD})},t}-u_{a_{(\mathrm{PRD})},t} \beta_{a_{(\mathrm{PRD})}}\right) \geq w_{a_{(\mathrm{EXP})},t}-u_{a_{(\mathrm{EXP})},t} \beta_{a_{(\mathrm{EXP})}}$  then
7:  $a\gets a_{(\mathrm{PRD})}$
8: else
9:  $a\gets a_{(\mathrm{EXP})}$
10: end if
11: Dispose of impressions with least value-size ratio currently allocated to  $a$  until there is  $u_{at}$  of free space and allocate  $t$  to  $a$
12: Let  $\frac{w_x}{u_x}$  as in (12) and update  $\beta_{a} \gets \frac{\alpha}{e^{\alpha} - 1} \int_{U_{a} - 1}^{U_{a}} \frac{w_{x}}{u_{x}} e^{\alpha (U_{a} - x)} dx$
13: end for
At the same time,
$$
\begin{array}{l} \beta_ {a} ^ {(t)} = \frac {\alpha}{e ^ {\alpha} - 1} \int_ {U _ {a} - 1} ^ {U _ {a}} \frac {w _ {x}}{u _ {x}} e ^ {\alpha (U _ {a} - x)} d x \\ = \frac {\alpha}{e ^ {\alpha} - 1} \left(\int_ {U _ {a} - 1 - u _ {a t}} ^ {U _ {a} - u _ {a t}} \frac {w _ {x}}{u _ {x}} e ^ {\alpha (U _ {a} - x)} d x + \int_ {U _ {a} - u _ {a t}} ^ {U _ {a}} \frac {w _ {x}}{u _ {x}} e ^ {\alpha (U _ {a} - x)} d x - \int_ {U _ {a} - 1 - u _ {a t}} ^ {U _ {a} - 1} \frac {w _ {x}}{u _ {x}} e ^ {\alpha (U _ {a} - x)} d x\right) \\ = \frac {\alpha}{e ^ {\alpha} - 1} \left(e ^ {\alpha u _ {a t}} \int_ {U _ {a} - 1 - u _ {a t}} ^ {U _ {a} - u _ {a t}} \frac {w _ {x}}{u _ {x}} e ^ {\alpha (U _ {a} - x - u _ {a t})} d x + w _ {a t} - \int_ {U _ {a} - 1 - u _ {a t}} ^ {U _ {a} - 1} \frac {w _ {x}}{u _ {x}} e ^ {\alpha (U _ {a} - x)} d x\right) \\ = e ^ {\alpha u _ {a t}} \beta_ {a} ^ {(t - 1)} + \frac {\alpha}{e ^ {\alpha} - 1} \left(w _ {a t} - \int_ {U _ {a} - 1 - u _ {a t}} ^ {U _ {a} - 1} \frac {w _ {x}}{u _ {x}} e ^ {\alpha (U _ {a} - x)} d x\right) \\ \end{array}
$$
We set  $z_{t} = \alpha \left(w_{at} - u_{at}\beta_{a}^{(t - 1)}\right)$  and obtain, since  $e^{\alpha u_{at}} - 1 = \alpha u_{at}$  due to  $u_{at} \to 0$ ,