SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
\begin{array}{l} H _ {\beta} = - \psi_ {\beta} (\mathring {\mathbf {x}}) \\ = \frac {1}{T} \sum_ {t = 1} ^ {T} \psi_ {\beta} (\mathring {\mathbf {x}} _ {t}) - \psi_ {\beta} (\mathring {\mathbf {x}}) \\ = \frac {1}{T} \sum_ {t = 1} ^ {T} B _ {\beta} (\mathring {\mathbf {x}} _ {t} | | \mathring {\bar {\mathbf {x}}}) \\ = \min  _ {\mathbf {x} \in \triangle_ {d}} \frac {1}{T} \sum_ {t = 1} ^ {T} B _ {\beta} (\stackrel {\circ} {\mathbf {x}} _ {t} | | \stackrel {\circ} {\mathbf {x}}) \\ \leq \min  _ {\delta \in (0, 1)} \frac {1}{T} \sum_ {t = 1} ^ {T} B _ {\beta} \left(\mathring {\mathbf {x}} _ {t} \left\| \frac {1 - \delta}{s} \mathbf {e} _ {S} + \frac {\delta}{d} \mathbf {1} _ {d}\right) \right. \\ = \min  _ {\delta \in (0, 1)} \frac {1}{T} \sum_ {t = 1} ^ {T} \frac {1}{1 - \beta} \sum_ {a = 1} ^ {d} \left(\frac {1 - \delta}{s} 1 _ {a \in S} + \frac {\delta}{d}\right) ^ {\beta} - \tilde {\mathbf {x}} _ {t [ a ]} ^ {\beta} + \frac {\beta \left(\stackrel {\circ} {\mathbf {x}} _ {t [ a ]} - \frac {1 - \delta}{s} 1 _ {a \in S} - \frac {\delta}{d}\right)}{\left(\frac {1 - \delta}{s} 1 _ {s \in S} + \frac {\delta}{d}\right) ^ {\beta}} \tag {81} \\ = \min  _ {\delta \in (0, 1)} \frac {1}{T} \sum_ {t = 1} ^ {T} \sum_ {a = 1} ^ {d} \left(\frac {1 - \delta}{s} 1 _ {a \in S} + \frac {\delta}{d}\right) ^ {\beta} - \frac {\mathring {\mathbf {x}} _ {t [ a ]} ^ {\beta}}{1 - \beta} + \frac {\beta \mathring {\mathbf {x}} _ {t [ a ]} ^ {\beta}}{(1 - \beta) (\frac {1 - \delta}{s} 1 _ {a \in S} + \frac {\delta}{d}) ^ {1 - \beta}} \\ \leq \min  _ {\delta \in (0, 1)} s ^ {1 - \beta} + \delta^ {\beta} d ^ {1 - \beta} + \frac {\beta}{(1 - \beta) T} \sum_ {t = 1} ^ {T} \sum_ {a = 1} ^ {d} \frac {1 _ {a = \hat {a} _ {t}}}{(1 - \beta) \left(\frac {1 - \delta}{s} 1 _ {a \in S} + \frac {\delta}{d}\right) ^ {1 - \beta}} \\ \leq \min  _ {\delta \in (0, 1)} \frac {s ^ {1 - \beta}}{1 - \beta} + \delta^ {\beta} d ^ {1 - \beta} + \mathcal {O} \left(\frac {\beta \left(\frac {d}{\delta}\right) ^ {1 - \beta}}{(1 - \beta) T ^ {1 - p}}\right) \\ = \mathcal {O} \left(s + \frac {d ^ {1 - \beta}}{T ^ {\beta (1 - p)}}\right) \\ \end{array}
$$
where the last line follows by considering  $\delta = 1 / T^{1 - p}$
E Online learning with self-concordant barrier regularizers
E.1 General results
Lemma E.1. Let  $\mathcal{K} \subset \mathbb{R}^d$  be a convex set and  $\psi: \mathcal{K}^\circ \mapsto \mathbb{R}^d$  be a self-concordant barrier. Suppose  $\ell_1, \ldots, \ell_T$  are a sequence of loss functions satisfying  $|\langle \ell_t, \mathbf{x} \rangle| \leq 1 \forall \mathbf{x} \in \mathcal{K}$ . Then if we run OMD with step-size  $\eta > 0$  as in Abernethy et al. [2, Alg. 1] on the sequence of estimators  $\hat{\ell}_t$  our estimated regret w.r.t. any  $\mathbf{x} \in \mathcal{K}_{\varepsilon}$  for  $\varepsilon > 0$  will satisfy
$$
\sum_ {t = 1} ^ {T} \left\langle \hat {\ell} _ {t}, \mathbf {x} _ {t} - \mathbf {x} \right\rangle \leq \frac {B (\mathbf {x} | | \mathbf {x} _ {1})}{\eta} + 3 2 d ^ {2} \eta T \tag {82}
$$
Proof. The result follows from Abernethy et al. [2] by stopping the derivation on the second inequality below Equation 10.  $\square$
Definition E.1. For any convex set  $\mathcal{K}$  and any point  $\mathbf{y} \in \mathcal{K}$ ,  $\pi_{\mathbf{y}}(\mathbf{x}) = \inf_{t \geq 0, \mathbf{y} + \frac{\mathbf{x} - \mathbf{y}}{t} \in \mathcal{K}} t$  is the
Minkowski function with pole y.
Lemma E.2. For any  $\mathbf{x} \in \mathcal{K} \subset \mathbb{R}^d$  and  $\psi: \mathcal{K}^\circ \mapsto \mathbb{R}$  a  $\nu$ -self-concordant regularizer with minimum  $\mathbf{x}_1 \in \mathcal{K}^\circ$ , the quantity  $\psi(\mathbf{c}_{\varepsilon}(\mathbf{x}))$  is  $\nu \sqrt{2}$ -Lipschitz w.r.t.  $\varepsilon \in [0,1]$ .
Proof. Consider any  $\varepsilon, \varepsilon' \in [0,1]$  s.t.  $\varepsilon' - \varepsilon \in (0,\frac{1}{2}]$ . Note that for  $t = \frac{\varepsilon' - \varepsilon}{1 + \varepsilon}$  we have
$$
\mathbf {c} _ {\varepsilon^ {\prime}} (\mathbf {x}) + \frac {\mathbf {c} _ {\varepsilon^ {\prime}} (\mathbf {x}) - \mathbf {c} _ {\varepsilon} (\mathbf {x})}{t} = \mathbf {x} _ {1} + \frac {\mathbf {x} - \mathbf {x} _ {1}}{1 + \varepsilon^ {\prime}} + \frac {\mathbf {x} _ {1} + \frac {\mathbf {x} - \mathbf {x} _ {1}}{1 + \varepsilon} - \mathbf {x} _ {1} - \frac {\mathbf {x} - \mathbf {x} _ {1}}{1 + \varepsilon^ {\prime}}}{t} = \mathbf {x} \in \mathcal {K} \tag {83}
$$
so  $\pi_{\mathbf{c}_{\varepsilon^{\prime}}(\mathbf{x})}(\mathbf{c}_{\varepsilon}(\mathbf{x})) \leq \frac{\varepsilon^{\prime} - \varepsilon}{1 + \varepsilon} \leq \varepsilon^{\prime} - \varepsilon$ . Therefore by Nesterov and Nemirovskii [42, Prop. 2.3.2] we have
$$
\psi \left(\mathbf {c} _ {\varepsilon} (\mathbf {x})\right) - \psi \left(\mathbf {c} _ {\varepsilon^ {\prime}} (\mathbf {x})\right) \leq \nu \log \left(\frac {1}{1 - \pi_ {\mathbf {c} _ {\varepsilon^ {\prime}} (\mathbf {x})} \left(\mathbf {c} _ {\varepsilon} (\mathbf {x})\right)}\right) \leq \nu \log \left(\frac {1}{1 + \varepsilon - \varepsilon^ {\prime}}\right) \leq \nu \left(\varepsilon^ {\prime} - \varepsilon\right) \sqrt {2} \tag {84}
$$
where for the last inequality we used  $-\log (1 - x)\leq x\sqrt{2}$  for  $x\in [0,\frac{1}{2} ]$  . The case of  $\varepsilon^{\prime} - \varepsilon \in (0,1]$  follows by considering  $\varepsilon '' = \frac{\varepsilon' + \varepsilon}{2}$  and applying the above twice.
Theorem E.1. In Algorithm 1, let  $\mathsf{OMD}_{\eta,\varepsilon}$  be online mirror descent over loss estimators specified in Abernethy et al. [2] with a  $\nu$ -self-concordant barrier regularizer  $\psi: \mathcal{K}^{\circ} \mapsto \mathbb{R}$  that satisfies  $\nu \geq 1$  and  $\| \nabla^{2}\psi(\mathbf{x}_{1})\|_{2} = S_{1} \geq 1$ . Let  $\Theta_{k}$  be a subset of  $[\frac{1}{m}, 1]$  and
$$
U _ {t} (\mathbf {x}, \eta , \varepsilon) = \frac {B \left(\mathbf {c} _ {\varepsilon} (\hat {\mathbf {x}}) \right\| \mathbf {x})}{\eta} + 3 2 \eta d ^ {2} + \varepsilon m \tag {85}
$$
Note that  $U_{t}^{(\rho)}(\mathbf{x},\eta ,\varepsilon) = U_{t}(\mathbf{x},\eta ,\varepsilon) + \frac{9\nu^{\frac{3}{2}}\rho^{2}Km\sqrt{S_{1}}}{\eta}$ . Then there exists settings of  $\underline{\eta},\overline{\eta},\alpha ,\lambda$  s.t. for all  $\varepsilon ,\rho \in (0,1)$  we have expected task averaged regret at most
$$
\begin{array}{l} \mathbb{E}\min_{\varepsilon \in [\frac{1}{m},1],\eta >0}\frac{512\nu^{2}K^{2}S_{1}m^{2}(1 + \log T)}{\eta} +\left(\frac{\hat{V}_{\varepsilon}^{2}}{\eta} +32\eta d^{2}m + \varepsilon m + \frac{\nu\sqrt{2} / \eta + m}{k}\right)T \\ + 3 \nu^ {\frac {3}{4}} m \min  \left\{\frac {3 \rho^ {2} \nu^ {\frac {3}{4}} K \sqrt {S _ {1}}}{\eta}, 4 d \rho \sqrt {2 K \sqrt {S _ {1}}} \right\} T \tag {86} \\ + \frac {7 d m}{\rho} \sqrt {2 K \sqrt {\nu^ {3} S _ {1}}} \left(7 \sqrt {T \log k} + \frac {1 + \log (T + 1)}{\rho}\right) \\ \end{array}
$$
Proof. Let  $\underline{\varepsilon} = \frac{1}{m}$ . For any  $\varepsilon \in [\underline{\varepsilon}, 1]$  and  $\mathbf{x} \in \mathcal{K}$  we have  $\pi_{\mathbf{x}_1}(\mathbf{c}_{\varepsilon}(\mathbf{x})) \leq \frac{1}{1 + \varepsilon}$ , so by Nesterov and Nemirovskii [42, Prop. 2.3.2] we have
$$
\left\| \nabla^ {2} \psi \left(\mathbf {c} _ {\varepsilon} (\mathbf {x})\right) \right\| _ {2} \leq \left(\frac {1 + 3 \nu}{1 - \pi_ {\mathbf {x} _ {1}} \left(\mathbf {c} _ {\varepsilon} (\mathbf {x})\right)}\right) ^ {2} \| \nabla^ {2} \psi \left(\mathbf {x} _ {1}\right) \| _ {2} \leq \frac {6 4 \nu^ {2} S _ {1}}{\varepsilon^ {2}} \tag {87}
$$
Thus  $S = \max_{\mathbf{x},\mathbf{y}\in \mathcal{K},\varepsilon \in [\underline{\varepsilon},1]}\| \nabla^2\psi (\mathbf{c}_\varepsilon (\mathbf{x}))\| _2 = \frac{64\nu^2S_1}{\varepsilon^2}$  and also
$$
\begin{array}{l} D _ {\varepsilon} ^ {2} = \max  _ {\mathbf {x}, \mathbf {y} \in \mathcal {K}} B (\mathbf {c} _ {\varepsilon} (\mathbf {x}) | | \mathbf {c} _ {\varepsilon} (\mathbf {y})) \\ = \max  _ {\mathbf {x}, \mathbf {y} \in \mathcal {K}} \psi (\mathbf {c} _ {\varepsilon} (\mathbf {x})) - \psi (\mathbf {c} _ {\varepsilon} (\mathbf {y})) - \langle \nabla \psi (\mathbf {c} _ {\varepsilon} (\mathbf {y})), \mathbf {x} - \mathbf {y} \rangle \\ \leq \max  _ {\mathbf {x}, \mathbf {y} \in \mathcal {K}} \nu \log \left(\frac {1}{1 - \pi_ {\mathbf {x} _ {1}} (\mathbf {c} _ {\varepsilon} (\mathbf {x}))}\right) + \sqrt {\nu \| \nabla^ {2} \psi (\mathbf {c} _ {\varepsilon} (\mathbf {y})) \| _ {2}} \| \mathbf {x} - \mathbf {y} \| _ {2} \tag {88} \\ \leq \nu \log \frac {2}{\varepsilon} + \frac {8 \nu^ {\frac {3}{2}} K \sqrt {S _ {1}}}{\varepsilon} \\ \leq \frac {9 \nu^ {\frac {3}{2}} K \sqrt {S _ {1}}}{\varepsilon} \\ \end{array}
$$
where the first inequality follows by Nesterov and Nemirovskii [42, Prop. 2.3.2] and the definition of a self-concordant barrier [2, Def. 5]. In addition, we have  $g(\varepsilon) = 32d^{2}$ ,  $f(\varepsilon) = \varepsilon$ ,  $M = 12d\sqrt{2Km / \underline{\varepsilon}}\sqrt[4]{\nu^3S_1}$ , and  $F = 1$ . We have
$$
\begin{array}{l} \mathbb {E} \sum_ {t = 1} ^ {T} \sum_ {i = 1} ^ {m} \langle \ell_ {t, i}, \mathbf {x} _ {t, i} - \mathring {\mathbf {x}} _ {t} \rangle \leq \mathbb {E} \sum_ {t = 1} ^ {T} \varepsilon_ {t} m + \sum_ {i = 1} ^ {m} \langle \ell_ {t, i}, \mathbf {x} _ {t, i} - \mathbf {c} _ {\varepsilon_ {t}} (\mathring {\mathbf {x}} _ {t}) \rangle \\ \leq \mathbb {E} \sum_ {t = 1} ^ {T} \varepsilon_ {t} m + \sum_ {i = 1} ^ {m} \left\langle \hat {\ell} _ {t, i}, \mathbf {x} _ {t, i} - \mathbf {c} _ {\varepsilon_ {t}} \left(\stackrel {\circ} {\mathbf {X}} _ {t}\right) \right\rangle \tag {89} \\ \leq \mathbb {E} \sum_ {t = 1} ^ {T} \varepsilon_ {t} m + \sum_ {i = 1} ^ {m} \langle \hat {\ell} _ {t, i}, \mathbf {x} _ {t, i} - \mathbf {c} _ {\varepsilon_ {t}} (\hat {\mathbf {x}} _ {t}) \rangle \\ \leq \sum_ {t = 1} ^ {T} \frac {\mathbb {E} B \left(\mathbf {c} _ {\varepsilon_ {t}} \left(\hat {\mathbf {x}} _ {t} \right| | \mathbf {x} _ {t , 1}\right)}{\eta_ {t}} + (3 2 \eta_ {t} d ^ {2} + \varepsilon_ {t}) m \\ \end{array}
$$
where the first inequality follows by Abernethy et al. [2, Lem. 8], the second by Abernethy et al. [2, Lem. 3], the third by optimality of  $\hat{\mathbf{x}}_t$ , and the fourth by Lemma E.1. Substituting into Theorem A.1 and simplifying yields the result.
E.2 Specialization to the unit sphere
Corollary E.1. Let  $\mathcal{K}$  be the unit sphere with the self-concordant barrier  $\psi (\mathbf{x}) = -\log (1 - \| \mathbf{x}\| _2^2)$ . Then Algorithm 1 attains expected task-averaged regret bounded by
$$
\tilde {\mathcal {O}} \left(\frac {d m ^ {\frac {3}{2}}}{T ^ {\frac {3}{4}}} + \frac {d m}{\sqrt [ 4 ]{T}}\right) + \min  _ {\varepsilon \in \left[ \frac {1}{m}, 1 \right]} 4 d \sqrt {2 m \log \left(1 + \frac {1 - \mathbb {E} \| \hat {\mathbf {x}} \| _ {2} ^ {2}}{2 \varepsilon + \varepsilon^ {2}}\right)} + \varepsilon m \tag {90}
$$
using  $k = \left\lceil \sqrt{T}\right\rceil$
Proof. Using the fact the  $\nu = 1$  and  $K = S_{1} = 2$ , we apply Theorem E.1 and simplify to obtain
$$
\mathbb {E} \min  _ {\varepsilon \in [ \frac {1}{m}, 1 ], \eta > 0} \frac {\hat {V} _ {\varepsilon} ^ {2}}{\eta} + 3 2 \eta d ^ {2} m + \varepsilon m + \tilde {\mathcal {O}} \left(\frac {m ^ {2}}{\eta T} + \frac {1}{\eta k} + \frac {m}{k} + m \min  \left\{\frac {\rho^ {2}}{\eta}, d \rho \right\} + \frac {d m}{\rho \sqrt {T}} + \frac {d m}{\rho^ {2} T}\right) \tag {91}
$$
Then substitute  $\eta = \frac{\hat{V}_{\varepsilon}}{4\sqrt{2dm}} +\frac{\sqrt{m}}{d^{\frac{4}{3}T}}$  , set  $\rho = \frac{1}{\sqrt[4]{T}}$  , and note that
$$
\begin{array}{l} \mathbb {E} \hat {V} _ {\varepsilon} = \mathbb {E} \sqrt {\log \left(\frac {1 - \| \mathbf {c} _ {\varepsilon} (\hat {\mathbf {x}}) \| _ {2} ^ {2}}{\sqrt [ t ]{\prod_ {t = 1} ^ {T} 1 - \| \mathbf {c} _ {\varepsilon} (\hat {\mathbf {x}} _ {t}) \| _ {2} ^ {2}}}\right)} = \mathbb {E} \sqrt {\log \left(\frac {1 - (1 + \varepsilon) ^ {- 2} \| \hat {\mathbf {x}} \| _ {2} ^ {2}}{1 - (1 + \varepsilon) ^ {- 2}}\right)} \tag {92} \\ \leq \sqrt {\log \left(1 + \frac {1 - \mathbb {E} \| \hat {\mathbf {x}} \| _ {2} ^ {2}}{2 \varepsilon + \varepsilon^ {2}}\right)} \\ \end{array}
$$
where we use the fact that  $\| \hat{\mathbf{x}}_t\| _2 = 1$  and the inequality is Jensen's.
E.3 Specialization to polytopes, specifically the bandit online shortest-path problem
Corollary E.2. Let  $\mathcal{K} = \{\mathbf{x}\in [0,1]^{|E|}:\langle \mathbf{a},\mathbf{x}\rangle \leq b\forall (\mathbf{a},b)\in \mathcal{C}\}$  be the set of flows from  $u$  to  $v$  on a graph  $G(V,E)$ , where  $\mathcal{C}\subset \mathbb{R}^{|E|}\times \mathbb{R}$  is a set of  $\mathcal{O}(|E|)$  linear constraints. Suppose we see  $T$  instances of the bandit online shortest path problem with  $m$  timesteps each. Then sampling from probability distributions over paths from  $u$  to  $v$  returned by running Algorithm 1 with regularizer  $\psi (\mathbf{x}) = -\sum_{\mathbf{a},b\in \mathcal{C}}\log (b - \langle \mathbf{a},\mathbf{x}\rangle)$  attains the following expected average regret across instances
$$
\tilde {\mathcal {O}} \left(\frac {| E | ^ {4} m ^ {\frac {3}{2}}}{T ^ {\frac {3}{4}}} + \frac {| E | ^ {\frac {5}{2}} m ^ {\frac {5}{6}}}{\sqrt [ 4 ]{T}}\right) + \min  _ {\varepsilon \in \left[ \frac {1}{m}, 1 \right]} 4 | E | \mathbb {E} \sqrt {2 m \sum_ {\mathbf {a} , b \in \mathcal {C}} \log \left(\frac {\frac {1}{T} \sum_ {t = 1} ^ {T} b - \langle \mathbf {a} , \mathbf {c} _ {\varepsilon} (\hat {\mathbf {x}} _ {t}) \rangle}{\sqrt [ T ]{\prod_ {t = 1} ^ {T} b - \langle \mathbf {a} , \mathbf {c} _ {\varepsilon} (\hat {\mathbf {x}} _ {t}) \rangle}}\right)} + \varepsilon m
$$
Proof. Using the fact that  $d = |E|$ ,  $\nu = \mathcal{O}(|E|)$ ,  $K = \sqrt{|E|}$ , and  $S_{1} \leq \sum_{\mathbf{a}, b \in \mathcal{C}} \frac{\|\mathbf{aa}^{T}\|_{2}}{(\langle\mathbf{a}, \mathbf{1}_{|E}| / |E|\rangle - \mathbf{b})^{2}} = \mathcal{O}(|E|^{3})$ , we apply Theorem E.1 and simplify to obtain
$$
\begin{array}{l} \mathbb {E} \min  _ {\varepsilon \in [ \frac {1}{m}, 1 ], \eta > 0} \frac {\hat {V} _ {\varepsilon} ^ {2}}{\eta} + 3 2 \eta | E | ^ {2} m + \varepsilon m \tag {94} \\ + \tilde {\mathcal {O}} \left(\frac {| E | ^ {6} m ^ {2}}{\eta T} + \frac {| E |}{\eta k} + \frac {m}{k} + m \min  \left\{\frac {\rho^ {2} | E | ^ {\frac {7}{2}}}{\eta}, \rho | E | ^ {\frac {1 1}{4}} \right\} + \frac {| E | ^ {\frac {1 1}{4}} m}{\rho} \left(\frac {1}{\sqrt {T}} + \frac {1}{\rho T}\right)\right) \\ \end{array}
$$
Then substitute  $\eta = \frac{\hat{V}_z}{4\sqrt{2dm}} +\frac{|E|^2\sqrt{m}}{\sqrt[4]{T}}$  , set  $\rho = \sqrt[4]{\frac{|E|}{T}}\sqrt[6]{m}$  , and note that
$$
\hat {V} _ {\varepsilon} ^ {2} = \sum_ {\mathbf {a}, b \in \mathcal {C}} \log \left(\frac {b - \langle \mathbf {a} , \mathbf {c} _ {\varepsilon} (\hat {\mathbf {x}}) \rangle}{\sqrt [ T ]{\prod_ {t = 1} ^ {T} b - \langle \mathbf {a} , \mathbf {c} _ {\varepsilon} (\hat {\mathbf {x}} _ {t}) \rangle}}\right) = \sum_ {\mathbf {a}, b \in \mathcal {C}} \log \left(\frac {\frac {1}{T} \sum_ {t = 1} ^ {T} b - \langle \mathbf {a} , \mathbf {c} _ {\varepsilon} (\hat {\mathbf {x}} _ {t}) \rangle}{\sqrt [ T ]{\prod_ {t = 1} ^ {T} b - \langle \mathbf {a} , \mathbf {c} _ {\varepsilon} (\hat {\mathbf {x}} _ {t}) \rangle}}\right) \tag {95}
$$