SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Without loss of generality, we may assume  $\mu(q) = 0$ , via a variable transformation  $x \gets x - \mu(q)$ . In fact, such a transformation preserves  $V(q)$  and  $V(p)$ . For any  $x \in \mathcal{X}$ , there exists  $\xi_x = (\xi_x(a))_{a \in \mathcal{A}} \in \mathcal{P}(\mathcal{A})$  such that  $\mu(\xi_x) = \sum_{a \in \mathcal{A}} \xi_x(a) = x$ . For any  $(\xi_x)_{x \in \mathcal{X}}$ , define  $p \in \mathcal{P}(\mathcal{A})$  by  $p(a) = \mathbf{E}_{x \sim q}[\xi_x(a)]$ . We then have  $\mu(p) = \mathbf{E}_{a \sim p}[x] = \mathbf{E}_{x \sim q}[\mathbf{E}_{a \sim \xi_x}[a]] = \mathbf{E}_{x \sim q}[x] = \mu(q) = 0$ . Fix an arbitrary non-zero vector  $u \in \mathbb{R}^d \setminus \{0\}$  and an arbitrary  $a^* \in \mathcal{A}$ . Denote  $\varepsilon = 1 - \operatorname{Pr}_{a \sim p}[a = a^*]$ .
Set  $v = u / \sqrt{u^{\top}V(q)u}$ . Let  $p_v$  and  $q_v$  denote the distributions of  $\langle v, a \rangle$  for  $a \sim p$  and  $\langle v, x \rangle$  for  $x \sim q$ , respectively. We then have  $V(q_v) = v^{\top}V(q)v = u^{\top}V(q)u / (u^{\top}V(q)u) = 1$ . Without loss of generality, we assume that  $\langle v, a^* \rangle \geq \langle v, \mu(q) \rangle = \mu(q_v) = 0$ . (If not, we redefine  $v = -u / \sqrt{u^{\top}V(q)u}$ .) Since  $q$  is a log-concave distribution, from Lemma 10,  $q_v$  is a log-concave distribution as well. Hence, from Lemma 11, we have
$$
\Pr_ {X \sim q _ {v}} \left[ X \leq - \frac {1}{2 \mathrm {e}} \right] = \Pr_ {X \sim q _ {v}} [ X \leq 0 ] - \Pr_ {X \sim q _ {v}} \left[ - \frac {1}{2 \mathrm {e}} <   X \leq 0 \right] \geq \frac {1}{\mathrm {e}} - \frac {1}{2 \mathrm {e}} = \frac {1}{2 \mathrm {e}}, \tag {48}
$$
where the inequality follows from (16) and (17).
We hence have
$$
\begin{array}{l} V (p _ {v}) = \underset {a \sim p} {\mathbf {E}} [ (\langle v, a \rangle) ^ {2} ] = \underset {x \sim q} {\mathbf {E}} \left[ \underset {a \sim \xi_ {x}} {\mathbf {E}} [ (\langle v, a \rangle) ^ {2} ] \right] \\ \geq \mathbf {E} _ {x \sim q} \left[ \mathbf {E} _ {a \sim \xi_ {x}} \left[ (\langle v, a \rangle) ^ {2} \right] | \langle v, x \rangle \leq - \frac {1}{2 \mathrm {e}} \right] \cdot \Pr_ {X \sim q _ {v}} \left[ X \leq - \frac {1}{2 \mathrm {e}} \right] \\ \geq \frac {1}{2 \mathrm {e}} \cdot \underset {x \sim q} {\mathbf {E}} \left[ \underset {a \sim \xi_ {x}} {\mathbf {E}} \left[ (\langle v, a \rangle) ^ {2} \right] \mid \langle v, x \rangle \leq - \frac {1}{2 \mathrm {e}} \right], \tag {49} \\ \end{array}
$$
where the last inequality follows from (48). Suppose  $\langle v, x \rangle \leq -\frac{1}{2\mathrm{e}}$ . Since  $\mu(\xi_x) = x$ , we have
$$
- \frac {1}{2 \mathrm {e}} \geq \langle v, x \rangle = \xi_ {x} \left(a ^ {*}\right) \langle v, a ^ {*} \rangle + \sum_ {a \in \mathcal {A} \backslash \{a ^ {*} \}} \xi_ {x} (a) \langle v, a \rangle \geq \sum_ {a \in \mathcal {A} \backslash \{a ^ {*} \}} \xi_ {x} (a) \langle v, a \rangle . \tag {50}
$$
$$
\begin{array}{l} \underset {a \sim \xi_ {x}} {\mathbf {E}} \left[ (\langle v, a \rangle) ^ {2} \right] \geq \sum_ {a \in \mathcal {A} \setminus \{a ^ {*} \}} \xi_ {x} (a) (\langle v, a \rangle) ^ {2} = (1 - \xi_ {x} (a ^ {*})) \frac {\sum_ {a \in \mathcal {A} \setminus \{a ^ {*} \}} \xi_ {x} (a) (\langle v , a \rangle) ^ {2}}{\sum_ {a \in \mathcal {A} \setminus \{a ^ {*} \}} \xi_ {x} (a)} \\ \geq \left(1 - \xi_ {x} \left(a ^ {*}\right)\right) \left(\frac {\sum_ {a \in \mathcal {A} \backslash \{a ^ {*} \}} \xi_ {x} (a) \langle v , a \rangle}{\sum_ {a \in \mathcal {A} \backslash \{a ^ {*} \}} \xi_ {x} (a)}\right) ^ {2} = \frac {\left(\sum_ {a \in \mathcal {A} \backslash \{a ^ {*} \}} \xi_ {x} (a) \langle v , a \rangle\right) ^ {2}}{1 - \xi_ {x} \left(a ^ {*}\right)} \geq \frac {1}{4 \mathrm {e} ^ {2} \left(1 - \xi_ {x} \left(a ^ {*}\right)\right)}, \\ \end{array}
$$
where the second inequality comes from Jensen's inequality, and the last inequality follows from (50). From this and (49), we have
$$
V \left(p _ {v}\right) \geq \frac {1}{8 \mathrm {e} ^ {3}} \mathbf {E} _ {x \sim q} \left[ \frac {1}{1 - \xi_ {x} \left(a ^ {*}\right)} | \langle v, x \rangle \leq - \frac {1}{2 \mathrm {e}} \right] \geq \frac {1}{8 \mathrm {e} ^ {3}} \cdot \frac {1}{\mathbf {E} _ {x \sim q} \left[ 1 - \xi_ {x} \left(a ^ {*}\right) \right| \langle v , x \rangle \leq - \frac {1}{2 \mathrm {e}}} \tag {51}
$$
We further have
$$
\begin{array}{l} \varepsilon = \underset {x \sim q} {\mathbf {E}} \left[ 1 - \xi_ {x} \left(a ^ {*}\right) \right] \geq \underset {x \sim q} {\mathbf {E}} \left[ 1 - \xi_ {x} \left(a ^ {*}\right) | \langle v, x \rangle \leq - \frac {1}{2 \mathrm {e}} \right] \cdot \Pr_ {x \sim q} \left[ \langle v, x \rangle \leq - \frac {1}{2 \mathrm {e}} \right] \\ \geq \frac {1}{2 \mathrm {e}} \underset {x \sim q} {\mathbf {E}} \left[ 1 - \xi_ {x} \left(a ^ {*}\right) \mid \langle v, x \rangle \leq - \frac {1}{2 \mathrm {e}} \right], \tag {52} \\ \end{array}
$$
where the last inequality follows from (48). Combining this with (51), we obtain
$$
V \left(p _ {v}\right) \geq \frac {1}{1 6 \mathrm {e} ^ {4} \varepsilon}. \tag {53}
$$
As it follows from the definition of  $v$  that
$$
V \left(p _ {v}\right) = v ^ {\top} V (p) v = \frac {u ^ {\top} V (p) u}{u ^ {\top} V (q) u}, \tag {54}
$$
$$
1 6 \mathrm {e} ^ {4} \varepsilon \cdot u ^ {\top} V (p) u \geq u ^ {\top} V (q) u. \tag {55}
$$
Since this holds for all  $u\in \mathbb{R}^d\setminus \{0\}$ , we have
$$
1 6 \mathrm {e} ^ {4} \varepsilon \cdot V (p) \succeq V (q) \tag {56}
$$
Letting  $a^* \in \arg \min_{a \in \mathcal{A}} \langle \ell^*, a \rangle$ , we obtain
$$
\begin{array}{l} \Delta_ {\ell^ {*}} (\mu (q)) = \Delta_ {\ell^ {*}} (\mu (p)) = \sum_ {a \in \mathcal {A}} p (a) \Delta_ {\ell^ {*}} (a) \\ \geq \sum_ {a \in \mathcal {A} \backslash \{a ^ {*} \}} p (a) \Delta_ {\ell^ {*}, \min } = \Delta_ {\ell^ {*}, \min } \cdot \left(1 - p \left(a ^ {*}\right)\right) = \Delta_ {\ell^ {*}, \min } \cdot \varepsilon \tag {57} \\ \end{array}
$$
Combining this with (56), we obtain
$$
1 6 \mathrm {e} ^ {4} \frac {\Delta_ {\ell^ {*}} (\mu (q))}{\Delta_ {\ell^ {*} , \min }} \cdot V (p) \succeq V (q), \tag {58}
$$
B.6 Proof of Lemma 4
Proof. Let  $q$  be a log-concave distribution over  $\mathcal{X} = \{x \in [0,1]^d \mid \| x \|_1 = 1\}$ . If  $x = (x_1, \ldots, x_d) \sim q$ , from Lemma 10,  $x_i$  for each  $i$  also follows a log-concave distribution over  $[0,1]$ . Hence, from Lemma 12, the variance of  $x_i$  (i.e., the  $(i, i)$  entry of  $V(q)$ ) is bounded as
$$
[ V (q) ] _ {i, i} \leq \mathrm {e} ^ {2} \min  \left\{\mu_ {i} (q) ^ {2}, (1 - \mu_ {i} (q)) ^ {2} \right\} \leq 4 \mathrm {e} ^ {2} \mu_ {i} (q) ^ {2} (1 - \mu_ {i} (q)) ^ {2} \tag {59}
$$
Consider the distribution  $p \in \mathcal{P}(\mathcal{A})$  such that  $\operatorname*{Pr}_{a \sim p}[a = e_i] = \mu(q)$ . We then have  $\mu_i(p) = \mu_i(q)$  and that  $H(p)$  is a diagonal matrix with diagonal entries  $(\mu_i(q))_{i=1}^d$ . Hence, from (59), we have
$$
\begin{array}{l} V (q) \bullet (H (p)) ^ {- 1} = \sum_ {i = 1} ^ {d} [ V (q) ] _ {i, i} (\mu_ {i} (q)) ^ {- 1} \leq 4 \mathrm {e} ^ {2} \sum_ {i = 1} ^ {d} \mu_ {i} (q) (1 - \mu_ {i} (q)) ^ {2} \\ \leq 4 \mathrm {e} ^ {2} \sum_ {i = 1} ^ {d} \mu_ {i} (q) \left(1 - \mu_ {i} (q)\right), \tag {60} \\ \end{array}
$$
which implies that  $\omega(q) = O\left(\sum_{i=1}^{d}\mu_i(q)(1 - \mu_i(q))\right)$ . Further, as we have  $\sum_{i=1}^{d}\mu_i(q) = 1$ , it holds for any  $i^* \in \{1,\dots,d\}$  that
$$
\begin{array}{l} \sum_ {i = 1} ^ {d} \mu_ {i} (q) (1 - \mu_ {i} (q)) = \mu_ {i ^ {*}} (q) (1 - \mu_ {i ^ {*}} (q)) + \sum_ {i \neq i ^ {*}} \mu_ {i} (q) (1 - \mu_ {i} (q)) \\ \leq 1 - \mu_ {i ^ {*}} (q) + \sum_ {i \neq i ^ {*}} \mu_ {i} (q) = 2 \left(1 - \mu_ {i ^ {*}} (q)\right), \tag {61} \\ \end{array}
$$
which complete the proof.
B.7 Proof of Corollary 2
Proof. From Lemma 4 and the fact that  $\sum_{i=1}^{d} \mu_i(q) = 1$ , it is clear that  $\omega(q) = O(1)$ . Let  $\ell^* \in \mathcal{L}$  be an loss vector such that  $e_{i^*} \in \arg \min_{1 \leq i \leq d} \ell_i^*$  is unique. We then have  $\Delta_{\ell^*}(\mu(q)) = \sum_{i \neq i^*} \Delta_{\ell^*}(e_i) \mu_i \geq \Delta_{\ell^*, \min}(1 - \mu_{i^*}(q))$ . Hence, from Lemma 4, we have  $\omega(q) = O(1 - \mu_{i^*}(q)) \leq O(\Delta_{\ell^*}(\mu(q)) / \Delta_{\ell^*, \min})$ . From this and Lemma 2, (10) holds with  $\alpha = O(1)$ ,  $\beta = O(\frac{1}{\Delta_{\ell^*, \min}})$ , and  $\eta_0 = O(1 / d)$ .
B.8 Proof of Lemma 5
Proof. For each  $j \in \{1, \ldots, m\}$ , let  $p^{(j)} \in \mathcal{P}(\mathcal{X}^{(j)})$  be such that  $\mu(p^{(j)}) = \mu(q^{(j)})$  and  $\omega(q^{(j)}) = V(q^{(j)}) \bullet (H(p^{(j)}))^{-1}$ . Let  $p \in \mathcal{P}(\mathcal{X})$  be the product measure of  $p^{(1)}, \ldots, p^{(m)}$ . As  $p$  is the product measure of  $q^{(1)}, \ldots, q^{(m)}$ ,  $V(q) \in \mathbb{R}^{d \times d}$  is a block diagonal matrix with submatrices  $V(q^{(1)}, \ldots, V(q^{(m)})$ . Similarly,  $H(p) \in \mathbb{R}^{d \times d}$  is a block diagonal matrix with submatrices  $H(p^{(1)}, \ldots, H(p^{(m)})$ . We hence have  $V(q) \bullet (H(p))^{-1} = \sum_{j=1}^{m} V(q^{(j)}) \bullet (H(p^{(j)}))^{-1} = \sum_{j=1}^{m} \omega(q^{(j)})$ , which implies  $\omega(q) \leq \sum_{j=1}^{m} \omega(q^{(j)})$ .
B.9 Proof of Corollary 3
Proof. Suppose  $\mathcal{A}$  is given by  $\mathcal{A} = \mathcal{A}_{d_1} \times \dots \times \mathcal{A}_{d_m}$ . For any  $\ell = (\ell^{(1)},\ell^{(2)},\ldots,\ell^{(m)}) \in \mathcal{L}$  and  $x = (x^{(1)},x^{(2)},\ldots,x^{(m)}) \in \mathcal{X}$ , where  $\ell^{(j)} \in \mathbb{R}^{d_j}$  and  $x^{(j)} \in \mathcal{X}^{(j)}$  for each  $j$ , denote  $\Delta_{\ell^{(j)}}^{(j)}(x^{(j)}) = \langle \ell^{(j)},x^{(j)} \rangle - \min_{a^{(j)} \in \mathcal{A}^{(j)}}\left\langle \ell^{(j)},a^{(j)}\right\rangle = \left\langle \ell^{(j)},x^{(j)} - a^{*(j)}\right\rangle$ . We then have  $\Delta_{\ell^*}(x) = \sum_{j=1}^m \Delta_{\ell^{(j)}}^{(j)}(x^{(j)})$ . Let  $\ell$  be a loss vector such that  $a^* = (a^{*(1)},a^{*(2)},\ldots,a^{*(m)}) \in \arg \min_{a \in \mathcal{A}}\langle \ell, a \rangle$  is unique. We then have  $\Delta_{\ell, \min} = \min_{1 \leq j \leq m} \Delta_{\ell^{(j)}, \min}^{(j)}$ .
If  $q \in \mathcal{X}$  is a product measure of  $q_j \in \mathcal{P}(\mathcal{X}^{(j)})$ , from Lemmas 4 and 5, we have
$$
\begin{array}{l} \omega (q) \leq O \left(\sum_ {j = 1} ^ {d} \omega \left(q ^ {(j)}\right)\right) \leq O \left(\sum_ {j = 1} ^ {m} \min  \left\{1, \frac {\Delta_ {\ell^ {*} (j)} ^ {(j)} (\mu \left(q ^ {(j)}\right))}{\Delta_ {\ell^ {*} (j) , \min }} \right\}\right) \\ \leq O \left(\min  \left\{m, \sum_ {j = 1} ^ {m} \frac {\Delta_ {\ell^ {*} (j)} ^ {(j)} (\mu (q ^ {(j)}))}{\Delta_ {\ell^ {*} (j) , \min }} \right\}\right) \leq O \left(\min  \left\{m, \sum_ {j = 1} ^ {m} \frac {\Delta_ {\ell^ {*} (j)} ^ {(j)} (\mu (q ^ {(j)}))}{\Delta_ {\ell^ {*} , \min }} \right\}\right) \\ = O \left(\min  \left\{m, \frac {\Delta_ {\ell^ {*}} ^ {(j)} (\mu (q))}{\Delta_ {\ell^ {*} , \min }} \right\}\right). \tag {62} \\ \end{array}
$$
This means that (10) holds with  $\alpha = O(m)$  and  $\beta = O\left(\frac{1}{\Delta_{\ell^{*},\min}}\right)$ .
The case of  $\mathcal{A} = \{0,1\}^d$  can be interpreted as a problem instance in which  $\mathcal{A}$  is a direct product of  $d$  copies of  $\mathcal{A}_2$ . Hence, we can see that (10) holds with  $\alpha = O(d)$  and  $\beta = O(\frac{1}{\Delta_{\ell^{*},\min}})$ .
B.10 Proof of Lemma 6
$$
\begin{array}{l} R _ {T} \left(a ^ {*}\right) = \sum_ {t = 1} ^ {T} \mathbf {E} _ {a \sim p _ {t}} \left[ \left\langle \ell_ {t}, a - a ^ {*} \right\rangle \right] \\ = \sum_ {t = 1} ^ {T} \left(\left\langle \ell_ {t}, w _ {t} - a ^ {*} \right\rangle + \left\langle \ell_ {t}, w _ {t} - \mu \left(p _ {t}\right) \right\rangle\right). \tag {63} \\ \end{array}
$$
From the definition of bias given in (14), the part of  $\langle \ell_t, w_t - a^* \rangle$  can be bounded as follows:
$$
\begin{array}{l} \left\langle \ell_ {t}, w _ {t} - a ^ {*} \right\rangle = \mathbf {E} _ {a _ {t}, f _ {t}} \left[ \left\langle \ell_ {t} - h _ {t} \left(a _ {t}, f _ {t}\right), w _ {t} - a ^ {*} \right\rangle + \left\langle h _ {t} \left(a _ {t}, f _ {t}\right), w _ {t} - a ^ {*} \right\rangle \right] \\ \leq \operatorname {b i a s} \left(h _ {t}, p _ {t}, w _ {t}, \ell_ {t}\right) + \underset {a _ {t}, f _ {t}} {\mathbf {E}} \left[ h _ {t} \left(a _ {t}, f _ {t}\right), w _ {t} - a ^ {*} \right]. \tag {64} \\ \end{array}
$$
From this and (63), we have
$$
R _ {T} \left(a ^ {*}\right) \leq \mathbf {E} \left[ \sum_ {t = 1} ^ {T} \left(\left\langle \ell_ {t}, w _ {t} - \mu \left(p _ {t}\right) \right\rangle + \operatorname {b i a s} \left(h _ {t}, p _ {t}, w _ {t}, \ell_ {t}\right)\right) + \sum_ {t = 1} ^ {T} \left\langle h _ {t} \left(a _ {t}, f _ {t}\right), w _ {t} - a ^ {*} \right\rangle \right]. \tag {65}
$$
By a standard analysis of the FTRL framework (see, e.g., [Lattimore and Szepesvári, 2020a, Exercise 28.12]), we have
$$
\sum_ {t = 1} ^ {T} \left\langle h _ {t} \left(a _ {t}, f _ {t}\right), w _ {t} - a ^ {*} \right\rangle \leq \sum_ {t = 1} ^ {T} \frac {1}{\eta_ {t}} \operatorname {s t a b} _ {\psi} \left(\eta_ {t} h _ {t} \left(a _ {t}, f _ {t}\right), w _ {t}\right) + \frac {1}{\eta_ {T + 1}} \psi \left(a ^ {*}\right). \tag {66}
$$
By combining (65) and (66), we obtain the bound of (14).