SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
\eta \gamma_ {i, t} + L - X _ {0} ^ {(t)} \leq - p _ {i, t} ^ {2} \quad \text {i f} r _ {t} = 0,
$$
$$
\eta \gamma_ {i, t} + L - X _ {1} ^ {(t)} \leq - (1 - p _ {i, t}) ^ {2} \quad \text {i f} r _ {t} = 1.
$$
Given that  $f(\gamma_{i,t}) \propto \exp(-\nu(\gamma_{i,t}))$ , if we define  $Y_0 = \eta \gamma_{i,t} + L - X_0^{(t)}$  and  $Y_1 = \eta \gamma_{i,t} + L - X_1^{(t)}$ , we have:
$$
f _ {0} (Y _ {0}) \propto \exp (- \nu (\frac {Y _ {0} - (L - X _ {0} ^ {(t)})}{\eta})),
$$
$$
f _ {1} (Y _ {1}) \propto \exp (- \nu (\frac {Y _ {1} - (L - X _ {1} ^ {(t)})}{\eta})).
$$
Therefore, if  $F_{0}$  and  $F_{1}$  denote the cdf, we can write the probability of expert  $i$  being chosen at round  $t + 1$  as  $F_{0}(-p_{i,t}^{2})$  and  $F_{1}(-(1 - p_{i,t})^{2})$  for the cases  $r_t = 0$  and  $r_t = 1$  respectively. Putting the above results together, we can write the expected utility of expert  $i$  (according to her belief  $b_{i,t}$ ) at round  $t$  as follows:
$$
\mathbb {E} _ {r _ {t} \sim \text {B e r n o u l l i} (b _ {i, t})} [ U _ {i, t} ] = (1 - b _ {i, t}) F _ {0} (- p _ {i, t} ^ {2}) + b _ {i, t} F _ {1} (- (1 - p _ {i, t}) ^ {2}).
$$
Taking the derivative and setting it to zero, we have:
$$
\begin{array}{l} \frac {d}{d p _ {i , t}} \mathbb {E} _ {r _ {t} \sim \mathrm {B e r n o u l l i} (b _ {i, t})} [ U _ {i, t} ] = - 2 p _ {i, t} (1 - b _ {i, t}) f _ {0} (- p _ {i, t} ^ {2}) + 2 (1 - p _ {i, t}) b _ {i, t} f _ {1} (- (1 - p _ {i, t}) ^ {2}) = 0, \\ 2 f _ {1} (- (1 - p _ {i, t}) ^ {2}) \big (- p _ {i, t} (1 - b _ {i, t}) \frac {f _ {0} (- p _ {i , t} ^ {2})}{f _ {1} (- (1 - p _ {i , t}) ^ {2})} + (1 - p _ {i, t}) b _ {i, t} \big) = 0, \\ - p _ {i, t} (1 - b _ {i, t}) \frac {\exp (- \nu (\frac {- p _ {i , t} ^ {2} - (L - X _ {0} ^ {(t)})}{\eta}))}{\exp (- \nu (\frac {- (1 - p _ {i , t}) ^ {2} - (L - X _ {1} ^ {(t)})}{\eta}))} + (1 - p _ {i, t}) b _ {i, t} = 0, \\ - p _ {i, t} (1 - b _ {i, t}) \underbrace {\exp \big (\nu (\frac {- (1 - p _ {i , t}) ^ {2} - (L - X _ {1} ^ {(t)})}{\eta}) - \nu (\frac {- p _ {i , t} ^ {2} - (L - X _ {0} ^ {(t)})}{\eta}) \big)} _ {= A} + (1 - p _ {i, t}) b _ {i, t} = 0. \\ \end{array}
$$
Ideally, we want  $A$  to be as close to 1 as possible because if  $A = 1$ ,  $p_{i,t}^{*} = b_{i,t}$  and the algorithm would be incentive-compatible. In general, we have:
$$
p _ {i, t} ^ {*} = \frac {b _ {i , t}}{b _ {i , t} + (1 - b _ {i , t}) A}.
$$
Note that this is not a closed-form solution for  $p_{i,t}^{*}$  because  $A$  is also a function of  $p_{i,t}$ . We can observe that for  $p_{i,t} < p_{i,t}^{*}$ , the derivative of the utility function is positive, and for  $p_{i,t} > p_{i,t}^{*}$ , the derivative is negative.
We can bound A as follows: Let  $g(u) = f\left(\frac{-p_{i,t}^2 - a - u(a' - a)}{\eta}\right)$  where  $a = L - X_1^{(t)} + 1 - 2p_{i,t}$ ,  $a' = L - X_0^{(t)}$  and  $f(z) = \exp(-\nu(z))$ . Taking derivative of  $g$  with respect to  $u$ , we obtain:
$$
g ^ {\prime} (u) = \frac {(a ^ {\prime} - a) \nu^ {\prime} (\frac {- p _ {i , t} ^ {2} - a - u (a ^ {\prime} - a)}{\eta})}{\eta} f (\frac {- p _ {i , t} ^ {2} - a - u (a ^ {\prime} - a)}{\eta}) = \frac {(a ^ {\prime} - a) \nu^ {\prime} (\frac {- p _ {i , t} ^ {2} - a - u (a ^ {\prime} - a)}{\eta})}{\eta} g (u).
$$
Therefore, we can write:
$$
\begin{array}{l} \ln f (\frac {- p _ {i , t} ^ {2} - a ^ {\prime}}{\eta}) - \ln f (\frac {- p _ {i , t} ^ {2} - a}{\eta}) = \ln g (1) - \ln g (0) \\ = \int_ {0} ^ {1} \frac {g ^ {\prime} (u)}{g (u)} d u \\ = \int_ {0} ^ {1} \frac {(a ^ {\prime} - a) \nu^ {\prime} (\frac {- p _ {i , t} ^ {2} - a - u (a ^ {\prime} - a)}{\eta})}{\eta} d u. \\ \end{array}
$$
We have  $a' - a = X_1^{(t)} - X_0^{(t)} + 2p_{i,t} - 1$ . Therefore,  $-2 \leq a' - a \leq 2$  holds. Moreover, we have  $-B \leq \nu' \left( \frac{-p_{i,t}^2 - a - u(a' - a)}{\eta} \right) \leq B$  due to Condition 1. Putting the above results together, the following holds:
$$
\frac {- 2 B}{\eta} \leq \ln A = \ln f (\frac {- p _ {i , t} ^ {2} - a ^ {\prime}}{\eta}) - \ln f (\frac {- p _ {i , t} ^ {2} - a}{\eta}) \leq \frac {2 B}{\eta}.
$$
Therefore,  $A \in [\exp(\frac{-2B}{\eta}), \exp(\frac{2B}{\eta})]$ . Let  $h(p_{i,t}) = p_{i,t} - p_{i,t}^{*} = p_{i,t} - \frac{b_{i,t}}{b_{i,t} + (1 - b_{i,t})A}$ . Since  $p_{i,t}^{*} \in [0,1]$ , we have  $h(0) < 0$  and  $h(1) > 0$ . Taking the derivative of  $h$  with respect to  $p_{i,t}$ , we have:
$$
\frac {d h}{d p _ {i , t}} = 1 + \frac {b _ {i , t} (1 - b _ {i , t}) A \big (\frac {2 (1 - p _ {i , t})}{\eta} \nu^ {\prime} (\frac {- (1 - p _ {i , t}) ^ {2} - (L - X _ {1} ^ {(t)})}{\eta}) + \frac {2 p _ {i , t}}{\eta} \nu^ {\prime} (\frac {- p _ {i , t} ^ {2} - (L - X _ {0} ^ {(t)})}{\eta}) \big)}{(b _ {i , t} + (1 - b _ {i , t}) A) ^ {2}}.
$$
Using Condition 1, we have  $1 - \frac{2Bb_{i,t}(1 - b_{i,t})A}{\eta(b_{i,t} + (1 - b_{i,t})A)^2} \leq \frac{dh}{dp_{i,t}}$ . Therefore, we can write:
$$
\frac {d h}{d p _ {i , t}} \geq 1 - \frac {2 B b _ {i , t} (1 - b _ {i , t}) A}{\eta (b _ {i , t} + (1 - b _ {i , t}) A) ^ {2}} = 1 - \frac {2 B b _ {i , t} (1 - b _ {i , t}) A}{\eta (b _ {i , t} ^ {2} + (1 - b _ {i , t}) ^ {2} A ^ {2} + 2 b _ {i , t} (1 - b _ {i , t}) A)} \geq 1 - \frac {2 B b _ {i , t} (1 - b _ {i , t}) A}{2 \eta b _ {i , t} (1 - b _ {i , t}) A} = 1 - \frac {B}{\eta} > 0.
$$
We choose  $\eta$  later such that  $\eta > B$  for the last inequality to hold. So,  $h$  is strictly increasing, there is exactly one solution  $p_{i,t}^{*}$ , and the derivative is positive below it and negative above it.
If we replace  $A$  with something larger in  $p_{i,t}^*$ , the value decreases and vice versa. Therefore, we can write:
$$
\begin{array}{l} p _ {i, t} ^ {*} \leq \frac {b _ {i , t}}{b _ {i , t} + (1 - b _ {i , t}) \exp (\frac {- 2 B}{\eta})} \\ = \frac {b _ {i , t}}{\exp (\frac {- 2 B}{\eta}) + b _ {i , t} (1 - \exp (\frac {- 2 B}{\eta}))} \\ \leq \frac {b _ {i , t}}{\exp (\frac {- 2 B}{\eta})} \\ \leq \frac {b _ {i , t}}{1 - \frac {2 B}{\eta}} \\ = \frac {\eta b _ {i , t}}{\eta - 2 B} \\ = b _ {i, t} + \frac {2 B b _ {i , t}}{\eta - 2 B} \\ \leq b _ {i, t} + \frac {2 B}{\eta - 2 B}. \\ \end{array}
$$
Similarly, we can lower bound  $p_{i,t}^{*}$  as follows:
$$
\begin{array}{l} p _ {i, t} ^ {*} \geq \frac {b _ {i , t}}{b _ {i , t} + (1 - b _ {i , t}) \exp (\frac {2 B}{\eta})} \\ = \frac {b _ {i , t}}{\exp (\frac {2 B}{\eta}) - b _ {i , t} \left(\exp (\frac {2 B}{\eta}) - 1\right)} \\ \geq \frac {b _ {i , t}}{\exp (\frac {2 B}{\eta})} \\ = b _ {i, t} \exp \left(\frac {- 2 B}{\eta}\right) \\ \geq b _ {i, t} \left(1 - \frac {2 B}{\eta}\right) \\ \geq b _ {i, t} - \frac {2 B}{\eta}. \\ \end{array}
$$
Putting the above results together, we conclude that  $|p_{i,t}^{*} - b_{i,t}| \leq \frac{2B}{\eta - 2B}$  holds.
Let  $\eta > 0$  and  $\mathcal{X}$  be the set of  $\binom{K}{m}$  feasible sets of size  $m$  for this problem. Denote  $L_{t} \in \mathbb{R}^{K}$  as the partial sum of losses for all experts before round  $t$ , i.e.,  $[L_t]_i = \sum_{s=1}^{t-1} \ell_{i,s}$ . The update rule of FTPL at round  $t \in [T]$  is  $\pi_t = \arg \min_{x \in \mathcal{X}} \langle x, L_t + \eta \gamma_t \rangle$ . First, we analyze the expected regret of the algorithm below.
Define  $\phi_t(\theta) = \mathbb{E}_{\gamma_t \sim \mathcal{D}^K}[\min_{x \in \mathcal{X}}\langle x, \theta + \eta \gamma_t\rangle]$ . Then, we have  $\nabla \phi_t(L_t) = \mathbb{E}_{\gamma_t \sim \mathcal{D}^K}[\arg \min_{x \in \mathcal{X}}\langle x, L_t + \eta \gamma_t\rangle] = \mathbb{E}_{\gamma_t \sim \mathcal{D}^K}[\pi_t]$  and  $\langle \nabla \phi_t(L_t), \ell_t\rangle = \mathbb{E}_{\gamma_t \sim \mathcal{D}^K}[\langle \pi_t, \ell_t\rangle]$ . Using the Taylor's expansion of  $\phi_t$ , we can write:
$$
\phi_ {t} (L _ {t + 1}) = \phi_ {t} (L _ {t}) + \langle \nabla \phi_ {t} (L _ {t}), \ell_ {t} \rangle + \frac {1}{2} \langle \ell_ {t}, \nabla^ {2} \phi_ {t} (L _ {t} ^ {\prime}) \ell_ {t} \rangle = \phi_ {t} (L _ {t}) + \mathbb {E} _ {\gamma_ {t} \sim \mathcal {D} ^ {K}} [ \langle \pi_ {t}, \ell_ {t} \rangle ] + \frac {1}{2} \langle \ell_ {t}, \nabla^ {2} \phi_ {t} (L _ {t} ^ {\prime}) \ell_ {t} \rangle ,
$$
where  $L_{t}^{\prime}$  is in the line segment between  $L_{t}$  and  $L_{t + 1} = L_{t} + \ell_{t}$ . By definition,  $\phi_t$  is the minimum of linear functions and therefore, it is concave. Thus,  $\nabla^2\phi_t(L_t')$  is negative semidefinite. Note that since  $\gamma_{t}$  is simply  $K$  i.i.d. samples of the noise distribution for all  $t\in [T]$ , we have  $\phi_t(\theta) = \phi (\theta) =$
$\mathbb{E}_{\gamma \sim \mathcal{D}^K}[\min_{x\in \mathcal{X}}\langle x,\theta +\eta \gamma \rangle ]$  . Taking the sum over  $t\in [T]$  and rearranging the terms, we obtain:
$$
\mathbb {E} [ \sum_ {t = 1} ^ {T} \langle \pi_ {t}, \ell_ {t} \rangle ] = \phi (L _ {T + 1}) - \phi (\underbrace {L _ {1}} _ {= 0}) - \frac {1}{2} \sum_ {t = 1} ^ {T} \langle \ell_ {t}, \nabla^ {2} \phi_ {t} (L _ {t} ^ {\prime}) \ell_ {t} \rangle .
$$
Using Jensen's inequality, we can write:
$$
\phi (L _ {T + 1}) = \mathbb {E} _ {\gamma \sim \mathcal {D} ^ {K}} [ \min  _ {x \in \mathcal {X}} \langle x, L _ {T + 1} + \eta \gamma \rangle ] \leq \min  _ {x \in \mathcal {X}} \mathbb {E} _ {\gamma \sim \mathcal {D} ^ {K}} [ \langle x, L _ {T + 1} + \eta \gamma \rangle ] = \min  _ {x \in \mathcal {X}} \langle x, L _ {T + 1} \rangle .
$$
Therefore, we can rearrange the terms and bound the expected regret as follows:
$$
\mathbb {E} [ R _ {T} ] = \frac {1}{m} \mathbb {E} _ {\gamma \sim \mathcal {D} ^ {K}} [ \sum_ {t = 1} ^ {T} \langle \pi_ {t}, \ell_ {t} \rangle ] - \frac {1}{m} \min _ {x \in \mathcal {X}} \langle x, L _ {T + 1} \rangle \leq - \frac {1}{m} \phi (0) - \frac {1}{2 m} \sum_ {t = 1} ^ {T} \langle \ell_ {t}, \nabla^ {2} \phi_ {t} (L _ {t} ^ {'}) \ell_ {t} \rangle .
$$
To bound the first term on the right hand side, we can use the fact that  $\mathcal{D}$  is symmetric to write:
$$
- \phi (0) = - \mathbb {E} _ {\gamma \sim \mathcal {D} ^ {K}} [ \min  _ {x \in \mathcal {X}} \langle x, \eta \gamma \rangle ] = \eta \mathbb {E} _ {\gamma \sim \mathcal {D} ^ {K}} [ \max  _ {x \in \mathcal {X}} \langle x, - \gamma \rangle ] = \eta \mathbb {E} _ {\gamma \sim \mathcal {D} ^ {K}} [ \max  _ {x \in \mathcal {X}} \langle x, \gamma \rangle ].
$$
Now, we move on to bound the second term in the regret bound. Given that the losses of the experts at each round  $t \in [T]$  are bounded between 0 and 1, we have  $\| \ell_t \|_{\infty} \leq 1$ . Therefore, we have:
$$
- \langle \ell_ {t}, \nabla^ {2} \phi_ {t} (L _ {t} ^ {\prime}) \ell_ {t} \rangle \leq \sum_ {i, j \in [ K ]} | \nabla_ {i, j} ^ {2} \phi_ {t} (L _ {t} ^ {\prime}) |.
$$
By definition, if we denote  $\hat{\pi} (\theta) = \arg \min_{x\in \mathcal{X}}\langle x,\theta \rangle$  , we have:
$$
\nabla_ {i, j} ^ {2} \phi (L _ {t} ^ {\prime}) = \frac {1}{\eta} \mathbb {E} _ {\gamma \sim \mathcal {D} ^ {K}} [ \hat {\pi} (L _ {t} ^ {\prime} + \eta \gamma) _ {i} \frac {d \nu (\gamma_ {j})}{d \gamma_ {j}} ].
$$
Given the concavity of  $\phi$ , diagonal entries of the Hessian  $\nabla^2\phi$  are non-positive. We can also use  $1^T\hat{\pi} (\theta) = m$  to show that the off-diagonal entries are non-negative and each row or column of the Hessian sums up to 0. Therefore, we have  $\sum_{i,j\in [K]}\left|\nabla_{i,j}^{2}\phi (L_{t}^{\prime})\right| = -2\sum_{i = 1}^{K}\nabla_{i,i}^{2}\phi (L_{t}^{\prime})$ . Putting the above results together, we can bound the regret as follows:
$$
\mathbb {E} [ R _ {T} ] \leq \frac {\eta}{m} \mathbb {E} _ {\gamma \sim \mathcal {D} ^ {K}} [ \max  _ {x \in \mathcal {X}} \langle x, \gamma \rangle ] + \frac {1}{\eta m} \sum_ {t = 1} ^ {T} \sum_ {i = 1} ^ {K} \mathbb {E} _ {\gamma \sim \mathcal {D} ^ {K}} [ \hat {\pi} (L _ {t} ^ {\prime} - \eta \gamma) _ {i} \frac {d \nu (\gamma_ {i})}{d \gamma_ {i}} ].
$$
We can bound the first term as follows:
$$
\begin{array}{l} \mathbb {E} _ {\gamma \sim \mathcal {D} ^ {K}} [ \max  _ {x \in \mathcal {X}} \langle x, \gamma \rangle ] \leq \inf  _ {s: s > 0} \frac {1}{s} \ln \left(\sum_ {x \in \mathcal {X}} \mathbb {E} _ {\gamma \sim \mathcal {D} ^ {K}} [ \exp (s \langle x, \gamma \rangle) ]\right) \\ = \inf  _ {s: s > 0} \frac {1}{s} \ln \left(\sum_ {x \in \mathcal {X}} \prod_ {i = 1} ^ {K} \mathbb {E} _ {\gamma_ {i} \sim \mathcal {D}} [ \exp (s x _ {i} \gamma_ {i}) ]\right) \\ = \inf  _ {s: s > 0} \frac {1}{s} \ln \left(| \mathcal {X} | \left(\mathbb {E} _ {\gamma_ {1} \sim \mathcal {D}} [ \exp (s \gamma_ {1}) ]\right) ^ {m}\right) \\ = \inf  _ {s: s > 0} \frac {1}{s} \ln | \mathcal {X} | + \frac {m}{s} \ln \mathbb {E} _ {\gamma_ {1} \sim \mathcal {D}} [ \exp (s \gamma_ {1}) ] \\ \leq \inf  _ {s: s > 0} \frac {m}{s} \ln \left(\frac {K e}{m}\right) + \frac {m}{s} \ln \mathbb {E} _ {\gamma_ {1} \sim \mathcal {D}} [ \exp (s \gamma_ {1}) ] \\ \leq \mathcal {O} (m \ln (\frac {K}{m})) + \mathcal {O} (m) \\ = \mathcal {O} (m \ln (\frac {K}{m})). \\ \end{array}
$$
To bound the second term in the regret bound, we can use Condition 1 to write:
$$
\sum_ {i = 1} ^ {K} \mathbb {E} _ {\gamma \sim \mathcal {D}} [ \hat {\pi} (L _ {t} ^ {\prime} - \eta \gamma) _ {i} \frac {d \nu (\gamma_ {i})}{d \gamma_ {i}} ] \leq B \sum_ {i = 1} ^ {K} \mathbb {E} _ {\gamma \sim \mathcal {D}} [ \hat {\pi} (L _ {t} ^ {\prime} - \eta \gamma) _ {i} ] = B \mathbb {E} _ {\gamma \sim \mathcal {D}} [ \sum_ {i = 1} ^ {K} \hat {\pi} (L _ {t} ^ {\prime} - \eta \gamma) _ {i} ] = B m.
$$
Putting the above results together, we have:
$$
\mathbb {E} [ R _ {T} ] \leq \mathcal {O} (\eta \ln (\frac {K}{m})) + \frac {B T}{\eta}.
$$
Therefore, if we set  $\eta = \sqrt{\frac{BT}{\ln(\frac{K}{m})}}$ , the regret bound is  $\mathcal{O}(\sqrt{BT\ln(\frac{K}{m})})$ . In the proof of Theorem 3, we assumed that  $\eta$  is chosen such that  $\eta > B$ . So, we assume  $T > B\ln \left(\frac{K}{m}\right)$ .
A.5 Proof of Theorem 6
For  $i\in [m]$  and  $t\in [T]$ , define:
$$
\phi_ {i, t} (S) = (1 - \frac {1}{m}) ^ {m - i} g _ {t} (S) + h _ {t} (S).
$$
For all  $i\in [m]$  , we can write
$$
\begin{array}{l} \phi_ {i, t} (S _ {i, t}) - \phi_ {i - 1, t} (S _ {i - 1, t}) = (1 - \frac {1}{m}) ^ {m - i} g _ {t} (S _ {i, t}) + h _ {t} (S _ {i, t}) - (1 - \frac {1}{m}) ^ {m - (i - 1)} g _ {t} (S _ {i - 1, t}) - h _ {t} (S _ {i - 1, t}) \\ = (1 - \frac {1}{m}) ^ {m - i} (g _ {t} (S _ {i, t}) - g _ {t} (S _ {i - 1, t})) + h _ {t} (v _ {i, t}) + \frac {1}{m} (1 - \frac {1}{m}) ^ {m - i} g _ {t} (S _ {i - 1, t}). \\ \end{array}
$$
Taking the sum over  $t\in [T]$  , we obtain:
$$
\begin{array}{l} \sum_ {t = 1} ^ {T} \left(\phi_ {i, t} \left(S _ {i, t}\right) - \phi_ {i - 1, t} \left(S _ {i - 1, t}\right)\right) = \left(1 - \frac {1}{m}\right) ^ {m - i} \sum_ {t = 1} ^ {T} \left(g _ {t} \left(S _ {i, t}\right) - g _ {t} \left(S _ {i - 1, t}\right)\right) + \sum_ {t = 1} ^ {T} h _ {t} \left(v _ {i, t}\right) + \frac {1}{m} \left(1 - \frac {1}{m}\right) ^ {m - i} \sum_ {t = 1} ^ {T} g _ {t} \left(S _ {i - 1, t}\right) \\ = (1 - \frac {1}{m}) ^ {m - i} \sum_ {t = 1} ^ {T} g _ {t} (v _ {i, t} | S _ {i - 1, t}) + \sum_ {t = 1} ^ {T} h _ {t} (v _ {i, t}) + \frac {1}{m} (1 - \frac {1}{m}) ^ {m - i} \sum_ {t = 1} ^ {T} g _ {t} (S _ {i - 1, t}). \\ \end{array}
$$
If the regret of  $\mathcal{A}_i$  is bounded above by  $R_T^{(i)}$  and the optimal benchmark solution is OPT =  $\{v_1^*,\ldots ,v_m^*\}$ , for all  $j\in [m]$ , we can write:
$$
\sum_ {t = 1} ^ {T} \left((1 - \frac {1}{m}) ^ {m - i} g _ {t} (v _ {j} ^ {*} | S _ {i - 1, t}) + h _ {t} (v _ {j} ^ {*})\right) - \sum_ {t = 1} ^ {T} \left((1 - \frac {1}{m}) ^ {m - i} g _ {t} (v _ {i, t} | S _ {i - 1, t}) + h _ {t} (v _ {i, t})\right) \leq R _ {T} ^ {(i)}.
$$
Putting the above inequalities together, we have:
$$
\begin{array}{l} \sum_ {t = 1} ^ {T} (\phi_ {i, t} (S _ {i, t}) - \phi_ {i - 1, t} (S _ {i - 1, t})) \geq \frac {1}{m} (1 - \frac {1}{m}) ^ {m - i} \sum_ {t = 1} ^ {T} \sum_ {j = 1} ^ {m} g _ {t} (v _ {j} ^ {*} | S _ {i - 1, t}) + \frac {1}{m} \sum_ {t = 1} ^ {T} \sum_ {j = 1} ^ {m} h _ {t} (v _ {j} ^ {*}) \\ + \frac {1}{m} (1 - \frac {1}{m}) ^ {m - i} \sum_ {t = 1} ^ {T} g _ {t} \left(S _ {i - 1, t}\right) - R _ {T} ^ {(i)}. \\ \end{array}
$$
We can use submodularity and monotonicity of  $g_{t}$  to write:
$$
g _ {t} (\mathrm {O P T}) - g _ {t} (S _ {i - 1, t}) \leq g _ {t} (\mathrm {O P T} \cup S _ {i - 1, t}) - g _ {t} (S _ {i - 1, t}) = \sum_ {j = 1} ^ {m} g _ {t} \left(v _ {j} ^ {*} \mid S _ {i - 1, t} \cup \{v _ {1} ^ {*}, \dots , v _ {j - 1} ^ {*} \}\right) \leq \sum_ {j = 1} ^ {m} g _ {t} \left(v _ {j} ^ {*} \mid S _ {i - 1, t}\right).
$$
We can combine the last two inequalities to write:
$$
\begin{array}{l} \sum_ {t = 1} ^ {T} (\phi_ {i, t} (S _ {i, t}) - \phi_ {i - 1, t} (S _ {i - 1, t})) \geq \frac {1}{m} (1 - \frac {1}{m}) ^ {m - i} \sum_ {t = 1} ^ {T} (g _ {t} (\mathrm {O P T}) - g _ {t} (S _ {i - 1, t})) + \frac {1}{m} \sum_ {t = 1} ^ {T} \sum_ {j = 1} ^ {m} h _ {t} (v _ {j} ^ {*}) \\ + \frac {1}{m} (1 - \frac {1}{m}) ^ {m - i} \sum_ {t = 1} ^ {T} g _ {t} (S _ {i - 1, t}) - R _ {T} ^ {(i)} \\ = \frac {1}{m} \left(1 - \frac {1}{m}\right) ^ {m - i} \sum_ {t = 1} ^ {T} g _ {t} (\mathrm {O P T}) + \frac {1}{m} \sum_ {t = 1} ^ {T} \sum_ {j = 1} ^ {m} h _ {t} \left(v _ {j} ^ {*}\right) - R _ {T} ^ {(i)}. \\ \end{array}
$$
Taking the sum over  $i \in [m]$ , we have:
$$
\begin{array}{l} \sum_ {t = 1} ^ {T} \sum_ {i = 1} ^ {m} \left(\phi_ {i, t} \left(S _ {i, t}\right) - \phi_ {i - 1, t} \left(S _ {i - 1, t}\right)\right) \geq \frac {1}{m} \sum_ {i = 1} ^ {m} \left(1 - \frac {1}{m}\right) ^ {m - i} \sum_ {t = 1} ^ {T} g _ {t} (\mathrm {O P T}) + \sum_ {t = 1} ^ {T} \sum_ {j = 1} ^ {m} h _ {t} \left(v _ {j} ^ {*}\right) - \sum_ {i = 1} ^ {m} R _ {T} ^ {(i)} \\ = \left(1 - \left(1 - \frac {1}{m}\right) ^ {m}\right) \sum_ {t = 1} ^ {T} g _ {t} (\mathrm {O P T}) + \sum_ {t = 1} ^ {T} h _ {t} (\mathrm {O P T}) - \sum_ {i = 1} ^ {m} R _ {T} ^ {(i)} \\ \geq \left(1 - \frac {1}{e}\right) \sum_ {t = 1} ^ {T} g _ {t} (\mathrm {O P T}) + \sum_ {t = 1} ^ {T} h _ {t} (\mathrm {O P T}) - \sum_ {i = 1} ^ {m} R _ {T} ^ {(i)}. \\ \end{array}
$$
On the other hand, we have:
$$
\sum_ {t = 1} ^ {T} \sum_ {i = 1} ^ {m} (\phi_ {i, t} (S _ {i, t}) - \phi_ {i - 1, t} (S _ {i - 1, t})) = \sum_ {t = 1} ^ {T} (\phi_ {m, t} (S _ {m, t}) - \phi_ {0, t} (S _ {0, t})) = \sum_ {t = 1} ^ {T} (g _ {t} (S _ {t}) + h _ {t} (S _ {t})) = \sum_ {t = 1} ^ {T} f _ {t} (S _ {t}).
$$
Combining the last two inequalities, we obtain:
$$
\begin{array}{l} \sum_ {t = 1} ^ {T} f _ {t} \left(S _ {t}\right) \geq \left(1 - \frac {1}{e}\right) \sum_ {t = 1} ^ {T} g _ {t} (\mathrm {O P T}) + \sum_ {t = 1} ^ {T} h _ {t} (\mathrm {O P T}) - \sum_ {i = 1} ^ {m} R _ {T} ^ {(i)} \\ = \left(1 - \frac {1}{e}\right) \sum_ {t = 1} ^ {T} f _ {t} (\mathrm {O P T}) - \left(1 - \frac {1}{e}\right) \sum_ {t = 1} ^ {T} h _ {t} (\mathrm {O P T}) + \sum_ {t = 1} ^ {T} h _ {t} (\mathrm {O P T}) - \sum_ {i = 1} ^ {m} R _ {T} ^ {(i)} \\ = \left(1 - \frac {1}{e}\right) \sum_ {t = 1} ^ {T} f _ {t} (\mathrm {O P T}) + \frac {1}{e} \sum_ {t = 1} ^ {T} h _ {t} (\mathrm {O P T}) - \sum_ {i = 1} ^ {m} R _ {T} ^ {(i)} \\ \geq \left(1 - \frac {1}{e}\right) \sum_ {t = 1} ^ {T} f _ {t} (\mathrm {O P T}) + \frac {1 - c _ {f}}{e} \sum_ {t = 1} ^ {T} f _ {t} (\mathrm {O P T}) - \sum_ {i = 1} ^ {m} R _ {T} ^ {(i)} \\ = \left(1 - \frac {c _ {f}}{e}\right) \sum_ {t = 1} ^ {T} f _ {t} (\text {O P T}) - \sum_ {i = 1} ^ {m} R _ {T} ^ {(i)}, \\ \end{array}
$$
where  $f = \sum_{t=1}^{T} f_t$ . Rearranging the terms, we obtain the  $(1 - \frac{c_f}{e})$ -regret bound of the online distorted greedy algorithm as follows:
$$
\left(1 - \frac {c _ {f}}{e}\right) \sum_ {t = 1} ^ {T} f _ {t} (\mathrm {O P T}) - \sum_ {t = 1} ^ {T} f _ {t} (S _ {t}) \leq \sum_ {i = 1} ^ {m} R _ {T} ^ {(i)}.
$$
To see why the online greedy algorithm is incentive-compatible, note that at round  $t \in [T]$ , the loss of expert  $j \in [K]$  for the instance  $\mathcal{A}_i$ ;  $i \in [K]$  is:
$$
- f _ {t} (j \mid S _ {i - 1, t}) = \prod_ {k \in S _ {i - 1, t}} \ell_ {k, t} (\ell_ {j, t} - 1).
$$
If we use  $\pi_{j,t + 1}^{(i)}$  to denote the probability of expert  $j$  being chosen at round  $t + 1$  by  $\mathcal{A}_i$ , we can write:
$$
\pi_ {j, t + 1} ^ {(i)} = \pi_ {j, t} ^ {(i)} \big (1 - \eta \prod_ {k \in S _ {i - 1, t}} \ell_ {k, t} (\ell_ {j, t} - \sum_ {s = 1} ^ {K} \pi_ {s, t} ^ {(i)} \ell_ {s, t}) \big).
$$
$\pi_{j,t+1}^{(i)}$  is linear in  $\ell_{j,t}$  and expert  $j$  does not have control over the term  $\prod_{k \in S_{i-1,t}} \ell_{k,t}$  (because  $S_{i-1,t}$  is unknown to the expert). Therefore, to maximize  $\pi_{j,t+1}^{(i)}$ , expert  $j$  can only aim to minimize  $\mathbb{E}_{r_t \sim \mathrm{Bem}(b_{j,t})}[\ell_{j,t} - \sum_{s=1}^K \pi_{s,t}^{(i)} \ell_{s,t}]$ . Given that the quadratic loss function is proper, we can conclude that  $\mathcal{A}_i$  is incentive-compatible. Moreover, since the online greedy algorithm simply outputs the predictions of  $\{\mathcal{A}_i\}_{i=1}^K$ , this algorithm is incentive-compatible as well. For the case with  $c_f = 0$ , the loss of expert  $j \in [K]$  for each instance  $\mathcal{A}_i$ ;  $i \in [K]$  in both online greedy algorithm and online distorted greedy algorithm is simply  $\frac{1}{m} (\ell_{j,t} - 1)$  and the same argument holds.
A.6 Proof of Theorem 7
First, assume that the losses are non-negative. Using the update rule of the algorithm, we can write:
$$
\begin{array}{l} 1 \geq \pi_ {i ^ {*}, T + 1} = \pi_ {i ^ {*}, 1} \prod_ {t = 1} ^ {T} (1 - \eta L _ {i ^ {*}, t}), \\ 1 \geq \frac {1}{K} \prod_ {t: L _ {i ^ {*}, t} \geq 0} (1 - \eta L _ {i ^ {*}, t}) \prod_ {t: L _ {i ^ {*}, t} <   0} (1 - \eta L _ {i ^ {*}, t}). \\ \end{array}
$$
We can use the inequalities  $1 - \eta x \geq (1 - \eta)^x$  for  $x \in [0, 1]$  and  $1 - \eta x \geq (1 + \eta)^{-x}$  for  $x \in [-1, 0]$  to write:
$$
1\geq \frac{1}{K}\prod_{t:L_{i^{*},t}\geq 0}(1 - \eta)^{L_{i^{*},t}}.\prod_{t:L_{i^{*},t} <   0}(1 + \eta)^{-L_{i^{*},t}},
$$
$$
0 \geq -\ln K + \sum_ {t:L _ {i ^ {*}, t} \geq 0} L _ {i ^ {*}, t} \ln (1 - \eta) - \sum_ {t: L _ {i ^ {*}, t} <   0} L _ {i ^ {*}, t} \ln (1 + \eta).
$$
Given the inequalities  $\ln (1 - x) \geq -x - x^2$  and  $\ln (1 + x) \geq x - x^2$  for  $x \leq \frac{1}{2}$ , we have:
$$
0 \geq - \ln K + (- \eta - \eta^ {2}) \sum_ {t: L _ {i ^ {*}, t} \geq 0} L _ {i ^ {*}, t} - (\eta - \eta^ {2}) \sum_ {t: L _ {i ^ {*}, t} <   0} L _ {i ^ {*}, t},
$$
$$
0 \geq - \ln K - \eta \sum_ {t = 1} ^ {T} L _ {i ^ {*}, t} - \eta^ {2} \sum_ {t = 1} ^ {T} | L _ {i ^ {*}, t} |,
$$
$$
R _ {T} = - \sum_ {t = 1} ^ {T} L _ {i ^ {*}, t} \leq \frac {\ln K}{\eta} + \eta \sum_ {t = 1} ^ {T} | L _ {i ^ {*}, t} |.
$$
Given that  $L_{i^{*},t} = \ell_{i^{*},t} - \pi_t^T\ell_t$  , we can write:
$$
R _ {T} \leq \frac {\ln K}{\eta} + \eta \sum_ {t = 1} ^ {T} | \ell_ {i ^ {*}, t} - \pi_ {t} ^ {T} \ell_ {t} | \leq \frac {\ln K}{\eta} + \eta (\sum_ {t = 1} ^ {T} | \ell_ {i ^ {*}, t} | + \sum_ {t = 1} ^ {T} | \pi_ {t} ^ {T} \ell_ {t} |) = \frac {\ln K}{\eta} + \eta (| L _ {T} ^ {*} | + | L _ {T} |),
$$
where  $|L_T^*|$  and  $|L_T|$  are the cumulative absolute loss of the benchmark and the algorithm respectively. Therefore, if we set  $\eta = \min \{1, \sqrt{\frac{\ln K}{|L_T| + |L_T^*|}}\}$ , we obtain an  $\mathcal{O}(\sqrt{\max \{|L_T|, |L_T^*|\} \ln K} + \ln K)$  regret bound. Given that  $R_T = L_T - L_T^*$ , if  $|L_T| = L_T < L_T^* = |L_T^*|$ , the regret is negative and if  $|L_T| = L_T \geq L_T^* = |L_T^*|$ , the regret bound is  $\mathcal{O}(\sqrt{|L_T| \ln K} + \ln K)$ . Therefore, the regret bound  $\mathcal{O}(\sqrt{|L_T| \ln K} + \ln K)$  always holds.