SlowGuess/NeurIPS_2023_OCR huggingface dataset:
In the subsequent sections, we propose two efficient algorithmic frameworks that exploit the modular or submodular structure of the utility function and obtain the desired regret and incentive compatibility guarantees.
4 Follow the Perturbed Leader (FTPL) algorithm
In this section, we study the well-known Follow the Perturbed Leader (FTPL) algorithm for the  $m$ -experts problem with modular utility functions and study its regret and incentive compatibility guarantees. The algorithm is as follows: At each round  $t \in [T]$ , we first take  $K$  i.i.d. samples  $\{\gamma_{i,t}\}_{i=1}^{K}$  from the noise distribution  $\mathcal{D}$ . In particular, we focus on zero-mean symmetric noise distributions from the exponential family, i.e.,  $f(\gamma_{i,t}) \propto \exp(-\nu(\gamma_{i,t}))$  where  $\nu : \mathbb{R} \to \mathbb{R}_+$  is symmetric about the origin. At round  $t$ , we simply keep track of  $\sum_{s=1}^{t-1} \ell_{i,s} + \eta \gamma_{i,t}$  for each  $i$  (where  $\eta$  is the step size) and pick the  $m$  experts for whom this quantity is the smallest. [8] previously studied the FTPL algorithm for a class of problems that includes the  $m$ -experts problem. However, they only focused on the setting with zero-mean Gaussian perturbations. In contrast, we not only extend this analysis to all zero-mean symmetric noise distributions from the exponential family, but we also analyze the incentive compatibility guarantees of the algorithm and determine a sufficient condition for the perturbation distribution under which the algorithm is approximately incentive-compatible. This condition is provided below.
Condition 1. For all  $z\in \mathbb{R}$ $|\nu '(z)|\leq B$  holds for some constant  $B > 0$
We have  $\nu(z) = |z|$  for Laplace distribution. Therefore,  $\nu'(z) = \mathrm{sign}(z)$  and  $B = 1$ . For symmetric hyperbolic distribution,  $\nu(z) = \sqrt{1 + z^2}$  holds. So,  $|\nu'(z)| = \frac{|z|}{\sqrt{1 + z^2}} \leq 1$  and  $B = 1$ .
Condition 1 is closely related to a boundedness assumption on the hazard rate of the perturbation distribution. We first define the hazard rate below.
Definition 2. The hazard rate of  $\mathcal{D}$  at  $z\in \mathbb{R}$  is defined as
$$
h a z _ {\mathcal {D}} (z) = \frac {f _ {\mathcal {D}} (z)}{1 - F _ {\mathcal {D}} (z)},
$$
where  $f_{\mathcal{D}}$  and  $F_{\mathcal{D}}$  are the probability density function (pdf) and the cumulative density function (cdf) of the noise distribution  $\mathcal{D}$ . The maximum hazard rate of  $\mathcal{D}$  is  $haz_{\mathcal{D}} = \sup_{z\in \mathbb{R}}haz_{\mathcal{D}}(z)$ .
The hazard rate is a statistical tool used in survival analysis that measures how fast the tail of a distribution decays. The theorem below shows the connection between Condition 1 and the bounded hazard rate assumption.
Theorem 2. If Condition 1 holds for the perturbation distribution  $\mathcal{D}$  with the constant  $B > 0$ , we have  $\text{haz}_{\mathcal{D}} \leq B$ .
However, there are distributions with bounded hazard rates for which  $\max_z |\nu'(z)|$  is unbounded (i.e., Condition 1 does not hold). For instance, consider the standard Gumbel distribution. In this case,  $\nu(z) = z + \exp(-z)$ . Therefore, we have  $\nu'(z) = 1 - \exp(-z)$ . So, if  $z \to -\infty$ ,  $|\nu'(z)| \to \infty$ . Therefore, Condition 1 is strictly stronger than the bounded hazard rate assumption for the noise distribution  $\mathcal{D}$ .
We show how Condition 1 guarantees an approximate notion of incentive compatibility for FTPL.
Theorem 3. For the FTPL algorithm with a noise distribution satisfying Condition 1 with a constant  $B > 0$ , at round  $t \in [T]$ , for an expert  $i \in [K]$ , the optimal report from the expert's perspective  $p_{i,t}^*$  is at most  $\frac{2B}{\eta - 2B}$  away from her belief  $b_{i,t}$ , i.e., the following holds:
$$
| p _ {i, t} ^ {*} - b _ {i, t} | \leq \frac {2 B}{\eta - 2 B}.
$$
Note that while we focused on the incentive structure in which at each round  $t \in [T]$ , experts wish to maximize their probability of being chosen at round  $t + 1$ , the same argument could be applied to a more general setting where the goal is to maximize a conic combination of probabilities of being chosen at all subsequent round  $s > t$ . Therefore, FTPL is approximately incentive compatible with respect to this more general incentive structure as well.
Theorem 3 allows us to bound the regret of the FTPL algorithm with respect to the true beliefs of the experts. First, note that FTPL obtains the following bound with respect to the reported beliefs of the experts.
Theorem 4. For the FTPL algorithm with noise distribution  $\mathcal{D}$  satisfying Condition 1 with the constant  $B > 0$ , if we set  $\eta = \sqrt{\frac{BT}{\ln(\frac{K}{m})}}$ , the following holds:
$$
\mathbb {E} [ \frac {1}{m} \sum_ {t = 1} ^ {T} \sum_ {i \in S _ {t}} \ell (p _ {i, t}, r _ {t}) - \min _ {S: | S | = m} \frac {1}{m} \sum_ {t = 1} ^ {T} \sum_ {j \in S} \ell (p _ {j, t}, r _ {t}) ] \leq \mathcal {O} (\sqrt {B T \ln (\frac {K}{m})}).
$$
Using the result of Theorem 3, we have  $|p_{i,t} - b_{i,t}| = |p_{i,t}^{*} - b_{i,t}| \leq \frac{2B}{\eta - 2B}$ . Moreover, one can easily show that the quadratic loss function is 2-Lipschitz. Therefore, for all  $t \in [T]$  and  $i \in [K]$ , we have:
$$
\left| \ell \left(p _ {i, t}, r _ {t}\right) - \ell \left(b _ {i, t}, r _ {t}\right) \right| \leq \frac {4 B}{\eta - 2 B}.
$$
Putting the above results together, we can obtain the following regret bound for the FTPL algorithm.
$$
\mathbb {E} [ 1 - R _ {T} ] = \mathbb {E} [ \frac {1}{m} \sum_ {t = 1} ^ {T} \sum_ {i \in S _ {t}} \ell (b _ {i, t}, r _ {t}) - \min _ {S: | S | = m} \frac {1}{m} \sum_ {t = 1} ^ {T} \sum_ {j \in S} \ell (b _ {j, t}, r _ {t}) ] \leq \mathcal {O} (\sqrt {B T \ln (\frac {K}{m})}) + \frac {8 B T}{\eta - 2 B}.
$$
Given that  $\eta = \sqrt{\frac{BT}{\ln(\frac{K}{m})}}$  in Theorem 4, the expected regret bound is  $\mathcal{O}(\sqrt{BT\ln(\frac{K}{m})})$ . This result is summarized in the following theorem.
Theorem 5. For the FTPL algorithm with noise distribution  $\mathcal{D}$  satisfying Condition 1 with the constant  $B > 0$ , if we set  $\eta = \sqrt{\frac{BT}{\ln(\frac{K}{m})}}$ , the regret bound is  $\mathcal{O}(\sqrt{BT\ln(\frac{K}{m})})$ .
In order to ensure approximate incentive compatibility, the probability density function of the noise distribution  $f$  needs to be such that  $\frac{f(z)}{f(z + 1)}$  does not grow to infinity for very large  $z$ . One way to enforce this condition is via a Lipschitzness assumption on  $\ln f$ . Condition 1 implies that  $\ln f$  is  $B$ -Lipschitz. That is why smaller values of  $B$  lead to better approximate incentive compatibility which in turn results in smaller regret bounds (given that the term  $4TC$  appears in the regret bound where  $C$  is the bound on the approximate incentive-compatibility derived in Theorem 3).
We can use the FTPL algorithm to obtain results for the partial information setting as well. [20] showed that if the hazard rate of the noise distribution is bounded by  $B$ , applying the FTPL algorithm to the partial information setting for the 1-expert problem leads to  $\mathcal{O}(\sqrt{BKT\ln K})$  regret bounds. Using the result of Theorem 2, we know that if Condition 1 holds, the hazard rate is bounded. Therefore, if the noise distribution satisfies Condition 1, FTPL applied to the  $m$ -experts problem is approximately incentive-compatible and achieves  $\mathcal{O}(\sqrt{BKT\ln(\frac{K}{m})})$  regret bound.
5 Online distorted greedy algorithm
In this section, we study the setting where the utility function is submodular. In this case, we have  $f_{t}(S_{t}) = 1 - \prod_{i\in S_{t}}\ell_{i,t} = 1 - \prod_{i\in S_{t}}\ell (b_{i,t},r_{t})$ . The problem in this setting could be
written as an online monotone submodular maximization problem subject to a cardinality constraint of size  $m$ . [21] proposed the online greedy algorithm for this problem whose  $(1 - \frac{1}{e})$ -regret is bounded by  $\mathcal{O}(\sqrt{mT\ln K})$ . The algorithm works as follows: There are  $m$  instantiations  $\mathcal{A}_1, \ldots, \mathcal{A}_m$  of no-regret algorithms for the 1-expert problem. At each round  $t \in [T]$ ,  $\mathcal{A}_i$  selects an expert  $v_{i,t} \in [K]$  and the set  $S_{t} = \{v_{1,t}, \ldots, v_{m,t}\}$  is selected.  $\mathcal{A}_i$  observes the reward  $f_{t}(v_{i,t}|S_{i-1,t})$  where  $S_{j,t} = \{v_{1,t}, \ldots, v_{j,t}\}$ .
Inspired by Algorithm 1 of [13] for online monotone submodular maximization subject to a matroid constraint, we propose the online distorted greedy in Algorithm 1 for the special case of a cardinality constraint. The algorithm is similar to the online greedy algorithm of [21] discussed above. However, in the online distorted greedy algorithm, after choosing the set  $S_{t}$  and observing the function  $f_{t}$ , we first compute the modular lower bound  $h_t$  defined as  $h_t(S) = \sum_{i\in S}f_t(i|[K]\setminus \{i\})$ . We define  $g_{t} = f_{t} - h_{t}$ . Note that  $g_{t}$  is monotone submodular as well. The reward of  $\mathcal{A}_i$  for choosing  $v_{i,t}$  at round  $t$  is  $(1 - \frac{1}{m})^{m - i}g_t(v_{i,t}|S_{i - 1,t}) + h_t(v_{i,t})$  (in case  $v_{i,t}\in S_{i - 1,t}$ , we repeatedly take samples from the weight distribution of  $\mathcal{A}_i$  over the experts until we observe an expert not in the set  $S_{i - 1,t}$ ). This technique was first introduced by [22] for the corresponding offline problem and it allows us to obtain  $(1 - \frac{c_f}{e})$ -regret bounds (where  $f = \sum_{t = 1}^{T}f_{t}$ ) with the optimal approximation ratio (optimality was shown by [23]) instead of the  $(1 - \frac{1}{e})$ -regret bounds for the online greedy algorithm.
One particular choice for  $\{\mathcal{A}_i\}_{i=1}^m$  is the WSU algorithm of [10]. We summarize the result for this choice in the theorem below.
Theorem 6. For all  $i \in [m]$ , let  $\mathcal{A}_i$  be an instantiation of the WSU algorithm of [10] for the 1-expert problem and denote  $f = \sum_{t=1}^{T} f_t$ . The online distorted greedy algorithm applied to the  $m$ -experts problem obtains the following regret bound:
$$
\mathbb {E} [ (1 - \frac {c _ {f}}{e}) \text {-} R _ {T} ] \leq \sum_ {i = 1} ^ {m} R _ {T} ^ {(i)},
$$
where  $R_T^{(i)}$  is the regret of algorithm  $\mathcal{A}_i$ . If  $c_f = 0$ , the algorithm is incentive-compatible.
If  $c_{f} \in (0,1]$ , we can use the online greedy algorithm of [21] instead to ensure incentive compatibility while maintaining similar bounds for the  $(1 - \frac{1}{e})$ -regret. [10] provided  $\mathcal{O}(\sqrt{T\ln K})$  regret bounds for the WSU algorithm. If we plug in this bound in the result of Theorem 6, the regret bound of the online distorted greedy algorithm is  $\mathcal{O}(m\sqrt{T\ln K})$ . However, this bound depends linearly on  $m$  which is suboptimal. To remedy this issue, we first provide an adaptive regret bound for the WSU algorithm below.
Theorem 7. The regret of the WSU algorithm of [10] is bounded by  $\mathcal{O}(\sqrt{|L_T| \ln K} + \ln K)$  where  $|L_T|$  is the cumulative absolute loss of the algorithm.
Note that the bound in Theorem 7 adapts to the hardness of the problem. In the worst case, we have  $|L_{T}| = T$  and recover the  $\mathcal{O}(\sqrt{T\ln K})$  bound proved in [10]. However, for smaller values of  $|L_{T}|$ , the bound in Theorem 7 improves that of [10]. For the non-strategic setting with modular utilities, [7] proposed the Component Hedge (CH) algorithm and obtained a regret bound of  $\sqrt{2m\ell^{*}\ln(\frac{K}{m})} + m\ln (\frac{K}{m})$  where  $\ell^{*}$  is the cumulative loss of the best-chosen set in hindsight. They also gave a matching lower bound for this problem. Applying the same analysis as in Theorem 7 to the setting of the naive approach with  $\binom{K}{m}$  meta-experts, we can show that the regret bound of WSU matches the aforementioned lower bound.
We can use the above adaptive regret bound to improve the regret bound of the online distorted greedy algorithm. First, note that at round  $t \in [T]$ , the sum of the absolute value of losses incurred by  $\{\mathcal{A}_i\}_{i=1}^m$  is bounded as follows:
$$
\sum_ {i = 1} ^ {m} \left((1 - \frac {1}{m}) ^ {m - i} g _ {t} (v _ {i, t} | S _ {i - 1, t}) + h _ {t} (v _ {i, t})\right) \leq \sum_ {i = 1} ^ {m} \underbrace {\left(g _ {t} (v _ {i , t} | S _ {i - 1 , t}) + h _ {t} (v _ {i , t})\right)} _ {= f _ {t} (v _ {i, t} | S _ {i - 1, t})} = f _ {t} (S _ {t}) \leq 1.
$$
Therefore, if we denote the cumulative absolute losses incurred by  $\mathcal{A}_i$  with  $|L_T^{(i)}|$ , we have:
$$
\sum_ {i = 1} ^ {m} \left| L _ {T} ^ {(i)} \right| \leq T.
$$
Algorithm 1 Online distorted greedy algorithm
Initialization: Initialize  $m$  instances  $\mathcal{A}_1,\ldots ,\mathcal{A}_m$  of online algorithms for the 1-expert problem.   
for  $t = 1,\dots ,T$  do   
for  $i = 1,\dots ,m$  do   
 $\mathcal{A}_i$  chooses the expert  $v_{i,t}$  and  $S_{i,t} = \{v_{1,t},\dots ,v_{i,t}\}$    
end for   
Play the set  $S_{t} = S_{m,t} = \{v_{1,t},\dots ,v_{m,t}\}$  and observe  $f_{t}$    
Compute the modular function  $h_t(S) = \sum_{i\in S}f_t(i|[K]\setminus \{i\})$  and set  $g_{t} = f_{t} - h_{t}$    
for  $i = 1,\dots ,m$  do   
Feedback the cost  $-(1 - \frac{1}{m})^{m - i}g_t(v_{i,t}|S_{i - 1,t}) - h_t(v_{i,t})$  to  $\mathcal{A}_i$    
end for   
end for
Using the result of Theorem 7, we know that the regret bound of the online distorted greedy algorithm is  $\sum_{i=1}^{m} \sqrt{|L_T^{(i)}| \ln K} + m \ln K$ .  $\sum_{i=1}^{m} \sqrt{|L_T^{(i)}}$  is maximized when  $|L_T^{(i)}| = \frac{T}{m}$  for all  $i \in [m]$ . Thus, in the worst case, the expected  $(1 - \frac{c_f}{e})$ -regret bound is  $\mathcal{O}(\sqrt{mT \ln K} + m \ln K)$ .
While we focused on submodular utility functions in this section, we can also apply the online distorted greedy algorithm to the setting with modular utilities. In this case, we have  $c_{f} = 0$ , and therefore, the algorithm is incentive-compatible and its 1-regret is bounded by  $\mathcal{O}(\sqrt{mT\ln K} + m\ln K)$ . Unlike the FTPL algorithm that is only approximately incentive-compatible, the online distorted greedy algorithm applied to modular utility functions is incentive-compatible. However, this comes at the price of an extra  $\sqrt{m}$  term in the regret bound.
In this section, we evaluate the performance of our proposed algorithms for modular utility functions on a publicly available dataset from a FiveThirtyEight forecasting competition $^4$  in which forecasters make predictions about the match outcomes of the 2022-2023 National Football League (NFL). Before each match, FiveThirtyEight provides information on the past performance of the two opposing teams. Forecasters observe this information and make probabilistic predictions about the likelihood of each team winning the match. Considering that there are 284 different matches in the dataset, we set  $T = 284$ . Out of the 9982 forecasters who participated in this competition, only 274 made predictions for every single match. We consider two cases:  $K = 20$  and  $K = 100$ . To reduce variance, for each case, we sample 5 groups of  $K$  forecasters from the 274 and run FTPL and Online Distorted Greedy (ODG) 10 times. We set  $m = 5$ . Given that FTPL is only approximately incentive-compatible and according to the result of Theorem 3, the reported beliefs could be  $\frac{2B}{\eta - 2B}$  distant from the true beliefs, we add a uniformly random value in the range  $\left[\frac{-2B}{\eta - 2B}, \frac{2B}{\eta - 2B}\right]$  to the true beliefs to model this fact. We use the standard Laplace distribution as the perturbation for FTPL. Hence, we set  $B = 1$ . For both algorithms, the step size  $\eta$  is chosen according to our theoretical results. In Figure 1, we plot the average regret  $\frac{1}{t}\mathbb{E}\left[\max_{S\subseteq [K]:|S| = m}\sum_{\tau = 1}^{t}f_{\tau}(S) - \sum_{\tau = 1}^{t}f_{\tau}(S_{\tau})\right]$  of the two algorithms over time (along with error bands corresponding to 20th and 80th percentiles) along with that of the FiveThirtyEight aggregated predictions for  $K = 20$  and  $K = 100$  settings. Note that while our proposed algorithms choose  $m$  predictions at each round  $t\in [T]$ , the FiveThirtyEight aggregated prediction is a single scalar value. The plots suggest that while the regret of all three algorithms converges to zero as  $t$  gets larger, both our proposed algorithms have superior performance compared to that of the FiveThirtyEight predictions.
In this paper, we studied the  $m$ -experts problem, a generalization of the standard binary prediction with expert advice problem where at each round  $t \in [T]$ : 1) The algorithm is allowed to pick  $m \geq 1$  experts and its utility is a modular or submodular function of the chosen experts, and 2) The experts are strategic and may misreport their true beliefs about the  $t$ -th event to increase their probability of
$^{4}$ https://github.com/fivethirtyeight/nfl-elo-game
Figure 1: Running average of regret over time for (a)  $K = 20$  and (b)  $K = 100$ .
being chosen at the next round (round  $t + 1$ ). The goal is to design algorithms that incentivize experts to report truthfully (i.e., incentive-compatible) and obtain sublinear regret bounds with respect to the true beliefs of the experts (i.e., no-regret). We proposed two algorithmic frameworks for this problem. In Section 4, we introduced the Follow the Perturbed Leader (FTPL) algorithm. Under a certain condition for the noise distribution (Condition 1), this algorithm is approximately incentive-compatible and achieves sublinear regret bounds for modular utility functions. Moreover, in Section 5, we proposed the online distorted greedy algorithm that applies to both modular and submodular utility functions. This algorithm is incentive-compatible but its regret bound is slightly worse than that of FTPL.
This work could be extended in several interesting directions. First, none of the algorithms discussed here or in prior works have taken into account the properties of the quadratic loss function. In particular, this loss function is exp-concave, and [4] showed that for exp-concave loss functions in the 1-expert problem, the regret bound could be improved to  $\mathcal{O}(\ln K)$  using the Hedge algorithm without the incentive compatibility property. Designing incentive-compatible algorithms with similarly improved regret bounds for the 1-expert and  $m$ -experts problems is yet to be done. In order to obtain the  $\mathcal{O}(\ln K)$  regret bound for the 1-expert problem (with squared loss) using the Hedge algorithm, the algorithm makes a single prediction  $\sum_{i=1}^{K} \pi_{i,t} p_{i,t}$  at round  $t \in [T]$  and its loss is  $(\sum_{i=1}^{K} \pi_{i,t} p_{i,t} - r_t)^2$ . In other words, choosing an expert  $i \in [K]$  with probability  $\pi_{i,t}$  at round  $t$  is not good enough to obtain the improved  $\mathcal{O}(\ln K)$  regret bound. Moving on to the  $m$ -expert problem, the main challenge for obtaining regret bounds better than  $\mathcal{O}(\sqrt{T})$  is to decide how to aggregate the  $K$  predictions as  $m$  scalar values. Second, while we focused on the particular choice of quadratic loss functions, the setting could be extended to other loss functions as well. It is not clear to what extent our results hold when moving beyond the quadratic loss function. Finally, [16] introduced a framework for augmenting online algorithms for various online problems with predictions or pieces of advice. An interesting future research direction is to extend this setting to the case where the predictions are given by strategic experts and study incentive compatibility guarantees for online problems beyond the  $m$ -experts problem.
This work was supported in part by the following grants: NSF TRIPODS II 2023166, NSF CIF 2212261, NSF CIF 2007036, NSF AI Inst 2112085.
[1] Volodimir G Vovk. Aggregating strategies. In Annual Workshop on Computational Learning Theory: Proceedings of the third annual workshop on Computational learning theory, 1990. Association for Computing Machinery, Inc, 1990.
[23] Maxim Sviridenko, Jan Vondrák, and Justin Ward. Optimal approximation for submodular and supermodular optimization with bounded curvature. Mathematics of Operations Research, 42(4):1197-1218, 2017.
At time  $t = 1$ , we set  $\pi_{S,1} = \frac{1}{\binom{K}{m}}$  for each set  $S$  where  $|S| = m$  and update these weights as follows:
$$
\pi_ {S, t + 1} = \pi_ {S, t} (1 - \eta L _ {S, t}),
$$
where  $L_{S,t} = \ell_{S,t} - \sum_{S':|S'| = m}\pi_{S',t}\ell_{S',t}$  and  $\ell_{S,t} = \frac{1}{m}\sum_{i\in S}\ell_{i,t}$ . If we denote the optimal set as  $S^{*}$ , we can write:
$$
1 \geq \pi_ {S ^ {*}, T + 1} = \frac {1}{\binom {K} {m}} \prod_ {t = 1} ^ {T} (1 - \eta L _ {S ^ {*}, t}).
$$
Taking the natural logarithm of both sides, we have:
$$
0 \geq - \ln \binom {K} {m} + \sum_ {t = 1} ^ {T} \ln (1 - \eta L _ {S ^ {*}, t}).
$$
We can use the inequality  $\ln (1 - x) \geq -x - x^2$  for  $x \leq \frac{1}{2}$  (we choose  $\eta$  later such that this inequality holds) to obtain:
$$
0 \geq - \ln \binom {K} {m} - \eta \sum_ {t = 1} ^ {T} L _ {S ^ {*}, t} - \eta^ {2} \sum_ {t = 1} ^ {T} L _ {S ^ {*}, t} ^ {2}.
$$
Rearranging the terms and dividing both sides by  $\eta$ , we have:
$$
- \sum_ {t = 1} ^ {T} L _ {S ^ {*}, t} \leq \frac {\ln ( \begin{array}{c} K \\ m \end{array} )}{\eta} + \eta \sum_ {t = 1} ^ {T} L _ {S ^ {*}, t} ^ {2}.
$$
Using the fact that  $R_{T} = -\sum_{t=1}^{T} L_{S^{*},t}$ , the inequality  $\binom{K}{m} \leq \left(\frac{Ke}{m}\right)^{m}$  and  $|L_{S,t}| \leq 1 \forall S, t$ , we can write:
$$
R _ {T} \leq \frac {m \ln (\frac {K e}{m})}{\eta} + \eta T.
$$
Setting  $\eta = \sqrt{\frac{m\ln(\frac{Ke}{m})}{T}}$ , we obtain the regret bound  $\mathcal{O}(\sqrt{mT\ln(\frac{K}{m})})$ . Note that the assumption  $\eta L_{S^{*},t}\leq \frac{1}{2}$  (used in the proof) holds if  $T\geq 4m\ln (\frac{Ke}{m})$ . Therefore, we assume  $T$  is large enough to satisfy this inequality.
We can show that if  $B = \max_{z} |\nu'(z)|$ , the hazard rate of the distribution is bounded above by  $B$  as well. To see this, fix  $x \geq 0$ . Note that  $\frac{f(-x)}{1 - F(-x)} = \frac{f(x)}{1 - F(-x)} \leq \frac{f(x)}{1 - F(x)}$  for a symmetric zero-mean distribution and therefore, we only need to bound the hazard rate at  $x > 0$  to bound  $\mathrm{haz}_{\mathcal{D}}$ . We can
$$
\begin{array}{l} \frac {f _ {\mathcal {D}} (x)}{1 - F _ {\mathcal {D}} (x)} = \frac {f _ {\mathcal {D}} (x)}{\int_ {x} ^ {\infty} f _ {\mathcal {D}} (z) d z} \\ = \frac {\exp (- \nu (x))}{\int_ {x} ^ {\infty} \exp (- \nu (z)) d z} \\ = \frac {1}{\int_ {x} ^ {\infty} \exp (\nu (x) - \nu (z)) d z} \\ \stackrel {\text {(a)}} {=} \frac {1}{\int_ {x} ^ {\infty} \exp ((x - z) \nu^ {\prime} (z _ {x})) d z} \\ \stackrel {(b)} {\leq} \frac {1}{\int_ {x} ^ {\infty} \exp (B (x - z)) d z} \\ = \frac {1}{(1 / B)} \\ = B, \\ \end{array}
$$
where we have used the mean-value theorem in (a) and  $z_{x}$  is in the line segment between  $x$  and  $z$ . Also, note that since  $x, z > 0$ ,  $z_{x} > 0$  holds as well and therefore,  $\nu'(z_{x}) > 0$ . We have used this fact along with  $x - z < 0$  to obtain (b). Therefore, we have  $\mathrm{haz}_{\mathcal{D}} \leq B$ .
Let's fix round  $t \in [T]$  and expert  $i \in [K]$ . For  $j \neq i$ , denote  $x_{j,0}^{(t)} = \sum_{s=1}^{t-1} \ell_{j,s} + p_{j,t}^2 + \eta \gamma_{j,t}$  as the total losses of expert  $j$  up to round  $t$  plus noise if  $r_t = 0$ . Similarly, we can define  $x_{j,1}^{(t)} = \sum_{s=1}^{t-1} \ell_{j,s} + (1 - p_{j,t})^2 + \eta \gamma_{j,t}$ . Define  $X_0^{(t)}(X_1^{(t)})$  as the  $m$ -th smallest value in  $\{x_{j,0}^{(t)}\}_{j \neq i} (\{x_{j,1}^{(t)}\}_{j \neq i})$ . Note that  $|X_0^{(t)} - X_1^{(t)}| \leq 1$  holds because for each  $j \neq i$ , we have  $|x_{j,0}^{(t)} - x_{j,1}^{(t)}| \leq 1$ . Also, let  $L = \sum_{s=1}^{t-1} \ell_{i,s}$ . If  $r_t = 0$ , expert  $i$  is chosen at round  $t+1$  if and only if  $L + p_{i,t}^2 + \eta \gamma_{i,t} < = X_0^{(t)}$ . Similarly, for the case  $r_t = 1$ , expert  $i$  is chosen if and only if  $L + (1 - p_{i,t})^2 + \eta \gamma_{i,t} < = X_1^{(t)}$ . Rearranging the terms, we can write: