SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Proposition 4.1 (Follows from (Neyman and Roughgarden, 2023, Theorem 5.1)). Let  $\mathbf{p}^1, \ldots, \mathbf{p}^m$  be forecasts over  $n$  outcomes,  $j \in [n]$  be an outcome, and  $\mathbf{w} \in \Delta^m$  be a weight vector. Let  $\mathbf{p}^*(\mathbf{w})$  be the logarithmic pool of the forecasts with weight vector  $\mathbf{w}$  and let  $L(\mathbf{w}) \coloneqq -\ln(p_j^*(\mathbf{w}))$  be the log loss of  $\mathbf{p}^*(\mathbf{w})$  if Outcome  $j$  is realized. Then  $L$  is a convex function.
In particular, all of our loss functions  $L^t$  are convex, which means that standard regret bounds apply. In particular, to bound the expected regret of Algorithm 1, we will use a well-known regret bound for follow the regularized leader (FTRL) with linearized losses (Hazan, 2021, Lemma 5.3), which in our case is equivalent to OMD.<sup>6</sup>
Lemma 4.2 (Follows from (Hazan, 2021, Lemma 5.3)). If  $\eta_t = \eta$  for all  $t$ , the regret of Algorithm 1 is at most
$$
\frac {1}{\eta} \left(\max _ {\mathbf {w} \in \Delta^ {m}} R (\mathbf {w}) - \min _ {\mathbf {w} \in \Delta^ {m}} R (\mathbf {w})\right) + \sum_ {t = 1} ^ {T} \nabla L ^ {t} (\mathbf {w} ^ {t}) \cdot (\mathbf {w} ^ {t} - \mathbf {w} ^ {t + 1}).
$$
Informally, this bound means that if the vectors  $\nabla L^t (\mathbf{w}^t)$  are small in magnitude, our regret is also small. Conversely, if some  $\nabla L^{t}(\mathbf{w}^{t})$  is large, this may be bad for our regret bound. We expect the gradient of the loss to be large if some expert is very wrong (assigns a very low probability to the correct outcome), since the loss would then be steeply increasing as a function of that expert's weight. Fortunately, the calibration property guarantees this to be unlikely. Specifically, we define the small gradient assumption as follows.
Definition 4.3. Define  $\gamma \coloneqq 12n\ln T$ . The small gradient assumption holds for a particular run of Algorithm 1 if for every  $t\in [T]$  and  $i\in [m]$ , we have
$$
- \frac {\gamma}{w _ {i} ^ {t}} \leq \partial_ {i} L ^ {t} (\mathbf {w} ^ {t}) \leq \gamma ,
$$
where  $\partial_i$  denotes the partial derivative with respect to the  $i$ -th weight.
In Section 4.4, we prove that the small gradient assumption is very likely to hold. This is a key conceptual step in our proof, as it is where we leverage the calibration property to prove bounds that ultimately let us bound our algorithm's regret. We then use the low likelihood of the small gradient assumption failing in order to bound the contribution to the expected regret from the case where the assumption fails to hold.
In Sections 4.2 and 4.3, we bound regret under the condition that the small gradient assumption holds. We show that under the assumption, for all  $i,t$  we have  $(w_i^t)^\alpha \geq \eta$ . Consequently,  $\eta = \frac{1}{\sqrt{T}\ln T} \cdot \frac{1}{12m^{(1 + \alpha) / 2}n}$  at all time steps, so we can apply Lemma 4.2. The first term in the bound is  $O(1 / \eta) = O(\sqrt{T}\log T)$ . As for the summation term, we upper bound it by keeping track of the following quantity:
$$
\varphi (t) := \sum_ {s = 1} ^ {t} \nabla L ^ {s} (\mathbf {w} ^ {s}) \cdot \left(\mathbf {w} ^ {s} - \mathbf {w} ^ {s + 1}\right) + 1 9 m ^ {2} \gamma^ {2} \eta (T - t) - 4 \gamma \sum_ {i = 1} ^ {m} \ln w _ {i} ^ {t + 1}.
$$
The first term is exactly the summation in Lemma 4.2 up through step  $t$ . The  $19m^2\gamma^2\eta$  is something akin to an upper bound on the value of  $L^{t}(\mathbf{w}^{t})\cdot (\mathbf{w}^{t} - \mathbf{w}^{t + 1})$  at a given time step (times  $T - t$  remaining time steps). This upper bound is not strict: in particular, large summands are possible when some weights are small (because of the fact that the lower bound in the small gradient assumption is inversely proportional to  $w_{i}^{t}$ ). However, attaining a large summand requires these small weights to increase, thus "spending potential" for future large summands. The last term keeps track of this potential.
We show that under the small gradient assumption,  $\varphi(t)$  necessarily decreases with  $t$ . This argument, which we give in Section 4.3, is another key conceptual step, and is arguably the heart of the proof. Since  $\varphi(T)$  is equal to the summation term in Lemma 4.2 (plus a positive number), and  $\varphi(0) \geq \varphi(T)$ , the summation term is less than or equal to  $\varphi(0)$ , which is at most  $O(m^{(3 - \alpha)/2} \sqrt{T} \log T)$ . This completes the proof.
This equivalence is due to our choice of regularizer, as we never need to project  $\mathbf{w}^t$ .
See Equation 8 for an expression of this quantity in terms of the experts' reports and weights.
4.2 Bounds on  $\mathbf{w}^t$  under the small gradient assumption
In this section, we state bounds on expert weights and how quickly they change from one time step to the next, conditional on the small gradient assumption. We use the following lemma, whose proof we defer to Appendix A.
Lemma 4.4. Consider a particular run of Algorithm 1. Let  $\zeta$  be a constant such that  $-\frac{\zeta}{w_i^t} \leq \partial_i L^t(\mathbf{w}^t) \leq \zeta$  for all  $i, t$ . Then for every  $i, t$ , we have
$$
(w _ {i} ^ {t}) ^ {\alpha - 1} - \left(\frac {1}{w _ {i} ^ {t}} + 1\right) \eta_ {t} \zeta \leq (w _ {i} ^ {t + 1}) ^ {\alpha - 1} \leq (w _ {i} ^ {t}) ^ {\alpha - 1} + \left(\frac {1}{\min _ {k} w _ {k}} + 1\right) \eta_ {t} \zeta .
$$
Furthermore, if  $\eta_t\zeta \leq (1 - \alpha)^2(w_i^t)^\alpha$  for all  $i$ , then for every  $i$  we have
$$
\left(w _ {i} ^ {t + 1}\right) ^ {\alpha - 1} \leq \left(w _ {i} ^ {t}\right) ^ {\alpha - 1} + (m + 1) \eta_ {t} \zeta .
$$
Intuitively, this result states that when the gradient update is small,  $w_{i}^{t + 1}$  is not too different from  $w_{i}^{t}$ . Note that the lower bound  $-\frac{\zeta}{w_i^t}$  that we place on the gradient is not a simple Lipschitz bound but instead depends on  $w_{i}^{t}$ ; this makes the bounds in Lemma 4.4 less straightforward to prove. In particular, we bound each component  $w_{i}^{t}$  individually, using bounds on the gradient of the loss for all other components and convexity arguments.
Lemma 4.4 can be translated into bounds on each  $w_{i}^{t}$  and on the change between  $w_{i}^{t}$  and  $w_{i}^{t + 1}$ :
Corollary 4.5. Under the small gradient assumption, for sufficiently large  $T$  we have for all  $i \in [m], t \in [T]$  that:
(#1)  $(w_{i}^{t})^{\alpha}\geq 4\eta \gamma$  and  $w_{i}^{t}\geq \frac{1}{10\sqrt{m}} T^{1 / (2(\alpha -1))}$
(#2)  $-32(w_i^t)^{1 - \alpha}\eta \gamma \leq w_i^t -w_i^{t + 1}\leq 2(w_i^t)^{2 - \alpha}(m + 1)\eta \gamma .$
We defer the proof of Corollary 4.5 to Appendix A. The key idea for (#1) is to proceed by induction on  $t$  on the two sub-statements in parallel: so long as  $(w_i^t)^\alpha \geq 4\eta \gamma$ , we may use the second part of Lemma 4.4 with  $\zeta = \gamma$  to bound  $(w_i^{t + 1})^{\alpha -1}$  in terms of  $(w_i^t)^{\alpha -1}$ , which we can leverage to prove both sub-statements for  $t + 1$ . (#2) then follows from (#1) by routine (though nontrivial) algebra.
Armed with the bounds of Corollary 4.5, we are now able to show that under the small gradient assumption, Algorithm 1 attains vanishing regret.
4.3 Bounding regret under the small gradient assumption
Assume the small gradient assumption. Note that since  $4\gamma \geq 1$ , by Corollary 4.5 (#1) we have that  $\eta_t = \eta$  for all  $t$ . This means that we may apply the bound in Lemma 4.2, and in particular we have
$$
\frac {1}{\eta} \left(\max _ {\mathbf {w} \in \Delta^ {m}} R (\mathbf {w}) - \min _ {\mathbf {w} \in \Delta^ {m}} R (\mathbf {w})\right) = \frac {1}{\eta} \cdot \frac {m}{\alpha} \left(\frac {1}{m}\right) ^ {\alpha} = \frac {m ^ {1 - \alpha}}{\alpha \eta} = \frac {1 2}{\alpha} m ^ {(3 - \alpha) / 2} n \sqrt {T} \ln T.
$$
It remains to bound the summation component of the regret bound in Lemma 4.2. To do so, we prove the following lemma, which we alluded to in Section 4.1 as the heart of the proof of Theorem 3.2.
Lemma 4.6. For  $t \in \{0,1,\dots,T\}$ , let
Under the small gradient assumption, for sufficiently large  $T$ ,  $\varphi(t)$  is a decreasing function of  $t$ .
To prove this claim, consider a particular  $t \in [T]$ . We may write
$$
\varphi (t) - \varphi (t - 1) = \sum_ {i = 1} ^ {m} \left(\left(w _ {i} ^ {t} - w _ {i} ^ {t + 1}\right) \partial_ {i} L ^ {t} \left(\mathbf {w} ^ {t}\right) - 1 9 m \gamma^ {2} \eta + 4 \gamma \left(\ln w _ {i} ^ {t} - \ln w _ {i} ^ {t + 1}\right)\right) \tag {1}
$$
and we wish to show that this quantity is negative. In fact, we show that the contribution from every  $i \in [m]$  is negative. The key idea is to consider two cases:  $w_i^{t+1} \leq w_i^t$  and  $w_i^{t+1} \geq w_i^t$ . In each case, Corollary 4.5 provides an upper bound on the magnitude of the difference between  $w_i^t$  and  $w_i^{t+1}$ . If  $w_i^{t+1} \leq w_i^t$  then the first and third terms in the summation are positive but small, and are dominated by the middle term. If  $w_i^{t+1} \geq w_i^t$  then the first term may be quite large, because of the asymmetric bound in the small gradient assumption (and the consequently asymmetric bound in Corollary 4.5). However, in this case the contribution of the third term is very negative, enough to make the overall expression negative. In this sense, the third term keeps track of unspent potential for future regret, which gets "spent down" whenever a large amount of regret is realized (as measured by the first term).
We now prove formally that each term of the summation in Equation 1 is negative.
Proof. First assume that  $w_i^t - w_i^{t+1} \leq 0$ . Note that by combining (#1) and (#2) of Corollary 4.5, we have
$$
w _ {i} ^ {t + 1} - w _ {i} ^ {t} \leq 3 2 \left(w _ {i} ^ {t}\right) ^ {1 - \alpha} \eta \gamma \leq 8 w _ {i} ^ {t}.
$$
By the small gradient assumption we have that
$$
(w _ {i} ^ {t} - w _ {i} ^ {t + 1}) \partial_ {i} L ^ {t} (\mathbf {w} ^ {t}) \leq \frac {\gamma (w _ {i} ^ {t + 1} - w _ {i} ^ {t})}{w _ {i} ^ {t}}.
$$
$$
4 \gamma (\ln w _ {i} ^ {t} - \ln w _ {i} ^ {t + 1}) = - 4 \gamma \ln \left(1 + \frac {w _ {i} ^ {t + 1} - w _ {i} ^ {t}}{w _ {i} ^ {t}}\right) \leq \frac {- \gamma (w _ {i} ^ {t + 1} - w _ {i} ^ {t})}{w _ {i} ^ {t}}
$$
for  $T$  large enough. (Here we use that  $w_{i}^{t + 1} - w_{i}^{t}\leq 8w_{i}^{t}$  and that  $\ln (1 + x)\geq \frac{x}{4}$  for  $x\leq 8$ .) Thus, the first and third terms in Equation 1 are net negative; meanwhile, the second term is also negative, so the expression is negative.
Now assume that  $w_{i}^{t} - w_{i}^{t + 1} \geq 0$ . Again by the small gradient assumption, we have that
$$
(w _ {i} ^ {t} - w _ {i} ^ {t + 1}) \partial_ {i} L ^ {t} (\mathbf {w} ^ {t}) \leq \gamma (w _ {i} ^ {t} - w _ {i} ^ {t + 1}) \leq 2 (m + 1) \eta \gamma^ {2} (w _ {i} ^ {t}) ^ {2 - \alpha} \leq 3 m \eta \gamma^ {2}
$$
$$
\begin{array}{l} 4 \gamma (\ln w _ {i} ^ {t} - \ln w _ {i} ^ {t + 1}) = - 4 \gamma \ln \left(1 - \frac {w _ {i} ^ {t} - w _ {i} ^ {t + 1}}{w _ {i} ^ {t}}\right) \leq - 4 \gamma \ln (1 - 2 (m + 1) \eta \gamma (w _ {i} ^ {t}) ^ {1 - \alpha}) \\ \leq - 4 \gamma \ln (1 - 2 (m + 1) \eta \gamma) \leq - 4 \gamma \ln (1 - 3 m \eta \gamma) \leq 1 6 m \eta \gamma^ {2} \\ \end{array}
$$
for  $T$  sufficiently large, where in the last step we use that  $\ln (1 - x) \geq -\frac{4}{3} x$  for  $x > 0$  sufficiently small (and we have  $\lim_{T \to \infty} 3mn\gamma = 0$ ). Since  $16 + 3 \leq 19$ , the right-hand side of Equation 1 is negative. This concludes the proof.
Corollary 4.7. For sufficiently large  $T$ , under the small gradient assumption, the regret of Algorithm 1 is at most  $\left(240 + \frac{12}{\alpha}\right)m^{(3 - \alpha) / 2}n\sqrt{T}\ln T$ .
Proof. We have already bounded the first term in the regret bound in Lemma 4.2. It remains only to bound the second term. This term is exactly equal to  $\varphi(T) + 4\gamma \sum_{i=1}^{m} \ln w_i^{T+1} \leq \varphi(T)$ , and  $\varphi(T) \leq \varphi(0)$ , by Lemma 4.6. We have
$$
\varphi (0) = 1 9 m ^ {2} \gamma^ {2} \eta T + 4 m \gamma \ln m \leq 2 0 m ^ {2} \gamma^ {2} \eta T
$$
for sufficiently large  $T$ . Plugging in  $\gamma = 12n\ln T$  and  $\eta = \frac{1}{\sqrt{T}\ln T} \cdot \frac{1}{12m^{(1 + \alpha) / 2n}}$  concludes the proof.
4.4 The case where the small gradient assumption fails
It remains to consider the case in which the small gradient assumption does not hold. This part of the proof consists primarily of technical lemmas, which we defer to Appendix A. The key lemma is a bound on the probability that the small gradient assumption fails by a given margin:
Lemma 4.8. For any weight vector  $\mathbf{w}$ ,  $i\in [m]$ , and  $\zeta \geq 0$ , we have that
$$
\mathbb {P} \left[ \partial_ {i} L (\mathbf {w}) \geq \zeta \right] \leq n e ^ {- \zeta} \tag {2}
$$
$$
\mathbb {P} \left[ \partial_ {i} L (\mathbf {w}) \leq - \frac {\zeta}{w _ {i}} \right] \leq m n ^ {2} e ^ {- \zeta / n}. \tag {3}
$$
Note that plugging in  $\zeta = \gamma$  yields a bound of  $mT(ne^{-\gamma} + mn^2 e^{-\gamma /n})$  on the probability that the small gradient assumption fails to hold. (Since  $\gamma = 12n\ln T$ , this quantity is on the order of  $T^{-11}$ .)
The proof of Lemma 4.8 is the only part of the proof of Theorem 3.2 that uses the calibration property. While we defer the full proof to Appendix A, we highlight how the calibration property is used to prove Equation 2. In brief, it is straightforward to show that  $\partial_i L(\mathbf{w}) \leq -\ln p_j^i$ , where  $J$  is the random variable corresponding to the realized outcome. $^8$  Therefore, we have
$$
\begin{array}{l} \mathbb {P} \left[ \partial_ {i} L (\mathbf {w}) \geq \zeta \right] \leq \mathbb {P} \left[ - \ln p _ {J} ^ {i} \geq \zeta \right] = \mathbb {P} \left[ p _ {J} ^ {i} \leq e ^ {- \zeta} \right] = \sum_ {j = 1} ^ {n} \mathbb {P} \left[ J = j \& p _ {j} ^ {i} \leq e ^ {- \zeta} \right] \\ = \sum_ {j = 1} ^ {n} \mathbb {P} \left[ p _ {j} ^ {i} \leq e ^ {- \zeta} \right] \mathbb {P} \left[ J = j \mid p _ {j} ^ {i} \leq e ^ {- \zeta} \right] \leq \sum_ {j = 1} ^ {n} \mathbb {P} \left[ J = j \mid p _ {j} ^ {i} \leq e ^ {- \zeta} \right] \leq n e ^ {- \zeta}, \\ \end{array}
$$
where the last step follows by the calibration property, thus proving Equation 2.
Combining Lemma 4.8 with an analysis of our algorithm using the standard regret bound for online mirror descent (Orabona, 2021, Theorem 6.8) gives us the following result as a corollary.
Corollary 4.9. The expected total regret of our algorithm conditional on the small gradient assumption not holding, times the probability of this event, is at most  $\tilde{O}(T^{(5 - \alpha) / (1 - \alpha) - 10})$ .
It follows that the contribution to expected regret from the case that the small gradient assumption does not hold is  $\tilde{O}(T^{-1})$ , which is negligible. Together with Corollary 4.7 (which bounds regret under the small gradient assumption), this proves Theorem 3.2. As a matter of fact, Theorem 3.2 holds even if experts are only approximately calibrated. As with other details of this section, we refer the reader to Appendix A.
In this work we have considered the problem of learning optimal weights for the logarithmic pooling of expert forecasts. It quickly became apparent that under the usual fully adversarial setup, attaining vanishing regret is impossible (Example 1.1). We chose to relax the environment by imposing the constraint on the adversary that experts must be calibrated. Put otherwise, the adversary is allowed to choose a joint probability distribution over the experts' reports and the outcome however it wants to, so long as the experts' reports are calibrated, after which the realized reports and outcome are selected at random from this distribution. To our knowledge, this setting is a novel contribution to the literature on prediction with expert advice. The setting may be of independent interest: we have demonstrated that no-regret bounds are possible in this setting when they are otherwise impossible, and it seems plausible that even in settings where no-regret bounds are attainable in a fully adversarial setting, the calibration property allows for stronger results.
Another important direction for future work is learning weights for other pooling methods. In particular, because of the close connection between a proper loss function and its associated quasi-arithmetic pool, it is natural to ask for which proper loss functions it is possible to achieve vanishing regret when learning weights for quasi-arithmetic pooling with respect to the loss function. (Neyman and Roughgarden, 2023, §5) showed that the loss of a quasi-arithmetic pool is convex in the experts' weights, and therefore the usual no-regret algorithms (e.g. online gradient descent) guarantee  $O(\sqrt{T})$  regret — so long as the loss function is bounded. In this work, we extended their result to the log loss (with the associated QA pooling method, i.e. logarithmic pooling.) Extending our techniques to other unbounded loss functions is a promising avenue for future exploration.
$^{8}$ Writing out the expression for  $L(\mathbf{w})$  and differentiating leaves us with  $-\ln p_{j}^{i}$  plus a negative term - see Equation 8.
Funding in direct support of this work: NSF grant DGE-2036197; NSF grant CCF-2006737; and ARO grant W911NF1910294.
We would like to thank Christian Kroer, Rafael Frongillo, and Bo Waggoner for discussion.
Ali E. Abbas. 2009. A Kullback-Leibler View of Linear and Log-Linear Pools. Decision Analysis 6, 1 (3 2009), 25-37. https://doi.org/10.1287/deca.1080.0133
D Allard, A Comunian, and Philippe Renard. 2012. Probability aggregation methods in geoscience. Mathematical Geosciences 44, 5 (2012), 545-581.
Dirk Bergemann and Stephen Morris. 2019. Information Design: A Unified Perspective. Journal of Economic Literature 57, 1 (3 2019), 44-95. https://doi.org/10.1257/jel.20181489
Jaroslaw Blasiok, Parikshit Gopalan, Lunjia Hu, and Preetum Nakkiran. 2023. When Does Optimizing a Proper Loss Yield Calibration? CoRR abs/2305.18764 (2023). https://doi.org/10.48550/arXiv.2305.18764 arXiv:2305.18764
Nicolò Cesa-Bianchi and Gábor Lugosi. 2006. Prediction, Learning, and Games. https://doi.org/10.1017/CB09780511546921
Thomas M. Cover. 1991. Universal Portfolios. Mathematical Finance 1, 1 (1991), 1-29. https://doi.org/10.1111/j.1467-9965.1991.tb00002.x arXiv:https://onlinelibrary.wiley.com/doi/pdf/10.1111/j.1467-9965.1991.tb00002.x
Thomas M. Cover and Erik Ordentlich. 1996. Universal portfolios with side information. IEEE Trans. Inf. Theory 42, 2 (1996), 348-363. https://doi.org/10.1109/18.485708
Shrey Desai and Greg Durrett. 2020. Calibration of Pre-trained Transformers. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, EMNLP 2020, Online, November 16-20, 2020, Bonnie Webber, Trevor Cohn, Yulan He, and Yang Liu (Eds.). Association for Computational Linguistics, 295-302. https://doi.org/10.18653/v1/2020.emnlp-main.21
Dean P. Foster and Rakesh V. Vohra. 1997. Calibrated Learning and Correlated Equilibrium. Games and Economic Behavior 21, 1 (1997), 40-55. https://doi.org/10.1006/game.1997.0595
Christian Genest. 1984. A Characterization Theorem for Externally Bayesian Groups. Ann. Statist. 12, 3 (09 1984), 1100-1105. https://doi.org/10.1214/aos/1176346726
Christian Genest and James V. Zidek. 1986. Combining Probability Distributions: A Critique and an Annotated Bibliography. Statist. Sci. 1, 1 (1986), 114-135. http://www.jstor.org/stable/2245510
Geof H. Givens and Paul J. Roback. 1999. Logarithmic Pooling of Priors Linked by a Deterministic Simulation Model. Journal of Computational and Graphical Statistics 8, 3 (1999), 452-478. http://www.jstor.org/stable/1390869
Elad Hazan. 2021. Introduction to Online Convex Optimization. CoRR abs/1909.05207 (2021). arXiv:1909.05207 http://arxiv.org/abs/1909.05207
Dan Hendrycks, Norman Mu, Ekin Dogus Cubuk, Barret Zoph, Justin Gilmer, and Balaji Lakshminarayanan. 2020. AugMix: A Simple Data Processing Method to Improve Robustness and Uncertainty. In 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020. OpenReview.net. https://openreview.net/forum?id=S1gmrxHFvB
Efficiency of Algorithm 1 The only nontrivial step of the algorithm is finding the weight vector satisfying the equation on the last line of the algorithm. To do so, it is first necessary to compute the gradient of the loss. This gradient, given by Equation 8 below, can clearly be computed in time  $O(mn)$ . After that, it is necessary to find the weight vector  $\mathbf{w}^{t+1}$  that satisfies the equation on the last line. This can be done efficiently through local search: the goal amounts to find weights  $(w_1, \ldots, w_m)$  such that the vector  $(w_1^{\alpha-1}, \ldots, w_m^{\alpha-1})$  is equal to a target vector (call it  $\mathbf{v}$ ) plus a constant  $c$  times the all-ones vector. That is, we need to simultaneously solve the equation  $w_i^{\alpha-1} = v_i + c$  for all  $i$ , with weights that add to 1. (Here, the  $v_i$  are knowns and the  $w_i$  and  $c$  are unknowns.)
We start by finding  $c$ , by solving the equation  $\sum_{i}(v_{i} + c)^{1 / (\alpha - 1)} = 1$ . Such a  $c$  exists because the left-hand side of this equation is continuous and monotone decreasing, going from infinity to zero as  $c$  ranges from  $-\min_{i} v_{i}$  to infinity. We can solve for  $c$  very efficiently, e.g. with Newton's method. Once we know  $c$ , we know each  $w_{i}$ : we have  $w_{i} = (v_{i} + c)^{1 / (\alpha - 1)}$ . Thus, Algorithm 1 takes  $O(mn)$  time.
Proof of Lemma 4.4. Fix any  $t$ . Note that since the space of possible weights is  $\Delta^m$ , it is most natural to think of  $\nabla R$  as a function from  $\Delta^m$  to  $\mathbb{R}^m / T(\mathbf{1}_m)$ , i.e.  $\mathbb{R}^m$  modulo translation by the all-ones vector (which is orthogonal to  $\Delta^m$  in  $\mathbb{R}^m$ ). That is,  $\nabla R(\mathbf{w}) = -((w_1)^{\alpha - 1}, \ldots, (w_m)^{\alpha - 1})$ , where this vector may be thought of as modulo translation by the all-ones vector. Nevertheless, we find it convenient to define  $\partial_i R(\mathbf{w}) := -(w_i)^{\alpha - 1}$ . We define  $\partial_i L^t(\mathbf{w})$  similarly (see Section 4.4).
Define  $\mathbf{h} \in \mathbb{R}^m$  to have coordinates  $h_i \coloneqq \partial_i R(\mathbf{w}^t) - \eta_t \partial_i L^t(\mathbf{w}^t)$ . Per the update rule, we have that  $h_i \equiv R(\mathbf{w}^{t+1}) \mod T(\mathbf{1}_m)$ . We have
$$
- \left(w _ {i} ^ {t}\right) ^ {\alpha - 1} - \eta_ {t} \zeta = \partial_ {i} R \left(\mathbf {w} ^ {t}\right) - \eta_ {t} \zeta \leq h _ {i} \leq \partial_ {i} R \left(\mathbf {w} ^ {t}\right) + \frac {\eta_ {t} \zeta}{w _ {i} ^ {t}} = - \left(w _ {i} ^ {t}\right) ^ {\alpha - 1} + \frac {\eta_ {t} \zeta}{w _ {i} ^ {t}} \tag {4}
$$
Applying the first and last claims of Lemma A.1 (below) with  $a = \alpha - 1$ ,  $\mathbf{v} = \mathbf{w}^t$ ,  $\kappa = \eta_t \zeta$ , and  $\mathbf{g} = -\mathbf{h}$ , we have that there exists a unique  $c \in \mathbb{R}$  such that
$$
\sum_ {i = 1} ^ {m} (- h _ {i} + c) ^ {1 / (\alpha - 1)} = 1,
$$
and in fact that  $-\eta_t\zeta \leq c \leq m\eta_t\zeta$ . (Equation 4 is relevant here because it is equivalent to the  $v_i^a - \frac{\kappa}{w_i^t} \leq g_i \leq v_i^a + \kappa$  conditions in Lemma A.1. This is also where we use that  $\eta_t\zeta \leq (1 - \alpha)^2(w_i^t)^\alpha$ , which is equivalent to  $\kappa \leq a^2 v_i^{a+1}$ .) The significance of this fact is that  $(-h_i + c)^{1 / (\alpha - 1)}$  is precisely  $w_i^{t+1}$ , since (in  $\mathbb{R}^m$ ) we have that  $(\partial_i R(\mathbf{w}^{t+1}), \ldots, \partial_i R(\mathbf{w}^{t+1})) = \mathbf{h} - c \cdot \mathbf{1}$  for some  $c$ , and in particular this  $c$  must be such that  $\sum_i w_i^{t+1} = 1$ . In particular, this means that for all  $i$ , we have
$$
(w _ {i} ^ {t + 1}) ^ {\alpha - 1} = - h _ {i} + c \leq (w _ {i} ^ {t}) ^ {\alpha - 1} + \eta_ {t} \zeta + \frac {1}{\min _ {k} w _ {k}} \eta_ {t} \zeta = (w _ {i} ^ {t}) ^ {\alpha - 1} + \left(\frac {1}{\min _ {k} w _ {k}} + 1\right) \eta_ {t} \zeta .
$$