SlowGuess/NeurIPS_2023_OCR huggingface dataset:
We can instantiate the bound in Theorem 3.4 in terms of  $T$  and  $\sigma$  for two particularly interesting cases: when  $\mathrm{VC}(\mathcal{F},\epsilon)$  scales as  $d\log \left(1 / \epsilon\right)$  (often referred to as parametric classes) and when  $\mathrm{VC}(\mathcal{F},\epsilon)$  scales as  $\epsilon^{-p}$  (often referred to as nonparametric classes). A canonical example of the former are VC classes; for a class with VC dimension  $d$ ,  $\mathrm{VC}(\mathcal{F}^{\mathrm{VC}},\epsilon) = Cd\log \left(\frac{1}{\epsilon}\right)$  (see for example [Vershynin, 2018, Theorem 8.3.18]). A canonical example of the latter are functions of bounded variation,  $\mathcal{F}^{\mathrm{BV}}$  which have  $\mathrm{VC}(\mathcal{F}^{\mathrm{BV}},\epsilon) = \frac{C}{\epsilon}$  (see for example [Musayeva, 2020, Bartlett et al., 1997]). This class is known to have unbounded sequential covering numbers [Rakhlin et al., 2010] and therefore is not learnable with a worst-case adversary—this can be seen as a simple consequence of the fact that  $\mathcal{F}^{\mathrm{BV}}$  contains all one-dimensional thresholds.
Corollary 3.4.1 (Rates for parametric and nonparametric classes). If  $\operatorname{VC}(\mathcal{F},\epsilon) = d\log \left(1 / \epsilon\right)$ , then for a large enough  $T$
$$
\mathcal {R} _ {T} (\mathcal {F}, \sigma) \leq O \left(d \cdot \operatorname {p o l y} \log \left(\frac {T}{\sigma}\right)\right).
$$
If  $\operatorname{VC}(\mathcal{F},\epsilon) = \epsilon^{-p}$ , then
$$
\mathcal {R} _ {T} (\mathcal {F}, \sigma) \leq O \left(T ^ {\frac {p}{p + 1}} \cdot \operatorname {p o l y} \log \left(\frac {T}{\sigma}\right)\right).
$$
In particular, note that Corollary 3.4.1 shows that for VC classes  $\mathcal{R}_T(\mathcal{F}^{\mathrm{VC}},\sigma) = \widetilde{O}\left(d\cdot \log \left(\frac{T}{\sigma}\right)\right)$  (tight, see also concurrent work [Wu et al., 2023] for a similar bound) and for functions of bounded variation  $\mathcal{R}_T(\mathcal{F}^{\mathrm{BV}},\sigma) = \widetilde{O}\left(\sqrt{T}\log \left(\frac{1}{\sigma}\right)\right)$ ; note that the minmax rates for the worst-case adversary scale as  $\Omega (T)$  for both these cases. We note that the above bound may be loose for general nonparametric classes, but should be improvable using a multiscale (chaining) version of Theorem 3.4 but we do not focus on this here.
Though the above results give satisfactory bounds in the minimax sense for many classes  $\mathcal{F}$  of interest, it is useful to consider explicit constructions of probability assignment rules. For the case of finite VC dimension, we give an explicit probability assignment rule by considering a discretization of the class and using a mixture probability assignment rule. In particular, this strategy (denoted by  $\mathcal{Q}^{\mathrm{VC}}$ ) yields optimal regret  $\mathcal{R}_T(\mathcal{F}^{\mathrm{VC}},\sigma ,\mathcal{Q}^{\mathrm{VC}})\leq O\left(d\log \left(\frac{T}{\sigma}\right)\right)$ . For the formal statements, proofs and detailed discussion see Appendix D.
3.2.2 Examples without Covering numbers
This reduction approach to characterizing minmax regret in the log-loss is interesting since all previous approaches have used covering numbers of some kind—either sequential covering numbers
or stronger notions of global covering. However, in stark contrast to the  $0/1$  loss and several other loss functions [Rakhlin et al., 2015b], covering numbers cannot capture the minmax regret for the log-loss, at least in the adversarial case. Consider the following class of functions on context set  $\mathcal{X} = \mathbb{B}_2$  (where  $\mathbb{B}_2$  denotes the unit  $\ell_2$  Euclidean ball)  $\mathcal{F}^{\mathrm{Lin}} := \left\{x \mapsto \frac{\langle x, w \rangle + 1}{2} \Bigg| w \in \mathbb{B}_2\right\}$ . For this class, Rakhlin and Sridharan [2015] construct a follow-the-regularized leader (FTRL) based algorithm achieving regret  $O(\sqrt{T})$ . However, Bilodeau et al. [2020] show an upper bound on the regret in terms of sequential covering numbers which is not improvable in general—this shows that sequential covering numbers are not adequate to capture the minmax regret rates for the log-loss. Wu et al. [2022b] further consolidate this by considering the following class, closely resembling  $\mathcal{F}^{\mathrm{Lin}}$ ,  $\mathcal{F}^{\mathrm{AbsLin}} := \left\{x \mapsto \big|\langle x, w \rangle \big| w \in \mathbb{B}_2\right\}$ . [Wu et al., 2022b, Example 2, Theorem 6] establish that the minmax regret for  $\mathcal{F}^{\mathrm{AbsLin}}$  is  $\widetilde{\Theta}(T^{2/3})$ , demonstrating the surprising fact that by a simple linear transformation of the hypothesis class (which does not change its covering number) one can obtain minmax rates that differ by a polynomial factor! On the other hand, our reduction-based approach bypasses the need for using any covering based arguments and therefore would lead to tight (at least up to poly  $\log(T/\sigma)$ ) rates.
We remark that exact characterizations of the minmax regret with log-loss (often referred to as the minmax redundancy in the information theory literature) in the no-context (adversarial) case is most often calculated by studying the so-called stochastic complexity of the class  $\mathcal{F}$  [Rissanen, 1996]. This can be extended to (worst-case) transductive learning with contexts  $x_{1},\ldots ,x_{T}$ ; in this case the minmax optimal regret for a fixed horizon is achieved by the normalized maximum likelihood (NML) probability assignment [Shtar'kov, 1987], and can be expressed as  $\underline{\mathcal{R}}_T^T (\mathcal{F}) = \max_{x_{1:T}}\log \left(\sum_{y_{1:T}\in \{0,1\} ^T}\max_{f\in \mathcal{F}}\prod_{t = 1}^{T}p_f(y_t|x_t)\right)$ . This expression has been evaluated previously for online logistic regression [Jacquet et al., 2021] and more general hypothesis classes [Wu et al., 2022b]. It is an intriguing question to understand what properties of  $\mathcal{F}$  the stochastic complexity depends on, given that the above examples illustrates that covering numbers do not capture it. Our reduction provides a technique to use such fine-grained understanding of the regret to directly lift the bounds to the more general smoothed adversary setting.
4 Oracle-Efficient Smoothed Sequential Probability Assignment
In the previous section, we consider a purely statistical perspective on the minimax value of the sequential probability assignment problem for smoothed adversaries. In this section, we will focus on an algorithmic perspective and design an algorithm that is efficiently implemented using calls to an MLE oracle. We will focus on the setting when the base measure is the uniform measure on the input space  $\mathcal{X}$  and the label space  $\mathcal{Y} = \{0,1\}$ . In this setting, we are given access to an oracle OPT which given a data set  $S = \{x_{i},y_{i}\}_{i = 1}^{m}$  outputs a hypothesis that minimizes the loss on  $S$ . That is,
$$
\operatorname {O P T} (S) = \underset {h \in \mathcal {F}} {\operatorname {a r g m i n}} \frac {1}{m} \sum_ {i = 1} ^ {m} \ell \left(h, \left(x _ {i}, y _ {i}\right)\right).
$$
In the context of the logarithmic loss, this corresponds to maximum likelihood estimation. Most of the analysis holds for a general loss function  $\ell$  (with regret scaling appropriate bounds on the values and derivatives), but for clarity one can think of  $\ell$  as the log-loss. In particular, Algorithm 1 is written for the log-loss.
The main framework we work in is the follow-the-perturbed-leader (FTPL) framework. Here, our algorithm uses the oracle on a data set consisting of the historical samples and a set of hallucinated samples. The hallucinated samples are intended to "stabilize" the predictions of the algorithm. This gives us a probability assignment  $\mathcal{D}^{\mathrm{FTPL}} = \{q^{\mathrm{FTPL}}(\cdot |x_{1:t},y_{1:t - 1})\}_{t = 1}^{T}$ .
In order to state the regret bound, we need the following notions. For any class  $\mathcal{F}$ , define the truncated class  $\mathcal{F}_{\alpha}$  as  $\mathcal{F}_{\alpha} = \left\{f_{\alpha}:f_{\alpha}(x) = \frac{f(x) + \alpha}{1 + 2\alpha}\text{ where } f\in \mathcal{F}\right\}$ . We will also need the notion of Rademacher complexity  $\operatorname{Rad}(\mathcal{F},T) = \sup_{X\subset \mathcal{X}|X| = T}\mathbb{E}_{\epsilon}\left[\sup_{f\in \mathcal{F}}\frac{1}{T}\sum_{x\in X}\epsilon_{x}f(x)\right]$ .
Algorithm 1: Probability assignment  $\mathcal{Q}^{\mathrm{FTPL}}$
Input: time horizon  $T$  , smoothness parameter  $\sigma$  , VC dimension  $d$  , Samples per step Parameter  $n$  Truncation parameter  $\alpha$    
1 for  $t\gets 1$  to  $T$  do Generate  $N^{(t)}\sim \mathrm{Poi}(n)$  fresh hallucinated samples  $(\widetilde{x}_1^{(t)},\widetilde{y}_1^{(t)}),\dots ,(\widetilde{x}_N^{(t)},\widetilde{y}_N^{(t)})$  , which are i.i.d. conditioned on  $N$  with  $\tilde{x}_i^{(t)}\sim \mathcal{U}(\mathcal{X})$  and  $\tilde{y}_i^{(t)}\sim \mathcal{U}(\{0,1\})$  Call the oracle to compute  $\tilde{h}_t\gets \mathrm{OPT}\left(\{(\widetilde{x}_i^{(t)},\widetilde{y}_i^{(t)})\}_{i\in [N^{(t)}]}\cup \{x_\tau ,y_\tau \}_{\tau \in [t - 1]}\right)$  Observe  $x_{t}$  Assign probability  $q^{\mathrm{FTPL}}(1|x_{1:t},y_{1:t - 1}) = \frac{\tilde{h}_t(x_t) + \alpha}{1 + 2\alpha}$  Receive  $y_{t}$
Theorem 4.1 (Main Regret Bound). For any hypothesis class  $\mathcal{F}$  and parameters  $n, \alpha$ , we have that the regret of Algorithm 1 for  $\sigma$ -smoothed adversaries is bounded as
$$
\begin{array}{l} \mathcal {R} _ {T} (\mathcal {F}, \sigma , \mathcal {Q} ^ {\mathrm {F T P L}}) \leq n \log \left(\frac {1}{\alpha}\right) + \alpha T + T \sqrt {\log \left(\frac {1}{\alpha}\right) \cdot \frac {1}{\sigma n}} \\ + T \cdot \inf  _ {m \leq n} \left\{\frac {1}{\alpha} \operatorname {R a d} \left(\mathcal {F} _ {\alpha}, n / m\right) + \frac {n (1 - \sigma) ^ {m} \log (1 / \alpha)}{m} + e ^ {- n / 8} \right\}. \\ \end{array}
$$
We will instantiate this bound for case when the class  $\mathcal{F}$  has bounded VC dimension. For such classes, it is known that the Rademacher complexity is bounded. We state this in the following corollary.
Corollary 4.1.1. Let  $\mathcal{F}$  be a hypothesis class such that the Rademacher complexity is bounded as  $\operatorname{Rad}(\mathcal{F}_{\alpha}, T) = cT^{-\omega}$ , then we have  $\mathcal{R}_T(\mathcal{F}, \sigma, \mathcal{Q}^{\mathrm{FTPL}}) \leq T^{\frac{2}{2 + \omega}} \sqrt{\frac{1}{\sigma}} \cdot \operatorname{poly} \log \left(\frac{Td}{\sigma}\right)$ .
Note that, in particular, for VC classes we have  $\mathcal{R}_T\left(\mathcal{F}^{\mathrm{VC}},\sigma\right)$  scales as  $T^{4 / 5}$ . Improving this to achieve the minimax rate discussed in Corollary 3.4.1 is an interesting open question.
Remark. A slightly improved regret scaling as  $T^{3/4}$  can be achieved by assuming access to an oracle that can optimize a mixed objective function involving the log-loss and signed sum of the functions in class. However, these oracles do not have the natural interpretation in terms of maximum likelihood estimation.
The main challenge in designing algorithms in the follow-the-perturbed-leader framework is designing the distribution of the hallucinated samples so as to balance the tradeoff between the "stability" of the algorithm i.e. how little the algorithm changes its prediction from time step to time step, and the "perturbation" i.e. how much the addition of the hallucinated samples changes the prediction of the algorithm from the outputting the best hypothesis on the historical samples. This is captured by the following lemma.
Lemma 4.2 (Follow the Perturbed Leader bound). Let  $\ell$  be a convex loss function, and let  $\mathcal{F}$  be a hypothesis class. Let  $\mathcal{D}_t$  denote the distribution of the adversary at time  $t$  and let  $\mathcal{Q}_t$  denote the distribution of the hypothesis  $h_t$  output by Algorithm 1. Then, we have that the regret of Algorithm 1 (where we use  $\widetilde{s}_t = (\widetilde{x}_t,\widetilde{y}_t)$  to denote a hallucinated data point) is bounded by
$$
\sum_{i = 1}^{T}\underbrace{\mathbb{E}_{s_{t}\sim\mathcal{D}_{t}}\left(\underset{h_{t}\sim\mathcal{Q}_{t}}{\mathbb{E}}\left[ \ell(h_{t},s_{t})\right] - \underset{h_{t + 1}\sim\mathcal{Q}_{t + 1}}{\mathbb{E}}\left[ \ell(h_{t + 1},s_{t})\right]\right)}_{\text{Stability}} + \underbrace{\mathbb{E}\left[\sup_{h\in\mathcal{F}_{\alpha}}\sum_{i = 1}^{N}\ell(h,\tilde{s}_{t}) - \ell(h^{*},\tilde{s}_{t})\right]}_{\text{Perturbation}} + \alpha T.
$$
where  $h^* = \operatorname{argmin}_{h \in \mathcal{F}_\alpha} \sum_{i=1}^T \ell(h, s_t)$ .
2With some abuse of notation, we consider  $\mathcal{D}_t$  to be over  $\mathcal{X}\times \{0,1\}$
We provide a proof in Appendix E for completeness. Given this decomposition of the regret, we need to handle both terms carefully. Just to appreciate the tradeoff, note that as we increase the number of hallucinated examples, the perturbation term generally increases, but the stability term generally decreases. First, let us focus on the stability term which is harder to deal with. The main approach we will use is a generalization of the decomposition of the stability term introduced in Haghtalab et al. [2022] even when the losses are unbounded, as is the case with the log-loss. The main idea is to decompose the stability term in terms of the distance between the distribution of the average prediction at the next time step and the distribution of the current time step, as captured by the  $\chi^2$  distance and a term that captures how different the predictions of the algorithm are when the sample is resampled from the same distribution. The proof can be found in Appendix F.
Lemma 4.3 ( $\chi^2 + \text{Generalization} \Rightarrow \text{Stability}$ ). Let  $\mathcal{Q}_t$  denote learner's distribution over  $\mathcal{F}$  in at round  $t$ ,  $\mathcal{D}_t$  be adversary's distribution at time  $t$  (given the history  $s_1, \dots, s_{t-1}$ ),  $s_t \sim \mathcal{D}_t$  be the realized adversarial instance at time  $t$ , and  $s_t'$  be an independent copy  $s_t' \sim \mathcal{D}_t$ . Let  $R^{(t+1)}$  refers to the randomness used by the algorithm in round  $t+1$ . Then,
$$
\begin{array}{l} \underset {s _ {t} \sim \mathcal {D} _ {t}} {\mathbb {E}} \left(\underset {h _ {t} \sim \mathcal {Q} _ {t}} {\mathbb {E}} [ \ell (h _ {t}, s _ {t}) ] - \underset {h _ {t + 1} \sim \mathcal {Q} _ {t + 1}} {\mathbb {E}} [ \ell (h _ {t + 1}, s _ {t}) ]\right) \\ \leq \sqrt {\frac {1}{2} \chi^ {2} (\underset {s _ {t} \sim \mathcal {D} _ {t}} {\mathbb {E}} [ \mathcal {Q} _ {t + 1} ] , \mathcal {Q} _ {t})} \cdot \log \left(\frac {1}{\alpha}\right) + \underset {s _ {t}, s _ {t} ^ {\prime} \sim \mathcal {D} _ {t}; R ^ {(t + 1)}} {\mathbb {E}} [ \ell (h _ {t + 1}, s _ {t} ^ {\prime}) - \ell (h _ {t + 1}, s _ {t}) ]. \\ \end{array}
$$
Given this lemma, we move on to bounding the  $\chi^2$  divergence between the distribution of the average prediction at the next time step and the distribution of the current time step. This is done using the Ingster method to bound the divergence of mixtures. We include a proof in Appendix G for completeness.
Lemma 4.4 (Bound on  $\chi^2$ ).  $\chi^2\left(\mathbb{E}_{s_t\sim \mathcal{D}_t}[\mathcal{Q}_{t + 1}],\mathcal{Q}_t\right)\leq \frac{2}{\sigma n}$ .
Next, we move on second term in Lemma 4.3. Note that this term involves the difference between the loss of the hypothesis output at time  $t + 1$  evaluated on two independent points  $s_t$  and  $s_t'$  drawn from  $\mathcal{D}_t$ . The main idea to bound the term is to use a stronger version of the coupling lemma Theorem 2.1 which allows us to extract subsequences of points sampled according to smooth distributions from iid samples from the base measure. This allows us to relate the required generalization bound to the Rademacher complexity of the class composed with the loss. Using the truncation and the contraction principle, we get the desired bound. The proof can be found in Appendix H.
Lemma 4.5 (Generalization). Let  $h_{t+1}$  denote the hypothesis output by the Algorithm 1 at time  $t + 1$ . Then, for any  $m \leq n$ , we have
$$
\underset {s _ {t}, s _ {t} ^ {\prime} \sim \mathcal {D} _ {t}; R ^ {(t + 1)}} {\mathbb {E}} \left[ \ell (h _ {t + 1}, s _ {t} ^ {\prime}) - \ell (h _ {t + 1}, s _ {t}) \right] \leq \frac {1}{\alpha} \mathrm {R A D} \left(\mathcal {F} _ {\alpha}, n / m\right) + \frac {n (1 - \sigma) ^ {m} \log (1 / \alpha)}{m} + e ^ {- n / 8}.
$$
The final term that we want to bound is the perturbation term. In order to bound this note that we set our truncation parameter  $\alpha$  such that the loss our the prediction made by our algorithm is bounded and consequently the perturbation term is bounded by  $n\log \frac{1}{\alpha}$ , see Appendix I. Theorem 4.1 follows by combining the above results.
5 Conclusions and Open Problems
In this paper, we initiated the study of sequential probability assignment with smoothed adversaries. We characterize the minimax regret in terms of the minimax regret for transductive learning and use this to provide tight regret bounds, e.g., for VC classes. Furthermore, we initiate the study of oracle efficiency in this setting and show that sublinear regret can be achieved for general classes.
Our work motivates several directions for future work. An interesting direction is whether the optimal  $O(\log (T))$  regret is achievable for some classes, such as VC classes, using oracle-efficient algorithms. More generally, are there computational barriers to obtaining fast rates in prediction with log-loss in this setting?
For the particular use in our analysis a simpler version of the lemma similar to Haghtalab et al. [2022] suffices but we prove a general version since we believe such a version is useful in providing improved regret bounds for the problem.
This work was supported in part by the National Science Foundation under grant CCF-2145898, a C3.AI Digital Transformation Institute grant, and Berkeley AI Research Commons grants. This work was partially done while authors were visitors at the Simons Institute for the Theory of Computing.
Jorma Rissanen. A universal data compression system. IEEE Transactions on information theory, 29 (5):656-664, 1983.
Jorma Rissanen. Universal coding, information, prediction, and estimation. IEEE Transactions on Information theory, 30(4):629-636, 1984.
Thomas M Cover. Universal portfolios. Mathematical finance, 1(1):1-29, 1991.
Meir Feder, Neri Merhav, and Michael Gutman. Universal prediction of individual sequences. IEEE transactions on Information Theory, 38(4):1258-1270, 1992.
Qun Xie and Andrew R Barron. Minimax redundancy for the class of memoryless sources. IEEE Transactions on Information Theory, 43(2):646-657, 1997.
Neri Merhav and Meir Feder. Universal prediction. IEEE Transactions on Information Theory, 44(6): 2124-2147, 1998.
Qun Xie and Andrew R Barron. Asymptotic minimax regret for data compression, gambling, and prediction. IEEE Transactions on Information Theory, 46(2):431-445, 2000.
Yuhong Yang and Andrew Barron. Information-theoretic determination of minimax rates of convergence. Annals of Statistics, pages 1564-1599, 1999.
Jiantao Jiao, Haim H Permuter, Lei Zhao, Young-Han Kim, and Tsachy Weissman. Universal estimation of directed information. IEEE Transactions on Information Theory, 59(10):6220-6242, 2013.
Francesco Orabona and David Pál. Coin betting and parameter-free online learning. Advances in Neural Information Processing Systems, 29, 2016.
Dylan J Foster, Satyen Kale, Haipeng Luo, Mehryar Mohri, and Karthik Sridharan. Logistic regression: The importance of being improper. In Conference On Learning Theory, pages 167-208. PMLR, 2018.
Alexander Rakhlin and Karthik Sridharan. Sequential probability assignment with binary alphabets and large classes of experts. arXiv preprint arXiv:1501.07340, 2015.
Yaniv Fogel and Meir Feder. On the problem of on-line learning with log-loss. In 2017 IEEE International Symposium on Information Theory (ISIT), pages 2995-2999. IEEE, 2017.
Yaniv Fogel and Meir Feder. Universal batch learning with log-loss. In 2018 IEEE International Symposium on Information Theory (ISIT), pages 21-25. IEEE, 2018.
Alankrita Bhatt and Young-Han Kim. Sequential prediction under log-loss with side information. In Algorithmic Learning Theory, pages 340-344. PMLR, 2021.
Blair Bilodeau, Dylan J Foster, and Daniel M Roy. Minimax rates for conditional density estimation via empirical entropy. arXiv preprint arXiv:2109.10461, 2021.
Changlong Wu, Mohsen Heidari, Ananth Grama, and Wojciech Szpankowski. Expected worst case regret via stochastic sequential covering. arXiv preprint arXiv:2209.04417, 2022a.
Dylan J Foster and Akshay Krishnamurthy. Efficient first-order contextual bandits: Prediction, allocation, and triangular discrimination. Advances in Neural Information Processing Systems, 34: 18907-18919, 2021.
Changlong Wu, Mohsen Heidari, Ananth Grama, and Wojciech Szpankowski. Precise regret bounds for log-loss via a truncated bayesian algorithm. Advances in Neural Information Processing Systems, 2022b.
Mark Rudelson and Roman Vershynin. Combinatorics of random processes and sections of convex bodies. Annals of Mathematics, pages 603-648, 2006.
Roman Vershynin. High-dimensional probability: An introduction with applications in data science. Cambridge university press, 2018.
Khadija Musayeva. Sample complexity result for multi-category classifiers of bounded variation. arXiv preprint arXiv:2003.09176, 2020.
Peter L Bartlett, Sanjeev R Kulkarni, and S Eli Posner. Covering numbers for real-valued function classes. IEEE transactions on information theory, 43(5):1721-1724, 1997.
Alexander Rakhlin, Karthik Sridharan, and Ambuj Tewari. Online learning: Random averages, combinatorial parameters, and learnability. Advances in Neural Information Processing Systems, 23, 2010.
Changlong Wu, Ananth Grama, and Wojciech Szpankowski. Online learning in dynamically changing environments. arXiv preprint arXiv:2302.00103, 2023.
Blair Bilodeau, Dylan Foster, and Daniel Roy. Tight bounds on minimax regret under logarithmic loss via self-concordance. In International Conference on Machine Learning, pages 919-929. PMLR, 2020.
Jorma J Rissanen. Fisher information and stochastic complexity. IEEE transactions on information theory, 42(1):40-47, 1996.
Yurii Mikhailovich Shtar'kov. Universal sequential coding of single messages. Problemy Peredachi Informatssii, 23(3):3-17, 1987.
Philippe Jacquet, Gil Shamir, and Wojciech Szpankowski. Precise minimax regret for logistic regression with categorical feature values. In Algorithmic Learning Theory, pages 755-771. PMLR, 2021.
Raphail Krichevsky and Victor Trofimov. The performance of universal encoding. IEEE Transactions on Information Theory, 27(2):199-207, 1981.
Sudeep Kamath, Alon Orlitsky, Dheeraj Pichapati, and Ananda Theertha Suresh. On learning distributions from their samples. In Conference on Learning Theory, pages 1066-1100. PMLR, 2015.
Daniel A Spielman and Shang-Hua Teng. Smoothed analysis of algorithms: Why the simplex algorithm usually takes polynomial time. Journal of the ACM (JACM), 51(3):385-463, 2004.
Adam Block and Max Simchowitz. Efficient and near-optimal smoothed online learning for generalized linear functions. Advances in neural information processing systems, 2022.
Adam Tauman Kalai and Santosh Vempala. Efficient algorithms for universal portfolios. Journal of Machine Learning Research, pages 423-440, 2002.
Haipeng Luo, Chen-Yu Wei, and Kai Zheng. Efficient online portfolio with logarithmic regret. Advances in neural information processing systems, 31, 2018.
Julian Zimmert, Naman Agarwal, and Satyen Kale. Pushing the efficiency-regret pareto frontier for online learning of portfolios and quantum states. In Conference on Learning Theory, pages 182-226. PMLR, 2022.
Rémi Jézéquel, Dmitrii M Ostrovskii, and Pierre Gaillard. Efficient and near-optimal online portfolio selection. arXiv preprint arXiv:2209.13932, 2022.