SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Alankrita Bhatt
California Institute of Technology abhatt@caltech.edu
Nika Haghtalab
University of California, Berkeley nika@berkeley.edu
shetty@berkeley.edu
We initiate the study of smoothed analysis for the sequential probability assignment problem with contexts. We study information-theoretically optimal minmax rates as well as a framework for algorithmic reduction involving the maximum likelihood estimator oracle. Our approach establishes a general-purpose reduction from minimax rates for sequential probability assignment for smoothed adversaries to minimax rates for transductive learning. This leads to optimal (logarithmic) fast rates for parametric classes and classes with finite VC dimension. On the algorithmic front, we develop an algorithm that efficiently taps into the MLE oracle, for general classes of functions. We show that under general conditions this algorithmic approach yields sublinear regret.
Sequential probability assignment — also known as online learning under the logarithmic loss — is a fundamental problem with far-reaching impact on information theory, statistics, finance, optimization, and sequential decision making [Rissanen, 1983, 1984, Cover, 1991, Feder et al., 1992, Xie and Barron, 1997, Merhav and Feder, 1998, Xie and Barron, 2000, Yang and Barron, 1999, Jiao et al., 2013, Orabona and Pal, 2016, Foster et al., 2018]. In recent years, methods for incorporating contexts or side information into sequential probability assignment have gained much attention [Rakhlin and Sridharan, 2015, Fogel and Feder, 2017, 2018, Foster et al., 2018, Bhatt and Kim, 2021, Bilodeau et al., 2021, Wu et al., 2022a], in part due to their newly forged connection to sequential decision making applications, the contextual bandit problem, and learning in Markov Decision Processes (MDPs) (see e.g. [Foster and Krishnamurthy, 2021] and [Foster et al., 2021a]). In this setting, a forecaster who has access to historical data  $x_{1:t-1}, y_{1:t-1}$  consisting of contexts  $x_{\tau}$  (e.g., day  $\tau$ 's meteorological information) and the outcomes  $y_{\tau} \in \{0,1\}$  (e.g., whether  $\tau$  was a rainy day) wishes to predict  $y_t$  given a new context  $x_t$ . The forecaster uses a probability assignment  $p_t$  to estimate the probability of  $y_t = 1$  outcome and incurs the logarithmic loss, i.e.,  $-\log p_t(y_t)$ , which rewards the forecasters for having assigned high probability to the realized outcome. The goal of the forecaster is to suffer low regret against a chosen reference class of predictors.
A large body of prior work on sequential probability assignment with contexts has focused on settings where contexts are presented i.i.d. from an unknown distribution (see [Fogel and Feder, 2017, Bhatt and Kim, 2021, Bilodeau et al., 2021, Wu et al., 2022a] and the references within); this problem is also referred to as conditional density estimation. In these cases, sequential probability assignment is known to enjoy small regret for several reference classes such as Vapnik-Chervonenkis (VC) classes. On the other hand, attempts to consider context distributions that evolve unpredictably and
adversarily have faced strong impossibility results even for simple reference classes of predictors. For example, for the reference class of simple one-dimensional thresholds assigning  $p_t = \theta_0 \mathbb{1}\{x_t \leq a\} + \theta_1 \mathbb{1}\{x_t > a\}$  for  $a \in [0, 1]$ , regret is bounded by  $O(\log T)$  in the i.i.d. case [Fogel and Feder, 2017] but is lower bounded by  $\Omega(T)$  when the sequence of contexts is chosen adversariably (folklore e.g. Littlestone [1988]). In the face of the increasing need to adapt to evolving contexts in modern applications, these impossibility results indicate that new models of adversarial behavior must be considered for obtaining rigorous guarantees that guide the design of sequential probability assignment in practical applications.
In recent years, smoothed analysis of adaptive adversaries [Haghtalab et al., 2021, Rakhlin et al., 2011] has emerged as a framework for going beyond the worst-case adversaries while making minimal assumptions about the adaptive process that generates a sequence. In this setting, contexts are chosen from an evolving sequence of so-called  $\sigma$ -smooth distributions, whose density is bounded above by  $1 / \sigma$  times that of a base measure (such as the uniform distribution). Remarkably, these methods, established by Haghtalab et al. [2021] for 0-1 loss and extended to regression by Haghtalab et al. [2022], Block et al. [2022], have established performance guarantees for the sequential prediction problem that matches the optimal performance in the i.i.d setting. This raises the question as to whether the sequential probability assignment problem may similarly enjoy improved minmax regret bounds for smoothed adaptive sequences.
Beyond minmax rates, an important feature of probability assignment and, its analogue, density estimation is the availability of fundamental and natural estimation techniques such as maximum likelihood estimation (MLE). For i.i.d. sequences, under general conditions, MLE is known to be optimal asymptotically and often serves as a starting point for designing more sophisticated estimators. Going beyond i.i.d. sequences, we ask whether MLE can be made to achieve good statistical behavior on adaptive sequences. More generally, algorithmic perspective is increasingly important for the sequential probability assignment problem and its applications to contextual bandits and reinforcement learning (where algorithm design is as fundamental a consideration as minmax rates [Agarwal et al., 2014, Simchi-Levi and Xu, 2022, Foster and Rakhlin, 2020, Langford et al., 2007, Foster et al., 2021b]). In this space, oracle-efficient sequential decision making algorithms that repurpose existing offline algorithmic routines have received special interest [Kalai and Vempala, 2005, Dudik et al., 2020, Wang et al., 2022, Kakade et al., 2007, Simchi-Levi and Xu, 2022]. Here again, recent progress on smoothed analysis for sequential prediction with 0-1 loss and regression loss [Haghtalab et al., 2021, Block et al., 2022] has shown promise in bridging the computational and information-theoretical gaps between what is obtainable in the i.i.d. case and for smoothed adaptive sequences.
In this paper, we initiate the study of smoothed analysis for sequential probability assignment and seek to understand fundamental information-theoretic limits on the minmax regret as well as design natural and oracle-efficient algorithms for this problem. Additionally, we investigate whether in the smoothed analysis setting, maximum likelihood estimation can efficiently address sequential probability assignment while achieving small regret. To the best of our knowledge, our work is the first to consider oracle-efficient algorithms (and particularly the MLE) for the sequential probability assignment problem.
Reduction to transductive learning. Our first main result is a reduction from regret bounds against a smoothed adversary to regret bounds against (a generalized version of) a transductive adversary. That is, we show that the minimax regret in the smoothed analysis setting is upper bounded by the minimax regret in the setting where a set of contexts is provided to the learner and the adversary is constrained to picking the contexts from this set. For  $\mathcal{F}$ , a class of hypotheses mapping contexts to  $[0,1]$ , let us define the minimax regret in the transductive case over  $T$  times steps when a context set of size  $M$  is provided to the learner to be  $\underline{\mathcal{R}}_T^M (\mathcal{F})$ . We establish in Theorem 3.1 that for all  $\sigma$ -smooth sequences the minimax regret  $\mathcal{R}_T(\mathcal{F},\sigma)$  satisfies, for any  $k > 1$ ,
$$
\mathcal {R} _ {T} (\mathcal {F}, \sigma) \leq \underline {{\mathcal {R}}} _ {T} ^ {k T} (\mathcal {F}) + T ^ {2} (1 - \sigma) ^ {k}.
$$
Furthermore, in Theorem 3.4 we upper bound  $\underline{\mathcal{R}}_T^{kT}(\mathcal{F})$  by connecting the worst case adversarial regret in this setting to the scale-sensitive VC dimension of  $\mathcal{F}$  which is a prototypical offline complexity of the class. Our results obtain a logarithmic dependence on  $1 / \sigma$ . In particular, in
Corollary 3.4.1, we show that for VC classes (and parametric classes) the regret is bounded by  $\mathcal{R}_T(\mathcal{F},\sigma)\leq O\left(d\log \left(\frac{T}{\sigma}\right)\right)$ , where  $d$  is the VC dimension of class  $\mathcal{F}$ .
Efficient Reduction from Sequential Probability Assignment to MLE. Our second contribution is initiating the study of oracle-efficient algorithm design for sequential probability assignment. In particular, for small alphabet size, we design a natural algorithm (Algorithm 1) that efficiently uses an MLE oracle and achieves sublinear regret in the smoothed setting. Our Theorem 4.1 gives a general regret bound in terms of the statistical complexity of the class  $\mathcal{F}$  and the smoothness parameter  $\sigma$ . For VC classes, this achieves regret rate of  $T^{4/5} \sqrt{\frac{d}{\sigma}}$ . To the best of our knowledge, this is the first oracle-efficient algorithm and analysis of the follow-the-perturbed-leader style algorithms for the logarithmic loss.
Probability assignment for VC classes. For VC classes  $\mathcal{F}$ , we explicitly construct sequential probability assignments and establish their regret guarantees in the smoothed setting. That is, we construct a probability assignment based on a Bayesian mixture over  $\mathcal{F}$  that satisfies  $\mathcal{R}_T(\mathcal{F},\sigma)\leq Cd\log \left(\frac{T}{\sigma}\right)$  where  $d$  is the VC dimension of class  $\mathcal{F}$ . While this approach is not oracle-efficient, it indeed achieves regret bound with optimal dependence on  $T$  and  $\sigma$ . This motivates a natural direction for future work as to whether such mixture-based methods can be implemented oracle efficiently or if there is a tradeoff between the regret and the computational complexity of sequential probability assignment.
Let  $\mathcal{X}$  be a set of contexts and  $\mathcal{Y} = \{0,1\}$ . Then, the problem being studied entails a sequential game where at each timestep  $t$ , based on the history of contexts  $x_{1:t} := (x_1,\ldots ,x_t)$  where  $x_{i}\in \mathcal{X}$  and associated bits  $y_{1:t - 1}\in \{0,1\}^{t - 1}$ , the player must assign a probability  $q(\cdot |x_{1:t},y_{1:t - 1})$  to what the upcoming bit  $y_{t}$  will be. Once the bit  $y_{t}$  is revealed (possibly in an adversarial fashion) the player incurs loss  $-\log q(y_{t}|x_{1:t},y_{1:t - 1})$  and the game proceeds to the next step. For a hypothesis class  $\mathcal{F}\subset \{\mathcal{X}\to [0,1]\}$ , the associated regret for a probability assignment strategy  $\mathcal{Q} = \{q(\cdot |x_{1:t},y_{1:t - 1})\}_{t = 1}^n$  for a fixed  $x_{1:T},y_{1:T}$  is
$$
\mathcal {R} _ {T} \left(\mathcal {F}, x _ {1: T}, y _ {1: T}, \mathcal {Q}\right) = \sum_ {t = 1} ^ {T} \log \frac {1}{q \left(y _ {t} \mid x _ {1 : t} , y _ {1 : t - 1}\right)} - \inf  _ {f \in \mathcal {F}} \sum_ {t = 1} ^ {T} \log \frac {1}{p _ {f} \left(y _ {t} \mid x _ {t}\right)} \tag {1}
$$
where  $p_f(1|x_t) = f(x_t)$ ; i.e. the function  $f$  assigns probability  $\mathrm{Bern}(f(x_t))$  to the upcoming bit given the context  $x_t$ .
Our statistical results apply to a general loss function  $\ell$  and general actions of the learner  $a_{t}$ . For a set of inputs  $\{(x_i,y_i)\}_{i = 1}^T$  specified by the adversary and a set of actions  $\{a_i\}_{i = 1}^T$  of the learner, regret is defined by
$$
\mathcal {R} _ {T} (\mathcal {F}, x _ {1: T}, y _ {1: T}, a _ {1: T}) = \sum_ {t = 1} ^ {T} \ell (a _ {t}, (x _ {t}, y _ {t})) - \inf  _ {f \in \mathcal {F}} \sum_ {t = 1} ^ {T} \ell (f (x _ {t}), (x _ {t}, y _ {t})),
$$
where for log-loss  $a_{t} = q(\cdot |x_{1:t},y_{1:t - 1})$ ; the action is a probability mass function (pmf) over  $\{0,1\}$ .
The regret in (1) is often studied under various adversary models; i.e. various different probabilistic assumptions (or lack thereof) on the model generating  $x_{t}$  and  $y_{t}$ . In this work, we consider worst-case  $y_{t}$  (in contrast to the realizable setting where  $Y_{t} \sim \mathrm{Bern}(f^{*}(x_{t}))$  for a fixed unknown  $f^{*} \in \mathcal{F}$ ) and  $X_{t} \sim \mathcal{D}_{t}$  where  $\mathcal{D}_{t}$ s form an adaptive sequence of smooth distributions.
Definition 2.1 (Smooth distribution and adversary [Haghtalab et al., 2020]). Consider a fixed and known base distribution  $\mu$  on  $\mathcal{X}$  (such as the uniform distribution if  $\mathcal{X}$  supports it). A distribution  $\mathcal{D}$  on  $\mathcal{X}$  is said to be  $\sigma$ -smooth if for all measurable sets  $A \subseteq \mathcal{X}$ ,  $\mathcal{D}(A) \leq \frac{\mu(A)}{\sigma}$ . We denote the set of all  $\sigma$ -smooth distributions by  $\Delta_{\sigma}(\mu)$ . An adversary, characterized by a joint distribution  $\mathcal{D}$  with  $X_{t} \sim \mathcal{D}_{t}$  (where  $\mathcal{D}_{t}$  may possibly depend on the history) is said to be a  $\sigma$ -smooth adaptive adversary if  $\mathcal{D}_{t} \in \Delta_{\sigma}(\mu)$  for all  $t \in \{1, \dots, T\}$ .
The minmax regret for  $\sigma$ -smooth adaptive adversaries is then given by
$$
\mathcal{R}_{T}(\mathcal{F},\sigma) = \inf_{\mathscr{D}}\sup_{\sigma \text{-smoothed}\mathscr{D}}\mathbb{E}_{X_{1:T}\sim \mathscr{D}}\left[\max_{y_{1:T}}\mathcal{R}_{T}(\mathcal{F},X_{1:T},y_{1:T},\mathscr{Q})\right],
$$
where  $\mathcal{Q}$  is the set of all probability assignment strategies.
We are particularly interested in how geometric properties of the function class  $\mathcal{F}$  affect  $\mathcal{R}_T(\mathcal{F},\sigma)$ . There are several notions of covering numbers and combinatorial dimensions that quantify the "richness" and complexity of a class, but the scale-sensitive VC dimension will be of particular interest to us and is invoked in our results.
Definition 2.2 (Scale-sensitive VC dimension). Let  $\mathcal{F}$  be a function class. For any  $\alpha >0$  and points  $x_{1},\ldots ,x_{m}\in \mathcal{X}$ , we say that  $\mathcal{F}$  shatters the set  $x_{1},\ldots ,x_{m}$  at scale  $\alpha$  if there exist  $s_1\dots s_m\in \mathbb{R}$  such that for each  $\epsilon \in \{-1,1\} ^n$  there exists a function  $f\in \mathcal{F}$  such that  $\epsilon_{i}\left(f(x_{i}) - s_{i}\right)\geq \frac{\alpha}{2}$ . The scale sensitive VC dimension at scale  $\alpha$  of  $\mathcal{F}$ , denoted by VC  $(\mathcal{F},\alpha)$  is defined as the largest  $m$  such that there is a set of  $m$  points  $x_{1},\ldots ,x_{m}\in \mathcal{X}$  such that  $\mathcal{F}$  shatters the set at scale  $\alpha$ . The (traditional) VC dimension of a binary class  $\mathcal{F}$  is defined as VC  $(\mathcal{F}) = \lim_{\alpha \to 0^{+}}\mathrm{VC}(\mathcal{F},\alpha)$ .
Throughout, we use the following result of Haghtalab et al. [2021] about  $\sigma$ -smooth distributions. This results aids us in reduction from smoothed learning to transductive learning.
Theorem 2.1 (Coupling Lemma of Haghtalab et al. [2021]). Let  $\mathcal{D}_{\sigma}$  be an adaptive sequence of  $t$ $\sigma$ -smooth distributions on  $\mathcal{X}$ . There is a coupling  $\Pi$  such that
$$
\left(X _ {1}, Z _ {1, 1}, \dots , Z _ {1, K}, \dots , X _ {t}, Z _ {t, 1}, \dots , Z _ {t, K}\right) \sim \Pi
$$
1.  $X_{1},\ldots ,X_{t}$  is distributed according  $\mathcal{D}_{\sigma}$
2. For every  $j \leq t$ ,  $\{Z_{i,k}\}_{i \geq j, k \in [K]}$  are uniformly and independently distributed on  $\mathcal{X}$ , conditioned on  $X_1, \ldots, X_{j-1}$ .
3. For any  $t$ , with probability at least  $1 - (1 - \sigma)^{K}$ ,  $X_{t} \in \{Z_{t,k}\}_{k=1:K}$ .
3 General reduction to transductive learning
In this section, we will consider the minimax regret for the smoothed online learning game with respect to the loss function  $\ell$  against a general class of functions  $\mathcal{F}$ . In Section 3.1, we will show that the minimax regret can be reduced to the minimax regret for a version of transductive learning with respect to the same loss function and class of functions. In Section 3.2, we give general upper bounds for the transductive setting. There is a subtle but important difference between reduction that directly involve regret compared to recent efforts (such as Haghtalab et al. [2021], Block et al. [2022], Haghtalab et al. [2022]) using reductions between proxies of regret, such as covering numbers and sequential complexities. This is particularly important for log loss since its complexity is not captured by covering numbers. We discuss this point further in Section 3.2.2.
3.1 Regret-to-regret Reduction
We work with a general loss function  $\ell$  and general actions of the learner  $a_{t}$ . We note that, we can write the minmax value of the smoothed setting in extensive form as
$$
\begin{array}{l} \mathcal {R} _ {T} (\mathcal {F}, \sigma) = \sup  _ {\mathcal {D} _ {1} \in \Delta_ {\sigma} (\mu)} \mathbb {E} _ {X _ {1} \sim \mathcal {D} _ {1}} \inf  _ {a _ {1}} \sup  _ {y _ {1}} \sup  _ {\mathcal {D} _ {2} \in \Delta_ {\sigma} (\mu)} \mathbb {E} _ {X _ {2} \sim \mathcal {D} _ {2}} \inf  _ {a _ {2}} \sup  _ {y _ {2}} \dots \tag {2} \\ \ldots \sup  _ {\mathcal {D} _ {T} \in \Delta_ {\sigma} (\mu)} \mathbb {E} _ {X _ {T} \sim \mathcal {D} _ {T}} \inf  _ {a _ {T}} \sup  _ {y _ {T}} \mathcal {R} (\mathcal {F}, X _ {1: T}, y _ {1: T}, a _ {1: T}). \\ \end{array}
$$
In order to bound this, we consider a generalization of the notion of online learning that is referred to as transductive learning. In this setting, at the start of the interaction the adversary chooses a set of contexts  $X = \{X_{i}\}_{i = 1}^{M}$  for some  $M\geq T$  and provides this to the player. The game proceeds as
The reduction works for general loss functions with the property that the worst-case regret for horizon  $T$  is bounded by  $T$ , but one can think about  $\ell$  being the log-loss throughout this section for concreteness.
before with the adversary picking  $(x_{t},y_{t})$  at time  $t$  and the learner picking an action  $a_{t}$  and suffering a loss  $\ell (a_t,(x_t,y_t))$ . However, the adversary is now constrained to pick  $x_{t}\in X$  at all times  $t$ . We can then define the minmax regret indexed by  $X$  as
$$
\underline {{\mathcal {R}}} _ {T} (\mathcal {F}, X) := \left[ \max  _ {x _ {1} \in X a _ {1}} \inf  _ {y _ {1}} \dots \max  _ {x _ {T} \in X a _ {T}} \inf  _ {y _ {T}} \sup  _ {y _ {T}} \mathcal {R} (\mathcal {F}, x _ {1: T}, y _ {1: T}, a _ {1: T}) \right].
$$
Furthermore, define the worst-case transductive learning regret for sets of size  $M$  as  $\underline{\mathcal{R}}_T^M(\mathcal{F}) = \max_{X \subseteq \mathcal{X}, |X| = M} \underline{\mathcal{R}}_T(\mathcal{F}, X)$ . In the following theorem, we show that the regret against  $\sigma$ -smoothed adversaries is bounded by the regret in the transductive learning setting when the set of contexts is drawn from the base distribution  $\mu$ .
Theorem 3.1. Let  $\mathcal{F}$  be any class of functions from  $\mathcal{X}$  to  $\mathbb{R}$  and let  $\sigma \in (0,1]$ . Then, for any  $T$  and  $k$ , we have
Theorem 3.1 shows that we can reduce the problem of evaluating the minimax regret for smoothed adversaries to evaluating the minimax regret for transductive learning. Note that the second term  $(1 - \sigma)^{k} \leq e^{-k\sigma}$  and thus, in order to get bounds that are sublinear one needs to consider  $k = c\log T / \sigma$  for an appropriate absolute constant  $c$ . As we will see in the next section, this leads to logarithmic dependence on  $\sigma^{-1}$ . Moreover, by Theorem 3.1 and since  $\mathbb{E}_{X \sim \mu^T} \underline{\mathcal{R}}_T(\mathcal{F}, X) \leq \mathcal{R}_T(\mathcal{F}, \sigma)$ , we can see that the transductive learning regret exactly captures the smoothed regret up to  $\mathrm{polylog}\left(\frac{T}{\sigma}\right)$  factors.
3.2 Bounds for Transductive Learning
In this section, we discuss ways to upper bound transductive learning regret  $\underline{\mathcal{R}}_T^M (\mathcal{F})$  so as to achieve bounds on  $\mathcal{R}_T(\mathcal{F},\sigma)$  via Theorem 3.1.
3.2.1 Using Covering Numbers
One of the approaches common in online learning is to characterize the regret in terms of geometric properties (such as covering numbers) of the function class  $\mathcal{F}$ . The notion of covering required varies depending on the loss function and the stochastic properties of the data—typically completely adversarial problems require stronger notions of sequential coverings [Rakhlin et al., 2015a,b] while for stochastic problems usually weaker offline coverings suffice. In our smoothed case, we show that the offline complexity notion of scale-sensitive VC dimension as defined in Definition 2.2 is adequate. Similar ideas were considered for the case of regression and convex Lipshitz losses in Håghtalab et al. [2022], Block et al. [2022].
Let us first define the notion of approximation according to which a cover will be constructed; we will consider a pointwise approximation. This notion is similar to the notion of global sequential covering in Wu et al. [2022b].
Definition 3.1. Let  $\mathcal{F}$  be a function class. A set of functions  $\tilde{\mathcal{F}}$  is said to be a  $\epsilon$ -covering of  $\mathcal{F}$  if for any  $f \in \mathcal{F}$  there exists  $g \in \tilde{\mathcal{F}}$  such that  $\sup_{x \in \mathcal{X}} |f(x) - g(x)| \leq \epsilon$ . We will use  $\mathcal{N}(\mathcal{F}, \epsilon)$  to denote the size of the minimal  $\epsilon$ -covering of  $\mathcal{F}$ .
Note that while the metric in Definition 3.1 is quite stringent, using this cover in the transductive learning case requires us to only consider function classes with bounded domain size. We capture this using the following theorem.
Theorem 3.2 (Upper bound on transductive learning). Let  $\mathcal{F}$  be a function class and  $\epsilon >0$ . Then,
$$
\underline {{\mathcal {R}}} _ {T} ^ {k T} \left(\mathcal {F}\right) \leq \inf  _ {\epsilon} \left\{\sup  _ {Z \subset \mathcal {X} | Z | = k T} \log \mathcal {N} \left(\mathcal {F} | _ {Z}, \epsilon\right) + 2 \epsilon T \right\},
$$
where  $\mathcal{F}|_{Z}$  is the projection of hypothesis class  $\mathcal{F}$  on the set  $Z$ .
The proof of this theorem follows from relating transductive learning to the worst sequential prediction on a finite set of points using formalism presented in Wu et al. [2022b]. The proof of this theorem is deferred to the Appendix C.
Next, we recall that the covering number  $\mathcal{N}(\mathcal{F},\epsilon)$  is bounded as a function of the scale sensitive VC dimension of the class  $\mathcal{F}$  and the number of points in the domain.
Theorem 3.3 (Rudelson and Vershynin [2006]). There exist universal constants  $c, C$  such that for all  $\alpha > 0$ , any function class  $\mathcal{F}$  defined on a finite set  $\mathcal{X}$ , and  $\epsilon > 0$ , we have
$$
\log \mathcal {N} (\mathcal {F}, \epsilon) \leq C \cdot \operatorname {V C} (\mathcal {F}, c \alpha \epsilon) \log^ {1 + \alpha} \left(\frac {C | \mathcal {X} |}{\operatorname {V C} (\mathcal {F} , c \epsilon) \epsilon}\right).
$$
Finally, putting together Theorem 3.1, Theorem 3.2 and Theorem 3.3 we get the following.
Theorem 3.4 (Minimax smoothed regret and scale-sensitive VC dimension).
$$
\mathcal {R} _ {T} (\mathcal {F}, \sigma) \leq \inf  _ {k, \alpha , \epsilon > 0} \left\{C \cdot \operatorname {V C} (\mathcal {F}, c \alpha \epsilon) \log^ {1 + \alpha} \left(\frac {C k T}{\operatorname {V C} (\mathcal {F} , c \epsilon) \epsilon}\right) + 2 \epsilon T + T ^ {2} (1 - \sigma) ^ {k} \right\}.
$$