SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Tim Van Erven, Dirk Van der Hoeven, Wojciech Kotlowski, and Wouter M Koolen. Open problem: Fast and optimal online portfolio selection. In Conference on Learning Theory, pages 3864-3869. PMLR, 2020.
Thomas M Cover and Erik Ordentlich. Universal portfolios with side information. IEEE Transactions on Information Theory, 42(2):348-363, 1996.
Jason E Cross and Andrew R Barron. Efficient universal portfolios for past-dependent target classes. Mathematical Finance: An International Journal of Mathematics, Statistics and Financial Economics, 13(2):245-276, 2003.
László Győrfi, Gábor Lugosi, and Frederic Udina. Nonparametric kernel-based sequential investment strategies. Mathematical Finance: An International Journal of Mathematics, Statistics and Financial Economics, 16(2):337-357, 2006.
Alankrita Bhatt, J Jon Ryu, and Young-Han Kim. On universal portfolios with continuous side information. In International Conference on Artificial Intelligence and Statistics. PMLR, 2023.
Nicolo Cesa-Bianchi and Gábor Lugosi. Prediction, learning, and games. Cambridge university press, 2006.
Sequential probability assignment is a classic topic in information theory with extensive literature, see the survey by Merhav and Feder [1998] and the references within. In particular, the idea of probability assignments that are Bayesian mixtures over the reference class of distributions [Krichevsky and Trofimov, 1981] is of central importance—such mixture probability assignments arise as the optimal solution to several operational information theoretic and statistical problems [Kamath et al., 2015]. It is also known that the Bayesian mixture approach often outperforms the "plug-in" approach of estimating a predictor from the reference class and then playing it [Merhav and Feder, 1998]. A similar Bayesian mixture probability assignment in the contextual probability assignment problem was used by Bhatt and Kim [2021], where the covering over the VC function class was obtained in a data-dependent manner. This idea of using a mixture over an empirical covering along with a so-called "add-  $\beta$  "probability assignment was then used by Bilodeau et al. [2021]. Combining this with the key idea of discretizing the class of functions as per the Hellinger divergence induced metric, they obtained matching rates for several interesting classes in the realizable case (i.e.  $y_{t}|x_{t}$  generated according to a fixed unknown distribution in the reference class); see also Yang and Barron [1999] for more intuition behind usage of Hellinger coverings for stochastic data. Recent work of Wu et al. [2022a,b] has also employed an empirical covering with an add-  $\beta$  probability assignment for both stochastic and adversarial adversaries.
A complementary approach, more common in the online learning literature is to study fundamental limits of sequential decision making problems non-constructively (i.e. providing bounds on the minmax regret without providing a probability assignment that achieves said regret). This sequential complexities based approach of Rakhlin et al. [2015b,a] has been employed for the log-loss by Rakhlin and Sridharan [2015] and Bilodeau et al. [2020]; however the latter suggests that sequential complexities might not fully capture the log-loss problem.
Smoothed analysis, initiated by Spielman and Teng [2004] for the study of efficiency of algorithms such as the simplex method, has recently shown to be effective in circumventing both statistical and computational lower bounds in online learning for classification and regression Haghtalab et al. [2021], Rakhlin et al. [2011], Block et al. [2022], Haghtalab et al. [2020], Block and Simchowitz [2022]. This line of work establishes that smoothed analysis is a viable line of attack to construct statistically and computationally efficient algorithms for sequential decision making problems and has developed tools such as the coupling lemma to understand smoothed sequential decision-making. But handling the unbounded and non-Lipschitz log-loss and MLE oracle is the main conceptual novelty introduced in this work.
Due to the fundamental nature of the problem, the notion of computational efficiency for sequential probability assignment and the closely related problem of portfolio selection has been considered in the literature. Kalai and Vempala [2002] presents an efficient implementation of Cover's universal portfolio algorithm using techniques from Markov chain Monte Carlo. Recently, there has been a flurry of interest in using follow the regularized leader (FTRL) type techniques to achieve low regret and low complexity simultaneously [Luo et al., 2018, Zimmert et al., 2022, Jézquel et al., 2022], see also Van Erven et al. [2020] and the references within. However, none of these methods consider the contextual version of the problem and are considerably different from the oracle-efficient approach. On the other hand, work studying portfolio selection with contexts [Cover and Ordentlich, 1996, Cross and Barron, 2003, Györfi et al., 2006, Bhatt et al., 2023] does not take oracle-efficiency into account.
Concurrent Work: Wu et al. [2023] also study the problem of sequential probability assignment (and general mixable losses) and for VC classes achieve the optimal regret of  $O(d\log (T / \sigma))$ . In addition to the smooth adversaries, they also studied general models capturing the setting where the base measures are not known. They work primarily in the information theoretical setting and do not present any results regarding efficient algorithms. A major contribution of our work is the study of oracle-efficient online learning for the log-loss, which has not been considered by Wu et al. [2023] Indeed, to the best of knowledge, oracle efficiency has not been considered at all for losses such as the log loss even for worst-case settings. Considering oracle-efficiency for log-loss is an important and natural line of research as the ERM oracle corresponds to the maximum likelihood estimator (MLE) that is commonly used in practice. Our paper presents sublinear regret oracle efficient algorithms for the log loss (which does not have an analog in Wu et al. [2023]).
B Deferred Proof from Section 3
In order to obtain an upper bound on  $\mathcal{R}_T(\mathcal{F},\sigma)$  in terms of  $\mathcal{R}_T^{kT}(\mathcal{F})$  for some  $k$ , we will consider (2) and proceed inductively. The main idea is to note that since  $\mathcal{D}_i$  is  $\sigma$ -smoothed, conditioned on the history thus far, we can invoke the coupling lemma given in Theorem 2.1.
For the sake of illustration, first consider the simple case of  $T = 1$ . Let  $X_{1}, Z_{1} \ldots Z_{k}$  denote the coupling alluded to in Theorem 2.1. Recall that  $X_{1} \sim \mathcal{D}_{1}$  and  $Z_{1:k} \sim \mu^{k}$ . Defining the event  $E_{1} := \{X_{1} \in Z_{1:k}\}$ , we have
$$
\begin{array}{l} \mathcal{R}_{1}(\mathcal{F},\mathcal{D}) = \mathbb{E}_{X_{1}\sim \mathcal{D}_{1}}\inf_{a_{1}}\sup_{y_{1}}\mathcal{R}_{1}(\mathcal{F},X_{1},y_{1},a_{1}) \\ = \mathbb {E} _ {X _ {1}, Z _ {1: k}} \left[ \inf  _ {a _ {1}} \sup  _ {y _ {1}} \mathcal {R} _ {1} (\mathcal {F}, X _ {1}, y _ {1}, a _ {1}) \right] \\ = \mathbb {E} _ {X _ {1}, Z _ {1: k}} \left[ \mathbb {1} \left\{E _ {1} \right\} \inf  _ {a _ {1}} \sup  _ {y _ {1}} \mathcal {R} _ {1} (\mathcal {F}, X _ {1}, y _ {1}, a _ {1}) \right] \\ + \mathbb {E} _ {X _ {1}, Z _ {1: k}} \left[ \mathbb {1} \left\{E _ {1} ^ {C} \right\} \inf  _ {a _ {1}} \sup  _ {y _ {1}} \mathcal {R} _ {1} \left(\mathcal {F}, X _ {1}, y _ {1}, a _ {1}\right) \right] \\ \leq \mathbb {E} _ {X _ {1}, Z _ {1: k}} \left[ \mathbb {1} \left\{E _ {1} \right\} \inf  _ {a _ {1}} \sup  _ {y _ {1}} \mathcal {R} _ {1} \left(\mathcal {F}, X _ {1}, y _ {1}, a _ {1}\right) \right] + \mathbb {P} \left(E _ {1} ^ {c}\right) (3) \\ \leq \mathbb {E} _ {Z _ {1: k}} \left[ \max  _ {X _ {1} \in Z _ {1: k}} \inf  _ {a _ {1}} \sup  _ {y _ {1}} \mathcal {R} _ {1} (\mathcal {F}, X _ {1}, y _ {1}, a _ {1}) \right] + (1 - \sigma) ^ {k} (4) \\ = \underline {{\mathcal {R}}} _ {T} ^ {k T} (\mathcal {F}) + (1 - \sigma) ^ {k}, (5) \\ \end{array}
$$
where (3) uses that  $\inf_{a_1}\sup_{y_1}\mathcal{R}_1(\mathcal{F},X_1,y_1,a_1)\leq 1^4$ , (4) follows by the coupling lemma and (5) follows from the definition of transductive learning regret. The next step is to generalize this to arbitrary  $T$ . The key aspect that makes this possible is that for all  $t\le T$ , we have  $D_{t}\in \Delta_{\sigma}(\mu)$  even conditioned on the past, allowing us to apply the coupling lemma. Furthermore, we need that  $\mathcal{R}_T\le T$  for arbitrary sequences which is indeed guaranteed for reasonable losses such as the log-loss as noted above.
We now move to general case. We will prove this inductively. Assume that we have used the coupling lemma till time  $t - 1$  and replaced the samples from the smooth distributions with samples from the uniform. That is assume the induction hypothesis, for time  $t$  as
$$
\begin{array}{l} \mathcal {R} _ {T} \leq \underset {\{Z _ {1: k} \} \sim \mu X _ {1} \in Z _ {1} ^ {k}} {\mathbb {E}} \max  _ {a _ {1}} \inf  _ {y _ {1}} \dots \sup  _ {\mathcal {D} _ {t}} \underset {X _ {t} \sim \mathcal {D} _ {t}} {\mathbb {E}} \inf  _ {a _ {t}} \sup  _ {y _ {t}} \dots \\ \dots \sup  _ {\mathcal {D} _ {T}} \underset {X _ {T} \sim \mathcal {D} _ {T}} {\mathbb {E}} \inf  _ {a _ {T}} \sup  _ {y _ {T}} \mathcal {R} (\mathcal {F}, X _ {1: T}, y _ {1: T}, a _ {1: T}) + T (t - 1) (1 - \sigma) ^ {k} \\ \end{array}
$$
Using the coupling lemma, we have that there exists a coupling  $\Pi_t$  such that  $X_t, Z_{t,1} \ldots Z_{t,k} \sim \Pi_t$  and an event  $E_t = \left\{X_t \in \{Z_{t,1} \ldots Z_{t,k}\}\right\}$  that occurs with probability  $1 - (1 - \sigma)^k$ . Using  $Z_t := \{Z_{t,1} \ldots Z_{t,k}\}$  we have
$$
\begin{array}{l} \mathop{\mathbb{E}}_{Z_{1}\sim \mu^{k}}\max_{X_{1}\in Z_{1}}\inf_{a_{1}}\sup_{y_{1}}\ldots \sup_{\mathcal{D}_{t}}\mathop{\mathbb{E}}_{X_{t}\sim \mathcal{D}_{t}}\inf_{a_{t}}\sup_{y_{t}}\ldots \sup_{\mathcal{D}_{T}}\mathop{\mathbb{E}}_{X_{T}\sim \mathcal{D}_{T}}\inf_{a_{T}}\sup_{y_{T}}\mathcal{R}(\mathcal{F},X_{1:T},y_{1:T},a_{1:T}) \\ \leq \underset {Z _ {1} \sim \mu^ {k}} {\mathbb {E}} \max  _ {X _ {1} \in Z _ {1}} \inf  _ {a _ {1}} \sup  _ {y _ {1}} \dots \sup  _ {\mathcal {D} _ {t}} \underset {X _ {t}, Z _ {t} \sim \Pi_ {t}} {\mathbb {E}} \inf  _ {a _ {t}} \sup  _ {y _ {t}} \dots \sup  _ {\mathcal {D} _ {T}} \underset {X _ {T} \sim \mathcal {D} _ {T}} {\mathbb {E}} \inf  _ {a _ {T}} \sup  _ {y _ {T}} \mathcal {R} (\mathcal {F}, X _ {1: T}, y _ {1: T}, a _ {1: T}) \\ \leq \underset {Z _ {1} \sim \mu^ {k}} {\mathbb {E}} \max  _ {X _ {1} \in Z _ {1}} \inf  _ {a _ {1}} \sup  _ {y _ {1}} \dots \sup  _ {\mathcal {D} _ {t}} \underset {X _ {t}, Z _ {t} \sim \Pi_ {t}} {\mathbb {E}} \left[ \mathbb {1} [ E _ {t} ] \left(\inf  _ {a _ {t}} \sup  _ {y _ {t}} \dots \sup  _ {\mathcal {D} _ {T}} \underset {X _ {T} \sim \mathcal {D} _ {T}} {\mathbb {E}} \inf  _ {a _ {T}} \sup  _ {y _ {T}} \mathcal {R} (\mathcal {F}, X _ {1: T}, y _ {1: T}, a _ {1: T})\right) \right] \\ + \underset {Z _ {1} \sim \mu^ {k}} {\mathbb {E}} \max  _ {X _ {1} \in Z _ {1}} \inf  _ {a _ {1}} \sup  _ {y _ {1}} \dots \sup  _ {\mathcal {D} _ {t}} \underset {X _ {t}, Z _ {t} \sim \Pi_ {t}} {\mathbb {E}} \left[ \mathbb {1} \left[ E _ {t} ^ {c} \right] \left(\inf  _ {a _ {t}} \sup  _ {y _ {t}} \dots \sup  _ {\mathcal {D} _ {T}} \underset {X _ {T} \sim \mathcal {D} _ {T}} {\mathbb {E}} \inf  _ {a _ {T}} \sup  _ {y _ {T}} \mathcal {R} (\mathcal {F}, X _ {1: T}, y _ {1: T}, a _ {1: T})\right) \right] \\ \end{array}
$$
This holds for the log-loss by using the trivial strategy of using a uniform probability assignment at each step.
$$
\begin{array}{l} \leq \mathbb {E} _ {Z _ {1} \sim \mu^ {k}} \max  _ {X _ {1} \in Z _ {1}} \inf  _ {a _ {1}} \sup  _ {y _ {1}} \dots \sup  _ {\mathcal {D} _ {t}} \mathbb {E} _ {X _ {t}, Z _ {t} \sim \Pi_ {t}} \left[ \mathbb {1} [ E _ {t} ] \left(\inf  _ {a _ {t}} \sup  _ {y _ {t}} \dots \sup  _ {\mathcal {D} _ {T}} \mathbb {E} _ {X _ {T} \sim \mathcal {D} _ {T}} \inf  _ {a _ {T}} \sup  _ {y _ {T}} \mathcal {R} (\mathcal {F}, X _ {1: T}, y _ {1: T}, a _ {1: T})\right) \right] \\ + T \left(1 - \sigma\right) ^ {k} \\ \leq \underset {Z _ {1} \sim \mu^ {k}} {\mathbb {E}} \max  _ {X _ {1} \in Z _ {1}} \inf  _ {a _ {1}} \sup  _ {y _ {1}} \dots \sup  _ {\mathcal {D} _ {t}} \underset {Z _ {t} \sim \Pi_ {t}} {\mathbb {E}} \max  _ {X _ {t} \in Z _ {t}} \left(\inf  _ {a _ {t}} \sup  _ {y _ {t}} \dots \sup  _ {\mathcal {D} _ {T}} \underset {X _ {T} \sim \mathcal {D} _ {T}} {\mathbb {E}} \inf  _ {a _ {T}} \sup  _ {y _ {T}} \mathcal {R} (\mathcal {F}, X _ {1: T}, y _ {1: T}, a _ {1: T})\right) \\ + T (1 - \sigma) ^ {k} \\ = \mathbb {E} _ {Z _ {1} \sim \mu^ {k}} \max  _ {X _ {1} \in Z _ {1}} \inf  _ {a _ {1}} \sup  _ {y _ {1}} \dots \mathbb {E} _ {Z _ {t} \sim \mu^ {k}} \max  _ {X _ {t} \in Z _ {t}} \left(\inf  _ {a _ {t}} \sup  _ {y _ {t}} \dots \sup  _ {\mathcal {D} _ {T}} \mathbb {E} _ {X _ {T} \sim \mathcal {D} _ {T}} \inf  _ {a _ {T}} \sup  _ {y _ {T}} \mathcal {R} (\mathcal {F}, X _ {1: T}, y _ {1: T}, a _ {1: T})\right) \\ + T (1 - \sigma) ^ {k} \\ \end{array}
$$
Combining with the induction hypothesis, gives us the induction hypothesis for the next  $t$  as required. The desired result follows by upper bounding the average with the supremum over all subsets of size  $kT$ .
First, recall the notion of the global sequential covering for a class Wu et al. [2022b].
Definition C.1 (Global Sequential Covering Wu et al. [2022b]). For any class  $\mathcal{F}$ , we say that  $\mathcal{F}_{\alpha}^{\prime} \subset \mathcal{X}^{*} \to [0,1]$  is a global sequential  $\alpha$ -covering of  $\mathcal{F}$  at scale  $T$  if for any sequence  $x_{1:T}$  and  $h \in \mathcal{F}$ , there is a  $h^{\prime} \in \mathcal{F}^{\prime}$  such that for all  $i$ ,
$$
\left| h \left(x _ {i}\right) - h ^ {\prime} \left(x _ {1: i}\right) \right| \leq \alpha .
$$
Theorem C.1 (Wu et al. [2022b]). If  $\mathcal{F}_{\alpha}^{\prime}$  is a global sequential  $\alpha$ -covering of  $\mathcal{F}$  at scale  $T$ , then
$$
\mathcal {R} _ {T} \left(\mathcal {F}\right) \leq \inf  _ {\alpha > 0} \left\{2 \alpha T + \log \left| \mathcal {F} _ {\alpha} ^ {\prime} \right| \right\}.
$$
To finish the proof note that a  $\epsilon$ -cover in the sense of Definition 3.1 gives a global sequential cover in the sense of Definition C.1.
D VC Classes
In this section, we construct a probability assignment for the case when  $\mathcal{F} \subset \{\mathcal{X} \to [0,1]\}$  is a VC class. To motivate this probability assignment, consider the no-context case, which is a classic problem in information theory, where the (asymptotically) optimal probability assignment is known to be the Krichevsky and Trofimov [1981] (KT) probability assignment which is a Bayesian mixture of the form
$$
q _ {\mathrm {K T}} (y _ {1: T}) = \int_ {0} ^ {1} p _ {\theta} (y _ {1: T}) w (\theta) d \theta
$$
for a particular prior  $w(\theta)$ . This can be written sequentially as  $q_{\mathrm{KT}}(1|y_{1:t - 1}) = \frac{\sum_{i = 1}^{t - 1}y_i + 1 / 2}{t - 1 + 1}$  leading to it sometimes being called the add-1/2 probability assignment; by choosing  $w(\theta)$  to be  $\operatorname{Beta}(\beta, \beta)$  prior one can achieve a corresponding add-  $\beta$  probability assignment. We extend the mixture idea to the contextual case. In particular, for functions  $f_1, \ldots, f_m \in \mathcal{F}$ , one can choose a mixture probability assignment as
$$
\prod_ {i = 1} ^ {t} q \left(y _ {i} \mid x _ {1: i}, y _ {1: i - 1}\right) =: q \left(y _ {1: t} \| x _ {1: t}\right) = \frac {1}{m} \sum_ {j = 1} ^ {m} \prod_ {i = 1} ^ {t} \left(\frac {p _ {f _ {j}} \left(y _ {i} \mid x _ {i}\right) + \beta}{1 + 2 \beta}\right).
$$
Note that once a mixture  $q(y_{1:t} \| x_{1:t})$  has been defined for arbitrary  $x_{1:t}, y_{1:t}$ , the probability assignment at time  $t$  (or equivalently, the predicted probability with which the upcoming bit is 1) can be defined as  $q(1|x_{1:t}, y_{1:t-1}) = \frac{q(y_{1:t-1} 1 \| x_{1:t})}{q(y_{1:t-1} \| x_{1:t-1})}$ ; in particular, this prediction depends only on the observed history  $x_{1:t}, y_{1:t-1}$  and not the future  $y_t$ .
This is the approach employed presently with a carefully chosen  $f_{1}, \ldots, f_{m}$ . We remark that for VC classes this mixture approach may be extended to any mixable [Cesa-Bianchi and Lugosi, 2006, Chapter 3] loss.
First consider VC classes more carefully: i.e. each  $f \in \mathcal{F}^{\mathrm{VC}}$  is characterized by three things: a set  $A \subseteq \mathcal{X}$ , where  $A \in \mathcal{A} \subset 2^{\mathcal{X}}$  with the VC dimension of the collection  $\mathcal{A}$  being  $d < \infty$ ; as well as two numbers  $\theta_0, \theta_1 \in [0,1]$ . Then, we have
$$
f _ {A, \theta_ {0}, \theta_ {1}} (x) = \theta_ {0} \mathbb {1} \{x \in A \} + \theta_ {1} \mathbb {1} \{x \in A ^ {C} \}.
$$
The following equivalent representation of this hypothesis class is more convenient to use. We consider each  $f$  to be characterized by a tuple  $f = (g, \theta_0, \theta_1)$  where
1.  $\theta_0, \theta_1 \in [0, 1]$
2.  $g\in \mathcal{G}\subset \{\mathcal{X}\to \{0,1\} \}$
In other words,  $g$  belongs to a class  $\mathcal{G}$  of binary functions—this is simply the class of functions  $\{x \mapsto \mathbb{1}\{x \notin A\} | A \in \mathcal{A}\}$  in the original notation; so that clearly  $\mathrm{VCdim}(\mathcal{G}) = d$ . Then, we have  $p_f(\cdot | x) = p_{g, \theta_0, \theta_1}(\cdot | x) = \operatorname{Bernoulli}(\theta_0)$  if  $g(x) = 0$ ; and  $p_{g, \theta_0, \theta_1}(\cdot | x) = \operatorname{Bernoulli}(\theta_1)$  otherwise.
Recalling the definition of regret against a particular  $f = (g, \theta_0, \theta_1)$  for a sequential probability assignment strategy  $\mathcal{Q} = \{q(\cdot | x_{1:t}, y_{1:t-1})\}_{t=1}^T$
$$
\begin{array}{l} \mathcal {R} _ {T} \left(f, x _ {1: T}, y _ {1: T}, \mathcal {Q}\right) = \sum_ {t = 1} ^ {T} \log \frac {1}{q \left(y _ {t} \mid x _ {1 : t} , y _ {1 : t - 1}\right)} - \sum_ {t = 1} ^ {T} \log \frac {1}{p _ {f} \left(y _ {t} \mid x _ {t}\right)} \tag {6} \\ = \log \frac {p _ {f} \left(y _ {1 : T} \mid x _ {1 : T}\right)}{q \left(y _ {1 : T} \| x _ {1 : T}\right)} \\ \end{array}
$$
where  $q(y_{1:T}\| x_{1:T})\coloneqq \prod_{t = 1}^{T}q(y_{t}|x_{1:t},y_{1:t - 1}).$
In the smoothed analysis case, we have  $X_{t}\sim \mathcal{D}_{t}$  where  $\mathcal{D}_t$  for all  $t$  is  $\sigma$ -smoothed. Recall that in this case, we are concerned with the regret
$$
\begin{array}{l} \mathcal {R} _ {T} (\mathcal {F}, \sigma , \mathcal {Q}) = \max  _ {\mathcal {D}: \sigma - \text {s m o o t h e d}} \mathbb {E} _ {X _ {1: T}} \left[ \max  _ {y _ {1: T}} \sup  _ {f \in \mathcal {F}} \frac {p _ {f} \left(y _ {1 : T} \mid X _ {1 : T}\right)}{q \left(y _ {1 : T} \| X _ {1 : T}\right)} \right] \\ = \max  _ {\mathcal {D}: \sigma \text {- s m o o t h e d}} \mathbb {E} _ {X _ {1: T}} \left[ \max  _ {y _ {1: T}} \sup  _ {g \in \mathcal {G}} \max  _ {\theta_ {0}, \theta_ {1}} \frac {p _ {g , \theta_ {0} , \theta_ {1}} (y _ {1 : T} | X _ {1 : T})}{q (y _ {1 : T} \| X _ {1 : T})} \right]. \\ \end{array}
$$
D.1 Proposed probability assignment
Let  $\mu$  be the dominating measure for the  $\sigma$ -smoothed distribution of  $X_{1:T}$ . Let  $g_1, \ldots, g_{m_\epsilon} \in \mathcal{G}$  be an  $\epsilon$ -cover of the function class  $\mathcal{G}$  under the metric  $\delta_\mu(g_1, g_2) = \operatorname{Pr}_{X \sim \mu}(g_1(X) \neq g_2(X))$ . The following lemma bounds  $m_\epsilon$ .
Lemma D.1 (Covering number of VC classes under the metric  $\delta$ , Vershynin [2018]).
$$
m _ {\epsilon} \leq \left(\frac {1}{\epsilon}\right) ^ {c d}
$$
for an absolute constant  $c$ .
Following the idea of using a mixture probability assignment, we take a uniform mixture over  $g_{1},\ldots ,g_{m_{\epsilon}}$  and  $\theta_0,\theta_1$  so that
$$
q \left(y _ {1: t} \| x _ {1: t}\right) = \frac {1}{m _ {\epsilon}} \sum_ {i = 1} ^ {m _ {\epsilon}} \int_ {0} ^ {1} \int_ {0} ^ {1} p _ {g _ {i}, \theta_ {0}, \theta_ {1}} \left(y _ {1: t} | x _ {1: t}\right) d \theta_ {0} d \theta_ {1}
$$
and consequently the sequential probability assignment (or equivalently, the probability assigned to 1) is
$$
q (1 | x _ {1: t}, y _ {1: t - 1}) = \frac {q (y _ {1 : t - 1} 1 \| x _ {1 : t})}{q (y _ {1 : t - 1} \| x _ {1 : t - 1})}.
$$
One can observe that  $q(0|x_{1:t},y_{1:t - 1}),q(1|x_{1:t},y_{1:t - 1}) > 0$  and  $q(0|x_{1:t},y_{1:t - 1}) + q(1|x_{1:t},y_{1:t - 1}) = 1$  so that  $q$  is a legitimate probability assignment. Let the strategy induced by this uniform mixture be called  $\mathcal{Q}^{\mathrm{VC}}$ .
D.2 Analysis of  $\mathcal{Q}^{\mathrm{VC}}$  for smoothed adversaries
We note from (6) that for the  $\mathcal{Q}^{\mathrm{VC}}$  as defined in the last section, we have
$$
\begin{array}{l} \mathcal {R} _ {T} \left(\left(g ^ {*}, \theta_ {0} ^ {*}, \theta_ {1} ^ {*}\right), x _ {1: T}, y _ {1: T}, \mathcal {Q} ^ {\mathrm {V C}}\right) = \log m _ {\epsilon} + \log \frac {p _ {g ^ {*} , \theta_ {0} ^ {*} , \theta_ {1} ^ {*}} \left(y _ {1 : T} \mid x _ {1 : T}\right)}{\sum_ {i = 1} ^ {m _ {\epsilon}} \int_ {0} ^ {1} \int_ {0} ^ {1} p _ {g _ {i} , \theta_ {0} , \theta_ {1}} \left(y _ {1 : T} \mid x _ {1 : T}\right) d \theta_ {0} d \theta_ {1}} \\ \leq \log m _ {\epsilon} + \log \frac {p _ {g ^ {*} , \theta_ {0} ^ {*}} , \theta_ {1} ^ {*} \left(y _ {1 : T} \mid x _ {1 : T}\right)}{\int_ {0} ^ {1} \int_ {0} ^ {1} p _ {g _ {i ^ {*}}, \theta_ {0}, \theta_ {1}} \left(y _ {1 : T} \mid x _ {1 : T}\right) d \theta_ {0} d \theta_ {1}} \tag {7} \\ \end{array}
$$
where  $g_{i^*} \in \{g_1, \ldots, g_{m_e}\}$  is the function  $i^* \in [m]$  that minimizes the Hamming distance between the binary strings  $(g_{i^*}(x_1), \ldots, g_{i^*}(x_T))$  and  $(g^*(x_1), \ldots, g^*(x_T))$ .
We now take a closer look at the second term of (7). Firstly, note that for any  $(g,\theta_0,\theta_1)$  we have  $p_{g,\theta_0,\theta_1}(y_{1:T}|x_{1:T}) =$
$$
\begin{array}{l} \prod_ {t = 1} ^ {T} p _ {g, \theta_ {0}, \theta_ {1}} (y _ {t} | x _ {t}) = \prod_ {t = 1} ^ {T} \theta_ {g \left(x _ {t}\right)} ^ {y _ {t}} (1 - \theta_ {g \left(x _ {t}\right)}) ^ {1 - y _ {t}} = \prod_ {t: g \left(x _ {t}\right) = 0} \theta_ {0} ^ {y _ {t}} (1 - \theta_ {0}) ^ {1 - y _ {t}} \prod_ {t: g \left(x _ {t}\right) = 1} \theta_ {1} ^ {y _ {t}} (1 - \theta_ {1}) ^ {1 - y _ {t}} \\ = \theta_ {0} ^ {k _ {0} (g; x _ {1: T}, y _ {1: T})} (1 - \theta_ {0}) ^ {n _ {0} (g; x _ {1: T}) - k _ {0} (g; x _ {1: T}, y _ {1: T})} \\ \theta_ {1} ^ {k _ {1} (g; x _ {1: T}, y _ {1: T})} (1 - \theta_ {1}) ^ {n _ {1} (g; x _ {1: T}) - k _ {1} (g; x _ {1: T}, y _ {1: T})}, \\ \end{array}
$$
where for  $j\in \{0,1\}$
$$
k _ {j} (g; x _ {1: T}, y _ {1: T}) = | \{t: y _ {t} = 1, g (x _ {t}) = j \} |
$$
$$
n _ {0} (g; x _ {1: T}) = | \{t: g (x _ {t}) = j \} |.
$$
Next, we note that for any  $g \in \mathcal{G}$
$$
\begin{array}{l} \int_ {0} ^ {1} \int_ {0} ^ {1} p _ {g, \theta_ {0}, \theta_ {1}} \left(y _ {1: T} \mid x _ {1: T}\right) d \theta_ {0} d \theta_ {1} \\ = \left(\int_ {0} ^ {1} \theta_ {0} ^ {k _ {0} (g; x _ {1: T}, y _ {1: T})} (1 - \theta_ {0}) ^ {n _ {0} (g; x _ {1: T}) - k _ {0} (g; x _ {1: T}, y _ {1: T})} d \theta_ {0}\right). \\ \left(\int_ {0} ^ {1} \theta_ {1} ^ {k _ {1} (g; x _ {1: T}, y _ {1: T})} (1 - \theta_ {1}) ^ {n _ {1} (g; x _ {1: T}) - k _ {0} (g; x _ {1: T}, y _ {1: T})} d \theta_ {1}\right) \\ = \frac {1}{\binom {n _ {0} (g ; x _ {1 : T})} {k _ {0} (g ; x _ {1 : T} , y _ {1 : T})}} \left(n _ {0} (g; x _ {1: T}) + 1\right) \frac {1}{\binom {n _ {1} (g ; x _ {1 : T})} {k _ {1} (g ; x _ {1 : T} , y _ {1 : T})}} \left(n _ {1} (g; x _ {1: T}) + 1\right) \tag {8} \\ \geq \frac{1}{n^{2}\binom{n_{0}(g;x_{1:T})}{k_{0}(g;x_{1:T},y_{1:T})}\binom{n_{1}(g;x_{1:T})}{k_{1}(g;x_{1:T},y_{1:T})}} \\ \end{array}
$$
where (8) follows from properties of the Laplace probability assignment (or that of the Beta/Gamma functions), captured by Lemma D.2.
Lemma D.2. For  $k \leq n \in \mathbb{N}$ ,
$$
\int_ {0} ^ {1} t ^ {k} (1 - t) ^ {n - k} d t = \frac {\Gamma (k + 1) \Gamma (n - k + 1)}{\Gamma (n + 2)} = \frac {1}{(n + 1) \binom {n} {k}}
$$
where  $\Gamma (\cdot)$  represents the Gamma function.
Putting this back into (7) (and rearranging), we have