SlowGuess/NeurIPS_2023_OCR huggingface dataset:
We prove by induction that for each  $i \in [k]$ , all labels chosen by the adversary for the subsequences prior to  $q_i$  are  $\mathcal{H}$ -realizable, and additionally there exists an instance in subsequence  $q_i$  on which the adversary can force a mistake regardless of the learners' predictions. The base case is that the adversary can always force a mistake on the first instance,  $q_1$ , by choosing the label opposite to the learner's prediction (both labels 0 and 1 are  $\mathcal{H}$ -realizable for this instance). Subsequently, for any  $i > 1$ , note that by the induction hypothesis, the labels chosen by the adversary for all instances in the previous subsequences are  $\mathcal{H}$ -realizable. In particular there exists an index  $a \in [N]$  such that instance  $x_a$  has already been labeled, and all the labels chosen so far are consistent with  $h_a$ . Let  $b$  be the minimal integer such that  $b > a$  and  $x_b$  has also been labeled. Then  $x_a$  and all labeled instances with smaller indices received label 1, while  $x_b$  and all labeled instances with larger indices received label 0. Because the sequence is dyadic, subsequence  $q_i$  contains an element  $x_m$  such that  $a < m < b$ . The adversary can force a mistake on  $x_m$ , because  $h_a$  and  $h_m$  agree on all previously labeled instances, but disagree on  $x_m$ .
Claim 3.4 is used in the proof of the trichotomy (Theorem 4.1, below).
Finally, we note that for every  $d \in \mathbb{N}$  there exists a hypothesis class  $\mathcal{H}$  such that  $d = \mathrm{TD}(\mathcal{H})$  and
$$
\forall n \in \mathbb {N}: M (\mathcal {H}, n) = \min  \{d, n \}.
$$
Indeed, take  $\mathcal{X} = [d]$  and  $\mathcal{H} = \{0,1\}^{\mathcal{X}}$ . The upper bound holds because  $|\mathcal{X}| = d$ , and the lower bound holds by Item 2 in Theorem 4.1, because  $\mathrm{VC}(\mathcal{H}) = d$ .
4 Trichotomy
Theorem 4.1. Let  $\mathcal{X}$  be a set, let  $\mathcal{H} \subseteq \{0,1\}^{\mathcal{X}}$ , and let  $n \in \mathbb{N}$  such that  $n \leq |\mathcal{X}|$ .
1. If  $\mathsf{VC}(\mathcal{H}) = \infty$  then  $M(\mathcal{H}, n) = n$ .
2. Otherwise, if  $\mathsf{VC}(\mathcal{H}) = d < \infty$  and  $\mathsf{LD}(\mathcal{H}) = \infty$  then
$$
\max  \left\{\min  \{d, n \}, \lfloor \log (n) \rfloor \right\} \leq M (\mathcal {H}, n) \leq O (d \log (n / d)). \tag {4}
$$
Furthermore, each of the bounds in Eq. (4) is tight for some classes. The  $\Omega(\cdot)$  and  $O(\cdot)$  notations hide universal constants that do not depend on  $\mathcal{X}$  or  $\mathcal{H}$ .
3. Otherwise, there exists an integer  $C(\mathcal{H}) \leq \mathrm{LD}(\mathcal{H})$  (that depends on  $\mathcal{X}$  and  $\mathcal{H}$  but does not depend on  $n$ ) such that  $M(\mathcal{H}, n) \leq C(\mathcal{H})$ .
Proof of Theorem 4.1. For Item 1, assume  $\mathsf{VC}(\mathcal{H}) = \infty$ . Then there exists a set  $X = \{x_{1},\ldots ,x_{n}\} \subseteq \mathcal{X}$  of cardinality  $n$  that is shattered by  $\mathcal{H}$ . The adversary can force the learner to make  $n$  mistakes by selecting the sequence  $(x_{1},\dots,x_{n})$ , and then selecting labels  $y_{t} = 1 - \hat{y}_{t}$  for all  $t\in [n]$ . This choice of labels is  $\mathcal{H}$ -realizable because  $X$  is a shattered set.
To obtain the upper bound in Item 2 the learner can use the halving algorithm, as follows. Let  $x = (x_{1},\ldots ,x_{n})$  be the sequence chosen by the adversary, and let  $\mathcal{H}|_x$  denote the collection of functions from elements of  $x$  to  $\{0,1\}$  that are restrictions of functions in  $\mathcal{H}$ . For each  $t\in \{0,\dots ,n\}$ , let
$$
\mathcal {H} _ {t} = \left\{f \in \mathcal {H} | _ {x}: (\forall i \in [ t ]: f (x _ {i}) = y _ {i}) \right\}
$$
be a set called the version space at time  $t$ . At each step  $t \in [n]$ , the learner makes prediction
$$
\hat {y} _ {t} = \operatorname {a r g m a x} _ {b \in \{0, 1 \}} \left| \left\{f \in \mathcal {H} _ {t - 1}: f (x _ {t}) = b \right\} \right|.
$$
In words, the learner chooses  $\hat{y}_t$  according to a majority vote among the functions in version space  $\mathcal{H}_{t-1}$ , and then any function whose vote was incorrect is excluded from the next version space,  $\mathcal{H}_t$ . This implies that for any  $t \in [n]$ , if the learner made a mistake at time  $t$  then
$$
\left| \mathcal {H} _ {t} \right| \leq \frac {1}{2} \cdot \left| \mathcal {H} _ {t - 1} \right|. \tag {5}
$$
Let  $M = M(\mathcal{H}, n)$ . The adversary selects  $\mathcal{H}$ -realizable labels, so  $\mathcal{H}_n$  cannot be empty. Hence, applying Eq. (5) recursively yields
$$
1 \leq | \mathcal {H} _ {n} | \leq 2 ^ {- M} \cdot | \mathcal {H} _ {0} | \leq 2 ^ {- M} \cdot O \left(\left(n / d\right) ^ {d}\right),
$$
where the last inequality follows from  $\mathsf{VC}(\mathcal{H}_0)\leq \mathsf{VC}(\mathcal{H}) = d$  and the Sauer-Shelah-Perles lemma (Theorem 2.6). Hence  $M = O(d\log (n / d))$ , as desired.
For the  $\min\{d, n\}$  lower bound in Item 2, if  $n \leq d$  then the adversary can force  $n$  mistakes by the same argument as in Item 1. For the logarithmic lower bound in Item 2, the assumption that  $\mathrm{LD}(\mathcal{H}) = \infty$  and Theorem 3.3 imply that  $\mathrm{TD}(\mathcal{H}) = \infty$ , and in particular  $\mathrm{TD}(\mathcal{H}) \geq n$ , and this implies the bound by Claim 3.4.
For Item 3, the assumption  $\mathsf{LD}(\mathcal{H}) = k < \infty$  and Theorem 2.5 imply that for any  $n$ , the learner will make at most  $k$  mistakes. This is because the adversary in the transductive setting is strictly weaker than the adversary in the standard online setting. So there exists some  $C(\mathcal{H}) \in \{0, \dots, k\}$  as desired.
Remark 4.2. One can use Theorem 3.1 to obtain a lower bound for the case of Item 2 in Theorem 4.1. However, that yields a lower bound of  $\Omega (\log \log (n))$ , which is exponentially weaker than the bound we show.
5 Multiclass Setting
The trichotomy of Theorem 4.1 can be generalized to the multiclass setting, in which the label set  $\mathcal{V}$  contains more than two labels. In this setting, the VC dimension is replaced by the Natarajan dimension [Nat89], denoted ND, and the Littlestone dimension is generalized in the natural way. The result holds for finite sets  $\mathcal{V}$ .
Theorem 5.1 (Informal Version of Theorem B.3). Let  $\mathcal{X}$  be a set, let  $\mathcal{Y}$  be a finite set, and let  $\mathcal{H} \subseteq \mathcal{Y}^{\mathcal{X}}$ . Then  $\mathcal{H}$  satisfies precisely one of the following:
1.  $M(\mathcal{H},n) = n$  . This happens if  $\mathrm{ND}(\mathcal{H}) = \infty$
2.  $M(\mathcal{H},n) = \Theta (\log (n))$  . This happens if  $\mathsf{ND}(\mathcal{H}) <   \infty$  and  $\mathsf{LD}(\mathcal{H}) = \infty$
3.  $M(\mathcal{H},n) = O(1)$ . This happens if  $\mathsf{LD}(\mathcal{H}) < \infty$ .
The proof of Theorem 5.1 appears in Appendix B, along with the necessary definitions. The main innovation in the proof involves the use of the multiclass threshold bounds developed in Appendix D, which in turn rely on a basic result from Ramsey theory, stated Appendix C.
5.1 The Case of an Infinite Label Set
It is interesting to observe that the analogy between the binary classification and multiclass classification settings breaks down when the label set  $\mathcal{Y}$  is not finite.
Example 5.2. There exists a class  $\mathcal{G} \subseteq \mathcal{Y}^{\mathcal{X}}$  such that  $\mathcal{V}$  is countable,  $\operatorname{LD}(\mathcal{G})$  is infinite, but the class is learnable with a mistake bound of  $M(\mathcal{G}, n) = 1$ . To see this, let  $\mathcal{X}$  be countable, and let  $\mathcal{H} \subseteq \{0, 1\}^{\mathcal{X}}$  be a class with  $\operatorname{LD}(\mathcal{H}) = \infty$ . For each  $i \in \mathbb{N}$ , let  $T_i$  be a Littlestone tree of depth  $i$  that is shattered by  $\mathcal{H}$ , and let  $\{h_1^i, \ldots, h_{2^{i+1}}^i\} \subseteq \mathcal{H}$  be a subset that witnesses the shattering. Let  $\mathcal{G} = \{g_j^i : i \in \mathbb{N} \land j \in [2^{i+1}]\}$  be a set of functions such that  $g_j^i(x) = (h_j^i(x), i, j)$  for all  $i, j$ . Let  $\mathcal{Y} = \{0, 1\} \times \mathbb{N} \times \mathbb{N}$ . Observe that  $\mathcal{G} \subseteq \mathcal{Y}^{\mathcal{X}}$  is a countable set of functions with a countable set of labels. Furthermore,  $\operatorname{LD}(\mathcal{G}) = \infty$  because  $\mathcal{G}$  shatters a sequence of suitable Littlestone trees corresponding to  $T_1, T_2, \ldots$ . However,  $\mathcal{G}$  can be learned with mistake bound 1, because a single example of the form  $(x, (h_j^i(x), i, j))$  reveals the correct labeling function  $h_j^i$ .
Recent work by  $\left[\mathrm{BCD}^{+}22\right]$  has shown that multiclass PAC learning with infinite  $\mathcal{V}$  is not characterized by the Natarajan dimension, and that instead it is characterized by the DS dimension (introduced by [DS14]). It is therefore natural to ask whether the DS dimension might also characterize multiclass transductive online learning with infinite  $\mathcal{V}$ . We show that the answer to that question is negative.
Recall the definition of the DS dimension.
Definition 5.3. Let  $d \in \mathbb{N}$ , let  $\mathcal{X}$  and  $\mathcal{Y}$  be sets, and let  $\mathcal{H} \subseteq \mathcal{Y}^{\mathcal{X}}$ . For an index  $i \in [d]$  and vectors  $y = (y_{1}, \ldots, y_{d}) \in \mathcal{Y}^{d}$ ,  $y' = (y_{1}', \ldots, y_{d}') \in \mathcal{Y}^{d}$ , we say that  $y$  and  $y'$  are  $i$ -neighbors, denoted  $y \sim_{i} y'$ , if  $\{j \in [d] : y_{j} \neq y_{j}'\} = \{i\}$ . We say that  $\mathcal{C} \subseteq \mathcal{Y}^{d}$  is a d-pseudocube if  $\mathcal{C}$  is non-empty and finite, and
$$
\forall y \in \mathcal {C} \forall i \in [ d ] \exists y ^ {\prime} \in \mathcal {C}: y \sim_ {i} y ^ {\prime}.
$$
For a vector  $x = (x_{1},\ldots ,x_{d})\in \mathcal{X}^{d}$ , we say that  $\underline{\mathcal{H}}$  DS-shatters  $x$  if the set
$$
\mathcal {H} | _ {x} := \left\{\left(h \left(x _ {1}\right), \dots , h \left(x _ {d}\right)\right): h \in \mathcal {H} \right\} \subseteq \mathcal {Y} ^ {d}
$$
contains a  $d$ -pseudocube.
Finally, the Daniely-Shalev-Shwartz (DS) dimension of  $\mathcal{H}$  is
$$
\mathsf {D S} (\mathcal {H}) = \sup  \left\{d \in \mathbb {N}: \left(\exists x \in \mathcal {X} ^ {d}: \mathcal {H} D S - s h a t t e r d x\right) \right\}.
$$
See  $\left[\mathrm{BCD}^{+}22\right]$  for figures and further discussion of the DS dimension.
The following claim shows that the DS dimension does not characterize transductive online learning, even when  $\mathcal{V}$  is finite.
Claim 5.4. For every  $n \in \mathbb{N}$ , there exists a hypothesis class  $\mathcal{H}_n$  such that  $\mathrm{DS}(\mathcal{H}_n) = 1$  but the adversary in transductive online learning can force at least  $M(\mathcal{H}_n, n) = n$  mistakes.
Proof. Fix  $n \in \mathbb{N}$  and let  $\mathcal{X} = \{0, 1, 2, \ldots, n\}$ . Consider a complete binary tree  $T$  of depth  $n$  such that for each  $x \in \mathcal{X}$ , all the nodes at depth  $x$  (at distance  $x$  from the root) are labeled by  $x$ , and each edge in  $T$  is labeled by a distinct label. Let  $\mathcal{H}$  be a minimal hypothesis class that shatters  $T$ , namely,  $\mathcal{H}$  shatters  $T$  and there does not exist a strict subset of  $\mathcal{H}$  that shatters  $T$ .
Observe that  $M(\mathcal{H}_n, n) = n$ , because the adversary can present the sequence  $0, 1, 2, \ldots, n - 1$  and force a mistake at each time step. To see that  $\mathrm{DS}(\mathcal{H}_n) = 1$ , assume for contradiction that there exists a vector  $x = (x_1, x_2) \in \mathcal{X}^2$  that is DS-shattered by  $\mathcal{H}_n$ , namely, there exists a 2-pseudocube  $\mathcal{C} \subseteq \mathcal{H}|_x$ . Note that  $x_1 \neq x_2$ , and without loss of generality  $x_1 < x_2$  ( $\mathcal{H}$  DS-shatters  $(x_1, x_2)$  if and only if it DS-shatters  $(x_2, x_1)$ ).
Fix  $y \in \mathcal{C}$ . So  $y = (h(x_1), h(x_2))$  for some  $h \in \mathcal{H}$ . Because  $\mathcal{C}$  is a 2-pseudocube, there exists  $y' \in \mathcal{C}$  that is a 1-neighbor of  $y$ . Namely, there exists  $g \in \mathcal{H}$  such that  $y' = (g(x_1), g(x_2)) \in \mathcal{C}$ ,  $y_1' \neq y_1$  and  $y_2' = y_2$ . However, because each edge in  $T$  has a distinct label, and  $\mathcal{H}$  is minimal, it follows that for any  $x \in \mathcal{X}$ ,
$$
g (x) = h (x) \Rightarrow (\forall x ^ {\prime} \in \{0, 1, \dots , x \}: g (x ^ {\prime}) = h (x ^ {\prime})).
$$
In particular,  $g(x_2) = y_2' = y_2 = h(x_2)$  implies  $y_1' = g(x_1) = h(x_1) = y_1$  which is a contradiction to the choice of  $y'$ .
6 Agnostic Setting
The agnostic transductive online learning setting is defined analogously to the realizable (non-agnostic) transductive online learning setting described in Section 1.1. An early work by [Cov65] observed that it is not possible for a learner to achieve vanishing regret in an agnostic online setting with complete information. Therefore, we consider a game with incomplete information, as follows.
First, the adversary selects an arbitrary sequence of instances,  $x_{1}, \ldots, x_{n} \in \mathcal{X}$ , and reveals the sequence to the learner. Then, for each  $t = 1, \ldots, n$ :
1. The adversary selects a label  $y_{t}\in \mathcal{V}$
2. The learner selects a prediction  $\hat{y}_t\in \mathcal{V}$  and reveals it to the adversary.
3. The adversary reveals  $y_{t}$  to the learner.
At each time step  $t \in [n]$ , the adversary may select any  $y_{t} \in \mathcal{V}$ , without restrictions. The learner, which is typically randomized, has the objective of minimizing the regret, namely
$$
R (A, \mathcal {H}, x, y) = \mathbb {E} [ | \{t \in [ n ]: \hat {y} _ {t} \neq y _ {t} \} | ] - \min  _ {h \in \mathcal {H}} | \{t \in [ n ]: h (x _ {t}) \neq y _ {t} \} |,
$$
where the expectation is over the learner's randomness. In words, the regret is the expected excess number of mistakes the learner makes when it plays strategy  $A$  and the adversary chooses the sequence  $x \in \mathcal{X}^n$  and labels  $y \in \mathcal{Y}^n$ , as compared to the number of mistakes made by the best fixed hypothesis  $h \in \mathcal{H}$ . We are interested in understanding the value of this game, namely
$$
R(\mathcal{H},n) = \inf_{A\in \mathcal{A}}\sup_{x\in \mathcal{X}^{n}}\sup_{y\in \mathcal{Y}^{n}}R(A,\mathcal{H},x,y),
$$
where  $\mathcal{A}$  is the set of all learner strategies. We show the following result.
Theorem 6.1. Let  $\mathcal{X}$  be a set, let  $\mathcal{H} \subseteq \{0,1\}^{\mathcal{X}}$ , and let  $n \in \mathbb{N}$  such that  $n \leq |\mathcal{X}|$ . Assume  $0 < \mathsf{VC}(\mathcal{H}) < \infty$ . Then the agnostic transductive regret for  $\mathcal{H}$  on sequences of length  $n$  is
$$
\Omega \Big (\sqrt {\mathsf {V C} (\mathcal {H}) \cdot n} \Big) \leq R (\mathcal {H}, n) \leq O \Big (\sqrt {\mathsf {V C} (\mathcal {H}) \cdot n \cdot \log (n / \mathsf {V C} (\mathcal {H}))} \Big).
$$
The upper bound in Theorem 6.1 follows directly from the Sauer-Shelah-Perles lemma (Theorem 2.6), together with the following well-known bound on the regret of the Multiplicative Weights algorithm (see, e.g., Theorem 21.10 in [SB14]).
Theorem 6.2. Let  $\mathcal{X}$  be a set and let  $\mathcal{H} \subseteq \{0,1\}^{\mathcal{X}}$  be finite. There exists an algorithm for the standard (non-transductive) agnostic online learning setting that satisfies
$$
R _ {\text {o n l i n e}} (\mathcal {H}, n) \leq \sqrt {2 \log (| \mathcal {H} |)}.
$$
Hence the name 'agnostic', implying that we make no assumptions concerning the choice of labels.
Theorem 6.2 implies the upper bound of Theorem 6.1, because the adversary in the transductive agnostic setting is weaker than the adversary in the standard agnostic setting.
We prove the lower bound of Theorem 6.1 using an anti-concentration technique from Lemma 14 of [BPS09]. The proof appears in Appendix E.
Remark 6.3. Additionally:
1. If  $\mathsf{VC}(\mathcal{H}) = 0$  (i.e., classes with a single function) then the regret is 0.
2. If  $\mathsf{VC}(\mathcal{H}) < \infty$  and  $\mathsf{LD}(\mathcal{H}) < \infty$  then the regret is  $R(\mathcal{H}, n) = O\left(\sqrt{\mathsf{LD}(\mathcal{H}) \cdot n}\right)$ , by [ABD+21] (as mentioned above). Namely, in some cases the  $\log(n)$  factor in Theorem 6.1 can be removed.
3. If  $\mathsf{VC}(\mathcal{H}) = \infty$  then the regret is  $\Omega (n)$
Some remaining open problems include:
1. Showing a sharper bound for the  $\Theta(1)$  case in the trichotomy (Theorem 4.1). Currently, there is an exponential gap between the best known upper and lower bounds for Littlestone classes.
2. Showing sharper bounds for the  $\Theta (\log n)$  cases in the trichotomy (Theorem 4.1) and multiclass trichotomy (Theorem B.3).
3. Showing a sharper bound for the agnostic case (Theorem 6.1).
4. Characterizing the number of mistakes in the multiclass setting with an infinite label set  $\mathcal{Y}$  (complementing Theorem B.3).
Zachary Chase contributed significantly to this paper. All errors are our own.
SM is a Robert J. Shillman Fellow; he acknowledges support by ISF grant 1225/20, by BSF grant 2018385, by an Azrieli Faculty Fellowship, by Israel PBC-VATAT, by the Technion Center for Machine Learning and Intelligent Systems (MLIS), and by the European Union (ERC, GENERALIZATION, 101039692). JS was supported by DARPA (Defense Advanced Research Projects Agency) contract # HR001120C0015 and the Simons Collaboration on The Theory of Algorithmic Fairness. Views and opinions expressed are those of the author(s) only and do not necessarily reflect those of the European Union, the European Research Council Executive Agency, DARPA, or the Simons Foundation. The European Union, the granting authority, DARPA, and the Simons Foundation cannot be held responsible for them.
$\left[\mathrm{ABD}^{+}21\right]$  Noga Alon, Omri Ben-Eliezer, Yuval Dagan, Shay Moran, Moni Naor, and Eylon Yogev. Adversarial laws of large numbers and optimal regret in online classification. In Samir Khuller and Virginia Vassilevska Williams, editors, STOC 2021: 53rd Annual ACM SIGACT Symposium on Theory of Computing, Virtual Event, Italy, June 21-25, 2021, pages 447-455. ACM, 2021. doi:10.1145/3406325.3451041.
[ALMM19] Noga Alon, Roi Livni, Maryanthe Malliaris, and Shay Moran. Private PAC learning implies finite littlestone dimension. In Moses Charikar and Edith Cohen, editors, Proceedings of the 51st Annual ACM SIGACT Symposium on Theory of Computing, STOC 2019, Phoenix, AZ, USA, June 23-26, 2019, pages 852-860. ACM, 2019. doi:10.1145/3313276.3316312.
[BB11] Shalev Ben-David and Shai Ben-David. Learning a classifier when the labeling is known. In Jyrki Kivinen, Csaba Szepesvári, Esko Ukkonen, and Thomas Zeugmann, editors, Algorithmic Learning Theory - 22nd International Conference, ALT 2011,
[KK05] Sham M. Kakade and Adam Kalai. From batch to transductive online learning. In Advances in Neural Information Processing Systems 18, Neural Information Processing Systems, NIPS 2005, December 5-8, 2005, Vancouver, British Columbia, Canada, pages 611-618, 2005. URL https://proceedings.neurips.cc/paper/2005/ hash/17693c91d9204b7a7646284bb3adb603-AAbstract.html.
[Kuh99] Christian Kuhlmann. On teaching and learning intersection-closed concept classes. In Paul Fischer and Hans Ulrich Simon, editors, Computational Learning Theory, 4th European Conference, EuroCOLT 1999, Nordkirchen, Germany, March 29-31, 1999, Proceedings, volume 1572 of Lecture Notes in Computer Science, pages 168-182. Springer, 1999. doi:10.1007/3-540-49097-3_14.
[Lit87] Nick Littlestone. Learning quickly when irrelevant attributes abound: A new linear-threshold algorithm. Mach. Learn., 2(4):285-318, 1987. doi:10.1007/BF00116827.
[Nat89] B. K. Natarajan. On learning sets and functions. Mach. Learn., 4:67-97, 1989. doi:10.1007/BF00114804.
[Pec08] Dmitry Pechyony. Theory and practice of transductive learning. PhD thesis, Technion - Israel Institute of Technology, Israel, 2008. URL https://technion.primo.exlibrisgroup.com/permalink/972TEC_INST/q1jq5o/alma990023032150203971.
[Sau72] Norbert Sauer. On the density of families of sets. J. Comb. Theory, Ser. A, 13(1):145-147, 1972. doi:10.1016/0097-3165(72)90019-2.
[SB14] Shai Shalev-Shwartz and Shai Ben-David. Understanding Machine Learning: From Theory to Algorithms. Cambridge University Press, 2014. doi:10.1017/CBO9781107298019.
[She72] Saharon Shelah. A combinatorial problem; stability and order for models and theories in infinitary languages. Pacific Journal of Mathematics, 41(1):247-261, 1972. doi:10.2140/pjm.1972.41.247.
[She90] Saharon Shelah. Classification Theory: And the Number of Non-Isomorphic Models, volume 92 of Studies in Logic and the Foundations of Mathematics. North-Holland, second edition, 1990.
[SKS16] Vasilis Syrgkanis, Akshay Krishnamurthy, and Robert E. Schapire. Efficient algorithms for adversarial contextual learning. In Maria-Florina Balcan and Kilian Q. Weinberger, editors, Proceedings of the 33nd International Conference on Machine Learning, ICML 2016, New York City, NY, USA, June 19-24, 2016, volume 48 of JMLR Workshop and Conference Proceedings, pages 2159-2168. JMLR.org, 2016. URL http://proceedings.mlr.press/v48/syrgkanis16.html.
[Vap82] V. Vapnik. Estimation of Dependencies Based on Empirical Data. Springer-Verlag, New York, 1982. doi:10.1007/0-387-34239-7.
[VC71] V. Vapnik and A. Chervonenkis. On the uniform convergence of relative frequencies of events to their probabilities. Theory of Probability and its Applications, 16(2):264-280, 1971. doi:10.1137/1116025.
[VC74] V. Vapnik and A. Chervonenkis. Theory of Pattern Recognition. Nauka, Moscow, 1974.
A Proof of Lower Bound for Littlestone Classes
Proof of Theorem 3.1. Let  $T$  be a Littlestone tree of depth  $d$  that is shattered by  $\mathcal{H}$ , and let  $\mathcal{H}_1 \subseteq \mathcal{H}$  be a collection of  $2^{d+1}$  functions that witness the shattering.  $T$  contains  $n_T = 2^{d+1} - 1$  nodes. The adversary selects the sequence
$$
x _ {1}, x _ {2}, \dots , x _ {n}
$$
consisting of the first  $n$  nodes of  $T$  in breadth-first order (if  $n > n_T$ , then the adversary chooses the suffix  $x_{n_T + 1},\ldots ,x_n$  arbitrarily). For each time step  $t\in [n]$ , let  $\mathcal{H}_t$  denote the version space, i.e., the subset of  $\mathcal{H}_1$  that is consistent with all previously-assigned labels. Namely, for any  $t > 1$
$$
\mathcal {H} _ {t} = \left\{h \in \mathcal {H} _ {1}: (\forall s \in [ t - 1 ]: h (x _ {s}) = y _ {s}) \right\}.
$$
Similarly, for each  $b \in \{0,1\}$ , let  $\mathcal{H}_{t,b} = \{h \in \mathcal{H}_t : h(x_t) = b\}$ .
send  $x_{1},\ldots ,x_{n}$  to learner   
 $k\gets 1$    
for  $t\gets 1,2,\dots ,n$  ..   
 $m_k\gets 2^{2^k}$ $\varepsilon_t\gets 1 / m_k$ $r_t\gets |\mathcal{H}_{t,1}| / |\mathcal{H}_t|$    
receive  $\hat{y}_t$  from learner   
 $y_{t}\gets \left\{ \begin{array}{ll}1 - \hat{y}_{t} & r_{t}\in [\varepsilon_{t},1 - \varepsilon_{t}]\\ 0 & r_{t}\in [0,\varepsilon_{t})\\ 1 & r_{t}\in (1 - \varepsilon_{t},1] \end{array} \right.$    
send  $y_{t}$  to learner   
if  $r_t\in [\varepsilon_t,1 - \varepsilon_t]$  ..   
 $k\gets k + 1$
Algorithm 1: An adversary that forces  $\Omega (\log (\mathsf{LD}(\mathcal{H}))$  mistakes.
The adversary operates according to Algorithm 1. Conceptually, at each time step  $t \in [n]$ , if  $\mathcal{H}_t$  is very unbalanced, meaning that a large majority of the functions in  $\mathcal{H}_t$  assign the same value to  $x_t$ , then the adversary chooses  $y_t$  to be that value. Otherwise, if  $\mathcal{H}_t$  is fairly balanced, the adversary forces a mistake. Note that if  $\mathcal{H}_t$  is fairly balanced then the adversary can force a mistake without violating  $\mathcal{H}$ -realizability.
We now argue that using this strategy, the adversary forces  $\Omega (\log (d))$  mistakes. Let  $F = \{t_{1},t_{2},\ldots \} = \{t\in [n]:r_{t}\in [\varepsilon_{t},1 - \varepsilon_{t}]\}$  be the set of time steps where the adversary forces a mistake. Note that in the for-loop in Algorithm 1, the value of  $k$  at the beginning of iteration  $t_k$  is  $k$  (e.g., at the beginning of iteration  $t_3$ ,  $k = 3$ ).
We argue by induction that for any  $k \in \mathbb{N}$ , if  $m_k \coloneqq 2^{2^{2k}} \leq n$  then: