SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Going beyond binary classification, recent work [HKLM22] shows that the equivalence of realizable and agnostic learnability extends across a wide variety of settings. These include models with no known characterization of learnability such as learning with arbitrary distributional assumptions and more general loss functions, as well as a host of other popular settings such as robust learning, partial learning, fair learning, and the statistical query model. This stands in contrast to our results for the distribution learning setting. We show that realizable learnability of a class of probability distributions does not imply its agnostic learnability. It is interesting and natural to explore the relationship between various notions of distribution learnability, which we have scratched the surface of in this work.
2 Learnability Implies Additive Robust Learnability
We recall Theorem 1.5, which shows that any class that is realizably learnable is also additive robustly learnable.
We prove this theorem by providing an algorithm based on classical tools for hypothesis selection [Yat85, DL96, DL97, DL01]. These methods take as input a set of samples from an unknown distribution and a collection of hypotheses distributions. If the unknown distribution is close to one of the hypotheses, then, given enough samples, the algorithm will output a close hypothesis. Roughly speaking, our algorithm looks at all large subsets of the dataset, such that at least one will correspond to an uncontaminated set of samples. A learner for the realizable setting (whose existence we assumed) is applied to each to generate a set of hypotheses, and we then use hypothesis selection to pick one with sufficient accuracy. The proof of Theorem 1.7 (showing that subtractively robust learnability implies robust learnability) follows almost the exact same recipe, except the realizable learner is replaced with a learner robust to subtractive contaminations. We recall some preliminaries in Section A. We then prove Theorem 1.5 in Section B, and we formalize and prove a version of Theorem 1.7 in Section 2.1.
We note that  $\alpha = 2$  and  $\alpha = 3$  are often the optimal factors to expect in distribution learning settings, even for the case of finite distribution classes. For example, for proper agnostic learning the factor  $\alpha = 3$  is known to be optimal for finite collections of distributions, which holds for classes with only 2 distributions [BKM19]. Similarly the factor of  $\alpha = 2$  is optimal if the notion of learning is relaxed to improper learners [BKM19, CDSS14b]. While we are not aware of lower bounds for the additive setting, a small constant factor such as 2 is within expectations for these problems.
For the proof of Theorem 1.5, we refer the reader to Section B in the appendix.
2.1 Subtractive Robust Learnability Implies Robust Learnability
Similarly, we can show that robustness with respect to a subtractive adversary implies robustness with respect to a general adversary. We note that this theorem requires a change in constants from  $\alpha$  to  $(2\alpha + 4)$ .
The proof follows a similar argument as the proof of Theorem 1.5 and can be found in Section C in the appendix.
3 Learnability Does Not Imply Robust (Agnostic) Learnability
In this section we show that there are classes of distributions which are realizably learnable, but not robustly learnable.
Theorem 3.1. There are classes of distributions  $\mathcal{Q}$ , such that  $\mathcal{Q}$  is realizably learnable, but for every  $\alpha \in \mathbb{R}$ ,  $\mathcal{Q}$  is not  $\alpha$ -robustly learnable. Moreover, the sample complexity of learning  $\mathcal{Q}$  can be arbitrarily close to (but larger than) linear. Namely, for any super-linear function  $g$ , there is a class  $\mathcal{Q}_g$ , with
-  $\mathcal{Q}_g$  is realizable learnable with sample complexity  $n_{\mathcal{Q}_g}^{re}(\varepsilon, \delta) \leq \log(1/\delta) g(1/\varepsilon)$ ;
for every  $\alpha \in \mathbb{R}$ ,  $\mathcal{Q}_g$  is not  $\alpha$ -robustly learnable.
Note that this statement appears slightly weaker than Theorem 1.6, in that it holds for  $\alpha$ -robust learnability rather than  $\eta$ -subtractive  $\alpha$ -robust learnability. In fact, the two statements are incomparable, due to the order of quantifiers in the construction. Here we provide a single class which is not  $\alpha$ -robustly learnable for every  $\alpha$ , whereas in the proof of Theorem 1.6 we give a different class for each  $\alpha$  (though the two constructions are similar). For simplicity we focus here on Theorem 3.1, whereas the proofs of Theorem 1.6 and other claims appear in Section D.
The key idea to the proof is to construct a class which is easy to learn in the realizable case, by having each distribution of the class have a unique support element that is not shared by any other distributions in the class. Distributions on which this "indicator element" has sufficient mass will be easily identified, independent of how rich the class is on other domain elements. That richness makes the class hard to learn from samples that miss those indicators. Furthermore, we construct the class in a way that its members are close in total variation distance to distributions that place no weight on those indicator elements.
This is done by making the mass on these indicator elements small, so that the members of a class of distributions that results from deleting these indicator bits are close to the initially constructed class,  $\mathcal{Q}_g$ . In order to make this work for every target accuracy and sample complexity, we need to have a union of such classes with decreasingly small mass on the indicator bits. In order for this to not interfere with the realizable learnability, we let the distributions with small mass on the indicator bits have most of their mass on one point (0, 0) that is the same for all distributions in the class. This ensures that distributions for which the indicator bit will likely not be observed because their mass is smaller than some  $\eta$  are still easily  $\varepsilon$ -approximated by a constant distribution  $(\delta_{(0,0)})$ . Lastly we ensure the impossibility of agnostic learnability, by controlling the rate at which  $\eta$  approaches zero to be faster than the rate at which  $\varepsilon$  approaches zero. With this intuition in mind, we will now describe the construction and proof of this theorem.
Proof. We first define the distributions in  $\mathcal{Q}_g$ . Let  $\{A_i \subset \mathbb{N} : i \in \mathbb{N}\}$  be an enumeration of all finite subsets of  $\mathbb{N}$ . Define distributions over  $\mathbb{N} \times \mathbb{N}$  as follows:
$$
q _ {i, j, k} = \left(1 - \frac {1}{j}\right) \delta_ {(0, 0)} + \left(\frac {1}{j} - \frac {1}{k}\right) U _ {A _ {i} \times \{2 j + 1 \}} + \frac {1}{k} \delta_ {(i, 2 j + 2)}, \tag {1}
$$
where, for every finite set  $W$ ,  $U_W$  denotes the uniform distribution over  $W$ . For a monotone, superlinear function  $g: \mathbb{N} \to \mathbb{N}$ , we now let  $\mathcal{Q}_g = \{q_{i,j,g(j)}: i,j \in \mathbb{N}\}$ . The first bullet point of the theorem (the class is learnable) follows from Claim 3.2 and the second bullet point (the class is not robustly learnable) follows from Claim 3.3.
Claim 3.2. For a monotone function  $g: \mathbb{N} \to \mathbb{N}$ , let  $\mathcal{Q}_g = \{q_{i,j,g(j)} : i, j \in \mathbb{N}\}$ . Then, the sample complexity of  $\mathcal{Q}_g$  in the realizable case is upper bounded by
$$
n _ {\mathcal {Q} _ {g}} ^ {r e} (\varepsilon , \delta) \leq \log (1 / \delta) g (1 / \varepsilon).
$$
This claim can be proved by showing that the following learner defined by
$$
\mathcal {A} (S) = \left\{ \begin{array}{l l} q _ {i, j, g (j)} & \text {i f} (i, 2 j + 2) \in S \\ \delta_ {(0, 0)} & \text {o t h e r w i s e} \end{array} \right.
$$
is a successful learner in the realizable case. Intuitively, this learner is successful for distributions  $q_{i,j,g(j)}$  for which  $j$  is large (i.e.,  $j > \frac{1}{\varepsilon}$ ), since this will mean that  $d_{\mathrm{TV}}(q_{i,j,g(j)}, \delta_{(0,0)})$  is small. Furthermore, it is successful for distributions  $q_{i,j,g(j)}$  for which  $j$  is small (i.e., upper bounded by some constant dependent on  $\varepsilon$ ), because this will lower bound the probability  $1 / g(j)$  of observing the indicator bit on  $(i, 2j + 2)$ . Once the indicator bit is observed the distribution will be uniquely identified.
Claim 3.3. For every function  $g \in \omega(n)$  the class  $\mathcal{Q}_g$  is not  $\alpha$ -robustly learnable for any  $\alpha > 0$ .
This claim can be proven by showing that for every  $\alpha$ , there is  $\eta$ , such that the class of distributions  $Q'$  such that for every  $q' \in Q'$  there is  $q \in \mathcal{Q}_g$  with  $d_{\mathrm{TV}}(q, q') < \eta$  which is not  $\alpha\eta$ -weakly learnable. In particular, those for every  $q' \in Q'$  there is  $q \in \mathcal{Q}_g$  and  $p$  such that  $q = (1 - \eta)q' + \eta q$ . We construct this class and show that it is not learnable by using the construction and Lemma 3 from [LBD23].
3.1 Existence of sample compression schemes
Sample compression schemes are combinatorial properties of classes that imply their learnability. For a variety of learning tasks, such as binary classification or expectation maximization a class has a sample compression scheme if and only if it is learnable [MY16, BHM+17]. For classification tasks, sample compression for realizable samples implies agnostic sample compression.  $\mathrm{[ABDH^{+}20]}$  used compression schemes to show learnability of classes of distributions in the realizable case, but left open the question if for learning probability distributions, the existence of realizable sample compression schemes implies the existence of similar schemes for the non-realizable (agnostic, or robust) settings. We provide a negative answer to this question.
More concretely, let  $\mathcal{Q}$  be a class of distributions over some domain  $X$ . A compression scheme for  $\mathcal{Q}$  involves two agents: an encoder and a decoder.
- The encoder knows a distribution  $q$  and receives a sample  $S$  generated by this distribution. The encoder picks a bounded size sub-sample and sends it, possibly with a few additional bits to the decoder.
- The decoder receives the message and uses an agreed upon decoding rule (that may depend on  $\mathcal{Q}$  but not on  $q$  or  $S$ ) to constructs a distribution  $p$  that is close to  $q$ .
Of course, there is some probability that the samples are not representative of the distribution  $q$ , in which case the compression scheme will fail. Thus, we only require that the decoding succeed with constant probability.
We say that a class  $\mathcal{Q}$  has a sample compression scheme (realizable or robust) if for every accuracy parameter  $\varepsilon > 0$ , the minimal required size of the sample  $S$ , and upper bounds on the size of the sub-sample and number of additional bits in the encoder's message depend only on  $\mathcal{Q}$  and  $\varepsilon$  (and are independent of the sample generating  $q$  and on the sample  $S$ ).
A realizable compression scheme is required to handle only  $q$ 's in  $\mathcal{Q}$  and output  $p$  such that  $d_{\mathrm{TV}}(p,q) \leq \varepsilon$ , while a robust compression scheme should handle any  $q$  but the decoder's output  $p$  is only required to be  $\min_{q \in \mathcal{Q}} \{d_{\mathrm{TV}}(p,q)\} + \varepsilon$  close to  $q$ .
Theorem 3.4 (Formal version of Theorem 1.8). For every  $\alpha \in \mathbb{R}$ , the existence of a realizable compression scheme, does not imply the existence of an  $\alpha$ -robust compression scheme. That is, there is a class  $\mathcal{Q}$  that has a realizable compression scheme, but for every  $\alpha \in \mathbb{R}$ ,  $\mathcal{Q}$  does not have an  $\alpha$ -robust compression scheme.
Proof. Consider the class  $\mathcal{Q} = \mathcal{Q}_g$  from Section 3. We note that this class has a compression scheme of size 1. However, from  $[\mathrm{ABDH}^+18]$ , we know that having a  $\alpha$ -robust compression scheme implies  $\alpha$ -agnostic learnability. We showed in Theorem 1.6 that for every  $\alpha$  and for every superlinear function  $g$ , the class  $\mathcal{Q}_g$  is not  $\alpha$ -agnostically learnable. It follows that  $\mathcal{Q}_g$  does not have an  $\alpha$ -robust compression scheme.
<sup>7</sup>We provide a definition for  $\varepsilon$ -weak learnability as Definition D.2. We note that the definition we provide is what would usually be referred to as  $(1/2 - \varepsilon)$ -weak learnability in the supervised learning literature. For simplicity, because  $\varepsilon$  is our parameter of interest, we reparameterized the definition to be more intuitive.
In Section E, we present a precise quantitative definition of sample compression schemes, as well as the proof that the class  $\mathcal{Q}_g$  has a sample compression scheme of size 1.
4 Implications of Private Learnability
Qualitatively speaking, differentially private algorithms offer a form of "robustness" – the output distribution of a differentially private algorithm is insensitive to the change of a single point in its input sample. The relationship between privacy and notions of "robustness" has been studied under various settings, where it has been shown that robust algorithms can be made private and vice versa [DL09, GH22, HKMN23].
For distribution learning, we find that: (1) the requirement of approximate differentially private learnability also does not imply (general) robust learnability; and (2) the stronger requirement of pure differentially private learnability does imply robust learnability.
Definition 4.1 (Differential Privacy [DMNS06]). Let  $X$  be an input domain and  $Y$  to be an output domain. A randomized algorithm  $A: X^m \to Y$  is  $(\varepsilon, \delta)$ -differentially private (DP) if for every  $x, x' \in X^n$  that differ in one entry,
$$
\mathbb {P} [ A (x) \in B ] \leq e ^ {\varepsilon} \cdot \mathbb {P} [ A \left(x ^ {\prime}\right) \in B ] + \delta \quad f o r a l l B \subseteq Y.
$$
If  $A$  is  $(\varepsilon, \delta)$ -DP for  $\delta > 0$ , we say it satisfies approximate DP. If it satisfies  $(\varepsilon, 0)$ -DP, we say it satisfies pure DP.
Definition 4.2 (DP learnable class). We say that a class  $\mathcal{C}$  of probability distributions is (approximate) DP learnable if there exists a randomized learner  $A$  and a function  $n_{\mathcal{C}}:(0,1)^4\to \mathbb{N}$ , such that for every probability distribution  $p\in \mathcal{C}$ , and every  $(\alpha ,\beta ,\varepsilon ,\delta)\in (0,1)^4$ , for  $n\geq n_{\mathcal{C}}(\alpha ,\beta ,\varepsilon ,\delta)$
1.  $A$  is  $(\varepsilon, \delta)$ -DP; and
2. The probability over samples  $S$  of size  $n$  drawn i.i.d. from the distribution  $p$ , as well as over the randomness of  $A$  that
$$
d _ {\mathrm {T V}} (p, A (S)) \leq \alpha
$$
is at least  $1 - \beta$
We say  $\mathcal{C}$  is pure DP learnable if a learner  $A$  can be found that satisfies  $(\varepsilon, 0)$ -DP, in which case the sample complexity function  $n_{\mathcal{C}}: (0,1)^3 \to \mathbb{N}$  does not take  $\delta$  as a parameter.
Theorem 4.3 (Approximate DP learnability vs. robust learnability).
1. If a class  $\mathcal{Q}$  is approximate DP learnable, then  $\mathcal{Q}$  is  $\eta$ -additive 2-robustly learnable for any  $\eta \in (0,1/4)$ .
2. There exists an approximate DP learnable class  $\mathcal{Q}$  that is not  $\alpha$ -robustly learnable for any  $\alpha \geq 1$ .
Note that the first claim is immediate from Theorem 1.5, since approximate DP learnability implies learnability. To prove the second claim, we show that the learner for the class  $\mathcal{Q}$  described in Theorem 3.1 can be made differentially private by employing stability-based histograms [BNS16]. The proof appears in Section F.
Theorem 4.4 (Pure DP learnable vs. robustly learnable). If a class  $\mathcal{Q}$  is pure DP learnable, then  $\mathcal{Q}$  is 3-robustly learnable.
The proof relies on the finite cover characterization of pure DP learnability.
Proposition 4.5 (Packing lower bound, Lemma 5.1 from [BKSW19]). Let  $\mathcal{C}$  be a class of distributions, and let  $\alpha, \varepsilon > 0$ . Suppose  $\mathcal{P}_{\alpha}$  is a  $\alpha$ -packing of  $\mathcal{C}$ , that is,  $\mathcal{P}_{\alpha} \subseteq \mathcal{C}$  such that for any  $p \neq q \in \mathcal{P}_{\alpha}$ ,  $d_{\mathrm{TV}}(p, q) > \alpha$ .
Any  $\varepsilon$ -DP algorithm  $A$  that takes  $n$  i.i.d. samples  $S$  from any  $p \in \mathcal{C}$  and has  $d_{\mathrm{TV}}(p, A(S)) \leq \alpha / 2$  with probability  $\geq 9 / 10$  requires
$$
n \geq \frac {\log | \mathcal {P} _ {\alpha} | - \log \frac {1 0}{9}}{\varepsilon}.
$$
Proof of Theorem 4.4. Let  $\alpha, \beta > 0$ . Pure DP learnability of  $\mathcal{Q}$  implies that there exists a 1-DP algorithm  $A_{DP}$  and  $n = n_{\mathcal{C}}(\alpha / 12, 1 / 10, 1)$  such that for any  $p \in \mathcal{Q}$ , with probability  $\geq 9 / 10$  over the sampling of  $n$  i.i.d. samples  $S$  from  $p$ , as well as over the randomness of the algorithm  $A_{DP}$ , we have  $d_{\mathrm{TV}}(p, A_{DP}(S)) \leq \alpha / 12$ . By Proposition 4.5, any  $\alpha / 6$ -packing  $\mathcal{P}_{\alpha / 6}$  of  $\mathcal{Q}$  has
$$
\left| \mathcal {P} _ {\alpha / 6} \right| \leq \exp (m) \cdot (1 0 / 9).
$$
Let  $\widehat{\mathcal{Q}}$  be such a maximal  $\alpha /6$ -packing. By maximality,  $\widehat{\mathcal{Q}}$  is also an  $\alpha /6$ -cover of  $\mathcal{Q}$ . Hence, running Yatracos' 3-robust finite class learner (Theorem A.1)  $A$  over  $\widehat{\mathcal{Q}}$  with
$$
n _ {\widehat {\mathcal {Q}}} (\alpha / 2, \beta) = O \left(\frac {\log | \widehat {\mathcal {Q}} | + \log (1 / \beta)}{(\alpha / 2) ^ {2}}\right)
$$
samples drawn i.i.d. from  $p$  yields, with probability  $\geq 1 - \beta$
$$
\begin{array}{l} d _ {\mathrm {T V}} (p, A (S)) \leq 3 \min \{d _ {\mathrm {T V}} (p, p ^ {\prime}): p ^ {\prime} \in \widehat {\mathcal {Q}} \} + \alpha / 2 \\ \leq 3 \left(\min  \left\{d _ {\mathrm {T V}} \left(p, p ^ {\prime}\right): p ^ {\prime} \in \mathcal {Q} \right\} + \alpha / 6\right) + \alpha / 2 \\ = 3 \min  \left\{d _ {\mathrm {T V}} \left(p, p ^ {\prime}\right): p ^ {\prime} \in \mathcal {Q} \right\} + \alpha . \\ \end{array}
$$
Note that Yatracos' algorithm for hypothesis selection can be replaced with a pure DP algorithm for hypothesis selection (Theorem 27 of [AAK21]) in order to achieve the following stronger implication.
Theorem 4.6 (Pure DP learnable vs. robustly learnable). If a class  $\mathcal{Q}$  is pure DP learnable, then  $\mathcal{Q}$  is pure DP 3-robustly learnable.
We examine the connection between learnability and robust learnability for general classes of probability distributions. Our main findings are somewhat surprising in that, in contrast to most known leaning scenarios, learnability does not imply robust learnability. We also show that learnability does imply additively robust learnability. We use our proof techniques to draw new insights related to compression schemes and differentially private distribution learning.
Thanks to Argyris Mouzakis for helpful conversations in the early stages of this work. AB was supported by an NSERC Discovery Grant and a David R. Cheriton Graduate Scholarship. GK was supported by a Canada CIFAR AI Chair, an NSERC Discovery Grant, and an unrestricted gift from Google. TL was supported by a Vector Research Grant and a Waterloo Apple PhD Fellowship in Data Science and Machine Learning.
[AAK21] Ishaq Aden-Ali, Hassan Ashtiani, and Gautam Kamath. On the sample complexity of privately learning unbounded high-dimensional gaussians. In Proceedings of the 32nd International Conference on Algorithmic Learning Theory, ALT '21, pages 185-216. JMLR, Inc., 2021.
[AAL21] Ishaq Aden-Ali, Hassan Ashtiani, and Christopher Liaw. Privately learning mixtures of axis-aligned gaussians. In Advances in Neural Information Processing Systems 34, NeurIPS '21. Curran Associates, Inc., 2021.
$\left[\mathrm{ABDH}^{+}18\right]$  Hassan Ashtiani, Shai Ben-David, Nicholas Harvey, Christopher Liaw, Abbas Mehrabian, and Yaniv Plan. Nearly tight sample complexity bounds for learning mixtures of Gaussians via sample compression schemes. In Advances in Neural Information Processing Systems 31, NeurIPS '18, pages 3412-3421. Curran Associates, Inc., 2018.
$\left[\mathrm{ABDH}^{+}20\right]$  Hassan Ashtiani, Shai Ben-David, Nicholas JA Harvey, Christopher Liaw, Abbas Mehrabian, and Yaniv Plan. Near-optimal sample complexity bounds for robust learning of gaussian mixtures via compression schemes. Journal of the ACM, 67(6):32:1-32:42, 2020.
[VC74] Vladimir Vapnik and Alexey Chervonenkis. Theory of Pattern Recognition. Nauka, 1974.
[Yat85] Yannis G. Yatracos. Rates of convergence of minimum distance estimators and Kolmogorov's entropy. The Annals of Statistics, 13(2):768-774, 1985.
A Additional Preliminaries
We recall a classic theorem for the problem of hypothesis selection. Given a set of candidate hypothesis distributions, the algorithm selects one which is close to an unknown distribution (to which we have sample access). The requisite number of samples from the unknown distribution is logarithmic in the size of the set of candidates.
Theorem A.1 (Yatracos' 3-robust learner for finite classes (Theorem 4.4 of  $\mathrm{[ABDH^{+}20]}$ , based on Theorem 1 of [Yat85])) Let  $\mathcal{C}$  be a finite class of distributions over a domain  $\mathcal{X}$ . There exists an algorithm  $A$  such that for any  $(\alpha, \beta) \in (0,1)^2$  and for any distribution  $p$  over  $\mathcal{X}$ , given a sample  $S$  of size
$$
m = O \left(\frac {\log | \mathcal {C} | + \log (1 / \delta)}{\varepsilon^ {2}}\right)
$$
drawn i.i.d. from  $p$ , we have that with probability  $\geq 1 - \delta$
$$
d _ {\mathrm {T V}} (p, A (S)) \leq 3 \cdot \min  \left\{d _ {\mathrm {T V}} \left(p, p ^ {\prime}\right): p ^ {\prime} \in \mathcal {C} \right\} + \varepsilon .
$$
We also use the following form of the Chernoff bound.
Proposition A.2 (Chernoff bounds). Let  $X = \sum_{i=1}^{m} X_i$  where  $X_1 \ldots X_m$  are independent draws from Bernoulli(p). For any  $t \in (0,1)$ ,  $\mathbb{P}[X / m \leq (1 - t)p] \leq \exp(-t^2 mp / 2)$ .
B Learnability Implies Additive Robust Learnability
In this section, we provide the proof for Theorem 1.5.
Proof. Recall that  $\mathcal{Q}$  is a class of probability distributions which is realizably learnable. We let  $\mathcal{A}_{\mathcal{Q}}^{re}$  be a realizable learner for  $\mathcal{Q}$ , with sample complexity  $n_{\mathcal{Q}}^{re}$ . Let accuracy parameters  $\varepsilon, \delta > 0$  be arbitrary.
We will define  $n_1 \geq \max \left\{ 2n_{\mathcal{Q}}^{re}(\frac{\varepsilon}{9},\frac{\delta}{5}),\frac{162(1 + \log(\frac{5}{\delta}))}{\varepsilon^2} \right\}$  and  $n_2 \geq \frac{162\left(2(\eta + \frac{2\varepsilon}{9})n_1\log\left(n_1\right) + \log\left(\frac{5}{\delta}\right)\right)}{\varepsilon^2} \geq \frac{162(1 + \log(\frac{5}{\delta}))}{\varepsilon^2}$ , and  $n = n_1 + n_2$  be their sum.
Our additive robust learner will receive a sample  $S \sim (\eta q + (1 - \eta)p)^n$  of size  $n$ . We can view a subset  $S' \subset S$  as the "clean" part, being i.i.d. generated by  $p$ . The size of this clean part  $|S'| = n'$  is distributed according to a binomial distribution  $\mathrm{Bin}(n, 1 - \eta)$ . By a Chernoff bound (Proposition A.2), we get
$$
\begin{array}{l} \mathbb {P} \left[ n ^ {\prime} \leq \left(1 - \frac {\varepsilon}{9} - \eta\right) n \right] \leq \mathbb {P} \left[ n ^ {\prime} \leq \left(1 - \frac {\varepsilon}{9} - \eta + \frac {\eta \varepsilon}{9}\right) n \right] = \mathbb {P} \left[ n ^ {\prime} \leq \left(1 - \frac {\varepsilon}{9}\right) (1 - \eta) n \right] \\ \leq \exp \left(- \left(\frac {\varepsilon}{9}\right) ^ {2} n (1 - \eta) / 2\right) = \exp \left(- \frac {\varepsilon^ {2}}{1 6 2} n (1 - \eta)\right) \leq \exp \left(- \frac {\varepsilon^ {2}}{1 6 2} n \left(1 - \frac {3}{4}\right)\right) \\ \end{array}
$$