SlowGuess/NeurIPS_2023_OCR huggingface dataset:
where the second inequality is by event  $\mathcal{E}_1$ . Similarly, to bound  $J_{3}$ , we have
$$
J _ {3} = \sqrt {\mathbb {P} (X \in A _ {L})} \left| \bar {y} _ {\mathcal {I} _ {A _ {L}}} - \mathbb {E} \left(f ^ {*} (X) \mid X \in A _ {L}\right) \right| \leq 2 0 U \sqrt {\bar {t} (\delta / 3 6)} \tag {34}
$$
To bound  $J_{2}$ , note that
$$
\begin{array}{l} J _ {2} \leq \left| \sqrt {\mathbb {P} \left(X \in A _ {L}\right)} - \sqrt {\left| \mathcal {I} _ {A _ {L}} \right| / n} \right| \cdot \left| \bar {y} _ {\mathcal {I} _ {A}} - \bar {y} _ {\mathcal {I} _ {A _ {L}}} \right| + \sqrt {\left| \mathcal {I} _ {A _ {L}} \right| / n} \cdot \left| \bar {y} _ {\mathcal {I} _ {A}} - \bar {y} _ {\mathcal {I} _ {A _ {L}}} \right| \tag {35} \\ \leq 5 \sqrt {\bar {t} (\delta / 1 2)} \cdot 2 U + \sqrt {| \mathcal {I} _ {A _ {L}} | / n} \cdot | \bar {y} _ {\mathcal {I} _ {A}} - \bar {y} _ {\mathcal {I} _ {A _ {L}}} | \\ \end{array}
$$
where the second inequality made use of the event  $\mathcal{E}_2$ . Combining (32) - (35), we have
$$
\begin{array}{l} \sqrt {\Delta_ {L} (A , j , b)} \leq 4 0 U \sqrt {\bar {t} (\delta / 3 6)} + 1 0 U \sqrt {\bar {t} (\delta / 1 2)} + \sqrt {| \mathcal {I} _ {A _ {L}} | / n} \cdot | \bar {y} _ {\mathcal {I} _ {A}} - \bar {y} _ {\mathcal {I} _ {A _ {L}}} | \\ \leq 5 0 U \sqrt {\bar {t} (\delta / 3 6)} + \sqrt {\left| \mathcal {I} _ {A _ {L}} \right| / n} \cdot \left| \bar {y} _ {\mathcal {I} _ {A}} - \bar {y} _ {\mathcal {I} _ {A _ {L}}} \right| \\ \end{array}
$$
which implies (by Young's inequality)
$$
\Delta_ {L} (A, j, a) \leq (1 + 1 / \alpha) \cdot 2 5 0 0 U ^ {2} \bar {t} (\delta / 3 6) + (1 + \alpha) \frac {\left| \mathcal {I} _ {A _ {L}} \right|}{n} \left| \bar {y} _ {\mathcal {I} _ {A}} - \bar {y} _ {\mathcal {I} _ {A _ {L}}} \right| ^ {2} \tag {36}
$$
$$
\Delta_ {R} (A, j, a) \leq (1 + 1 / \alpha) \cdot 2 5 0 0 U ^ {2} \bar {t} (\delta / 3 6) + (1 + \alpha) \frac {\left| \mathcal {I} _ {A _ {R}} \right|}{n} \left| \bar {y} _ {\mathcal {I} _ {A}} - \bar {y} _ {\mathcal {I} _ {A _ {R}}} \right| ^ {2} \tag {37}
$$
Summing up (36) and (37), and by (31), we have
$$
\Delta (A, j, a) \leq (1 + 1 / \alpha) \cdot 5 0 0 0 U ^ {2} \bar {t} (\delta / 3 6) + (1 + \alpha) \widehat {\Delta} (A, j, a)
$$
This completes the proof of (29). The proof of (30) is by a similar argument.
Lemma A.4 provides upper bounds between  $\Delta(A, j, b)$  and  $\widehat{\Delta}(A, j, b)$ , which serves as a link to translate the population impurity decrease to sample impurity decrease. With all these technical lemmas at hand, we are ready to present the proof Theorem 2.3, as shown in the next subsection.
A.3 Completing the proof of Theorem 2.3
Define events
$$
\mathcal {E} _ {1} := \left\{\sup  _ {A \in \mathcal {A} _ {p, d}} \sqrt {\mathbb {P} (X \in A)} \left| \mathbb {E} (f ^ {*} (X) | X \in A) - \bar {y} _ {\mathcal {I} _ {A}} \right| \leq 2 0 U \sqrt {\bar {t} (\delta / 2 4)} \right\}
$$
$$
\mathcal {E} _ {2} := \left\{\Delta (A, j, a) \leq (1 + \alpha) \widehat {\Delta} (A, j, a) + (1 + 1 / \alpha) \cdot 5 0 0 0 U ^ {2} \bar {t} (\delta / 7 2) \quad \forall A \in \mathcal {A} _ {p, d - 1},   j \in [ p ],   a \in \mathbb {R} \right\}
$$
$$
\mathcal {E} _ {3} := \left\{\widehat {\Delta} (A, j, a) \leq (1 + \alpha) \Delta (A, j, a) + (1 + 1 / \alpha) \cdot 5 0 0 0 U ^ {2} \bar {t} (\delta / 7 2) \quad \forall A \in \mathcal {A} _ {p, d - 1},   j \in [ p ],   a \in \mathbb {R} \right\}
$$
Then by Lemmas A.1 and A.4, and note that from the statement of Theorem 2.3,  $\bar{t}_2(\delta /72) < 3 / 4$ , so we have  $\mathbb{P}(\mathcal{E}_1)\geq 1 - \delta /2$  and  $\mathbb{P}(\mathcal{E}_2\cup \mathcal{E}_3)\geq 1 - \delta /2$ , which implies  $\mathbb{P}(\cup_{i = 1}^{3}\mathcal{E}_{i})\geq 1 - \delta$ . In the following, we prove (10) using a deterministic argument conditioned on  $\cup_{i = 1}^{3}\mathcal{E}_{i}$ .
For any  $k \in [d]$  and any leave node  $t$  of  $\widehat{f}^{(k)}$  (recall that  $\widehat{f}^{(k)}$  is the decision tree by CART with depth  $k$ ), let  $A_{t}^{(k)}$  be the corresponding cube, that is, for any  $x \in \mathbb{R}^p$ ,  $x \in A_{t}^{(k)}$  if and only if  $x$  is routed to  $t$  in  $\widehat{f}^{(k)}$ . Let  $\mathcal{L}^{(k)}$  be the set of all leave nodes of  $\widehat{f}^{(k)}$ . Then we have
$$
\hat {f} ^ {(k)} (x) = \sum_ {t \in \mathcal {L} ^ {(k)}} \bar {y} _ {\mathcal {I} _ {A _ {t} ^ {(k)}}} 1 _ {\{x \in A _ {t} ^ {(k)} \}} \tag {38}
$$
Define a function
$$
\widetilde {f} ^ {(k)} (x) := \sum_ {t \in \mathcal {L} ^ {(k)}} \mathbb {E} \left(f ^ {*} (X) \mid X \in A _ {t} ^ {(k)}, \mathcal {X} _ {1} ^ {n}\right) \cdot 1 _ {\{x \in A _ {t} ^ {(k)} \}} \tag {39}
$$
where  $\mathcal{X}_1^n$  is the set of iid random variables  $\{x_1, \ldots, x_n\}$ , and  $X$  is a random variable having the same distribution as  $x_1$  but independent of  $\mathcal{X}_1^n$ . In other words,  $\widehat{f}^{(k)}$  is a tree with the same splitting structure as  $\widehat{f}^{(k)}$  and replaces the prediction value of each leave node as the populational conditional mean of  $f^*(\cdot)$ .
First, using Cauchy-Schwarz inequality, we have
$$
\left\| \widehat {f} ^ {(k)} - f ^ {*} \right\| _ {L ^ {2} (X)} ^ {2} \leq 2 \| f ^ {*} - \widetilde {f} ^ {(k)} \| _ {L ^ {2} (X)} ^ {2} + 2 \| \widetilde {f} ^ {(k)} - \widehat {f} ^ {(k)} \| _ {L ^ {2} (X)} ^ {2} := 2 J _ {1} (k) + 2 J _ {2} (k) \tag {40}
$$
To bound  $J_{1}(d)$ , we derive recursive inequalities between  $J_{1}(k)$  and  $J_{1}(k + 1)$  for all  $0 \leq k \leq d - 1$ . Note that
$$
\begin{array}{l} J _ {1} (k) = \mathbb {E} \left(\left(f ^ {*} (X) - \widetilde {f} ^ {(k)} (X)\right) ^ {2} \mid \mathcal {X} _ {1} ^ {n}\right) \\ = \sum_ {t \in \mathcal {L} ^ {(k)}} \mathbb {P} \left(X \in A _ {t} \mid \mathcal {X} _ {1} ^ {n}\right) \cdot \operatorname {V a r} \left(f ^ {*} (X) \mid X \in A _ {t}, \mathcal {X} _ {1} ^ {n}\right) \tag {41} \\ \end{array}
$$
For each  $t \in \mathcal{L}^{(k)}$ , let  $t_L$  and  $t_R$  be the two children of  $t$ , then we have
$$
\begin{array}{l} \mathbb {P} \left(X \in A _ {t} \mid \mathcal {X} _ {1} ^ {n}\right) \cdot \operatorname {V a r} \left(f ^ {*} (X) \mid X \in A _ {t}, \mathcal {X} _ {1} ^ {n}\right) \\ = \mathbb {P} \left(X \in A _ {t _ {L}} \mid \mathcal {X} _ {1} ^ {n}\right) \cdot \operatorname {V a r} \left(f ^ {*} (X) \mid X \in A _ {t _ {L}}, \mathcal {X} _ {1} ^ {n}\right) \tag {42} \\ + \mathbb {P} (X \in A _ {t _ {R}} | \mathcal {X} _ {1} ^ {n}) \cdot \operatorname {V a r} \left(f ^ {*} (X) \mid X \in A _ {t _ {R}}, \mathcal {X} _ {1} ^ {n}\right) + \Delta \left(A _ {t}, \hat {j} _ {t}, \hat {b} _ {t}\right) \\ \end{array}
$$
$$
(\hat{j}_{t},\hat{b}_{t})\in \operatorname *{argmax}_{j\in [p],b\in \mathbb{R}}\widehat{\Delta} (A_{t},j,b)
$$
$$
(j_{t},b_{t})\in \operatorname *{argmax}_{j\in [p],b\in \mathbb{R}}\Delta (A_{t},j,b)
$$
$$
\begin{array}{l} \Delta \left(A _ {t}, \hat {j} _ {t}, \hat {b} _ {t}\right) \geq \frac {1}{1 + \alpha} \widehat {\Delta} \left(A _ {t}, \hat {j} _ {t}, \hat {b} _ {t}\right) - (5 0 0 0 / \alpha) U ^ {2} \bar {t} (\delta / 7 2) \\ \geq \frac {1}{1 + \alpha} \widehat {\Delta} \left(A _ {t}, j _ {t}, b _ {t}\right) - (5 0 0 0 / \alpha) U ^ {2} \bar {t} (\delta / 7 2) \tag {43} \\ \geq \frac {1}{(1 + \alpha) ^ {2}} \Delta \left(A _ {t}, j _ {t}, b _ {t}\right) - \frac {2 + \alpha}{\alpha (1 + \alpha)} 5 0 0 0 U ^ {2} \bar {t} (\delta / 7 2) \\ \end{array}
$$
where the first inequality is by event  $\mathcal{E}_3$ , the second inequality is by the definition of  $(\hat{j}_t,\hat{b}_t)$ , and the third inequality is because of event  $\mathcal{E}_2$ . By Assumption 2.2, we have
$$
\Delta \left(A _ {t}, j _ {t}, b _ {t}\right) = \sup  _ {j \in [ p ], b \in \mathbb {R}} \Delta \left(A _ {t}, j, b\right) \geq \lambda \cdot \mathbb {P} \left(X \in A _ {t} \mid \mathcal {X} _ {1} ^ {n}\right) \operatorname {V a r} \left(f ^ {*} (X) \mid X \in A _ {t}, \mathcal {X} _ {1} ^ {n}\right) \tag {44}
$$
Combining (42), (43) and (44), we have
$$
\begin{array}{l} \mathbb {P} \left(X \in A _ {t _ {L}} \mid \mathcal {X} _ {1} ^ {n}\right) \cdot \operatorname {V a r} \left(f ^ {*} (X) \mid X \in A _ {t _ {L}}, \mathcal {X} _ {1} ^ {n}\right) + \mathbb {P} \left(X \in A _ {t _ {R}} \mid \mathcal {X} _ {1} ^ {n}\right) \cdot \operatorname {V a r} \left(f ^ {*} (X) \mid X \in A _ {t _ {R}}, \mathcal {X} _ {1} ^ {n}\right) \\ \leq \left(1 - \frac {\lambda}{(1 + \alpha) ^ {2}}\right) \mathbb {P} (X \in A _ {t} | \mathcal {X} _ {1} ^ {n}) \cdot \operatorname {V a r} \left(f ^ {*} (X) | X \in A _ {t}, \mathcal {X} _ {1} ^ {n}\right) + \frac {2 + \alpha}{\alpha (1 + \alpha)} 5 0 0 0 U ^ {2} \bar {t} (\delta / 7 2) \\ \end{array}
$$
Summing up the inequality above for all  $t \in \mathcal{L}^{(k)}$ , we have
$$
J _ {1} (k + 1) \leq \left(1 - \frac {\lambda}{(1 + \alpha) ^ {2}}\right) J _ {1} (k) + 2 ^ {k} \cdot \frac {2 + \alpha}{\alpha (1 + \alpha)} 5 0 0 0 U ^ {2} \bar {t} (\delta / 7 2)
$$
Using the inequality above recursively for  $k = 0,1,\dots,d - 1$ , we have
$$
\begin{array}{l} J _ {1} (d) \leq \left(1 - \frac {\lambda}{(1 + \alpha) ^ {2}}\right) ^ {d} J _ {1} (0) + \frac {2 + \alpha}{\alpha (1 + \alpha)} 5 0 0 0 U ^ {2} \bar {t} (\delta / 7 2) \sum_ {k = 1} ^ {d} 2 ^ {k - 1} \tag {45} \\ \leq \left(1 - \frac {\lambda}{(1 + \alpha) ^ {2}}\right) ^ {d} \operatorname {V a r} \left(f ^ {*} (X)\right) + 2 ^ {d} \cdot \frac {2 + \alpha}{\alpha (1 + \alpha)} 5 0 0 0 U ^ {2} \bar {t} (\delta / 7 2) \\ \end{array}
$$
To bound  $J_{2}(d)$ , we have
$$
\begin{array}{l} J _ {2} (d) = \sum_ {t \in \mathcal {L} ^ {(d)}} \mathbb {P} (X \in A _ {t}) \left(\mathbb {E} \left(f ^ {*} (X) \mid X \in A _ {t}, \mathcal {X} _ {1} ^ {n}\right) - \bar {y} _ {\mathcal {I} _ {A _ {t}}}\right) ^ {2} \tag {46} \\ \leq 2 ^ {d} \cdot 4 0 0 U ^ {2} \bar {t} (\delta / 2 4) \\ \end{array}
$$
where the inequality made use of event  $\mathcal{E}_1$ .
Using (45) and (46), and recalling (40), we have
$$
\begin{array}{l} \| \hat {f} ^ {(k)} - f ^ {*} \| _ {L ^ {2} (X)} ^ {2} \leq 2 \left(1 - \frac {\lambda}{(1 + \alpha) ^ {2}}\right) ^ {d} \operatorname {V a r} \left(f ^ {*} (X)\right) + 2 ^ {d + 1} \cdot \frac {2 + \alpha}{\alpha (1 + \alpha)} 5 0 0 0 U ^ {2} \bar {t} (\delta / 7 2) \\ + 2 ^ {d + 1} \cdot 4 0 0 U ^ {2} \bar {t} (\delta / 2 4) \\ \lesssim \operatorname {V a r} \left(f ^ {*} (X)\right) \cdot \left(1 - \lambda / (1 + \alpha) ^ {2}\right) ^ {d} + \frac {2 + \alpha}{\alpha (1 + \alpha)} \frac {2 ^ {d} \left(d \log (n p) + \log (1 / \delta)\right)}{n} U ^ {2} \tag {47} \\ \lesssim \operatorname {V a r} (f ^ {*} (X)) \cdot (1 - \lambda / (1 + \alpha) ^ {2}) ^ {d} + \frac {2 ^ {d} (d \log (n p) + \log (1 / \delta))}{\alpha n} U ^ {2} \\ \end{array}
$$
This completes the proof of (10). To prove (11), by taking  $\alpha = 1 / d$  and  $d = \lceil \log_2(n) / (1 - \log_2(1 - \lambda))\rceil$ , we have
$$
\begin{array}{l} \left(1 - \frac {\lambda}{(1 + \alpha) ^ {2}}\right) ^ {d} = (1 - \lambda) ^ {d} \left(1 + \frac {\lambda}{1 - \lambda} (1 - (1 + \alpha) ^ {- 2})\right) ^ {d} \tag {48} \\ = (1 - \lambda) ^ {d} \left(1 + \frac {\lambda}{1 - \lambda} \frac {2 / d + 1 / d ^ {2}}{(1 + 1 / d) ^ {2}}\right) ^ {d} \lesssim_ {\lambda} (1 - \lambda) ^ {d} \\ \end{array}
$$
Note that for  $s = \log_2(n) / (1 - \log_2(1 - \lambda))$  we have  $(1 - \lambda)^s = 2^s / n$ , hence by taking  $d = \lceil \log_2(n) / (1 - \log_2(1 - \lambda)) \rceil$ , we have
$$
(1 - \lambda) ^ {d} \leq \frac {2 ^ {d}}{n} \leq 2 n ^ {- 1 + \frac {1}{1 - \log_ {2} (1 - \lambda)}} = 2 n ^ {- \phi (\lambda)}. \tag {49}
$$
Combining (47), (48) and (49) and note that  $\operatorname{Var}\big(f^{*}(X)\big) \leq M < U$ , we have
$$
\begin{array}{l} \left\| \widehat {f} ^ {(k)} - f ^ {*} \right\| _ {L ^ {2} (X)} ^ {2} \lesssim_ {\lambda , U} n ^ {- \phi (\lambda)} \left(d ^ {2} \log (n p) + d \log (1 / \delta)\right) \\ \lesssim_ {\lambda , U} n ^ {- \phi (\lambda)} (\log^ {2} (n) \log (n p) + \log (n) \log (1 / \delta)) \\ \end{array}
$$
this completes the proof of (11).
A.4 Proof of Lemma A.1
The main idea of proving Lemma A.1 is to find a proper finite net of the set  $\mathcal{A}_{p,d}$ , control the gap on this net, and finally prove the result for all  $A \in \mathcal{A}_{p,d}$  based on the approximation gap of the net. We need a few auxiliary results. Let  $S := \{0, 1/n, 2/n, \ldots, (n-1)/n, 1\}$ , and define
$$
\widetilde {\mathcal {A}} _ {p, d} := \left\{\prod_ {j = 1} ^ {p} \left[ \ell_ {j}, u _ {j} \right] \in \mathcal {A} _ {p, d} \middle | \ell_ {j}, u _ {j} \in \mathcal {S} \text {f o r a l l} j \in [ p ] \right\}
$$
For any  $A = \prod_{j=1}^{p} [\ell_j, u_j] \in \mathcal{A}_{p,d}$ , define
$$
A ^ {\prime} = \prod_ {j = 1} ^ {p} \left[ \ell_ {j} ^ {\prime}, u _ {j} ^ {\prime} \right]
$$
where  $\ell_j^{\prime} := \max \{s \in S \mid s \leq \ell_j\}$ , and  $u_j^{\prime} := \min \{s \in S \mid s \geq u_j\}$ . Roughly speaking,  $A^{\prime}$  is the smallest box with all edges in  $\mathcal{S}$  that contains  $A$ . For any  $\widetilde{A} = \prod_{j=1}^{p} [\tilde{\ell}_j, \tilde{u}_j] \in \widetilde{\mathcal{A}}_{p,d}$  with  $\tilde{u}_j - \tilde{\ell}_j \geq 2/n$  for all  $j \in [p]$ , define
$$
B (\widetilde {A}) := \widetilde {A} \setminus \prod_ {j = 1} ^ {p} \left[ \tilde {\ell} _ {j} + (1 / n) \cdot 1 _ {\{\tilde {\ell} _ {j} \neq 0 \}}, \tilde {u} _ {j} - (1 / n) \cdot 1 _ {\{\tilde {u} _ {j} \neq 1 \}} \right].
$$
and define  $\mathcal{B}_{p,d}$  to be the set of all such sets, that is
$$
\mathcal {B} _ {p, d} := \left\{B (\widetilde {A}) \middle | \widetilde {A} = \prod_ {j = 1} ^ {p} [ \tilde {\ell} _ {j}, \tilde {u} _ {j} ] \in \widetilde {\mathcal {A}} _ {p, d} \text {w i t h} \tilde {u} _ {j} - \tilde {\ell} _ {j} \geq 2 / n \right\}
$$
The following lemma can be easily verified from the definitions of  $\widetilde{\mathcal{A}}_{p,d}$  and  $\mathcal{B}_{p,d}$ .
Lemma A.5 (1) For any  $A \in \mathcal{A}_{p,d}$ , there exists  $B \in \mathcal{B}_{p,d}$  such that  $A' \setminus A \subseteq B$ .
(2)  $\mathbb{P}(X\in B)\leq 2\bar{\theta} d / n$  for all  $B\in \mathcal{B}_{p,d}$
(3) The cardinality
$$
\left| \mathcal {B} _ {p, d} \right| \leq \left| \tilde {\mathcal {A}} _ {p, d} \right| \leq \binom {p} {d} (n + 1) ^ {2 d} \leq p ^ {d} (n + 1) ^ {2 d}
$$
Finally, for any  $t \geq 0$ , we define
$$
\mathcal {A} _ {p, d} (t) := \left\{A \in \mathcal {A} _ {p, d} \mid \mathbb {P} (X \in A) \leq t \right\}, \quad \text {a n d} \quad \widetilde {\mathcal {A}} _ {p, d} (t) := \left\{A \in \widetilde {\mathcal {A}} _ {p, d} \mid \mathbb {P} (X \in A) \leq t \right\}
$$
Lemma A.6 Suppose Assumption 2.1 holds true. Let  $z_{1}, \ldots, z_{n}$  be i.i.d. bounded random variables with  $|z_{1}| \leq V < \infty$  almost surely. Assume that for each  $i \in [n]$ ,  $z_{i}$  is independent of  $\{x_{j}\}_{j \neq i}$ , but may be dependent on  $x_{i}$ . Given any  $\delta \in (0,1)$ , with probability at least  $1 - \delta$ , it holds
$$
\max  _ {A \in \widetilde {\mathcal {A}} _ {p, d} \backslash \widetilde {\mathcal {A}} _ {p, d} (\bar {t} _ {1} (\delta))} \frac {1}{\sqrt {\mathbb {P} (X \in A)}} \left| \frac {1}{n} \sum_ {i = 1} ^ {n} z _ {i} 1 _ {\{x _ {i} \in A \}} - \mathbb {E} (z _ {1} 1 _ {\{x _ {1} \in A \}}) \right| \leq 2 V \sqrt {\bar {t} _ {1} (\delta)}
$$
where  $U = M + m$ .
Proof. For each fixed  $A \in \widetilde{\mathcal{A}}_{p,d} \setminus \widetilde{\mathcal{A}}_{p,d}(\bar{t}_1(\delta))$ , note that
$$
\left. \right.\left| \mathbb {E} \left(\left(z _ {1} 1 _ {\{x _ {1} \in A \}} - \mathbb {E} \left(z _ {1} 1 _ {\{x _ {1} \in A \}}\right)\right) ^ {k}\right)\right| \leq (2 V) ^ {k} \mathbb {P} (X \in A) \quad \forall k \geq 2
$$
so by Lemma D.1 with  $t = 2V\sqrt{\mathbb{P}(X\in A)}\sqrt{\bar{t}_1(\delta)},\gamma^2 = (2V)^2\mathbb{P}(X\in A)$  and  $b = 2V$  , we have
$$
\begin{array}{l} \mathbb {P} \left(\frac {1}{\sqrt {\mathbb {P} (X \in A)}} \left| \frac {1}{n} \sum_ {i = 1} ^ {n} z _ {i} 1 _ {\{x _ {i} \in A \}} - \mathbb {E} \left(z _ {1} 1 _ {\{x _ {1} \in A \}}\right) \right| > 2 V \sqrt {\bar {t} _ {1} (\delta)}\right) \\ \leq 2 \exp \left(- \frac {n}{4} \left(\frac {4 V ^ {2} \mathbb {P} (X \in A) \bar {t} _ {1} (\delta)}{4 V ^ {2} \mathbb {P} (X \in A)} \wedge \frac {2 V \sqrt {\mathbb {P} (X \in A) \bar {t} _ {1} (\delta)}}{2 V}\right)\right) \\ \stackrel {(i)} {=} 2 \exp \left(- \frac {n}{4} \bar {t} _ {1} (\delta)\right) = \delta / (p ^ {d} (n + 1) ^ {2 d}) \\ \end{array}
$$
where  $(i)$  is because  $\mathbb{P}(X\in A)\geq \bar{t}_1(\delta)$  (since  $A\in \widetilde{\mathcal{A}}_{p,d}\setminus \widetilde{\mathcal{A}}_{p,d}(\bar{t}_1(\delta))$ ). As a result, we have
$$
\begin{array}{l} \mathbb {P} \left(\max  _ {A \in \tilde {\mathcal {A}} _ {p, d} \backslash \tilde {\mathcal {A}} _ {p, d} (\bar {t} _ {1} (\delta))} \frac {1}{\sqrt {\mathbb {P} (X \in A)}} \Big | \frac {1}{n} \sum_ {i = 1} ^ {n} z _ {i} 1 _ {\{x _ {i} \in A \}} - \mathbb {E} (z _ {1} 1 _ {\{x _ {1} \in A \}}) \Big | > 2 V \sqrt {\bar {t} _ {1} (\delta)}\right) \\ \leq \sum_ {A \in \tilde {\mathcal {A}} _ {p, d} \backslash \tilde {\mathcal {A}} _ {p, d} (\bar {t} _ {1} (\delta))} \mathbb {P} \left(\frac {1}{\sqrt {\mathbb {P} (X \in A)}} \Big | \frac {1}{n} \sum_ {i = 1} ^ {n} z _ {i} 1 _ {\{x _ {i} \in A \}} - \mathbb {E} (z _ {1} 1 _ {\{x _ {1} \in A \}}) \Big | > 2 V \sqrt {\bar {t} _ {1} (\delta)}\right) \\ \leq \left| \tilde {\mathcal {A}} _ {p, d} \backslash \tilde {\mathcal {A}} _ {p, d} \left(\bar {t} _ {1} (\delta)\right) \right| \cdot \delta / \left(p ^ {d} (n + 1) ^ {2 d}\right) \leq \delta \\ \end{array}
$$
where the last inequality makes use of Lemma A.5 (3).
Lemma A.7 Let  $\mathcal{D}$  be a finite collection of measurable subsets of  $[0,1]^p$  satisfying  $\mathbb{P}(X\in D)\leq \bar{\alpha}$  for all  $D\in \mathcal{D}$  (for some constant  $\bar{\alpha}\in (0,1)$ ). Given any  $\delta \in (0,1)$ , if
$$
w (\bar {\alpha}, \delta) := (e ^ {2} \bar {\alpha}) \vee \frac {\log (| \mathcal {D} | / \delta)}{n} \leq 3 / 4
$$
then with probability at least  $1 - \delta$  it holds
$$
\max  _ {D \in \mathcal {D}} \left\{\frac {1}{n} \sum_ {i = 1} ^ {n} 1 _ {\{x _ {i} \in D \}} \right\} \leq w (\bar {\alpha}, \delta)
$$
Proof. For any fixed  $D \in \mathcal{D}$ , denote  $\alpha = \mathbb{P}(X \in D)$ , then by Lemma D.2, for any  $t \in (0, 3/4]$ , we have
$$
\begin{array}{l} \mathbb {P} \left(\frac {1}{n} \sum_ {i = 1} ^ {n} 1 _ {\{x _ {i} \in D \}} > t\right) \leq \exp \left(- n \left(t \log (t / \alpha) + (1 - t) \log \left(\frac {1 - t}{1 - \alpha}\right)\right)\right) \\ \leq \exp (- n (t \log (t / \alpha) + (1 - t) \log (1 - t))) \\ \leq \exp \left(- n \left(t \log (t / \alpha) + (1 - t) (- t - t ^ {2})\right)\right) \\ = \exp \left(- n \left(t \left(\log (t / \alpha) - 1\right) + t ^ {3}\right)\right) \\ \leq \exp (- n t (\log (t / \alpha) - 1)) \\ \end{array}
$$
where the third inequality makes use of Lemma D.3 and the assumption  $t \leq 3/4$ . Take  $t = w(\bar{\alpha}, \delta)$ , and note that
$$
\log (w (\bar {\alpha}, \delta) / \alpha) - 1 \geq \log (w (\bar {\alpha}, \delta) / \bar {\alpha}) - 1 \geq \log (e ^ {2}) - 1 \geq 1
$$
$$
\mathbb {P} \left(\frac {1}{n} \sum_ {i = 1} ^ {n} 1 _ {\{x _ {i} \in B \}} > w (\bar {\alpha}, \delta)\right) \leq \exp (- n w (\bar {\alpha}, \delta)) \leq \delta / | \mathcal {D} |
$$
where the last inequality is because of the definition of  $w(\bar{\alpha}, \delta)$ . Taking the union bound we have
$$
\mathbb {P} \left(\max  _ {D \in \mathcal {D}} \left\{\frac {1}{n} \sum_ {i = 1} ^ {n} 1 _ {\{x _ {i} \in D \}} \right\} > w (\bar {\alpha}, \delta)\right) \leq | \mathcal {D} | \cdot \delta / | \mathcal {D} | = \delta
$$
Corollary A.8 Under Assumption 2.1 and given  $\delta \in (0,1)$ , suppose  $\bar{t}_2(\delta) < 3/4$ , then with probability at least  $1 - \delta$ , it holds
$$
\max  _ {B \in \mathcal {B} _ {p, d}} \left\{\frac {1}{n} \sum_ {i = 1} ^ {n} 1 _ {\{x _ {i} \in B \}} \right\} \leq \bar {t} _ {2} (\delta)
$$
Proof. Apply Lemma A.7 with  $\mathcal{D} = \mathcal{B}_{p,d}$  and  $\bar{\alpha} = 2\bar{\theta} d / n$ , and note that  $|\mathcal{B}_{p,d}| \leq (n + 1)^{2d}p^{d}$  (by Lemma A.5 (3)) and the definition  $\bar{t}_2(\delta) = \frac{2\bar{\theta}e^2d}{n} \vee \frac{\log(p^d(n + 1)^{2d} / \delta)}{n}$ .
Lemma A.9 Suppose Assumption 2.1 holds true. Let  $z_{1}, \ldots, z_{n}$  be i.i.d. bounded random variables with  $|Z| \leq V < \infty$  almost surely. Assume that for each  $i \in [n]$ ,  $z_{i}$  is independent of  $\{x_{j}\}_{j \neq i}$ , but may be dependent on  $x_{i}$ . Given any  $\delta \in (0,1)$ , suppose  $\bar{t}_{2}(\delta / 2) < 3/4$ , then with probability at least  $1 - \delta$ , it holds
$$
\sup  _ {A \in \mathcal {A} _ {p, d} \backslash \mathcal {A} _ {p, d} (\bar {t} (\delta / 2))} \frac {1}{\sqrt {\mathbb {P} (X \in A)}} \left| \frac {1}{n} \sum_ {i = 1} ^ {n} z _ {i} 1 _ {\{x _ {i} \in A \}} - \mathbb {E} \left(z _ {1} 1 _ {\{x _ {1} \in A \}}\right) \right| \leq 5 V \sqrt {\bar {t} (\delta / 2)}. \tag {50}
$$
Proof. Define events  $\mathcal{E}_1$  and  $\mathcal{E}_2$ :
$$
\mathcal {E} _ {1} := \left\{\max  _ {B \in \mathcal {B} _ {p, d}} \left\{\frac {1}{n} \sum_ {i = 1} ^ {n} 1 _ {\{x _ {i} \in B \}} \right\} \leq \bar {t} _ {2} (\delta / 2) \right\}
$$
$$
\mathcal {E} _ {2} := \left\{\max  _ {A \in \tilde {\mathcal {A}} _ {p, d} \backslash \tilde {\mathcal {A}} _ {p, d} (\bar {t} _ {1} (\delta / 2))} \frac {1}{\sqrt {\mathbb {P} (X \in A)}} \left| \frac {1}{n} \sum_ {i = 1} ^ {n} z _ {i} 1 _ {\{x _ {i} \in A \}} - \mathbb {E} \left(z _ {1} 1 _ {\{x _ {1} \in A \}}\right) \right| \leq V \sqrt {\bar {t} _ {1} (\delta / 2)} \right\}
$$
Then by Lemma A.6 and Corollary A.8, we have  $\mathbb{P}(\mathcal{E}_1) \geq 1 - \delta / 2$  and  $\mathbb{P}(\mathcal{E}_2) \geq 1 - \delta / 2$ , hence  $\mathbb{P}(\mathcal{E}_1 \cap \mathcal{E}_2) \geq 1 - \delta$ . Below we prove that when  $\mathcal{E}_1$  and  $\mathcal{E}_2$  hold true, inequality (50) holds true.
Note that for any  $A\in \mathcal{A}_{p,d}\setminus \mathcal{A}_{p,d}(\bar{t} (\delta /2))$
$$
\begin{array}{l} \frac {1}{\sqrt {\mathbb {P} (X \in A)}} \left| \frac {1}{n} \sum_ {i = 1} ^ {n} z _ {i} 1 _ {\{x _ {i} \in A \}} - \mathbb {E} \left(z _ {1} 1 _ {\{x _ {1} \in A \}}\right) \right| \\ \leq \frac {1}{\sqrt {\mathbb {P} (X \in A)}} \left| \frac {1}{n} \sum_ {i = 1} ^ {n} z _ {i} 1 _ {\{x _ {i} \in A \}} - \frac {1}{n} \sum_ {i = 1} ^ {n} z _ {i} 1 _ {\{x _ {i} \in A ^ {\prime} \}} \right| \\ + \frac {1}{\sqrt {\mathbb {P} (X \in A)}} \left| \frac {1}{n} \sum_ {i = 1} ^ {n} z _ {i} 1 _ {\{x _ {i} \in A ^ {\prime} \}} - \mathbb {E} \left(z _ {1} 1 _ {\{x _ {1} \in A ^ {\prime} \}}\right) \right| \tag {51} \\ + \frac {1}{\sqrt {\mathbb {P} (X \in A)}} \left| \mathbb {E} \left(z _ {1} 1 _ {\{x _ {1} \in A ^ {\prime} \}}\right) - \mathbb {E} \left(z _ {1} 1 _ {\{x _ {1} \in A \}}\right) \right| \\ := T _ {1} + T _ {2} + T _ {3} \\ \end{array}
$$
To bound  $T_{1}$ , we have
$$
\begin{array}{l} T _ {1} = \frac {1}{\sqrt {\mathbb {P} (X \in A)}} \left| \frac {1}{n} \sum_ {i = 1} ^ {n} z _ {i} 1 _ {\{x _ {i} \in A ^ {\prime} \backslash A \}} \right| \leq \frac {V}{\sqrt {\mathbb {P} (X \in A)}} \left(\frac {1}{n} \sum_ {i = 1} ^ {n} 1 _ {\{x _ {i} \in A ^ {\prime} \backslash A \}}\right) \tag {52} \\ \leq \frac {V}{\sqrt {\mathbb {P} (X \in A)}} \max  _ {B \in \mathcal {B} _ {p, d}} \left\{\frac {1}{n} \sum_ {i = 1} ^ {n} 1 _ {\{x _ {i} \in B \}} \right\} \leq \frac {V}{\sqrt {\mathbb {P} (X \in A)}} \bar {t} _ {2} (\delta / 2) \leq V \sqrt {\bar {t} _ {2} (\delta / 2)} \\ \end{array}
$$
where the second inequality makes use of Lemma A.5 (1), and the third inequality is by  $\mathcal{E}_1$ .
To bound  $T_{2}$ , note that
$$
\begin{array}{l} T _ {2} = \sqrt {\frac {\mathbb {P} \left(X \in A ^ {\prime}\right)}{\mathbb {P} \left(X \in A\right)}} \frac {1}{\sqrt {\mathbb {P} \left(X \in A ^ {\prime}\right)}} \left| \frac {1}{n} \sum_ {i = 1} ^ {n} z _ {i} 1 _ {\left\{x _ {i} \in A ^ {\prime} \right\}} - \mathbb {E} \left(z _ {1} 1 _ {\left\{x _ {1} \in A ^ {\prime} \right\}}\right) \right| \tag {53} \\ \leq \sqrt {\frac {\mathbb {P} (X \in A ^ {\prime})}{\mathbb {P} (X \in A)}} 2 V \sqrt {\bar {t} _ {1} (\delta / 2)} \\ \end{array}
$$