SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
\mathcal {R} _ {L, \mathrm {P}} (\overline {{f}} _ {\pi}) - \mathcal {R} _ {L, \mathrm {P}} (f ^ {*}) = \int_ {x \in \mathcal {X}} \left(\frac {\int_ {A (x)} f (x ^ {\prime}) d \mathrm {P} _ {X} (x ^ {\prime})}{\int_ {A (x)} d \mathrm {P} _ {X} (x ^ {\prime})} - f (x)\right) ^ {2} d P _ {X} (x).
$$
For each  $x$ , if Assumption 3.2 holds, the point-wise error can be bounded by
$$
\begin{array}{l} \frac {\int_ {A (x)} f (x ^ {\prime}) d \mathrm {P} _ {X} (x ^ {\prime})}{\int_ {A (x)} d \mathrm {P} _ {X} (x ^ {\prime})} - f (x) \leq \frac {\int_ {A (x)} | f (x ^ {\prime}) - f (x) | d \mathrm {P} _ {X} (x ^ {\prime})}{\int_ {A (x)} d \mathrm {P} _ {X} (x ^ {\prime})} \\ \leq \frac {\int_ {A (x)} c _ {L} \| x ^ {\prime} - x \| ^ {\alpha} d \mathrm {P} _ {X} \left(x ^ {\prime}\right)}{\int_ {A (x)} d \mathrm {P} _ {X} \left(x ^ {\prime}\right)} \leq c _ {L} \operatorname {d i a m} (A (x)) ^ {\alpha} \\ \end{array}
$$
Then we can bound the integral of point-wise error by the sum of the integral of errors on each  $A_{j}$ , namely
$$
\int_ {x \in \mathcal {X}} \left(\frac {\int_ {A (x)} f \left(x ^ {\prime}\right) d \mathrm {P} _ {X} \left(x ^ {\prime}\right)}{\int_ {A (x)} d \mathrm {P} _ {X} \left(x ^ {\prime}\right)} - f (x)\right) ^ {2} d \mathrm {P} (x) \leq \sum_ {j \in \mathcal {I}} c _ {L} ^ {2} \operatorname {d i a m} \left(A _ {j}\right) ^ {2 \alpha} \int_ {A _ {j}} d \mathrm {P} _ {X} (x). \tag {24}
$$
Apply Lemma C.5 and C.6, we have
$$
\begin{array}{l} \sum_ {j \in \mathcal {I}} c _ {L} ^ {2} \operatorname {d i a m} \left(A _ {j}\right) ^ {2 \alpha} \int_ {A _ {j}} d \mathrm {P} _ {X} (x) \leq \sum_ {j \in \mathcal {I}} 2 c _ {L} ^ {2} \sqrt {d} 2 ^ {- 2 \alpha \cdot \operatorname {d e p t h} \left(A _ {j}\right) / d - s} \\ = \sum_ {j \in \mathcal {I}} 2 c _ {L} ^ {2} \sqrt {d} 2 ^ {s - \operatorname {d e p t h} (A _ {j})} \cdot 2 ^ {- 2 \alpha \cdot \operatorname {d e p t h} (A _ {j}) / d + \operatorname {d e p t h} (A _ {j}) - 2 s}. \tag {25} \\ \end{array}
$$
For each  $j$ , we can guarantee
$$
2 ^ {- 2 \alpha \cdot \operatorname {d e p t h} (A _ {j}) / d + \operatorname {d e p t h} (A _ {j})} \leq 2 ^ {(1 - (2 \alpha / d) \wedge 1) \operatorname {d e p t h} (A _ {j})} \leq 2 ^ {(1 - (2 \alpha / d) \wedge 1) s} \tag {26}
$$
since  $(2\alpha /d)\wedge 1\leq 1$  . Bringing (26) into (25), we get
$$
\sum_ {j \in \mathcal {I}} c _ {L} ^ {2} \operatorname {d i a m} \left(A _ {j}\right) ^ {2 \alpha} \int_ {A _ {j}} d P (x) \leq 2 ^ {- s - (2 \alpha s / d) \wedge s} \sum_ {j \in \mathcal {I}} 2 c _ {L} ^ {2} \sqrt {d} 2 ^ {s - \operatorname {d e p t h} \left(A _ {j}\right)}
$$
By applying Lemma C.7, we immediately have
$$
\sum_ {j \in \mathcal {I}} c _ {L} ^ {2} \mathrm {d i a m} (A _ {j}) ^ {2 \alpha} \int_ {A _ {j}} d \mathrm {P} (x) \leq 2 c _ {L} ^ {2} 2 ^ {- (2 \alpha s / d) \wedge s}.
$$
This together with (24) yields the desired conclusion.
C.5 Proofs of results in Section C.4
Proof of Lemma C.1. The conclusion follows from Example 2.4 and Proposition 2.5 in [55].  $\square$
Proof of Lemma C.2. The conclusion follows from (2.18) in [55].
In the subsequent proof, we define the empirical measure  $D := \frac{1}{n} \sum_{i=1}^{n} \delta_{(X_i, Y_i)}$  given samples  $D = \{(X_1, Y_1), \dots, (X_n, Y_n)\}$ , where  $\delta$  is the Dirac function. To conduct our analysis, we first need to recall the definitions of VC dimension (VC index) and covering number, which are frequently used in capacity-involved arguments and measure the complexity of the underlying function class [54, 40, 31].
Definition C.8 (VC dimension). Let  $\mathcal{B}$  be a class of subsets of  $\mathcal{X}$  and  $A\subset \mathcal{X}$  be a finite set. The trace of  $\mathcal{B}$  on  $A$  is defined by  $\{B\cap A:B\subset \mathcal{B}\}$ . Its cardinality is denoted by  $\Delta^{\mathcal{B}}(A)$ . We say that  $\mathcal{B}$  shatters  $A$  if  $\Delta^{\mathcal{B}}(A) = 2^{\# (A)}$ , that is, if for every  $A^{\prime}\subset A$ , there exists a  $B\subset \mathcal{B}$  such that  $A^{\prime} = B\cap A$ . For  $n\in \mathbb{N}$ , let
$$
m ^ {\mathcal {B}} (n) := \sup  _ {A \subset \mathcal {X}, \# (A) = n} \Delta^ {\mathcal {B}} (A). \tag {27}
$$
Then, the set  $\mathcal{B}$  is a Vapnik-Chervonenkis class if there exists  $n < \infty$  such that  $m^{\mathcal{B}}(n) < 2^{n}$  and the minimal of such  $n$  is called the VC dimension of  $\mathcal{B}$ , and abbreviate as  $\mathrm{VC}(\mathcal{B})$ .
Since an arbitrary set of  $n$  points  $\{x_1, \ldots, x_n\}$  possesses  $2^n$  subsets, we say that  $\mathcal{B}$  picks out a certain subset from  $\{x_1, \ldots, x_n\}$  if this can be formed as a set of the form  $B \cap \{x_1, \ldots, x_n\}$  for a  $B \in \mathcal{B}$ . The collection  $\mathcal{B}$  shatters  $\{x_1, \ldots, x_n\}$  if each of its  $2^n$  subsets can be picked out in this manner. From Definition C.8 we see that the VC dimension of the class  $\mathcal{B}$  is the smallest  $n$  for which no set of size  $n$  is shattered by  $\mathcal{B}$ , that is,
$$
\operatorname {V C} (\mathcal {B}) = \inf  \left\{n: \max  _ {x _ {1}, \dots , x _ {n}} \Delta^ {\mathcal {B}} (\{x _ {1}, \dots , x _ {n} \}) \leq 2 ^ {n} \right\},
$$
where  $\Delta^{\mathcal{B}}(\{x_1,\ldots ,x_n\}) = \# \{B\cap \{x_1,\ldots ,x_n\} :B\in \mathcal{B}\}$ . Clearly, the more refined  $\mathcal{B}$  is, the larger is its index.
To further bound the capacity of the function sets, we need to introduce the following fundamental descriptions of covering number which enables an approximation of an infinite set by finite subsets.
Definition C.9 (Covering Number). Let  $(\mathcal{X},d)$  be a metric space and  $A\subset \mathcal{X}$ . For  $\varepsilon >0$ , the  $\varepsilon$ -covering number of  $A$  is denoted as
$$
\mathcal {N} (A, d, \varepsilon) := \min  \left\{n \geq 1: \exists x _ {1}, \dots , x _ {n} \in \mathcal {X} \text {s u c h t h a t} A \subset \bigcup_ {i = 1} ^ {n} B (x _ {i}, \varepsilon) \right\},
$$
where  $B(x,\varepsilon)\coloneqq \{x^{\prime}\in \mathcal{X}:d(x,x^{\prime})\leq \varepsilon \}$
To prove Lemma C.10, we need the following fundamental lemma concerning with the VC dimension of random partitions in Section 2.3, which follows the idea put forward by [30] of the construction of random forest. To this end, let  $p \in \mathbb{N}$  be fixed and  $\pi_p$  be a partition of  $\mathcal{X}$  with number of splits  $s$  and  $\pi_{(p)}$  denote the collection of all partitions  $\pi_p$ .
Lemma C.10. Let  $\tilde{\mathcal{A}}$  be the collection of all cells  $\times_{i=1}^{d}[a_i, b_i]$  in  $\mathbb{R}^d$ . The VC index of  $\tilde{\mathcal{A}}$  equals  $2d + 1$ . Moreover, for all  $0 < \varepsilon < 1$ , there exists a universal constant  $C$  such that
$$
\mathcal {N} (\mathbf {1} _ {\tilde {\mathcal {A}}}, \| \cdot \| _ {L _ {1} (Q)}, \varepsilon) \leq C (2 d + 1) (4 e) ^ {2 d + 1} (1 / \varepsilon) ^ {2 d}.
$$
Proof of Lemma C.10. The first result of VC index follows from Example 2.6.1 in [54]. The second result of covering number follows directly from Theorem 9.2 in [40].  $\square$
Before we proceed, we list the well-known Bernstein's inequality that will be used frequently in the proofs. Lemma C.11 was introduced in [8] and can be found in many statistical learning textbooks, see e.g., [42, 19, 51].
Lemma C.11 (Bernstein's inequality). Let  $B > 0$  and  $\sigma > 0$  be real numbers, and  $n \geq 1$  be an integer. Furthermore, let  $\xi_1, \ldots, \xi_n$  be independent random variables satisfying  $\mathbb{E}_P \xi_i = 0$ ,  $\| \xi_i \|_{\infty} \leq B$ , and  $\mathbb{E}_P \xi_i^2 \leq \sigma^2$  for all  $i = 1, \ldots, n$ . Then for all  $\tau > 0$ , we have
$$
P \bigg (\frac {1}{n} \sum_ {i = 1} ^ {n} \xi_ {i} \geq \sqrt {\frac {2 \sigma^ {2} \tau}{n}} + \frac {2 B \tau}{3 n} \bigg) \leq e ^ {- \tau}.
$$
Proof of Lemma C.3. Let  $\tilde{\mathcal{A}}$  be the collection of all cells  $\times_{i=1}^{d}[a_i, b_i]$  in  $\mathbb{R}^d$ . Applying Lemma C.10 with  $Q := (\mathrm{D}_X + \mathrm{P}_X)/2$ , there exists an  $\varepsilon$ -net  $\{\tilde{A}_k\}_{k=1}^K \subset \tilde{\mathcal{A}}$  with
$$
K \leq C (2 d + 1) (4 e) ^ {2 d + 1} (1 / \varepsilon) ^ {2 d} \tag {28}
$$
such that for any  $j\in \mathcal{I}_p$  , there exist some  $k\in \{1,\dots ,K\}$  such that
$$
\| \mathbf {1} \{x \in A _ {p} ^ {j} \} - \mathbf {1} \{x \in \tilde {A} _ {k} \} \| _ {L _ {1} ((\mathrm {D} _ {X} + \mathrm {P} _ {X}) / 2)} \leq \varepsilon ,
$$
$$
\begin{array}{l} \left\| \mathbf {1} \{x \in A _ {p} ^ {j} \} - \mathbf {1} \{x \in \tilde {A} _ {k} \} \right\| _ {L _ {1} \left(\left(\mathrm {D} _ {X} + \mathrm {P} _ {X}\right) / 2\right)} \\ = 1 / 2 \cdot \| \mathbf {1} \{x \in A _ {p} ^ {j} \} - \mathbf {1} \{x \in \tilde {A} _ {k} \} \| _ {L _ {1} (\mathrm {D} _ {X})} + 1 / 2 \cdot \| \mathbf {1} \{x \in A _ {p} ^ {j} \} - \mathbf {1} \{x \in \tilde {A} _ {k} \} \| _ {L _ {1} (\mathrm {P} _ {X})}, \\ \end{array}
$$
$$
\left\| \mathbf {1} \{x \in A _ {p} ^ {j} \} - \mathbf {1} \{x \in \tilde {A} _ {k} \} \right\| _ {L _ {1} (\mathrm {D} _ {X})} \leq 2 \varepsilon , \quad \left\| \mathbf {1} \{x \in A _ {p} ^ {j} \} - \mathbf {1} \{x \in \tilde {A} _ {k} \} \right\| _ {L _ {1} (\mathrm {P} _ {X})} \leq 2 \varepsilon . \tag {29}
$$
Consequently, by the definition of the covering number and the triangle inequality, for any  $j \in \mathcal{I}_p$ , there holds
$$
\begin{array}{l} \left| \frac {1}{n} \sum_ {i = 1} ^ {n} \mathbf {1} \{x \in A _ {p} ^ {j} \} \left(X _ {i}\right) - \int_ {\tilde {A} _ {p} ^ {j}} d \mathrm {P} _ {X} \left(x ^ {\prime}\right) \right| \\ \leq \left| \frac {1}{n} \sum_ {i = 1} ^ {n} \mathbf {1} \{x \in \tilde {A} _ {k} \} (X _ {i}) - \int_ {\tilde {A} _ {k}} d \mathrm {P} _ {X} (x ^ {\prime}) \right| + \| \mathbf {1} \{x \in A _ {p} ^ {j} \} - \mathbf {1} \{x \in \tilde {A} _ {k} \} \| _ {L _ {1} (\mathrm {D} _ {X})} \\ + \| \mathbf {1} \{x \in A _ {p} ^ {j} \} - \mathbf {1} \{x \in \tilde {A} _ {k} \} \| _ {L _ {1} (\mathrm {P} _ {X})} \leq \left| \frac {1}{n} \sum_ {i = 1} ^ {n} \mathbf {1} \{x \in \tilde {A} _ {k} \} (X _ {i}) - \int_ {\tilde {A} _ {k}} d \mathrm {P} _ {X} \left(x ^ {\prime}\right) \right| + 4 \varepsilon . \\ \end{array}
$$
$$
\sup  _ {j \in \mathcal {I}} \left| \frac {1}{n} \sum_ {i = 1} ^ {n} \mathbf {1} \{x \in A _ {p} ^ {j} \} \left(X _ {i}\right) - \int_ {\tilde {A} _ {p} ^ {j}} d \mathrm {P} _ {X} \left(x ^ {\prime}\right) \right| \leq \sup  _ {1 \leq k \leq K} \left| \frac {1}{n} \sum_ {i = 1} ^ {n} \mathbf {1} \{x \in \tilde {A} _ {k} \} \left(X _ {i}\right) - \int_ {\tilde {A} _ {k}} d \mathrm {P} _ {X} \left(x ^ {\prime}\right) \right| + 4 \varepsilon . \tag {30}
$$
For any fixed  $1 \leq k \leq K$ , let the random variable  $\xi_{i}$  be defined by  $\xi_{i} := \mathbf{1}\{X_{i} \in \tilde{A}_{k}\} - \int_{\tilde{A}_{k}} d\mathrm{P}_{X}(x')$ . Then we have  $\mathbb{E}_{\mathrm{P}_X} \xi_i = 0$ ,  $\| \xi \|_{\infty} \leq 1$ , and  $\mathbb{E}_{\mathrm{P}_X} \xi_i^2 \leq \int_{\tilde{A}_k} d\mathrm{P}_X(x')$ . According to Assumption 3.2, there holds  $\mathbb{E}_{\mathrm{P}_X} \xi_i^2 \leq \overline{c} \cdot 2^{-s}$ . Applying Bernstein's inequality in Lemma C.11, we obtain
$$
\left| \frac {1}{n} \sum_ {i = 1} ^ {n} {\bf 1} \{X _ {i} \in \tilde {A} _ {k} \} - \int_ {\tilde {A} _ {k}} d \mathrm {P} _ {X} (x ^ {\prime}) \right| \leq \sqrt {\frac {\overline {{c}} \cdot 2 ^ {1 - s} \cdot \tau}{n}} + \frac {2 \tau \log n}{3 n}
$$
with probability  $\mathrm{P}^n$  at least  $1 - 2e^{-\tau}$ . Then the union bound together with the covering number estimate (28) implies that
$$
\sup  _ {1 \leq k \leq K} \left| \frac {1}{n} \sum_ {i = 1} ^ {n} \mathbf {1} \{X _ {i} \in \tilde {A} _ {k} \} - \int_ {\tilde {A} _ {k}} d \mathrm {P} _ {X} (x ^ {\prime}) \right| \leq \sqrt {\frac {\overline {{c}} \cdot 2 ^ {1 - s} (\tau + \log (2 K))}{n}} + \frac {2 (\tau + \log (2 K)) \log n}{3 n}
$$
with probability  $\mathrm{P}^n$  at least  $1 - e^{-\tau}$ . Let  $\tau = 2\log n$  and  $\varepsilon = 1 / n$ . Then for any  $n > N_1 \coloneqq (2C) \wedge (2d + 1) \wedge (4e)$ , we have  $\tau + \log(2K) = 2\log n + \log(2C) + \log(2d + 1) + (2d + 1)\log(4e) + 2d\log n \leq (4d + 5)\log n$ . Therefore, we have
$$
\sup  _ {1 \leq k \leq K} \left| \frac {1}{n} \sum_ {i = 1} ^ {n} \mathbf {1} \left\{X _ {i} \in \tilde {A} _ {k} \right\} - \int_ {\tilde {A} _ {k}} d \mathrm {P} _ {X} \left(x ^ {\prime}\right) \right| \leq \sqrt {\frac {\bar {c} \cdot 2 ^ {1 - s} (4 d + 5) \log n}{n}} + \frac {2 (4 d + 5) \log n}{3 n} \tag {31}
$$
with probability  $\mathrm{P}^n$  at least  $1 - 1 / n^{2}$ . This together with (30) yields that
$$
\sup _ {j \in \mathcal {I}} \left| \frac {1}{n} \sum_ {i = 1} ^ {n} \mathbf {1} \{x \in A _ {p} ^ {j} \} - \int_ {\tilde {A} _ {p} ^ {j}} d \mathrm {P} _ {X} (x ^ {\prime}) \right| \leq \sqrt {\frac {\overline {{c}} \cdot 2 ^ {1 - s} (4 d + 5) \log n}{n}} + \frac {2 (4 d + 5) \log n}{3 n} + \frac {4}{n}.
$$
Proof of Lemma C.4. Let  $\tilde{\mathcal{A}}$  be the collection of all cells  $\times_{i=1}^{d}[a_i, b_i]$  in  $\mathbb{R}^d$ . Then there exists an  $\varepsilon$ -net  $\{\tilde{A}_k\}_{k=1}^K \subset \tilde{\mathcal{A}}$  with  $K$  bounded by (28) such that for any  $j \in \mathcal{I}$ , (29) holds for some  $k \in \{1, \ldots, K\}$ . Consequently, by the definition of the covering number and the triangle inequality, for any  $j \in \mathcal{I}_p$ , there holds
$$
\begin{array}{l} \left| \sum_ {i = 1} ^ {n} {\bf 1} \{X _ {i} \in A _ {p} ^ {j} \} Y _ {i} - \int_ {A _ {p} ^ {j}} f ^ {*} (x ^ {\prime}) d \mathrm {P} _ {X} (x ^ {\prime}) \right| \\ \leq \left| \sum_ {i = 1} ^ {n} \mathbf {1} \left\{X _ {i} \in \tilde {A} _ {k} \right\} Y _ {i} - \int_ {\tilde {A} _ {k}} f ^ {*} \left(x ^ {\prime}\right) d \mathrm {P} _ {X} \left(x ^ {\prime}\right) \right| \\ + \int_ {\mathbb {R} ^ {d}} \left| \mathbf {1} \left\{x ^ {\prime} \in A _ {p} ^ {j} \right\} - \mathbf {1} \left\{x ^ {\prime} \in \tilde {A} _ {k} \right\} \right| \left| f ^ {*} \left(x ^ {\prime}\right) \right| d P _ {X} \left(x ^ {\prime}\right) + \sum_ {i = 1} ^ {n} \left| \mathbf {1} \left\{X _ {i} \in \tilde {A} _ {k} \right\} - \mathbf {1} \left\{X _ {i} \in A _ {p} ^ {j} \right\} \right| \left| Y _ {i} \right| \\ \leq \left| \sum_ {i = 1} ^ {n} \mathbf {1} \left\{X _ {i} \in \tilde {A} _ {k} \right\} Y _ {i} - \int_ {\tilde {A} _ {k}} f ^ {*} \left(x ^ {\prime}\right) d \mathrm {P} _ {X} \left(x ^ {\prime}\right) \right| \\ + \max  _ {1 \leq i \leq n} | Y _ {i} | \cdot \| \mathbf {1} \{x \in A _ {p} ^ {j} \} - \mathbf {1} \{x \in \tilde {A} _ {k} \} \| _ {L _ {1} (\mathrm {D} _ {X})} + M \cdot \| \mathbf {1} \{x \in A _ {p} ^ {j} \} - \mathbf {1} \{x \in \tilde {A} _ {k} \} \| _ {L _ {1} (\mathrm {P} _ {X})} \\ \leq \left| \sum_ {i = 1} ^ {n} \mathbf {1} _ {\tilde {A} _ {k}} \left(X _ {i}\right) Y _ {i} - \int_ {\tilde {A} _ {k}} f ^ {*} \left(x ^ {\prime}\right) d \mathrm {P} _ {X} \left(x ^ {\prime}\right) \right| + 4 M \varepsilon . \tag {32} \\ \end{array}
$$
where the last inequality follows from the condition  $\mathcal{V} \subset [-M, M]$ .
For any fixed  $1 \leq k \leq K$ , let the random variable  $\tilde{\xi}_i$  be defined by  $\tilde{\xi}_i := \mathbf{1}\{X_i \in \tilde{A}_k\} Y_i - \int_{\tilde{A}_k} f^*(x') d\mathrm{P}_X(x')$ . Then we have  $\mathbb{E}_{\mathrm{P}} \tilde{\xi}_i = 0$ ,  $\| \xi \|_{\infty} \leq 1$ , and  $\mathbb{E}_{\mathrm{P}} \tilde{\xi}_i^2 \leq M^2 \int_{\tilde{A}_k} d\mathrm{P}(x')$ . According to Assumption 3.2, there holds  $\mathbb{E}_{\mathrm{P}} \tilde{\xi}_i^2 \leq M^2 \cdot \overline{c} \cdot 2^{-s}$ . Applying Bernstein's inequality in Lemma C.11, we obtain
$$
\left| \sum_ {i = 1} ^ {n} {\bf 1} \{X _ {i} \in \tilde {A} _ {k} \} Y _ {i} - \int_ {\tilde {A} _ {k}} f ^ {*} (x ^ {\prime}) d \mathrm {P} _ {X} (x ^ {\prime}) \right| \leq \sqrt {\frac {M ^ {2} \cdot \overline {{c}} \cdot 2 ^ {1 - s} \cdot \tau}{n}} + \frac {2 M \tau \log n}{3 n}
$$
with probability  $\mathrm{P}^n$  at least  $1 - 2e^{-\tau}$ . Similar to the proof of Lemma C.3, one can show that for any  $n \geq N_1$ , there holds
$$
\sup _ {1 \leq k \leq K} \left| \sum_ {i = 1} ^ {n} {\bf 1} \{X _ {i} \in \tilde {A} _ {k} \} Y _ {i} - \int_ {\tilde {A} _ {k}} f ^ {*} (x ^ {\prime}) d \mathrm {P} _ {X} (x ^ {\prime}) \right| \leq M \sqrt {\frac {\overline {{c}} \cdot 2 ^ {1 - s} \cdot \tau}{n}} + \frac {2 M \tau \log n}{3 n}
$$
with probability  $\mathrm{P}^n$  at least  $1 - 1 / n^{2}$ . This together with (32) yields that
$$
\begin{array}{l} \left| \sum_ {i = 1} ^ {n} \mathbf {1} \left\{X _ {i} \in A _ {p} ^ {j} \right\} Y _ {i} - \int_ {A _ {p} ^ {j}} f ^ {*} \left(x ^ {\prime}\right) d \mathrm {P} _ {X} \left(x ^ {\prime}\right) \right| (33) \\ \leq M \sqrt {\frac {\bar {c} \cdot 2 ^ {1 - s} (4 d + 5) \log n}{n}} + \frac {2 M (4 d + 5) \log n}{3 n} + \frac {4 M}{n}. (34) \\ \end{array}
$$
Proof of Lemma C.5. Since  $\pi$  is generated by Algorithm 2, the number of public samples in each of its leaf cell  $A_{j}, j \in \mathcal{I}_{p}$  is no fewer than  $n_{l}$ , i.e.
$$
\sum_ {i = 1} ^ {n} \mathbf {1} \left\{X _ {i} ^ {p u b} \in A _ {j} \right\} \geq n _ {l}, j \in \mathcal {I} _ {p}
$$
which implies  $n_q^{-1}\sum_{i = 1}^n\mathbf{1}\{X_i^{pub}\in A_j\} \geq 2^{-s}$  Hence, by Lemma C.3, there holds
$$
\begin{array}{l} \int_ {A _ {j}} d \mathrm {Q} _ {X} (x ^ {\prime}) \geq \frac {1}{n _ {q}} \sum_ {i = 1} ^ {n} \mathbf {1} \{X _ {i} ^ {p u b} \in A _ {j} \} - \sqrt {\frac {\overline {{c}} \cdot 2 ^ {1 - s} (4 d + 5) \log n _ {q}}{n _ {q}}} - \frac {2 (4 d + 5) \log n _ {q}}{3 n _ {q}} - \frac {4}{n _ {q}} \\ \geq \frac {1}{2 ^ {s}} - \sqrt {\frac {\bar {c} \cdot 2 ^ {1 - s} (4 d + 5) \log n _ {q}}{n _ {q}}} - \frac {2 (4 d + 5) \log n _ {q}}{3 n _ {q}} - \frac {4}{n _ {q}} \\ \end{array}
$$
with probability  $\mathrm{Q}^{n_q}$  at least  $1 - 1 / n_q^2$ . By Assumption 3.3, for sufficiently large  $n$  with  $\log n \geq 12 \cdot \max(\overline{c}, 1) \cdot (4d + 5)$ , there holds
$$
\int_ {A _ {j}} d \mathrm {P} _ {X} \left(x ^ {\prime}\right) \geq \frac {1}{\tau} \int_ {A _ {j}} d \mathrm {Q} _ {X} \left(x ^ {\prime}\right) \geq \frac {1}{\tau} \left(\frac {1}{2 ^ {s}} - 3 \cdot \frac {1}{6 \cdot 2 ^ {s}}\right) = \frac {1}{2 \tau \cdot 2 ^ {s}} \tag {35}
$$
where we used condition  $n_q \geq 2^s \log^2 n$ . For the opposite direction, notice that by Assumption 3.3 and 3.2, there holds
$$
\int_ {A _ {j}} d \mathrm {P} _ {X} \left(x ^ {\prime}\right) \leq \tau \int_ {A _ {j}} d \mathrm {Q} _ {X} \left(x ^ {\prime}\right) \leq \frac {\bar {c} \cdot \tau}{2 ^ {s}}. \tag {36}
$$
(35) and (36) together yields the first conclusion. For the second conclusion, again by Lemma C.3, we have
$$
\begin{array}{l} \left| \frac {1}{n} \sum_ {i = 1} ^ {n} \mathbf {1} \{X _ {i} \in A _ {j} \} - \int_ {A _ {j}} d \mathrm {P} _ {X} \left(x ^ {\prime}\right) \right| \leq \sqrt {\frac {\bar {c} \cdot 2 ^ {1 - s} (4 d + 5) \log n}{n}} + \frac {2 (4 d + 5) \log n}{3 n} + \frac {4}{n} \\ \lesssim \frac {1}{2 ^ {s} \log n}. \tag {37} \\ \end{array}
$$
for  $j \in \mathcal{I}_p$  with probability  $\mathrm{P}^n$  at least  $1 - 1/n^2$ . Consequently, there holds
$$
\frac {1}{n} \sum_ {i = 1} ^ {n} \mathbf {1} \left\{X _ {i} \in A _ {j} \right\} \geq \int_ {A _ {j}} d \mathrm {P} _ {X} \left(x ^ {\prime}\right) - \frac {1}{2 ^ {s} \log n} \geq \frac {1}{2 \tau \cdot 2 ^ {s}} - \frac {1}{4 \tau \cdot 2 ^ {s}} = \frac {1}{4 \tau \cdot 2 ^ {s}}
$$
for sufficiently large  $n$ . Also,
$$
\frac {1}{n} \sum_ {i = 1} ^ {n} {\bf 1} \{X _ {i} \in A _ {j} \} \leq \int_ {A _ {j}} d \mathrm {P} _ {X} (x ^ {\prime}) + \frac {1}{2 ^ {s} \log n} \leq \frac {\overline {{c}} \cdot \tau}{2 ^ {s}} + \frac {\overline {{c}} \cdot \tau}{2 ^ {s}} = \frac {2 \overline {{c}} \cdot \tau}{2 ^ {s}}
$$
for large  $n$ . In conclusion, we have
$$
\frac {1}{n} \sum_ {i = 1} ^ {n} \mathbf {1} \left\{X _ {i} \in A _ {j} \right\} \asymp \frac {1}{2 ^ {s}}. \tag {38}
$$
(38) holds when the first conclusion and (37) both hold, which yields a probability  $\mathrm{Q}^{n_q} \times \mathrm{P}^n$  at least  $1 - 1 / n_q^2 - 1 / n^2$ .
Proof of Lemma C.6. According to the max-edge partition rule, when the depth of the tree  $i$  is a multiple of dimension  $d$ , each cell of the tree partition is a high-dimensional cube with a side length  $2^{-i / d}$ . On the other hand, when the depth of the tree  $s$  is not a multiple of dimension  $d$ , we consider the max-edge tree partition with depth  $\lfloor i / d \rfloor$  and  $\lceil i / d \rceil$ , whose corresponding side length of the higher dimensional cube is  $2^{-\lfloor i / d \rfloor}$  and  $2^{-\lceil i / d \rceil}$ . Note that in the splitting procedure of max-edge partition, the side length of each sub-rectangle decreases monotonically with the increase of  $s$ , so the side length of a random tree partition cell is between  $2^{-\lceil i / d \rceil}$  and  $2^{-\lfloor i / d \rfloor}$ . This implies that
$$
\sqrt {d} \cdot 2 ^ {- \lceil i / d \rceil} \leq \operatorname {d i a m} \left(A _ {i} ^ {j}\right) \leq \sqrt {d} \cdot 2 ^ {- \lfloor i / d \rfloor}
$$
Since  $i / d - 1\leq \lfloor i / d\rfloor \leq \lceil i / d\rceil \leq i / d + 1$  , we immediately get  $2^{-1}\sqrt{d}\cdot 2^{-i / d}\leq \mathrm{diam}(A_i^j)\leq$ $2\sqrt{d}\cdot 2^{-i / d}$
Proof of Lemma C.7. For binary tree partition  $\pi$ , we can grow  $\pi$  to a perfect binary tree  $\overline{\pi}$  with depth  $s$ , meaning that all interior nodes have two children and all leaf nodes have the same depth  $s$ . Then, we can compute the number of leaf nodes on  $\overline{\pi}$ . On one hand, a depth- $s$  perfect binary tree has  $2^s$  leaf nodes [17]. On the other hand, consider the perfect tree grown by  $\pi$ . Each leaf node  $A_j$ , when grown to depth  $s$ , is the root node of a depth- $(s - \mathrm{depth}(A_j))$  perfect binary tree. Thus, the node  $A_j$  with induce  $2^{s - \mathrm{depth}(A_j)}$  leaf nodes. Immediately, the total number of  $\overline{\pi}$  can be computed as  $\sum_{j \in \mathcal{I}} 2^{s - \mathrm{depth}(A_j)}$ . Matching the results of these two computations yields the desired conclusion.
C.6 Derivation of removing the range parameter
At present, we assume that  $\mathrm{P}_X = \mathrm{Q}_X$ , but the analogous conclusion holds as long as their density ratio is bounded. Let  $X$  be scaled to  $\ddot{X}$ . The excess risk can be decomposed by
$$
\begin{array}{l} \mathcal {R} _ {L, \mathrm {P}} \left(f _ {\pi} ^ {\mathrm {D P}}\right) - \mathcal {R} _ {L, \mathrm {P}} ^ {*} = \int_ {[ 0, 1 ] ^ {d}} L (x, y, f _ {\pi} ^ {\mathrm {D P}} (x)) d \mathrm {P} (x, y) - \int_ {[ 0, 1 ] ^ {d}} L (x, y, f ^ {*} (x)) d \mathrm {P} (x, y) \\ + \int_ {[ 0, 1 ] ^ {d c}} L (x, y, f _ {\pi} ^ {\mathrm {D P}} (x)) d \mathrm {P} (x, y) - \int_ {[ 0, 1 ] ^ {d c}} L (x, y, f ^ {*} (x)) d \mathrm {P} (x, y) \\ \lesssim \left(\frac {\log n}{n \varepsilon^ {2}}\right) ^ {\frac {\alpha}{\alpha + d} \wedge \frac {1}{3}} + M ^ {2} \mathbb {P} (\check {X} \notin \times_ {j = 1} ^ {d} [ 0, 1 ]). \\ \end{array}
$$
$$
\mathbb {P} \left(\check {X} \notin \times_ {j = 1} ^ {d} [ 0, 1 ]\right) = \mathbb {P} \left(X \notin \times_ {j = 1} ^ {d} \left[ \widehat {a ^ {j}}, \widehat {b ^ {j}} \right]\right) \leq \sum_ {j = 1} ^ {d} \mathbb {P} \left(X ^ {j} \notin \left[ \widehat {a ^ {j}}, \widehat {b ^ {j}} \right]\right).
$$
Denote the CDF of  $X^j$  as  $F^j$ . Since
$$
\begin{array}{l} \mathbb {P} \left(\left| F ^ {j} (\widehat {a ^ {j}}) - F ^ {j} (a ^ {j}) \right| > \frac {2 \log n _ {q}}{n _ {q}}\right) = \mathrm {P} \left(F ^ {j} (\widehat {a ^ {j}}) > \frac {2 \log n _ {q}}{n _ {q}}\right) \\ = \left(1 - \frac {2 \log n _ {q}}{n _ {q}}\right) ^ {n _ {q}} = \frac {1}{n _ {q} ^ {2}} \\ \end{array}
$$