SlowGuess/NeurIPS_2023_OCR huggingface dataset:
(a)  $\bar{l}_{\tau}$  is twicely differentiable and convex;
(b)  $\bar{l}_{\tau}$  is identical to  $\mathbb{E}_{U\sim \bar{F}}[l_{\tau}(u,U)]$  up to a constant;
(c) For sufficiently small  $h \leq \frac{r}{2L}$ ,  $\exists r_0 \geq \frac{r}{2}$ , s.t. the minimizer of  $\bar{l}_{\tau}$  is unique (denoted by  $u^{*}$ ),  $u^{*} \in [-M, M]$ , and  $\bar{l}_{\tau}$  is a strongly convex within a ball of radius  $r_0$  around its minimizer  $u^{*}$ .
To prove Lemma 5, we introduce the following Lemma 6, whose proof is postponed to Appendix J.
Lemma 6. For any distribution with cdf  $F(u) = \mathbb{P}(U \leq u)$ , the expected pinball loss with respect to  $F$  is semi-derivative with respect to  $u$ , and
$$
\partial_ {-} \mathbb {E} _ {U \sim F} [ l _ {\tau} (u, U) ] = F (u ^ {-}) - \tau ,
$$
$$
\partial_ {+} \mathbb {E} _ {U \sim F} [ l _ {\tau} (u, U) ] = F (u) - \tau .
$$
Furthermore, for those  $F(u^{-}) = F(u)$ , the derivative exists.
Proof of Lemma 5. By Lemma 6, we have
$$
\nabla_ {u} \bar {l} _ {\tau} (u) = \frac {1}{n (x)} \sum_ {k = 1} ^ {n (x)} \nabla_ {u} \mathbb {E} _ {U \sim F _ {k}} [ l _ {\tau} (u, U) ] = \frac {1}{n (x)} \sum_ {k = 1} ^ {n (x)} F _ {k} (u) - \tau .
$$
Since we assume that  $(X,U)$  follows a continuous distribution, and the conditional pdf exists, then
$$
\nabla^ {2} \bar {l} _ {\tau} (u) = \frac {1}{n (x)} \sum_ {k = 1} ^ {n (x)} p _ {k} (u),
$$
where  $p_k$  is the pdf of  $F_{k}$ $\forall k\in [n(x)]$  , which proves part (a).
For part (b), one only needs to notice that  $\bar{F}$  has the cdf of the form  $\sum_{k=1}^{n(x)} \frac{1}{n(x)} F_k$ , which implies that  $\mathbb{E}_{U \sim \bar{F}}[l_\tau(u, U)]$  has the same derivative as  $\bar{l}_\tau$ .
For part (c), by Assumption 1 (c), we have  $n(x)$  balls  $B(Q_{\tau}(U|X = X_{i_k}), \underline{r})$ $k \in [n(x)]$ , such that
$$
p _ {k} (u) \geq \underline {{p}}, \quad \forall u \in B \bigl (Q _ {\tau} (U | X = X _ {i _ {k}}), \underline {{r}} \bigr).
$$
At the same time, from the proof of Lemma 4, we know that
$$
\left| Q _ {\tau} (\bar {F}) - Q _ {\tau} \left(U | X = X _ {i _ {k}}\right) \right| \leq L h \leq \frac {1}{2} \cdot \underline {{r}},
$$
$$
\begin{array}{l} \max  _ {k} \left\{Q _ {\tau} \left(U | X = X _ {i _ {k}}\right) - \underline {{r}} \right\} + \frac {1}{2} \cdot \underline {{r}} \leq \min  _ {k} Q _ {\tau} \left(U | X = X _ {i _ {k}}\right) \\ \leq Q _ {\tau} (\bar {F}) \\ \leq \max  _ {k} Q _ {\tau} (U | X = X _ {i _ {k}}) \\ \leq \min  _ {k} \left\{Q _ {\tau} \left(U | X = X _ {i _ {k}}\right) + \underline {{r}} \right\} - \frac {1}{2} \cdot \underline {{r}}. \\ \end{array}
$$
Such an inequality implies that for at least a ball of radius  $\frac{r}{2}$ ,
$$
\nabla^ {2} \mathbb {E} _ {U \sim \bar {F}} [ l _ {\tau} (u, U) ] = \frac {1}{n (x)} \sum_ {k = 1} ^ {n (x)} p _ {k} (u) \geq \underline {{p}}, \quad \forall u \in B (Q _ {\tau} (\bar {F}), \frac {r}{2}),
$$
which proves the uniqueness of  $u^{*}$  and the local strong convexity.
Since  $\min_k Q_\tau(U|X = X_{i_k}) \leq Q_\tau(\bar{F}) \leq \max_k Q_\tau(U|X = X_{i_k})$  and  $Q_\tau(U|X = X_{i_k}) \in [-M, M]$ , we have
$$
u ^ {*} = Q _ {\tau} (\bar {F}) \in [ - M, M ].
$$
Now we go back to prove Step 1. To get the optimal convergence rate, our arguments share the same spirits of Bartlett et al. [2005], Koltchinskii [2006] of getting fast convergence rates for the M-estimators of low variance. But for simplicity, we derive our proof of uniform convergence via Bernstein's inequality and covering number arguments, which is more similar to the way of Maurer and Pontil [2009].
We are now interested in
$$
\xi_ {k} (u) := l _ {\tau} (u, U _ {i _ {k}}) - l _ {\tau} (Q _ {\tau} (\bar {F}), U _ {i _ {k}}), \quad \forall u \in [ - M, M ], U _ {i _ {k}} \sim U | X = X _ {i _ {k}}, k \in [ n (x) ].
$$
A direct computation shows that
Lemma 7.  $\forall u\in [-M,M],$  we have
$$
\left| \xi_ {k} (u) \right| \leq \max  \left\{\tau , 1 - \tau \right\} \left| u - Q _ {\tau} (\bar {F}) \right| \leq \left| u - Q _ {\tau} (\bar {F}) \right|,
$$
$$
\operatorname {V a r} (\xi_ {k} (u)) \leq \frac {1}{3} \max  \{\tau^ {2}, (1 - \tau) ^ {2} \} | u - Q _ {\tau} (\bar {F}) | ^ {2} \leq \frac {1}{3} | u - Q _ {\tau} (\bar {F}) | ^ {2}.
$$
Proof of Lemma 7. Note that the pinball loss can be expressed in an alternative way:
$$
l _ {\tau} (u, U) = \max  \{(1 - \tau) (u - U), \tau (U - u) \},
$$
$$
l _ {\tau} \left(Q _ {\tau} (\bar {F}), U\right) = \max  \left\{\left(1 - \tau\right) \left(Q _ {\tau} (\bar {F}) - U\right), \tau \left(U - Q _ {\tau} (\bar {F})\right) \right\}.
$$
From the fact that  $\left| (1 - \tau)(u - U) - (1 - \tau)(Q_{\tau}(\bar{F}) - U) \right| \leq (1 - \tau) \left| u - Q_{\tau}(\bar{F}) \right|$
and that  $\left|\tau(U - u) - \tau\big(U - Q_{\tau}(\bar{F})\big)\right| \leq \tau|u - Q_{\tau}(\bar{F})|$ , we prove the first statement via Lemma 15.
For the second claim, one just needs to notice that any almost surely bounded by  $[a,b]$  random variables have at most a variance of  $\frac{1}{12} (b - a)^2$ .
Combining Lemma 1 and Lemma 7, we directly prove the following
Lemma 8 (Pointwise Convergence).  $\forall \delta >0,n(x) > 0,u\in [-M,M]$ , with probability at least  $1 - \delta$  the following holds
$$
\begin{array}{l} \left| \frac {1}{n (x)} \sum_ {k = 1} ^ {n (x)} \mathbb {E} [ \xi_ {k} (u) ] - \frac {1}{n (x)} \sum_ {k = 1} ^ {n (x)} \xi_ {k} (u) \right| \leq \frac {2}{3 n (x)} | u - Q _ {\tau} (\bar {F}) | \log \left(\frac {1}{\delta}\right) \\ + \sqrt {\frac {2}{3 n (x)} \left| u - Q _ {\tau} (\bar {F}) \right| ^ {2} \log \left(\frac {1}{\delta}\right)}. \\ \end{array}
$$
Proof. Direct corollary from Lemma 1 and Lemma 7.
To reach a uniform convergence guarantee from pointwise convergence, we utilize the covering number arguments.
Definition 4 ( $\epsilon$ -Covering Numbers).  $\forall \epsilon > 0$ , a function class  $\mathcal{F}$  is said to have an  $\epsilon$ -covering number of
$$
N \left(\epsilon , \mathcal {F}, \| \cdot \| _ {\infty}\right) := \inf  \left\{\left| \mathcal {F} _ {0} \right|: \mathcal {F} _ {0} \subset \mathcal {F}, \forall f \in \mathcal {F}, \exists f _ {0} \in \mathcal {F} _ {0}, \text {s . t .} \| f - f _ {0} \| _ {\infty} \leq \epsilon \right\}.
$$
We compute the function class formed up by the functions  $l_{\tau}(u,\cdot) - l_{\tau}(Q_{\tau}(\bar{F}),\cdot)$ .
Lemma 9 (Covering Number). Assume Assumption 1 (b) holds with parameter  $M$ . Define  $\mathcal{L}_{[-M,M]} := \{l_{\tau}(u,\cdot) - l_{\tau}(Q_{\tau}(\bar{F}),\cdot): u \in [-M,M]\}$ . Then  $\forall \epsilon > 0$ ,
$$
N \left(\epsilon , \mathcal {L} _ {[ - M, M ]}, \| \cdot \| _ {\infty}\right) \leq \frac {\operatorname {d i a m} ([ - M , M ])}{\epsilon} = \frac {2 M}{\epsilon}.
$$
Proof of Lemma 9. By a similar argument to that in Lemma 7, we have
$$
\left| \left(l _ {\tau} (u _ {1}, \cdot) - l _ {\tau} (Q _ {\tau} (\bar {F}), \cdot)\right) - \left(l _ {\tau} (u _ {2}, \cdot) - l _ {\tau} (Q _ {\tau} (\bar {F}), \cdot)\right) \right| \leq | u _ {1} - u _ {2} |,
$$
where the left-hand-side is the function infinity form by itself.
Combining the above fact with another that
$$
|u_{1} - u_{2}|\leq \sup_{u_{1}^{\prime},u_{2}^{\prime}\in [-M,M]}|u_{1}^{\prime} - u_{2}^{\prime}| = \operatorname {diam}([-M,M]),
$$
we complete the proof.
Since there will be many coefficients dependence, we abbreviate any constant  $C$  that depends polynomially on some other factors  $(c_{1},\dots ,c_{q})$  by  $C(c_{1},\dots ,c_{q})$ . Now we provide the uniform convergence result by combining the pointwise convergence and the covering number.
Lemma 10 (Uniform Convergence).  $\forall \epsilon, \delta > 0$ , assume  $n(x)$  is sufficiently large such that  $n(x) \geq \frac{2}{3} \log \left( \frac{2M}{\epsilon \delta} \right)$ , we have with probability at least  $1 - \delta$ ,
$$
\begin{array}{l} \left| \frac {1}{n (x)} \sum_ {k = 1} ^ {n (x)} \mathbb {E} [ \xi_ {k} (u) ] - \frac {1}{n (x)} \sum_ {k = 1} ^ {n (x)} \xi_ {k} (u) \right| \leq 4 \epsilon + \frac {2}{3 n (x)} | u - Q _ {\tau} (\bar {F}) | \log \left(\frac {2 M}{\epsilon \delta}\right) \\ + \sqrt {\frac {2}{3 n (x)} \left| u - Q _ {\tau} (\bar {F}) \right| ^ {2} \log \left(\frac {2 M}{\epsilon \delta}\right)} \\ \end{array}
$$
holds for  $\forall u\in [-M,M]$
Specifically, if we select  $\epsilon = \frac{1}{n(x)}$ ,  $\delta = \frac{1}{n(x)M}$ ,
we have  $\forall n(x) \geq C_1(\log(M)) \coloneqq 6\log(\max\{2M, 10\})$ ,
$$
\begin{array}{l} \left| \frac {1}{n (x)} \sum_ {k = 1} ^ {n (x)} \mathbb {E} [ \xi_ {k} (u) ] - \frac {1}{n (x)} \sum_ {k = 1} ^ {n (x)} \xi_ {k} (u) \right| \leq \frac {1}{n (x)} \left(4 + \frac {8 M}{3} \log (2 n (x) M)\right) \\ + \sqrt {\frac {4}{3 n (x)} \left| u - Q _ {\tau} (\bar {F}) \right| ^ {2} \log (2 n (x) M)} \\ \end{array}
$$
Proof of Lemma 10.  $\forall \epsilon > 0$ , we can select an  $\epsilon$ -cover of the function class  $\mathcal{L}_{[-M,M]}$ , denoted by
$$
\mathcal {S} = \left\{L [ u _ {j} ] = l _ {\tau} (u _ {j}, \cdot) - l _ {\tau} (Q _ {\tau} (\bar {F}), \cdot) \right\} _ {j = 1} ^ {N (\epsilon , \mathcal {L} _ {[ - M, M ]}, \| \cdot \| _ {\infty})}.
$$
$$
\xi_ {k} (u) = l _ {\tau} \left(u, U _ {i _ {k}}\right) - l _ {\tau} \left(Q _ {\tau} (\bar {F}), U _ {i _ {k}}\right) = L [ u ] \left(U _ {i _ {k}}\right), \quad \forall u \in [ - M, M ],
$$
which means that  $\exists u_{j}\in S$  , s.t.
$$
\left| \xi_ {k} (u) - \xi_ {k} (u _ {j}) \right| = \left| L [ u ] \left(U _ {i _ {k}}\right) - L [ u _ {j} ] \left(U _ {i _ {k}}\right) \right| \leq \| L [ u ] - L [ u _ {j} ] \| _ {\infty} \leq \epsilon .
$$
Now  $\forall \delta > 0$ , we apply Lemma 8 on  $\{\xi_k(u_j)\}_{k=1}^m, u_j \in S$  with probability guarantee of at least  $1 - \frac{\delta}{N(\epsilon, \mathcal{L}_{[-M,M]}, \| \cdot \|_{\infty})}$ , we have  $\forall u_j \in S$ ,
$$
\begin{array}{l} \left| \frac {1}{m} \sum_ {k = 1} ^ {m} \mathbb {E} [ \xi_ {k} (u _ {j}) ] - \frac {1}{m} \sum_ {k = 1} ^ {m} \xi_ {k} (u _ {j}) \right| \leq \frac {2}{3 m} | u _ {j} - Q _ {\tau} (\bar {F}) | \log \left(\frac {N (\epsilon , \mathcal {L} _ {[ - M , M ]} , \| \cdot \| _ {\infty})}{\delta}\right) \\ + \sqrt {\frac {2}{3 m} \left| u _ {j} - Q _ {\tau} (\bar {F}) \right| ^ {2} \log \left(\frac {N (\epsilon , \mathcal {L} _ {[ - M , M ]} , \| \cdot \| _ {\infty})}{\delta}\right)}. \\ \end{array}
$$
Combining above inequality with the fact that  $|\xi_k(u) - \xi_k(u_j)| \leq \epsilon$  and the way we select the cover in Lemma 9 such that  $|u - u_j| \leq \epsilon$ , we complete the proof.
To verify the specific conclusion, one just needs to notice that our choice of  $n(x)$  ensures that  $n(x) \geq \frac{4}{3} \log (2n(x)M)$  for  $\epsilon = \frac{1}{n(x)}, \delta = \frac{1}{n(x)M}$ .
Lemma 11 (Generalization Bound for  $\hat{Q}_{\tau}$ ). If we take  $\hat{Q}_{\tau}(x) = \arg \min_{u} \sum_{k=1}^{n(x)} \frac{1}{n(x)} l_{\tau}(u, U_{i_k})$  as we do in Algorithm 1, then for sufficiently large  $m \geq C_2(\underline{p}^{-1}, \underline{r}^{-1}, M)$ , we have with probability at least  $1 - \frac{1}{n(x)M}$ ,
$$
0 \leq \bar {l} _ {\tau} (\hat {Q} _ {\tau} (x)) - \bar {l} _ {\tau} (Q _ {\tau} (\bar {F})) \leq \frac {C _ {3} (\underline {{p}} ^ {- 1} , M , \log (n (x) M))}{n (x)} = \tilde {O} \left(\frac {1}{n (x)}\right),
$$
where  $C_3 \coloneqq 8 + \frac{8M}{3} \log (n(x)M) + \frac{16 \log (n(x)M)}{3\underline{p}}$ .
More specifically, with probability at least  $1 - \frac{1}{n(x)M}$ ,
$$
\left| \hat {Q} _ {\tau} (x) - Q _ {\tau} (\bar {F}) \right| \leq \frac {C _ {4} (\underline {{p}} ^ {- 1} , M , \log (n (x) M))}{\sqrt {n (x)}} = \tilde {O} \left(\frac {1}{\sqrt {n (x)}}\right),
$$
where  $C_4 \coloneqq \sqrt{C_3 \cdot \frac{2}{p}}$ .
Proof of Lemma 11. We first claim that for sufficiently large  $n(x) \geq C_2$ , the empirical minimizer  $\hat{Q}_{\tau}(x)$  must fall into the neighborhood  $B(Q_{\tau}(\hat{F}), \frac{r}{2})$  stated in Lemma 5 (c). In fact, one just needs to verify that
$$
\frac {1}{n (x)} \left(4 + \frac {8 M}{3} \log (2 n (x) M)\right) + \frac {1}{\sqrt {n (x)}} \cdot \sqrt {\frac {1 6}{3} M ^ {2} \log (2 n (x) M)} \leq \frac {\underline {{p}} \cdot \underline {{r}} ^ {2}}{8},
$$
which will definitely hold for sufficiently large  $n(x)\geq C_2(\underline{p}^{-1},\underline{r}^{-1},M)$
From the fact that minimizing  $\sum_{k=1}^{n(x)} \frac{1}{n(x)} l_{\tau}(u, U_{i_k})$  is equivalent to minimizing it minus some constant, we know that
$$
\sum_ {k = 1} ^ {n (x)} \frac {1}{n (x)} \xi_ {k} (\hat {Q} _ {\tau} (x)) \leq 0.
$$
By Lemma 5 and Lemma 10, we have
$$
\begin{array}{l} 0 \leq \bar {l} _ {\tau} (\hat {Q} _ {\tau} (x)) - \bar {l} _ {\tau} (Q _ {\tau} (\bar {F})) \\ = \frac {1}{n (x)} \sum_ {k = 1} ^ {n (x)} \mathbb {E} \left[ \xi_ {k} \left(\hat {Q} _ {\tau} (x) \right. \right] \\ \leq \frac {1}{n (x)} \left(4 + \frac {8 M}{3} \log (2 n (x) M)\right) + \sqrt {\frac {1 6}{3 n (x)} | \hat {Q} _ {\tau} (x) - Q _ {\tau} (\bar {F}) | ^ {2} \log (2 n (x) M)} \\ \leq \frac {1}{n (x)} \left(4 + \frac {8 M}{3} \log (2 n (x) M)\right) + \sqrt {\frac {1 6}{3 n (x)} \cdot \frac {2}{\underline {{p}}} \left(\bar {l} _ {\tau} (\hat {Q} _ {\tau} (x)) - \bar {l} _ {\tau} (Q _ {\tau} (\bar {F}))\right) \log (2 n (x) M)}, \\ \end{array}
$$
where the first inequality follows from Lemma 5 (a), the second inequality from Lemma 10, and the last from Lemma 5 (c).
Solve the quadratic inequality
$$
w ^ {2} \leq \frac {\alpha}{n (x)} + \sqrt {\frac {\beta}{n (x)}} w,
$$
where  $\alpha = 4 + \frac{8M}{3}\log (2n(x)M),\beta = \frac{32\log(2n(x)M)}{3\underline{p}}$  we have
$$
w \leq \frac {\sqrt {\frac {\beta}{n (x)}} + \sqrt {\frac {\beta}{n (x)} + \frac {4 \alpha}{n (x)}}}{2} \leq \frac {1}{\sqrt {n (x)}} \left(\sqrt {\beta} + \sqrt {\alpha}\right),
$$
where  $w$  is exactly  $\sqrt{\bar{l}_{\tau}(\hat{Q}_{\tau}(x)) - \bar{l}_{\tau}(Q_{\tau}(\bar{F}))}$
$$
\bar {l} _ {\tau} (\hat {Q} _ {\tau} (x)) - \bar {l} _ {\tau} (Q _ {\tau} (\bar {F})) \leq \frac {1}{n (x)} (2 \beta + 2 \alpha) = \frac {1}{n (x)} \left(8 + \frac {1 6 M}{3} \log (2 n (x) M) + \frac {6 4 \log (2 n (x) M)}{3 p}\right).
$$
The latter conclusion follows from the local strong convexity of  $\bar{l}_{\tau}$  again, as is shown in Lemma 5 (c).
Lemma 11 proves our Step 1, finally. We can combine Step 1 and Step 2 now to get an upper bound that is critical for the following analysis:
Lemma 12 (Bounding  $\left|\hat{Q}_{\tau}(x) - Q_{\tau}(U|X = x)\right|$ ). Suppose Assumption 1 holds. For sufficiently small  $h \leq \frac{r}{2L}$ , we have
$$
\begin{array}{l} \left| \hat {Q} _ {\tau} (x) - Q _ {\tau} (U | X = x) \right| \leq L h + \mathbb {1} \{n (x) <   C _ {2} \} M \\ + \mathbb {1} \{n (x) \geq C _ {2} \} \left(M \mathbb {1} \{\text {B o u n d s i n L e m m a 1 1 f a i l} \} + \frac {C _ {4}}{\sqrt {n (x)}}\right), \\ \end{array}
$$
where  $C_2 = C_2(\underline{p}^{-1}, \underline{r}^{-1}, M)$ ,  $C_4 = C_4(\underline{p}^{-1}, M, \log(n(x)M))$  in Lemma 11, and the failing probability in Lemma 11 is no larger than  $\frac{1}{n(x)M}$  provided  $n(x) \geq C_2$ .
Proof of Lemma 12. A direct corollary from Lemma 4 and Lemma 11.
Proof of the pointwise consistency part of Theorem 1. Since the  $\tilde{O}\left(\frac{1}{\sqrt{n(x)}}\right)$  term and the failing probability term in Lemma 12 tend to zero as  $n(x)\to \infty$ , we only need to verify that we can select  $h$  such that as  $n\to \infty$ , we have
(a)  $h\to 0$
(b)  $\forall C > 0, \mathbb{P}(n(x) \geq C) \to 1$ .
We will show that both conditions are true for any  $x_0$  where the pdf  $p(x)$  is positive and continuous at  $x = x_0$ , if we select  $h = C_5 n^{-\frac{1}{d + 2}}$ .
The first argument automatically holds as  $n\to \infty$  . We turn to prove the second one.
Since  $p(x)$  is continuous at  $p(x_0)$ , we can find sufficiently large  $n \geq N_0$ , such that
$$
p (x) \geq \frac {p \left(x _ {0}\right)}{2}, \quad \forall x \in B \left(x _ {0}, h\right).
$$
Then the probability of  $\mathbb{1}\{X_i\in B(x_0,h)\}$  can be lower bounded by
$$
\mathbb {P} (X _ {i} \in B (x _ {0}, h)) \geq \frac {p (x _ {0})}{2} V h ^ {d} = C _ {6} n ^ {- \frac {d}{d + 2}},
$$
where  $V$  is the volume of the unit ball in  $\mathcal{X} \subset \mathbb{R}^d$ ,  $C_6 \coloneqq \frac{p(x_0)}{2} VC_5$ .
$\forall C > 0, j \in \mathbb{N}_{+}$ , we can define a sequence of parameters  $\{\lambda_j\}_{j=1}^{\infty}$ , such that for any  $\eta_j \sim \mathrm{Poisson}(\lambda_j)$
$$
\mathbb {P} (\eta_ {j} \geq C) \geq 1 - \frac {1}{j}, \quad j = 1, 2, \ldots ,
$$
where we assume without loss of generality that  $\lambda_{j}$  is non-decreasing.
For each  $j$ , we can define an auxiliary sequence of independent random variables  $\{\nu_{l,q,j}\}$  for  $l = [q], q = 1,2,\ldots ,j = 1,2,\ldots$ , such that each random variable  $\nu_{l,q,j} \sim \mathrm{Bernoulli}\left(\frac{\lambda_j}{q}\right)$ .
By Lemma 3, we can find a sequence of  $\{q_j\}_{j=1}^{\infty}$ , such that
$$
\mathbb {P} \left(\sum_ {l = 1} ^ {q} \nu_ {l, q, j} \geq C\right) \geq \mathbb {P} (\eta_ {j} \geq C) - \frac {1}{j} \geq 1 - \frac {2}{j}, \quad \forall q \geq q _ {j}, j = 1, 2, \ldots ,
$$
where we select  $q_{j}$  without loss of generality that  $\frac{\lambda_j}{q_j}$  is non-increasing.
As  $\{\mathbb{1}\{X_i\in B(x_0,h)\} \}_{i = 1}^n$  is also an independent Bernoulli random variable sequence with parameter  $C_6n^{-\frac{d}{d + 2}}$ , we can select a non-decreasing sequence of  $\{N_j\}_{j = 1}^{\infty}$ , such that  $\forall n\geq N_{j}$
$$
C _ {6} n ^ {- \frac {d}{d + 2}} \geq \frac {\lambda_ {j}}{n}, \quad n \geq q _ {j}.
$$
Then  $\forall n\geq N_j$  , we have
$$
\mathbb {P} \left(n (x _ {0}) = \sum_ {i = 1} ^ {n} \mathbb {1} \{X _ {i} \in B (x _ {0}, h) \} \geq C\right) \geq \mathbb {P} \left(\sum_ {i = 1} ^ {n} \nu_ {i, n, j} \geq C\right) \geq 1 - \frac {2}{j},
$$
where the first inequality follows from the result of stochastically dominant of  $\mathbb{1}\{X_i\in B(x_0,h)\}$  over  $\nu_{l,n,j}$ .
$\forall \epsilon > 0$ , we can select sufficiently large  $j \geq \frac{2}{\epsilon}$ , then  $\forall n \geq \max \{N_0, N_j\}$ ,
$$
\mathbb {P} (n (x _ {0}) \geq C) \geq 1 - \epsilon ,
$$
To prove the mean squared error part of Theorem 1, we state a lemma on the Binomial random variable that will be proved later.
Lemma 13. For any  $\zeta \sim \mathrm{Binomial}(n,p)$ , we have
(a)  $\mathbb{E}\left[\frac{1}{\zeta}\mathbb{1}\{\zeta >0\}\right]\leq \frac{2}{(n + 1)p};$
(b)  $\forall r\in \mathbb{N},r < np$  we have  $\mathbb{P}(\zeta \leq r)\leq \frac{(n - r)p}{(np - r)^2}$
Now we return to the proof of the mean squared error part of Theorem 1.
Proof of the mean squared error part of Theorem 1. By direct computation from Lemma 12, we have
$$
\begin{array}{l} \mathbb {E} \left[ | \hat {Q} _ {\tau} (X _ {\mathrm {t e s t}}) - Q _ {\tau} (U | X = X _ {\mathrm {t e s t}}) | ^ {2} \right] \\ = \mathbb {E} _ {X _ {\text {t e s t}}} \left[ \mathbb {E} _ {X _ {1: n}} \left[ \mathbb {E} _ {U _ {1: n}} \left[ | \hat {Q} _ {\tau} (X _ {\text {t e s t}}) - Q _ {\tau} (U | X = X _ {\text {t e s t}}) | ^ {2} \right] \right] \right] \\ \leq \mathbb {E} _ {X _ {\text {t e s t}}} \left[ \mathbb {E} _ {X _ {1: n}} \left[ 4 L ^ {2} h ^ {2} \right. \right. \\ + \left[ \frac {4 C _ {4} (\underline {{p}} ^ {- 1} , \underline {{M}} , \log (n (X _ {\text {t e s t}}) M)) ^ {2}}{n (X _ {\text {t e s t}})} + 4 M ^ {2} \left(\frac {1}{n (X _ {\text {t e s t}}) M}\right) ^ {2} \right] \mathbb {1} \left\{n \left(X _ {\text {t e s t}}\right) \geq C _ {2} \right\} \\ \left. \left. + 4 M ^ {2} \mathbb {1} \left\{n \left(X _ {\text {t e s t}}\right) <   C _ {2} \right\} \right] \right] \\ \leq 4 L ^ {2} h ^ {2} + \mathbb {E} _ {X _ {\text {t e s t}}} \left[ \mathbb {E} _ {X _ {1: n}} \left[ \frac {C _ {4} ^ {\prime} (\underline {{p}} ^ {- 1} , \underline {{M}} , \log (n M))}{n (X _ {\text {t e s t}})} \mathbb {1} \{n (X _ {\text {t e s t}}) \geq C _ {2} \} + 4 M ^ {2} \mathbb {1} \{n (X _ {\text {t e s t}}) <   C _ {2} \} \right] \right], \\ \end{array}
$$
where the first inequality comes from the fact that  $(a + b + c + d)^2 \leq 4(a^2 + b^2 + c^2 + d^2)$ , and the second from that  $n(X_{\mathrm{test}}) \leq n$ . Note that we replace the original  $C_4$  that depends on the logarithmic term of  $n(X_{\mathrm{test}})$  with a new  $C_4'$  that does not depend logarithmically on  $n(X_{\mathrm{test}})$  but logarithmically on the entire  $n$ . From the definition of  $C_4$  in Lemma 11, we recall that the dependence on the logarithmic term is polynomial.
By Lemma 13 (a), we have
$$
\begin{array}{l} \mathbb {E} _ {X _ {1: n}} \left[ \frac {1}{n (X _ {\text {t e s t}})} \mathbb {1} \{n (X _ {\text {t e s t}} \geq C _ {2}) \} \right] \leq \mathbb {E} _ {X _ {1: n}} \left[ \frac {1}{n (X _ {\text {t e s t}})} \mathbb {1} \{n (X _ {\text {t e s t}} > 0) \} \right] \\ \leq \frac {2}{(n + 1) \mathbb {P} (X _ {i} \in B (X _ {\mathrm {t e s t}} , h))}. \\ \end{array}
$$
We try to bound  $\mathbb{E}_X\left[\frac{1}{\mathbb{P}(X_i\in B(X,h))}\right]$ .
Since  $\mathcal{X} \subset [0,1]^d$ , we can generate an  $\frac{h}{2}$ -cover of  $\mathcal{X}$ , denoted by  $\mathcal{S} = \{x_j\}_{j=1}^{C_d}$ , where
$$
C _ {d} \leq \frac {C}{h ^ {d}},
$$
where  $C$  is some intrinsic constant with respect to the space  $\mathbb{R}^d$  equipped with metric  $\| \cdot \|$ .
$$
\begin{array}{l} \mathbb {E} _ {X} \left[ \frac {1}{\mathbb {P} (X _ {i} \in B (X , h))} \right] = \int_ {\mathcal {X}} \frac {1}{P (B (x , h))} \mathrm {d} P \\ \leq \sum_ {j = 1} ^ {C _ {d}} \int_ {\mathcal {X}} \frac {\mathbb {1} \{x \in B (x _ {j} , \frac {h}{2}) \}}{P (B (x , h))} d P \\ \leq \sum_ {j = 1} ^ {C _ {d}} \int_ {\mathcal {X}} \frac {\mathbb {1} \{x \in B (x _ {j} , \frac {h}{2}) \}}{P (B (x _ {j} , \frac {h}{2}))} d P \\ = C _ {d} \leq \frac {C}{h ^ {d}}, \\ \end{array}
$$
where  $P$  is the probability measure with respect to  $X$ , and the first inequality follows from that  $\mathcal{S}$  is a  $\frac{h}{2}$ -cover of  $\mathcal{X}$ , the second from the fact that  $B(x_{j},\frac{h}{2})\subset B(x,h)$  for those  $\| x - x_j\| \leq \frac{h}{2}$ , and the inequality from the fact that  $\int \mathbb{1}\{x\in B(x_j,\frac{h}{2})\} \mathrm{d}P = P(B(x_j,\frac{h}{2}))$ .
We now try to bound  $\mathbb{E}_{X_{\mathrm{test}}}\left[\mathbb{E}_{X_{1:n}}\left[\mathbb{1}\{n(X_{\mathrm{test}}) < C_2\}\right]\right]$ .
If  $nP(B(x, h)) \geq 2\lfloor C_2 \rfloor$ , then by Lemma 13 (b), we have
$$
\mathbb {P} (n (x) <   C _ {2}) \leq \frac {n \lfloor C _ {2} \rfloor}{(n P (B (x , h)) - \lfloor C _ {2} \rfloor) ^ {2}} \leq \frac {4}{n P (B (x , h))}.
$$
If  $nP(B(x,h)) < 2\lfloor C_2 \rfloor$ , then for sufficiently large  $n \geq 4\lfloor C_2 \rfloor - 1$ , by direct calculation we have,
$$
\frac {P (B (x , h))}{1 - P (B (x , h))} \leq 2 \lfloor C _ {2} \rfloor \cdot \frac {2}{n - 1} \leq \frac {4}{3} \lfloor C _ {2} \rfloor \cdot \frac {3}{n - 2} \leq \lfloor C _ {2} \rfloor \cdot \frac {4}{n - 3} \leq \dots \leq \lfloor C _ {2} \rfloor \cdot \frac {\lfloor C _ {2} \rfloor}{n + 1 - \lfloor C _ {2} \rfloor},
$$
$$
\left. \right. \mathrm {C} _ {n} ^ {l} \left(1 - P (B (x, h))\right) ^ {n - l} \left(P (B (x, h))\right) ^ {l} \leq 2 \cdot \frac {4}{3} \cdot \left\lfloor C _ {2} \right\rfloor \cdot \mathrm {C} _ {n} ^ {0} \left(1 - P (B (x, h))\right) ^ {n}, \quad \forall l \in [ \lfloor C _ {2} \rfloor ],
$$
where  $\mathrm{C}_n^l$  is the number of combinations of  $(n,l)$ . Hence
$$
\begin{array}{l} \mathbb {P} (n (x) <   C _ {2}) \\ \leq \mathbb {P} (n (x) \leq \left\lfloor C _ {2} \right\rfloor) \\ = \sum_ {l = 0} ^ {\lfloor C _ {2} \rfloor} C _ {n} ^ {l} \left(1 - P (B (x, h))\right) ^ {n - l} \left(P (B (x, h))\right) ^ {l} \\ \leq \frac {8 (C _ {2}) ^ {2}}{3} \left(1 - P (B (x, h))\right) ^ {n} \\ \leq \frac {8 (C _ {2}) ^ {2}}{3} \exp (- n P (B (x, h))). \\ \end{array}
$$
Since  $\exp (-a)\leq \frac{1}{a}\cdot \max_{b}\{b\exp (-b)\} = \frac{1}{ea}$ , we have
$$
\mathbb {P} (n (x) <   C _ {2}) \leq \frac {C _ {7}}{n P (B (x , h))},
$$
where  $C_7 = \frac{8(C_2)^2}{3e}$
Therefore, we can conclude that (w.l.o.g. we assume  $C_7\geq 4$
$$
\mathbb {P} (n (x) <   C _ {2}) \leq \frac {\max  \{C _ {7} , 4 \}}{n P (B (x , h))} = \frac {C _ {7}}{n P (B (x , h))}.
$$
Hence, we can finally give the bound by