Combining this with (40) proves that

$$ (\overline{F} \circ m_1^{-1})^{-1}(t) = (\overline{F} \circ m_2^{-1})^{-1}(t) $$

for all $t \in (0, 1)$. Using the equivalence in (42) (that holds for all $t \in (0, 1]$ and $y \in \mathbb{R}$), we also have

$$ \begin{aligned} (\overline{F} \circ m_j^{-1})^{-1}(t) &= \sup\{y \in \mathbb{R}, m_j \circ \overline{F}^{-1}(t) \ge y\} \\ &= m_j \circ \overline{F}^{-1}(t). \end{aligned} $$

Hence,

$$ m_1 \circ \overline{F}^{-1}(t) = m_2 \circ \overline{F}^{-1}(t) $$

for all $t \in (0, 1)$. This in turn implies that $m_1 = m_2$ on the support of $X$ since the range of $\overline{F}^{-1}$ is the support of $X$. $\square$

**Proof of Proposition 2.** *Proof of Claim 1.* Recall that any element $m \in \mathcal{M}$ is bounded, and hence there exists $K > 0$ such that $\|m\|_{\infty} \le K$. Denote by $Y_{(1)} \le \dots \le Y_{(n)}$ the order statistics corresponding to $Y_1, \dots, Y_n$. We have for all $y < Y_{(1)}$ that $\mathbb{H}_n(y) = 0$. Moreover, it follows from monotonicity of $\Phi_{\epsilon}$ and $m$ that $0 \le \Phi_{\epsilon}(y - m(X_i)) \le \Phi_{\epsilon}(y - m(X_{(1)})) \le 1$ for all $i \in \{1, \dots, n\}$ and therefore,

$$ \begin{aligned} \left\{ \mathbb{H}_n(y) - n^{-1} \sum_{i=1}^n \Phi_{\epsilon}(y - m(X_i)) \right\}^2 &= n^{-2} \left( \sum_{i=1}^n \Phi_{\epsilon}(y - m(X_i)) \right)^2 \\ &\le \Phi_{\epsilon}(y - m(X_{(1)}))^2 \\ &\le \Phi_{\epsilon}(y - m(X_{(1)})). \end{aligned} $$

Now, existence of expectation of $\epsilon$ implies that

$$ \int_{-\infty}^{c} \Phi_{\epsilon}(t) dt < \infty, \quad \text{and} \quad \int_{c}^{\infty} (1 - \Phi_{\epsilon}(t)) dt < \infty \qquad (43) $$

for arbitrary $c \in \mathbb{R}$ and therefore,

$$ \begin{aligned} \int_{-\infty}^{Y_{(1)}} \left\{ \mathbb{H}_n(y) - n^{-1} \sum_{i=1}^{n} \Phi_{\epsilon}(y - m(X_i)) \right\}^2 dy &\le \int_{-\infty}^{Y_{(1)}} \Phi_{\epsilon}(y - m(X_{(1)})) dy \\ &= \int_{-\infty}^{Y_{(1)} - m(X_{(1)})} \Phi_{\epsilon}(y) dy \\ &< \infty. \end{aligned} \qquad (44) $$

Similarly, $\mathbb{H}_n(y) = 1$ for $y > Y_{(n)}$ and hence

$$ \begin{aligned} \left\{ \mathbb{H}_n(y) - n^{-1} \sum_{i=1}^{n} \Phi_{\epsilon}(y - m(X_i)) \right\}^2 &= \left(1 - n^{-1} \sum_{i=1}^{n} \Phi_{\epsilon}(y - m(X_i))\right)^2 \\ &\le \left(1 - \Phi_{\epsilon}(y - m(X_{(n)}))\right)^2 \\ &\le 1 - \Phi_{\epsilon}(y - m(X_{(n)})). \end{aligned} $$
Combined with (43), this proves that

$$ \int_{Y_{(n)}}^{\infty} \left\{ \mathbb{H}_n(y) - n^{-1} \sum_{i=1}^{n} \Phi_{\epsilon}(y - m(X_i)) \right\}^2 dy < \infty. \quad (45) $$

Since the integrand is bounded, (44) and (45) yield

$$ \int_{\mathbb{R}} \left\{ \mathbb{H}_n(y) - n^{-1} \sum_{i=1}^{n} \Phi_{\epsilon}(y - m(X_i)) \right\}^2 dy < \infty, $$

which proves that $M_n(m)$ is finite.

*Proof of Claim 2.* Assume that Assumption A1 holds and consider $m \in \mathcal{M}$. Then, we have that

$$ \begin{aligned} \mathbb{M}(m) &= \int_{\mathbb{R}} \left\{ \int_{\mathbb{R}} \left( \Phi_{\epsilon}(y - m_0(x)) - \Phi_{\epsilon}(y - m(x)) \right) dF_0(x) \right\}^2 dy \\ &= \int_{[0, \infty)} \left\{ \int_{\mathbb{R}} \left( 1 - \Phi_{\epsilon}(y - m(x)) - (1 - \Phi_{\epsilon}(y - m_0(x))) \right) dF_0(x) \right\}^2 dy \end{aligned} \quad (46) $$

$$ + \int_{(-\infty,0]} \left\{ \int_{\mathbb{R}} \left( \Phi_{\epsilon}(y - m_0(x)) - \Phi_{\epsilon}(y - m(x)) \right) dF_0(x) \right\}^2 dy. \quad (47) $$

We further bound above the integral in (46) by

$$ \begin{aligned} &\le 2 \int_{[0,\infty)} \left\{ \int_{\mathbb{R}} (1 - \Phi_{\epsilon}(y - m(x))) dF_0(x) \right\}^2 dy \\ &+ 2 \int_{[0,\infty)} \left\{ \int_{\mathbb{R}} (1 - \Phi_{\epsilon}(y - m_0(x))) dF_0(x) \right\}^2 dy \\ &\le 4 \int_{[0,\infty)} (1 - \Phi_{\epsilon}(y - \max(K_0, K)))^2 dy \end{aligned} \quad (48) $$

where we recall that $K \ge \|m\|_{\infty}$. The latter integral is finite using again (43). Similarly, we argue that the integral in (47) can be also bounded above by

$$ 4 \int_{(-\infty,0]} \left( \Phi_{\epsilon}(y + \max(K_0, K)) \right)^2 dy < \infty. $$

This completes the proof that $M(m)$ is finite.

*Proof of Claim 3.* Using again that $\mathbb{H}_n(y) = 0$ for all $y < Y_{(1)}$, together with monotonicity of $\Phi_{\epsilon}$ and $m$, we have that

$$ \begin{aligned} \mathbb{M}_n(m) &\ge \int_{-\infty}^{Y_{(1)}} n^{-2} \left( \sum_{i=1}^{n} \Phi_{\epsilon}(y - m(X_i)) \right)^2 dy \\ &\ge n^{-2} \int_{-\infty}^{Y_{(1)}} \Phi_{\epsilon}(y - m(X_{(1)}))^2 dy \\ &= n^{-2} \int_{-\infty}^{Y_{(1)} - m(X_{(1)})} \Phi_{\epsilon}(t)^2 dt \\ &\to \infty, \text{ if } m(X_{(1)}) \to -\infty. \end{aligned} $$
Combined with (43), this proves that

$$ \int_{Y_{(n)}}^{\infty} \left\{ \mathbb{H}_n(y) - n^{-1} \sum_{i=1}^{n} \Phi_{\epsilon}(y - m(X_i)) \right\}^2 dy < \infty. \quad (45) $$

Since the integrand is bounded, (44) and (45) yield

$$ \int_{\mathbb{R}} \left\{ \mathbb{H}_n(y) - n^{-1} \sum_{i=1}^{n} \Phi_{\epsilon}(y - m(X_i)) \right\}^2 dy < \infty, $$

which proves that $M_n(m)$ is finite.

*Proof of Claim 2.* Assume that Assumption A1 holds and consider $m \in \mathcal{M}$. Then, we have that

$$ \begin{aligned} \mathbb{M}(m) &= \int_{\mathbb{R}} \left\{ \int_{\mathbb{R}} \left( \Phi_{\epsilon}(y - m_0(x)) - \Phi_{\epsilon}(y - m(x)) \right) dF_0(x) \right\}^2 dy \\ &= \int_{[0, \infty)} \left\{ \int_{\mathbb{R}} \left( 1 - \Phi_{\epsilon}(y - m(x)) - (1 - \Phi_{\epsilon}(y - m_0(x))) \right) dF_0(x) \right\}^2 dy \end{aligned} \quad (46) $$

$$ + \int_{(-\infty, 0]} \left\{ \int_{\mathbb{R}} \left( \Phi_{\epsilon}(y - m_0(x)) - \Phi_{\epsilon}(y - m(x)) \right) dF_0(x) \right\}^2 dy. \quad (47) $$

We further bound above the integral in (46) by

$$ \begin{aligned} &\le 2 \int_{[0, \infty)} \left\{ \int_{\mathbb{R}} (1 - \Phi_{\epsilon}(y - m(x))) dF_0(x) \right\}^2 dy \\ &+ 2 \int_{[0, \infty)} \left\{ \int_{\mathbb{R}} (1 - \Phi_{\epsilon}(y - m_0(x))) dF_0(x) \right\}^2 dy \\ &\le 4 \int_{[0, \infty)} (1 - \Phi_{\epsilon}(y - \max(K_0, K)))^2 dy \end{aligned} \quad (48) $$

where we recall that $K \ge \|m\|_{\infty}$. The latter integral is finite using again (43). Similarly, we argue that the integral in (47) can be also bounded above by

$$ 4 \int_{(-\infty, 0]} \left( \Phi_{\epsilon}(y + \max(K_0, K)) \right)^2 dy < \infty. $$

This completes the proof that $M(m)$ is finite.

*Proof of Claim 3.* Using again that $\mathbb{H}_n(y) = 0$ for all $y < Y_{(1)}$, together with monotonicity of $\Phi_{\epsilon}$ and $m$, we have that

$$ \begin{aligned} \mathbb{M}_n(m) &\ge \int_{-\infty}^{Y_{(1)}} n^{-2} \left( \sum_{i=1}^{n} \Phi_{\epsilon}(y - m(X_i)) \right)^2 dy \\ &\ge n^{-2} \int_{-\infty}^{Y_{(1)}} \Phi_{\epsilon}(y - m(X_{(1)}))^2 dy \\ &= n^{-2} \int_{-\infty}^{Y_{(1)} - m(X_{(1)})} \Phi_{\epsilon}(t)^2 dt \\ &\to \infty, \text{ if } m(X_{(1)}) \to -\infty. \end{aligned} $$
Similarly, for $y \ge Y_{(n)}$, it holds that

$$ 
\begin{aligned}
\mathbb{M}_n(m) &\ge n^{-2} \int_{Y_{(n)}}^{\infty} (1 - \Phi_{\epsilon}(y - m(X_{(n)})))^2 dy \\
&\ge n^{-2} \int_{Y_{(n)} - m(X_{(n)})}^{\infty} (1 - \Phi_{\epsilon}(t))^2 dt \\
&\to \infty, \quad \text{if } m(X_{(n)}) \to \infty.
\end{aligned}
$$

Hence, there exists some $K > 0$ (which may depend on $n$) such that any candidate $m \in \mathcal{M}$ for the minimization problem in (8) should satisfy $-K \le m(X_{(1)}) \le \dots \le m(X_{(n)}) \le K$. By identifying an element $m \in \mathcal{M}_K$ by the corresponding vector $\theta = (m(X_{(1)}), \dots, m(X_{(n)}))^T$, it is easy to see that the original minimization problem is equivalent to minimizing

$$ \tilde{\mathbb{M}}_n(\theta) := \int_{\mathbb{R}} \left\{ \mathbb{H}_n(y) - n^{-1} \sum_{i=1}^n \Phi_{\epsilon}(y - \theta_i) \right\}^2 dy $$

on the compact finite dimensional subset

$$ \mathcal{S}_K := \{ (\theta_1, \dots, \theta_n)^T \in \mathbb{R}^n : -K \le \theta_1 \le \dots \le \theta_n \le K \}. $$

Now, the function $\tilde{\mathbb{M}}_n$ is continuous on $\mathcal{S}_K$ since for any sequence $(\theta_p)_{p \ge 0}$ in $\mathcal{S}_K^N$ converging (in any distance) to $\theta \in \mathcal{S}_K$, the sequence of functions

$$ y \mapsto \left( \mathbb{H}_n(y) - n^{-1} \sum_{i=1}^n \Phi_{\epsilon}(y - \theta_{p,i}) \right)^2 $$

converges pointwise by continuity of $\Phi_{\epsilon}$ (see Assumption A2) to the limit

$$ y \mapsto \left( \mathbb{H}_n(y) - n^{-1} \sum_{i=1}^n \Phi_{\epsilon}(y - \theta_i) \right)^2. \qquad (49) $$

Also, for $y \in \mathbb{R}$, we have that (49) is no larger than

$$ 
\left\{ 
\begin{array}{ll}
(n^{-1} \sum_{i=1}^n \Phi_{\epsilon}(y+K))^2, & \text{for } y < Y_{(1)} \\
4, & \text{for } Y_{(1)} \le y \le Y_{(n)} \\
(1 - n^{-1} \sum_{i=1}^n \Phi_{\epsilon}(y-K))^2, & \text{for } y > Y_{(n)}
\end{array}
\right. 
$$

where the function on the right side can be shown to be integrable using similar arguments as above. By the Lebesgue dominated convergence theorem, it follows that

$$ \lim_{p \to \infty} \tilde{\mathbb{M}}_n(\theta_p) = \tilde{\mathbb{M}}_n(\theta). $$

Thus, $\tilde{\mathbb{M}}_n$ admits at least a minimizer in $\mathcal{S}_K$, $\hat{\theta}_n$ say. We conclude that $\mathbb{M}_n$ admits at least a minimizer $\hat{m}_n$ which is bounded by $K$, and such that $(\hat{m}_n(X_{(1)}), \dots, \hat{m}_n(X_{(n)}))^T = \hat{\theta}_n$. The values of the minimizer being given by $\hat{\theta}_n$ at the observed covariates $X_1, \dots, X_n$,
Similarly, for $y \ge Y_{(n)}$, it holds that

$$ 
\begin{aligned}
\mathbb{M}_n(m) &\ge n^{-2} \int_{Y_{(n)}}^{\infty} (1 - \Phi_{\epsilon}(y - m(X_{(n)})))^2 dy \\
&\ge n^{-2} \int_{Y_{(n)} - m(X_{(n)})}^{\infty} (1 - \Phi_{\epsilon}(t))^2 dt \\
&\to \infty, \text{ if } m(X_{(n)}) \to \infty.
\end{aligned}
$$

Hence, there exists some $K > 0$ (which may depend on $n$) such that any candidate $m \in \mathcal{M}$ for the minimization problem in (8) should satisfy $-K \le m(X_{(1)}) \le \dots \le m(X_{(n)}) \le K$. By identifying an element $m \in \mathcal{M}_K$ by the corresponding vector $\theta = (m(X_{(1)}), \dots, m(X_{(n)}))^T$, it is easy to see that the original minimization problem is equivalent to minimizing

$$ \tilde{\mathbb{M}}_n(\theta) := \int_{\mathbb{R}} \left\{ \mathbb{H}_n(y) - n^{-1} \sum_{i=1}^n \Phi_{\epsilon}(y - \theta_i) \right\}^2 dy $$

on the compact finite dimensional subset

$$ \mathcal{S}_K := \{ (\theta_1, \dots, \theta_n)^T \in \mathbb{R}^n : -K \le \theta_1 \le \dots \le \theta_n \le K \}. $$

Now, the function $\tilde{\mathbb{M}}_n$ is continuous on $\mathcal{S}_K$ since for any sequence $(\theta_p)_{p \ge 0}$ in $\mathcal{S}_K^N$ converging (in any distance) to $\theta \in \mathcal{S}_K$, the sequence of functions

$$ y \mapsto \left( \mathbb{H}_n(y) - n^{-1} \sum_{i=1}^n \Phi_{\epsilon}(y - \theta_{p,i}) \right)^2 $$

converges pointwise by continuity of $\Phi_{\epsilon}$ (see Assumption A2) to the limit

$$ y \mapsto \left( \mathbb{H}_n(y) - n^{-1} \sum_{i=1}^n \Phi_{\epsilon}(y - \theta_i) \right)^2. \quad (49) $$

Also, for $y \in \mathbb{R}$, we have that (49) is no larger than

$$ \left\{ \begin{array}{ll} (n^{-1} \sum_{i=1}^n \Phi_{\epsilon}(y+K))^2, & \text{for } y < Y_{(1)} \\ 4, & \text{for } Y_{(1)} \le y \le Y_{(n)} \\ (1 - n^{-1} \sum_{i=1}^n \Phi_{\epsilon}(y-K))^2, & \text{for } y > Y_{(n)} \end{array} \right. $$

where the function on the right side can be shown to be integrable using similar arguments as above. By the Lebesgue dominated convergence theorem, it follows that

$$ \lim_{p \to \infty} \tilde{\mathbb{M}}_n(\theta_p) = \tilde{\mathbb{M}}_n(\theta). $$

Thus, $\tilde{\mathbb{M}}_n$ admits at least a minimizer in $\mathcal{S}_K$, $\hat{\theta}_n$ say. We conclude that $\mathbb{M}_n$ admits at least a minimizer $\hat{m}_n$ which is bounded by $K$, and such that $(\hat{m}_n(X_{(1)}), \dots, \hat{m}_n(X_{(n)}))^T = \hat{\theta}_n$. The values of the minimizer being given by $\hat{\theta}_n$ at the observed covariates $X_1, \dots, X_n$,
any monotone interpolation of these values gives a solution to (8). In particular, there exists a solution $\hat{m}_n$ that takes constant values between successive covariates and that is right continuous.

*Proof of Claim 4.* Without loss of generality, and possibly changing scale, we can assume that $\epsilon$ is supported on $[-1, 1]$. We show below that there exists at least a solution to (8) that is bounded in sup-norm by $K_0 + 2$, where $K_0$ is taken from Assumption A1. For an arbitrary $m \in \mathcal{M}$, we define the truncated version $\bar{m}$ by

$$ \bar{m}(x) = \begin{cases} K_0 + 2 & \text{if } m(x) \ge K_0 + 2 \\ -K_0 - 2 & \text{if } m(x) \le -K_0 - 2. \\ m(x) & \text{otherwise} \end{cases} $$

In the following, we place ourselves in the event $\|\epsilon\|_{\infty} \le 1$ which occurs with probability 1. Consider $y > K_0 + 1$. Since $|Y_i| \le K_0 + 1$ for all $i$, we then have $\mathbb{H}_n(y) = 1$, and

$$ \Phi_{\epsilon}(y - m(X_i)) = \Phi_{\epsilon}(y - \bar{m}(X_i)) = 1 $$

for all $X_i$'s such that $m(X_i) \le -K_0 - 2$. Also, for all $X_i$'s such that $m(X_i) \ge K_0 + 2$ we have that

$$ \Phi_{\epsilon}(y - m(X_i)) \le \Phi_{\epsilon}(y - \bar{m}(X_i)) \le 1. $$

This implies that

$$ \begin{aligned} & \int_{K_0+1}^{\infty} \left\{ \mathbb{H}_n(y) - n^{-1} \sum_{i=1}^{n} \Phi_{\epsilon}(y - \bar{m}(X_i)) \right\}^2 dy \\ & \le \int_{K_0+1}^{\infty} \left\{ \mathbb{H}_n(y) - n^{-1} \sum_{i=1}^{n} \Phi_{\epsilon}(y - m(X_i)) \right\}^2 dy. \end{aligned} \quad (50) $$

Similarly, it can be shown that

$$ \begin{aligned} & \int_{-\infty}^{-K_0-1} \left\{ \mathbb{H}_n(y) - n^{-1} \sum_{i=1}^{n} \Phi_{\epsilon}(y - \bar{m}(X_i)) \right\}^2 dy \\ & \le \int_{-\infty}^{-K_0-1} \left\{ \mathbb{H}_n(y) - n^{-1} \sum_{i=1}^{n} \Phi_{\epsilon}(y - m(X_i)) \right\}^2 dy. \end{aligned} \quad (51) $$

Now, consider $y$ such that $|y| \le K_0 + 1$. If for some $i$ we have $y > m(X_i) + 1$ (or $y < m(X_i) - 1$) then we have that

$$ \Phi_{\epsilon}(y - m(X_i)) = \Phi_{\epsilon}(y - \bar{m}(X_i)). $$

Indeed, if $y > m(X_i) + 1$, then $m(X_i) < K_0$. In case $m(X_i) > -K_0 - 2$ we have $m(X_i) = \bar{m}(X_i)$ and $\Phi_{\epsilon}(y - m(X_i)) = \Phi_{\epsilon}(y - \bar{m}(X_i)) = 1$. If $m(X_i) \le -K_0 - 2$, then $\bar{m}(X_i) = -K_0 - 2$ and hence $y - \bar{m}(X_i) \ge 1$ implying again that $\Phi_{\epsilon}(y - m(X_i)) = \Phi_{\epsilon}(y - \bar{m}(X_i)) = 1$. Similar arguments can be used in case $y < m(X_i) - 1$.

Now, the equality in the above display holds also if $|y - m(X_i)| \le 1$ since in that case, $|m(X_i)| \le K_0 + 2$, implying that $\bar{m}(X_i) = m(X_i)$. Combining this with (50) and (51) shows that

$$ \begin{aligned} & \int_{\mathbb{R}} \left\{ \mathbb{H}_n(y) - n^{-1} \sum_{i=1}^{n} \Phi_{\epsilon}(y - \bar{m}(X_i)) \right\}^2 dy \\ & \le \int_{\mathbb{R}} \left\{ \mathbb{H}_n(y) - n^{-1} \sum_{i=1}^{n} \Phi_{\epsilon}(y - m(X_i)) \right\}^2 dy. \end{aligned} $$
any monotone interpolation of these values gives a solution to (8). In particular, there exists a solution $\hat{m}_n$ that takes constant values between successive covariates and that is right continuous.

*Proof of Claim 4.* Without loss of generality, and possibly changing scale, we can assume that $\epsilon$ is supported on $[-1, 1]$. We show below that there exists at least a solution to (8) that is bounded in sup-norm by $K_0 + 2$, where $K_0$ is taken from Assumption A1. For an arbitrary $m \in \mathcal{M}$, we define the truncated version $\bar{m}$ by

$$ \bar{m}(x) = \begin{cases} K_0 + 2 & \text{if } m(x) \ge K_0 + 2 \\ -K_0 - 2 & \text{if } m(x) \le -K_0 - 2. \\ m(x) & \text{otherwise} \end{cases} $$

In the following, we place ourselves in the event $\|\epsilon\|_{\infty} \le 1$ which occurs with probability 1. Consider $y > K_0 + 1$. Since $|Y_i| \le K_0 + 1$ for all $i$, we then have $\mathbb{H}_n(y) = 1$, and

$$ \Phi_{\epsilon}(y - m(X_i)) = \Phi_{\epsilon}(y - \bar{m}(X_i)) = 1 $$

for all $X_i$'s such that $m(X_i) \le -K_0 - 2$. Also, for all $X_i$'s such that $m(X_i) \ge K_0 + 2$ we have that

$$ \Phi_{\epsilon}(y - m(X_i)) \le \Phi_{\epsilon}(y - \bar{m}(X_i)) \le 1. $$

This implies that

$$ \begin{aligned} & \int_{K_0+1}^{\infty} \left\{ \mathbb{H}_n(y) - n^{-1} \sum_{i=1}^{n} \Phi_{\epsilon}(y - \bar{m}(X_i)) \right\}^2 dy \\ & \le \int_{K_0+1}^{\infty} \left\{ \mathbb{H}_n(y) - n^{-1} \sum_{i=1}^{n} \Phi_{\epsilon}(y - m(X_i)) \right\}^2 dy. \end{aligned} \quad (50) $$

Similarly, it can be shown that

$$ \begin{aligned} & \int_{-\infty}^{-K_0-1} \left\{ \mathbb{H}_n(y) - n^{-1} \sum_{i=1}^{n} \Phi_{\epsilon}(y - \bar{m}(X_i)) \right\}^2 dy \\ & \le \int_{-\infty}^{-K_0-1} \left\{ \mathbb{H}_n(y) - n^{-1} \sum_{i=1}^{n} \Phi_{\epsilon}(y - m(X_i)) \right\}^2 dy. \end{aligned} \quad (51) $$

Now, consider $y$ such that $|y| \le K_0 + 1$. If for some $i$ we have $y > m(X_i) + 1$ (or $y < m(X_i) - 1$) then we have that

$$ \Phi_{\epsilon}(y - m(X_i)) = \Phi_{\epsilon}(y - \bar{m}(X_i)). $$

Indeed, if $y > m(X_i) + 1$, then $m(X_i) < K_0$. In case $m(X_i) > -K_0 - 2$ we have $m(X_i) = \bar{m}(X_i)$ and $\Phi_{\epsilon}(y - m(X_i)) = \Phi_{\epsilon}(y - \bar{m}(X_i)) = 1$. If $m(X_i) \le -K_0 - 2$, then $\bar{m}(X_i) = -K_0 - 2$ and hence $y - \bar{m}(X_i) \ge 1$ implying again that $\Phi_{\epsilon}(y - m(X_i)) = \Phi_{\epsilon}(y - \bar{m}(X_i)) = 1$. Similar arguments can be used in case $y < m(X_i) - 1$.

Now, the equality in the above display holds also if $|y - m(X_i)| \le 1$ since in that case, $|m(X_i)| \le K_0 + 2$, implying that $\bar{m}(X_i) = m(X_i)$. Combining this with (50) and (51) shows that

$$ \begin{aligned} & \int_{\mathbb{R}} \left\{ \mathbb{H}_n(y) - n^{-1} \sum_{i=1}^{n} \Phi_{\epsilon}(y - \bar{m}(X_i)) \right\}^2 dy \\ & \le \int_{\mathbb{R}} \left\{ \mathbb{H}_n(y) - n^{-1} \sum_{i=1}^{n} \Phi_{\epsilon}(y - m(X_i)) \right\}^2 dy. \end{aligned} $$
From Claim 3 in Proposition 2, there exists at least a solution to (8) and from the arguments above, its truncated version also is a solution. Hence, there exists at least a solution that is bounded in the sup-norm by $K_0+2$ with probability 1. This completes the proof of the proposition. □

### E.3 Proofs for Section 3

We first prove the propositions in Section 3 and finish with the proof of Theorems 1, 2, 3, 4 and 5.

**Proof of Proposition 3.** Let $\hat{\mathbb{H}}_n$ and $\mathbb{H}_n^0$ be the distribution functions defined as

$$ \hat{\mathbb{H}}_n(y) = \frac{1}{n} \sum_{i=1}^{n} \Phi_{\epsilon}(y - \hat{m}_n(X_i)), \quad \text{and} \quad \mathbb{H}_n^0(y) = \frac{1}{n} \sum_{i=1}^{n} \Phi_{\epsilon}(y - m_0(X_i)), \qquad (52) $$

for $y \in \mathbb{R}$. Recall the Plancherel's identity for Fourier transforms: for a function $g \in L_1(\mathbb{R}) \cap L_2(\mathbb{R})$, where $L_1(\mathbb{R})$ and $L_2(\mathbb{R})$ denote respectively the set of integrable, and the set of square integrable functions from $\mathbb{R}$ to $\mathbb{R}$ with respect to the Lebesgue measure it holds that

$$ \int_{\mathbb{R}} g(x)^2 dx = \frac{1}{2\pi} \int_{\mathbb{R}} |\phi_g(x)|^2 dx $$

where $\phi_g$ is defined in (10). If $F_1$ and $F_2$ are two distribution functions with finite expectations, it follows using integration by parts that

$$ \psi_{F_2}(x) - \psi_{F_1}(x) = -ix \int_{\mathbb{R}} (F_2(t) - F_1(t))e^{itx} dt $$

implying that

$$ \phi_{F_2-F_1}(x) = i \frac{\psi_{F_2}(x) - \psi_{F_1}(x)}{x} \qquad (53) $$

for $x \neq 0$. Moreover, if $F_1$ and $F_2$ have finite expectations then

$$ \int_{-\infty}^{0} F_{j}(x) dx < \infty \quad \text{and} \quad \int_{0}^{\infty} (1 - F_{j}(x)) dx < \infty, $$

for $j \in \{1, 2\}$, implying that $F_1 - F_2 \in L_1(\mathbb{R}) \cap L_2(\mathbb{R})$. Therefore, the Plancherel identity implies that

$$ \int_{\mathbb{R}} (F_2(x) - F_1(x))^2 dx = \frac{1}{2\pi} \int_{\mathbb{R}} \frac{1}{x^2} |\psi_{F_2}(x) - \psi_{F_1}(x)|^2 dx. $$

We apply below this identity with $F_1$ and $F_2$ replaced respectively by $\hat{\mathbb{L}}_n$ and $\mathbb{L}_n$, defined in (14). Note that the two corresponding distributions have finite expectations since they are supported on a finite set. Hence,

$$ \int_{\mathbb{R}} (\hat{\mathbb{L}}_n(w) - \mathbb{L}_n(w))^2 dw = \frac{1}{2\pi} \int_{\mathbb{R}} \frac{1}{t^2} |\psi_{\hat{\mathbb{L}}_n}(t) - \psi_{\mathbb{L}_n}(t)|^2 dt. $$
From Claim 3 in Proposition 2, there exists at least a solution to (8) and from the arguments above, its truncated version also is a solution. Hence, there exists at least a solution that is bounded in the sup-norm by $K_0+2$ with probability 1. This completes the proof of the proposition. □

### E.3 Proofs for Section 3

We first prove the propositions in Section 3 and finish with the proof of Theorems 1, 2, 3, 4 and 5.

**Proof of Proposition 3.** Let $\hat{\mathbb{H}}_n$ and $\mathbb{H}_n^0$ be the distribution functions defined as

$$ \hat{\mathbb{H}}_n(y) = \frac{1}{n} \sum_{i=1}^{n} \Phi_{\epsilon}(y - \hat{m}_n(X_i)), \quad \text{and} \quad \mathbb{H}_n^0(y) = \frac{1}{n} \sum_{i=1}^{n} \Phi_{\epsilon}(y - m_0(X_i)), \qquad (52) $$

for $y \in \mathbb{R}$. Recall the Plancherel's identity for Fourier transforms: for a function $g \in L_1(\mathbb{R}) \cap L_2(\mathbb{R})$, where $L_1(\mathbb{R})$ and $L_2(\mathbb{R})$ denote respectively the set of integrable, and the set of square integrable functions from $\mathbb{R}$ to $\mathbb{R}$ with respect to the Lebesgue measure it holds that

$$ \int_{\mathbb{R}} g(x)^2 dx = \frac{1}{2\pi} \int_{\mathbb{R}} |\phi_g(x)|^2 dx $$

where $\phi_g$ is defined in (10). If $F_1$ and $F_2$ are two distribution functions with finite expectations, it follows using integration by parts that

$$ \psi_{F_2}(x) - \psi_{F_1}(x) = -ix \int_{\mathbb{R}} (F_2(t) - F_1(t))e^{itx} dt $$

implying that

$$ \phi_{F_2-F_1}(x) = i \frac{\psi_{F_2}(x) - \psi_{F_1}(x)}{x} \qquad (53) $$

for $x \neq 0$. Moreover, if $F_1$ and $F_2$ have finite expectations then

$$ \int_{-\infty}^{0} F_{j}(x) dx < \infty \quad \text{and} \quad \int_{0}^{\infty} (1 - F_{j}(x)) dx < \infty, $$

for $j \in \{1, 2\}$, implying that $F_1 - F_2 \in L_1(\mathbb{R}) \cap L_2(\mathbb{R})$. Therefore, the Plancherel identity implies that

$$ \int_{\mathbb{R}} (F_2(x) - F_1(x))^2 dx = \frac{1}{2\pi} \int_{\mathbb{R}} \frac{1}{x^2} |\psi_{F_2}(x) - \psi_{F_1}(x)|^2 dx. $$

We apply below this identity with $F_1$ and $F_2$ replaced respectively by $\hat{\mathbb{L}}_n$ and $\mathbb{L}_n$, defined in (14). Note that the two corresponding distributions have finite expectations since they are supported on a finite set. Hence,

$$ \int_{\mathbb{R}} (\hat{\mathbb{L}}_n(w) - \mathbb{L}_n(w))^2 dw = \frac{1}{2\pi} \int_{\mathbb{R}} \frac{1}{t^2} |\psi_{\hat{\mathbb{L}}_n}(t) - \psi_{\mathbb{L}_n}(t)|^2 dt. $$
By Assumption A5, we can find $T^* > 0$ such that $|\phi_{f_e}(t)| \ge |\phi_{f_e}(T)| > 0$ for all $T > T^*$ and $|t| \le T$. Using that $|\psi_F| \le 1$ for any distribution function $F$, it follows from the previous display that for all $T > T^*$ we have

$$ \int_{\mathbb{R}} (\hat{\mathbb{L}}_n(w) - \mathbb{L}_n(w))^2 dw \le \frac{1}{2\pi|\phi_{f_e}(T)|^2} \int_{-T}^{T} \frac{|\phi_{f_e}(t)|^2}{t^2} |\psi_{\hat{\mathbb{L}}_n}(t) - \psi_{\mathbb{L}_n}(t)|^2 dt + \frac{4}{\pi T} \\ \le \frac{1}{2\pi|\phi_{f_e}(T)|^2} \int_{\mathbb{R}} \frac{|\phi_{f_e}(t)|^2}{t^2} |\psi_{\hat{\mathbb{L}}_n}(t) - \psi_{\mathbb{L}_n}(t)|^2 dt + \frac{4}{\pi T}. $$

Now, using again Plancherel's identity we have

$$ \int_{\mathbb{R}} \frac{|\phi_{f_e}(t)|^2}{t^2} |\psi_{\hat{\mathbb{L}}_n}(t) - \psi_{\mathbb{L}_n}(t)|^2 dt = \int_{\mathbb{R}} |\phi_{f_e}(t)|^2 |\phi_{\hat{\mathbb{L}}_n - \mathbb{L}_n}(t)|^2 dt \\ = \int_{\mathbb{R}} |\phi_{f_e \star (\hat{\mathbb{L}}_n - \mathbb{L}_n)}(t)|^2 dt \\ = 2\pi \int_{\mathbb{R}} (\hat{\mathbb{H}}_n(y) - \mathbb{H}_n^0(y))^2 dy $$

since $\hat{\mathbb{H}}_n = f_e \star \hat{\mathbb{L}}_n$ and $\mathbb{H}_n^0 = f_e \star \mathbb{L}_n$. Here, $(f \star g)(y) := \int_{\mathbb{R}} f(z)g(y-z)dz$. Hence, it follows from Assumption A4 that for sufficiently large $T$,

$$ \int_{\mathbb{R}} (\hat{\mathbb{L}}_n(w) - \mathbb{L}_n(w))^2 dw \le \frac{T^{2\beta}}{d_0^2} \int_{\mathbb{R}} (\hat{\mathbb{H}}_n(y) - \mathbb{H}_n^0(y))^2 dy + \frac{4}{\pi T}. \quad (54) $$

Assuming that we have

$$ \int_{\mathbb{R}} E (\hat{\mathbb{H}}_n(y) - \mathbb{H}_n^0(y))^2 dy = O(n^{-1}), \quad (55) $$

it will follow that for all sufficiently large $T$,

$$ \int_{\mathbb{R}} E (\hat{\mathbb{L}}_n(w) - \mathbb{L}_n(w))^2 dw \le O(T^{2\beta}n^{-1}) + \frac{4}{\pi T}. $$

For $T = T_n \sim n^{1/(2\beta+1)}$ we get

$$ \int_{\mathbb{R}} E (\hat{\mathbb{L}}_n(w) - \mathbb{L}_n(w))^2 dw \le O \left( \frac{1}{n^{1/(2\beta+1)}} \right), $$

which proves Proposition 3.

Now, we will show (55). From the inequality $(a+b)^2 \le 2(a^2+b^2)$, which holds for any $a$ and $b$ in $\mathbb{R}$, and the definition of $\hat{m}_n$ it follows that

$$ \int_{\mathbb{R}} (\hat{\mathbb{H}}_n(y) - \mathbb{H}_n^0(y))^2 dy \le 2 \int_{\mathbb{R}} (\hat{\mathbb{H}}_n(y) - \mathbb{H}_n(y))^2 dy + 2 \int_{\mathbb{R}} (\mathbb{H}_n^0(y) - \mathbb{H}_n(y))^2 dy \\ \le 4 \int_{\mathbb{R}} (\mathbb{H}_n^0(y) - \mathbb{H}_n(y))^2 dy \\ \le 8 \int_{\mathbb{R}} (\mathbb{H}_n^0(y) - \mathbb{H}_0(y))^2 dy + 8 \int_{\mathbb{R}} (\mathbb{H}_n(y) - \mathbb{H}_0(y))^2 dy $$

where

$$ E[(\mathbb{H}_n(y) - H_0(y))^2] = n^{-1}H_0(y)(1 - H_0(y)), $$
By Assumption A5, we can find $T^* > 0$ such that $|\phi_{f_e}(t)| \ge |\phi_{f_e}(T)| > 0$ for all $T > T^*$ and $|t| \le T$. Using that $|\psi_F| \le 1$ for any distribution function $F$, it follows from the previous display that for all $T > T^*$ we have

$$ 
\begin{aligned}
\int_{\mathbb{R}} (\hat{\mathbb{L}}_n(w) - \mathbb{L}_n(w))^2 dw &\le \frac{1}{2\pi|\phi_{f_e}(T)|^2} \int_{-T}^{T} \frac{|\phi_{f_e}(t)|^2}{t^2} |\psi_{\hat{\mathbb{L}}_n}(t) - \psi_{\mathbb{L}_n}(t)|^2 dt + \frac{4}{\pi T} \\
&\le \frac{1}{2\pi|\phi_{f_e}(T)|^2} \int_{\mathbb{R}} \frac{|\phi_{f_e}(t)|^2}{t^2} |\psi_{\hat{\mathbb{L}}_n}(t) - \psi_{\mathbb{L}_n}(t)|^2 dt + \frac{4}{\pi T}.
\end{aligned}
$$

Now, using again Plancherel's identity we have

$$ 
\begin{aligned}
\int_{\mathbb{R}} \frac{|\phi_{f_e}(t)|^2}{t^2} |\psi_{\hat{\mathbb{L}}_n}(t) - \psi_{\mathbb{L}_n}(t)|^2 dt &= \int_{\mathbb{R}} |\phi_{f_e}(t)|^2 |\phi_{\hat{\mathbb{L}}_n - \mathbb{L}_n}(t)|^2 dt \\
&= \int_{\mathbb{R}} |\phi_{f_e \star (\hat{\mathbb{L}}_n - \mathbb{L}_n)}(t)|^2 dt \\
&= 2\pi \int_{\mathbb{R}} (\hat{\mathbb{H}}_n(y) - \mathbb{H}_n^0(y))^2 dy
\end{aligned}
$$

since $\hat{\mathbb{H}}_n = f_e \star \hat{\mathbb{L}}_n$ and $\mathbb{H}_n^0 = f_e \star \mathbb{L}_n$. Here, $(f \star g)(y) := \int_{\mathbb{R}} f(z)g(y-z)dz$. Hence, it follows from Assumption A4 that for sufficiently large $T$,

$$ \int_{\mathbb{R}} (\hat{\mathbb{L}}_n(w) - \mathbb{L}_n(w))^2 dw \le \frac{T^{2\beta}}{d_0^2} \int_{\mathbb{R}} (\hat{\mathbb{H}}_n(y) - \mathbb{H}_n^0(y))^2 dy + \frac{4}{\pi T}. \quad (54) $$

Assuming that we have

$$ \int_{\mathbb{R}} E \left( \hat{\mathbb{H}}_n(y) - \mathbb{H}_n^0(y) \right)^2 dy = O(n^{-1}), \quad (55) $$

it will follow that for all sufficiently large $T$,

$$ \int_{\mathbb{R}} E \left( \hat{\mathbb{L}}_n(w) - \mathbb{L}_n(w) \right)^2 dw \le O(T^{2\beta}n^{-1}) + \frac{4}{\pi T}. $$

For $T = T_n \sim n^{1/(2\beta+1)}$ we get

$$ \int_{\mathbb{R}} E \left( \hat{\mathbb{L}}_n(w) - \mathbb{L}_n(w) \right)^2 dw \le O \left( \frac{1}{n^{1/(2\beta+1)}} \right), $$

which proves Proposition 3.

Now, we will show (55). From the inequality $(a+b)^2 \le 2(a^2+b^2)$, which holds for any $a$ and $b$ in $\mathbb{R}$, and the definition of $\hat{m}_n$ it follows that

$$ 
\begin{aligned}
\int_{\mathbb{R}} (\hat{\mathbb{H}}_n(y) - \mathbb{H}_n^0(y))^2 dy &\le 2 \int_{\mathbb{R}} (\hat{\mathbb{H}}_n(y) - \mathbb{H}_n(y))^2 dy + 2 \int_{\mathbb{R}} (\mathbb{H}_n^0(y) - \mathbb{H}_n(y))^2 dy \\
&\le 4 \int_{\mathbb{R}} (\mathbb{H}_n^0(y) - \mathbb{H}_n(y))^2 dy \\
&\le 8 \int_{\mathbb{R}} (\mathbb{H}_n^0(y) - H_0(y))^2 dy + 8 \int_{\mathbb{R}} (\mathbb{H}_n(y) - H_0(y))^2 dy
\end{aligned}
$$

where

$$ E[(\mathbb{H}_n(y) - H_0(y))^2] = n^{-1}H_0(y)(1 - H_0(y)), $$
and

$$ 
\begin{aligned}
E[(\mathbb{H}_n^0(y) - H_0(y))^2] &= \frac{1}{n} \mathrm{Var} \Phi_\epsilon(y - m_0(X)) \\
&= \frac{1}{n} \int_{\mathbb{R}} (\Phi_\epsilon(y - m_0(x)) - H_0(y))^2 dF_0(x).
\end{aligned}
$$

Both the integrals

$$ I_1 = \int_{\mathbb{R}} H_0(y)(1 - H_0(y))dy \qquad (56) $$

and

$$ I_2 = \int_{\mathbb{R}} \int_{\mathbb{R}} (\Phi_{\epsilon}(y - m_0(x)) - H_0(y))^2 dF_0(x)dy \qquad (57) $$

are finite, see Appendix A. This yields the result. □

**Proof of Proposition 4.** In the sequel we denote by $\mathbb{P}_n^X$ and $P^X$ the empirical probability measure associated with $X_1, \dots, X_n$ and the true corresponding probability measure. Then, the two integrals in Proposition 4 are the integrated square of the empirical processes

$$ \hat{\mathbb{L}}_n(w) - \hat{\mathbb{L}}_n^0(w) = (\mathbb{P}_n^X - P^X)1_{\{\hat{m}_n(\cdot) \le w\}} $$

and

$$ \mathbb{L}_n(w) - L_0(w) = (\mathbb{P}_n^X - P^X)1_{\{m_0(\cdot) \le w\}}. $$

In Appendix B, we recall some of the basic tools of empirical processes that we need in this proof. In what follows, the notation $\lesssim$ means smaller or equal modulo a universal positive multiplicative constant. For all fixed $w \in \mathbb{R}$ and $m \in \mathcal{M}$, let $k_{w,m}$ be the function defined by $k_{w,m}(x) = 1_{m(x) \le w}$ for all $x \in [0, 1]$. Consider the set of functions

$$ \mathcal{I} := \{k_{w,m}, \text{ with } m \in \mathcal{M} \text{ and } w \in [A, B]\}. $$

Using the same notation as in van der Vaart and Wellner (1996) (for completeness, we provide definitions in Appendix B), let us write $\mathbb{G}_nk = \sqrt{n}(\mathbb{P}_n^X - P^X)k$ for $k \in \mathcal{I}$. Since $\mathcal{I}$ is a subset of the class of monotone non-increasing functions $f: \mathbb{R} \to [0, 1]$, it follows from van der Vaart and Wellner (1996, Theorem 2.7.5) that there exists a universal constant $C > 0$, such that for any $\delta > 0$ and any probability measure $Q$,

$$ \log N_B(\delta, \mathcal{I}, L_2(Q)) \le \frac{C}{\delta} $$

(where $N_B(\cdot, \cdot, \cdot)$ is defined in Appendix B). Since $\mathcal{I}$ admits $F(t) = 1$ as an envelope, this and the inequality in (26) imply that

$$ 
\begin{aligned}
J(1, \mathcal{I}) &\le \sup_Q \int_0^1 \sqrt{1 + \log N_B(2\delta, \mathcal{I}, L_2(Q))} d\delta \\
&\le \int_0^1 \sqrt{1 + \frac{C}{2\delta}} d\delta \le 1 + \sqrt{2C} < \infty,
\end{aligned}
$$
and

$$ 
\begin{aligned}
E[(\mathbb{H}_n^0(y) - H_0(y))^2] &= \frac{1}{n} \mathrm{Var} \Phi_\epsilon(y - m_0(X)) \\
&= \frac{1}{n} \int_{\mathbb{R}} (\Phi_\epsilon(y - m_0(x)) - H_0(y))^2 dF_0(x).
\end{aligned}
$$

Both the integrals

$$ I_1 = \int_{\mathbb{R}} H_0(y)(1 - H_0(y))dy \qquad (56) $$

and

$$ I_2 = \int_{\mathbb{R}} \int_{\mathbb{R}} (\Phi_{\epsilon}(y - m_0(x)) - H_0(y))^2 dF_0(x)dy \qquad (57) $$

are finite, see Appendix A. This yields the result. □

**Proof of Proposition 4.** In the sequel we denote by $\mathbb{P}_n^X$ and $P^X$ the empirical probability measure associated with $X_1, \dots, X_n$ and the true corresponding probability measure. Then, the two integrals in Proposition 4 are the integrated square of the empirical processes

$$ \hat{\mathbb{L}}_n(w) - \hat{\mathbb{L}}_n^0(w) = (\mathbb{P}_n^X - P^X)1_{\{\hat{m}_n(\cdot) \le w\}} $$

and

$$ \mathbb{L}_n(w) - L_0(w) = (\mathbb{P}_n^X - P^X)1_{\{m_0(\cdot) \le w\}}. $$

In Appendix B, we recall some of the basic tools of empirical processes that we need in this proof. In what follows, the notation $\lesssim$ means smaller or equal modulo a universal positive multiplicative constant. For all fixed $w \in \mathbb{R}$ and $m \in \mathcal{M}$, let $k_{w,m}$ be the function defined by $k_{w,m}(x) = 1_{m(x) \le w}$ for all $x \in [0, 1]$. Consider the set of functions

$$ \mathcal{I} := \{k_{w,m}, \text{ with } m \in \mathcal{M} \text{ and } w \in [A, B]\}. $$

Using the same notation as in van der Vaart and Wellner (1996) (for completeness, we provide definitions in Appendix B), let us write $\mathbb{G}_nk = \sqrt{n}(\mathbb{P}_n^X - P^X)k$ for $k \in \mathcal{I}$. Since $\mathcal{I}$ is a subset of the class of monotone non-increasing functions $f: \mathbb{R} \to [0, 1]$, it follows from van der Vaart and Wellner (1996, Theorem 2.7.5) that there exists a universal constant $C > 0$, such that for any $\delta > 0$ and any probability measure $Q$,

$$ \log N_B(\delta, \mathcal{I}, L_2(Q)) \le \frac{C}{\delta} $$

(where $N_B(\cdot, \cdot, \cdot)$ is defined in Appendix B). Since $\mathcal{I}$ admits $F(t) = 1$ as an envelope, this and the inequality in (26) imply that

$$ 
\begin{aligned}
J(1, \mathcal{I}) &\le \sup_Q \int_0^1 \sqrt{1 + \log N_B(2\delta, \mathcal{I}, L_2(Q))} d\delta \\
&\le \int_0^1 \sqrt{1 + \frac{C}{2\delta}} d\delta \le 1 + \sqrt{2C} < \infty,
\end{aligned}
$$
where $J(\delta, \mathcal{F})$ is defined in (27). Since $X_1, \dots, X_n$ are i.i.d. it follows now from van der Vaart and Wellner (1996, Theorem 2.14.1) that

$$ (E[\|\mathbb{G}_n\|_{\mathcal{I}}^2])^{1/2} \lesssim J(1, \mathcal{I}). \qquad (58) $$

Let us denote

$$ M_n = \max \left( \int_A^B (\hat{\mathbb{L}}_n(w) - \hat{\mathbb{L}}_n^0(w))^2 dw, \int_A^B (\mathbb{L}_n(w) - L_0(w))^2 dw \right). $$

Then,

$$ 0 \le M_n \le \frac{B-A}{n} \|\mathbb{G}_n\|_{\mathcal{I}}^2. $$

The first two claims in the proposition now follow from (58) combined to the Markov's inequality.

Now, using the inequality $(a+b+c)^2 \le 3(a^2 + b^2 + c^2)$ for any real numbers $a, b$ and $c$, we have

$$ \int_A^B (\hat{\mathbb{L}}_n^0(w) - L_0(w))^2 dw \le 3 \int_A^B (\hat{\mathbb{L}}_n(w) - \hat{\mathbb{L}}_n^0(w))^2 dw \\ + 3 \int_A^B (\hat{\mathbb{L}}_n(w) - \mathbb{L}_n(w))^2 dw \\ + 3 \int_A^B (\mathbb{L}_n(w) - L_0(w))^2 dw. $$

It thus follows from Proposition 3 that (16) holds provided that $B-A = O_P(n^{2\beta/(2\beta+1)})$. □

**Proof of Proposition 5.** The first equality in Proposition 5 follows from Lemma 3 above combined with the definition of $\hat{\mathbb{L}}_n^0$ and $L_n^0$, while the second equality follows from Proposition 4. □

**Proof of Proposition 6.** It follows from Lemma 4 below that

$$ \int_a^b |\hat{m}_n(x) - m_0(x)| dF_0(x) \le \int_{A_n}^{B_n} |F_0 \circ \hat{m}_n^{-1}(x) - F_0 \circ m_0^{-1}(x)| dx. $$

The proposition then follows from applying the Cauchy-Schwarz inequality. □

**Lemma 4.** Let $f : [0, 1] \to \mathbb{R}$ and $g : [0, 1] \to \mathbb{R}$ be right-continuous non-decreasing functions. Let $f^{-1}$ and $g^{-1}$ be the corresponding generalized inverses, see (6) where the infimum of an empty set is defined to be one. Let $H : [0, 1] \to [0, 1]$ be a continuous non-decreasing function. Then, for all $a < b$ in $[0, 1]$ we have

$$ \int_a^b |f(t) - g(t)| dH(t) \le \int_{I(a)}^{S(b)} |H \circ g^{-1}(x) - H \circ f^{-1}(x)| dx $$

where

$$ I(a) = f(a) \wedge g(a) ; S(b) = f(b) \vee g(b). $$
where $J(\delta, \mathcal{F})$ is defined in (27). Since $X_1, \dots, X_n$ are i.i.d. it follows now from van der Vaart and Wellner (1996, Theorem 2.14.1) that

$$ (E [\|\mathbb{G}_n\|_{\mathcal{I}}^2])^{1/2} \lesssim J(1, \mathcal{I}). \qquad (58) $$

Let us denote

$$ M_n = \max \left( \int_A^B (\hat{\mathbb{L}}_n(w) - \hat{\mathbb{L}}_n^0(w))^2 dw, \int_A^B (\mathbb{L}_n(w) - L_0(w))^2 dw \right). $$

Then,

$$ 0 \le M_n \le \frac{B-A}{n} \|\mathbb{G}_n\|_{\mathcal{I}}^2. $$

The first two claims in the proposition now follow from (58) combined to the Markov's inequality.

Now, using the inequality $(a+b+c)^2 \le 3(a^2 + b^2 + c^2)$ for any real numbers $a, b$ and $c$, we have

$$ \int_A^B (\hat{\mathbb{L}}_n^0(w) - L_0(w))^2 dw \le 3 \int_A^B (\hat{\mathbb{L}}_n(w) - \hat{\mathbb{L}}_n^0(w))^2 dw \\ + 3 \int_A^B (\hat{\mathbb{L}}_n(w) - \mathbb{L}_n(w))^2 dw \\ + 3 \int_A^B (\mathbb{L}_n(w) - L_0(w))^2 dw. $$

It thus follows from Proposition 3 that (16) holds provided that $B-A = O_P(n^{2\beta/(2\beta+1)})$. $\square$

**Proof of Proposition 5.** The first equality in Proposition 5 follows from Lemma 3 above combined with the definition of $\hat{\mathbb{L}}_n^0$ and $L_n^0$, while the second equality follows from Proposition 4. $\square$

**Proof of Proposition 6.** It follows from Lemma 4 below that

$$ \int_a^b |\hat{m}_n(x) - m_0(x)| dF_0(x) \le \int_{A_n}^{B_n} |F_0 \circ \hat{m}_n^{-1}(x) - F_0 \circ m_0^{-1}(x)| dx. $$

The proposition then follows from applying the Cauchy-Schwarz inequality. $\square$

**Lemma 4.** Let $f : [0, 1] \to \mathbb{R}$ and $g : [0, 1] \to \mathbb{R}$ be right-continuous non-decreasing functions. Let $f^{-1}$ and $g^{-1}$ be the corresponding generalized inverses, see (6) where the infimum of an empty set is defined to be one. Let $H : [0, 1] \to [0, 1]$ be a continuous non-decreasing function. Then, for all $a < b$ in $[0, 1]$ we have

$$ \int_a^b |f(t) - g(t)| dH(t) \le \int_{I(a)}^{S(b)} |H \circ g^{-1}(x) - H \circ f^{-1}(x)| dx $$

where

$$ I(a) = f(a) \wedge g(a) ; S(b) = f(b) \vee g(b). $$
**Proof** [Proof of Lemma 4.]

For all real numbers $u$, let $u_+ = \max(u, 0)$. We then have

$$ \int_a^b |f(t) - g(t)| dH(t) = I_1 + I_2 \qquad (59) $$

where

$$ I_1 = \int_a^b (f(t) - g(t))_+ dH(t) \text{ and } I_2 = \int_a^b (g(t) - f(t))_+ dH(t). $$

Let us deal first with $I_1$. We have

$$ \begin{aligned} I_1 &= \int_a^b \int_0^\infty \mathbb{I}_{\{x \le f(t) - g(t)\}} dx dH(t) = \int_a^b \int_{g(t)}^\infty \mathbb{I}_{\{x \le f(t)\}} dx dH(t) \\ &= \int_a^b \int_{g(t)}^{f(b)} \mathbb{I}_{\{x \le f(t)\}} dx dH(t), \end{aligned} $$

where we use a change of variable for the second equality and the monotonicity of $f$ for the third one. Similar to (35), the equivalence

$$ t \ge f^{-1}(x) \iff f(t) \ge x $$

holds for all $t \in [0, 1)$ and $x \in \mathbb{R}$. Combining this with the Fubini theorem, we arrive at

$$ \begin{aligned} I_1 &= \int_a^b \int_{g(t)}^{f(b)} \mathbb{I}_{\{t \ge f^{-1}(x)\}} dx dH(t) \\ &= \int_{g(a)}^{f(b)} \int_a^b \mathbb{I}_{\{t \ge f^{-1}(x)\}} \mathbb{I}_{\{t < g^{-1}(x)\}} dH(t) dx. \end{aligned} $$

Hence, it follows from the continuity of $H$ that

$$ \begin{aligned} I_1 &= \int_{g(a)}^{f(b)} (H(g^{-1}(x) \wedge b) - H(f^{-1}(x) \vee a))_+ dx \\ &\le \int_{g(a)}^{f(b)} (H(g^{-1}(x)) - H(f^{-1}(x)))_+ dx, \end{aligned} $$

since $H$ is non-decreasing. Since $I(a) \le g(a)$ and $S(b) \ge f(b)$, this implies that

$$ I_1 \le \int_{I(a)}^{S(b)} (H \circ g^{-1}(x) - H \circ f^{-1}(x))_+ dx. $$

Interchanging the roles of $f$ and $g$, we obtain

$$ I_2 \le \int_{I(a)}^{S(b)} (H \circ f^{-1}(x) - H \circ g^{-1}(x))_+ dx $$

and therefore,

$$ I_1 + I_2 \le \int_{I(a)}^{S(b)} |H \circ f^{-1}(x) - H \circ g^{-1}(x)| dx. $$
**Proof** [Proof of Lemma 4.]

For all real numbers $u$, let $u_+ = \max(u, 0)$. We then have

$$ \int_a^b |f(t) - g(t)| dH(t) = I_1 + I_2 \qquad (59) $$

where

$$ I_1 = \int_a^b (f(t) - g(t))_+ dH(t) \text{ and } I_2 = \int_a^b (g(t) - f(t))_+ dH(t). $$

Let us deal first with $I_1$. We have

$$ \begin{aligned} I_1 &= \int_a^b \int_0^\infty \mathbb{I}_{\{x \le f(t) - g(t)\}} dx dH(t) = \int_a^b \int_{g(t)}^\infty \mathbb{I}_{\{x \le f(t)\}} dx dH(t) \\ &= \int_a^b \int_{g(t)}^{f(b)} \mathbb{I}_{\{x \le f(t)\}} dx dH(t), \end{aligned} $$

where we use a change of variable for the second equality and the monotonicity of $f$ for the third one. Similar to (35), the equivalence

$$ t \ge f^{-1}(x) \iff f(t) \ge x $$

holds for all $t \in [0, 1)$ and $x \in \mathbb{R}$. Combining this with the Fubini theorem, we arrive at

$$ \begin{aligned} I_1 &= \int_a^b \int_{g(t)}^{f(b)} \mathbb{I}_{\{t \ge f^{-1}(x)\}} dx dH(t) \\ &= \int_{g(a)}^{f(b)} \int_a^b \mathbb{I}_{\{t \ge f^{-1}(x)\}} \mathbb{I}_{\{t < g^{-1}(x)\}} dH(t) dx. \end{aligned} $$

Hence, it follows from the continuity of $H$ that

$$ \begin{aligned} I_1 &= \int_{g(a)}^{f(b)} (H(g^{-1}(x) \wedge b) - H(f^{-1}(x) \vee a))_+ dx \\ &\le \int_{g(a)}^{f(b)} (H(g^{-1}(x)) - H(f^{-1}(x)))_+ dx, \end{aligned} $$

since $H$ is non-decreasing. Since $I(a) \le g(a)$ and $S(b) \ge f(b)$, this implies that

$$ I_1 \le \int_{I(a)}^{S(b)} (H \circ g^{-1}(x) - H \circ f^{-1}(x))_+ dx. $$

Interchanging the roles of $f$ and $g$, we obtain

$$ I_2 \le \int_{I(a)}^{S(b)} (H \circ f^{-1}(x) - H \circ g^{-1}(x))_+ dx $$

and therefore,

$$ I_1 + I_2 \le \int_{I(a)}^{S(b)} |H \circ f^{-1}(x) - H \circ g^{-1}(x)| dx. $$
Lemma 4 then follows from (59).

■

**Proof of Theorem 1.** *Proof of the first claim.* If we have (12), then $A_n$ and $B_n$ from Proposition 6 are both of the order $O_P(1)$. Hence, the first claim in Theorem 1 is an immediate consequence of Proposition 6 combined with Proposition 5.

*Proof of the second claim.* Now, we show that (12) holds true for all $a$ and $b$ such that $0 < F_0(a) \le F_0(b) < 1$. It follows from the definition of $\hat{\mathbb{L}}_n$ and $\mathbb{L}_n$ together with the Hölder inequality and Proposition 3 that

$$ 
\begin{aligned}
\int_{\|m_0\|_{\infty}}^{2\|m_0\|_{\infty}} \left(1 - \frac{1}{n} \sum_{i=1}^{n} 1_{\{\hat{m}_n(X_i) \le w\}}\right) dw &= \int_{\|m_0\|_{\infty}}^{2\|m_0\|_{\infty}} \left|\hat{\mathbb{L}}_n(w) - \mathbb{L}_n(w)\right| dw \\
&\le \left(\|m_0\|_{\infty} \int_{\mathbb{R}} \left(\hat{\mathbb{L}}_n(w) - \mathbb{L}_n(w)\right)^2 dw\right)^{1/2} \\
&= o_P(1).
\end{aligned}
$$

On the other hand,

$$ \int_{\|m_0\|_{\infty}}^{2\|m_0\|_{\infty}} \left(1 - \frac{1}{n} \sum_{i=1}^{n} 1_{\{\hat{m}_n(X_i) \le w\}}\right) dw \ge \|m_0\|_{\infty} n^{-1} \sum_{i=1}^{n} 1_{\{\hat{m}_n(X_i) > 2\|m_0\|_{\infty}\}} $$

and therefore,

$$ \sum_{i=1}^{n} 1_{\{\hat{m}_n(X_i) > 2\|m_0\|_{\infty}\}} = o_P(n). $$

By monotonicity of $\hat{m}_n$ this implies that

$$ 1_{\{\hat{m}_n(b)>2\|m_0\|_{\infty}\}} \sum_{i=1}^{n} 1_{\{X_i>b\}} \le \sum_{i=1}^{n} 1_{\{\hat{m}_n(X_i)>2\|m_0\|_{\infty}\}} = o_P(n). $$

By the law of large numbers, $n^{-1} \sum_{i=1}^{n} 1_{\{X_i>b\}}$ converges in probability to $1 - F_0(b) > 0$ and therefore, it follows from the previous display that

$$ 1_{\{\hat{m}_n(b)>2\|m_0\|_{\infty}\}} = o_P(1). $$

This implies that

$$ \lim_{n \to \infty} P(\hat{m}_n(b) > 2\|m_0\|_{\infty}) = 0. $$

One can prove similarly that

$$ \lim_{n \to \infty} P(\hat{m}_n(a) < -2\|m_0\|_{\infty}) = 0. $$

This implies (12) by monotonicity of $\hat{m}_n$, which completes the proof of the second claim in Theorem 1. □

**Proof of Theorem 2** As the arguments are very similar to those used in the proof of Theorem 1, we focus here on how the convergence rate is obtained in the supersmooth
Lemma 4 then follows from (59).

■

**Proof of Theorem 1.** *Proof of the first claim.* If we have (12), then $A_n$ and $B_n$ from Proposition 6 are both of the order $O_P(1)$. Hence, the first claim in Theorem 1 is an immediate consequence of Proposition 6 combined with Proposition 5.

*Proof of the second claim.* Now, we show that (12) holds true for all $a$ and $b$ such that $0 < F_0(a) \le F_0(b) < 1$. It follows from the definition of $\hat{\mathbb{L}}_n$ and $\mathbb{L}_n$ together with the Hölder inequality and Proposition 3 that

$$ \int_{\|m_0\|_{\infty}}^{2\|m_0\|_{\infty}} \left(1 - \frac{1}{n} \sum_{i=1}^{n} 1_{\{\hat{m}_n(X_i) \le w\}}\right) dw = \int_{\|m_0\|_{\infty}}^{2\|m_0\|_{\infty}} |\hat{\mathbb{L}}_n(w) - \mathbb{L}_n(w)| dw \\ \le \left(\|m_0\|_{\infty} \int_{\mathbb{R}} (\hat{\mathbb{L}}_n(w) - \mathbb{L}_n(w))^2 dw\right)^{1/2} \\ = o_P(1). $$

On the other hand,

$$ \int_{\|m_0\|_{\infty}}^{2\|m_0\|_{\infty}} \left(1 - \frac{1}{n} \sum_{i=1}^{n} 1_{\{\hat{m}_n(X_i) \le w\}}\right) dw \ge \|m_0\|_{\infty} n^{-1} \sum_{i=1}^{n} 1_{\{\hat{m}_n(X_i) > 2\|m_0\|_{\infty}\}} $$

and therefore,

$$ \sum_{i=1}^{n} 1_{\{\hat{m}_n(X_i) > 2\|m_0\|_{\infty}\}} = o_P(n). $$

By monotonicity of $\hat{m}_n$ this implies that

$$ 1_{\{\hat{m}_n(b)>2\|m_0\|_{\infty}\}} \sum_{i=1}^{n} 1_{\{X_i>b\}} \le \sum_{i=1}^{n} 1_{\{\hat{m}_n(X_i)>2\|m_0\|_{\infty}\}} = o_P(n). $$

By the law of large numbers, $n^{-1} \sum_{i=1}^{n} 1_{\{X_i>b\}}$ converges in probability to $1 - F_0(b) > 0$ and therefore, it follows from the previous display that

$$ 1_{\{\hat{m}_n(b)>2\|m_0\|_{\infty}\}} = o_P(1). $$

This implies that

$$ \lim_{n \to \infty} P(\hat{m}_n(b) > 2\|m_0\|_{\infty}) = 0. $$

One can prove similarly that

$$ \lim_{n \to \infty} P(\hat{m}_n(a) < -2\|m_0\|_{\infty}) = 0. $$

This implies (12) by monotonicity of $\hat{m}_n$, which completes the proof of the second claim in Theorem 1. □

**Proof of Theorem 2** As the arguments are very similar to those used in the proof of Theorem 1, we focus here on how the convergence rate is obtained in the supersmooth
case. Under Assumptions A0–A3, Assumption A4' and Assumption A5, we can show
that this rate of convergence is driven by

$$
O\left(\exp\left(\frac{2T^{\beta}}{\gamma}\right) T^{-2\alpha} n^{-1}\right) + \frac{4}{\pi T}
$$

for $T = T_n \to \infty$ as $n \to \infty$ which should be determined so that the above expression
is smallest. This means that the first term should converge to 0 or equivalently that
there exists a sequence $(K_n)_n$ such that $\log(T_n) = K_n \to \infty$ and $K_n \le \log n$ such that

$$
\frac{2T_n^\beta}{\gamma} = \log n + (2\alpha - 1)K_n
$$

or equivalently

$$
T_n = c \left( \log n + (2\alpha - 1)K_n \right)^{1/\beta}, \quad \text{for } c = (\gamma/2)^{1/\beta}.
$$

It is not difficult to see that the optimal choice of the sequence $(K_n)_n$ is $K_n = (1 - a) \log n$ for some $a \in (0, 1)$ (the case $a = 0$ is impossible because otherwise we would have $T_n = c(2\alpha)^{1/\beta}(\log(T_n))^{1/\beta}$). This in turn yields

$$
T_n = c \left( a \log n + 2\alpha \log(T_n) \right)^{1/\beta},
$$

implying that $T_n \sim ca^{1/\beta}(\log n)^{1/\beta}$ and that rate of convergence is $(\log n)^{-1/(2\beta)}$. $\square$

**Proof of Theorem 3.** Without loss of generality, we can assume that the support points of $\epsilon$ are all in $[-1, 1]$. From Proposition 2 we know that with probability 1 there exists a solution to (8) which is bounded in the sup-norm by $K_0 + 2$. In the sequel, we denote by $\hat{m}_n$ such a solution.

Recall $\hat{\mathbb{H}}_n$ and $\mathbb{H}_n^0$ from (52). Using the Cauchy-Schwarz inequality it follows that

$$
\int_{\mathbb{R}} |\hat{\mathbb{H}}_n(y) - \mathbb{H}_n^0(y)| dy = \int_{-K_0-3}^{K_0+3} |\hat{\mathbb{H}}_n(y) - \mathbb{H}_n^0(y)| dy \\
\le \left( (2K_0 + 6) \int_{\mathbb{R}} |\hat{\mathbb{H}}_n(y) - \mathbb{H}_n^0(y)|^2 dy \right)^{1/2}.
$$

Since the equality in (55) holds under Assumptions A0 to A2 it follows that

$$
\int_{\mathbb{R}} \left| \hat{\mathbb{H}}_n(y) - \mathbb{H}_n^0(y) \right| dy = O_P(n^{-1/2}).
$$

Now note that $\hat{\mathbb{H}}_n$ and $\mathbb{H}_n^0$ are distribution functions with bounded support, and hence they admit a finite first moment. Therefore, denoting by $W_1(F, G)$ the Wasserstein-distance of first order between two probability distributions with respective distribution functions $F$ and $G$, it follows from Proposition 9 in Appendix C that

$$
W_1(\hat{\mathbb{H}}_n, \mathbb{H}_n^0) = O_P(n^{-1/2}).
$$

With $\hat{\mathbb{L}}_n$ and $\mathbb{L}_n$ taken from (14), it follows from Proposition 10 in Appendix C that there exists some constant $C > 0$ that depends only on the distribution of $\epsilon$ and $K_0$ such that

$$
W_1(\hat{\mathbb{L}}_n, \mathbb{L}_n) \le C W_1(\hat{\mathbb{H}}_n, \mathbb{H}_n^0).
$$
case. Under Assumptions A0–A3, Assumption A4' and Assumption A5, we can show that this rate of convergence is driven by

$$O\left(\exp\left(\frac{2T^\beta}{\gamma}\right) T^{-2\alpha}n^{-1}\right) + \frac{4}{\pi T}$$

for $T = T_n \to \infty$ as $n \to \infty$ which should be determined so that the above expression is smallest. This means that the first term should converge to 0 or equivalently that there exists a sequence $(K_n)_n$ such that $\log(T_n) = K_n \to \infty$ and $K_n \le \log n$ such that

$$\frac{2T_n^\beta}{\gamma} = \log n + (2\alpha - 1)K_n$$

or equivalently

$$T_n = c\left(\log n + (2\alpha - 1)K_n\right)^{1/\beta}, \quad \text{for } c = (\gamma/2)^{1/\beta}.$$

It is not difficult to see that the optimal choice of the sequence $(K_n)_n$ is $K_n = (1 - a) \log n$ for some $a \in (0, 1)$ (the case $a = 0$ is impossible because otherwise we would have $T_n = c(2\alpha)^{1/\beta}(\log(T_n))^{1/\beta}$). This in turn yields

$$T_n = c\left(a \log n + 2\alpha \log(T_n)\right)^{1/\beta},$$

implying that $T_n \sim ca^{1/\beta}(\log n)^{1/\beta}$ and that rate of convergence is $(\log n)^{-1/(2\beta)}$. $\square$

**Proof of Theorem 3.** Without loss of generality, we can assume that the support points of $\epsilon$ are all in $[-1, 1]$. From Proposition 2 we know that with probability 1 there exists a solution to (8) which is bounded in the sup-norm by $K_0 + 2$. In the sequel, we denote by $\hat{m}_n$ such a solution.

Recall $\hat{\mathbb{H}}_n$ and $\mathbb{H}_n^0$ from (52). Using the Cauchy-Schwarz inequality it follows that

$$\int_{\mathbb{R}} |\hat{\mathbb{H}}_n(y) - \mathbb{H}_n^0(y)| dy = \int_{-K_0-3}^{K_0+3} |\hat{\mathbb{H}}_n(y) - \mathbb{H}_n^0(y)| dy \\ \le \left( (2K_0 + 6) \int_{\mathbb{R}} |\hat{\mathbb{H}}_n(y) - \mathbb{H}_n^0(y)|^2 dy \right)^{1/2}.$$

Since the equality in (55) holds under Assumptions A0 to A2 it follows that

$$\int_{\mathbb{R}} |\hat{\mathbb{H}}_n(y) - \mathbb{H}_n^0(y)| dy = O_P(n^{-1/2}).$$

Now note that $\hat{\mathbb{H}}_n$ and $\mathbb{H}_n^0$ are distribution functions with bounded support, and hence they admit a finite first moment. Therefore, denoting by $W_1(F, G)$ the Wasserstein-distance of first order between two probability distributions with respective distribution functions $F$ and $G$, it follows from Proposition 9 in Appendix C that

$$W_1(\hat{\mathbb{H}}_n, \mathbb{H}_n^0) = O_P(n^{-1/2}).$$

With $\hat{\mathbb{L}}_n$ and $\mathbb{L}_n$ taken from (14), it follows from Proposition 10 in Appendix C that there exists some constant $C > 0$ that depends only on the distribution of $\epsilon$ and $K_0$ such that

$$W_1(\hat{\mathbb{L}}_n, \mathbb{L}_n) \le C W_1(\hat{\mathbb{H}}_n, \mathbb{H}_n^0).$$
Thus,

$$W_1(\hat{\mathbb{L}}_n, \mathbb{L}_n) = O_P(n^{-1/2}).$$

Using again Proposition 9 in Appendix C the latter rate yields

$$\int_{\mathbb{R}} |\hat{\mathbb{L}}_n(y) - \mathbb{L}_n(y)| dy = O_P(n^{-1/2}).$$

Combining this with Proposition 4 and the Cauchy-Schwarz inequality, we obtain

$$\begin{aligned} \int_{\mathbb{R}} |\hat{\mathbb{L}}_n^0(y) - L_0(y)| dy &= \int_{-K_0-2}^{K_0+2} |\hat{\mathbb{L}}_n^0(y) - L_0(y)| dy \\ &= O_P(n^{-1/2}). \end{aligned}$$

Using the result of Lemma 4, it follows that

$$\begin{aligned} \int_0^1 |\hat{m}_n(x) - m_0(x)| dF_0(x) &\le \int_{m_0(0) \wedge \hat{m}_n(0)}^{m_0(1) \vee \hat{m}_n(1)} |F_0 \circ \hat{m}_n^{-1}(y) - F_0 \circ m_0^{-1}(y)| dy \\ &= \int_{m_0(0) \wedge \hat{m}_n(0)}^{m_0(1) \vee \hat{m}_n(1)} |\hat{\mathbb{L}}_n^0(y) - L_0(y)| dy \\ &\le \int_{-K_0-2}^{K_0+2} |\hat{\mathbb{L}}_n^0(y) - L_0(y)| dy \end{aligned}$$

implying that $\int_0^1 |\hat{m}_n(x) - m_0(x)| dF_0(x) = O_P(n^{-1/2})$. $\square$

**Proof of Theorem 4.** *Proof of Claim 1.* We start with the case where the noise has an absolutely continuous density. We restrict attention to the ordinary smooth case because the arguments are very similar in the supersmooth one. Now, since the proof in the ordinary smooth case follows the same lines of the proof of Theorem 1, details are omitted and the reader is referred to the latter proof. Here, we only point out the main existing differences. Similar to (14) and (52), we define

$$\hat{\mathbb{L}}_{n_x, n_y}(w) := \frac{1}{n_x} \sum_{i=1}^{n_x} \mathbb{1}_{\{\hat{m}_{n_x, n_y}(X_i) \le w\}} \quad \text{and} \quad \mathbb{L}_{n_x}(w) := \frac{1}{n_x} \sum_{i=1}^{n_x} \mathbb{1}_{\{m_0(X_i) \le w\}}$$

for all $w \in \mathbb{R}$, and

$$\hat{\mathbb{H}}_{n_x, n_y}(y) = \frac{1}{n_x} \sum_{i=1}^{n_x} \Phi_{\epsilon}(y - \hat{m}_{n_x, n_y}(X_i)), \quad \text{and} \quad \mathbb{H}_{n_x}^0(y) = \frac{1}{n_x} \sum_{i=1}^{n_x} \Phi_{\epsilon}(y - m_0(X_i))$$

for all $y \in \mathbb{R}$. With similar arguments as for the proof of (54) we obtain that for all sufficiently large $T$,

$$\int_{\mathbb{R}} (\hat{\mathbb{L}}_{n_x, n_y}(w) - \mathbb{L}_{n_x}(w))^2 dw \le \frac{T^{2\beta}}{d_0^2} \int_{\mathbb{R}} (\hat{\mathbb{H}}_{n_x, n_y}(y) - \mathbb{H}_{n_x}^0(y))^2 dy + \frac{4}{\pi T}. \quad (60)$$

Moreover, it follows from the definition of $\hat{m}_{n_x, n_y}$ that

$$\begin{aligned} \int_{\mathbb{R}} (\hat{\mathbb{H}}_{n_x, n_y}(y) - \mathbb{H}_{n_x}^0(y))^2 dy &\le 2 \int_{\mathbb{R}} (\hat{\mathbb{H}}_{n_x, n_y}(y) - \mathbb{H}_{n_y}(y))^2 dy \\ &\quad + 2 \int_{\mathbb{R}} (\mathbb{H}_{n_x}^0(y) - \mathbb{H}_{n_y}(y))^2 dy \end{aligned}$$
Thus,

$$W_1(\hat{\mathbb{L}}_n, \mathbb{L}_n) = O_P(n^{-1/2}).$$

Using again Proposition 9 in Appendix C the latter rate yields

$$\int_{\mathbb{R}} |\hat{\mathbb{L}}_n(y) - \mathbb{L}_n(y)| dy = O_P(n^{-1/2}).$$

Combining this with Proposition 4 and the Cauchy-Schwarz inequality, we obtain

$$\begin{aligned} \int_{\mathbb{R}} |\hat{\mathbb{L}}_n^0(y) - L_0(y)| dy &= \int_{-K_0-2}^{K_0+2} |\hat{\mathbb{L}}_n^0(y) - L_0(y)| dy \\ &= O_P(n^{-1/2}). \end{aligned}$$

Using the result of Lemma 4, it follows that

$$\begin{aligned} \int_0^1 |\hat{m}_n(x) - m_0(x)| dF_0(x) &\le \int_{m_0(0) \wedge \hat{m}_n(0)}^{m_0(1) \vee \hat{m}_n(1)} |F_0 \circ \hat{m}_n^{-1}(y) - F_0 \circ m_0^{-1}(y)| dy \\ &= \int_{m_0(0) \wedge \hat{m}_n(0)}^{m_0(1) \vee \hat{m}_n(1)} |\hat{\mathbb{L}}_n^0(y) - L_0(y)| dy \\ &\le \int_{-K_0-2}^{K_0+2} |\hat{\mathbb{L}}_n^0(y) - L_0(y)| dy \end{aligned}$$

implying that $\int_0^1 |\hat{m}_n(x) - m_0(x)| dF_0(x) = O_P(n^{-1/2})$. $\square$

**Proof of Theorem 4.** *Proof of Claim 1.* We start with the case where the noise has an absolutely continuous density. We restrict attention to the ordinary smooth case because the arguments are very similar in the supersmooth one. Now, since the proof in the ordinary smooth case follows the same lines of the proof of Theorem 1, details are omitted and the reader is referred to the latter proof. Here, we only point out the main existing differences. Similar to (14) and (52), we define

$$\hat{\mathbb{L}}_{n_x, n_y}(w) := \frac{1}{n_x} \sum_{i=1}^{n_x} \mathbb{1}_{\{\hat{m}_{n_x, n_y}(X_i) \le w\}} \quad \text{and} \quad \mathbb{L}_{n_x}(w) := \frac{1}{n_x} \sum_{i=1}^{n_x} \mathbb{1}_{\{m_0(X_i) \le w\}}$$

for all $w \in \mathbb{R}$, and

$$\hat{\mathbb{H}}_{n_x, n_y}(y) = \frac{1}{n_x} \sum_{i=1}^{n_x} \Phi_{\epsilon}(y - \hat{m}_{n_x, n_y}(X_i)), \quad \text{and} \quad \mathbb{H}_{n_x}^0(y) = \frac{1}{n_x} \sum_{i=1}^{n_x} \Phi_{\epsilon}(y - m_0(X_i))$$

for all $y \in \mathbb{R}$. With similar arguments as for the proof of (54) we obtain that for all sufficiently large $T$,

$$\int_{\mathbb{R}} (\hat{\mathbb{L}}_{n_x, n_y}(w) - \mathbb{L}_{n_x}(w))^2 dw \le \frac{T^{2\beta}}{d_0^2} \int_{\mathbb{R}} (\hat{\mathbb{H}}_{n_x, n_y}(y) - \mathbb{H}_{n_x}^0(y))^2 dy + \frac{4}{\pi T}. \quad (60)$$

Moreover, it follows from the definition of $\hat{m}_{n_x, n_y}$ that

$$\begin{aligned} \int_{\mathbb{R}} (\hat{\mathbb{H}}_{n_x, n_y}(y) - \mathbb{H}_{n_x}^0(y))^2 dy &\le 2 \int_{\mathbb{R}} (\hat{\mathbb{H}}_{n_x, n_y}(y) - \mathbb{H}_{n_y}(y))^2 dy \\ &\quad + 2 \int_{\mathbb{R}} (\mathbb{H}_{n_x}^0(y) - \mathbb{H}_{n_y}(y))^2 dy \end{aligned}$$
which is less than or equal to

$$4 \int_{\mathbb{R}} (\mathbb{H}_{n_x}^0(y) - \mathbb{H}_{n_y}(y))^2 dy \le 8 \int_{\mathbb{R}} (\mathbb{H}_{n_x}^0(y) - H_0(y))^2 dy + 8 \int_{\mathbb{R}} (\mathbb{H}_{n_y}(y) - H_0(y))^2 dy$$

where

$$E[(\mathbb{H}_{n_y}(y) - H_0(y))^2] = n_y^{-1} H_0(y)(1 - H_0(y)),$$

and

$$\begin{aligned} E[(\mathbb{H}_{n_x}^0(y) - H_0(y))^2] &= \frac{1}{n_x} \mathrm{Var} \Phi_{\epsilon}(y - m_0(X)) \\ &= \frac{1}{n_x} \int_{\mathbb{R}} (\Phi_{\epsilon}(y - m_0(x)) - H_0(y))^2 dF_0(x). \end{aligned}$$

The integrals $I_1$ and $I_2$ defined in (56) and (57) are finite, since it can be shown that $\int_{-\infty}^0 H_0(y)dy < \infty$ and $\int_0^\infty (1 - H_0(y))dy < \infty$; see Appendix A. Hence, we obtain that

$$\int_{\mathbb{R}} E \left( \hat{\mathbb{H}}_{n_x, n_y}(y) - \mathbb{H}_{n_x}^0(y) \right)^2 dy = O\left( (n_x \wedge n_y)^{-1} \right). \quad (61)$$

Combining this with (60) proves that for all sufficiently large $T$,

$$\int_{\mathbb{R}} E \left( \hat{\mathbb{L}}_{n_x, n_y}(w) - \mathbb{L}_{n_x}(w) \right)^2 dw \le T^{2\beta} O\left( (n_x \wedge n_y)^{-1} \right) + \frac{4}{\pi T}.$$

For $T \sim (n_x \wedge n_y)^{1/(2\beta+1)}$ we get

$$\int_{\mathbb{R}} E \left( \hat{\mathbb{L}}_{n_x, n_y}(w) - \mathbb{L}_{n_x}(w) \right)^2 dw \le O \left( \frac{1}{(n_x \wedge n_y)^{1/(2\beta+1)}} \right), \quad (62)$$

which proves an analogue of Proposition 3 in the case of possibly unequal sample sizes.

Next, we consider an analogue of Proposition 4. For this task, we denote by $\mathbb{P}_{n_x}^X$ and $\mathbb{P}^X$ the empirical probability measure associated with $X_1, \dots, X_{n_x}$ and the true corresponding probability measure. We consider the empirical processes

$$\hat{\mathbb{L}}_{n_x, n_y}(w) - \hat{\mathbb{L}}_{n_x, n_y}^0(w) = (\mathbb{P}_{n_x}^X - \mathbb{P}^X) \mathbb{1}_{\{\hat{m}_{n_x, n_y}(\cdot) \le w\}}$$

and

$$\mathbb{L}_{n_x}(w) - L_0(w) = (\mathbb{P}_{n_x}^X - \mathbb{P}^X) \mathbb{1}_{\{m_0(\cdot) \le w\}};$$

where

$$\hat{\mathbb{L}}_{n_x, n_y}^0(w) = \int \mathbb{1}_{\{\hat{m}_{n_x, n_y}(x) \le w\}} dF_0(x)$$

and $L_0$ is defined in (15). Then, with similar arguments as in the proof of Proposition 4, we obtain that for all random variables $A < B$ (that may depend on $n$) it holds that

$$\int_A^B (\hat{\mathbb{L}}_{n_x, n_y}(w) - \hat{\mathbb{L}}_{n_x, n_y}^0(w))^2 dw \le (B-A)O_P(1/n_x) \le (B-A)O_P(1/(n_x \wedge n_y)),$$

$$\int_A^B (\mathbb{L}_{n_x}(w) - L_0(w))^2 dw \le (B-A)O_P(1/n_x) \le (B-A)O_P(1/(n_x \wedge n_y)),$$
which is less than or equal to

$$4 \int_{\mathbb{R}} (\mathbb{H}_{n_x}^0(y) - \mathbb{H}_{n_y}(y))^2 dy \le 8 \int_{\mathbb{R}} (\mathbb{H}_{n_x}^0(y) - H_0(y))^2 dy + 8 \int_{\mathbb{R}} (\mathbb{H}_{n_y}(y) - H_0(y))^2 dy$$

where

$$E[(\mathbb{H}_{n_y}(y) - H_0(y))^2] = n_y^{-1} H_0(y)(1 - H_0(y)),$$

and

$$\begin{aligned} E[(\mathbb{H}_{n_x}^0(y) - H_0(y))^2] &= \frac{1}{n_x} \mathrm{Var} \Phi_{\epsilon}(y - m_0(X)) \\ &= \frac{1}{n_x} \int_{\mathbb{R}} (\Phi_{\epsilon}(y - m_0(x)) - H_0(y))^2 dF_0(x). \end{aligned}$$

The integrals $I_1$ and $I_2$ defined in (56) and (57) are finite, since it can be shown that $\int_{-\infty}^{0} H_0(y)dy < \infty$ and $\int_{0}^{\infty} (1 - H_0(y))dy < \infty$; see Appendix A. Hence, we obtain that

$$\int_{\mathbb{R}} E \left( \hat{\mathbb{H}}_{n_x, n_y}(y) - \mathbb{H}_{n_x}^0(y) \right)^2 dy = O\left( (n_x \wedge n_y)^{-1} \right). \quad (61)$$

Combining this with (60) proves that for all sufficiently large $T$,

$$\int_{\mathbb{R}} E \left( \hat{\mathbb{L}}_{n_x, n_y}(w) - \mathbb{L}_{n_x}(w) \right)^2 dw \le T^{2\beta} O\left( (n_x \wedge n_y)^{-1} \right) + \frac{4}{\pi T}.$$

For $T \sim (n_x \wedge n_y)^{1/(2\beta+1)}$ we get

$$\int_{\mathbb{R}} E \left( \hat{\mathbb{L}}_{n_x, n_y}(w) - \mathbb{L}_{n_x}(w) \right)^2 dw \le O \left( \frac{1}{(n_x \wedge n_y)^{1/(2\beta+1)}} \right), \quad (62)$$

which proves an analogue of Proposition 3 in the case of possibly unequal sample sizes.

Next, we consider an analogue of Proposition 4. For this task, we denote by $\mathbb{P}_{n_x}^X$ and $\mathbb{P}^X$ the empirical probability measure associated with $X_1, \dots, X_{n_x}$ and the true corresponding probability measure. We consider the empirical processes

$$\hat{\mathbb{L}}_{n_x, n_y}(w) - \hat{\mathbb{L}}_{n_x, n_y}^0(w) = (\mathbb{P}_{n_x}^X - \mathbb{P}^X) \mathbb{1}_{\{\hat{m}_{n_x, n_y}(\cdot) \le w\}}$$

and

$$\mathbb{L}_{n_x}(w) - L_0(w) = (\mathbb{P}_{n_x}^X - \mathbb{P}^X) \mathbb{1}_{\{m_0(\cdot) \le w\}};$$

where

$$\hat{\mathbb{L}}_{n_x, n_y}^0(w) = \int \mathbb{1}_{\{\hat{m}_{n_x, n_y}(x) \le w\}} dF_0(x)$$

and $L_0$ is defined in (15). Then, with similar arguments as in the proof of Proposition 4, we obtain that for all random variables $A < B$ (that may depend on $n$) it holds that

$$\int_A^B (\hat{\mathbb{L}}_{n_x, n_y}(w) - \hat{\mathbb{L}}_{n_x, n_y}^0(w))^2 dw \le (B - A)O_P(1/n_x) \le (B - A)O_P(1/(n_x \wedge n_y)),$$

$$\int_A^B (\mathbb{L}_{n_x}(w) - L_0(w))^2 dw \le (B - A)O_P(1/n_x) \le (B - A)O_P(1/(n_x \wedge n_y)),$$
where $O_P(1/n_x)$ is uniform in A and B. Moreover, if $B - A = O_P((n_x \wedge n_y)^{2\beta/(2\beta+1)})$, then

$$ \int_A^B (\hat{L}_{n_x, n_y}^0(w) - L_0(w))^2 dw = O_P((n_x \wedge n_y)^{-1/(2\beta+1)}). $$

Next, similar to Proposition 5 we obtain that for all random variables $A < B$ such that $B - A = O_P(1)$ it holds that

$$ \begin{aligned} \int_A^B (F_0 \circ \hat{m}_{n_x, n_y}^{-1}(w) - F_0 \circ m_0^{-1}(w))^2 dw &= \int_A^B (\hat{L}_n^0(w) - L_0(w))^2 dw \\ &= O_P((n_x \wedge n_y)^{-1/(2\beta+1)}). \end{aligned} $$

Proposition 6 still holds in the case of possibly different sample sizes with $\hat{m}_n$ replaced by $\hat{m}_{n_x, n_y}$. If (12) also holds, then $A_n$ and $B_n$ from Proposition 6 are both of the order $O_P(1)$. In that case, the second assertion in Theorem 4 is an immediate consequence of Proposition 6 combined with the preceding display. Hence, it remains to prove that (12) holds. It follows from the definition of $\hat{\mathbb{L}}_{n_x, n_y}$ and $\mathbb{L}_{n_x}$ together with the Cauchy-Schwarz inequality and (62) that

$$ \int_{\|m_0\|_{\infty}}^{2\|m_0\|_{\infty}} \left( 1 - \frac{1}{n_x} \sum_{i=1}^{n_x} \mathbb{1}_{\{\hat{m}_{n_x, n_y}(X_i) \le w\}} \right) dw = \int_{\|m_0\|_{\infty}}^{2\|m_0\|_{\infty}} \left| \hat{\mathbb{L}}_{n_x, n_y}(w) - \mathbb{L}_{n_x}(w) \right| dw $$

which is bounded above by

$$ \sqrt{\left( \|m_0\|_{\infty} \int_{\mathbb{R}} \left( \hat{\mathbb{L}}_{n_x, n_y}(w) - \mathbb{L}_{n_x}(w) \right)^2 dw \right)} = o_P(1). $$

On the other hand,

$$ \int_{\|m_0\|_{\infty}}^{2\|m_0\|_{\infty}} \left( 1 - \frac{1}{n_x} \sum_{i=1}^{n_x} \mathbb{1}_{\{\hat{m}_{n_x, n_y}(X_i) \le w\}} \right) dw \ge \|m_0\|_{\infty} n_x^{-1} \sum_{i=1}^{n_x} \mathbb{1}_{\{\hat{m}_{n_x, n_y}(X_i) > 2\|m_0\|_{\infty}\}} $$

and therefore,

$$ \sum_{i=1}^{n_x} \mathbb{1}_{\{\hat{m}_{n_x, n_y}(X_i) > 2\|m_0\|_{\infty}\}} = o_P(n_x). $$

By monotonicity of $\hat{m}_{n_x, n_y}$ this implies that

$$ \mathbb{1}_{\{\hat{m}_{n_x, n_y}(b) > 2\|m_0\|_{\infty}\}} \sum_{i=1}^{n_x} \mathbb{1}_{\{X_i > b\}} \le \sum_{i=1}^{n_x} \mathbb{1}_{\{\hat{m}_{n_x, n_y}(X_i) > 2\|m_0\|_{\infty}\}} = o_P(n_x). $$

By the law of large numbers, $n_x^{-1} \sum_{i=1}^{n_x} \mathbb{1}_{\{X_i > b\}}$ converges in probability to $1 - F_0(b) > 0$ and therefore, it follows from the previous display that

$$ \mathbb{1}_{\{\hat{m}_{n_x, n_y}(b) > 2\|m_0\|_{\infty}\}} = o_P(1). $$

This implies that

$$ \lim_{n \to \infty} P(\hat{m}_{n_x, n_y}(b) > 2\|m_0\|_{\infty}) = 0. $$
where $O_P(1/n_x)$ is uniform in A and B. Moreover, if $B - A = O_P((n_x \wedge n_y)^{2\beta/(2\beta+1)})$, then

$$ \int_A^B (\hat{L}_{n_x, n_y}^0(w) - L_0(w))^2 dw = O_P((n_x \wedge n_y)^{-1/(2\beta+1)}). $$

Next, similar to Proposition 5 we obtain that for all random variables $A < B$ such that $B - A = O_P(1)$ it holds that

$$ \begin{aligned} \int_A^B (F_0 \circ \hat{m}_{n_x, n_y}^{-1}(w) - F_0 \circ m_0^{-1}(w))^2 dw &= \int_A^B (\hat{L}_n^0(w) - L_0(w))^2 dw \\ &= O_P((n_x \wedge n_y)^{-1/(2\beta+1)}). \end{aligned} $$

Proposition 6 still holds in the case of possibly different sample sizes with $\hat{m}_n$ replaced by $\hat{m}_{n_x, n_y}$. If (12) also holds, then $A_n$ and $B_n$ from Proposition 6 are both of the order $O_P(1)$. In that case, the second assertion in Theorem 4 is an immediate consequence of Proposition 6 combined with the preceding display. Hence, it remains to prove that (12) holds. It follows from the definition of $\hat{L}_{n_x, n_y}$ and $L_{n_x}$ together with the Cauchy-Schwarz inequality and (62) that

$$ \int_{\|m_0\|_{\infty}}^{2\|m_0\|_{\infty}} \left( 1 - \frac{1}{n_x} \sum_{i=1}^{n_x} \mathbb{1}_{\{\hat{m}_{n_x, n_y}(X_i) \le w\}} \right) dw = \int_{\|m_0\|_{\infty}}^{2\|m_0\|_{\infty}} \left| \hat{L}_{n_x, n_y}(w) - L_{n_x}(w) \right| dw $$

which is bounded above by

$$ \sqrt{\left( \|m_0\|_{\infty} \int_{\mathbb{R}} \left( \hat{L}_{n_x, n_y}(w) - L_{n_x}(w) \right)^2 dw \right)} = o_P(1). $$

On the other hand,

$$ \int_{\|m_0\|_{\infty}}^{2\|m_0\|_{\infty}} \left( 1 - \frac{1}{n_x} \sum_{i=1}^{n_x} \mathbb{1}_{\{\hat{m}_{n_x, n_y}(X_i) \le w\}} \right) dw \ge \|m_0\|_{\infty} n_x^{-1} \sum_{i=1}^{n_x} \mathbb{1}_{\{\hat{m}_{n_x, n_y}(X_i) > 2\|m_0\|_{\infty}\}} $$

and therefore,

$$ \sum_{i=1}^{n_x} \mathbb{1}_{\{\hat{m}_{n_x, n_y}(X_i) > 2\|m_0\|_{\infty}\}} = o_P(n_x). $$

By monotonicity of $\hat{m}_{n_x, n_y}$ this implies that

$$ \mathbb{1}_{\{\hat{m}_{n_x, n_y}(b) > 2\|m_0\|_{\infty}\}} \sum_{i=1}^{n_x} \mathbb{1}_{\{X_i > b\}} \le \sum_{i=1}^{n_x} \mathbb{1}_{\{\hat{m}_{n_x, n_y}(X_i) > 2\|m_0\|_{\infty}\}} = o_P(n_x). $$

By the law of large numbers, $n_x^{-1} \sum_{i=1}^{n_x} \mathbb{1}_{\{X_i > b\}}$ converges in probability to $1 - F_0(b) > 0$ and therefore, it follows from the previous display that

$$ \mathbb{1}_{\{\hat{m}_{n_x, n_y}(b) > 2\|m_0\|_{\infty}\}} = o_P(1). $$

This implies that

$$ \lim_{n \to \infty} P(\hat{m}_{n_x, n_y}(b) > 2 \|m_0\|_{\infty}) = 0. $$
One can prove similarly that

$$ \lim_{n \to \infty} P(\hat{m}_{n_x, n_y}(a) < -2\|m_0\|_{\infty}) = 0. $$

This implies (12) by monotonicity of $\hat{m}_n$.

**Proof of Claim 2.** Now, we turn to the case where $\epsilon$ is supported on a finite set of points. Without loss of generality we assume that $\|\epsilon\|_{\infty} \le 1$ with probability 1. The same proof of Claim 4 in Proposition 2 can be again used to show existence with probability 1 of an estimator $\hat{m}_{n_x, n_y}$ which is bounded in the sup-norm by $K_0 + 2$. Now, the rate obtained above in (61) and the Cauchy-Schwarz inequality allow us to write that

$$ \begin{aligned} \int_{\mathbb{R}} |\hat{\mathbb{H}}_{n_x, n_y}(y) - \mathbb{H}_{n_x}^0(y)| dy &= \int_{-K_0-3}^{K_0+3} |\hat{\mathbb{H}}_{n_x, n_y}(y) - \mathbb{H}_{n_x}^0(y)| dy \\ &= O_P((n_x \wedge n_y)^{-1/2}). \end{aligned} $$

Using the same arguments as in the proof of Theorem 3 (in particular Proposition 9 in Appendix C) this implies that $W_1(\hat{\mathbb{H}}_{n_x, n_y}, \mathbb{H}_{n_x}^0) = O_P((n_x \wedge n_y)^{-1/2})$. Since $\hat{\mathbb{H}}_{n_x, n_y}(y) = \int_{\mathbb{R}} \hat{\mathbb{L}}_{n_x, n_y}(y-t)d\Phi_{\epsilon}(t)$ and $\mathbb{H}_{n_x}^0(y) = \int_{\mathbb{R}} \mathbb{L}_{n_x}(y-t)d\Phi_{\epsilon}(t)$, we can use again Proposition 10 in Appendix C to find a constant $D > 0$ depending only on the distribution of $\epsilon$ and $K_0$ such that

$$ W_1(\hat{\mathbb{L}}_{n_x, n_y}, \mathbb{L}_{n_x}) \le D W_1(\hat{\mathbb{H}}_{n_x, n_y}, \mathbb{H}_{n_x}^0). $$

implying that

$$ \int_{\mathbb{R}} |\hat{\mathbb{L}}_{n_x, n_y}(w) - \mathbb{L}_{n_x}(w)| dw = \int_{-K_0-2}^{K_0+2} |\hat{\mathbb{L}}_{n_x, n_y}(w) - \mathbb{L}_{n_x}(w)| dw = O_P((n_x \wedge n_y)^{-1/2}) $$

using again Proposition 2 of Meis and Mammen (2020). Therefore,

$$ \begin{aligned} \int_{-K_0-2}^{K_0+2} |\hat{\mathbb{L}}_{n_x, n_y}^0(w) - L_0(w)| dw &= \int_{-K_0-2}^{K_0+2} |F_0 \circ \hat{m}_{n_x, n_y}^{-1}(w) - F_0 \circ m_0^{-1}(w)| dw \\ &= O_P((n_x \wedge n_y)^{-1/2}) \end{aligned} $$

and hence

$$ \begin{aligned} \int_0^1 |\hat{m}_{n_x, n_y}(x) - m_0(x)| dF_0(x) &\le \int_{\hat{m}_{n_x, n_y}(0) \wedge m_0(0)}^{\hat{m}_{n_x, n_y}(1) \vee m_0(1)} |F_0 \circ \hat{m}_{n_x, n_y}^{-1}(w) - F_0 \circ m_0^{-1}(w)| dw \\ &\le \int_{-K_0-2}^{K_0+2} |F_0 \circ \hat{m}_{n_x, n_y}^{-1}(w) - F_0 \circ m_0^{-1}(w)| dw \\ &= O_P((n_x \wedge n_y)^{-1/2}), \end{aligned} $$

which completes the proof of Theorem 4. □

**Proof of Theorem 5.** Theorem 1, Theorem 2, or Theorem 3 imply that along any subsequence of $\{n\}_{n=1}^{\infty}$ we can find a further subsequence $\{n_i\}_{i=1}^{\infty}$ such that with probability 1

$$ \lim_{i \to \infty} \int_a^b |\hat{m}_{n_i}(x) - m_0(x)| dF_0(x) = 0. $$
One can prove similarly that

$$ \lim_{n \to \infty} P(\hat{m}_{n_x, n_y}(a) < -2 \|m_0\|_{\infty}) = 0. $$

This implies (12) by monotonicity of $\hat{m}_n$.

**Proof of Claim 2.** Now, we turn to the case where $\epsilon$ is supported on a finite set of points. Without loss of generality we assume that $\|\epsilon\|_{\infty} \le 1$ with probability 1. The same proof of Claim 4 in Proposition 2 can be again used to show existence with probability 1 of an estimator $\hat{m}_{n_x, n_y}$ which is bounded in the sup-norm by $K_0 + 2$. Now, the rate obtained above in (61) and the Cauchy-Schwarz inequality allow us to write that

$$ \begin{aligned} \int_{\mathbb{R}} |\hat{\mathbb{H}}_{n_x, n_y}(y) - \mathbb{H}_{n_x}^0(y)| dy &= \int_{-K_0-3}^{K_0+3} |\hat{\mathbb{H}}_{n_x, n_y}(y) - \mathbb{H}_{n_x}^0(y)| dy \\ &= O_P((n_x \wedge n_y)^{-1/2}). \end{aligned} $$

Using the same arguments as in the proof of Theorem 3 (in particular Proposition 9 in Appendix C) this implies that $W_1(\hat{\mathbb{H}}_{n_x, n_y}, \mathbb{H}_{n_x}^0) = O_P((n_x \wedge n_y)^{-1/2})$. Since $\hat{\mathbb{H}}_{n_x, n_y}(y) = \int_{\mathbb{R}} \hat{\mathbb{L}}_{n_x, n_y}(y-t)d\Phi_{\epsilon}(t)$ and $\mathbb{H}_{n_x}^0(y) = \int_{\mathbb{R}} \mathbb{L}_{n_x}(y-t)d\Phi_{\epsilon}(t)$, we can use again Proposition 10 in Appendix C to find a constant $D > 0$ depending only on the distribution of $\epsilon$ and $K_0$ such that

$$ W_1(\hat{\mathbb{L}}_{n_x, n_y}, \mathbb{L}_{n_x}) \le D W_1(\hat{\mathbb{H}}_{n_x, n_y}, \mathbb{H}_{n_x}^0). $$

implying that

$$ \int_{\mathbb{R}} |\hat{\mathbb{L}}_{n_x, n_y}(w) - \mathbb{L}_{n_x}(w)| dw = \int_{-K_0-2}^{K_0+2} |\hat{\mathbb{L}}_{n_x, n_y}(w) - \mathbb{L}_{n_x}(w)| dw = O_P((n_x \wedge n_y)^{-1/2}) $$

using again Proposition 2 of Meis and Mammen (2020). Therefore,

$$ \begin{aligned} \int_{-K_0-2}^{K_0+2} |\hat{\mathbb{L}}_{n_x, n_y}^0(w) - L_0(w)| dw &= \int_{-K_0-2}^{K_0+2} |F_0 \circ \hat{m}_{n_x, n_y}^{-1}(w) - F_0 \circ m_0^{-1}(w)| dw \\ &= O_P((n_x \wedge n_y)^{-1/2}) \end{aligned} $$

and hence

$$ \begin{aligned} \int_0^1 |\hat{m}_{n_x, n_y}(x) - m_0(x)| dF_0(x) &\le \int_{\hat{m}_{n_x, n_y}(0) \wedge m_0(0)}^{\hat{m}_{n_x, n_y}(1) \vee m_0(1)} |F_0 \circ \hat{m}_{n_x, n_y}^{-1}(w) - F_0 \circ m_0^{-1}(w)| dw \\ &\le \int_{-K_0-2}^{K_0+2} |F_0 \circ \hat{m}_{n_x, n_y}^{-1}(w) - F_0 \circ m_0^{-1}(w)| dw \\ &= O_P((n_x \wedge n_y)^{-1/2}), \end{aligned} $$

which completes the proof of Theorem 4. $\square$

<figure><img src="image_7.png" /></figure>

**Proof of Theorem 5.** Theorem 1, Theorem 2, or Theorem 3 imply that along any subsequence of $\{n\}_{n=1}^{\infty}$ we can find a further subsequence $\{n_i\}_{i=1}^{\infty}$ such that with probability 1

$$ \lim_{i \to \infty} \int_a^b |\hat{m}_{n_i}(x) - m_0(x)| dF_0(x) = 0. $$
Call this event $E_1$. For notational ease let $\hat{m}_{n_i} \equiv \hat{m}_i$. Further, by Corollary 2.3 of Stein and Shakarchi (2005) (stated for Lebesgue measure but the proof does not rely on the Lebesgue measure at all and the result holds for a general measure space), there exists another subsequence (which we call again $\{n_i\}_{i=1}^\infty$ for convenience) such that $\hat{m}_i$ converges $F_0$-a.e. to $m_0$.

Recall $C$ is a compact in the interior of $[a, b] \cap S_0$. Then since $m_0$ is continuous on $C$, $\hat{m}_i$ converges on a dense subset of $[a, b] \cap S_0$ to $m_0$ (for any points $\alpha, \beta \in [a, b] \cap S_0$, the $F_0$ measure of $(\alpha, \beta)$ is given by $F_0(\beta) - F_0(\alpha)$, so if $F_0(\beta) - F_0(\alpha) > 0$ then there must be a point of convergence in $(\alpha, \beta)$, since convergence is $F_0$-a.e.), and both $\hat{m}_i$ and $m_0$ are monotone, it follows that $\hat{m}_i$ converges pointwise on all of $C$ to $m_0$ (one can sandwich any point in $C$, including its boundary points, by sequences of points above and below at which $\hat{m}_i$ converges to $m_0$ and appeal to monotonicity).

And, we can strengthen the convergence to uniform convergence on $C$, since $m_0$ and $\hat{m}_i$ are monotone and $m_0$ is continuous on $C$, again, for any $\omega \in E_1$. The elementary proof is as follows. Fix $\epsilon > 0$. By uniform continuity of $m_0$ on (the compact) $[a, b]$, there exists $\delta > 0$ such that $|m_0(x) - m_0(y)| \le \epsilon$ for all $x, y$ such that $|x - y| \le \delta$. Cover $C$ with the set of open (in $C$'s subspace topology) sets $A(x, \delta) := \{y \in C : |x - y| < \delta/2\}$ for all $x \in C$, and extract by compactness a finite subcover of these open sets, $A(x_i, \delta)$, for $i = 1, \dots, N$. Let $x_{i1} := \inf A(x_i, \delta)$ and $x_{i2} := \sup A(x_i, \delta)$. Since $C$ is closed, $x_{ij} \in C$, $j = 1, 2$. By pointwise convergence $\hat{m}_n(x_{ij})$ is within $\epsilon$ of $m_0(x_{ij})$, $j = 1, 2$, for all $i$ and $n$ large enough. Now, take any $x \in C$; let $j$ be such that $x_{j1} \le x \le x_{j2}$. Using monotonicity of $\hat{m}_n$ and of $m_0$, we have for $n$ large enough that $\hat{m}_n(x) \le \hat{m}_n(x_{j2}) \le m_0(x_{j2}) + \epsilon \le m_0(x) + 2\epsilon$. Similarly, using $x_{j1}$, we have $\hat{m}_n(x) \ge m_0(x) - 2\epsilon$, which proves the uniform convergence on $C$.

Hence, for all $\omega \in E_1$, for all subsequences of $\{n_i\}_{i=1}^\infty$, we can find a further subsequence (depending on $\omega$) along which $\sup_{x \in C} |\hat{m}_i(x) - m_0(x)|$ converges to zero. Hence, for all $\omega \in E_1$, this supremum distance converges to zero along the subsequence $\{n_i\}_{i=1}^\infty$. Therefore,

$$P\left(\limsup_{i \to \infty} \sup_{x \in C} |\hat{m}_i(x) - m_0(x)| = 0\right) = 1.$$

Since for any subsequence of $\{n\}_{n=1}^\infty$, we have almost sure convergence along a subsequence, it follows that along the original sequence $\{n\}$

$$\sup_{x \in C} |\hat{m}_n(x) - m_0(x)| \to 0, \text{ in probability}$$

which completes the proof. $\square$

## E.4 Proofs for Section 4

We begin with an auxiliary lemma. Below, we use the same notation $\hat{H}_n$ and $\mathbb{H}_n^0$ as in (52). We recall that $H_0$ denotes the distribution function of $Y$. We use the same notation $P_n^X$ and $P^X$ as in the proof of Proposition 4 and denote, moreover, by $E^X$ the expectation corresponding to $P^X$.

**Lemma 5.** Let Assumptions A0 and A1 hold. Assume that $\epsilon$ is supported on $[-1; 1]$ and independent of $X$. For any solution $\hat{m}_n \in \mathcal{M}$ to (8) that is bounded in the sup-
Call this event $E_1$. For notational ease let $\hat{m}_{n_i} \equiv \hat{m}_i$. Further, by Corollary 2.3 of Stein and Shakarchi (2005) (stated for Lebesgue measure but the proof does not rely on the Lebesgue measure at all and the result holds for a general measure space), there exists another subsequence (which we call again $\{n_i\}_{i=1}^\infty$ for convenience) such that $\hat{m}_i$ converges $F_0$-a.e. to $m_0$.

Recall $C$ is a compact in the interior of $[a, b] \cap S_0$. Then since $m_0$ is continuous on $C$, $\hat{m}_i$ converges on a dense subset of $[a, b] \cap S_0$ to $m_0$ (for any points $\alpha, \beta \in [a, b] \cap S_0$, the $F_0$ measure of $(\alpha, \beta)$ is given by $F_0(\beta) - F_0(\alpha)$, so if $F_0(\beta) - F_0(\alpha) > 0$ then there must be a point of convergence in $(\alpha, \beta)$, since convergence is $F_0$-a.e.), and both $\hat{m}_i$ and $m_0$ are monotone, it follows that $\hat{m}_i$ converges pointwise on all of $C$ to $m_0$ (one can sandwich any point in $C$, including its boundary points, by sequences of points above and below at which $\hat{m}_i$ converges to $m_0$ and appeal to monotonicity).

And, we can strengthen the convergence to uniform convergence on $C$, since $m_0$ and $\hat{m}_i$ are monotone and $m_0$ is continuous on $C$, again, for any $\omega \in E_1$. The elementary proof is as follows. Fix $\epsilon > 0$. By uniform continuity of $m_0$ on (the compact) $[a, b]$, there exists $\delta > 0$ such that $|m_0(x) - m_0(y)| \le \epsilon$ for all $x, y$ such that $|x - y| \le \delta$. Cover $C$ with the set of open (in $C$'s subspace topology) sets $A(x, \delta) := \{y \in C : |x - y| < \delta/2\}$ for all $x \in C$, and extract by compactness a finite subcover of these open sets, $A(x_i, \delta)$, for $i = 1, \dots, N$. Let $x_{i1} := \inf A(x_i, \delta)$ and $x_{i2} := \sup A(x_i, \delta)$. Since $C$ is closed, $x_{ij} \in C$, $j = 1, 2$. By pointwise convergence $\hat{m}_n(x_{ij})$ is within $\epsilon$ of $m_0(x_{ij})$, $j = 1, 2$, for all $i$ and $n$ large enough. Now, take any $x \in C$; let $j$ be such that $x_{j1} \le x \le x_{j2}$. Using monotonicity of $\hat{m}_n$ and of $m_0$, we have for $n$ large enough that $\hat{m}_n(x) \le \hat{m}_n(x_{j2}) \le m_0(x_{j2}) + \epsilon \le m_0(x) + 2\epsilon$. Similarly, using $x_{j1}$, we have $\hat{m}_n(x) \ge m_0(x) - 2\epsilon$, which proves the uniform convergence on $C$.

Hence, for all $\omega \in E_1$, for all subsequences of $\{n_i\}_{i=1}^\infty$, we can find a further subsequence (depending on $\omega$) along which $\sup_{x \in C} |\hat{m}_i(x) - m_0(x)|$ converges to zero. Hence, for all $\omega \in E_1$, this supremum distance converges to zero along the subsequence $\{n_i\}_{i=1}^\infty$. Therefore,

$$P\left(\limsup_{i \to \infty} \sup_{x \in C} |\hat{m}_i(x) - m_0(x)| = 0\right) = 1.$$

Since for any subsequence of $\{n\}_{n=1}^\infty$, we have almost sure convergence along a subsequence, it follows that along the original sequence $\{n\}$

$$\sup_{x \in C} |\hat{m}_n(x) - m_0(x)| \to 0, \text{ in probability}$$

which completes the proof. $\square$

## E.4 Proofs for Section 4

We begin with an auxiliary lemma. Below, we use the same notation $\hat{H}_n$ and $\mathbb{H}_n^0$ as in (52). We recall that $H_0$ denotes the distribution function of $Y$. We use the same notation $P_n^X$ and $P^X$ as in the proof of Proposition 4 and denote, moreover, by $E^X$ the expectation corresponding to $P^X$.

**Lemma 5.** Let Assumptions A0 and A1 hold. Assume that $\epsilon$ is supported on $[-1; 1]$ and independent of $X$. For any solution $\hat{m}_n \in \mathcal{M}$ to (8) that is bounded in the sup-
norm by $K_0 + 2$, we then have

$$ \int_{\mathbb{R}} \{H_0(y) - E^X[\Phi_\epsilon(y - \hat{m}_n(X))]\}^2 dy \le 2 \int_{\mathbb{R}} \{\mathbb{H}_n(y) - \hat{\mathbb{H}}_n(y)\}^2 dy + O_P(n^{-1}) \\ = O_P(n^{-1}). \qquad (63) $$

Moreover,

$$ \left\{ \iint_{\mathbb{R}} (\Phi_{\epsilon}(y - m_0(x)) - \Phi_{\epsilon}(y - \hat{m}_n(x))) dy dF_0(x) \right\}^2 = O_P(n^{-1}). \qquad (64) $$

**Proof** [Proof of Lemma 5.] We can write

$$ n^{-1} \sum_{i=1}^{n} \Phi_{\epsilon}(y - \hat{m}_{n}(X)) - E^{X}[\Phi_{\epsilon}(y - \hat{m}_{n}(X))] = (\mathbb{P}_{n}^{X} - P^{X}) \Phi_{\epsilon}(y - \hat{m}_{n}(\cdot)). $$

For a fixed $y$, consider the class of non-decreasing functions

$$ \{x \mapsto -\Phi(y - m(x)), x \in [0, 1], m \text{ monotone non-decreasing and } \|m\|_{\infty} \le K_0 + 2\}. $$

Using entropy arguments as in the proof of Proposition 4 by replacing the class $\mathcal{I}$ with the one defined above we can show that for all random variables $A < B$ such that $B - A = O_P(1)$, it holds that

$$ \int_{A}^{B} \left\{ n^{-1} \sum_{i=1}^{n} \Phi_{\epsilon}(y - \hat{m}_{n}(X_{i})) - E^{X}[\Phi_{\epsilon}(y - \hat{m}_{n}(X))] \right\}^{2} dy = O_{P}(n^{-1}). $$

Since $\epsilon$ is supported on $[-1, 1]$, and $\hat{m}_n$ is assumed to be bounded in the sup-norm by $K_0 + 2$, the above integral over $[A, B]$ with $A = -K_0 - 3$ and $B = K_0 + 3$, is equal to the same integral over the whole real line $\mathbb{R}$. Hence, we get

$$ \int_{\mathbb{R}} \left\{ n^{-1} \sum_{i=1}^{n} \Phi_{\epsilon}(y - \hat{m}_{n}(X_{i})) - E^{X}[\Phi_{\epsilon}(y - \hat{m}_{n}(X))] \right\}^{2} dy = O_{P}(n^{-1}). \qquad (65) $$

On the other hand, with $H_0$ the distribution function of $Y$ we have

$$ \begin{aligned} E \int_{\mathbb{R}} (\mathbb{H}_n(y) - H_0(y))^2 dy &= \int_{\mathbb{R}} E(\mathbb{H}_n(y) - H_0(y))^2 dy \\ &= n^{-1} \int_{\mathbb{R}} H_0(y)(1 - H_0(y)) dy \end{aligned} $$

since $n\mathbb{H}_n(y)$ is a binomial random variable with parameter $n$ and probability of success $H_0(y)$. The integral on the right hand side is finite since $Y$ has bounded support (included in $[-K_0 - 1, K_0 + 1]$) and therefore,

$$ \int_{\mathbb{R}} (\mathbb{H}_n(y) - H_0(y))^2 dy = O_P(n^{-1}). $$

Combining this with (65) together with the fact that for all real numbers $a$ and $b$, we have $(a + b)^2 \le 2a^2 + 2b^2$, we conclude that

$$ \begin{aligned} & \int_{\mathbb{R}} \{H_0(y) - E^X[\Phi_\epsilon(y - \hat{m}_n(X))]\}^2 dy \\ & \le 2 \int_{\mathbb{R}} \{\mathbb{H}_n(y) - n^{-1} \sum_{i=1}^n \Phi_\epsilon(y - \hat{m}_n(X_i))\}^2 dy + O_P(n^{-1}). \end{aligned} $$
norm by $K_0 + 2$, we then have

$$ \int_{\mathbb{R}} \{H_0(y) - E^X[\Phi_\epsilon(y - \hat{m}_n(X))]\}^2 dy \le 2 \int_{\mathbb{R}} \{\mathbb{H}_n(y) - \hat{\mathbb{H}}_n(y)\}^2 dy + O_P(n^{-1}) \\ = O_P(n^{-1}). \qquad (63) $$

Moreover,

$$ \left\{ \iint_{\mathbb{R}} (\Phi_{\epsilon}(y - m_0(x)) - \Phi_{\epsilon}(y - \hat{m}_n(x))) dy dF_0(x) \right\}^2 = O_P(n^{-1}). \qquad (64) $$

**Proof** [Proof of Lemma 5.] We can write

$$ n^{-1} \sum_{i=1}^{n} \Phi_{\epsilon}(y - \hat{m}_{n}(X)) - E^{X}[\Phi_{\epsilon}(y - \hat{m}_{n}(X))] = (\mathbb{P}_{n}^{X} - P^{X}) \Phi_{\epsilon}(y - \hat{m}_{n}(\cdot)). $$

For a fixed $y$, consider the class of non-decreasing functions

$$ \{x \mapsto -\Phi(y - m(x)), x \in [0, 1], m \text{ monotone non-decreasing and } \|m\|_{\infty} \le K_0 + 2\}. $$

Using entropy arguments as in the proof of Proposition 4 by replacing the class $\mathcal{I}$ with the one defined above we can show that for all random variables $A < B$ such that $B - A = O_P(1)$, it holds that

$$ \int_{A}^{B} \left\{ n^{-1} \sum_{i=1}^{n} \Phi_{\epsilon}(y - \hat{m}_{n}(X_{i})) - E^{X}[\Phi_{\epsilon}(y - \hat{m}_{n}(X))] \right\}^{2} dy = O_{P}(n^{-1}). $$

Since $\epsilon$ is supported on $[-1, 1]$, and $\hat{m}_n$ is assumed to be bounded in the sup-norm by $K_0 + 2$, the above integral over $[A, B]$ with $A = -K_0 - 3$ and $B = K_0 + 3$, is equal to the same integral over the whole real line $\mathbb{R}$. Hence, we get

$$ \int_{\mathbb{R}} \left\{ n^{-1} \sum_{i=1}^{n} \Phi_{\epsilon}(y - \hat{m}_{n}(X_{i})) - E^{X}[\Phi_{\epsilon}(y - \hat{m}_{n}(X))] \right\}^{2} dy = O_{P}(n^{-1}). \qquad (65) $$

On the other hand, with $H_0$ the distribution function of $Y$ we have

$$ \begin{aligned} E \int_{\mathbb{R}} (\mathbb{H}_n(y) - H_0(y))^2 dy &= \int_{\mathbb{R}} E(\mathbb{H}_n(y) - H_0(y))^2 dy \\ &= n^{-1} \int_{\mathbb{R}} H_0(y)(1 - H_0(y)) dy \end{aligned} $$

since $n\mathbb{H}_n(y)$ is a binomial random variable with parameter $n$ and probability of success $H_0(y)$. The integral on the right hand side is finite since $Y$ has bounded support (included in $[-K_0 - 1, K_0 + 1]$) and therefore,

$$ \int_{\mathbb{R}} (\mathbb{H}_n(y) - H_0(y))^2 dy = O_P(n^{-1}). $$

Combining this with (65) together with the fact that for all real numbers $a$ and $b$, we have $(a + b)^2 \le 2a^2 + 2b^2$, we conclude that

$$ \begin{aligned} & \int_{\mathbb{R}} \left\{ H_0(y) - E^X[\Phi_\epsilon(y - \hat{m}_n(X))] \right\}^2 dy \\ & \le 2 \int_{\mathbb{R}} \left\{ \mathbb{H}_n(y) - n^{-1} \sum_{i=1}^n \Phi_\epsilon(y - \hat{m}_n(X_i)) \right\}^2 dy + O_P(n^{-1}). \end{aligned} $$
The first inequality follows by definition of $\hat{\mathbb{H}}_n$.

Now, it follows from the definition of $\hat{m}_n$ that

$$ \int_{\mathbb{R}} \{\mathbb{H}_n(y) - \hat{\mathbb{H}}_n(y)\}^2 dy \le \int_{\mathbb{R}} \{\mathbb{H}_n(y) - \mathbb{H}_n^0(y)\}^2 dy \\ \le 2 \int_{\mathbb{R}} \{\mathbb{H}_n(y) - H_0(y)\}^2 dy + 2 \int_{\mathbb{R}} \{\mathbb{H}_n^0(y) - H_0(y)\}^2 dy. $$

Since $\epsilon$ is independent of $X$, both $n\mathbb{H}_n(y)$ and $n\mathbb{H}_n^0(y)$ are the average of $n$ i.i.d. bounded random variables with mean $H_0(y)$ and therefore,

$$ E \int_{\mathbb{R}} \{\mathbb{H}_n(y) - \hat{\mathbb{H}}_n(y)\}^2 dy \le 2n^{-1} \int_{\mathbb{R}} \mathrm{Var}(1_{\{Y \le y\}}) dy + 2n^{-1} \int_{\mathbb{R}} \mathrm{Var}(\Phi_{\epsilon}(y - m_0(X))) dy. $$

Both integrals on the right-hand side are finite since the integrands are bounded and equal to zero for all $y \le -K_0 - 1$ and all $y \ge K_0 + 1$. Hence,

$$ \int_{\mathbb{R}} \{\mathbb{H}_n(y) - \hat{\mathbb{H}}_n(y)\}^2 dy = O_P(n^{-1}). $$

This completes the proof of (63).

Now, with $F_0$ the distribution function of $X$, it follows from the assumption that $\hat{m}_n$ (as well as $m_0$) is bounded in sup-norm by $K_0 + 2$ that

$$ \int_{\mathbb{R}} \{H_0(y) - E^X [\Phi_\epsilon(y - \hat{m}_n(X))]\}^2 dy \\ = \int_{\mathbb{R}} \left\{ \int \left( \Phi_\epsilon(y - m_0(x)) - \Phi_\epsilon(y - \hat{m}_n(x)) \right) dF_0(x) \right\}^2 dy \\ = \int_{-K_0-3}^{K_0+3} \left\{ \int \left( \Phi_\epsilon(y - m_0(x)) - \Phi_\epsilon(y - \hat{m}_n(x)) \right) dF_0(x) \right\}^2 dy $$

so it follows from the Jensen inequality and the Fubini theorem that

$$ \int_{\mathbb{R}} \{H_0(y) - E^X [\Phi_\epsilon(y - \hat{m}_n(X))]\}^2 dy \\ \ge \frac{1}{2K_0 + 6} \left\{ \int_{\mathbb{R}} \int \left( \Phi_\epsilon(y - m_0(x)) - \Phi_\epsilon(y - \hat{m}_n(x)) \right) dF_0(x) dy \right\}^2 \\ = \frac{1}{2K_0 + 6} \left\{ \int_{\mathbb{R}} \int \left( \Phi_\epsilon(y - m_0(x)) - \Phi_\epsilon(y - \hat{m}_n(x)) \right) dy dF_0(x) \right\}^2. $$

Combining this with (63) yields (64) and completes the proof of the lemma.

<figure><img src="image_7.png" /></figure>

### Proof of Theorem 6.

*Case of error with finite support:* Suppose that $\epsilon$ is supported on a finite set such that all the points of the support belong to $[-1, 1]$ and let $k \ge 2$. For any solution $\hat{m}_n$ to (8) which is bounded by $K_0 + 2$ (which exists with probability 1 in view of Proposition 2), it follows from Theorem 3 and the identity $a^k - b^k = (a-b)(a^{k-1}+a^{k-2}b+...+b^{k-1})$ that

$$ \left| \int_{0}^{1} (\hat{m}_{n}^{k}(x) - m_{0}^{k}(x)) dF_{0}(x) \right| \le k(K_{0} + 2)^{k-1} \left( \int_{0}^{1} |\hat{m}_{n}(x) - m_{0}(x)| dF_{0}(x) \right) \\ = O_{P}(1/\sqrt{n}). $$
The first inequality follows by definition of $\hat{\mathbb{H}}_n$.

Now, it follows from the definition of $\hat{m}_n$ that

$$ \int_{\mathbb{R}} \{\mathbb{H}_n(y) - \hat{\mathbb{H}}_n(y)\}^2 dy \le \int_{\mathbb{R}} \{\mathbb{H}_n(y) - \mathbb{H}_n^0(y)\}^2 dy \\ \le 2 \int_{\mathbb{R}} \{\mathbb{H}_n(y) - H_0(y)\}^2 dy + 2 \int_{\mathbb{R}} \{\mathbb{H}_n^0(y) - H_0(y)\}^2 dy. $$

Since $\epsilon$ is independent of $X$, both $n\mathbb{H}_n(y)$ and $n\mathbb{H}_n^0(y)$ are the average of $n$ i.i.d. bounded random variables with mean $H_0(y)$ and therefore,

$$ E \int_{\mathbb{R}} \{\mathbb{H}_n(y) - \hat{\mathbb{H}}_n(y)\}^2 dy \le 2n^{-1} \int_{\mathbb{R}} \mathrm{Var}(1_{\{Y \le y\}}) dy + 2n^{-1} \int_{\mathbb{R}} \mathrm{Var}(\Phi_{\epsilon}(y - m_0(X))) dy. $$

Both integrals on the right-hand side are finite since the integrands are bounded and equal to zero for all $y \le -K_0 - 1$ and all $y \ge K_0 + 1$. Hence,

$$ \int_{\mathbb{R}} \{\mathbb{H}_n(y) - \hat{\mathbb{H}}_n(y)\}^2 dy = O_P(n^{-1}). $$

This completes the proof of (63).

Now, with $F_0$ the distribution function of $X$, it follows from the assumption that $\hat{m}_n$ (as well as $m_0$) is bounded in sup-norm by $K_0 + 2$ that

$$ \begin{aligned} & \int_{\mathbb{R}} \{H_0(y) - E^X [\Phi_{\epsilon}(y - \hat{m}_n(X))]\}^2 dy \\ &= \int_{\mathbb{R}} \left\{ \int_{\mathbb{R}} (\Phi_{\epsilon}(y - m_0(x)) - \Phi_{\epsilon}(y - \hat{m}_n(x))) dF_0(x) \right\}^2 dy \\ &= \int_{-K_0 - 3}^{K_0 + 3} \left\{ \int_{\mathbb{R}} (\Phi_{\epsilon}(y - m_0(x)) - \Phi_{\epsilon}(y - \hat{m}_n(x))) dF_0(x) \right\}^2 dy \end{aligned} $$

so it follows from the Jensen inequality and the Fubini theorem that

$$ \begin{aligned} & \int_{\mathbb{R}} \{H_0(y) - E^X [\Phi_{\epsilon}(y - \hat{m}_n(X))]\}^2 dy \\ & \ge \frac{1}{2K_0 + 6} \left\{ \int_{\mathbb{R}} \int (\Phi_{\epsilon}(y - m_0(x)) - \Phi_{\epsilon}(y - \hat{m}_n(x))) dF_0(x) dy \right\}^2 \\ & = \frac{1}{2K_0 + 6} \left\{ \int_{\mathbb{R}} \int (\Phi_{\epsilon}(y - m_0(x)) - \Phi_{\epsilon}(y - \hat{m}_n(x))) dy dF_0(x) \right\}^2. \end{aligned} $$

Combining this with (63) yields (64) and completes the proof of the lemma. ■

### Proof of Theorem 6.

*Case of error with finite support:* Suppose that $\epsilon$ is supported on a finite set such that all the points of the support belong to $[-1, 1]$ and let $k \ge 2$. For any solution $\hat{m}_n$ to (8) which is bounded by $K_0 + 2$ (which exists with probability 1 in view of Proposition 2), it follows from Theorem 3 and the identity $a^k - b^k = (a-b)(a^{k-1}+a^{k-2}b+...+b^{k-1})$ that

$$ \begin{aligned} \left| \int_{0}^{1} (\hat{m}_{n}^{k}(x) - m_{0}^{k}(x)) dF_{0}(x) \right| & \le k(K_{0} + 2)^{k-1} \left( \int_{0}^{1} |\hat{m}_{n}(x) - m_{0}(x)| dF_{0}(x) \right) \\ & = O_{P}(1/\sqrt{n}). \end{aligned} $$
To show (20), it is enough to show that

$$ \int_0^1 \hat{m}_n^k(x) d(F_n(x) - F_0(x)) \equiv \frac{1}{\sqrt{n}} \mathbb{G}_n \hat{m}_n^k \quad (66) $$

is $O_P(1/\sqrt{n})$. Define

$$ \mathcal{M}_c = \{m : m \text{ non-decreasing on } [0, 1] \text{ and } \|m\|_{\infty} \le c\}, $$

and

$$ \mathcal{G}_{k,c} = \{m^k : m \in \mathcal{M}_c\}, $$

for $c > 0$. If $k = 1$, then $\hat{m}_n \in \mathcal{M}_{K_0+2}$ and it follows from similar arguments as in the proof of Proposition 4 that there exists $M > 0$ depending only on $K_0$ such that $\|\mathbb{G}_n\|_{\mathcal{M}_{K_0+2}} = O_P(1)$ which implies from (66) that (20) holds true. Now, suppose that $k \ge 2$. Using the decomposition $m^k = m^k 1_{m \ge 0} + m^k 1_{m < 0}$ we see that for any $m \in \mathcal{M}_{K_0+2}$, $m^k$ is either the sum or the difference (depending on whether $k$ is odd or even) of two functions in $\mathcal{M}_{(K_0+2)^k}$. Using Proposition 8 with $(\lambda_1, \lambda_2) = (1, -1)$ or $(\lambda_1, \lambda_2) = (1, 1)$, it follows that for any discrete measure $Q$ and $\delta > 0$

$$ \log N(\delta, \mathcal{G}_{k,K_0+2}, L_2(Q)) \le 2 \log N(\delta/2, \mathcal{M}_{(K_0+2)^k}, L_2(Q)). $$

Using (26) and similar arguments as in the proof of Proposition 4, we conclude again that $\|\mathbb{G}_n\|_{\mathcal{G}_{k,K_0+2}} = O_P(1)$. Together with (66), it follows that (20) holds true.

*Case of uniform error:* Now, we turn to the case where $\epsilon \sim U[-1, 1]$. To compute the integral on the left-hand side of (64), we distinguish between several cases. We recall that, because $\Phi_\epsilon$ is the distribution function of a uniformly distributed random variable over [-1, 1], $\Phi_\epsilon(t)$ is equal to 0 if $t < -1$, to 1 if $t > 1$, and to $(t+1)/2$ otherwise. For all $x$ such that $m_0(x) \le \hat{m}_n(x) \le m_0(x) + 2$ we have

$$ \Phi_{\epsilon}(y - m_0(x)) - \Phi_{\epsilon}(y - \hat{m}_n(x)) = \begin{cases} (y - m_0(x) + 1)/2 & \text{if } m_0(x) - 1 \le y \le \hat{m}_n(x) - 1 \\ (\hat{m}_n(x) - m_0(x))/2 & \text{if } \hat{m}_n(x) - 1 \le y \le m_0(x) + 1 \\ 1 - (y - \hat{m}_n(x) + 1)/2 & \text{if } m_0(x) + 1 \le y \le \hat{m}_n(x) + 1 \\ 0 & \text{otherwise.} \end{cases} $$

Hence, for all $x$ such that $m_0(x) \le \hat{m}_n(x) \le m_0(x) + 2$ we have

$$ \int_{\mathbb{R}} (\Phi_{\epsilon}(y - m_0(x)) - \Phi_{\epsilon}(y - \hat{m}_n(x))) dy \\ = \frac{1}{2}(\hat{m}_n(x) - m_0(x))^2 + \frac{1}{2}(\hat{m}_n(x) - m_0(x))(m_0(x) - \hat{m}_n(x) + 2) \\ = (\hat{m}_n(x) - m_0(x)). $$

Similarly, for all $x$ such that $m_0(x) + 2 < \hat{m}_n(x)$ we have

$$ \Phi_{\epsilon}(y - m_0(x)) - \Phi_{\epsilon}(y - \hat{m}_n(x)) = \begin{cases} (y - m_0(x) + 1)/2 & \text{if } m_0(x) - 1 \le y \le m_0(x) + 1 \\ 1 & \text{if } m_0(x) + 1 \le y \le \hat{m}_n(x) - 1 \\ 1 - (y - \hat{m}_n(x) + 1)/2 & \text{if } \hat{m}_n(x) - 1 \le y \le \hat{m}_n(x) + 1 \\ 0 & \text{otherwise,} \end{cases} $$
To show (20), it is enough to show that

$$ \int_0^1 \hat{m}_n^k(x) d(\mathbb{F}_n(x) - F_0(x)) \equiv \frac{1}{\sqrt{n}} \mathbb{G}_n \hat{m}_n^k \quad (66) $$

is $O_P(1/\sqrt{n})$. Define

$$ \mathcal{M}_c = \{m : m \text{ non-decreasing on } [0, 1] \text{ and } \|m\|_{\infty} \le c\}, $$

and

$$ \mathcal{G}_{k,c} = \{m^k : m \in \mathcal{M}_c\}, $$

for $c > 0$. If $k = 1$, then $\hat{m}_n \in \mathcal{M}_{K_0+2}$ and it follows from similar arguments as in the proof of Proposition 4 that there exists $M > 0$ depending only on $K_0$ such that $\|\mathbb{G}_n\|_{\mathcal{M}_{K_0+2}} = O_P(1)$ which implies from (66) that (20) holds true. Now, suppose that $k \ge 2$. Using the decomposition $m^k = m^k 1_{m \ge 0} + m^k 1_{m < 0}$ we see that for any $m \in \mathcal{M}_{K_0+2}$, $m^k$ is either the sum or the difference (depending on whether $k$ is odd or even) of two functions in $\mathcal{M}_{(K_0+2)^k}$. Using Proposition 8 with $(\lambda_1, \lambda_2) = (1, -1)$ or $(\lambda_1, \lambda_2) = (1, 1)$, it follows that for any discrete measure $Q$ and $\delta > 0$

$$ \log N(\delta, \mathcal{G}_{k,K_0+2}, L_2(Q)) \le 2 \log N(\delta/2, \mathcal{M}_{(K_0+2)^k}, L_2(Q)). $$

Using (26) and similar arguments as in the proof of Proposition 4, we conclude again that $\|\mathbb{G}_n\|_{\mathcal{G}_{k,K_0+2}} = O_P(1)$. Together with (66), it follows that (20) holds true.

*Case of uniform error:* Now, we turn to the case where $\epsilon \sim U[-1, 1]$. To compute the integral on the left-hand side of (64), we distinguish between several cases. We recall that, because $\Phi_\epsilon$ is the distribution function of a uniformly distributed random variable over $[-1, 1]$, $\Phi_\epsilon(t)$ is equal to 0 if $t < -1$, to 1 if $t > 1$, and to $(t+1)/2$ otherwise. For all $x$ such that $m_0(x) \le \hat{m}_n(x) \le m_0(x) + 2$ we have

$$ \Phi_{\epsilon}(y - m_0(x)) - \Phi_{\epsilon}(y - \hat{m}_n(x)) = \begin{cases} (y - m_0(x) + 1)/2 & \text{if } m_0(x) - 1 \le y \le \hat{m}_n(x) - 1 \\ (\hat{m}_n(x) - m_0(x))/2 & \text{if } \hat{m}_n(x) - 1 \le y \le m_0(x) + 1 \\ 1 - (y - \hat{m}_n(x) + 1)/2 & \text{if } m_0(x) + 1 \le y \le \hat{m}_n(x) + 1 \\ 0 & \text{otherwise.} \end{cases} $$

Hence, for all $x$ such that $m_0(x) \le \hat{m}_n(x) \le m_0(x) + 2$ we have

$$ \int_{\mathbb{R}} (\Phi_{\epsilon}(y - m_0(x)) - \Phi_{\epsilon}(y - \hat{m}_n(x))) dy \\ = \frac{1}{2}(\hat{m}_n(x) - m_0(x))^2 + \frac{1}{2}(\hat{m}_n(x) - m_0(x))(m_0(x) - \hat{m}_n(x) + 2) \\ = (\hat{m}_n(x) - m_0(x)). $$

Similarly, for all $x$ such that $m_0(x) + 2 < \hat{m}_n(x)$ we have

$$ \Phi_{\epsilon}(y - m_0(x)) - \Phi_{\epsilon}(y - \hat{m}_n(x)) = \begin{cases} (y - m_0(x) + 1)/2 & \text{if } m_0(x) - 1 \le y \le m_0(x) + 1 \\ 1 & \text{if } m_0(x) + 1 \le y \le \hat{m}_n(x) - 1 \\ 1 - (y - \hat{m}_n(x) + 1)/2 & \text{if } \hat{m}_n(x) - 1 \le y \le \hat{m}_n(x) + 1 \\ 0 & \text{otherwise,} \end{cases} $$
which implies that

$$ \int_{\mathbb{R}} (\Phi_{\epsilon}(y - m_0(x)) - \Phi_{\epsilon}(y - \hat{m}_n(x))) dy = (\hat{m}_n(x) - m_0(x)). $$

Hence,

$$ \int 1_{\{m_0(x) \le \hat{m}_n(x)\}} \int_{\mathbb{R}} (\Phi_{\epsilon}(y - m_0(x)) - \Phi_{\epsilon}(y - \hat{m}_n(x))) dy dF_0(x) \\ = \int 1_{\{m_0(x) \le \hat{m}_n(x)\}} (\hat{m}_n(x) - m_0(x)) dF_0(x) $$

Similarly,

$$ \int 1_{\{\hat{m}_n(x) \le m_0(x)\}} \int_{\mathbb{R}} (\Phi_{\epsilon}(y - m_0(x)) - \Phi_{\epsilon}(y - \hat{m}_n(x))) dy dF_0(x) \\ = \int 1_{\{\hat{m}_n(x) \le m_0(x)\}} (\hat{m}_n(x) - m_0(x)) dF_0(x) $$

Combining the two previous displays yields

$$ \int \int_{\mathbb{R}} (\Phi_{\epsilon}(y - m_0(x)) - \Phi_{\epsilon}(y - \hat{m}_n(x))) dy dF_0(x) \\ = \int (\hat{m}_n(x) - m_0(x)) dF_0(x). $$

Now, from (64) it follows that

$$ \left| \int_{0}^{1} (\hat{m}_{n}(x) - m_{0}(x)) dF_{0}(x) \right| = O_{P}(1/\sqrt{n}). $$

As we already know that $\int_{0}^{1} \hat{m}_{n}(x) d(F_{n}(x) - F_{0}(x)) = O_{P}(1/\sqrt{n})$, the second claim of the proposition now follows. $\square$

## Appendix F. Proof of Proposition 7

To make the notation less cumbersome, we write in the following $Z_i$ for the i-th order statistic $X_{(i)}$. Suppose that $\hat{m}_n$ takes at least two distinct values and let $1 \le j < j' \le n$ be such that $\hat{m}_n$ is constant on $[Z_j, Z_{j'})$, where $Z_j < Z_{j'}$ are two successive jump points of $\hat{m}_n$. Consider the function $m_\delta$ which is right-continuous, constant between the order statistics $Z_1, \dots, Z_n$, and

$$ m_{\delta}(Z_i) = \begin{cases} \hat{m}_n(Z_i) + \delta, & i \in \{j, \dots, j' - 1\} \\ \hat{m}_n(Z_i), & \text{otherwise.} \end{cases} $$

Then, the function $m_\delta$ as defined above belongs to $\mathcal{M}$, provided that $|\delta|$ is small enough. It follows from the definition (8) that $M_n(m_\delta) \ge M_n(\hat{m}_n)$. Using Taylor expansion of $\Phi_\epsilon$ with the integral remainder term we can write that for $i \in \{j, \dots, j' - 1\}$

$$ \Phi_{\epsilon}(y - \hat{m}_{n}(Z_{i}) - \delta) = \Phi_{\epsilon}(y - \hat{m}_{n}(Z_{i})) - \delta f_{\epsilon}(y - \hat{m}_{n}(Z_{i})) + R_{\delta,i}(y) $$
which implies that

$$ \int_{\mathbb{R}} (\Phi_{\epsilon}(y - m_0(x)) - \Phi_{\epsilon}(y - \hat{m}_n(x))) dy = (\hat{m}_n(x) - m_0(x)). $$

Hence,

$$ \int_1 \int_{\{m_0(x) \le \hat{m}_n(x)\}} (\Phi_{\epsilon}(y - m_0(x)) - \Phi_{\epsilon}(y - \hat{m}_n(x))) dy dF_0(x) \\ = \int_1 \int_{\{m_0(x) \le \hat{m}_n(x)\}} (\hat{m}_n(x) - m_0(x)) dF_0(x) $$

Similarly,

$$ \int_1 \int_{\{\hat{m}_n(x) \le m_0(x)\}} (\Phi_{\epsilon}(y - m_0(x)) - \Phi_{\epsilon}(y - \hat{m}_n(x))) dy dF_0(x) \\ = \int_1 \int_{\{\hat{m}_n(x) \le m_0(x)\}} (\hat{m}_n(x) - m_0(x)) dF_0(x) $$

Combining the two previous displays yields

$$ \int_1 \int_\mathbb{R} (\Phi_\epsilon(y - m_0(x)) - \Phi_\epsilon(y - \hat{m}_n(x))) dy dF_0(x) \\ = \int_1 (\hat{m}_n(x) - m_0(x)) dF_0(x). $$

Now, from (64) it follows that

$$ \left| \int_0^1 (\hat{m}_n(x) - m_0(x)) dF_0(x) \right| = O_P(1/\sqrt{n}). $$

As we already know that $\int_0^1 \hat{m}_n(x)d(F_n(x) - F_0(x)) = O_P(1/\sqrt{n})$, the second claim of the proposition now follows. $\square$

## Appendix F. Proof of Proposition 7

To make the notation less cumbersome, we write in the following $Z_i$ for the $i$-th order statistic $X_{(i)}$. Suppose that $\hat{m}_n$ takes at least two distinct values and let $1 \le j < j' \le n$ be such that $\hat{m}_n$ is constant on $[Z_j, Z_{j'})$, where $Z_j < Z_{j'}$ are two successive jump points of $\hat{m}_n$. Consider the function $m_\delta$ which is right-continuous, constant between the order statistics $Z_1, \dots, Z_n$, and

$$ m_\delta(Z_i) = \begin{cases} \hat{m}_n(Z_i) + \delta, & i \in \{j, \dots, j' - 1\} \\ \hat{m}_n(Z_i), & \text{otherwise.} \end{cases} $$

Then, the function $m_\delta$ as defined above belongs to $\mathcal{M}$, provided that $|\delta|$ is small enough. It follows from the definition (8) that $M_n(m_\delta) \ge M_n(\hat{m}_n)$. Using Taylor expansion of $\Phi_\epsilon$ with the integral remainder term we can write that for $i \in \{j, \dots, j' - 1\}$

$$ \Phi_{\epsilon}(y - \hat{m}_{n}(Z_{i}) - \delta) = \Phi_{\epsilon}(y - \hat{m}_{n}(Z_{i})) - \delta f_{\epsilon}(y - \hat{m}_{n}(Z_{i})) + R_{\delta,i}(y) $$
where the remainder term $R_{\delta,i}$ is given below. Hence,

$$
\begin{align}
0 &\le \mathbb{M}_n(m_\delta) - \mathbb{M}_n(\hat{m}_n) \tag{67} \\
&= \int_{\mathbb{R}} \left\{ \mathbb{H}_n(y) - \frac{1}{n} \sum_{i \notin \{j, \dots, j'-1\}} \Phi_\epsilon(y - \hat{m}_n(Z_i)) - \frac{1}{n} \sum_{i=j}^{j'-1} \Phi_\epsilon(y - \hat{m}_n(Z_i) - \delta) \right\}^2 dy \nonumber \\
&\quad - \int_{\mathbb{R}} \left\{ \mathbb{H}_n(y) - \frac{1}{n} \sum_{i=1}^n \Phi_\epsilon(y - \hat{m}_n(X_i)) \right\}^2 dy \nonumber \\
&= \int_{\mathbb{R}} \left\{ \mathbb{H}_n(y) - \frac{1}{n} \sum_{i \notin \{j, \dots, j'-1\}} \Phi_\epsilon(y - \hat{m}_n(Z_i)) \right. \nonumber \\
&\qquad \left. - \frac{1}{n} \sum_{i=j}^{j'-1} \Phi_\epsilon(y - \hat{m}_n(Z_i)) + \delta \frac{1}{n} \sum_{i=j}^{j'-1} f_\epsilon(y - \hat{m}_n(Z_i)) - \frac{1}{n} \sum_{i=j}^{j'-1} R_{\delta,i}(y) \right\}^2 dy \nonumber \\
&\quad - \int_{\mathbb{R}} \left\{ \mathbb{H}_n(y) - \frac{1}{n} \sum_{i=1}^n \Phi_\epsilon(y - \hat{m}_n(X_i)) \right\}^2 dy \nonumber
\end{align}
$$

which equals

$$
\int_{\mathbb{R}} \left\{ \mathbb{H}_n(y) - \frac{1}{n} \sum_{i=1}^{n} \Phi_{\epsilon}(y - \hat{m}_n(X_i)) + \delta \frac{1}{n} \sum_{i=j}^{j'-1} f_{\epsilon}(y - \hat{m}_n(Z_i)) - \frac{1}{n} \sum_{i=j}^{j'-1} R_{\delta,i}(y) \right\}^2 dy \\
- \int_{\mathbb{R}} \left\{ \mathbb{H}_n(y) - \frac{1}{n} \sum_{i=1}^{n} \Phi_{\epsilon}(y - \hat{m}_n(X_i)) \right\}^2 dy
$$

which equals

$$
\frac{2}{n} \int_{\mathbb{R}} \left( \mathbb{H}_n(y) - \frac{1}{n} \sum_{i=1}^{n} \Phi_{\epsilon}(y - \hat{m}_n(X_i)) \right) \left( \delta \sum_{i=j}^{j'-1} f_{\epsilon}(y - \hat{m}_n(Z_i)) - \sum_{i=j}^{j'-1} R_{\delta,i}(y) \right) dy \\
+ \frac{1}{n^2} \int_{\mathbb{R}} \left( \delta \sum_{i=j}^{j'-1} f_{\epsilon}(y - \hat{m}_n(Z_i)) - \sum_{i=j}^{j'-1} R_{\delta,i}(y) \right)^2 dy
$$

which equals

$$
\frac{2}{n} \int_{\mathbb{R}} \left( \mathbb{H}_n(y) - \frac{1}{n} \sum_{i=1}^{n} \Phi_{\epsilon}(y - \hat{m}_n(X_i)) \right) \left( \delta(j' - j) f_{\epsilon}(y - \hat{m}_n(Z_j)) - \sum_{i=j}^{j'-1} R_{\delta,i}(y) \right) dy \\
+ \frac{1}{n^2} \int_{\mathbb{R}} \left( \delta(j' - j) f_{\epsilon}(y - \hat{m}_n(Z_j)) - \sum_{i=j}^{j'-1} R_{\delta,i}(y) \right)^2 dy
$$

where for $i = j, \dots, j' - 1$

$$
\begin{align*}
R_{\delta,i}(y) &= \int_{y-\hat{m}_n(Z_i)}^{y-\hat{m}_n(Z_i)-\delta} f'_{\epsilon}(t) \cdot (y - \hat{m}_n(Z_i) - \delta - t) dt \\
&= - \int_{0}^{\delta} f'_{\epsilon}(y - \hat{m}_n(Z_i) - u) \cdot (u - \delta) du, \text{ letting } u = y - \hat{m}_n(Z_i) - t \\
&= -\delta^2 \int_{0}^{1} f'_{\epsilon}(y - \hat{m}_n(Z_i) - \delta v) (v - 1) dv, \text{ letting } v = u/\delta.
\end{align*}
$$
where the remainder term $R_{\delta,i}$ is given below. Hence,

$$
\begin{align}
0 &\le \mathbb{M}_n(m_\delta) - \mathbb{M}_n(\hat{m}_n) \tag{67} \\
&= \int_{\mathbb{R}} \left\{ \mathbb{H}_n(y) - \frac{1}{n} \sum_{i \notin \{j, \dots, j'-1\}} \Phi_\epsilon(y - \hat{m}_n(Z_i)) - \frac{1}{n} \sum_{i=j}^{j'-1} \Phi_\epsilon(y - \hat{m}_n(Z_i) - \delta) \right\}^2 dy \nonumber \\
&\quad - \int_{\mathbb{R}} \left\{ \mathbb{H}_n(y) - \frac{1}{n} \sum_{i=1}^n \Phi_\epsilon(y - \hat{m}_n(X_i)) \right\}^2 dy \nonumber \\
&= \int_{\mathbb{R}} \left\{ \mathbb{H}_n(y) - \frac{1}{n} \sum_{i \notin \{j, \dots, j'-1\}} \Phi_\epsilon(y - \hat{m}_n(Z_i)) \right. \nonumber \\
&\qquad \left. - \frac{1}{n} \sum_{i=j}^{j'-1} \Phi_\epsilon(y - \hat{m}_n(Z_i)) + \delta \frac{1}{n} \sum_{i=j}^{j'-1} f_\epsilon(y - \hat{m}_n(Z_i)) - \frac{1}{n} \sum_{i=j}^{j'-1} R_{\delta,i}(y) \right\}^2 dy \nonumber \\
&\quad - \int_{\mathbb{R}} \left\{ \mathbb{H}_n(y) - \frac{1}{n} \sum_{i=1}^n \Phi_\epsilon(y - \hat{m}_n(X_i)) \right\}^2 dy \nonumber
\end{align}
$$

which equals

$$
\int_{\mathbb{R}} \left\{ \mathbb{H}_n(y) - \frac{1}{n} \sum_{i=1}^{n} \Phi_{\epsilon}(y - \hat{m}_n(X_i)) + \delta \frac{1}{n} \sum_{i=j}^{j'-1} f_{\epsilon}(y - \hat{m}_n(Z_i)) - \frac{1}{n} \sum_{i=j}^{j'-1} R_{\delta,i}(y) \right\}^2 dy \\
- \int_{\mathbb{R}} \left\{ \mathbb{H}_n(y) - \frac{1}{n} \sum_{i=1}^{n} \Phi_{\epsilon}(y - \hat{m}_n(X_i)) \right\}^2 dy
$$

which equals

$$
\frac{2}{n} \int_{\mathbb{R}} \left( \mathbb{H}_n(y) - \frac{1}{n} \sum_{i=1}^{n} \Phi_{\epsilon}(y - \hat{m}_n(X_i)) \right) \left( \delta \sum_{i=j}^{j'-1} f_{\epsilon}(y - \hat{m}_n(Z_i)) - \sum_{i=j}^{j'-1} R_{\delta,i}(y) \right) dy \\
+ \frac{1}{n^2} \int_{\mathbb{R}} \left( \delta \sum_{i=j}^{j'-1} f_{\epsilon}(y - \hat{m}_n(Z_i)) - \sum_{i=j}^{j'-1} R_{\delta,i}(y) \right)^2 dy
$$

which equals

$$
\frac{2}{n} \int_{\mathbb{R}} \left( \mathbb{H}_n(y) - \frac{1}{n} \sum_{i=1}^{n} \Phi_{\epsilon}(y - \hat{m}_n(X_i)) \right) \left( \delta(j' - j)f_{\epsilon}(y - \hat{m}_n(Z_j)) - \sum_{i=j}^{j'-1} R_{\delta,i}(y) \right) dy \\
+ \frac{1}{n^2} \int_{\mathbb{R}} \left( \delta(j' - j)f_{\epsilon}(y - \hat{m}_n(Z_j)) - \sum_{i=j}^{j'-1} R_{\delta,i}(y) \right)^2 dy
$$

where for $i = j, \dots, j' - 1$

$$
\begin{align*}
R_{\delta,i}(y) &= \int_{y-\hat{m}_n(Z_i)}^{y-\hat{m}_n(Z_i)-\delta} f'_\epsilon(t) \cdot (y - \hat{m}_n(Z_i) - \delta - t) dt \\
&= - \int_0^\delta f'_\epsilon(y - \hat{m}_n(Z_i) - u) \cdot (u - \delta) du, \text{ letting } u = y - \hat{m}_n(Z_i) - t \\
&= -\delta^2 \int_0^1 f'_\epsilon(y - \hat{m}_n(Z_i) - \delta v) (v-1) dv, \text{ letting } v = u/\delta.
\end{align*}
$$
Thus, $(M_n(m_\delta) - M_n(\hat{m}_n))/\delta$ equals

$$ \frac{2}{n}(j' - j) \int_{\mathbb{R}} \left( \mathbb{H}_n(y) - \frac{1}{n} \sum_{i=1}^{n} \Phi_{\epsilon}(y - \hat{m}_n(X_i)) \right) f_{\epsilon}(y - \hat{m}_n(Z_j)) dy \\ - \frac{2}{n} \int_{\mathbb{R}} \left( \mathbb{H}_n(y) - \frac{1}{n} \sum_{i=1}^{n} \Phi_{\epsilon}(y - \hat{m}_n(X_i)) \right) \frac{1}{\delta} \sum_{i=j}^{j'-1} R_{\delta,i}(y) dy \\ + \frac{1}{n^2} \int_{\mathbb{R}} \left\{ \delta(j' - j)^2 f_{\epsilon}(y - \hat{m}_n(Z_j))^2 - 2(j' - j) f_{\epsilon}(y - \hat{m}_n(Z_j)) \sum_{i=j}^{j'-1} R_{\delta,i}(y) \right. \\ \left. + \frac{\left( \sum_{i=j}^{j'-1} R_{\delta,i}(y) \right)^2}{\delta} \right\} dy $$

which equals

$$ \frac{2}{n}(j' - j) \int_{\mathbb{R}} \left( \mathbb{H}_n(y) - \frac{1}{n} \sum_{i=1}^{n} \Phi_{\epsilon}(y - \hat{m}_n(X_i)) \right) f_{\epsilon}(y - \hat{m}_n(Z_j)) dy \\ + \frac{2\delta}{n} \int_{\mathbb{R}} \left( \mathbb{H}_n(y) - \frac{1}{n} \sum_{i=1}^{n} \Phi_{\epsilon}(y - \hat{m}_n(X_i)) \right) \times \\ \left( \sum_{i=j}^{j'-1} \int_{0}^{1} f'_{\epsilon}(y - \hat{m}_n(Z_i) - \delta v) (v-1) dv \right) dy \\ + \frac{\delta(j' - j)^2}{n^2} \int_{\mathbb{R}} f_{\epsilon}(y - \hat{m}_n(Z_j))^2 dy \\ + \frac{2\delta(j' - j)}{n^2} \int_{\mathbb{R}} f_{\epsilon}(y - \hat{m}_n(Z_j)) \sum_{i=j}^{j'-1} \int_{0}^{1} f'_{\epsilon}(y - \hat{m}_n(Z_i) - \delta v) (v-1) dv dy \\ + \frac{\delta^3}{n^2} \int_{\mathbb{R}} \left( \int_{0}^{1} \sum_{i=j}^{j'-1} f'_{\epsilon}(y - \hat{m}_n(Z_i) - \delta v) (v-1) dv \right)^2 dy. $$

We show below that each term on the right hand side that depends on $\delta$ takes the form of $\delta^i$, $i = 1, 2, 3$ times a finite integral, so that it tends to zero as $\delta \to 0$. From Assumption A6, it follows that

$$ \left| \left( \mathbb{H}_n(y) - \frac{1}{n} \sum_{i=1}^{n} \Phi_{\epsilon}(y - \hat{m}_n(X_i)) \right) \cdot \left( \sum_{i=j}^{j'-1} \int_{0}^{1} f'_{\epsilon}(y - \hat{m}_n(Z_i) - \delta v) (v-1) dv \right) \right| \\ \le D(j' - j) \left| \mathbb{H}_n(y) - \frac{1}{n} \sum_{i=1}^{n} \Phi_{\epsilon}(y - \hat{m}_n(X_i)) \right| $$

which can be shown to be integrable on $\mathbb{R}$ using the property of $\Phi_\epsilon$ in (43). Also, Assumption A6 implies that there exists $D' > 0$ such that $\sup_{t \in \mathbb{R}} f_\epsilon(t) \le D'$. Then,

$$ \int_{\mathbb{R}} f_{\epsilon}(y - \hat{m}_{n}(Z_{j}))^{2} dy \leq D' \int_{\mathbb{R}} f_{\epsilon}(y - \hat{m}_{n}(Z_{j})) dy = D', $$
Thus, $(M_n(m_\delta) - M_n(\hat{m}_n))/\delta$ equals

$$
\begin{align*}
& \frac{2}{n}(j' - j) \int_{\mathbb{R}} \left( \mathbb{H}_n(y) - \frac{1}{n} \sum_{i=1}^{n} \Phi_{\epsilon}(y - \hat{m}_n(X_i)) \right) f_{\epsilon}(y - \hat{m}_n(Z_j)) dy \\
& - \frac{2}{n} \int_{\mathbb{R}} \left( \mathbb{H}_n(y) - \frac{1}{n} \sum_{i=1}^{n} \Phi_{\epsilon}(y - \hat{m}_n(X_i)) \right) \frac{1}{\delta} \sum_{i=j}^{j'-1} R_{\delta,i}(y) dy \\
& + \frac{1}{n^2} \int_{\mathbb{R}} \left\{ \delta(j' - j)^2 f_{\epsilon}(y - \hat{m}_n(Z_j))^2 - 2(j' - j) f_{\epsilon}(y - \hat{m}_n(Z_j)) \sum_{i=j}^{j'-1} R_{\delta,i}(y) \right. \\
& \qquad \left. + \frac{\left( \sum_{i=j}^{j'-1} R_{\delta,i}(y) \right)^2}{\delta} \right\} dy
\end{align*}
$$

which equals

$$
\begin{align*}
& \frac{2}{n}(j' - j) \int_{\mathbb{R}} \left( \mathbb{H}_n(y) - \frac{1}{n} \sum_{i=1}^{n} \Phi_{\epsilon}(y - \hat{m}_n(X_i)) \right) f_{\epsilon}(y - \hat{m}_n(Z_j)) dy \\
& + \frac{2\delta}{n} \int_{\mathbb{R}} \left( \mathbb{H}_n(y) - \frac{1}{n} \sum_{i=1}^{n} \Phi_{\epsilon}(y - \hat{m}_n(X_i)) \right) \times \\
& \qquad \left( \sum_{i=j}^{j'-1} \int_0^1 f'_{\epsilon}(y - \hat{m}_n(Z_i) - \delta v) (v-1) dv \right) dy \\
& + \frac{\delta(j' - j)^2}{n^2} \int_{\mathbb{R}} f_{\epsilon}(y - \hat{m}_n(Z_j))^2 dy \\
& + \frac{2\delta(j' - j)}{n^2} \int_{\mathbb{R}} f_{\epsilon}(y - \hat{m}_n(Z_j)) \sum_{i=j}^{j'-1} \int_0^1 f'_{\epsilon}(y - \hat{m}_n(Z_i) - \delta v) (v-1) dv dy \\
& + \frac{\delta^3}{n^2} \int_{\mathbb{R}} \left( \int_0^1 \sum_{i=j}^{j'-1} f'_{\epsilon}(y - \hat{m}_n(Z_i) - \delta v) (v-1) dv \right)^2 dy.
\end{align*}
$$

We show below that each term on the right hand side that depends on $\delta$ takes the form of $\delta^i$, $i = 1, 2, 3$ times a finite integral, so that it tends to zero as $\delta \to 0$. From Assumption A6, it follows that

$$
\left| \left( \mathbb{H}_n(y) - \frac{1}{n} \sum_{i=1}^n \Phi_\epsilon(y - \hat{m}_n(X_i)) \right) \cdot \left( \sum_{i=j}^{j'-1} \int_0^1 f'_\epsilon(y - \hat{m}_n(Z_i) - \delta v) (v-1) dv \right) \right| \\
\le D(j' - j) \left| \mathbb{H}_n(y) - \frac{1}{n} \sum_{i=1}^n \Phi_\epsilon(y - \hat{m}_n(X_i)) \right|
$$

which can be shown to be integrable on $\mathbb{R}$ using the property of $\Phi_\epsilon$ in (43). Also, Assumption A6 implies that there exists $D' > 0$ such that $\sup_{t \in \mathbb{R}} f_\epsilon(t) \le D'$. Then,

$$
\int_{\mathbb{R}} f_{\epsilon}(y - \hat{m}_{n}(Z_{j}))^{2} dy \leq D' \int_{\mathbb{R}} f_{\epsilon}(y - \hat{m}_{n}(Z_{j})) dy = D',
$$
and by Fubini's Theorem

$$
\begin{align*}
& \int_{\mathbb{R}} \sum_{i=j}^{j'-1} \int_{0}^{1} f_{\epsilon}(y - \hat{m}_{n}(Z_{j})) |f'_{\epsilon}(y - \hat{m}_{n}(Z_{i}) - \delta v)| (v-1) \, dv dy \\
&= \sum_{i=j}^{j'-1} \int_{0}^{1} \left( \int_{\mathbb{R}} f_{\epsilon}(y - \hat{m}_{n}(Z_{j})) |f'_{\epsilon}(y - \hat{m}_{n}(Z_{i}) - \delta v)| dy \right) (v-1) \, dv \\
&\leq D \sum_{i=j}^{j'-1} \int_{0}^{1} (v-1) \, dv = \frac{D(j' - j)}{2}
\end{align*}
$$

using Assumption A6 and the fact that $f_\epsilon$ is a density. Finally, using again Assumption A6 and Fubini's Theorem we have

$$
\begin{align*}
& \int_{\mathbb{R}} \left( \int_{0}^{1} \sum_{i=j}^{j'-1} f'_{\epsilon}(y - \hat{m}_{n}(Z_{i}) - \delta v) (v-1) \, dv \right)^{2} dy \\
& \leq \frac{D(j' - j)}{2} \int_{\mathbb{R}} \int_{0}^{1} \sum_{i=j}^{j'-1} |f'_{\epsilon}(y - \hat{m}_{n}(Z_{i}) - \delta v)| (1-v) \, dv dy
\end{align*}
$$

which equals

$$
\begin{align*}
& \frac{D(j' - j)}{2} \int_0^1 \left( \sum_{i=j}^{j'-1} \int_{\mathbb{R}} |f'_\epsilon(y - \hat{m}_n(Z_i) - \delta v)| dy \right) (1-v) dv \\
& = \frac{D(j' - j)^2}{4} \int_{\mathbb{R}} |f'_\epsilon(t)| dt < \infty,
\end{align*}
$$

by Assumption A6. By using (67) and distinguishing between the cases of positive and negative values of $\delta$ it follows that

$$
\begin{align*}
0 &= \lim_{\delta \to 0} \frac{M_n(m_\delta) - M_n(\hat{m}_n)}{\delta} \\
&= \frac{2}{n}(j' - j) \int_{\mathbb{R}} \left( \mathbb{H}_n(y) - \frac{1}{n} \sum_{i=1}^n \Phi_\epsilon(y - \hat{m}_n(X_i)) \right) f_\epsilon(y - \hat{m}_n(Z_j)) dy
\end{align*}
$$

and therefore,

$$
0 = \int_{\mathbb{R}} \left( \mathbb{H}_n(y) - n^{-1} \sum_{i=1}^{n} \Phi_{\epsilon}(y - \hat{m}_n(X_i)) \right) f_{\epsilon}(y - \hat{m}_k) dy
$$

where $\hat{m}_k = \hat{m}_n(Z_j) = \dots = \hat{m}_n(Z_{j'-1})$. This is precisely the condition given in (21).

In the case $\hat{m}_n$ takes a unique value, a similar reasoning give the same result, characterizing $\hat{m}_k$ for $k=1$.

Now, the alternative expression in (22) follows from the fact that for any $a \in \mathbb{R}$

$$
\int_{\mathbb{R}} \mathbb{H}_n(y) f_\epsilon(y-a) dy = \frac{1}{n} \sum_{i=1}^{n} \int_{Y_i}^{\infty} f_\epsilon(y-a) dy = 1 - \frac{1}{n} \sum_{i=1}^{n} \Phi_\epsilon(Y_i - a)
$$

which completes the proof.

<figure><img src="image_8.png" /></figure>
and by Fubini's Theorem

$$
\begin{align*}
& \int_{\mathbb{R}} \sum_{i=j}^{j'-1} \int_{0}^{1} f_{\epsilon}(y - \hat{m}_{n}(Z_{j})) |f'_{\epsilon}(y - \hat{m}_{n}(Z_{i}) - \delta v)| (v-1) \, dv \, dy \\
&= \sum_{i=j}^{j'-1} \int_{0}^{1} \left( \int_{\mathbb{R}} f_{\epsilon}(y - \hat{m}_{n}(Z_{j})) |f'_{\epsilon}(y - \hat{m}_{n}(Z_{i}) - \delta v)| \, dy \right) (v-1) \, dv \\
&\leq D \sum_{i=j}^{j'-1} \int_{0}^{1} (v-1) \, dv = \frac{D(j' - j)}{2}
\end{align*}
$$

using Assumption A6 and the fact that $f_\epsilon$ is a density. Finally, using again Assumption A6 and Fubini's Theorem we have

$$
\begin{align*}
& \int_{\mathbb{R}} \left( \left( \int_{0}^{1} \sum_{i=j}^{j'-1} f'_{\epsilon}(y - \hat{m}_{n}(Z_{i}) - \delta v) (v-1) \, dv \right)^{2} \right) dy \\
& \leq \frac{D(j' - j)}{2} \int_{\mathbb{R}} \int_{0}^{1} \sum_{i=j}^{j'-1} |f'_{\epsilon}(y - \hat{m}_{n}(Z_{i}) - \delta v)| (1-v) \, dv \, dy
\end{align*}
$$

which equals

$$
\begin{align*}
& \frac{D(j' - j)}{2} \int_0^1 \left( \sum_{i=j}^{j'-1} \int_{\mathbb{R}} |f'_\epsilon(y - \hat{m}_n(Z_i) - \delta v)| dy \right) (1-v) dv \\
& = \frac{D(j' - j)^2}{4} \int_{\mathbb{R}} |f'_\epsilon(t)| dt < \infty,
\end{align*}
$$

by Assumption A6. By using (67) and distinguishing between the cases of positive and
negative values of $\delta$ it follows that

$$
\begin{align*}
0 &= \lim_{\delta \to 0} \frac{M_n(m_\delta) - M_n(\hat{m}_n)}{\delta} \\
&= \frac{2}{n}(j' - j) \int_{\mathbb{R}} \left( \mathbb{H}_n(y) - \frac{1}{n} \sum_{i=1}^n \Phi_\epsilon(y - \hat{m}_n(X_i)) \right) f_\epsilon(y - \hat{m}_n(Z_j)) dy
\end{align*}
$$

and therefore,

$$
0 = \int_{\mathbb{R}} \left( \mathbb{H}_n(y) - n^{-1} \sum_{i=1}^{n} \Phi_{\epsilon}(y - \hat{m}_n(X_i)) \right) f_{\epsilon}(y - \hat{m}_k) dy
$$

where $\hat{m}_k = \hat{m}_n(Z_j) = \dots = \hat{m}_n(Z_{j'-1})$. This is precisely the condition given in (21).

In the case $\hat{m}_n$ takes a unique value, a similar reasoning give the same result,
characterizing $\hat{m}_k$ for $k=1$.

Now, the alternative expression in (22) follows from the fact that for any $a \in \mathbb{R}$

$$
\int_{\mathbb{R}} \mathbb{H}_n(y) f_{\epsilon}(y-a) dy = \frac{1}{n} \sum_{i=1}^{n} \int_{Y_i}^{\infty} f_{\epsilon}(y-a) dy = 1 - \frac{1}{n} \sum_{i=1}^{n} \Phi_{\epsilon}(Y_i - a)
$$

which completes the proof.

<figure><img src="image_8.png" /></figure>
<figure><img src="image_9.png" /><figcaption>Figure 3: Plots of gamma densities with varying shape parameter.</figcaption></figure>

References

Abubakar Abid, Ada Poon, and James Zou. Linear regression with shuffled labels. *arXiv*, May 2017.

R. E. Barlow, D. J. Bartholomew, J. M. Bremner, and H. D. Brunk. *Statistical Inference under Order Restrictions. The Theory and Application of Isotonic Regression*. John Wiley, London-New York-Sydney, 1972.

Sergey Bobkov and Michel Ledoux. *One-dimensional empirical measures, order statistics, and Kantorovich transport distances*, volume 261, number 1259. American Mathematical Society, 2019.

H. D. Brunk. Estimation of isotonic regression. In *Nonparametric Techniques in Statistical Inference (Proc. Sympos., Indiana Univ., Bloomington, Ind., 1969)*, pages 177-197. Cambridge Univ. Press, London, 1970.

Chris Carolan and Richard Dykstra. Asymptotic behavior of the Grenander estimator at density flat regions. *The Canadian Journal of Statistics*, 27(3):557-566, 1999. ISSN 0319-5724.

Alexandra Carpentier and Teresa Schlüter. Learning relationships between data obtained independently. In *Proceedings of the 19th International Conference on Artificial Intelligence and Statistics*, pages 658-666, 2016.

Raymond J. Carroll, David Ruppert, Leonard A. Stefanski, and Ciprian M. Crainiceanu. *Measurement error in nonlinear models*, volume 105 of *Monographs on Statistics and Applied Probability*. Chapman & Hall/CRC, Boca Raton, FL, second edition, 2006. ISBN 978-1-58488-633-4; 1-58488-633-1. doi: 10.1201/9781420010138. URL https://doi.org/10.1201/9781420010138. A modern perspective.

Eric Cator. Adaptivity and optimality of the monotone least-squares estimator. *Bernoulli*, 17(2):714-735, 2011. ISSN 1350-7265. doi: 10.3150/10-BEJ289. URL https://doi.org/10.3150/10-BEJ289.
<figure><img src="image_9.png" /><figcaption>Figure 3: Plots of gamma densities with varying shape parameter.</figcaption></figure>

References

Abubakar Abid, Ada Poon, and James Zou. Linear regression with shuffled labels. *arXiv*, May 2017.

R. E. Barlow, D. J. Bartholomew, J. M. Bremner, and H. D. Brunk. *Statistical Inference under Order Restrictions. The Theory and Application of Isotonic Regression*. John Wiley, London-New York-Sydney, 1972.

Sergey Bobkov and Michel Ledoux. *One-dimensional empirical measures, order statistics, and Kantorovich transport distances*, volume 261, number 1259. American Mathematical Society, 2019.

H. D. Brunk. Estimation of isotonic regression. In *Nonparametric Techniques in Statistical Inference (Proc. Sympos., Indiana Univ., Bloomington, Ind., 1969)*, pages 177-197. Cambridge Univ. Press, London, 1970.

Chris Carolan and Richard Dykstra. Asymptotic behavior of the Grenander estimator at density flat regions. *The Canadian Journal of Statistics*, 27(3):557-566, 1999. ISSN 0319-5724.

Alexandra Carpentier and Teresa Schlüter. Learning relationships between data obtained independently. In *Proceedings of the 19th International Conference on Artificial Intelligence and Statistics*, pages 658-666, 2016.

Raymond J. Carroll, David Ruppert, Leonard A. Stefanski, and Ciprian M. Crainiceanu. *Measurement error in nonlinear models*, volume 105 of *Monographs on Statistics and Applied Probability*. Chapman & Hall/CRC, Boca Raton, FL, second edition, 2006. ISBN 978-1-58488-633-4; 1-58488-633-1. doi: 10.1201/9781420010138. URL https://doi.org/10.1201/9781420010138. A modern perspective.

Eric Cator. Adaptivity and optimality of the monotone least-squares estimator. *Bernoulli*, 17(2):714-735, 2011. ISSN 1350-7265. doi: 10.3150/10-BEJ289. URL https://doi.org/10.3150/10-BEJ289.
Sabyasachi Chatterjee, Adityanand Guntuboyina, and Bodhisattva Sen. On risk bounds in isotonic and other shape restricted regression problems. *The Annals of Statistics*, 43(4):1774–1800, 2015.

Itai Dattner, Alexander Goldensluger, and Anatoli Juditsky. On deconvolution of distribution functions. *The Annals of Statistics*, 39(5):2477–2501, 2011.

Itai Dattner, Markus Reiß, Mathias Trabs, et al. Adaptive quantile estimation in deconvolution with unknown error distribution. *Bernoulli*, 22(1):143–192, 2016.

Morris H DeGroot and Prem K Goel. Estimation of the correlation coefficient from a broken random sample. *The Annals of Statistics*, 8(2):264–278, March 1980.

A Delaigle and I Gijbels. Bootstrap bandwidth selection in kernel density estimation from a contaminated sample. *Ann. Inst. Statist. Math.*, 56(1):19–47, 2004.

Cécile Durot. Sharp asymptotics for isotonic regression. *Probab. Theory Related Fields*, 122(2):222–240, 2002. ISSN 0178-8051. doi: 10.1007/s004400100171. URL https://doi.org/10.1007/s004400100171.

Cécile Durot. On the $L_p$-error of monotonicity constrained estimators. *Ann. Statist.*, 35(3):1080–1104, 2007. ISSN 0090-5364. doi: 10.1214/009053606000001497. URL https://doi.org/10.1214/009053606000001497.

Cécile Durot and Hendrik P. Lopuhaä. A Kiefer-Wolfowitz type of result in a general setting, with an application to smooth monotone estimation. *Electron. J. Stat.*, 8(2):2479–2513, 2014. ISSN 1935-7524. doi: 10.1214/14-EJS958. URL https://doi.org/10.1214/14-EJS958.

Cécile Durot, Vladimir N. Kulikov, and Hendrik P. Lopuhaä. The limit distribution of the $L_\infty$-error of Grenander-type estimators. *Ann. Statist.*, 40(3):1578–1608, 2012. ISSN 0090-5364. doi: 10.1214/12-AOS1015. URL https://doi.org/10.1214/12-AOS1015.

David Edelman. Estimation of the mixing distribution for a normal mean with applications to the compound decision problem. *Ann. Statist.*, 16(4):1609–1622, 1988. ISSN 0090-5364. doi: 10.1214/aos/1176351056. URL https://doi.org/10.1214/aos/1176351056.

Bob E Ellison. Two theorems for inferences about the normal distribution with applications in acceptance sampling. *Journal of the American Statistical Association*, 59 (305):89–95, 1964. doi: 10.2307/2282860.

Jianqing Fan. On the optimal rates of convergence for nonparametric deconvolution problems. *Ann. Statist.*, 19(3):1257–1272, 1991. ISSN 0090-5364. doi: 10.1214/aos/1176348248. URL https://doi.org/10.1214/aos/1176348248.

R Fletcher. *Practical methods of optimization*. A Wiley-Interscience Publication. John Wiley & Sons, Ltd., Chichester, second edition, 1987.

P. Groeneboom. Estimating a monotone density. In *Proceedings of the Berkeley conference in honor of Jerzy Neyman and Jack Kiefer, Vol. II (Berkeley, Calif., 1983)*, Wadsworth Statist./Probab. Ser., pages 539–555. Wadsworth, Belmont, CA, 1985.
Sabyasachi Chatterjee, Adityanand Guntuboyina, and Bodhisattva Sen. On risk bounds in isotonic and other shape restricted regression problems. *The Annals of Statistics*, 43(4):1774–1800, 2015.

Itai Dattner, Alexander Goldensluger, and Anatoli Juditsky. On deconvolution of distribution functions. *The Annals of Statistics*, 39(5):2477–2501, 2011.

Itai Dattner, Markus Reiß, Mathias Trabs, et al. Adaptive quantile estimation in deconvolution with unknown error distribution. *Bernoulli*, 22(1):143–192, 2016.

Morris H DeGroot and Prem K Goel. Estimation of the correlation coefficient from a broken random sample. *The Annals of Statistics*, 8(2):264–278, March 1980.

A Delaigle and I Gijbels. Bootstrap bandwidth selection in kernel density estimation from a contaminated sample. *Ann. Inst. Statist. Math.*, 56(1):19–47, 2004.

Cécile Durot. Sharp asymptotics for isotonic regression. *Probab. Theory Related Fields*, 122(2):222–240, 2002. ISSN 0178-8051. doi: 10.1007/s004400100171. URL https://doi.org/10.1007/s004400100171.

Cécile Durot. On the $L_p$-error of monotonicity constrained estimators. *Ann. Statist.*, 35(3):1080–1104, 2007. ISSN 0090-5364. doi: 10.1214/009053606000001497. URL https://doi.org/10.1214/009053606000001497.

Cécile Durot and Hendrik P. Lopuhaä. A Kiefer-Wolfowitz type of result in a general setting, with an application to smooth monotone estimation. *Electron. J. Stat.*, 8(2):2479–2513, 2014. ISSN 1935-7524. doi: 10.1214/14-EJS958. URL https://doi.org/10.1214/14-EJS958.

Cécile Durot, Vladimir N. Kulikov, and Hendrik P. Lopuhaä. The limit distribution of the $L_\infty$-error of Grenander-type estimators. *Ann. Statist.*, 40(3):1578–1608, 2012. ISSN 0090-5364. doi: 10.1214/12-AOS1015. URL https://doi.org/10.1214/12-AOS1015.

David Edelman. Estimation of the mixing distribution for a normal mean with applications to the compound decision problem. *Ann. Statist.*, 16(4):1609–1622, 1988. ISSN 0090-5364. doi: 10.1214/aos/1176351056. URL https://doi.org/10.1214/aos/1176351056.

Bob E Ellison. Two theorems for inferences about the normal distribution with applications in acceptance sampling. *Journal of the American Statistical Association*, 59 (305):89–95, 1964. doi: 10.2307/2282860.

Jianqing Fan. On the optimal rates of convergence for nonparametric deconvolution problems. *Ann. Statist.*, 19(3):1257–1272, 1991. ISSN 0090-5364. doi: 10.1214/aos/1176348248. URL https://doi.org/10.1214/aos/1176348248.

R Fletcher. *Practical methods of optimization*. A Wiley-Interscience Publication. John Wiley & Sons, Ltd., Chichester, second edition, 1987.

P. Groeneboom. Estimating a monotone density. In *Proceedings of the Berkeley conference in honor of Jerzy Neyman and Jack Kiefer, Vol. II (Berkeley, Calif., 1983)*, Wadsworth Statist./Probab. Ser., pages 539–555. Wadsworth, Belmont, CA, 1985.
Piet Groeneboom. The concave majorant of Brownian motion. Ann. Probab., 11(4): 1016-1027, 1983.

Piet Groeneboom. Brownian motion with a parabolic drift and Airy functions. Probab. Theory Related Fields, 81(1):79-109, 1989. ISSN 0178-8051. doi: 10.1007/BF00343738. URL https://doi.org/10.1007/BF00343738.

Piet Groeneboom and Geurt Jongbloed. *Nonparametric estimation under shape constraints*, volume 38 of *Cambridge Series in Statistical and Probabilistic Mathematics*. Cambridge University Press, New York, Cambridge, 2014.

Peter Hall and Soumendra N Lahiri. Estimation of distributions, moments and quantiles in deconvolution problems. *The Annals of Statistics*, 36(5):2110-2134, 2008.

Thomas N Herzog, Fritz J Scheuren, and William E Winkler. *Data quality and record linkage techniques*. Springer Science & Business Media, 2007.

Enno Mammen. Estimating a smooth monotone regression function. Ann. Statist., 19(2):724-740, 1991. ISSN 0090-5364. doi: 10.1214/aos/1176348117. URL https://doi.org/10.1214/aos/1176348117.

Jan Meis and Enno Mammen. Uncoupled isotonic regression with discrete errors. *Personal communication*, 2020.

Jorge Nocedal and Stephen J Wright. *Numerical Optimization*. Springer Series in Operations Research. Springer-Verlag, New York, New York, 1999.

Ashwin Pananjady, Martin J Wainwright, and Thomas A Courtade. Linear regression with an unknown permutation: Statistical and computational limits. In *Proceedings of the 2016 54th Annual Allerton Conference on Communication, Control, and Computing*, pages 417-424. IEEE, 2016.

Ashwin Pananjady, Martin J Wainwright, and Thomas A Courtade. Denoising linear models with permuted data. In *2017 IEEE International Symposium on Information Theory (ISIT)*, pages 446-450. IEEE, 2017.

Ashwin Pananjady, Martin J Wainwright, and Thomas A Courtade. Linear regression with shuffled data: statistical and computational limits of permutation recovery. *IEEE Trans. Inform. Theory*, 64(5):3286-3300, 2018.

Philippe Rigollet and Jonathan Weed. Uncoupled isotonic regression via minimum wasserstein deconvolution. *Information and Inference (to appear)*, 2019.

Tim Robertson, F T Wright, and R L Dykstra. *Order restricted statistical inference*. Wiley Series in Probability and Mathematical Statistics: Probability and Mathematical Statistics. John Wiley & Sons, Ltd., Chichester, 1988.

Steven Ruggles, Sarah Flood, Ronald Goeken, Josiah Grover, Erin Meyer, Jose Pacas, and Matthew Sobek. *IPUMS, USA: Version 10.0 [dataset]*. Minneapolis, MN: IPUMS, 2020.

Elias M Stein and Rami Shakarchi. *Real analysis*, volume 3 of *Princeton Lectures in Analysis*. Princeton University Press, Princeton, NJ, 2005.
Piet Groeneboom. The concave majorant of Brownian motion. Ann. Probab., 11(4): 1016-1027, 1983.

Piet Groeneboom. Brownian motion with a parabolic drift and Airy functions. Probab. Theory Related Fields, 81(1):79-109, 1989. ISSN 0178-8051. doi: 10.1007/BF00343738. URL https://doi.org/10.1007/BF00343738.

Piet Groeneboom and Geurt Jongbloed. *Nonparametric estimation under shape constraints*, volume 38 of *Cambridge Series in Statistical and Probabilistic Mathematics*. Cambridge University Press, New York, Cambridge, 2014.

Peter Hall and Soumendra N Lahiri. Estimation of distributions, moments and quantiles in deconvolution problems. *The Annals of Statistics*, 36(5):2110-2134, 2008.

Thomas N Herzog, Fritz J Scheuren, and William E Winkler. *Data quality and record linkage techniques*. Springer Science & Business Media, 2007.

Enno Mammen. Estimating a smooth monotone regression function. Ann. Statist., 19(2):724-740, 1991. ISSN 0090-5364. doi: 10.1214/aos/1176348117. URL https://doi.org/10.1214/aos/1176348117.

Jan Meis and Enno Mammen. Uncoupled isotonic regression with discrete errors. *Personal communication*, 2020.

Jorge Nocedal and Stephen J Wright. *Numerical Optimization*. Springer Series in Operations Research. Springer-Verlag, New York, New York, 1999.

Ashwin Pananjady, Martin J Wainwright, and Thomas A Courtade. Linear regression with an unknown permutation: Statistical and computational limits. In *Proceedings of the 2016 54th Annual Allerton Conference on Communication, Control, and Computing*, pages 417-424. IEEE, 2016.

Ashwin Pananjady, Martin J Wainwright, and Thomas A Courtade. Denoising linear models with permuted data. In *2017 IEEE International Symposium on Information Theory (ISIT)*, pages 446-450. IEEE, 2017.

Ashwin Pananjady, Martin J Wainwright, and Thomas A Courtade. Linear regression with shuffled data: statistical and computational limits of permutation recovery. *IEEE Trans. Inform. Theory*, 64(5):3286-3300, 2018.

Philippe Rigollet and Jonathan Weed. Uncoupled isotonic regression via minimum wasserstein deconvolution. *Information and Inference (to appear)*, 2019.

Tim Robertson, F T Wright, and R L Dykstra. *Order restricted statistical inference*. Wiley Series in Probability and Mathematical Statistics: Probability and Mathematical Statistics. John Wiley & Sons, Ltd., Chichester, 1988.

Steven Ruggles, Sarah Flood, Ronald Goeken, Josiah Grover, Erin Meyer, Jose Pacas, and Matthew Sobek. *IPUMS, USA: Version 10.0 [dataset]*. Minneapolis, MN: IPUMS, 2020.

Elias M Stein and Rami Shakarchi. *Real analysis*, volume 3 of *Princeton Lectures in Analysis*. Princeton University Press, Princeton, NJ, 2005.
Jayakrishnan Unnikrishnan, Saeid Haghighatshoar, and Martin Vetterli. Unlabeled sensing with random linear measurements. *IEEE Trans. Inform. Theory*, 64(5): 3237-3253, 2018.

Aad W. van der Vaart and Jon A. Wellner. *Weak convergence and empirical processes*. Springer Series in Statistics. Springer-Verlag, 1996.

Cun-Hui Zhang. Risk bounds in isotonic regression. *The Annals of Statistics*, 30(2): 528-555, 2002.
Jayakrishnan Unnikrishnan, Saeid Haghighatshoar, and Martin Vetterli. Unlabeled sensing with random linear measurements. *IEEE Trans. Inform. Theory*, 64(5): 3237-3253, 2018.

Aad W. van der Vaart and Jon A. Wellner. *Weak convergence and empirical processes*. Springer Series in Statistics. Springer-Verlag, 1996.

Cun-Hui Zhang. Risk bounds in isotonic regression. *The Annals of Statistics*, 30(2): 528-555, 2002.