Now, for all $i \in \mathcal{Y}$, let us write

$$ \mathbb{E}_i \left[ \left\| \hat{b}_i - \bar{b}_i \right\|_{n,i}^2 \right] = \mathbb{E}_i \left[ \left\| \hat{b}_i - \bar{b}_i \right\|_{n,i}^2 \right] - 2\mathbb{E}_i \left[ \left\| \hat{b}_i - \bar{b}_i \right\|_{n,N_i}^2 \right] + 2\mathbb{E}_i \left[ \left\| \hat{b}_i - \bar{b}_i \right\|_{n,N_i}^2 \right]. \quad (A.33) $$

For $h \in S_{K_{N_i}, M}$, we denote by $\bar{h}$ its thresholded counterpart

$$ \bar{h}(\cdot) := h(\cdot) \mathbb{1}_{\{|h(\cdot)| \le A_{N_i} \log^{1/2}(N)\}} + \operatorname{sgn}(h(\cdot)) A_{N_i} \log^{1/2}(N) \mathbb{1}_{\{|h(\cdot)| > A_{N_i} \log^{1/2}(N)\}}. $$

We also denote $\mathcal{H}_{K_{N_i}, M} := \{\bar{h}, h \in S_{K_{N_i}, M}\}$. Then, on the event $\{N_i > 1\}$, we have that

$$ \begin{aligned} \mathbb{E}_i \left[ \left\| \hat{b}_i - \bar{b}_i \right\|_{n,i}^2 \right] - 2\mathbb{E}_i \left[ \left\| \hat{b}_i - \bar{b}_i \right\|_{n,N_i}^2 \right] &\le \mathbb{E}_i \left[ \sup_{\bar{h} \in \mathcal{H}_{K_{N_i}, M}} \left\| \bar{h} - \bar{b}_i \right\|_{n,i}^2 - 2 \left\| \bar{h} - \bar{b}_i \right\|_{n,N_i}^2 \right] \\ &\le \mathbb{E}_i \left[ \sup_{g \in \mathcal{G}_{K_{N_i}, M}} \mathbb{E}_{X|Y=i} \left[ g(\bar{X}) - \frac{2}{N_i} \sum_{j \in \mathcal{I}_i} g(\bar{X}^j) \right] \right], \end{aligned} $$

with $\mathcal{G}_{K_{N_i}, M} = \{(x_1, \dots, x_n) \mapsto \frac{1}{n} \sum_{k=1}^n |\bar{h}(x_k) - \bar{b}_i(x_k)|^2, \bar{h} \in \mathcal{H}_{K_{N_i}, M}\}$. For each $g \in \mathcal{G}_{K_{N_i}, M}$ and $x \in \mathbb{R}$, we have on the event $\{N_i > 1\}$,

$$ 0 \le g(x) \le 4A_{N_i}^2 \log(N). $$

Furthermore, we have that (see [30])

$$ \mathcal{N}_{\infty}(\varepsilon, \mathcal{G}_{K_{N_i}, M}) \le \left( \frac{12(K_{N_i} + M)A_{N_i}^2 \log(N)}{\varepsilon} \right)^{K_{N_i} + M} \le \left( \frac{12(K_N + M) \log^3(N)}{\varepsilon} \right)^{K_N + M}. $$

Therefore, we deduce from Lemma A.2 in [30] with $\varepsilon = \frac{12(K_N + M) \log^3(N)}{N_i}$, Equation (A.32), and Equation (A.33), that on the event $\{N_i > 1\}$ with $A_{N_i} = \log(N_i)$

$$ \mathbb{E}_i \left[ \left\| \hat{b}_i - \bar{b}_i \right\|_{n,i}^2 \right] \le C \left( \frac{\log^2(N_i)}{K_{N_i}^2} + \sqrt{\frac{K_{N_i} \log^2(N_i) \log(N)}{N_i}} + \frac{\log^2(N_i) \log^2(N) K_{N_i}}{N_i} + \Delta \right). \quad (A.34) $$

Thus, choosing $K_{N_i} \propto (N_i \log(N_i))^{1/5}$ and for $\log(N_i) \le \log(N)$, we obtain from Equation (A.34) that

$$ \mathbb{E}_i \left[ \left\| \hat{b}_i - \bar{b}_i \right\|_{n,i}^2 \right] \le C \left( \frac{\log^{8/5}(N)}{N_i^{2/5}} + \frac{\log^{21/5}(N)}{N_i^{4/5}} + \Delta \right). \quad (A.35) $$

Using Jensen's inequality, we have

$$ \mathbb{E} \left[ \left\| \hat{b}_i - b_i^* \right\|_{n,i}^2 \mathbb{1}_{\{N_i > 1\}} \right] \le \sqrt{\mathbb{E} \left[ \left\| \hat{b}_i - \bar{b}_i \right\|_{n,i}^2 \mathbb{1}_{\{N_i > 1\}} \right]} + \mathbb{E} \left[ \left\| \bar{b}_i - b_i^* \right\|_{n,i}^2 \mathbb{1}_{\{N_i > 1\}} \right]. $$

Finally, let us study then the error $\|\bar{b}_i - b_i^*\|_{n,i}^2$. On the event $\{N_i > 1\}$, we observe with the Cauchy-Schwarz inequality

$$ \begin{aligned} \|\bar{b}_i - b_i^*\|_{n,i}^2 &= \mathbb{E}_{X|Y=i} \left[ \frac{1}{n} \sum_{k=1}^n (b_i^*(X_{k\Delta}))^2 \mathbb{1}_{\{|X_{k\Delta}| > A_{N_i}\}} \mathbb{1}_{Y_1=i, \dots, Y_{N_i}=i} \right] \\ &\le C \sqrt{\sup_{t \in [0,1]} \mathbb{P}_{X|Y=i} (|X_t| \ge A_{N_i} \mathbb{1}_{Y_1=i, \dots, Y_{N_i}=i})}, \end{aligned} $$

since $\sup_{t \in [0,1]} \mathbb{E}[b_i^*(X_t)^4] \le C$. For $A_{N_i} = \log(N_i)$ and from Lemma A.7.3, we obtain on the event $\{N_i > 1\}$

$$ \|\bar{b}_i - b_i^*\|_{n,i}^2 \le C \exp \left( -\frac{C_2}{2} \log^2(N_i) \right), $$
which, for $N_i$ a.s. large enough yields

$$ \|\bar{b}_i - b_i^*\|_{n,i} \le CN_i^{-1/2}. $$

This result leads us to obtain, from Equation (A.35), that

$$ \mathbb{E} \left[ \left\| \hat{b}_i - b_i^* \right\|_{n,i} \mathbb{1}_{\{N_i > 1\}} \right] \le C \left( \mathbb{E} \left[ \left( \frac{\log^{4/5}(N)}{N_i^{1/5}} + \frac{\log^{21/10}(N)}{N_i^{2/5}} \right) \mathbb{1}_{\{N_i > 1\}} \right] + \sqrt{\Delta} \right). $$

Using Jensen's inequality, we obtain

$$ \mathbb{E} \left[ \left\| \hat{b}_i - b_i^* \right\|_{n,i} \mathbb{1}_{\{N_i > 1\}} \right] \le C \left( \log^{4/5}(N) \left( \mathbb{E} \left[ \frac{\mathbb{1}_{N_i > 1}}{N_i} \right] \right)^{1/5} + \log^{21/10}(N) \left( \mathbb{E} \left[ \frac{\mathbb{1}_{N_i > 1}}{N_i} \right] \right)^{2/5} + \sqrt{\Delta} \right). $$

To finish the proof, since for all $i \in \mathcal{Y}$, $N_i \sim \mathcal{B}(N, \mathfrak{p}_i^*)$ we use Lemma 4.1 in [57] to deduce that

$$ \mathbb{E} \left[ \frac{\mathbb{1}_{N_i > 1}}{N_i} \right] \le \mathbb{E} \left[ \frac{\mathbb{1}_{N_i > 0}}{N_i} \right] \le \frac{2}{\mathfrak{p}_i^* N} \le \frac{2}{\mathfrak{p}_0^* N}, $$

and finally, there exists a constant $C > 0$ such that

$$ \mathbb{E} \left[ \left\| \hat{b}_i - b_i^* \right\|_{n,i} \mathbb{1}_{\{N_i > 1\}} \right] \le C \left( \left( \frac{\log^4(N)}{N \mathfrak{p}_0^*} \right)^{1/5} + \sqrt{\Delta} \right). \quad (A.36) $$

To conclude the proof for the rates of convergence of the drift coefficient, we observe that since $\hat{b}_i$ is bounded by $\log^{3/2}(N)$ and $\sup_{t \in [0,1]} \mathbb{E}[b_i^*(X_t)^2] < +\infty$, we have for $N$ large enough,

$$ \mathbb{E} \left[ \left\| \hat{b}_i - b_i^* \right\|_{n,i} \mathbb{1}_{\{N_i \le 1\}} \right] \le 2 \log^{3/2}(N) (\mathbb{P}(N_i = 0) + \mathbb{P}(N_i = 1)). \quad (A.37) $$

Since $N_i$ is distributed according to a Binomial distribution with parameters $(N, \mathfrak{p}_i^*)$. We deduce that

$$ \mathbb{P}(N_i = 0) = \exp(N \log(1 - \mathfrak{p}_i^*)), \quad \mathbb{P}(N_i = 1) = \frac{\mathfrak{p}_i^*}{1 - \mathfrak{p}_i^*} \exp(N \log(1 - \mathfrak{p}_i^*)). \quad (A.38) $$

Hence, gathering Equation (A.36), Equation (A.37) and Equation (A.38), and choosing $\Delta = O(1/N)$, we obtain for each label $i \in \mathcal{Y}$,

$$ \mathbb{E} \left[ \left\| \hat{b}_i - b_i^* \right\|_{n,i} \right] = \mathbb{E} \left[ \left\| \hat{b}_i - b_i^* \right\|_{n,i} \mathbb{1}_{\{N_i > 1\}} \right] + \mathbb{E} \left[ \left\| \hat{b}_i - b_i^* \right\|_{n,i} \mathbb{1}_{\{N_i \le 1\}} \right] = O \left( \left( \frac{\log^4(N)}{N \mathfrak{p}_0^*} \right)^{1/5} \right). $$

**Diffusion coefficient: rates of convergence.** We estimate the square $\sigma^{*2}$ of the diffusion coefficient as solution of the following regression model

$$ \frac{(X_{(k+1)\Delta}^j - X_{k\Delta}^j)^2}{\Delta} = \sigma^{*2}(X_{k\Delta}^j) + \zeta_{k\Delta}^j + R_{k\Delta}^j, \quad (A.39) $$

where $\zeta_{k\Delta}^j := \zeta_{k\Delta}^{j,1} + \zeta_{k\Delta}^{j,2} + \zeta_{k\Delta}^{j,3}$ with

$$ \zeta_{k\Delta}^{j,1} := \frac{1}{\Delta} \left[ \left( \int_{k\Delta}^{(k+1)\Delta} \sigma^* (X_s^j) dW_s^j \right)^2 - \int_{k\Delta}^{(k+1)\Delta} \sigma^{*2} (X_s^j) ds \right], $$

$$ \zeta_{k\Delta}^{j,2} := \frac{2}{\Delta} \int_{k\Delta}^{(k+1)\Delta} ((k+1)\Delta - s) \sigma^{*\prime} (X_s^j) \sigma^{*2} (X_s^j) dW_s^j, $$

$$ \zeta_{k\Delta}^{j,3} := 2b_Y^* (X_{k\Delta}^j) \int_{k\Delta}^{(k+1)\Delta} \sigma^* (X_s^j) dW_s^j, $$
and $R_{k\Delta}^j := R_{k\Delta}^{j,1} + R_{k\Delta}^{j,2} + R_{k\Delta}^{j,3}$ with,

$$R_{k\Delta}^{j,1} := \frac{1}{\Delta} \left( \int_{k\Delta}^{(k+1)\Delta} b_Y^*(X_s^j) ds \right)^2, \quad R_{k\Delta}^{j,2} := \frac{1}{\Delta} \int_{k\Delta}^{(k+1)\Delta} ((k+1)\Delta - s) \phi_Y(X_s^j) ds, \quad (A.40)$$

$$R_{k\Delta}^{j,3} := \frac{2}{\Delta} \left( \int_{k\Delta}^{(k+1)\Delta} (b_Y^*(X_s^j) - b_Y^*(X_{k\Delta})) ds \right) \left( \int_{k\Delta}^{(k+1)\Delta} \sigma^*(X_s^j) dW_s^j \right), \quad (A.41)$$

where $\phi_Y := b_Y^*\sigma^{*\prime}\sigma^* + [\sigma^{*\prime\prime}\sigma^* + (\sigma^{*\prime})^2]\sigma^{*2}$. We prove in the sequel that $\zeta_{k\Delta}^{j,1}$ is the error term, and all the other terms are negligible residuals. We remind the reader that the estimator $\hat{\sigma}^2$ of $\sigma^{*2}$ is given in (A.10). We rely on the following result:

**Lemma A.7.6.** Under Assumption A.2.1 and for $\tilde{A}_N = \log(N)$, the following holds

$$\mathbb{E} \|\hat{\sigma}^2 - \sigma^{*2}\|_{n,N}^2 \le 3 \inf_{h \in S_{\tilde{K}_N, M}} \|h - \sigma^{*2}\|_n^2 + C \left( \sqrt{\frac{\tilde{K}_N \log^3(N)}{Nn}} + \Delta_n^2 \right),$$

where $C > 0$ is a constant depending on $\sigma_1$, and where

$$\|h\|_{n,N}^2 = \frac{1}{nN} \sum_{j=1}^{N} \sum_{k=0}^{n-1} h^2(X_{k\Delta}^j).$$

The empirical error of the estimator $\hat{\sigma}^2$ is given by

$$\mathbb{E} \|\hat{\sigma}^2 - \sigma^{*2}\|_n^2 = 2\mathbb{E} \|\hat{\sigma}^2 - \sigma^{*2}\|_{n,N}^2 + [\mathbb{E} \|\hat{\sigma}^2 - \sigma^{*2}\|_n^2 - 2\mathbb{E} \|\hat{\sigma}^2 - \sigma^{*2}\|_{n,N}^2].$$

Since $\tilde{A}_N = \log(N)$, let us define $\mathcal{H}^\sigma$ as the set of functions $\bar{h}$ such that there exists a function $h \in S_{\tilde{K}_N, M}$ satisfying

$$\bar{h} = h(x) \mathbb{1}_{\{\frac{1}{\log(N)} \le h(x) \le \log^{3/2}(N)\}} + \log^{3/2}(N) \mathbb{1}_{\{h(x) > \log^{3/2}(N)\}} + \frac{1}{\log(N)} \mathbb{1}_{\{h(x) \le \frac{1}{\log(N)}\}}.$$

Using then an $\varepsilon$-net $\mathcal{H}^{\sigma, \varepsilon}$ of $\mathcal{H}^\sigma$ with $\varepsilon = \frac{12(\tilde{K}_N + M)\log^3(N)}{N}$, we finally obtain (see [30], Lemma A.2)

$$\mathbb{E} \|\hat{\sigma}^2 - \sigma^{*2}\|_n^2 - 2\mathbb{E} \|\hat{\sigma}^2 - \sigma^{*2}\|_{n,N}^2 \le \mathbb{E} \left[ \sup_{\bar{h} \in \mathcal{H}^\sigma} \left\{ \mathbb{E} \|\bar{h} - \sigma^{*2}\|_n^2 - 2\mathbb{E} \|\bar{h} - \sigma^{*2}\|_{n,N}^2 \right\} \right] \\ \le C \frac{\tilde{K}_N \log^4(N)}{N}.$$

Thus, as $\Delta_n = O(1/N)$,

$$\mathbb{E} \left[ \|\hat{\sigma}^2 - \sigma^{*2}\|_n^2 \right] \le 3 \inf_{h \in S_{\tilde{K}_N, M}} \|h - \sigma^{*2}\|_n^2 + C \left( \frac{\sqrt{\tilde{K}_N \log^3(N)}}{N} + \frac{\tilde{K}_N \log^4(N)}{N} + \frac{1}{N^2} \right) \\ \le 3 \inf_{h \in S_{\tilde{K}_N, M}} \|h - \sigma^{*2}\|_n^2 + C \frac{\tilde{K}_N \log^4(N)}{N},$$

for $N$ large enough. According to Proposition A.3.3, the bias term satisfies

$$\inf_{h \in S_{\tilde{K}_N, M}} \|h - \sigma^{*2}\|_n^2 \le C \frac{\log^2(N)}{\tilde{K}_N^2}.$$

Taking $\tilde{K}_N = (N \log(N))^{1/5}$ leads to

$$\mathbb{E} \|\hat{\sigma}^2 - \sigma^{*2}\|_n \le C_2 \left( \frac{\log^4(N)}{N} \right)^{1/5}.$$

This concludes the proof of Theorem A.3.4. □
**Proof of Lemma A.7.6.** Denote by

$$ \gamma_{N,n}(h) = \frac{1}{nN} \sum_{j=1}^{N} \sum_{k=0}^{n-1} (U_{k\Delta}^{j} - h(X_{k\Delta}^{j}))^2, $$

the least square contrast appearing in (A.9). For all $h \in S_{K_N,M}$, we deduce that

$$ \gamma_{n,N}(\hat{\sigma}^2) - \gamma_{n,N}(\sigma^{*2}) \leq \gamma_{n,N}(h) - \gamma_{n,N}(\sigma^{*2}). \quad (A.42) $$

Using (A.39), we have for all $h \in S_{\tilde{K}_N,M'}$,

$$ \gamma_{n,N}(h) - \gamma_{n,N}(\sigma^{*2}) = \|h - \sigma^{*2}\|_{n,N}^2 + 2\nu_1(\sigma^{*2} - h) + 2\nu_2(\sigma^{*2} - h) + 2\nu_3(\sigma^{*2} - h) + 2\mu(\sigma^{*2} - h), \quad (A.43) $$

where

$$ \nu_i(h) = \frac{1}{nN} \sum_{j=1}^{N} \sum_{k=0}^{n-1} h(X_{k\Delta}^{j}) \zeta_{k\Delta}^{j,i}, \quad i \in \{1, 2, 3\}, \quad \mu(h) = \frac{1}{nN} \sum_{j=1}^{N} \sum_{k=0}^{n-1} h(X_{k\Delta}^{j}) R_{k\Delta}^{j}, \quad (A.44) $$

we derive from Equations (A.42) and (A.43) that for all $h \in S_{\tilde{K}_N,M'}$,

$$ \mathbb{E} \|\hat{\sigma}^2 - \sigma^{*2}\|_{n,N}^2 \le \inf_{h \in S_{\tilde{K}_N,M}} \|h - \sigma^{*2}\|_n^2 + 2 \sum_{i=1}^3 \mathbb{E}[\nu_i(\hat{\sigma}^2 - h)] + 2 \mathbb{E}[\mu(\hat{\sigma}^2 - h)]. \quad (A.45) $$

For all $i \in \{1, 2, 3\}$ and for all $h \in S_{\tilde{K}_N,M'}$, taking the constraints (A.6) into account, one has

$$ \mathbb{E}[\nu_i(\hat{\sigma}^2 - h)] \le \sqrt{2(\tilde{K}_N + M) \log^3(N)} \sqrt{\sum_{\ell=-M}^{\tilde{K}_N-1} \mathbb{E}[\nu_i^2(B_{\ell,M,\mathbf{u}})]}. \quad (A.46) $$

1. Upper bound of $\sum_{\ell=-M}^{\tilde{K}_N-1} \mathbb{E}[\nu_1^2(B_{\ell,M,\mathbf{u}})]$. According to Equation (A.44), we have

$$ \forall \ell \in [-M, \tilde{K}_N - 1], \nu_1(B_{\ell,M,\mathbf{u}}) = \frac{1}{nN} \sum_{j=1}^{N} \sum_{k=0}^{n-1} B_{\ell,M,\mathbf{u}}(X_{k\Delta}^{j}) \zeta_{k\Delta}^{j,1}, $$

where $\zeta_{k\Delta}^{j,1} = \frac{1}{\Delta} \left[ \left( \int_{k\Delta}^{(k+1)\Delta} \sigma^*(X_s^j) dW_s^j \right)^2 - \int_{k\Delta}^{(k+1)\Delta} \sigma^{*2}(X_s^j) ds \right]$ is a martingale satisfying

$$ \mathbb{E}[\zeta_{k\Delta}^{1,1} | \mathcal{F}_{k\Delta}^{1}] = 0 \quad \text{and} \quad \mathbb{E}\left[(\zeta_{k\Delta}^{1,1})^2 | \mathcal{F}_{k\Delta}^{1}\right] \le \frac{1}{\Delta^2} \mathbb{E}\left[\left(\int_{k\Delta}^{(k+1)\Delta} \sigma^{*2}(X_s^1) ds\right)^2\right] \le C\sigma_1^{*4}, $$

with $(\mathcal{F}_t^1)_{t \ge 0}$ the natural filtration associated with the Brownian motion $W^1$. We derive that

$$ \begin{aligned} \sum_{\ell=-M}^{\tilde{K}_N-1} \mathbb{E}[\nu_1^2(B_{\ell,M,\mathbf{u}})] &= \frac{1}{Nn^2} \sum_{\ell=-M}^{\tilde{K}_N-1} \mathbb{E} \left[ \left( \sum_{k=0}^{n-1} B_{\ell,M,\mathbf{u}}(X_{k\Delta}^j) \zeta_{k\Delta}^{1,1} \right)^2 \right] \\ &= \frac{1}{Nn^2} \mathbb{E} \left[ \sum_{k=0}^{n-1} \sum_{\ell=-M}^{\tilde{K}_N-1} B_{\ell,M,\mathbf{u}}^2(X_{k\Delta}^1) (\zeta_{k\Delta}^{1,1})^2 \right] \\ &\le \frac{C}{Nn}, \end{aligned} $$

where $C$ is a constant depending on $\sigma^*$, for each $k \in [0, n-1]$, $\sum_{\ell=-M}^{\tilde{K}_N-1} B_{\ell,M,\mathbf{u}}^2(X_{k\Delta}^1) \le 1$ since $\sum_{\ell=-M}^{\tilde{K}_N-1} B_{\ell,M,\mathbf{u}}(X_{k\Delta}^1) = 1$ and $B_{\ell,M,\mathbf{u}}(X_{k\Delta}^1) \le 1$ for all $\ell = -M, \dots, \tilde{K}_N - 1$.
2. Upper bound of $\sum_{\ell=-M}^{\tilde{K}_N-1} \mathbb{E}[\nu_2^2(B_{\ell,M,\mathbf{u}})]$. For all $k \in [0, n-1]$ and for all $s \in [0, 1]$, set $\xi(s) = k\Delta$ if $s \in [k\Delta, (k+1)\Delta)$. We have:

$$ 
\begin{aligned}
& \sum_{\ell=-M}^{\tilde{K}_N-1} \mathbb{E}[\nu_2^2(B_{\ell,M,\mathbf{u}})] \\
&= \frac{4}{Nn^2} \sum_{\ell=-M}^{\tilde{K}_N-1} \mathbb{E} \left[ \left( \sum_{k=0}^{n-1} \int_{k\Delta}^{(k+1)\Delta} B_{\ell,M,\mathbf{u}}(X_{k\Delta}^1)((k+1)\Delta - s)\sigma^{*\prime}(X_s^1)\sigma^{*2}(X_s^1) dW_s \right)^2 \right] \\
&= \frac{4}{Nn^2} \sum_{\ell=-M}^{\tilde{K}_N-1} \mathbb{E} \left[ \left( \int_0^1 B_{\ell,M,\mathbf{u}}(X_{\xi(s)}^1)(\xi(s) + \Delta - s)\sigma^{*\prime}(X_s^1)\sigma^{*2}(X_s^1) dW_s \right)^2 \right] \\
&\le \frac{C}{Nn^2},
\end{aligned}
$$

where the constant $C > 0$ depends on the diffusion coefficient.

3. Upper bound of $\sum_{\ell=-M}^{\tilde{K}_N-1} \mathbb{E}[\nu_3^2(B_{\ell,M,\mathbf{u}})]$. We have:

$$ 
\begin{aligned}
\sum_{\ell=-M}^{\tilde{K}_N-1} \mathbb{E}[\nu_3^2(B_{\ell,M,\mathbf{u}})] &= \frac{4}{Nn^2} \sum_{\ell=-M}^{\tilde{K}_N-1} \mathbb{E} \left[ \left( \sum_{k=0}^{n-1} \int_{k\Delta}^{(k+1)\Delta} B_{\ell,M,\mathbf{u}}(X_{k\Delta}^1)b_Y^*(X_{k\Delta}^1)\sigma^*(X_s^1) dW_s \right)^2 \right] \\
&= \frac{4}{Nn^2} \sum_{\ell=-M}^{\tilde{K}_N-1} \mathbb{E} \left[ \left( \int_0^1 B_{\ell,M,\mathbf{u}}(X_{\eta(s)}^1)b_Y^*(X_{\eta(s)}^1)\sigma^*(X_s^1) dW_s \right)^2 \right] \\
&\le \frac{4}{Nn^2} \mathbb{E} \left[ \int_0^1 \sum_{\ell=-M}^{\tilde{K}_N-1} B_{\ell,M,\mathbf{u}}^2(X_{\eta(s)}^1)b_Y^{*2}(X_{\eta(s)}^1)\sigma^{*2}(X_s^1) ds \right].
\end{aligned}
$$

Since for all $x \in \mathbb{R}$, $b_Y^{*2}(x) \le C_0(1+x^2)$, $\sigma^{*2}(x) \le \sigma_1^{*2}$ and $\sup_{t \in [0,1]} \mathbb{E}(|X_t|^2) < \infty$, there exists a constant $C > 0$ depending on the upper bound $\sigma_1^{*2}$ of the diffusion coefficient such that

$$ \sum_{\ell=-M}^{\tilde{K}_N-1} \mathbb{E}[\nu_3^2(B_{\ell,M,\mathbf{u}})] \le \frac{C}{Nn^2}. $$

We finally deduce from Equations (2.50) and (A.46) that for all $h \in S_{\tilde{K}_N, M'}$

$$ \mathbb{E} \|\hat{\sigma}^2 - \sigma^{*2}\|_{n,N}^2 \le \inf_{h \in S_{\tilde{K}_N, M}} \|h - \sigma^{*2}\|_n^2 + C \sqrt{\frac{(\tilde{K}_N + M) \log^3(N)}{Nn}} + 2\mathbb{E}[\mu(\hat{\sigma}^2 - h)]. \quad (A.47) $$

It remains to obtain an upper bound of the term $\mu(\hat{\sigma}^2 - h)$. Notice that for $a > 0$, $x$ and $y \in \mathbb{R}$,

$$ 2xy = 2 \frac{x}{\sqrt{a}} \times \sqrt{ay} \le \frac{x^2}{a} + ay^2. $$

Then, for all $h \in S_{\tilde{K}_N, M}$ and $a > 0$,

$$ 2\mu(\hat{\sigma}^2 - h) \le \frac{2}{a} \|\hat{\sigma}^2 - \sigma^{*2}\|_{n,N}^2 + \frac{2}{a} \|h - \sigma^{*2}\|_{n,N}^2 + \frac{a}{Nn} \sum_{j=1}^{N} \sum_{k=0}^{n-1} (R_{k\Delta}^j)^2. $$

We set $a = 4$ and from Equation (A.47) we deduce that,

$$ \mathbb{E} \|\hat{\sigma}^2 - \sigma^{*2}\|_{n,N}^2 \le 3 \inf_{h \in S_{\tilde{K}_N, M}} \|h - \sigma^{*2}\|_n^2 + C \sqrt{\frac{(\tilde{K}_N + M) \log^3(N)}{Nn}} + \frac{4}{Nn} \sum_{j=1}^{N} \sum_{k=0}^{n-1} \mathbb{E} \left[ (R_{k\Delta}^j)^2 \right]. \quad (A.48) $$
We have

$$ \mathbb{E} \left[ \left( R_{k\Delta}^{j} \right)^{2} \right] \leq 3 \left( \mathbb{E} \left[ \left( R_{k\Delta}^{j,1} \right)^{2} \right] + \mathbb{E} \left[ \left( R_{k\Delta}^{j,2} \right)^{2} \right] + \mathbb{E} \left[ \left( R_{k\Delta}^{j,3} \right)^{2} \right] \right), $$

where for all $j \in [1, N]$ and $k \in [0, n-1]$, $R_{k\Delta}^{j,1}$, $R_{k\Delta}^{j,2}$ and $R_{k\Delta}^{j,3}$ are given in Equations (A.40) and (A.41). There exist constants $C_1, C_2, C_3 > 0$ such that

$$ \mathbb{E} \left[ \left( R_{k\Delta}^{j,1} \right)^2 \right] \leq \mathbb{E} \left[ \left( \int_{k\Delta}^{(k+1)\Delta} b_Y^{*2} (X_{k\Delta}^j) ds \right)^2 \right] \leq \Delta \mathbb{E} \left[ \int_{k\Delta}^{(k+1)\Delta} b_Y^{*4} (X_{k\Delta}^j) ds \right] \leq C_1 \Delta^2 $$

$$ \mathbb{E} \left[ \left( R_{k\Delta}^{j,2} \right)^2 \right] \leq \frac{1}{\Delta^2} \int_{k\Delta}^{(k+1)\Delta} ((k+1)\Delta - s)^2 ds \int_{k\Delta}^{(k+1)\Delta} \mathbb{E} \left[ \phi_Y^2 (X_s^j) \right] ds \leq C_2 \Delta^2 $$

$$ \mathbb{E} \left[ \left( R_{k\Delta}^{j,3} \right)^2 \right] \leq \frac{4}{\Delta^2} \mathbb{E} \left[ \Delta \int_{k\Delta}^{(k+1)\Delta} L_0^2 \left| X_s^j - X_{k\Delta}^j \right|^2 ds \left( \int_{k\Delta}^{(k+1)\Delta} \sigma^*(X_s^j) dW_s \right)^2 \right] \leq C_3 \Delta^2. $$

We deduce from Equation (A.48) that there exists a constant $C > 0$ depending on $\sigma_1^*$ and $M$ such that,

$$ \mathbb{E} \left\| \hat{\sigma}^2 - \sigma^{*2} \right\|_{n,N}^2 \leq 3 \inf_{h \in S_{\tilde{K}_N, M}} \left\| h - \sigma^{*2} \right\|_n^2 + C \left( \sqrt{\frac{\tilde{K}_N \log^3(N)}{Nn}} + \Delta_n^2 \right). $$

This is the announced result.

**Proof of Theorem A.3.5.** $i \in \mathcal{Y}$, define once again $\mathbb{E}_i = \mathbb{E}[. | 1_{Y_1}, \dots, 1_{Y_N=i}]$. On the event $\{N_i > 1\}$, we have for all $A_i > 0$

$$ \begin{aligned} \mathbb{E}_i \left[ \left\| \hat{b}_i - b_i^* \right\|_n^2 \right] &= \mathbb{E}_i \left[ \frac{1}{n} \sum_{k=0}^{n-1} \left( \hat{b}_i(X_{k\Delta}) - b_i^*(X_{k\Delta}) \right)^2 \right] \\ &= \mathbb{E}_i \left[ \frac{1}{n} \sum_{k=0}^{n-1} \left( \hat{b}_i(X_{k\Delta}) - b_i^*(X_{k\Delta}) \right)^2 1_{\{|X_{k\Delta}| \le A_i\}} \right] \\ &\quad + \mathbb{E}_i \left[ \frac{1}{n} \sum_{k=0}^{n-1} \left( \hat{b}_i(X_{k\Delta}) - b_i^*(X_{k\Delta}) \right)^2 1_{\{|X_{k\Delta}| > A_i\}} \right]. \end{aligned} \quad (A.49) $$

We bound each term of the r.h.s. of the above inequality. From Lemma A.7.3, and Cauchy-Schwarz Inequality, under Assumption A.2.1, we have for the second term of (A.49),

$$ \mathbb{E}_i \left[ \frac{1}{n} \sum_{k=0}^{n-1} \left( \hat{b}_i(X_{k\Delta}) - b_i^*(X_{k\Delta}) \right)^2 1_{\{|X_{k\Delta}| > A_i\}} \right] \leq C \sqrt{\exp(-CA_i^2)}. \quad (A.50) $$

For the first term of (A.49), we observe that

$$ \mathbb{E}_i \left[ \frac{1}{n} \sum_{k=0}^{n-1} \left( \hat{b}_i(X_{k\Delta}) - b_i^*(X_{k\Delta}) \right)^2 1_{\{|X_{k\Delta}| \le A_i\}} \mathcal{D}_N \right] = \int_{-A_i}^{A_i} \left( \hat{b}_i(x) - b_i^*(x) \right)^2 \left( \frac{1}{n} \sum_{k=1}^{n-1} p(k\Delta, x) \right) dx \\ + \frac{1}{n} \left( \hat{b}_i(0) - b_i^*(0) \right)^2. $$

For $A_i = (\log(N_i))^{1/4}$ and from Lemma A.7.4, we then deduce that

$$ \begin{aligned} \mathbb{E} \left[ \frac{1}{n} \sum_{k=0}^{n-1} \left( \hat{b}_i(X_{k\Delta}) - b_i^*(X_{k\Delta}) \right)^2 1_{N_i > 1} 1_{\{|X_{k\Delta}| \le A_i\}} \mathcal{D}_N \right] \\ \le C_1 e^{C_2 \sqrt{\log(N)}} \mathbb{E} \left[ \frac{1}{n} \sum_{k=0}^{n-1} \left( \hat{b}_i(X_{k\Delta}) - b_i^*(X_{k\Delta}) \right)^2 1_{\{|X_{k\Delta}| \le A_i\}} \mathcal{D}_N, Y = i \right] \\ \le C_1 e^{C_2 \sqrt{\log(N)}} \mathbb{E} \left[ \left\| \hat{b}_i - b_i^* \right\|_{n,i}^2 \right]. \end{aligned} $$
From the above key equation, Equation (A.49), Equation (A.50), and Theorem A.3.4, we deduce,

$$ \mathbb{E} \left[ \left\| \hat{b}_i - b_i^* \right\|_n^2 \right] \le C \left( \exp \left( C_2 \sqrt{\log(N)} \right) \left( \frac{\log(N)^4}{N} \right)^{1/5} + \mathbb{E} \left[ \exp(-C_2 \sqrt{\log(N_i)}) \mathbb{1}_{N_i>1} \right] \right). $$

Since $\exp(-C_2\sqrt{\log(N_i)})\mathbb{1}_{N_i>1} \to 0$ a.s. as $N \to \infty$, and $\exp(-C_2\sqrt{\log(N_i)}) \le 1$ for almost all $N_i > 1$, the theorem of dominated convergence implies

$$ \mathbb{E} \left[ \exp(-C_2 \sqrt{\log(N_i)}) \mathbb{1}_{N_i > 1} \right] \to 0 \text{ as } N \to \infty. $$

Besides from Theorem A.3.4, we also have

$$ \mathbb{E} \left[ \left\| \hat{\sigma}^2 - \sigma^{2*} \right\|_n \right] \le \left( \frac{\log(N)^4}{N} \right)^{1/5}. $$

Therefore, applying Theorem A.3.2 with $b_{\max} = \log(N)^{3/2}$, $\sigma_0^{-2} = \log(N)$, we get the desired result. $\square$

**Proof of Proposition A.3.7.** For all $i \in \mathcal{Y}$, let $\mathbb{P}_i = \mathbb{P}(.|Y = i)$ and denote by $\mathbb{P}_0$ the probability measure under which the diffusion process $X = (X_t)_{t \ge 0}$ is solution of $dX_t = d\tilde{W}_t$ where $\tilde{W}$ is a Brownian motion under $\mathbb{P}_0$. We deduce from the Girsanov's Theorem (see e.g. [64], Chapter III) that

$$ \forall i \in \mathcal{Y}, \forall t \in [0, 1], \quad \frac{d\mathbb{P}_i}{d\mathbb{P}_0}|_{\mathcal{F}_t^X} = \exp \left( \int_0^t b_i^*(X_s)dX_s - \frac{1}{2} \int_0^t b_i^{*2}(X_s)ds \right), $$

where $(\mathcal{F}_t^X)_{t \in [0,1]}$ is the natural filtration of $X$. Then, for all $i, j \in \mathcal{Y}$ such that $i \ne j$,

$$ \forall t \in [0, 1], \quad \frac{d\mathbb{P}_i}{d\mathbb{P}_j}|_{\mathcal{F}_t^X} = \exp \left( \int_0^t (b_i^* - b_j^*)(X_s)dX_s - \frac{1}{2} \int_0^t (b_i^{*2} - b_j^{*2})(X_s)ds \right) \le C \exp(M_t^{i,j}), \quad (A.51) $$

where the constant $C$ depends on $C_{b^*}$ given in Assumption A.3.6 and

$$ \forall i, j \in \mathcal{Y} : i \ne j, \quad M_t^{i,j} = \int_0^t (b_i^* - b_j^*)(X_s)dW_s, \quad t \in [0, 1]. $$

Then, for all $i, j \in \mathcal{Y}$ such that $i \ne j$ and for all $a > 0$, since $\|\hat{b}_i - b_i^*\|_{\infty}^2 \le 2A^2 \log(N)$, and using Equation (A.51) we have

$$ \begin{aligned} \|\hat{b}_i - b_i^*\|_{n,i}^2 &= \frac{1}{n} \sum_{k=0}^{n-1} \mathbb{E}_{X|Y=i} \left[ (\hat{b}_i - b_i^*)^2 (X_{k\Delta}) \right] = \frac{1}{n} \sum_{k=0}^{n-1} \mathbb{E}_{X|Y=j} \left[ (\hat{b}_i - b_i^*)^2 (X_{k\Delta}) \frac{d\mathbb{P}_i}{d\mathbb{P}_j} |_{\mathcal{F}_{k\Delta}^X} \right] \\ &\le \frac{C}{n} \sum_{k=0}^{n-1} \mathbb{E}_{X|Y=j} \left[ (\hat{b}_i - b_i^*)^2 (X_{k\Delta}) \exp(M_{k\Delta}^{i,j}) \right] \\ &\le C \exp(a) \|\hat{b}_i - b_i^*\|_{n,j}^2 + CA^2 \log(N) \mathbb{E}_{X|Y=j} \left[ \exp(M_{k\Delta}^{i,j}) \mathbb{1}_{M_{k\Delta}^{i,j}>a} \right]. \end{aligned} $$

Using the Cauchy-Schwarz inequality and Lemma 2.1 in [97], there exist constants $C > 0$ and $c > 0$ depending on $C_{b^*}$ such that,

$$ \begin{aligned} \mathbb{E} \left[ \exp \left( M_t^{i,j} \right) \mathbb{1}_{M_t^{i,j} > a} \right] &\le \sqrt{\mathbb{P} \left( M_t^{i,j} > a \right)} \sqrt{\mathbb{E} \left[ \exp \left( 2M_t^{i,j} - 2 \langle M^{i,j}, M^{i,j} \rangle_t \right) \exp \left( 2 \langle M^{i,j}, M^{i,j} \rangle_t \right) \right]} \\ &\le C \exp(-a^2/2c) \sqrt{\mathbb{E} \left[ \exp \left( 2M_t^{i,j} - 2 \langle M^{i,j}, M^{i,j} \rangle_t \right) \right]}, \end{aligned} $$

where $\mathbb{P}(M_t^{i,j} > a) \le \exp(-a^2/c)$ ([97]) and $\exp(2 \langle M^{i,j}, M^{i,j} \rangle_t) < \infty$ a.s since the drift functions are bounded. Moreover, since $(M_t^{i,j})_{t \le 1}$ is a martingale and

$$ \mathbb{E} \left[ \exp \left( \langle M^{i,j}, M^{i,j} \rangle_1 \right) \right] < \infty, $$
according to the Novikov assumption, thus $\mathcal{E}(M^{i,j}) := \{\exp(2M_t^{i,j} - 2\langle M^{i,j}, M^{i,j} \rangle_t)\}_{t \le 1}$ is a martingale with respect to the natural filtration $\mathcal{F}^M$ of $M^{i,j}$ (see [71], Proposition 5.8 and Theorem 5.9). We deduce that for all $t \in [0, 1]$,

$$ \mathbb{E}[\exp(2M_t^{i,j} - 2\langle M^{i,j}, M^{i,j} \rangle_t)] = \mathbb{E}[\mathbb{E}(\mathcal{E}(M^{i,j})_t | \mathcal{F}_0^M)] = \mathbb{E}[\exp(2M_0^{i,j} - 2\langle M^{i,j}, M^{i,j} \rangle_0)] = 1. $$

Thus, for all $a > 0$, we obtain $\mathbb{E}[\exp(M_t^{i,j}) 1_{M_t^{i,j}>a}] \le C \exp(-a^2/c)$. Finally, set $a = \sqrt{c \log(N)}$, it follows that for all $i, j \in \mathcal{Y}$ such that $i \ne j$, there exists a constant $C > 0$ such that

$$ \|\hat{b}_i - b_i^*\|_{n,j}^2 \le C \exp(\sqrt{c \log(N)}) \|\hat{b}_i - b_i^*\|_{n,i}^2 + C \frac{A^2 \log(N)}{N}. $$

□

**Proof of Theorem A.3.8.** From Theorem A.3.2, and its assumptions, we have

$$ \mathbb{E}[\mathcal{R}(\hat{g}) - \mathcal{R}(g^*)] \le C \left( \sqrt{\Delta} + \frac{1}{\mathfrak{p}_0^* \sqrt{N}} + \mathbb{E} \left[ b_{\max} \sigma_0^{-2} \sum_{i=1}^K \|\hat{b}_i - b_i^*\|_n \right] + \mathbb{E} \left[ \sigma_0^{-2} \|\hat{\sigma}^2 - \sigma^{2*}\|_n \right] \right). $$

For all $i \in \mathcal{Y}$ we obtain from Proposition A.3.7 with $A_{N_i} = \log(N_i) \le \log(N)$ that there exist constants $C_1, c > 0$ such that

$$ \mathbb{E} \left[ \left\| \hat{b}_i - b_i^* \right\|_n \right] = \sum_{j=1}^{K} \mathfrak{p}_j^* \mathbb{E} \left[ \left\| \hat{b}_i - b_i^* \right\|_{n,j} \right] \le C_1 \exp \left( \sqrt{c \log(N)} \right) \mathbb{E} \left[ \left\| \hat{b}_i - b_i^* \right\|_{n,i} \right] + C_1 \frac{\log^3(N)}{N}. $$

Then, from Theorem A.3.4 with $A_{N_i} = \log(N_i)$, $K_{N_i} \propto (N_i \log(N_i))^{1/5}$ on the event $\{N_i > 1\}$ for each $i \in \mathcal{Y}$, and $\tilde{A}_N = \log(N)$, $\tilde{K}_N \propto (N \log(N))^{1/5}$ and $\Delta = O(1/N)$, there exist constants $C_2, C_3 > 0$ such that

$$ \forall i \in \mathcal{Y}, \mathbb{E} \left\| \hat{b}_i - b_i^* \right\|_{n,i} \le C_2 \left( \frac{\log^4(N)}{N} \right)^{1/5}, \quad \text{and} \quad \mathbb{E} \left\| \hat{\sigma}^2 - \sigma^{2*} \right\|_n \le C_3 \left( \frac{\log^4(N)}{N} \right)^{1/5}. $$

Finally, by (A.12), we deduce that there exist constants $C, c > 0$ such that

$$ \mathbb{E}[\mathcal{R}(\hat{g}) - \mathcal{R}(g^*)] \le C \exp\left(\sqrt{c \log(N)}\right) N^{-1/5}. $$

□

Let us now turn to the proof of Theorem A.4.3. We have the following lemma.

**Lemma A.7.7.** Let $\beta \ge 1$ be a real number and suppose that $K_{N_i} = O(\log^{-5/2}(N_i)N_i^{1/(2\beta+1)})$ with $N_i$ a.s large enough, and $A_{N_i} = \sqrt{\frac{3\beta}{2\beta+1} \log(N_i)}$. Under Assumption A.2.1, the following holds:

$$ \mathbb{P}_i \left( \Omega_{n,N_i,K_{N_i}}^c \right) \le c \frac{K_{N_i}}{N_i}, $$

where $c > 0$ is a constant.

### A.7.3 Proofs of Section A.4

**Proof of Theorem A.4.3.** Note that throughout the proof we work conditional on the random variables $1_{Y_{1=i}}, \cdots, 1_{Y_{N=i}}$ and on the event $\{N_i > 1\}$, so that $N_i$ can be viewed as a deterministic variable. Then, to alleviate the notations, let use denote

$$ \mathbb{P}_i := \mathbb{P}(\cdot | 1_{Y_{1=i}}, \cdots, 1_{Y_{N=i}}) \quad \text{and} \quad \mathbb{E}_i = \mathbb{E}[\cdot | 1_{Y_{1=i}}, \cdots, 1_{Y_{N=i}}]. $$
For each class $i \in \mathcal{Y}$, the drift function $b_i^*$ is the solution of the following regression model

$$Z_{k\Delta}^j = b_i^*(X_{k\Delta}^j) + \xi_{k\Delta}^j + R_{k\Delta}^j, \quad j \in \mathcal{I}_i, \ k \in [0, n-1],$$

where we recall that $\mathcal{I}_i$ is the set of indices $j$ such that $Y_j = i$, and

$$\xi_{k\Delta}^j := \frac{1}{\Delta} \int_{k\Delta}^{(k+1)\Delta} \sigma^*(X_s^j) dW_s^j, \quad R_{k\Delta}^j := \frac{1}{\Delta} \int_{k\Delta}^{(k+1)\Delta} (b_i^*(X_s^j) - b_i^*(X_{k\Delta}^j)) ds. \quad (A.52)$$

We first focus on the error $\mathbb{E}_i \left[ \left\| \hat{b}_i - b_{A_{N_i}, i}^* \right\|_{n, N_i}^2 \right]$ for each label $i \in \mathcal{Y}$. Therefore, we consider the following decomposition:

$$\mathbb{E}_i \left[ \left\| \hat{b}_i - b_{A_{N_i}, i}^* \right\|_{n, N_i}^2 \right] = \mathbb{E}_i \left[ \left\| \hat{b}_i - b_{A_{N_i}, i}^* \right\|_{n, N_i}^2 \mathbb{1}_{\Lambda_i} \right] + \mathbb{E}_i \left[ \left\| \hat{b}_i - b_{A_{N_i}, i}^* \right\|_{n, N_i}^2 \mathbb{1}_{\Lambda'_i} \right], \quad (A.53)$$

where

$$\Lambda_i = \Omega_{n, N_i, K_{N_i}} \quad \text{and} \quad \Lambda'_i = \Omega_{n, N_i, K_{N_i}}^c.$$

**Upper bound of** $\mathbb{E}_i \left[ \left\| \hat{b}_i - b_{A_{N_i}, i}^* \right\|_{n, N_i}^2 \mathbb{1}_{\Lambda_i} \right]$. From the proof of Proposition 4.4 in [30], Equation (D.5), we see that for all $h \in S_{K_{N_i}, M}$ and for all $a, d > 0$, we have on the event $\Lambda_i = \Omega_{n, N_i, K_{N_i}}$,

$$\left(1 - \frac{2}{a} - \frac{4}{d}\right) \left\| \hat{b}_i - b_{A_{N_i}, i}^* \right\|_{n, N_i}^2 \le \left(1 + \frac{2}{a} + \frac{4}{d}\right) \left\| h - b_{A_{N_i}, i}^* \right\|_{n, N_i}^2 + d \sup_{\{h \in S_{K_{N_i}, M}, \|h\|_{n,i}=1\}} \nu^2(t) + aC\Delta,$$

where $C > 0$ is a constant and where for all $h \in S_{K_{N_i}, M}$,

$$\nu(h) = \frac{1}{N_i n} \sum_{j \in I_i} \sum_{k=0}^{n-1} h(X_{k\Delta}^j) \xi_{k\Delta}^j. \quad (A.54)$$

We set $a = d = 8$, and we obtain,

$$\mathbb{E}_i \left[ \left\| \hat{b}_i - b_{A_{N_i}, i}^* \right\|_{n, N_i}^2 \mathbb{1}_{\Lambda_i} \right] \le 7 \inf_{h \in S_{K_{N_i}, M}} \left\| h - b_{A_{N_i}, i}^* \right\|_{n, i}^2 + 32 \mathbb{E}_i \left[ \sup_{\{h \in S_{K_{N_i}, M}, \|h\|_{n,i}=1\}} \nu^2(h) \right] + 32C\Delta.$$

For $h \in S_{K_{N_i}, M}$, $h = \sum_{\ell=-M}^{K_{N_i}-1} w_\ell B_{\ell, M, \mathbf{u}}$ and $\|h\|_{n,i}^2 = w' \Psi_{K_{N_i}}^i w$ equals to one here, then $w = \Psi_{K_{N_i}}^{-1/2} w$ where the vector $u$ satisfies $\|u\|_{2, K_{N_i}+M} = 1$. Finally, one obtains,

$$h = \sum_{\ell=-M}^{K_{N_i}-1} w_\ell B_{\ell, M, \mathbf{u}} = \sum_{\ell=-M}^{K_{N_i}-1} u_\ell \left( \sum_{\ell'=-M}^{K_{N_i}-1} \left[ \Psi_{K_{N_i}}^{-1/2} \right]_{\ell', \ell} B_{\ell', M, \mathbf{u}} \right). \quad (A.55)$$

For all $h \in S_{K_{N_i}, M}$ such that $\|h\|_{n,i} = 1$, using Equation (A.54) and (A.55), gives

$$\nu^2(h) = \left( \sum_{\ell=-M}^{K_{N_i}-1} u_\ell \frac{1}{N_i n} \sum_{j=1}^{N_i} \sum_{k=0}^{n-1} \sum_{\ell'=-M}^{K_{N_i}-1} \left[ \Psi_{K_{N_i}}^{-1/2} \right]_{\ell', \ell} B_{\ell', M, \mathbf{u}} (X_{k\Delta}^{ij}) \xi_{k\Delta}^{ij} \right)^2.$$

Cauchy-Schwarz inequality together with $\|u\|_2 = 1$, produce

$$\nu^2(h) \le \sum_{\ell=-M}^{K_{N_i}-1} \left( \frac{1}{N_i n} \sum_{j=1}^{N_i} \sum_{k=0}^{n-1} \sum_{\ell'=-M}^{K_{N_i}-1} \left[ \Psi_{K_{N_i}}^{-1/2} \right]_{\ell', \ell} B_{\ell', M, \mathbf{u}} (X_{k\Delta}^{ij}) \xi_{k\Delta}^{ij} \right)^2.$$
Finally we obtain,

$$
\begin{align*}
\mathbb{E}_i \left[ \sup_{h \in S_{K_{N_i}, M}, \|h\|_{n,i}=1} \nu^2(h) \right] &\le \frac{1}{N_i} \mathbb{E}_i \left[ \frac{1}{n^2} \sum_{\ell=-M}^{K_{N_i}-1} \left( \sum_{k=0}^{n-1} \sum_{\ell'=-M}^{K_{N_i}-1} \left[ \Psi_{K_{N_i}}^{-1/2} \right]_{\ell',\ell} B_{\ell',M,\mathbf{u}}(X_{k\Delta}^{i_1}) \xi_{k\Delta}^{i_1} \right)^2 \right] \\
&= \frac{1}{N_i} \mathbb{E}_i \left[ \frac{1}{n^2} \sum_{\ell=-M}^{K_{N_i}-1} \sum_{k=0}^{n-1} \left( \sum_{\ell'=-M}^{K_{N_i}-1} \left[ \Psi_{K_{N_i}}^{-1/2} \right]_{\ell',\ell} B_{\ell',M,\mathbf{u}}(X_{k\Delta}^{i_1}) \right)^2 \left( \xi_{k\Delta}^{i_1} \right)^2 \right].
\end{align*}
$$

According to Equation (A.52) and considering the natural filtration ($\mathcal{F}_t$)$_{t \ge 0}$ of the Brownian motion,
for all $k \in [0, n-1]$, we have $\mathbb{E}_i (\xi_{k\Delta}^{i_1} | \mathcal{F}_{k\Delta}) = 0$ and

$$
\mathbb{E}_i \left[ \left( \xi_{k\Delta}^{i_1} \right)^2 | \mathcal{F}_{k\Delta} \right] = \frac{1}{\Delta^2} \mathbb{E} \left[ \sigma^{*2} \left( X_{k\Delta}^{i_1} \right) \mathbb{E} \left( \left( \int_{k\Delta}^{(k+1)\Delta} \sigma^*(X_s^{i_1}) \right)^2 | \mathcal{F}_{k\Delta} \right) \right] \le \frac{\sigma_1^{*2}}{\Delta}.
$$

By definition of the Gram matrix $\Psi_{K_{N_i}}$, we deduce that

$$
\begin{align*}
\mathbb{E}_i \left[ \sup_{h \in S_{K_{N_i}, M}, \|h\|_{n,i}=1} \nu^2(h) \right] &\le \frac{\sigma_1^{*2}}{N_i} \mathbb{E}_i \left[ \frac{1}{n} \sum_{\ell=-M}^{K_{N_i}-1} \sum_{k=0}^{n-1} \left( \sum_{\ell'=-M}^{K_{N_i}-1} \left[ \Psi_{K_{N_i}}^{-1/2} \right]_{\ell',\ell} B_{\ell',M,\mathbf{u}}(X_{k\Delta}^{1,i}) \right)^2 \right] \\
&\le \frac{\sigma_1^{*2}}{N_i} \mathbb{E}_i \left( \sum_{\ell, \ell', \ell''=-M}^{K_{N_i}-1} \left[ \Psi_{K_{N_i}}^{-1/2} \right]_{\ell',\ell} \left[ \Psi_{K_{N_i}}^{-1/2} \right]_{\ell'',\ell} \left[ \Psi_{K_{N_i}} \right]_{\ell',\ell''} \right) \\
&= \frac{\sigma_1^{*2}}{N_i} \mathbb{E}_i \left( \mathrm{Tr} \left( \Psi_{K_{N_i}}^{-1} \Psi_{K_{N_i}} \right) \right).
\end{align*}
$$

Besides,

$$
\mathrm{Tr} \left( \Psi_{K_{N_i}}^{-1} \Psi_{K_{N_i}} \right) = K_{N_i} + M.
$$

Thus, finally, there exists a constant $C_1 > 0$ depending on $\sigma_1^*$ and $M$ such that

$$
\mathbb{E}_i \left[ \sup_{h \in S_{K_{N_i}, M}, \|h\|_{n,i}=1} \nu^2(h) \right] \le C_1 \frac{K_{N_i}}{N_i}.
$$

Thus, there exists a constant $C > 0$ such that,

$$
\mathbb{E}_i \left[ \left\| \hat{b}_i - b_{A_{N_i}, i}^* \right\|_{n, N_i}^2 \mathbf{1}_{\Lambda_i} \right] \le 7 \inf_{h \in S_{K_{N_i}, M}} \left\| h - b_{A_{N_i}, i}^* \right\|_{n, i}^2 + C \left( \frac{K_{N_i}}{N_i} + \Delta \right). \quad (A.56)
$$

Upper bound of $\mathbb{E} \left[ \left\| \hat{b}_i - b_{A_{N_i}, i}^* \right\|_{n, N_i}^2 \mathbf{1}_{\Lambda'_i} \right]$. Using the Cauchy-Schwarz inequality, we have

$$
\mathbb{E}_i \left[ \left\| \hat{b}_i - b_{A_{N_i}, i}^* \right\|_{n, N_i}^2 \mathbf{1}_{\Lambda'_i} \right] \le C_0 \log^2(N_i) \mathbb{P}_i \left( \Omega_{n, N_i, K_{N_i}}^c \right),
$$

since for $N$ large enough, using (A.8), we have,

$$
\left\| \hat{b}_i - b_{A_{N_i}, i}^* \right\|_{n, N_i}^2 \le 2 \| \hat{b}_i \|_{\infty}^2 + 2 \| b_{A_{N_i}, i}^* \|_{\infty}^2 \le 4 A_{N_i}^2 \log(N_i) \le C_0 \log^2(N_i),
$$

where $C_0 > 0$ is a constant. Using Lemma 2.9.9, we have

$$
\mathbb{P}_i(\Lambda_i') = \mathbb{P}_i(\Omega_{n,N_i,K_{N_i}}^c) \le c \frac{K_{N_i}}{N_i}. \qquad (A.57)
$$

Then, from Equation (A.57), there exists a constant $C > 0$ such that

$$
\mathbb{E}_i \left[ \left\| \hat{b}_i - b_{A,N,i}^* \right\|_{n,N_i}^2 \mathbf{1}_{\Lambda_i'} \right] \le C \log^2(N_i) \frac{K_{N_i}}{N_i}. \quad (A.58)
$$
**Upper bound of** $\mathbb{E}_i [\|\hat{b}_i - b_{A_{N_i},i}^*\|_{n,N_i}^2]$. From Equations (A.53), (A.56) and (A.58), there exists a constant $C > 0$ such that

$$ \mathbb{E}_i \left[ \|\hat{b}_i - b_{A_{N_i},i}^*\|_{n,N_i}^2 \right] \le 7 \inf_{h \in S_{K_{N_i},M}} \left\| h - b_{A_{N_i},i}^* \right\|_{n,i}^2 + C \left( \frac{\log^2(N_i)K_{N_i}}{N_i} + \Delta \right). \quad (A.59) $$

**Upper bound of** $\mathbb{E}_i [\|\hat{b}_i - b_{A_{N_i},i}^*\|_{n,i}^2]$. Using Equation (A.59), we have

$$ \begin{aligned} \mathbb{E}_i \left[ \|\hat{b}_i - b_{A_{N_i},i}^*\|_{n,i}^2 \right] &= \mathbb{E}_i \left[ \|\hat{b}_i - b_{A_{N_i},i}^*\|_{n,i}^2 \right] - 2\mathbb{E}_i \left[ \|\hat{b}_i - b_{A_{N_i},i}^*\|_{n,N_i}^2 \right] \\ &\quad + 2\mathbb{E}_i \left[ \|\hat{b}_i - b_{A_{N_i},i}^*\|_{n,N_i}^2 \right] \\ &\le \mathbb{E}_i \left[ \|\hat{b}_i - b_{A_{N_i},i}^*\|_{n,i}^2 \right] - 2\mathbb{E}_i \left[ \|\hat{b}_i - b_{A_{N_i},i}^*\|_{n,N_i}^2 \right] \\ &\quad + 7 \inf_{h \in S_{K_{N_i},M}} \left\| h - b_{A_{N_i},i}^* \right\|_{n,i}^2 + C \left( \frac{\log^2(N_i)K_{N_i}}{N_i} + \Delta \right). \end{aligned} $$

From the proof of Theorem A.3.4, we deduce that

$$ \mathbb{E}_i \left[ \|\hat{b}_i - b_{A_{N_i},i}^*\|_{n,i}^2 \right] - 2\mathbb{E}_i \left[ \|\hat{b}_i - b_{A_{N_i},i}^*\|_{n,N_i}^2 \right] \le C \log^3(N_i) K_{N_i} / N_i $$

with $C > 0$ a constant depending on $\mathfrak{p}_0 = \min_{i \in \mathcal{Y}} \mathfrak{p}_i^*$. Besides, since $b_i^* \in \Sigma(\beta, R)$, we have

$$ \inf_{h \in S_{K_{N_i},M}} \|h - b_{A_{N_i},i}^*\|_{n,i}^2 \le C \left( \frac{A_{N_i}}{K_{N_i}} \right)^{2\beta}, $$

where $C > 0$ is a constant (see [30], Lemma D.2). Then it comes that

$$ \mathbb{E}_i \left[ \|\hat{b}_i - b_{A_{N_i},i}^*\|_{n,i}^2 \right] \le C \left( \left( \frac{A_{N_i}}{K_{N_i}} \right)^{2\beta} + \frac{K_{N_i} \log^3(N_i)}{N_i} + \Delta \right), $$

where $C > 0$ is a constant depending on $\beta, \Delta = O(1/N)$. Since

$$ K_{N_i} = O \left( \log^{-5/2} (N_i) N_i^{1/(2\beta+1)} \right), $$

we obtain

$$ \mathbb{E}_i \left[ \|\hat{b}_i - b_{A_{N_i},i}^*\|_{n,i}^2 \right] \le C \log^{6\beta}(N_i) N_i^{-\frac{2\beta}{2\beta+1}} \le C \log^{6\beta}(N) N_i^{-\frac{2\beta}{2\beta+1}}. $$

Using the Jensen's inequality,

$$ \mathbb{E} \left[ \mathbb{1}_{N_i > 1} \| \hat{b}_i - b_{A_{N_i},i}^* \|_{n,i}^2 \right] \le C \log^{6\beta}(N) \mathbb{E} \left[ \mathbb{1}_{N_i > 1} N_i^{-\frac{2\beta}{2\beta+1}} \right] \le C \log^{6\beta}(N) \left( \mathbb{E} \left[ \frac{\mathbb{1}_{N_i > 1}}{N_i} \right] \right)^{\frac{2\beta}{2\beta+1}}. $$

Using again Lemma 4.1 from [57], we obtain

$$ \mathbb{E} \left[ \mathbb{1}_{N_i > 1} \| \hat{b}_i - b_{A_{N_i},i}^* \|_{n,i}^2 \right] \le C \log^{6\beta}(N) \left( \mathbb{E} \left[ \frac{\mathbb{1}_{N_i > 1}}{N_i} \right] \right)^{\frac{2\beta}{2\beta+1}} \le C \log^{6\beta}(N) N_i^{-\frac{2\beta}{2\beta+1}}. $$

□
**Proof of Theorem A.4.4.** For all $i \in \mathcal{Y}$, recall that $b_{A_{N_i}, i}^* = b_i^* \mathbb{1}_{[-A_{N_i}, A_{N_i}]}$. Furthermore, set

$$N_0 := \min_{i \in \mathcal{Y}} N_i, \text{ then } A_{N_0} := \min_{i \in \mathcal{Y}} A_{N_i}. \qquad (A.60)$$

We have

$$\mathbb{E}[\mathcal{R}(\hat{g}) - \mathcal{R}(g^*)] = \mathbb{E}[(1 - \mathcal{R}(g^*)) \mathbb{1}_{N_0 \le 1}] + \mathbb{E}[(\mathcal{R}(\hat{g}) - \mathcal{R}(g^*)) \mathbb{1}_{N_0 > 1}].$$

Then, from Proposition A.3.1, we deduce that

$$\begin{aligned} \mathbb{E}[\mathcal{R}(\hat{g}) - \mathcal{R}(g^*)] &\le \sum_{i=1}^{K} \mathbb{P}(N_i \le 1) + 2 \sum_{i=1}^{K} \mathbb{E}[|\hat{\pi}_i(X) - \pi_i^*(X)| \mathbb{1}_{N_0 > 1}] \\ &\le 2KN(1 - \mathfrak{p}_0^*)^{N-1} + 2 \sum_{i=1}^{K} \mathbb{E}[|\hat{\pi}_i(X) - \pi_i^*(X)| \mathbb{1}_{N_0 > 1}], \end{aligned}$$

since $\hat{g} = 1$ on the event $\{N_0 \le 1\}$. For all $i \in \mathcal{Y}$ and on the event $\{N_0 > 1\}$,

$$|\hat{\pi}_i(X) - \pi_i^*(X)| \le |\hat{\pi}_i(X) - \bar{\pi}_i^{A_{N_0}}(X)| + |\bar{\pi}_i^{A_{N_0}}(X) - \pi_i^*(X)| + |\bar{\pi}_i^*(X) - \pi_i^*(X)|,$$

where $\bar{\pi}_i^{A_{N_0}}(X) := \phi_i(\bar{\mathfrak{F}}^{A_{N_0}})$ and $\bar{\mathfrak{F}}^{A_{N_0}} = (\bar{\mathfrak{F}}_1^{A_{N_0}}, \dots, \bar{\mathfrak{F}}_K^{A_{N_0}})$ with

$$\forall i \in \mathcal{Y}, \quad \bar{\mathfrak{F}}_i^{A_{N_0}} = \sum_{k=0}^{n-1} b_{A_{N_0}, i}^* (X_{k\Delta}) (X_{(k+1)\Delta} - X_{k\Delta}) - \frac{\Delta}{2} b_{A_{N_0}, i}^{*2} (X_{k\Delta}).$$

Then, there exists a constant $c > 0$ such that

$$\begin{aligned} \mathbb{E}[\mathcal{R}(\hat{g}) - \mathcal{R}(g^*)] &\le 2 \left( \sum_{i=1}^{K} \mathbb{E} \left( |\hat{\pi}_i(X) - \bar{\pi}_i^{A_{N_0}}(X)| \mathbb{1}_{N_0 > 1} \right) + \sum_{i=1}^{K} \mathbb{E} \left( |\bar{\pi}_i^{A_{N_0}}(X) - \pi_i^*(X)| \mathbb{1}_{N_0 > 1} \right) \right) \\ &\quad + c(1 - \mathfrak{p}_0^*)^{N/2} + 2 \sum_{i=1}^{K} \mathbb{E} |\bar{\pi}_i^*(X) - \pi_i^*(X)|. \end{aligned}$$

From the proof of Theorem A.3.2, there exists a constant $C_1 > 0$ depending on $K, \mathfrak{p}_0^*$ and $C_{b^*}$ and a constant $C_2 > 0$ depending on $K$ such that

$$\begin{aligned} \sum_{i=1}^{K} \mathbb{E} |\hat{\pi}_i(X) - \bar{\pi}_i^{A_{N_0}}(X)| &\le C_1 \left( \frac{1}{\sqrt{N}} + \sum_{i=1}^{K} \mathbb{E} \left[ \mathbb{1}_{N_0 > 1} \left\| \hat{b}_i - b_{A_{N_0}, i}^* \right\|_n \right] \right), \\ \sum_{i=1}^{K} \mathbb{E} |\bar{\pi}_i^*(X) - \pi_i^*(X)| &\le C_2 \sqrt{\Delta}. \end{aligned}$$

Thus, we have

$$\begin{aligned} \mathbb{E}[\mathcal{R}(\hat{g}) - \mathcal{R}(g^*)] &\le 2C_1 \left( \frac{1}{\sqrt{N}} + \sum_{i=1}^{K} \mathbb{E} \left[ \mathbb{1}_{N_0 > 1} \left\| \hat{b}_i - b_{A_{N_0}, i}^* \right\|_n \right] \right) + 2C_2\sqrt{\Delta} + c(1 - \mathfrak{p}_0^*)^{N/2} \\ &\quad + 2K \sum_{i=1}^{K} \mathbb{E} \left[ |\bar{\mathfrak{F}}_i^{A_{N_0}}(X) - \bar{\mathfrak{F}}_i(X)| \mathbb{1}_{N_0 > 1} \right]. \end{aligned}$$

For all $i \in \mathcal{Y}$,

$$\begin{aligned} \mathbb{E} \left[ |\bar{\mathfrak{F}}_i^{A_{N_0}}(X) - \bar{\mathfrak{F}}_i(X)| \mathbb{1}_{N_0 > 1} \right] &\le \mathbb{E} \left[ \left[ \sum_{k=0}^{n-1} b_i^*(X_{k\Delta}) \mathbb{1}_{|X_{k\Delta}| > A_{N_0}} \int_{k\Delta}^{(k+1)\Delta} b_i^*(X_s) ds \right] \middle| \mathbb{1}_{N_0 > 1} \right] \\ &\quad + \frac{\Delta}{2} \sum_{k=0}^{n-1} \mathbb{E} \left[ \mathbb{1}_{N_0 > 1} b_i^{*2}(X_{k\Delta}) \mathbb{1}_{|X_{k\Delta}| > A_{N_0}} \right] \\ &\quad + \mathbb{E} \left[ \sum_{k=0}^{n-1} b_i^*(X_{k\Delta}) \mathbb{1}_{N_0 > 1} \mathbb{1}_{|X_{k\Delta}| > A_{N_0}} (W_{(k+1)\Delta} - W_{k\Delta}) \right]. \end{aligned}$$
Under Assumption A.3.6, we easily obtain that

$$ \mathbb{E} \left| \sum_{k=0}^{n-1} b_i^*(X_{k\Delta}) \mathbf{1}_{N_0>1} \mathbf{1}_{|X_{k\Delta}|>A_{N_0}} \int_{k\Delta}^{(k+1)\Delta} b_i^*(X_s) ds \right| \le C_{\mathbf{b}^*}^2 \sup_{t \in [0,1]} \mathbb{P}(\{N_0>1\} \cap \{|X_t|>A_{N_0}\}), $$

and

$$ \frac{\Delta}{2} \sum_{k=0}^{n-1} \mathbb{E} [b_i^{*2}(X_{k\Delta}) \mathbf{1}_{|X_{k\Delta}|>A_{N_0}}] \le \frac{C_{\mathbf{b}^*}^2}{2} \sup_{t \in [0,1]} \mathbb{P}(\{N_0>1\} \cap \{|X_t|>A_{N_0}\}). $$

For the last term, consider the natural filtration ($\mathcal{F}_t$)$_{t \ge 0}$ of the Brownian motion ($W_t$)$_{t \ge 0}$. For all $k \in [0, n-1]$, $X_{k\Delta}$ is measurable with respect to $\mathcal{F}_{k\Delta}$ and $W_{(k+1)\Delta} - W_{k\Delta}$ is independent of $\mathcal{F}_{k\Delta}$ since the Brownian motion is an independently increasing process. Consequently, setting,

$$ \mathcal{Z} = \mathbb{E} \left| \sum_{k=0}^{n-1} b_i^*(X_{k\Delta}) \mathbf{1}_{N_0>1} \mathbf{1}_{|X_{k\Delta}|>A_{N_0}} (W_{(k+1)\Delta} - W_{k\Delta}) \right|, $$

and using the Cauchy Schwarz inequality, we obtain

$$ \begin{aligned} \mathcal{Z} & \le \left\{ \mathbb{E} \left[ \left( \sum_{k=0}^{n-1} b_i^*(X_{k\Delta}) \mathbf{1}_{N_0>1} \mathbf{1}_{|X_{k\Delta}|>A_{N_0}} (W_{(k+1)\Delta} - W_{k\Delta}) \right)^2 \right] \right\}^{1/2} \\ & \le \left\{ \mathbb{E} \left[ \sum_{k,l=0}^{n-1} b_i^*(X_{k\Delta}) b_i^*(X_{l\Delta}) \mathbf{1}_{N_0>1} \mathbf{1}_{|X_{k\Delta}|>A_{N_0}} \mathbf{1}_{|X_{l\Delta}|>A_{N_0}} (W_{(k+1)\Delta} - W_{k\Delta}) (W_{(l+1)\Delta} - W_{l\Delta}) \right] \right\}^{1/2} \\ & \le \left\{ 2 \mathbb{E} \left[ \sum_{k>l} b_i^*(X_{k\Delta}) b_i^*(X_{l\Delta}) \mathbf{1}_{N_0>1} \mathbf{1}_{|X_{k\Delta}|>A_{N_0}} \mathbf{1}_{|X_{l\Delta}|>A_{N_0}} (W_{(k+1)\Delta} - W_{k\Delta}) (W_{(l+1)\Delta} - W_{l\Delta}) \right] \right\}^{1/2} \\ & \qquad + \left\{ \mathbb{E} \left[ \sum_{k=0}^{n-1} b_i^{*2}(X_{k\Delta}) \mathbf{1}_{N_0>1} \mathbf{1}_{|X_{k\Delta}|>A_{N_0}} (W_{(k+1)\Delta} - W_{k\Delta})^2 \right] \right\}^{1/2} \\ & \le T_1 + T_2. \end{aligned} $$

We recall that ($\mathcal{F}_t$)$_{t \ge 0}$ is the natural filtration of the Brownian motion ($W_t$)$_{t \ge 0}$. Since for all $k \in [0, n-1]$, $X_{k\Delta}$ is $\mathcal{F}_{k\Delta}$-measurable, we have

$$ \begin{aligned} T_2^2 & \le \mathbb{E} \left[ \sum_{k=0}^{n-1} b_i^{*2}(X_{k\Delta}) \mathbf{1}_{N_0>1} \mathbf{1}_{|X_{k\Delta}|>A_{N_0}} \mathbb{E} \left[ (W_{(k+1)\Delta} - W_{k\Delta})^2 \middle| \mathcal{F}_{k\Delta} \right] \right] \\ & \le C_{\mathbf{b}^*}^2 \sup_{t \in [0,1]} \mathbb{P}(N_0 > 1, |X_t| > A_{N_0}). \end{aligned} $$

On the other hand, for all $k, \ell \in [0, n-1]$ such that $k > \ell$, we remark that

$$ \begin{aligned} T_1^2 & \le 2\mathbb{E} \left[ \sum_{k>\ell} b_i^*(X_{k\Delta}) b_i^*(X_{\ell\Delta}) \mathbf{1}_{N_0>1} \mathbf{1}_{|X_{k\Delta}|>A_{N_0}} \mathbf{1}_{|X_{\ell\Delta}|>A_{N_0}} (W_{(\ell+1)\Delta} - W_{\ell\Delta}) \mathbb{E} [W_{(k+1)\Delta} - W_{k\Delta} | \mathcal{F}_{k\Delta}] \right] \\ & = 0. \end{aligned} $$

Thus, we deduce that

$$ \mathcal{Z} = \mathbb{E} \left| \sum_{k=0}^{n-1} b_i^*(X_{k\Delta}) \mathbf{1}_{N_0>1} \mathbf{1}_{|X_{k\Delta}|>A_{N_0}} (W_{(k+1)\Delta} - W_{k\Delta}) \right| \le C_{\mathbf{b}^*} \sqrt{\sup_{t \in [0,1]} \mathbb{P}(N_0 > 1, |X_t| > A_{N_0})}. $$

Finally, there exists a constant $C > 0$ such that

$$ \mathbb{E}[\mathcal{R}(\hat{g}) - \mathcal{R}(g^*)] \le C \left( \frac{1}{\sqrt{N}} + \sum_{i=1}^{K} \sum_{j=1}^{K} \mathbf{p}_j^* \mathbb{E} \left[ \left\| \hat{b}_i - b_{A_{N_0}, i}^* \right\|_{n,j} \mathbf{1}_{N_0>1} \right] + \sqrt{\sup_{t \in [0,1]} \mathbb{P}(N_0 > 1, |X_t| > A_{N_0})} \right). \quad (A.61) $$
From Proposition A.3.7 with $\alpha = 1$, for all $i, j \in \mathcal{Y}$ such that $i \neq j$, we have

$$ \mathbb{E} \left[ \left\| \hat{b}_i - b_{A_{N_0},i}^* \right\|_{n,j} \mathbb{1}_{N_0>1} \right] \le C \exp(\sqrt{c \log(N)}) \mathbb{E} \left[ \left\| \hat{b}_i - b_{A_{N_0},i}^* \right\|_{n,i} \mathbb{1}_{N_0>1} \right] + C \frac{\log(N)}{N}. \quad (A.62) $$

Furthermore, for all $i \in \mathcal{Y}$, we have

$$ \begin{aligned} \mathbb{E} \left[ \left\| \hat{b}_i - b_{A_{N_0},i}^* \right\|_{n,i} \mathbb{1}_{N_0>1} \right] &\le \mathbb{E} \left[ \left\| \hat{b}_i - b_{A_{N_i},i}^* \right\|_{n,i} \mathbb{1}_{N_i>1} \right] + \mathbb{E} \left[ \left\| b_{A_{N_i},i}^* - b_{A_{N_0},i}^* \right\|_{n,i} \mathbb{1}_{N_0>1} \right] \\ &\le \mathbb{E} \left[ \left\| \hat{b}_i - b_{A_{N_i},i}^* \right\|_{n,i} \mathbb{1}_{N_i>1} \right] + \|b_i^*\|_{\infty} \sup_{t \in [0,1]} \mathbb{P}(\{A_{N_i} \ge |X_t| > A_{N_0}\} \cap \{N_0 > 1\}) \\ &\le \mathbb{E} \left[ \left\| \hat{b}_i - b_{A_{N_i},i}^* \right\|_{n,i} \mathbb{1}_{N_i>1} \right] + C_{\mathbf{b}^*} \sup_{t \in [0,1]} \sum_{j \neq i} \mathbb{P}(\{|X_t| > A_{N_j}\} \cap \{N_j > 1\}). \end{aligned} $$

We deduce from Equations (A.61) and (A.62) that there exists a constant $C > 0$ depending on $C_{\mathbf{b}^*}$, $K$ and $p_0$ such that

$$ \mathbb{E}[\mathcal{R}(\hat{g}) - \mathcal{R}(g^*)] \le C \left( \frac{1}{\sqrt{N}} + \exp\left(\sqrt{c \log(N)}\right) \sum_{i=1}^{K} \mathbb{E}\left[\left\|\hat{b}_i - b_{A_{N_i},i}^*\right\|_{n,i} \mathbb{1}_{N_i>1}\right] \right) \\ + C \exp\left(\sqrt{c \log(N)}\right) \sqrt{\sup_{t \in [0,1]} \sum_{i=1}^{K} \mathbb{P}(\{|X_t| > A_{N_i}\} \cap \{N_i > 1\})}. $$

Under the Assumptions of the Proposition and according to Theorem A.4.3, there exist two constants $C_1, C_2 > 0$ such that $\forall i \in \mathcal{Y}$,

$$ \mathbb{E} \left[ \left\| \hat{b}_i - b_{A_{N_i},i}^* \right\|_{n,i} \mathbb{1}_{N_i>1} \right] \le C_1 \log^{3\beta}(N) N^{-\beta/(2\beta+1)}, $$

and we deduce from Lemma A.7.5 with $q = 3/2$, for all $i \in \mathcal{Y}$, and for all $t \in [0, 1]$,

$$ \begin{aligned} \mathbb{P}(\{|X_t| > A_{N_i}\} \cap \{N_0 > 1\}) &= \mathbb{E}[\mathbb{P}(\{|X_t| > A_{N_i}\} \cap \{N_i > 1\}|\mathbb{1}_{Y_1=i}, \dots, \mathbb{1}_{Y_N=i})] \\ &\le C_2 \mathbb{E} \left[ \frac{\mathbb{1}_{N_i>1}}{A_{N_i}} \exp \left( -\frac{A_{N_i}^2}{3} \right) \right]. \end{aligned} $$

Thus, we obtain

$$ \mathbb{E}[\mathcal{R}(\hat{g}) - \mathcal{R}(g^*)] \le C \left( \exp\left(2\sqrt{c \log(N)}\right) N^{-\beta/(2\beta+1)} + \sqrt{\sum_{i=1}^{K} \mathbb{E}\left[\mathbb{1}_{N_i>1} \exp\left(-\frac{A_{N_i}^2}{3}\right)\right]} \right), $$

where $C > 0$ is a constant depending on $\beta, C_{\mathbf{b}^*}, K, \mathfrak{p}_0^*$. Finally, choosing $A_{N_i} = \sqrt{\frac{6\beta}{2\beta+1} \log(N_i)}$ for each $i \in \mathcal{Y}$ leads to the attended result applying the Jensen's inequality together with Lemma 4.1 in [57]. $\square$

**Proof of Theorem A.4.6.** From Theorem A.3.2, as we assumed $\sigma^*(.) = 1$, the excess risk of $\hat{g}$ satisfies

$$ \mathbb{E}[\mathcal{R}(\hat{g}) - \mathcal{R}(g^*)] \le C \left( \sqrt{\Delta} + \frac{1}{\mathfrak{p}_0^* \sqrt{N}} + \sum_{i=1}^{K} \mathbb{E} \left[ \left\| \hat{b}_i - b_i^* \right\|_n \mathbb{1}_{N_i>1} \right] + \sum_{i=1}^{K} \mathbb{P}(N_i \le 1) \right), \quad (A.63) $$

where the constant $C > 0$ depends on $b^* = (b_1^*, \dots, b_K^*)$ and $K$. For each $i \in \mathcal{Y}$, we have

$$ \mathbb{P}(N_i \le 1) \le 2N(1 - \mathfrak{p}_0^*)^{N-1}, $$
and

$$
\mathbb{E} \left[ \left\| \hat{b}_i - b_i^* \right\|_n \mathbf{1}_{N_i > 1} \right] \le \sqrt{\mathbb{E} \left[ \left\| \hat{b}_i - b_{A_{N_i}, i}^* \right\|_n^2 \mathbf{1}_{N_i > 1} \right] + \mathbb{E} \left[ \left\| b_i^* \mathbf{1}_{[-A_{N_i}, A_{N_i}]^c} \right\|_n^2 \mathbf{1}_{N_i > 1} \right]}.
$$

Using the Cauchy-Schwarz inequality and Assumption A.2.1, there exists a constant $C' > 0$ such that

$$
\mathbb{E} \left[ \left\| b_i^* \mathbf{1}_{[-A_{N_i}, A_{N_i}]^c} \right\|_n^2 \mathbf{1}_{N_i > 1} \right] \le C' \sqrt{\sup_{t \in [0,1]} \mathbb{P}(\{|X_t| > A_{N_i}\} \cap \{N_i > 1\})}.
$$

Thus, for all $i \in \mathcal{Y}$, we obtain

$$
\mathbb{E} \left\| \hat{b}_i - b_i^* \right\|_n \le \sqrt{\mathbb{E} \left[ \left\| \hat{b}_i - b_{A_{N_i}, i}^* \right\|_n^2 \mathbf{1}_{N_i > 1} \right] + C' \sqrt{\sup_{t \in [0,1]} \mathbb{P}(\{|X_t| > A_{N_i}\} \cap \{N_i > 1\})}}. \quad (A.64)
$$

For each label $i \in \mathcal{Y}$,

$$
\mathbb{E} \left[ \left\| \hat{b}_i - b_{A_{N_i}, i}^* \right\|_n^2 \mathbf{1}_{N_i > 1} \right] = E \left( \mathbf{1}_{N_i > 1} \int_{-A_{N_i}}^{A_{N_i}} (\hat{b}_i - b_{A_{N_i}, i}^*)^2 (x) f_{n,Y}(x) dx \right) + \frac{2 \log^3(N)}{n},
$$

where

$$
f_{n,Y}(x) := \frac{1}{n} \sum_{k=1}^{n-1} p_{Y,X}(k\Delta, x).
$$

From the proof of Lemma A.4.2, under Assumption A.2.1, there exist constants $C_1, C_2 > 0$ such that on the event $\{N_i > 1\}$,

$$
\forall x \in [-A_{N_i}, A_{N_i}], f_{n,Y}(x) \ge \frac{C_1}{\log(N)} \exp \left( -\frac{2A_{N_i}^2}{3(1 - \log^{-1}(N))} \right) \ge \frac{C_2}{\log(N)} \exp \left( -\frac{2}{3} A_{N_i}^2 \right) \text{ a.s.}
$$

and from Lemma A.7.4 there exists another constant $C_0 > 0$ such that $f_{n,Y}(x) \le C_0$ for all $x \in \mathbb{R}$.
Then we have

$$
\forall i \in \mathcal{Y}, \forall x \in [-A_{N_i}, A_{N_i}], \frac{f_{n,Y}(x)}{f_{n,i}(x)} \le \frac{C_0}{C_2} \log(N) \exp\left(\frac{2}{3}A_N^2\right).
$$

Then, for all $i \in \mathcal{Y}$, we obtain

$$
\begin{aligned}
\mathbb{E} \left[ \left\| \hat{b}_i - b_{A_{N_i}, i}^* \right\|_n^2 \mathbf{1}_{N_i > 1} \right] &\le \mathbb{E} \left[ \mathbf{1}_{N_i > 1} \int_{-A_{N_i}}^{A_{N_i}} (\hat{b}_i - b_{A_{N_i}, i}^*)^2 (x) f_{n,i}(x) \frac{f_{n,Y}(x)}{f_{n,i}(x)} \right] + \frac{2 \log^3(N)}{n} \\
&\le \frac{C_0}{C_2} \log(N) \exp\left(\frac{2}{3} A_N^2\right) \mathbb{E} \left[ \left\| \hat{b}_i - b_{A_{N_i}, i}^* \right\|_{n,i}^2 \mathbf{1}_{N_i > 1} \right] + \frac{2 \log^3(N)}{n}.
\end{aligned}
$$

From Theorem A.4.3, Equation (A.64) and for $n \propto N$, there exists a constant $C_3 > 0$ such that

$$
\mathbb{E} \left[ \left\| \hat{b}_i - b_i^* \right\|_n \mathbf{1}_{N_i > 1} \right] \le C_3 \sqrt{\exp\left(\frac{2}{3} A_N^2\right) \log^{6\beta+1}(N) N^{-\frac{2\beta}{2\beta+1}} + \sqrt{\sup_{t \in [0,1]} \mathbb{P}(\{|X_t| > A_{N_i}\} \cap \{N_i > 1\})}}.
$$

Using the Markov inequality, for all $t \in [0, 1]$, we have

$$
\begin{aligned}
\mathbb{P}(\{|X_t| > A_{N_i}\} \cap \{N_i > 1\}) &= \mathbb{E} \left[ \mathbb{P} \left( \{\exp(4|X_t|^2) > \exp(4A_{N_i}^2)\} \cap \{N_i > 1\} \middle| \mathbf{1}_{Y_1=i}, \dots, \mathbf{1}_{Y_N=i} \right) \right] \\
&\le \mathbb{E} \left[ \exp(4|X_t|^2) \right] \mathbb{E} \left[ \exp(-4A_{N_i}^2) \mathbf{1}_{N_i > 1} \right],
\end{aligned}
$$

and since $\sigma^*(.) = 1$ and under Assumption A.4.5, there exists a strictly positive constant $C_*$ such that
$\mathbb{E}[\exp(4|X_t|^2)] \le C_*$ (according to [55], Proposition 1.1). Thus, there exists a constant $C > 0$ such
that

$$
\mathbb{E} \left[ \left\| \hat{b}_i - b_i^* \right\|_n \mathbf{1}_{N_i > 1} \right] \le C \left( \exp \left( \frac{1}{3} A_N^2 \right) \log^{3\beta+1}(N) N^{-\beta/(2\beta+1)} \right) + C \mathbb{E} \left[ \exp(-4A_{N_i}^2) \mathbf{1}_{N_i > 1} \right]. \quad (A.65)
$$

From Equations (A.65) and (A.63), we finally obtain

$$
\mathbb{E}[\mathcal{R}(\hat{g}) - \mathcal{R}(g^*)] \le C \log^{3\beta+1}(N) N^{-3\beta/4(2\beta+1)},
$$

with $A_{N_i} \le \sqrt{\frac{3\beta}{4(2\beta+1)}} \log(N_i)$ and $C > 1$ a new constant. $\square$
## A.8 Appendix

**Proof of Lemma A.7.1.** Let $s, t \in [0, 1]$ with $s < t$, and $q \ge 1$. By convexity of $x \mapsto |x|^{2q}$, we have

$$|X_t - X_s|^{2q} \le 2^{2q-1} \left( \left| \int_s^t b_Y^*(X_u) du \right|^{2q} + \left| \int_s^t \sigma(X_u) dW_u \right|^{2q} \right).$$

Then, from Jensen's inequality, we have

$$\left| \int_s^t b_Y^*(X_u) du \right|^{2q} \le (t-s)^{2q-1} \int_s^t |b_Y^*(X_u)|^{2q} du,$$

Hence, under Assumption A.2.1 on function $b_Y^*$, we deduce that

$$\mathbb{E} \left[ \left| \int_s^t b_Y^*(X_u) du \right|^{2q} \right] \le C_q (t-s)^{2q} \left( 1 + \mathbb{E} \left[ \sup_{t \in [0,1]} |X_s|^{2q} \right] \right),$$

and using Burkholder-Davis-Gundy inequality, we obtain

$$\forall m > 0, \quad \mathbb{E} \left[ \left( \int_s^t \sigma(X_u) dW_u \right)^{2m} \right] \le C_m \mathbb{E} \left[ \left( \int_s^t \sigma^2(X_u) du \right)^m \right] \le C_m \sigma_1^{2m} (t-s)^m.$$

From the above equalities, we get

Finally, as the process has finite moments, we obtain that

$$\mathbb{E} |X_t - X_s|^{2q} \le C(t-s)^q,$$

where $C$ is a constant depending on $q, L_0$, and $\sigma_1$.

**Proof of Lemma A.4.2.** For all $i \in \mathcal{Y}$ and on the event $\{N_i > 1\}$, let us consider a vector

$(x_{-M}, \dots, x_{K_{N_i}-1}) \in \mathbb{R}^{K_{N_i}+M}$ such that $x_j \in [u_{j+M}, u_{j+M+1})$ and $B_{j,M,\mathbf{u}}(x_j) \ne 0$. Since we have $[u_{j+M}, u_{j+M+1}) \cap [u_{j'+M}, u_{j'+M+1}) = \emptyset$ for all $j, j' \in \{-M, \dots, K_{N_i}-1\}$ such that $j \ne j'$, then for all $j, j' \in \{-M, \dots, K_{N_i}-1\}$ such that $j \ne j'$, $B_{j,M,(u)}(x_{j'}) = 0$. Consequently, we obtain:

$$\begin{aligned} \det \left( (B_{\ell,M,\mathbf{u}}(x_{\ell'})\right)_{-M \le \ell, \ell' \le K_{N_i}-1} &= \det \left( \mathrm{diag} \left( B_{-M,M,\mathbf{u}}(x_M), \dots, B_{K_{N_i}-1,M,\mathbf{u}}(x_{K_{N_i}-1}) \right) \right) \\ &= \prod_{\ell=-M}^{K_{N_i}-1} B_{\ell,M,\mathbf{u}}(x_\ell) \ne 0. \end{aligned}$$

Then, we deduce from [18], Lemma 1 that the matrix $\Psi_{K_{N_i}}$ is invertible for all $K_{N_i} \in \mathcal{K}_{N_i}$, where the interval $[-A_{N_i}, A_{N_i}]$ and the function $f_T$ is replaced by $f_n: x \mapsto \frac{1}{n} \sum_{k=0}^{n-1} p(k\Delta, x)$ with $\lambda([-A_{N_i}, A_{N_i}] \cap \mathrm{supp}(f_n)) > 0$, $\lambda$ being the Lebesgue measure.

For all $w \in \mathbb{R}^{K_{N_i}+M}$ such that $\|w\|_{2,K_{N_i}+M} = 1$, we have:

$$w' \Psi_{K_{N_i}} w = \|h_w\|_n^2 = \int_{-A_{N_i}}^{A_{N_i}} h_w^2(x) f_n(x) dx + \frac{h_w^2(x_0)}{n} \quad \text{with} \quad h_w = \sum_{\ell=-M}^{K_{N_i}-1} w_\ell B_{\ell,M,\mathbf{u}}.$$

Since $\sigma^* = 1$, according to Lemma A.7.5, under Assumption A.2.1, the transition density satisfies:

$$\forall (t, x) \in (0, 1] \times \mathbb{R}, \quad \frac{1}{K_q \sqrt{t}} \exp \left( -\frac{(2q-1)x^2}{2qt} \right) \le p_X(t, x) \le \frac{K_q}{\sqrt{t}} \exp \left( -\frac{x^2}{2qt} \right) \quad \text{where } K_q > 1 \text{ and } q > 1.$$
We set $q = 3/2$, thus, since $s \mapsto \exp(-(2q-1)x^2/2qs)$ is an increasing function, we have on the event $\{N_i > 1\}$ and for all $x \in [-A_{N_i}, A_{N_i}]$,

$$ 
\begin{aligned}
f_n(x) &\ge \frac{1}{Cn} \sum_{k=1}^{n-1} \exp\left(-\frac{2x^2}{3k\Delta}\right) \ge \frac{1}{C} \int_0^{(n-1)\Delta} \exp\left(-\frac{2x^2}{3s}\right) ds \\
&\ge \frac{1}{C} \int_{1-\log^{-1}(N_i)}^{1-2^{-1}\log^{-1}(N_i)} \exp\left(-\frac{2x^2}{3s}\right) ds \\
&\ge \frac{1}{2C \log(N_i)} \exp\left(-\frac{2A_{N_i}^2}{3(1-\log^{-1}(N_i))}\right).
\end{aligned}
$$

Finally, since there exists a constant $C_1 > 0$ such that $\|h_w\|^2 \ge C_1 A_{N_i} K_{N_i}^{-1}$ (see [30], Lemma 2.6), for all $w \in \mathbb{R}^{K_{N_i}+M}$ such that $\|w\|_{2,K_{N_i}+M} = 1$, there exists constants $C', C > 0$ such that,

$$ w'\Psi_{K_{N_i}} w \ge \frac{C' A_{N_i}}{K_{N_i} \log(N_i)} \exp\left(-\frac{2A_{N_i}^2}{3(1-\log^{-1}(N_i))}\right) \ge \frac{C A_{N_i}}{K_{N_i} \log(N_i)} \exp\left(-\frac{2}{3} A_{N_i}^2\right). $$

Furthermore, we set $w_0 = e_{K_{N_i}-1} \in \mathbb{R}^{K_{N_i}+M}$ where for all $\ell \in [-M, K_{N_i}-1]$,

$$ [e_{K_{N_i}-1}]_{\ell} := \delta_{\ell, K_{N_i}-1} = \begin{cases} 0 & \text{if } \ell \ne K_{N_i}-1 \\ 1 & \text{else.} \end{cases} $$

We have,

$$ 
\begin{aligned}
w_0' \Psi_{K_{N_i}} w_0 &= \int_{-A_{N_i}}^{A_{N_i}} B_{K_{N_i}-1,M,\mathbf{u}}^2(x) f_n(x) + \frac{B_{K_{N_i}-1,M,\mathbf{u}}(0)}{n} \\
&\le \frac{C}{n} \sum_{k=1}^{n-1} \frac{1}{\sqrt{k\Delta}} \exp\left(-\frac{u_{K_{N_i}-1}^2}{3k\Delta}\right) \left\|B_{K_{N_i}-1,M,\mathbf{u}}\right\|^2 + \frac{1}{n} \\
&\le \frac{C C_1 A_{N_i} K_{N_i}^{-1}}{n} \sum_{k=1}^{n-1} \frac{1}{\sqrt{k\Delta}} \exp\left(-\frac{\alpha_{N_i}^2}{3k\Delta}\right) + \frac{1}{n},
\end{aligned}
$$

where $\alpha_{N_i} = A_{N_i}(K_{N_i}-2)/K_{N_i}$, $\|B_{K_{N_i}-1,M,\mathbf{u}}\|^2 \le C_1 A_{N_i} K_{N_i}^{-1}$ (see [30], Lemma 2.6) and $C_1 > 0$ is a constant. Since the function $s \mapsto \exp(-\alpha_{N_i}^2/3s)/\sqrt{s}$ is increasing, we deduce that

$$ n^{-1} \sum_{k=1}^{n-1} \frac{1}{\sqrt{k\Delta}} \exp(-\alpha_{N_i}^2/3k\Delta) \le n^{-1} \sum_{k=1}^{n-1} \exp(-\alpha_{N_i}^2/3), $$

and for $N$ large enough,

$$ w_0' \Psi_{K_{N_i}} w_0 \le \frac{C A_{N_i}}{K_{N_i}} \exp\left(-\frac{A_{N_i}^2}{3} \left(\frac{K_{N_i}-2}{K_{N_i}}\right)^2\right) + \frac{1}{n} \le \frac{C' A_{N_i}}{K_{N_i}} \exp\left(-\frac{A_{N_i}^2}{3} \left(\frac{K_{N_i}-2}{K_{N_i}}\right)^2\right), $$

where $C' > 0$ is a constant and $n \ge N \ge N_i$.

<figure><img src="image_16.png" /></figure>

**Proof of Lemma 2.9.9.** Let us remind the reader of the Gram matrix $\Psi_{K_{N_i}}$ given in Equation (A.13) for $i \in \mathcal{Y}$,

$$ \Psi_{K_{N_i}} = \mathbb{E} \left[ \frac{1}{N_i n} \mathbf{B}'_{K_{N_i}} \mathbf{B}_{K_{N_i}} \right] = \mathbb{E} \left( \hat{\Psi}_{K_{N_i}} \right), $$
where, on the event {$N_i > 1$}, and denoting by $\mathcal{I}_i := \{i_1, \dots, i_{N_i}\}$ the indices $j$ such that $Y_j = i$,

$$ \mathbf{B}_{K_{N_i}} := \begin{pmatrix} B_{-M}(X_0^{i_1}) & \dots & B_{K_{N_i}-1}(X_0^{i_1}) \\ \vdots & \ddots & \vdots \\ B_{-M}(X_{(n-1)\Delta}^{i_1}) & \dots & B_{K_{N_i}-1}(X_{(n-1)\Delta}^{i_1}) \\ \vdots & \ddots & \vdots \\ B_{-M}(X_0^{i_{N_i}}) & \dots & B_{K_{N_i}-1}(X_0^{i_{N_i}}) \\ \vdots & \ddots & \vdots \\ B_{-M}(X_{(n-1)\Delta}^{i_{N_i}}) & \dots & B_{K_{N_i}-1}(X_{(n-1)\Delta}^{i_{N_i}}) \end{pmatrix} \in \mathbb{R}^{N_i n \times (K_{N_i}+M)}. \quad (A.66) $$

The empirical counterpart $\hat{\Psi}$ is the random matrix given by $\hat{\Psi}_{K_{N_i}}$ of size $(K_{N_i} + M) \times (K_{N_i} + M)$ is given by

$$ \hat{\Psi}_{K_{N_i}} := \frac{1}{N_i n} \mathbf{B}'_{K_{N_i}} \mathbf{B}_{K_{N_i}} = \left( \frac{1}{N_i n} \sum_{j=1}^{N_i} \sum_{k=0}^{n-1} B_{\ell}(X_{k\Delta}^{ij}) B_{\ell'}(X_{k\Delta}^{ij}) \right)_{\ell, \ell' \in [-M, K_{N_i}-1]} . \quad (A.67) $$

We build an orthonormal basis $\theta = (\theta_{-M}, \dots, \theta_{K_{N_i}-1})$ of the subspace $S_{K_{N_i}, M}$ with respect to the $\mathbb{L}^2$ inner product $\langle \cdot, \cdot \rangle$ through the Gram-Schmidt orthogonalization of the spline basis $(B_{-M}, \dots, B_{K_{N_i}-1})$. Then, we have

$$ \mathrm{Span}(B_{-M}, \dots, B_{K_{N_i}-1}) = \mathrm{Span}(\theta_{-M}, \dots, \theta_{K_{N_i}-1}) = S_{K_{N_i}, M}, $$

and the matrix given in Equation (A.66) is factorized as follows

$$ \mathbf{B}_{K_{N_i}} = \mathbf{\Theta}_{K_{N_i}} \mathbf{R}_{K_{N_i}}, \quad (A.68) $$

where

$$ \mathbf{\Theta}_{K_{N_i}} = \left( (\theta_{\ell}(X_0^{ij}), \theta_{\ell}(X_{\Delta}^{ij}), \dots, \theta_{\ell}(X_{n\Delta}^{ij}))' \right)_{\substack{1 \le j \le N_i \\ -M \le \ell \le K_{N_i}-1}} \in \mathbb{R}^{N_i n \times (K_{N_i}+M)}, $$

and $\mathbf{R}_{K_{N_i}}$ is an upper triangular matrix of size $(K_{N_i} + M) \times (K_{N_i} + M)$ see [72]). Let $\Phi_{K_{N_i}}$ be the Gram matrix under the orthonormal basis $\theta = (\theta_{-M}, \dots, \theta_{K_{N_i}-1})$ and given by

$$ \Phi_{K_{N_i}} = \mathbb{E} \left[ \frac{1}{N_i n} \mathbf{\Theta}'_{K_{N_i}} \mathbf{\Theta}_{K_{N_i}} \right] = \mathbb{E} \left( \hat{\Phi}_{K_{N_i}} \right), $$

where,

$$ \hat{\Phi}_{K_{N_i}} := \frac{1}{N_i n} \mathbf{\Theta}'_{K_{N_i}} \mathbf{\Theta}_{K_{N_i}} = \left( \frac{1}{N_i n} \sum_{j=1}^{N_i} \sum_{k=0}^{n-1} \theta_{\ell}(X_{k\Delta}^{ij}) \theta_{\ell'}(X_{k\Delta}^{ij}) \right)_{\ell, \ell' \in [-M, K_{N_i}-1]} . \quad (A.69) $$

The matrices $\Psi_{K_{N_i}}$ and $\hat{\Psi}_{K_{N_i}}$ are respectively linked to the matrices $\Phi_{K_{N_i}}$ and $\hat{\Phi}_{K_{N_i}}$ through the following relations

$$ \Psi_{K_{N_i}} = \mathbf{R}'_{K_{N_i}} \Phi_{K_{N_i}} \mathbf{R}_{K_{N_i}} \quad \text{and} \quad \hat{\Psi}_{K_{N_i}} = \mathbf{R}'_{K_{N_i}} \hat{\Phi}_{K_{N_i}} \mathbf{R}_{K_{N_i}}. $$

Since for all $h = \sum_{\ell=-M}^{K_{N_i}-1} a_\ell B_{\ell,M,u} \in S_{K_{N_i}, M}$ one has

$$ \|h\|_{n, N_i}^2 = a' \hat{\Psi}_{K_{N_i}} a \quad \text{and} \quad \|h\|_{n, i}^2 = a' \Psi_{K_{N_i}} a, \quad \text{with} \quad a = (a_{-M}, \dots, a_{K_{N_i}-1})', $$
we deduce that

$$ \|h\|_{n,N_i}^2 = w' \hat{\Phi}_{K_{N_i}} w \quad \text{and} \quad \|h\|_{n,i}^2 = w' \Phi_{K_{N_i}} w, \quad \text{with} \quad w = \mathbf{R}_{K_{N_i}} a. $$

Under Assumption A.2.1, we follow the lines of [19] Proposition 2.3 and Lemma 6.2. Then,

$$ \begin{aligned} \sup_{h \in S_{K_{N_i}, M}, \|h\|_{n,i} = 1} & \|h\|_{n,N_i}^2 - \|h\|_{n,i}^2 \\ &= \sup_{w \in \mathbb{R}^{K_{N_i}+M}, \|\Phi_{K_{N_i}}^{1/2} w\|_{2,K_{N_i}+M} = 1} \left| w' (\hat{\Phi}_{K_{N_i}} - \Phi_{K_{N_i}}) w \right| \\ &= \sup_{u \in \mathbb{R}^{K_{N_i}+M}, \|u\|_{2,K_{N_i}+M} = 1} \left| u' \Phi_{K_{N_i}}^{-1/2} (\hat{\Phi}_{K_{N_i}} - \Phi_{K_{N_i}}) \Phi_{K_{N_i}}^{-1/2} u \right| \\ &= \left\| \Phi_{K_{N_i}}^{-1/2} \hat{\Phi}_{K_{N_i}} \Phi_{K_{N_i}}^{-1/2} - \mathrm{Id}_{K_{N_i}+M} \right\|_{\mathrm{op}}. \end{aligned} $$

Therefore,

$$ \Omega_{n,N_i,K_{N_i}}^c = \left\{ \left\| \Phi_{K_{N_i}}^{-1/2} \hat{\Phi}_{K_{N_i}} \Phi_{K_{N_i}}^{-1/2} - \mathrm{Id}_{K_{N_i}+M} \right\|_{\mathrm{op}} > 1/2 \right\}. $$

Then, we apply here Theorem 1 of [16], it yields

$$ \mathbb{P}_i \left( \Omega_{n, N_i, K_{N_i}}^c \right) \le 2(K_{N_i} + M) \exp \left( -c_{1/2} \frac{N_i}{\mathcal{L}(K_{N_i} + M) (\|\Phi_{K_{N_i}}^{-1}\|_{\mathrm{op}} \vee 1)} \right), \quad (A.70) $$

with $c_{1/2} = (3 \log(3/2) - 1)/2$ and $\mathcal{L}(K_{N_i} + M) := \sup_{x \in [-A_{N_i}, A_{N_i}]} \sum_{\ell=-M}^{K_{N_i}-1} \theta_\ell^2(x)$ (from application of

Lemma 6.2 from [19]). For all $h = \sum_{\ell=-M}^{K_{N_i}-1} w_\ell \theta_\ell \in \mathrm{Span} (\theta_{-M}, \dots, \theta_{K_{N_i}-1}) = S_{K_{N_i}, M}$, we have

$$ \|h\|^2 = \|w\|^2_{2, K_{N_i} + M} \quad \text{and} \quad \|h\|_{n,i}^2 = 1 \quad \text{implies} \quad w = \Phi_{K_{N_i}}^{-1/2} u \quad \text{where} \quad u \in \mathbb{R}^{K_{N_i}+M} : \|u\|_{2, K_{N_i}+M} = 1. $$

We deduce that

$$ \sup_{h \in S_{K_{N_i}+M}, \|h\|_{n,i}^2=1} \|h\|^2 = \sup_{u \in \mathbb{R}^{K_{N_i}+M}, \|u\|_{2,K_{N_i}+M}=1} u' \Phi_{K_{N_i}}^{-1} u = \left\| \Phi_{K_{N_i}}^{-1} \right\|_{\mathrm{op}}. $$

Furthermore, for all $h = \sum_{\ell=-M}^{K_{N_i}-1} a_\ell B_\ell \in \mathrm{Span} (B_{-M}, \dots, B_{K_{N_i}-1}) = S_{K_{N_i}, M}$, we have on one side

$$ \|h\|_{n,i}^2 = 1 \quad \text{implies} \quad a = \Psi_{K_{N_i}}^{-1/2} u \quad \text{where} \quad u \in \mathbb{R}^{K_{N_i}+M} : \|u\|_{2, K_{N_i}+M} = 1, $$

and on the other side, for all $h \in S_{K_{N_i}+M}$ such that $\|h\|_{n,i}^2 = 1$, from [30] Lemma 2.6, there exists a constant $C > 0$ such that,

$$ \|h\|^2 \le C A_{N_i} K_{N_i}^{-1} \|a\|^2_{2, K_{N_i}+M} = C A_{N_i} K_{N_i}^{-1} u' \Psi_{K_{N_i}}^{-1} u. $$

Then we have a.s

$$ \left\| \Phi_{K_{N_i}}^{-1} \right\|_{\mathrm{op}} = \sup_{h \in S_{K_{N_i}+M}, \|h\|_{n,i}^2=1} \|h\|^2 \le \frac{C A_{N_i}}{K_{N_i}} \sup_{u \in \mathbb{R}^{K_{N_i}+M}, \|u\|_{2,K_{N_i}+M}=1} u' \Psi_{K_{N_i}}^{-1} u = \frac{C A_{N_i}}{K_{N_i}} \left\| \Psi_{K_{N_i}}^{-1} \right\|_{\mathrm{op}}. \quad (A.71) $$

From Equations (A.70) and (A.71), there exists a constant $C > 0$ such that

$$ \mathbb{P}_i \left( \Omega_{n, N_i, K_{N_i}}^c \right) \le 2(K_{N_i} + M) \exp \left( -C \frac{N_i K_{N_i}}{A_{N_i} \mathcal{L}(K_{N_i} + M) \left\| \Psi_{K_{N_i}}^{-1} \right\|_{\mathrm{op}}} \right). \quad (A.72) $$
We have $\mathcal{L}(K_{N_i} + M) := \sup_{x \in [-A_{N_i}, A_{N_i}]} \sum_{\ell=-M}^{K_{N_i}-1} \theta_{\ell}^2(x)$ and the functions $\theta_{\ell}, \ell = -M, \dots, K_{N_i} - 1$ are given by

$$ 
\begin{aligned}
\theta_{-M} &= \frac{f_{-M}}{\|f_{-M}\|} \quad \text{with} \quad f_{-M} = B_{-M} \\
\theta_{\ell} &= \frac{f_{\ell}}{\|f_{\ell}\|} \quad \text{with} \quad f_{\ell} = B_{\ell} - \sum_{k=-M}^{\ell-1} \langle B_{\ell}, \theta_k \rangle \theta_k, \quad \ell = -M+1, \dots, K_{N_i}-1.
\end{aligned}
$$

Note that for all $x \in [-A_{N_i}, A_{N_i}]$, there exists $\ell \in [-M, K_{N_i} - 1]$ such that $x \in [u_\ell, u_{\ell+1})$. Then, $x \in [u_\ell, u_{\ell+M+1})$ for all $\ell' \in [\ell - M, \ell]$ if $\ell \ge 0$ and $\ell' \in [-M, \ell]$ for $\ell \le -1$. Thus, for each $x \in [-A_{N_i}, A_{N_i}]$, there exists at most $M + 1$ spline functions that don't vanish at $x$. As a result, we have on one side,

$$ \forall x \in [-A_{N_i}, A_{N_i}], \quad \sum_{\ell=-M}^{K_{N_i}-1} \theta_{\ell}^2(x) = \sum_{j=1}^{M+1} \theta_{\ell_j}^2(x), \qquad (A.73) $$

where for all $x \in [-A_{N_i}, A_{N_i}]$, there exists integers $\ell_j, j \in [1, M+1]$ such that

$$ x \in \bigcap_{j=1}^{M+1} [u_{\ell_j}, u_{\ell_j+M+1}) \quad \text{and} \quad x \notin [-A_{N_i}, A_{N_i}] \setminus \bigcap_{j=1}^{M+1} [u_{\ell_j}, u_{\ell_j+M+1}). $$

One the other side, for all $\ell \in [1, K_{N_i} - 1]$ and for all $x \in [u_\ell, u_{\ell+M+1})$ there exists at most $M + 1$ integers $\ell_1, \dots, \ell_{M+1}$ such that

$$ \theta_{\ell}(x) = \frac{f_{\ell}(x)}{\|f_{\ell}\|} \quad \text{and} \quad f_{\ell}(x) = B_{\ell}(x) - \sum_{j=1}^{M+1} \langle B_{\ell}, \theta_{\ell_j} \rangle \theta_{\ell_j}(x). $$

Now we focus on the supremum norm of each basis function $\theta_\ell, \ell = -M, \dots, K_{N_i} - 1$. For all each $\ell \in [-M, K_{N_i} - 1]$, since the spline function $B_\ell$ is non-zero, positive and continuous on the interval $[u_\ell, u_{\ell+M+1})$, there exists an interval $[\alpha_\ell, \beta_\ell] \subset [u_\ell, u_{\ell+M+1})$ such that $c_\ell = \inf_{x \in [\alpha_\ell, \beta_\ell]} B_\ell(x) > 0$ where $(\alpha_\ell - \beta_\ell) \propto A_{N_i}/K_{N_i}$ since $\int_{\alpha_\ell}^{\beta_\ell} B_\ell(x)dx \propto A_{N_i}/K_{N_i}$. Then we have

$$ \forall \ell \in [-M, K_{N_i} - 1], \quad \|B_\ell\|^2 = \int_{u_\ell}^{u_{\ell+M+1}} B_\ell^2(x)dx \ge c_\ell \int_{\alpha_\ell}^{\beta_\ell} B_\ell(x)dx = C_\ell \frac{A_{N_i}}{K_{N_i}}, \qquad (A.74) $$

where the constant $C_\ell > 0$ depends on $c_\ell = \inf_{x \in [\alpha_\ell, \beta_\ell]} B_\ell(x) > 0$. Then, for $\ell = -M$, there exists a constant $C_{-M}$ such that $\theta_{-M}^2(x) \le C_{-M} K_{N_i}$ and for each $\ell \ge -M + 1$, since the function $f_\ell$ depends on splines functions $B_{-M}, \dots, B_\ell$ and only $B_\ell$ does not vanish on the interval $[u_{\ell+M}, u_{\ell+M+1})$, we obtain that

$$ \|f_\ell\|^2 = \int_{-A_{N_i}}^{A_{N_i}} f_\ell^2(x) \ge \int_{u_{\ell+M}}^{u_{\ell+m+1}} B_\ell^2(x)dx. $$

Moreover, since $B_\ell$ is non-zero, positive and continue on the interval $[u_{\ell+M}, u_{\ell+M+1})$, there exists an interval $[\alpha_\ell, \beta_\ell] \subset [u_{\ell+M}, u_{\ell+M+1})$ with $(\alpha_\ell - \beta_\ell) \propto A_{N_i}/K_{N_i}$ such that $c_\ell = \inf_{x \in [\alpha_\ell, \beta_\ell]} B_\ell(x) > 0$. Then we obtain

$$ \|f_\ell\|^2 \ge c_\ell \int_{\alpha_\ell}^{\beta_\ell} B_\ell(x)dx = C \frac{A_{N_i}}{K_{N_i}}, \quad \ell \in [-M+1, \dots, K_{N_i}-1], \qquad (A.75) $$

where $C > 0$ is a constant depending on $\min_{\ell=-M+1, \dots, K_{N_i}-1} c_\ell > 0$. On the other side, for all $\ell \in [-M+1, K_{N_i} - 1]$ and for all $x \in [-A_{N_i}, A_{N_i}]$,

$$ |f_\ell(x)| \le |B_\ell(x)| + \sum_{j=-M}^{\ell-1} \frac{\langle B_\ell, f_j \rangle}{\|f_j\|^2} |f_j(x)| \le 1 + C \sum_{j=-M}^{\ell-1} |f_j(x)|, \qquad (A.76) $$
where the constant $C > 0$ is the upper-bound of $\langle B_\ell, f_j \rangle / \|f_j\|^2 \le \|B_\ell\| / \|f_j\|$ according to Equations (A.74) and (A.75). For $\ell = -M$, we have $\|f_{-M}\|_\infty < \infty$. Let $\ell \in [-M+1, K_{N_i}-1]$. Assume that the functions $f_{-M}, \dots, f_{\ell-1}$ are all bounded, then by recurrence hypothesis, we have from Equation (A.76) that

$$ \|f_\ell\|_\infty \le 1 + C \sum_{j=-M}^{\ell-1} \|f_j\|_\infty < \infty. $$

Thus, we obtain by recurrence that the functions $f_\ell$, $\ell = -M, \dots, K_{N_i}-1$ are bounded and finally conclude from Equation (A.73) that

$$ \mathcal{L}(K_{N_i} + M) := \sup_{x \in [-A_{N_i}, A_{N_i}]} \sum_{\ell=-M}^{K_{N_i}-1} \theta_\ell^2(x) \le CK_{N_i}, $$

where the constant $C > 0$ depends on the spline basis. We deduce from Equation (A.72) that there exists a constant $C > 0$ such that

$$ \mathbb{P}_i \left( \Omega_{n,N_i,K_{N_i}}^c \right) \le 2(K_{N_i} + M) \exp \left( -C \frac{N_i}{A_{N_i} \| \Psi_{K_{N_i}}^{-1} \|_{\text{op}}} \right). \quad (A.77) $$

Furthermore, since $A_{N_i} \le \sqrt{\frac{3\beta}{2\beta+1} \log(N_i)}$, $K_{N_i} \propto \log^{-5/2}(N_i) N_i^{1/(2\beta+1)}$ and from Lemma A.4.2, we obtain from Equation (A.77),

$$ \mathbb{P}_i \left( \Omega_{n,N_i,K_{N_i}}^c \right) \le 2(K_{N_i} + M) \exp \left( -C \log^{3/2}(N_i) \right), \quad (A.78) $$

where $C > 0$ is a new constant depending on $C_\theta, \beta$ and $M$. Since $N_i \to \infty$ a.s. as $N \to \infty$, one has

$$ \exp \left( \log(N_i) - C \log^{3/2}(N_i) \right) \to 0 \text{ a.s. as } N \to \infty. $$

Then, for $N$ large enough, $\exp(\log(N_i) - C \log^{3/2}(N_i)) \le 1$ a.s. and from Equation (A.78),

$$ \mathbb{P}_i \left( \Omega_{n,N_i,K_{N_i}}^c \right) \le \frac{2(K_{N_i} + M)}{N_i} \le c \frac{K_{N_i}}{N_i}, $$

where the constant $c > 0$ depends on $M$. $\square$
<figure><img src="image_17.png" /></figure>
# Appendix B

## Classifieur non-paramétrique de type ERM pour les trajectoires de diffusion

**Résumé:** Dans le contexte de la classification multiclasse de trajectoires de diffusion, nous considérons un mélange de processus de diffusion, solutions d'équations différentielles stochastiques avec un même coefficient de diffusion, et des coefficients de dérive qui dépendent des classes modélisées par une variable aléatoire discrète. Nous supposons que les coefficients des processus de diffusion et la loi discrète génèrant les étiquettes sont inconnus, et nous construisons une procédure de classification non-paramétrique basée sur la minimisation du risque empirique de classification. Nous prouvons la consistance du classifieur empirique sous des hypothèses suffisamment faibles, et nous établissons ensuite des vitesses de convergence sous un ensemble d'hypothèses sur le modèle étudié. Enfin, nous complétons nos résultats théoriques par une étude numérique à partir de données simulées.

**Mots clés.** Classification multiclasse, Minimisation du risque empirique, Processus de diffusion, Hypothèse de marge, Observations répétées.

**Abstract:** In the context of multiclass classification of diffusion paths, we consider a mixture of diffusion processes, solutions of stochastic differential equations with a common diffusion coefficient, and the drift coefficients which depend on the classes modeled by a discrete random variable. We assume that the coefficients of the diffusion processes together with the discrete law that generate the classes are unknown, and we construct a nonparametric classification procedure based on the minimization of the empirical classification risk. We prove the consistency of the empirical classifier under mild assumptions and derive some rates of convergence under different sets of assumptions. Finally, we complete our theoretical results with a numerical study over simulated diffusion paths.

**Keywords.** Multiclass classification, Empirical risk minimization, Diffusion process, Margin assumption, Repeated observations.

## Contents

<table><tr><td>B.1</td><td>Introduction</td><td>134</td></tr><tr><td>B.2</td><td>Model and assumptions</td><td>135</td></tr><tr><td>B.3</td><td>Empirical classification procedure</td><td>136</td></tr><tr><td>B.3.1</td><td>Convexification of the problem</td><td>136</td></tr><tr><td>B.3.2</td><td>The nonparametric ERM type classifier</td><td>137</td></tr><tr><td>B.4</td><td>Theoretical properties of the ERM type classifier</td><td>138</td></tr><tr><td>B.5</td><td>Faster rate of convergence under margin assumption</td><td>139</td></tr><tr><td>B.5.1</td><td>The Bayes Classifier</td><td>139</td></tr><tr><td>B.5.2</td><td>Control of the margin</td><td>140</td></tr><tr><td>B.6</td><td>Numerical illustration</td><td>140</td></tr></table>
<table><tr><td>B.6.1</td><td>Models and simulations</td><td>140</td></tr><tr><td>B.6.2</td><td>Numerical results</td><td>141</td></tr><tr><td>B.7</td><td>Conclusion and discussion</td><td>141</td></tr><tr><td>B.8</td><td>Proofs</td><td>142</td></tr><tr><td>B.8.1</td><td>Proof of Theorem B.4.1</td><td>142</td></tr><tr><td>B.8.2</td><td>Proof of Proposition B.5.2</td><td>150</td></tr></table>

## B.1 Introduction

In this chapter, we tackle the multiclass supervised classification of trajectories of time-homogeneous diffusion processes. More precisely, we consider labelled trajectories generated by a random pair $(X, Y)$ from a probability space $(\Omega, \mathcal{F}, \mathbb{P}_{X,Y})$, with the feature $X$ coming from a mixture of diffusion processes, and the label $Y \in \mathcal{Y} = \{1, \dots, K\}, K \ge 2$, a random variable of discrete law $p^* = (p_1^*, \dots, p_K^*)$. We place ourselves in a nonparametric setting where for each label $i \in \mathcal{Y}$, and given $\{Y = i\}$, the diffusion process $X$ is solution of a stochastic differential equation whose drift function $b_i^*$ is assumed to be unknown and depends on the class $i$, and whose diffusion coefficient $\sigma^*$, also unknown, is common to all classes. Moreover, the discrete law $p^*$ of the label $Y$ is also assumed to be unknown. We suppose to have at our disposal a learning sample $\mathcal{D}_N = \{(\bar{X}^j, Y_j), j = 1, \dots, N\}$ composed of $N$ independent copies of the random couple $(X, Y)$, where the diffusion process $X$ is observed at discrete times. We build from $\mathcal{D}_N$, an Empirical Risk Minimization (ERM) type classifier $\hat{g}$ that mimics the Bayes classifier $g^*$, minimizer of the classification risk, that is,

$$g^* \in \arg \min_{g} \mathbb{P}_{X,Y}(g(X) \neq Y),$$

and whose performance is assessed by its excess risk $\mathbb{P}(\hat{g}(X) \neq Y) - \mathbb{P}(g^*(X) \neq Y)$ with respect to the Bayes classifier.

The ERM type classifier is studied in the literature, for finite-dimensional data (see e.g. [76], [98]), and for functional data modelled by diffusion processes (see [11], [29]). Not far from our framework, the ERM-type classifiers proposed in [11] and [29] reach rates of convergence of order $N^{-\alpha/(2\alpha+1)}$ with $\alpha$ a regularity parameter. The rate is obtained in [11] in the context of binary classification, and in [29] in the multiclass classification setup, where the diffusion coefficient is assumed to be known. However, both ERM type classifiers are not implementable in practice. In this chapter, we propose an implementable multiclass classification procedure based on the convexification of the model (see [102], [8]), where the drift functions $b_i^*$, $i \in \mathcal{Y}$, and the diffusion coefficient $\sigma^*$ are unknown.

The results in this chapter are resumed as follows:

i) We prove the consistency of the ERM type classifier, and derive a rate of convergence of order $N^{-\beta/(2\beta+1)}$ over the Hölder space $\Sigma(\beta, R)$ of regularity parameter $\beta \ge 1$, given by

$$\Sigma(\beta, R) = \{ f \in C^{[\beta]+1}(\mathbb{R}) : |f^{(\ell)}(x) - f^{(\ell)}(y)| \le R |x - y|, x, y \in \mathbb{R} \}$$

with $\ell = \lfloor \ell \rfloor$, and $R > 0$.

ii) In the binary classification setup, we make a margin assumption on the regression function associated to the diffusion model, and establish a rate of convergence of the resulting ERM type classifier of order $N^{-4\beta/3(2\beta+1)}$ over the Hölder space $\Sigma(\beta, R)$ with $\beta \ge 1$.

In Section B.2, we present the diffusion model and the assumptions on the drift and diffusion coefficients. Section B.3 is devoted to the construction of the nonparametric ERM type classifier. The theoretical properties of the empirical classifier are given in Section B.4, and a faster rate of a binary ERM type classifier is established in Section B.5. A numerical study over simulated data is made in Section B.6, and Section B.8 is devoted to the proofs of our results.
## B.2 Model and assumptions

We consider a multiclass classification model in which the feature $X = (X_t)_{t \in [0,1]}$ is a diffusion process, solution of the following mixture model:

$$dX_t = b_Y^*(X_t)dt + \sigma^*(X_t)dW_t, \quad t \in [0, 1], \quad X_0 = 0, \qquad (B.1)$$

where $Y \in \mathcal{Y} = \{1, \dots, K\}$, $K \ge 2$, is a discrete random variable with an unknown distribution $\mathfrak{p}^* = (\mathfrak{p}_1^*, \dots, \mathfrak{p}_K^*)$, and $(W_t)_{t \ge 0}$ is a standard Brownian motion independent of $Y$. Thus, each label data is generated by the random couple $(X, Y)$ where $X$ is the feature and $Y$ the corresponding class. We assume that the distribution of the random pair $(X, Y)$ is unknown. More precisely, we assume that the coefficients $b_i^*$, $i \in \mathcal{Y}$, and $\sigma^*$ of the diffusion processes are unknown. Moreover, we assume that $0 < \mathfrak{p}_0^* = \min_{i \in \mathcal{Y}} \mathfrak{p}_i^*$, and we set $\mathfrak{b}^* = (b_1^*, \dots, b_K^*)$.

The goal is to construct a measurable function $g$ over the set of diffusion processes solutions of (B.1), and called classifier, whose role is to predict, for each feature $X$, a value $g(X) \in \mathcal{Y}$ which corresponds to the associated label $Y$. Let $\mathcal{G}$ be a set of classifiers $g: \mathcal{X} \to \mathcal{Y}$ that will be defined in Section B.3.2. The performance of each $g \in \mathcal{G}$ is assessed by $\mathcal{R}(g)$, where $\mathcal{R}$ is the classification risk defined in Chapter 1, Section A.2.

**Assumptions.** We assume that the drift functions $b_i^*$, $i \in \mathcal{Y}$ and diffusion coefficient $\sigma^*$ satisfy Assumptions A.2.1 and A.2.2 defined in Chapter A. We recall that under Assumption A.2.1, there exists a diffusion process $X$, unique strong solution of the model (B.1), which admits a transition density $p_X: (t, x) \mapsto p_X(t, x)$ given by

$$p_X = \sum_{i=1}^{K} \mathfrak{p}_i^* p_{i,X},$$

where each $p_{i,X}$ is the transition density of the diffusion process $X$ given $\{Y = i\}$ (see e.g. [46], [55]).

**Bayes classifier.** We also recall that, under Assumptions A.2.1 and A.2.2, the Bayes classifier $g^*$ defined in Chapter A, Section A.2.2 is characterized for all $X \in \mathcal{X}$ by

$$g^*(X) = \arg\max_{i \in \mathcal{Y}} \pi_i^*(X),$$

where,

$$\forall i \in \mathcal{Y}, \quad \pi_i^*(X) = \phi_i^*(\mathbf{F}^*(X)), \quad \mathbf{F}^* = (F_1^*, \dots, F_k^*)$$

with, for each $i \in \mathcal{Y}$,

$$F_i^*(X) := \int_0^1 \frac{b_i^*}{\sigma^{*2}}(X_s) dX_s - \frac{1}{2} \int_0^1 \frac{b_i^{*2}}{\sigma^{*2}}(X_s) ds,$$

and $\phi_i^*: (x_1, \dots, x_K) \mapsto \frac{\mathfrak{p}_i^* e^{x_i}}{\sum_{k=1}^K \mathfrak{p}_k^* e^{x_k}}$ are the softmax functions.

As we observe the diffusion paths in discrete time, denote by $\bar{g}^*$ the discrete time version of the Bayes classifier $g^*$ defined by

$$\bar{g}^*(X) = \arg\max_{i \in \mathcal{Y}} \bar{\pi}_i^*(X),$$

where $\bar{\pi}_i^*(X) = \phi_i^*(\bar{\mathbf{F}}^*(X))$, with $\bar{\mathbf{F}}^* = (\bar{F}_1^*, \dots, \bar{F}_K^*)$, and for each $i \in \mathcal{Y}$,

$$\bar{F}_i^* = \sum_{k=0}^{n-1} \frac{b_i^*}{\sigma^{*2}}(X_{k\Delta_n}) (X_{(k+1)\Delta_n} - X_{k\Delta_n}) - \frac{\Delta_n}{2} \sum_{k=0}^{n-1} \frac{b_i^{*2}}{\sigma^{*2}}(X_{k\Delta_n}).$$
## B.3 Empirical classification procedure

Consider the learning sample $\mathcal{D}_N = \{(\bar{X}^j, Y_j), j = 1, \dots, N\}$ composed of $N$ independent copies of the random pair $(\bar{X}, Y)$ with $\bar{X} = (X_{k\Delta_n})_{0\le k\le n}$ with $\Delta_n = 1/n$ the time-step. The empirical classification risk $\hat{\mathcal{R}}: \mathcal{G} \to [0, 1]$ is defined for all $g \in \mathcal{G}$ by

$$ \hat{\mathcal{R}}(g) := \frac{1}{N} \sum_{j=1}^{N} 1_{g(\bar{X}^j) \neq Y_j}, $$

and converges almost surely toward the theoretical risk $\mathcal{R}(g)$. The ERM type classifier $\hat{g}$ is given by

$$ \hat{g} \in \arg \min_{g \in \mathcal{G}} \hat{\mathcal{R}}(g). \qquad (B.2) $$

In the literature, classification procedures of the ERM type have been proposed both for functional data (see [11], [29]), and for multivariate data (see e.g. [98], [76]).

### B.3.1 Convexification of the problem

Since the set $\mathcal{G}$ of classifiers together with the empirical classification risk $\hat{\mathcal{R}}$ are non-convex, the classifier $\hat{g}$, solution of a non-convex optimization problem (B.2) is NP-hard to compute in practical situations, or even intractable considering the nature of the procedure (see e.g. [8], [3]). To overcome this problem and construct a classification procedure easier to compute in practice, we propose convex surrogates of both the loss function and the set $\mathcal{G}$ of classifiers as follows:

* The set of classifiers $\mathcal{G}$ is replaced by the convex space of score functions

$$ \mathcal{H} = \{ \mathbf{h} = (h_1, \dots, h_K) : \mathcal{X} \to \mathbb{R}^K \} $$

so that for each classifier $g \in \mathcal{G}$, there exists a score function $h \in \mathcal{H}$ such that for all $X \in \mathcal{X}$,

$$ g(X) = \arg \max_{i \in \mathcal{Y}} h^i(X). $$

From now, we set $g = g_h$ to link a classifier $g \in \mathcal{G}$ to its corresponding score function $h \in \mathcal{H}$.

* The non-convex loss function $(y, y') \in \mathcal{Y}^2 \mapsto 1_{y \neq y'}$ is replaced by the quadratic loss function $x \mapsto (1-x)^2$. Then, the score functions $h$ are now regarded as new classifiers whose performance is assessed by the new classification risk $\mathcal{R}: \mathcal{H} \to \mathbb{R}_+$ given for each $h \in \mathcal{H}$ by

$$ \mathcal{R}(h) := \sum_{i=1}^{K} \mathbb{E} \left[ (1 - Z_i h^i(X))^2 \right] $$

where $Z = (Z_1, \dots, Z_K)$ is a random vector with, for each $i \in \mathcal{Y}$, $Z_i = 2 \mathbb{1}_{Y=i} - 1 \in \{-1, 1\}$. So the new Bayes classifier $h^*$ is the minimizer of the classification risk $\mathcal{R}$ over $\mathcal{H}$, and is characterized for all $X \in \mathcal{X}$ by

$$ h^*(X) := 2\pi_i^*(X) - 1. $$

We define the empirical classifier $\hat{h}$ by

$$ \hat{h} \in \arg \min_{h \in \mathcal{H}} \hat{\mathcal{R}}(h) \qquad (B.3) $$

where $\hat{\mathcal{R}}$ is the new empirical risk of classification defined from the learning sample $\mathcal{D}_N$ and for each classifier $h \in \mathcal{H}$ by

$$ \hat{\mathcal{R}}(h) := \frac{1}{N} \sum_{j=1}^{N} \sum_{i=1}^{K} (1 - Z_i h^i(\bar{X}^j))^2. $$
As a result, the empirical classifier $\hat{g} = g_{\hat{h}}$ is given for all $X \in \mathcal{X}$ by

$$ \hat{g}(X) = g_{\hat{h}}(X) = \arg\max_{i \in \mathcal{Y}} \hat{h}^i(X). $$

We establish the following result relating the excess risk $\mathcal{R}(g_{\hat{h}}) - \mathcal{R}(g_{h^*})$ of the empirical classifier $g_{\hat{h}}$ to the excess risk $\mathfrak{R}(\hat{h}) - \mathfrak{R}(h^*)$ of the corresponding empirical classifier $\hat{h}$.

**Proposition B.3.1** (Zhang (2004) [102]). The empirical classifiers $\hat{g} = g_{\hat{h}}$ and $\hat{h}$, respective solutions of Equations (B.3) and (B.4), satisfy the following relation

$$ \mathcal{R}(g_{\hat{h}}) - \mathcal{R}(g_{h^*}) \le \frac{1}{\sqrt{2}} \sqrt{\mathfrak{R}(\hat{h}) - \mathfrak{R}(h^*)}. $$

The result of Proposition B.3.1 links the excess risk of any classifier $g = g_h \in \mathcal{G}$ to the excess risk of its corresponding classifier $h \in \mathcal{H}$. Thus, the consistency of the empirical classifier $\hat{h}$ solution of the convex programming problem (B.3) implies, through Proposition B.3.1, the consistency of the corresponding empirical classifier $\hat{g} = g_{\hat{h}}$ solution of Equation (B.3). Furthermore, the result of Proposition B.3.1 allows us to deduce the rate of convergence of the ERM type classifier $g_{\hat{h}}$ from the rate of $\hat{h}$.

### B.3.2 The nonparametric ERM type classifier

Let $S_{K_N}$ be the space of approximation of the unknown coefficients of the diffusion processes solution of (B.1), generated by the B-spline basis $\{B_\ell, \ell = -M, \dots, K_N - 1\}$ built on the interval $[-\log(N), \log(N)]$, with $M, K_N \ge 1$, and given as follows:

$$ \begin{aligned} S_{K_N} &:= \left\{ \mathfrak{f} = \sum_{\ell=-M}^{K_N-1} a_\ell B_\ell, \sum_{\ell=-M}^{K_N-1} a_\ell^2 \le (K_N + M) \log(N) \right\}, \\ \tilde{S}_{K_N} &:= \left\{ x \mapsto \mathfrak{s}^2(x) = \mathfrak{f}(x) \mathbb{1}_{f(x) \ge 1/\log^2(N)} + \frac{1}{\log^2(N)} \mathbb{1}_{f(x) \le 1/\log^2(N)}, \mathfrak{f} \in S_{K_N} \right\}. \end{aligned} $$

Let $\mathcal{X}$ be the set of diffusion paths, and define the set of classifiers $\mathcal{G}$ by

$$ \mathcal{G} = \{g_{\mathbf{b},\sigma^2,\mathbf{p}} : \mathcal{X} \to \mathcal{Y} \mid (\mathbf{b}, \sigma^2) \in (S_{K_N})^K \times \tilde{S}_{K_N}, \mathbf{p} \in (0,1)^K\}, $$

where each $g_{\mathbf{b},\sigma^2,\mathbf{p}}$ is a plug-in classifier of $g^* = g_{\mathbf{b}^*,\sigma^2,\mathbf{p}^*}$. Thus, the ERM type classifier $\hat{g}$ is the minimizer of the empirical classification risk $\hat{\mathcal{R}}$ on the set $\mathcal{G}$, that is,

$$ \hat{g} \in \arg\min_{g \in \mathcal{G}} \hat{\mathcal{R}}(g). \qquad (B.4) $$

Recall that the Bayes classifier $h^*$ is given for all $X \in \mathcal{X}$ and for all $i \in \mathcal{Y}$ by

$$ h^{*i}(X) = 2\pi_i^*(X) - 1. $$

Then, each coordinate function $h^{*i}$ takes its values in the compact interval $[-1, 1]$. As a result, setting $\pi_i^* = \pi_{b_i^*, \sigma^2, \mathbf{p}^*}$ for each $i \in \mathcal{Y}$, we restrict the convex space $\mathcal{H}$ to the space $\mathcal{H}$ defined by

$$ \mathcal{H} := \{h = (h^1, \dots, h^K) \in \mathcal{H} \mid \forall i \in \mathcal{Y}, h^i(X) = 2\pi_{\mathbf{b},\sigma^2,\mathbf{p}}(X) - 1, \mathbf{b} \in (S_{K_N}^b)^K, \sigma^2 \in \tilde{S}_{K_N}, \mathbf{p} \in (0,1)^K\}, \qquad (B.5) $$

and we define the empirical classifier $\hat{g} = g_{\hat{h}}$, where $\hat{h}$ is given by

$$ \hat{h} = \arg\min_{h \in \mathcal{H}} \mathfrak{R}(h). $$

We rather consider in the sequel, the empirical classifier $\hat{g}$ which has the benefit of being implementable thanks to the convexification of the problem.
## B.4 Theoretical properties of the ERM type classifier

We study in this section the consistency of our ERM type classifier $\hat{g}$ and derive a rate of convergence under mild assumptions on the unknown functions $b^*$ and $\sigma^{*2}$. We define from the learning sample $D_N$, the following two learning subsamples

$$D_N^{(0)} = \{(\bar{X}^{0,j}, Y_j^0) \in D_N, j = 1, \dots, N_0\},$$

and

$$D_N^{(1)} = \{(\bar{X}^{1,j}, Y_j^1) \in D_N, j = 1, \dots, N_1\}$$

of respective sizes $N_0$ and $N_1$, with $N_0 + N_1 = N$, and $N_0, N_1 \to \infty$ as $N \to \infty$ (for example $N_0 = \lfloor \frac{1}{4}N \rfloor$ and $N_1 = N - N_0$). Then, we build from the learning sample $D_N^{(0)}$, the estimator $\hat{\mathfrak{p}}$ of $\mathfrak{p}^*$ given by

$$\hat{\mathfrak{p}}_i := \frac{1}{N_0} \sum_{j=1}^{N_0} \mathbb{1}_{Y_j^0=i}, \quad i \in \mathcal{Y},$$

and from the learning subsample $D_N^{(1)}$, we build the empirical classifier $\hat{g} = g_{\hat{\mathfrak{b}}, \hat{\sigma}^2, \hat{\mathfrak{p}}}$ as follows:

$$\hat{g} \in \arg\min_{(\mathbf{b}, \sigma^2) \in \mathcal{W}_{K_N, L_N}} \mathcal{R}(g_{\mathbf{b}, \sigma^2, \hat{\mathfrak{p}}})$$

where $\mathcal{W}_{K_N} = (S_{K_N})^K \times \tilde{S}_{K_N}$. After the convexification of the model, we have that $\hat{g} = g_{\hat{h}}$ where the score function $\hat{h}$ is built from the learning sample $D_N^{(1)}$ by

$$\hat{h} \in \arg\min_{(\mathbf{b}, \sigma^2) \in \mathcal{W}_{K_N}} \hat{\mathcal{R}}(h_{\mathbf{b}, \sigma^2, \hat{\mathfrak{p}}}).$$

We establish below an upper-bound of the average excess risk of the empirical classifier $\hat{g}$.

**Theorem B.4.1.** Under Assumptions A.2.1 and A.2.2, the following holds,

$$\mathbb{E}[\mathcal{R}(g_{\hat{h}})] - \mathcal{R}(g_{h^*}) \le C \left( \frac{\log^{3/2}(N)}{K_N} + \left( \frac{K_N \log(N)}{N} \right)^{1/2} + \Delta_n^{1/2} + \sqrt{\frac{n}{\log(N)}} \exp\left(-\frac{c}{2} \log^2(N)\right) \right)$$

where $C > 0$ is a constant independent of $N$ and $n$.

We obtain the consistency of the ERM type classifier $\hat{g} = g_{\hat{h}}$ from the result of Theorem B.4.1 with $K_N$ of polynomial growth, and $n \propto N$. Note that the above result is obtained in the general case where the functions $b_i^*$ and $\sigma^*$ are unknown and non-constant.

If we assume that the functions $b_i^*$ and $\sigma^{*2}$ belong to the Hölder space $\Sigma(\beta, R)$, with $\beta \ge 1$ and $R > 0$, given by

$$\Sigma(\beta, R) := \{ f \in \mathbb{C}^{\lfloor \beta \rfloor+1}(\mathbb{R}), \quad |f^{(\ell)}(x) - f^{(\ell)}(y)| \le R |x - y|^{\beta-\ell}, \ x, y \in \mathbb{R} \},$$

then we obtain the following result.

**Corollary B.4.2.** Suppose that $b_i^*, \sigma^{*2} \in \Sigma(\beta, R)$, $K_N \propto N^{1/(2\beta+1)}$ and $N \propto n$. Under Assumptions A.2.1 and A.2.2, the following holds:

$$\mathbb{E}[\mathcal{R}(g_{\hat{h}})] - \mathcal{R}(g_{h^*}) = O(N^{-\beta/(2\beta+1)}).$$

The obtained rate of convergence is of the same order than the one established in chapter A for the plug-in type classifier where the diffusion coefficient is known and constant. Note that, contrary to the case of the plug-in type classifier which is based on the nonparametric estimators of the coefficients of the diffusion processes, the ERM type classifier focuses on the minimization of the empirical
classification risk which approximated the performance measurement for classifiers. In a nonpara-
metric multiclass classification framework, and in a non-convex setup, Denis et al. (2020) ([29]) built
a ERM type classifier, where the diffusion coefficient is assumed to be known. They established a
rate of order $N^{-\beta/(2\beta+1)}$ over a Hölder space of smoothness parameter $\beta \ge 1$. In our framework, we
consider the set of classifiers

$$
\mathcal{G} = \{g_b, \mathbf{b} = (b_1, \dots, b_K) \in \mathcal{S}_{K_N}^K\},
$$

and we derive the following result.

**Theorem B.4.3.** Suppose that $b_i^*, \sigma^{*2} \in \Sigma(\beta, R)$ with $\beta \ge 1$ and $R > 0$. Under Assumptions A.2.1 and A.2.2, the following holds:

$$
\mathbb{E}[\mathcal{R}(\hat{g})] - \mathcal{R}(g^*) = O\left(N^{-\beta/(2\beta+1)}\right).
$$

The result of Theorem B.4.3 is obtained using the similar arguments as in Denis et al. (see [29]). In the next section, we study a faster rate of convergence under stronger assumptions on our model.

B.5 Faster rate of convergence under margin assumption

We suppose that $K = 2$, $\mathbf{p}^* = (1/2, 1/2)$ the known discrete law of the label $Y = \{-1, 1\}$ and the diffusion coefficient $\sigma^*$ is known with $\sigma^* = 1$. Denote by $\eta^*$ the regression function given for each feature $X$ by $\eta^*(X) = \mathbb{P}(Y = 1|X)$. Let $\tilde{\mathbb{P}}$ be a probability measure under which the diffusion process $X = (X_t)_{t \in [0,1]}$ is solution of the S.D.E. $dX_t = \sigma(X_t)d\tilde{W}_t$, where $\tilde{W}$ is a brownian motion under $\tilde{\mathbb{P}}$. Under Assumptions A.2.1 and A.2.2, the Girsanov's theorem (see [86]) implies

$$
\frac{d\mathbb{P}_0}{d\tilde{\mathbb{P}}}(X) = \exp\left(\int_0^1 \frac{b_0^*}{\sigma^2}(X_s)dX_s - \int_0^1 \frac{b_0^{*2}}{2\sigma^2}(X_s)ds\right) =: q_0(X)
$$

$$
\frac{d\mathbb{P}_1}{d\tilde{\mathbb{P}}}(X) = \exp\left(\int_0^1 \frac{b_1^*}{\sigma^2}(X_s)dX_s - \int_0^1 \frac{b_1^{*2}}{2\sigma^2}(X_s)ds\right) =: q_1(X)
$$

where $\mathbb{P} = \frac{1}{2}\mathbb{P}_0 + \frac{1}{2}\mathbb{P}_1$. Then we have

$$
\eta^*(X) := \mathbb{P}(Y = 1|X) = \frac{\mathbb{P}(Y = 1) \frac{d\mathbb{P}_1}{d\mathbb{P}}(X)}{\mathbb{P}(Y = 0) \frac{d\mathbb{P}_0}{d\mathbb{P}}(X) + \mathbb{P}(Y = 1) \frac{d\mathbb{P}_1}{d\mathbb{P}}(X)} = \frac{\frac{d\mathbb{P}_1}{d\mathbb{P}}(X)}{\frac{d\mathbb{P}_0}{d\mathbb{P}}(X) + \frac{d\mathbb{P}_1}{d\mathbb{P}}(X)}.
$$

B.5.1 The Bayes Classifier

In binary classification, the Bayes classifier $g^*$ is defined from the regression function $\eta^*$ by $g^* = 1_{\eta^* \ge 1/2}$. Thus, for any diffusion process $X$, we have

$$
g^*(X) = 1_{\eta(X) \ge \frac{1}{2}} = 1_{\frac{d\mathbb{P}_1}{d\mathbb{P}}(X) \ge \frac{d\mathbb{P}_0}{d\mathbb{P}}(X)} = 1_{\int_0^1 \frac{b_1^* - b_0^*}{\sigma^2}(X_s)dX_s \ge \int_0^1 \frac{b_1^{*2} - b_0^{*2}}{2\sigma^2}(X_s)ds}.
$$

As we already know, the Bayes classifier $g^*$ minimizes the classification error $g \mapsto \mathcal{R}(g) = \mathbb{P}(g(X) \neq Y)$. As a result, the performance of any classifier $g \in \mathcal{G}$ is assessed through its excess risk with respect to $g^*$ given as follows:

$$
\mathcal{R}(g) - \mathcal{R}(g^*) = \mathbb{E}[|2\eta^*(X) - 1| \mathbf{1}_{g(X) \neq g^*(X)}],
$$

(see e.g. [47], [29]).
## B.5.2 Control of the margin

We make the following assumptions on the drift functions $b_0^*$ and $b_1^*$.

**Assumption B.5.1.** The drift functions $b_0^*$ and $b_1^*$ are bounded on the real line, and the random variable

$$Z := \frac{1}{\Delta_{b^*}} \int_{0}^{1} (b_{1}^{*} - b_{0}^{*})(X_{s}) dW_{s}, \quad \text{with} \quad \Delta_{b^{*}}^{2} = \mathbb{E} \left[ \int_{0}^{1} (b_{1}^{*} - b_{0}^{*})^{2} (X_{s}) ds \right] > 0,$$

has a density function that is bounded on $\mathbb{R}$.

The existence of a density function of the random variable $Z$ is proven using Maliavin calculus and the Hörmander's condition (see [79], Chapter 2, Theorem 2.3.3). The strictly positive constant $\Delta_b$ which is the minimum separation distance between the respective curves of the drift functions $b_0^*$ and $b_1^*$ characterizes the margin between the two classes. The condition $\Delta_b > 0$ implies that the two classes do not mix. We obtain the following result.

**Proposition B.5.2.** Under Assumptions A.2.1 and B.5.1 and for all $\varepsilon \in (0, 1/8)$, there exists a constant $C > 0$ such that

$$\mathbb{P} \left( 0 < \left| \eta^*(X) - \frac{1}{2} \right| \le \varepsilon \right) \le C \frac{12}{\Delta_{b^*}} \varepsilon.$$

We deduce from [8], Lemma 5 (and its proof) that the result of Proposition B.5.2 implies that for all classifier $g \in \mathcal{G}$,

$$\mathbb{P}(g(X)(\eta^*(X) - 1/2) < 0) \le \frac{c}{\sqrt{\Delta_{b^*}}} (\mathcal{R}(g) - \mathcal{R}(g^*))^{1/2}$$

where $c > 0$ is a constant. Moreover, Theorem 3 in [8] implies that for all classifier $g \in \mathcal{G}$, there exist a classifier $h \in \mathcal{H}$ and a constant $c > 0$ such that

$$\mathcal{R}(g) - \mathcal{R}(g^*) \le \frac{c}{\Delta_{b^*}^{1/3}} (\mathfrak{R}(h) - \mathfrak{R}(h^*))^{2/3}. \qquad (B.6)$$

We finally deduce from Theorem B.4.1, Assumption B.5.1 and Equation (B.6) the following result.

**Theorem B.5.3.** Suppose that $b_0^*, b_1^* \in \Sigma(\beta, R)$. Under Assumptions A.2.1, A.2.2 and B.5.1, the following holds:

$$\mathbb{E} [\mathcal{R}(g_{\hat{h}})] - \mathcal{R}(g_{h^*}) = O(N^{-4\beta/3(2\beta+1)}).$$

where $g_{\hat{h}} = \hat{g}$ and $g_{h^*} = g^*$.

Thus, we have a faster rate of convergence compared to the result of Theorem B.4.2 thanks to the margin assumption characterized by Proposition B.5.2 which minimizes the wrong prediction of labels when the probability of belonging of the new observation to the class 1 is too close to 1/2.

# B.6 Numerical illustration

In this section, we make a numerical study of the ERM type classifier over synthetic data. We present the diffusion models in Section B.6.1, and the numerical results in Section B.6.2.

## B.6.1 Models and simulations

We fix the number of classes to $K = 3$, and the discrete law of the label $Y \in \mathcal{Y} = \{1, \dots, 3\}$ is $\mathfrak{p}^* = \{1/3, 1/3, 1/3\}$. We study the two following diffusion models:

Model 1: $b(\theta, x) = -(x - \theta)$, $\theta \in \{-1, 0, 1\}$, $\sigma(x) = 1$

Model 2: $b(\theta, x) = \theta [1/4 + (3/4) \cos^2 x]$, $\theta \in \{1, 3/2, -3/2\}$, $\sigma(x) = 0.1 + 0.9/\sqrt{1 + x^2}$.
<table><caption>Table B.1: Classification risks of the Bayes classifier $g^*$ from learning samples of size $N = 4000$ with $n = 500$.</caption><thead><tr><th></th><th>Model 1</th><th>Model 2</th></tr></thead><tbody><tr><th>ŷ(g*)</th><td>0.41 (0.01)</td><td>0.36 (0.01)</td></tr></tbody></table>

The two diffusion models satisfy Assumptions A.2.1 and A.2.2. We investigate the numerical consistency of the empirical classifier $\hat{g} = g_{\hat{h}}$ using learning samples of size $N \in \{100, 1000\}$ composed of trajectories of length $n = 100$. We use a learning sample of size $N = 4000$ with diffusion paths of length $n = 500$ for the evaluation of the Bayes classifier $g^*$.

We deduce from the result of Table B.1 that Model 2 is simpler than Model 1 since the Bayes classifier of Model 2 has a better performance. Moreover, from the definition of the Bayes classifier, we known that the ERM type classifier could have at most the performance of the Bayes classifier given in Table B.1.

## B.6.2 Numerical results

We build the ERM type classifier from learning samples of size $N \in \{100, 1000\}$, with each diffusion path of length $n = 100$. The approximation subspace $S_{KN}$ spanned by the B-spline basis is chosen such that $M = 3, K_N = 4$ for the approximation of the drift function and of the square of the diffusion coefficient. The B-spline basis is built on the compact interval $[-A_N, A_N]$ with $A_N = 10$. We evaluate the performance of the empirical classifier on test samples of size $N_{test} = 10000$ over 20 repetitions. The table below compares the performance of our classifier with that of the classifiers for functional data based on the random forest and the notion of depth for functional data.

<table><caption>Table B.2: Performance of the ERM type classifier, and comparison with that of the random forest and the depth based classifier for diffusion paths collected at discrete times with the grid $n = 100$.</caption><thead><tr><th rowspan="2">R̂(ĝ)</th><th colspan="2">ERM</th><th colspan="2">Random Forest</th><th colspan="2">Depth</th></tr><tr><th>N = 100</th><th>N = 1000</th><th>N = 100</th><th>N = 1000</th><th>N = 100</th><th>N = 1000</th></tr></thead><tbody><tr><th>Model 1</th><td>0.43 (0.01)</td><td>0.41 (0.01)</td><td>0.46 (0.02)</td><td>0.44 (0.01)</td><td>0.44 (0.02)</td><td>0.43 (0.01)</td></tr><tr><th>Model 2</th><td>0.38 (0.02)</td><td>0.36 (0.01)</td><td>0.40 (0.01)</td><td>0.38 (0.01)</td><td>0.40 (0.02)</td><td>0.39 (0.01)</td></tr></tbody></table>

We observe from Table B.2 that the ERM type classifier performs better with respect to the classifiers for functional data based respectively on the random forest and the depth of functional data. The main benefit with our classifier is that its construction takes into account the theoretical properties of the diffusion processes that generate the data.

The diffusion models used for this illustration are exactly the same chosen for the numerical study of the performance of the plug-in type classifier studied in Chapter A. We remark that the ERM type classifier performs better compared to the plug-in type classifier. This result is explained by the fact that contrary to the plug-in type classifier, the construction of the ERM type classifier is based on the minimization of the empirical risk of classification. Note that the evaluation of the numerical performance of the ERM type classifier is not optimal since there is not a data-driven selection of dimension of the approximation space.

## B.7 Conclusion and discussion

In this chapter, we have built a nonparametric classification procedure by minimizing the empirical risk of classification from the learning sample. We placed ourselves in the case where the drift functions $b_i^*$, $i \in \mathcal{Y}$ and the diffusion coefficient $\sigma^*$ are unknown, and we derived, under mild assumptions on these unknown functions, a rate of convergence of the resulting ERM type classifier of
order $N^{-\beta/(2\beta+1)}$ over the Hölder space of smoothness parameter $\beta \ge 1$. In addition to the fact that our empirical classifier is implementable thanks to the convexification of the model, this theoretical result extends the results established in [11] and [29] to a nonparametric setup with an unknown diffusion coefficient. Finally, under a margin assumption on the regression function in binary classification setup, we derived a faster rate of order $N^{-4\beta/3(2\beta+1)}$ over the Hölder space $\Sigma(\beta, R)$ with $\beta \ge 1$.

It should be noted, however, that the nonparametric classification procedure built in this chapter is based on the minimization of a convexified optimization problem over a finite-dimensional approximation space whose dimension is not optimally selected from the data. As a result, a immediate perspective could be to propose an adaptive ERM type classifier based on a data-driven selection of the dimension of the approximation space. Moreover, one may also focus on the study of the optimality of the obtained rates of convergence both under mild assumptions on the coefficients of the diffusion processes and under a margin assumption on the regression function associated to the model. We could also extend the obtained results to the case of in-homogeneous diffusion processes.

## B.8 Proofs

For simplicity, we denote the Bayes classifier $g_{h^*}$ by $g^*$, and any other classifier $g_h$ by $g$, where $h \in \mathfrak{H}$ is a score function.

### B.8.1 Proof of Theorem B.4.1

*Proof.* According to the result of Proposition B.3.1

$$ \mathbb{E}[\mathcal{R}(\hat{g})] - \mathcal{R}(g^*) \le \sqrt{\mathbb{E}[\mathcal{R}(\hat{h})] - \mathcal{R}(h^*)} $$

Denote by $\bar{h}_*$, the discrete time version of the Bayes classifier $h^*$. We have the following decomposition of the excess risk of $\hat{h}$,

$$ \mathfrak{R}(\hat{h}) - \mathfrak{R}(h^*) = \mathfrak{R}(\hat{h}) - \mathfrak{R}(\bar{h}_*) + \mathfrak{R}(\bar{h}_*) - \mathfrak{R}(h^*) $$

**Upper bound of $\mathfrak{R}(\bar{h}_*) - \mathfrak{R}(h^*)$**

$$ \begin{aligned} \mathfrak{R}(\bar{h}_*) - \mathfrak{R}(h^*) &= 4 \sum_{i=1}^{K} \mathbb{E} |\bar{\pi}_i^*(X) - \pi_i^*(X)|^2 \\ &= 4 \sum_{i=1}^{K} \mathbb{E} |\phi_i^*(\bar{\mathbf{F}}^*(X)) - \phi_i^*(\mathbf{F}^*(X))|^2 \\ &\le 4K \sum_{i=1}^{K} \mathbb{E} |\bar{F}_i^*(X) - F_i^*(X)|^2. \end{aligned} $$

We set $\xi(s) = k\Delta$ if $s \in [k\Delta, (k+1)\Delta[$. For all $i \in \mathcal{Y}$,

$$ \begin{aligned} F_i^*(X) - \bar{F}_i^*(X) = & \int_0^1 \left( \frac{b_i^*}{\sigma^{*2}}(X_{\xi(s)}) - \frac{b_i^*}{\sigma^{*2}}(X_s) \right) b_Y^*(X_s) ds \\ & - \frac{1}{2} \int_0^1 \left( \frac{b_i^{*2}}{\sigma^{*2}}(X_{\xi(s)}) - \frac{b_i^{*2}}{\sigma^{*2}}(X_s) \right) ds \\ & + \int_0^1 \left( \frac{b_i^*}{\sigma^{*2}}(X_{\xi(s)}) - \frac{b_i^*}{\sigma^{*2}}(X_s) \right) \sigma(X_s) dW_s. \end{aligned} $$
Then we have:

$$
\begin{aligned}
\mathbb{E} |F_i^*(X) - \bar{F}_i^*(X)|^2 &\le 3\mathbb{E} \left| \int_0^1 \left( \frac{b_i^*}{\sigma^{*2}}(X_{\xi(s)}) - \frac{b_i^*}{\sigma^{*2}}(X_s) \right) b_Y^*(X_s) ds \right|^2 \\
&\quad + \frac{3}{4} \mathbb{E} \left| \int_0^1 \left( \frac{b_i^{*2}}{\sigma^{*2}}(X_{\xi(s)}) - \frac{b_i^{*2}}{\sigma^{*2}}(X_s) \right) ds \right|^2 \\
&\quad + 3\mathbb{E} \left| \int_0^1 \left( \frac{b_i^*}{\sigma^{*2}}(X_{\xi(s)}) - \frac{b_i^*}{\sigma^{*2}}(X_s) \right) \sigma^*(X_s) dW_s \right|^2.
\end{aligned}
$$

According to the proof of Theorem 5.1 or Theorem 3.3 in [28], we obtain:

$$
\forall i \in \mathcal{Y}, \quad \mathbb{E} |F_i^*(X) - \bar{F}_i^*(X)|^2 \le C\Delta
$$

where $C > 0$ is a constant. Finally, there exists a constant $C > 0$ such that

$$
\mathfrak{R}(\bar{h}_*) - \mathfrak{R}(h^*) \leq C\Delta.
$$

Thus, the excess risk of the empirical classifier $\hat{h}$ satisfies:

$$
\mathbb{E} [\mathfrak{R}(\hat{h})] - \mathfrak{R}(h^*) \le \mathbb{E} [\mathfrak{R}(\hat{h})] - \mathfrak{R}(\bar{h}_*) + C\Delta \quad (B.7)
$$

where $C > 0$ is a constant.

**Upper bound of** $\mathbb{E}[\mathfrak{R}(\hat{h})] - \mathfrak{R}(\bar{h}_*)$

Let $\tilde{h}$ be the approximation of the discrete version $\bar{h}_*$ of the Bayes classifier $h^*$ given by

$$
\tilde{h}^i(X) = 2\tilde{\pi}_i(X) - 1, \quad i \in [1, K] \text{ and } X \in \mathcal{X}
$$

where $\tilde{\pi}_i$ depends on the functions $\tilde{b} = (\tilde{b}_1, \dots, \tilde{b}_K)$ and $\tilde{\sigma}^2$, the respective approximations of $b^* = (b_1^*, \dots, b_K^*)$ and $\sigma^{*2}$, with $\tilde{b}_i \in S_{K_N}$ and $\tilde{\sigma}^2 \in \tilde{S}_{K_N}$, such that $\|\tilde{b}_i\|_{\infty} \le \log^{3/2}(N)$, $\|\tilde{\sigma}^2\|_{\infty} \le \log^{3/2}(N)$, and for all $x \in (-\log(N), \log(N))$,

$$
|\tilde{b}_i(x) - b_i^*(x)| \le C \frac{\log(N)}{K_N}, \quad \text{and} \quad |\tilde{\sigma}^2(x) - \sigma^{*2}(x)| \le C \frac{\log(N)}{K_N}, \qquad (B.8)
$$

where $C > 0$ is a constant (see [28]). Moreover, define

$$
h^0 = (h^{0,1}, \dots, h^{0,K}) = \arg \min_{h \in \mathcal{H}} \mathcal{R}(h),
$$

where the space $\mathcal{H}$ is given by Equation (B.5). For all classifier $h \in \mathcal{H}$, set $\mathcal{D}_h := \mathfrak{R}(h) - \mathfrak{R}(h^0)$ and $\hat{\mathcal{D}}_h := \hat{\mathfrak{R}}(h) - \hat{\mathfrak{R}}(h^0)$. One obtains that:

$$
\begin{align*}
\mathfrak{R}(\hat{h}) - \mathfrak{R}(\bar{h}_*) &= \mathfrak{R}(\hat{h}) - \mathfrak{R}(h^0) + \mathfrak{R}(h^0) - \mathfrak{R}(\tilde{h}) + \mathfrak{R}(\tilde{h}) - \mathfrak{R}(\bar{h}_*) \\
&= \mathcal{D}_{\hat{h}} + \mathfrak{R}(h^0) - \mathfrak{R}(\tilde{h}) + \mathfrak{R}(\tilde{h}) - \mathfrak{R}(\bar{h}_*) \\
&\le \mathcal{D}_{\hat{h}} - 2\hat{\mathcal{D}}_{\hat{h}} + \mathfrak{R}(h^0) - \mathfrak{R}(\tilde{h}) + \mathfrak{R}(\tilde{h}) - \mathfrak{R}(\bar{h}_*),
\end{align*}
$$

where $\hat{\mathcal{D}}_{\hat{h}} \le 0$. We have $\mathfrak{R}(h^0) - \mathfrak{R}(\tilde{h}) \le 0$ by definition of $h^0$, then

$$
\mathfrak{R}(\hat{h}) - \mathfrak{R}(\bar{h}_*) \le \mathcal{D}_{\hat{h}} - 2\hat{\mathcal{D}}_{\hat{h}} + \mathfrak{R}(\tilde{h}) - \mathfrak{R}(\bar{h}_*) \quad (B.9)
$$

$$
\mathfrak{R}(\tilde{h}) - \mathfrak{R}(\bar{h}_*) = 4 \sum_{i=1}^{K} \mathbb{E} \left[ |\tilde{\pi}_i(X) - \bar{\pi}_i^*(X)|^2 \right] \le 4K \sum_{i=1}^{K} \mathbb{E} \left[ \left| \tilde{F}_i(X) - \bar{F}_i^*(X) \right|^2 \right].
$$
For each $i \in [1, K]$,

$$ \mathbb{E} \left[ \left| \widetilde{F}_i(X) - \bar{F}_i^*(X) \right|^2 \right] \le 3\mathbb{E} \left[ \left| \int_0^1 \left( \frac{\tilde{b}_i}{\tilde{\sigma}^2}(X_{\xi(s)}) - \frac{b_i^*}{\sigma^{*2}}(X_{\xi(s)}) \right) b_Y^*(X_s) ds \right|^2 \right] \\ + \frac{3}{4} \mathbb{E} \left[ \left| \int_0^1 \left( \frac{\tilde{b}_i^2}{\tilde{\sigma}^2}(X_{\xi(s)}) - \frac{b_i^{*2}}{\sigma^{*2}}(X_{\xi(s)}) \right) ds \right|^2 \right] \\ + 3\mathbb{E} \left[ \left| \int_0^1 \left( \frac{\tilde{b}_i}{\tilde{\sigma}^2}(X_{\xi(s)}) - \frac{b_i^*}{\sigma^{*2}}(X_{\xi(s)}) \right) \sigma^*(X_s) dW_s \right|^2 \right]. $$

Since for all $i \in [1, K]$ and for all $x \in \mathbb{R}$, we have

$$ \left\{ \begin{array}{l} \left| \frac{\tilde{b}_i}{\tilde{\sigma}^2}(x) - \frac{b_i^*}{\sigma^{*2}}(x) \right| \le \log^2(N) \left| \tilde{b}_i(x) - b_i^*(x) \right| + \frac{\log^2(N)}{\sigma_0^{*2}} |b_i^*(x)| |\tilde{\sigma}^2(x) - \sigma^{*2}(x)|, \\ \\ \left| \frac{\tilde{b}_i^2}{\tilde{\sigma}^2}(x) - \frac{b_i^{*2}}{\sigma^{*2}}(x) \right| \le \log^2(N) \left| \tilde{b}_i(x) + b_i^*(x) \right| \left| \tilde{b}_i(x) - b_i^*(x) \right| + \frac{\log^2(N)}{\sigma_0^{*2}} |b_i^*(x)|^2 \left| \tilde{\sigma}^2(x) - \sigma^{*2}(x) \right|. \end{array} \right. $$

Then, from Equation (B.8), there exists a constant $C > 0$ such that for all $x \in (-\log(N), \log(N))$,

$$ \left\{ \begin{array}{l} \left| \frac{\tilde{b}_i}{\tilde{\sigma}^2}(x) - \frac{b_i^*}{\sigma^{*2}}(x) \right| \le C(1 + b_i^*(x)) \frac{\log(N)}{K_N}, \\ \\ \left| \frac{\tilde{b}_i^2}{\tilde{\sigma}^2}(x) - \frac{b_i^{*2}}{\sigma^{*2}}(x) \right| \le C \left( \sqrt{L_N} + |b_i^*(x)| + |b_i^*(x)|^2 \right) \frac{\log(N)}{K_N}. \end{array} \right. $$

Then, under Assumption A.2.1, we have $\sup_{t \in [0,1]} \mathbb{E}[|b_Y^*(X_t)|^p] < \infty$ for all $p \in \mathbb{N}$, and there exists a constant $C > 0$ such that

$$ \mathbb{E} \left[ \left| \widetilde{F}_i(X) - \bar{F}_i^*(X) \right|^2 \mathbb{1}_{\left\{ \max_{k=1,...,n} |X_{k\Delta}| < \log(N) \right\}} \right] \le C \frac{\log^3(N)}{K_N^2}. \quad (B.10) $$

Moreover, under Assumption A.2.1, we have

$$ \mathbb{E} \left[ \left| \widetilde{F}_i(X) - \bar{F}_i^*(X) \right|^2 \mathbb{1}_{\left\{ \max_{k=1,...,n} |X_{k\Delta}| \ge \log(N) \right\}} \right] \le C \mathbb{P} \left( \max_{k=1,...,n} |X_{k\Delta}| \ge \log(N) \right) \\ \le C \sum_{k=1}^n \mathbb{P} (|X_{k\Delta}| \ge \log(N)) \\ \le C n \sup_{t \in [0,1]} \mathbb{P} (|X_t| \ge \log(N)). $$

From [28], Lemma 7.3, there exist constants $C, c > 0$ such that

$$ \mathbb{E} \left[ \left| \widetilde{F}_i(X) - \bar{F}_i^*(X) \right|^2 \mathbb{1}_{\left\{ \max_{k=1,...,n} |X_{k\Delta}| \ge \log(N) \right\}} \right] \le C \frac{n}{\log(N)} \exp(-c \log^2(N)). \quad (B.11) $$

Finally, we deduce from Equations (B.10) and (B.11) that

$$ \Re(\tilde{h}) - \Re(\bar{h}_*) \le C \left( \frac{\log^3(N)}{K_N^2} + \frac{n}{\log(N)} \exp(-c \log^2(N)) \right). \quad (B.12) $$

Then we obtain from Equation (B.9) and (B.12) that

$$ \mathbb{E} \left[ \Re(\hat{h}) - \Re(\bar{h}_*) \right] \le \mathbb{E} \left[ \mathcal{D}_{\hat{h}} - 2\hat{\mathcal{D}}_{\hat{h}} \right] + C \left( \frac{\log^3(N)}{K_N^2} + \frac{n}{\log(N)} \exp(-c \log^2(N)) \right). \quad (B.13) $$
We recall that the score function $\hat{h}$ depends on functions $\hat{b}_i$, $i \in \mathcal{Y}$ and $\hat{\sigma}^2$ of respective coordinate vectors $\hat{a}^{(i)}$, $i \in \mathcal{Y}$ and $\hat{\alpha}$. $\hat{\mathbf{a}} = (\hat{a}^{(1)}, \dots, \hat{a}^{(K)})$. The coordinate vectors $\hat{a}^{(1)}, \dots, \hat{a}^{(K)}$, $\hat{\alpha}$ belong to the closed ball

$$ \bar{B}' := \{ \theta \in \mathbb{R}^{K_N+M}, \|\theta\|_2 \le \sqrt{(K_N+M)L_N} \} = \bar{B}_{\|\cdot\|_2} (0, \sqrt{(K_N+M)L_N}), $$

and $\hat{\mathfrak{p}}$ belong to the unit ball $\bar{B} = \bar{B}_{\|\cdot\|_2}(0, 1)$. Let $\bar{B}'_\epsilon$ and $\bar{B}_\epsilon$ be the respective $\epsilon$-net of $\bar{B}'$ and $\bar{B}$ for all $\epsilon > 0$. Let $\hat{a}^{(1)}_\epsilon, \dots, \hat{a}^{(K)}_\epsilon$, $\hat{\alpha}_\epsilon \in \bar{B}'_\epsilon$ and $\hat{\mathfrak{p}}_\epsilon \in \bar{B}_\epsilon$ such that:

$$ \|\hat{a}^{(i)}_\epsilon - \hat{a}^{(i)}\|_2 \le \epsilon, \quad i \in [1, K], \quad \|\hat{\alpha}_\epsilon - \hat{\alpha}\|_2 \le \epsilon, \quad \|\hat{\mathfrak{p}}_\epsilon - \hat{\mathfrak{p}}\|_2 \le \epsilon. $$

Set $\hat{h}_\epsilon = \hat{h}_{\hat{a}_\epsilon, \hat{\alpha}_\epsilon, \hat{p}_\epsilon}$ with $\hat{\mathbf{a}}_\epsilon = (\hat{a}^1_\epsilon, \dots, \hat{a}^K_\epsilon)$. We have:

$$ \mathcal{D}_{\hat{h}} - 2\mathcal{D}_{\hat{h}} = \mathcal{D}_{\hat{h}} - \mathcal{D}_{\hat{h}_\epsilon} + 2(\mathcal{D}_{\hat{h}_\epsilon} - \mathcal{D}_{\hat{h}}) + \mathcal{D}_{\hat{h}_\epsilon} - 2\mathcal{D}_{\hat{h}_\epsilon}. $$

It results that,

$$ \mathbb{E} [\mathcal{D}_{\hat{h}} - 2\mathcal{D}_{\hat{h}}] \le \mathbb{E} (\mathcal{D}_{\hat{h}} - \mathcal{D}_{\hat{h}_\epsilon}) + 2\mathbb{E} (\mathcal{D}_{\hat{h}_\epsilon} - \mathcal{D}_{\hat{h}}) + \mathbb{E} (\mathcal{D}_{\hat{h}_\epsilon} - 2\mathcal{D}_{\hat{h}_\epsilon}). \quad (B.14) $$

1. Upper bound of $\mathbb{E}(\mathcal{D}_{\hat{h}} - \mathcal{D}_{\hat{h}_\epsilon})$

Set $\hat{b}_{i,\epsilon} = \sum_{\ell=-M}^{K_N-1} \hat{a}^i_\epsilon B_\ell$ for all label $i \in [1, K]$, $\hat{b}_\epsilon = (\hat{b}_{1,\epsilon}, \dots, \hat{b}_{K,\epsilon})$, and $\hat{\sigma}_\epsilon^2 = \sum_{\ell=-M}^{K_N-1} \hat{\alpha}_\epsilon B_\ell$. We have:

$$ \begin{aligned} \mathcal{D}_{\hat{h}} - \mathcal{D}_{\hat{h}_\epsilon} &= \mathfrak{R}(\hat{h}) - \mathfrak{R}(\hat{h}_\epsilon) = 4 \sum_{i=1}^{K} \mathbb{E} |\hat{\pi}_i(X) - \hat{\pi}_i^\epsilon(X)|^2 \le C \sum_{i=1}^{K} \mathbb{E} |\hat{F}_i(X) - \hat{F}_i^\epsilon(X)|^2 \\ &\le C \sum_{i=1}^{K} \|\hat{b}_i - \hat{b}_{i,\epsilon}\|_n^2 + \sum_{i=1}^{K} \|\hat{b}_i (\hat{\sigma}^2 - \hat{\sigma}_\epsilon^2)\|_n^2 \end{aligned} $$

according to [28], *Proof of Theorem 3.3*. For all function $g = \sum_{\ell=-M}^{K_N-1} a_\ell B_\ell$ belonging to any of the spaces $S_{K_N}$ and $\tilde{S}_{K_N}$, using the Cauchy Schwartz inequality, we have:

$$ \forall x \in \mathbb{R}, \quad g^2(x) = \left( \sum_{\ell=-M}^{K_N-1} a_\ell B_\ell(x) \right)^2 \le \|a\|_2^2 \le (K_N + M) \log^3(N). $$

It results that for all $g$ in $S_{K_N}$ or $\tilde{S}_{K_N}$, $\|g\|_\infty \le \|a\|_2 \le \sqrt{(K_N + M) \log^3(N)}$ and,

$$ \mathcal{D}_{\hat{h}} - \mathcal{D}_{\hat{h}_\epsilon} \le C \left( \sum_{i=1}^{K} \|\hat{b}_i - \hat{b}_{i,\epsilon}\|_n^2 + (K_N + M)L_N \sum_{i=1}^{K} \|\hat{\sigma}^2 - \hat{\sigma}_\epsilon^2\|_n^2 \right) $$

where the empirical norm $\|\cdot\|_n$ is given for all $g$ by

$$ \|g\|_n := \sqrt{\mathbb{E} \left[ \frac{1}{n} \sum_{k=0}^{n-1} g^2(X_{k\Delta}) \right]} \le \|g\|_\infty. $$

Finally, there exists a constant $C > 0$ such that

$$ \begin{aligned} \mathcal{D}_{\hat{h}} - \mathcal{D}_{\hat{h}_\epsilon} &\le C \left( \sum_{i=1}^{K} \|\hat{b}_i - \hat{b}_{i,\epsilon}\|_\infty^2 + K_N L_N \|\hat{\sigma}^2 - \hat{\sigma}_\epsilon^2\|_\infty^2 \right) \\ &\le C \left( \sum_{i=1}^{K} \|\hat{a}^i_\epsilon - \hat{a}^i\|_2^2 + K_N L_N \|\hat{\alpha}_\epsilon - \hat{\alpha}\|_2^2 \right) \\ &\le CK_N \log^3(N)\epsilon^2. \end{aligned} $$

So we obtain

$$ \mathcal{D}_{\hat{h}} - \mathcal{D}_{\hat{h}_\epsilon} \le CK_N \log^3(N)\epsilon^2. \quad (B.15) $$
## 2. Upper bound of $2\mathbb{E}(\hat{D}_{\hat{h}_{\varepsilon}} - \hat{D}_{\hat{h}})$

We have $\hat{D}_{\hat{h}_{\varepsilon}} - \hat{D}_{\hat{h}} = \hat{\mathfrak{R}}(\hat{h}_{\varepsilon}) - \hat{\mathfrak{R}}(\hat{h})$, and

$$ \hat{\mathfrak{R}}(\hat{h}_{\varepsilon}) - \hat{\mathfrak{R}}(\hat{h}) = \frac{1}{N} \sum_{j=1}^{N} \sum_{i=1}^{K} \{ \phi(Z_{i}^{j} \hat{h}_{\varepsilon}^{i}(\bar{X}^{j})) - \phi(Z_{i}^{j} \hat{h}^{i}(\bar{X}^{j})) \}. \quad (B.16) $$

For all $j \in [1, N]$ and $i \in \mathcal{Y}$, one has $Z_{i}^{j} \in \{-1, 1\}$ and,

$$ \begin{aligned} |\phi(Z_{i}^{j} \hat{h}_{\varepsilon}^{i}(\bar{X}^{j})) - \phi(Z_{i}^{j} \hat{h}^{i}(\bar{X}^{j}))| &= |\hat{h}^{i} - \hat{h}_{\varepsilon}^{i}| (\bar{X}^{j}) \times |2 - Z_{i}^{j}(\hat{h}^{i}(\bar{X}^{(j)}) + \hat{h}_{\varepsilon}^{i}(\bar{X}^{j}))| \\ &\le 4 |\hat{h}^{i} - \hat{h}_{\varepsilon}^{i}| (\bar{X}^{j}). \end{aligned} $$

Then, we obtain:

$$ \begin{aligned} |\hat{\mathfrak{R}}(\hat{h}_{\varepsilon}) - \hat{\mathfrak{R}}(\hat{h})| &\le \frac{1}{N} \sum_{j=1}^{N} \sum_{i=1}^{K} |\phi(Z_{i}^{j} \hat{h}_{\varepsilon}^{i}(\bar{X}^{j})) - \phi(Z_{i}^{j} \hat{h}^{i}(\bar{X}^{j}))| \\ &\le \frac{8}{N} \sum_{j=1}^{N} \sum_{i=1}^{K} |\hat{\pi}_{i}(\bar{X}^{j}) - \hat{\pi}_{i}^{\varepsilon}(\bar{X}^{j})| \end{aligned} $$

and,

$$ \hat{\mathfrak{R}}(\hat{h}_{\varepsilon}) - \hat{\mathfrak{R}}(\hat{h}) \le \frac{8K}{N} \sum_{j=1}^{N} \sqrt{\sum_{i=1}^{K} |\hat{F}_{i}(\bar{X}^{j}) - \hat{F}_{i}^{\varepsilon}(\bar{X}^{j})|^{2}} \quad (B.17) $$

For all $j \in [1, N]$ and $i \in [1, K]$, one has:

$$ \hat{F}_{i}(\bar{X}^{j}) := \sum_{k=0}^{n-1} \left( \frac{\hat{b}_{i}}{\hat{\sigma}^{2}}(X_{k\Delta}^{j})(X_{(k+1)\Delta}^{j} - X_{k\Delta}^{j}) - \frac{\Delta}{2} \frac{\hat{b}_{i}^{2}}{\hat{\sigma}^{2}}(X_{k\Delta}^{j}) \right) $$

and,

$$ \begin{aligned} |\hat{F}_{i}(\bar{X}^{j}) - \hat{F}_{i}^{\varepsilon}(\bar{X}^{j})|^{2} &\le 2 \left( \sum_{k=0}^{n-1} \left( \frac{\hat{b}_{i}}{\hat{\sigma}^{2}} - \frac{\hat{b}_{i,\varepsilon}}{\hat{\sigma}_{\varepsilon}^{2}} \right) (X_{k\Delta}^{j})(X_{(k+1)\Delta}^{j} - X_{k\Delta}^{j}) \right)^{2} \\ &\quad + \frac{\Delta^{2}}{2} \left( \sum_{k=0}^{n-1} \left( \frac{\hat{b}_{i}^{2}}{\hat{\sigma}^{2}} - \frac{\hat{b}_{i,\varepsilon}^{2}}{\hat{\sigma}_{\varepsilon}^{2}} \right) (X_{k\Delta}^{j}) \right)^{2} \\ &\le 2 \left( \sum_{k=0}^{n-1} \left| \frac{\hat{b}_{i}}{\hat{\sigma}^{2}} - \frac{\hat{b}_{i,\varepsilon}}{\hat{\sigma}_{\varepsilon}^{2}} \right| (X_{k\Delta}^{j}) \left| X_{(k+1)\Delta}^{j} - X_{k\Delta}^{j} \right| \right)^{2} \\ &\quad + \frac{\Delta}{2} \sum_{k=0}^{n-1} \left| \frac{\hat{b}_{i}^{2}}{\hat{\sigma}^{2}} - \frac{\hat{b}_{i,\varepsilon}^{2}}{\hat{\sigma}_{\varepsilon}^{2}} \right|^{2} (X_{k\Delta}^{j}). \end{aligned} $$

For all $i \in \mathcal{Y}$, we have:

$$ \begin{aligned} \left| \frac{\hat{b}_i}{\hat{\sigma}^2} - \frac{\hat{b}_{i,\varepsilon}}{\hat{\sigma}_{\varepsilon}^2} \right| &= \frac{\left| \hat{b}_i \hat{\sigma}_{\varepsilon}^2 - \hat{b}_{i,\varepsilon} \hat{\sigma}^2 \right|}{\hat{\sigma}^2 \hat{\sigma}_{\varepsilon}^2} \le \frac{\left| \hat{b}_i - \hat{b}_{i,\varepsilon} \right|}{\hat{\sigma}^2} + \frac{\left| \hat{b}_{i,\varepsilon} \right| \times \left| \hat{\sigma}^2 - \hat{\sigma}_{\varepsilon}^2 \right|}{\hat{\sigma}^2 \hat{\sigma}_{\varepsilon}^2} \\ &\le \log^4(N) \left| \hat{b}_i - \hat{b}_{i,\varepsilon} \right| + \log^8(N) \sqrt{(K_N + M) \log^3(N)} \left| \hat{\sigma}^2 - \hat{\sigma}_{\varepsilon}^2 \right| \\ &\le \log^4(N) \left\| \hat{b}_i - \hat{b}_{i,\varepsilon} \right\|_{\infty} + \log^8(N) \sqrt{(K_N + M) \log^3(N)} \left\| \hat{\sigma}^2 - \hat{\sigma}_{\varepsilon}^2 \right\|_{\infty} \\ &\le C \log^8(N) \sqrt{K_N \log^3(N)} \varepsilon \end{aligned} $$
where $C > 0$ is a constant, and

$$
\begin{aligned}
\left| \frac{\hat{b}_i^2}{\hat{\sigma}^2} - \frac{\hat{b}_{i,\varepsilon}^2}{\hat{\sigma}_{\varepsilon}^2} \right| &\le 2 \log^4(N) \sqrt{(K_N + M) \log^3(N)} |\hat{b}_i - \hat{b}_{i,\varepsilon}| + \log^8(N) (K_N + M) \log^3(N) |\hat{\sigma}^2 - \hat{\sigma}_{\varepsilon}^2| \\
&\le 2 \log^4(N) \sqrt{(K_N + M)L \log^3} \left\| \hat{b}_i - \hat{b}_{i,\varepsilon} \right\|_{\infty} + \log_N^8 (K_N + M) \log^3 \left\| \hat{\sigma}^2 - \hat{\sigma}_{\varepsilon}^2 \right\|_{\infty} \\
&\le C \log^{11}(N) K_N \varepsilon.
\end{aligned}
$$

It results that for all $i \in \mathcal{Y}$, we have:

$$
\begin{aligned}
\left| \hat{F}_i(\bar{X}^j) - \hat{F}_i^\varepsilon(\bar{X}^j) \right|^2 &\le \log^{19}(N)(K_N + M)\varepsilon^2 \left( \sum_{k=0}^{n-1} \left| X_{(k+1)\Delta}^j - X_{k\Delta}^j \right| \right)^2 \\
&\quad + \log^{22}(N)(K_N + M)^2\varepsilon^2.
\end{aligned}
$$

We deduce that for all $j \in [1, N]$,

$$
\sqrt{\sum_{i=1}^{K} \left| \hat{F}_i(\bar{X}^j) - \hat{F}_i^\varepsilon(\bar{X}^j) \right|^2} \le C \log^8(N) \sqrt{K_N \log^3(N)} \varepsilon \left( \sum_{k=0}^{n-1} \left| X_{(k+1)\Delta}^{(j)} - X_{k\Delta}^{(j)} \right| + \sqrt{K_N \log^3(N)} \right).
$$

and from E.1 and (B.17), we have:

$$
\hat{\mathfrak{R}}(\hat{h}_{\varepsilon}) - \hat{\mathfrak{R}}(\hat{h}) \le C \left( \log^8(N) \sqrt{K_N \log^3(N)} \frac{\varepsilon}{N} \sum_{j=1}^N \sum_{k=0}^{n-1} \left| X_{(k+1)\Delta}^j - X_{k\Delta}^j \right| + \log^8(N) \sqrt{K_N \log^3(N)} \varepsilon \right). \quad (B.18)
$$

For all $k \in [0, n-1]$,

$$
\begin{aligned}
\left| X_{(k+1)\Delta}^{1} - X_{k\Delta}^{1} \right| &\le \left| \int_{k\Delta}^{(k+1)\Delta} b_{Y}^{*}(X_{s})ds \right| + \left| \int_{k\Delta}^{(k+1)\Delta} \sigma^{*}(X_{s})dW_{s} \right| \\
&\le C\sqrt{K_{N}\log^{3}(N)\Delta} + \left| \int_{k\Delta}^{(k+1)\Delta} \sigma^{*}(X_{s})dW_{s} \right| \\
\sum_{k=0}^{n-1} \left| X_{(k+1)\Delta}^{(1)} - X_{k\Delta}^{(1)} \right| &\le C \left( \sqrt{K_{N}\log^{3}(N)} + \sum_{k=0}^{n-1} \left| \int_{k\Delta}^{(k+1)\Delta} \sigma^{*}(X_{s})dW_{s} \right| \right).
\end{aligned}
$$

For $N$ large enough, we obtain,

$$
\mathbb{E} \left[ \sum_{k=0}^{n-1} \left| X_{(k+1)\Delta}^{(1)} - X_{k\Delta}^{(1)} \right| \right] \le C \left( \sqrt{K_N \log^3(N)} + \sqrt{\sum_{k=0}^{n-1} \mathbb{E} \left( \int_{k\Delta}^{(k+1)\Delta} \sigma^{*2}(X_s) ds \right)} \right),
$$

then

$$
\mathbb{E} \left[ \sum_{k=0}^{n-1} \left| X_{(k+1)\Delta}^{(1)} - X_{k\Delta}^{(1)} \right| \right] \le C \sqrt{K_N \log^3(N)}. \quad (B.19)
$$

We deduce from Equations (B.19) and (B.18) that there exists a constant $C > 0$ such that

$$
2\mathbb{E}(\hat{D}_{h_{\varepsilon}} - \hat{D}_{\hat{h}}) = 2\mathbb{E}[\hat{\mathfrak{R}}(\hat{h}_{\varepsilon}) - \hat{\mathfrak{R}}(\hat{h})] \le C \log^{11}(N) K_N \varepsilon \quad (B.20)
$$

For any function $h \in \mathcal{H}$ such that $h^i = 2\pi - 1$, we denote by $h = h_{\mathbf{a},\alpha,\mathbf{p}}$ where $\mathbf{a} = (a^{(1)}, \dots, a^{(K)})$
and $a^{(i)}$, $i \in \mathcal{Y}$ and $\alpha$ are coordinate vectors of functions of the approximations spaces $S_{K_N}$ and $\tilde{S}_{K_N}$.
We conclude from Equations (B.14), (B.15) and (B.20) that

$$
\begin{aligned}
\mathbb{E} [\mathcal{D}_{\hat{h}} - 2\hat{\mathcal{D}}_{\hat{h}}] &\le CK_N \log^3(N)\varepsilon^2 + C \log^{11}(N)K_N\varepsilon + \mathbb{E} (\mathcal{D}_{\hat{h}_\varepsilon} - 2\hat{\mathcal{D}}_{\hat{h}_\varepsilon}) \\
&\le CK_N \log^3(N)\varepsilon^2 + C \log^{11}(N)K_N\varepsilon + \mathbb{E} \left\{ \sup_{(\mathbf{a},\alpha) \in \tilde{B}'^{K+1}_\varepsilon, \mathbf{p} \in \bar{B}_\varepsilon} \mathcal{D}_{h_{\mathbf{a},\alpha,\mathbf{p}}} - 2\hat{\mathcal{D}}_{h_{\mathbf{a},\alpha,\mathbf{p}}} \right\}.
\end{aligned}
$$
Thus, we have:

$$ \mathbb{E}[\mathcal{D}_{\hat{h}} - 2\hat{\mathcal{D}}_{\hat{h}}] \leq CK_N \log^3(N)\epsilon^2 + \log^{11}(N)K_N\epsilon + T_1 \quad (B.21) $$

where $T_1 = \mathbb{E}\left\{\sup_{(\mathbf{a},\alpha) \in \bar{B}'^{K+1}_{\epsilon}, \mathbf{p} \in \bar{B}_{\epsilon}} \mathcal{D}_{h_{\mathbf{a},\alpha,\mathbf{p}}} - 2\hat{\mathcal{D}}_{h_{\mathbf{a},\alpha,\mathbf{p}}}\right\}$. For all $u > 0$,

$$ \begin{aligned} T_1 &= \int_0^{+\infty} \mathbb{P} \left( \sup_{(\mathbf{a},\alpha) \in \bar{B}'^{K+1}_{\epsilon}, \mathbf{p} \in \bar{B}_{\epsilon}} \mathcal{D}_{h_{\mathbf{a},\alpha,\mathbf{p}}} - 2\hat{\mathcal{D}}_{h_{\mathbf{a},\alpha,\mathbf{p}}} \geq t \right) dt \\ &\leq u + \int_u^{+\infty} \mathbb{P} \left( \sup_{(\mathbf{a},\alpha) \in \bar{B}'^{K+1}_{\epsilon}, \mathbf{p} \in \bar{B}_{\epsilon}} \mathcal{D}_{h_{\mathbf{a},\alpha,\mathbf{p}}} - 2\hat{\mathcal{D}}_{h_{\mathbf{a},\alpha,\mathbf{p}}} \geq t \right) dt \\ &= u + \int_u^{+\infty} L_t dt \end{aligned} $$

where $L_t = \mathbb{P}\left(\sup_{(\mathbf{a},\alpha) \in \bar{B}'^{K+1}_{\epsilon}, \mathbf{p} \in \bar{B}_{\epsilon}} \mathcal{D}_{h_{\mathbf{a},\alpha,\mathbf{p}}} - 2\hat{\mathcal{D}}_{h_{\mathbf{a},\alpha,\mathbf{p}}} \geq t\right)$, and for all $t \geq u$,

$$ \begin{aligned} L_t &= \mathbb{P} \left( \bigcup_{(\mathbf{a},\alpha) \in \bar{B}'^{K+1}_{\epsilon}, \mathbf{p} \in \bar{B}_{\epsilon}} \left\{ \mathcal{D}_{h_{\mathbf{a},\alpha,\mathbf{p}}} - 2\hat{\mathcal{D}}_{h_{\mathbf{a},\alpha,\mathbf{p}}} \geq t \right\} \right) \\ &\leq \sum_{a^1 \in \bar{B}'_{\epsilon}} \dots \sum_{a^K \in \bar{B}'_{\epsilon}} \sum_{\alpha \in \bar{B}'_{\epsilon}} \sum_{\mathbf{p} \in \bar{B}_{\epsilon}} \mathbb{P} \left( \mathcal{D}_{h_{\mathbf{a},\alpha,\mathbf{p}}} - 2\hat{\mathcal{D}}_{h_{\mathbf{a},\alpha,\mathbf{p}}} \geq t \right) \end{aligned} $$

For all $(\mathbf{a}, \alpha) \in \bar{B}'^{K+1}_{\epsilon}$ and for all $\mathbf{p} \in \bar{B}_{\epsilon}$, one has:

$$ \begin{aligned} \hat{\mathcal{D}}_{h_{\mathbf{a},\alpha,\mathbf{p}}} &= \hat{\mathfrak{R}}(h_{\mathbf{a},\alpha,\mathbf{p}}) - \hat{\mathfrak{R}}(\bar{h}_*) = \frac{1}{N} \sum_{j=1}^{N} \sum_{i=1}^{K} \{(1 - Z_i^j h_{\mathbf{a},\alpha,\mathbf{p}}^i(\bar{X}^j))^2 - (1 - Z_i^j \bar{h}_*^i(\bar{X}^j))^2\} \\ &= \frac{1}{N} \sum_{j=1}^{N} S_j \end{aligned} $$

with, for each $j \in [1, N]$,

$$ S_j = \sum_{i=1}^{K} \{(1 - Z_i^j h_{\mathbf{a},\alpha,\mathbf{p}}^i(\bar{X}^j))^2 - (1 - Z_i^j \bar{h}_*^i(\bar{X}^j))^2\} = \ell_{h_{\mathbf{a},\alpha,\mathbf{p}}}(Z, X) - \ell_{\bar{h}_*}(Z, X) $$

where,

$$ \ell_{h_{\mathbf{a},\alpha,\mathbf{p}}}(Z, X) = \sum_{i=1}^{K} (1 - Z_i^j h_{\mathbf{a},\alpha,\mathbf{p}}^i(\bar{X}^j))^2. $$

We deduce for all $j \in [1, N]$ that

$$ \begin{aligned} \mathbb{E}(S_j^2) &\leq \mathbb{E} \left[ (\ell_{h_{\mathbf{a},\alpha,\mathbf{p}}}(Z, X) - \ell_{\bar{h}_*}(Z, X))^2 \right] \\ &\leq K \sum_{i=1}^{K} \mathbb{E} \left[ (h_{\mathbf{a},\alpha,\mathbf{p}}^i - \bar{h}_*^i)^2 (\bar{X}^j) (2 - Z_i^j (h_{\mathbf{a},\alpha,\mathbf{p}}^i - \bar{h}_*^i)(\bar{X}^j))^2 \right] \\ &\leq 16K \sum_{i=1}^{K} \mathbb{E} \left[ (h_{\mathbf{a},\alpha,\mathbf{p}}^i - \bar{h}_*^i)^2 (\bar{X}^j) \right] \end{aligned} $$

since $|Z_i^j| = 1$ and $|h_{\mathbf{a},\alpha,\mathbf{p}}^i|, |\bar{h}_*^i| \leq 1$. Then, for all $j \in [1, N]$,

$$ \mathbb{E}(S_j^2) \leq 16K \sum_{i=1}^{K} \mathbb{E} \left[ (h_{\mathbf{a},\alpha,\mathbf{p}}^i - \bar{h}_*^i)^2 (\bar{X}^j) \right] = 16K \left[ \mathfrak{R}(h_{\mathbf{a},\alpha,\mathbf{p}}) - \mathfrak{R}(\bar{h}_*) \right]. \quad (B.22) $$
We have

$$
\begin{aligned}
\mathfrak{R}(h_{\mathbf{a},\alpha,\mathbf{p}}) - \mathfrak{R}(\bar{h}_*) &= \mathfrak{R}(h_{\mathbf{a},\alpha,\mathbf{p}}) - \mathfrak{R}(h^0) + \mathfrak{R}(h^0) - \mathfrak{R}(\tilde{h}) + \mathfrak{R}(\tilde{h}) - \mathfrak{R}(\bar{h}_*) \\
&\leq \mathcal{D}_{h_{\mathbf{a},\alpha,\mathbf{p}}} + C \left( \frac{\log^3(N)}{K_N^2} + \frac{n}{\log(N)} \exp(-c \log^2(N)) \right).
\end{aligned}
$$

From Equation (B.22), there exists a constant $C_K > 0$ depending on $K$ such that

$$ \forall j \in [1, N], \quad \mathbb{E}(S_j^2) \le C_K(\mathcal{D}_{h_{\mathbf{a},\alpha,\mathbf{p}}} + t) = v, \quad \forall t \ge \frac{\log^3(N)}{K_N^2} + \frac{n}{\log(N)} \exp(-c \log^2(N)). $$

Furthermore, for all $k \ge 3$, $\sum_{j=1}^N \|S_j^k\| \le \frac{vk!c^{k-2}}{2}$ where $v = CK(\mathcal{D}_{h_{\mathbf{a},\alpha,\mathbf{p}}} + t)$ and $c = 64K^2$. Thus, we obtain from the Bernstein inequality that $\forall \mathbf{a}, \alpha \in \bar{B}'_\varepsilon, \forall \mathbf{p} \in \bar{B}_\varepsilon, \forall t \ge \sqrt{\Delta}$,

$$
\begin{aligned}
\mathbb{P} (\mathcal{D}_{h_{\mathbf{a},\alpha,\mathbf{p}}} - 2\hat{\mathcal{D}}_{h_{\mathbf{a},\alpha,\mathbf{p}}} \ge t) &= \mathbb{P} \left( \mathcal{D}_{h_{\mathbf{a},\alpha,\mathbf{p}}} - \hat{\mathcal{D}}_{h_{\mathbf{a},\alpha,\mathbf{p}}} \ge \frac{\mathcal{D}_{h_{\mathbf{a},\alpha,\mathbf{p}}} + t}{2} \right) \\
&\le \exp \left( - \frac{N (\mathcal{D}_{h_{\mathbf{a},\alpha,\mathbf{p}}} + t)^2}{8CK (\mathcal{D}_{h_{\mathbf{a},\alpha,\mathbf{p}}} + t) + c (\mathcal{D}_{h_{\mathbf{a},\alpha,\mathbf{p}}} + t)} \right) \\
&= \exp \left( - \frac{N (\mathcal{D}_{h_{\mathbf{a},\alpha,\mathbf{p}}} + t)}{8(CK + c)} \right).
\end{aligned}
$$

Finally, since $\exp(-\frac{N\mathcal{D}_{h_{\mathbf{a},\alpha,\mathbf{p}}}}{8(CK+c)}) \le 1$, we obtain that $\forall \mathbf{a}, \alpha \in \bar{B}'_\varepsilon, \forall \mathbf{p} \in \bar{B}_\varepsilon$,

$$ \forall t \ge \frac{\log^3(N)}{K_N^2} + \frac{n}{\log(N)} \exp(-c \log^2(N)), \quad \mathbb{P}(\mathcal{D}_{h_{\mathbf{a},\alpha,\mathbf{p}}} - 2\hat{\mathcal{D}}_{h_{\mathbf{a},\alpha,\mathbf{p}}} \ge t) \le \exp\left(-\frac{Nt}{8(CK+c)}\right). $$

It results that for all $t \ge \frac{\log^3(N)}{K_N^2} + \frac{n}{\log(N)} \exp(-c \log^2(N))$,

$$ L_t \le \mathcal{N}_2(\varepsilon, \bar{B}')^{K+1} \mathcal{N}_2(\varepsilon, \bar{B}) \exp \left( -\frac{Nt}{8(CK+c)} \right) $$

and for all $u \ge \frac{\log^3(N)}{K_N^2} + \frac{n}{\log(N)} \exp(-c \log^2(N))$,

$$
\begin{aligned}
T_1 &\le u + \int_u^{+\infty} L_t dt = u + \mathcal{N}_2(\varepsilon, \bar{B}')^{K+1} \mathcal{N}_2(\varepsilon, \bar{B}) \int_u^{+\infty} \exp\left(-\frac{Nt}{8(CK+c)}\right) dt \\
&\le u + \frac{8(CK+c)\mathcal{N}_2(\varepsilon, \bar{B}')^{K+1} \mathcal{N}_2(\varepsilon, \bar{B})}{N} \exp\left(-\frac{Nu}{8(CK+c)}\right)
\end{aligned}
$$

where $\mathcal{N}_2(\varepsilon, \bar{B}')$ and $\mathcal{N}_2(\varepsilon, \bar{B})$ are respective covering numbers of $\bar{B}'$ and $\bar{B}$.

We set

$$ u = \frac{\log^3(N)}{K_N^2} + \frac{n}{\log(N)} \exp(-c \log^2(N)) + \frac{88K \left( (K+1) \log(\mathcal{N}_2(\varepsilon, \bar{B}')) + \log(\mathcal{N}_2(\varepsilon, \bar{B})) \right)}{N} $$

and we deduce that

$$ T_1 \le C \left( \frac{\log^3(N)}{K_N^2} + \frac{n}{\log(N)} \exp(-c \log^2(N)) + \frac{\log(\mathcal{N}_2(\varepsilon, \bar{B}')) + \log(\mathcal{N}_2(\varepsilon, \bar{B}))}{N} \right). $$

For every real number $R > 0$, the euclidean ball of radius $R$ denoted by $\bar{B}_2(0, R) \subset \mathbb{R}^d$ has a covering number $\mathcal{N}_2(\varepsilon, \bar{B}_2(0, R))$ that satisfies:

$$ \mathcal{N}_2(\varepsilon, \bar{B}_2(0, R)) \le \left( \frac{3R}{\varepsilon} \right)^d $$
(see [74], Chapter 15 Prop 1.3). We deduce that

$$ \mathcal{N}_2(\varepsilon, \bar{B}') \le \left( \frac{3\sqrt{(K_N + M) \log^3(N)}}{\varepsilon} \right)^{K_N + M}, \quad \mathcal{N}_2(\varepsilon, \bar{B}) \le \left( \frac{3}{\varepsilon} \right)^K $$

We set $\varepsilon = 1/N\sqrt{(K_N + M) \log^3(N)}$ and we obtain,

$$ T_1 \le C \left( \frac{\log^3(N)}{K_N^2} + \frac{n}{\log(N)} \exp(-c \log^2(N)) + \frac{K_N \log(N)}{N} \right) $$

and it follows from (B.21) that,

$$ \mathbb{E}[\mathcal{D}_{\hat{h}} - 2\mathcal{D}_{\hat{h}}] \le C \left( \frac{\log^3(N)}{K_N^2} + \frac{n}{\log(N)} \exp(-c \log^2(N)) + \frac{K_N \log(N)}{N} \right). \quad (B.23) $$

Finally, from Equations (B.23), (B.13) and (B.7), there exists a constant $C > 0$ such that

$$ \mathbb{E}[\mathfrak{R}(\hat{h})] - \mathfrak{R}(h^*) \le C \left( \frac{\log^3(N)}{K_N^2} + \frac{K_N \log(N)}{N} + \frac{n}{\log(N)} \exp(-c \log^2(N)) \right), $$

and the excess risk of the empirical classifier $\hat{g} = g_{\hat{h}}$ satisfies:

$$ \mathbb{E}[\mathcal{R}(\hat{g})] - \mathcal{R}(g^*) \le C \left( \frac{\log^{3/2}(N)}{K_N} + \left( \frac{K_N \log(N)}{N} \right)^{1/2} + \Delta^{1/2} + \sqrt{\frac{n}{\log(N)}} \exp\left(-\frac{c}{2}\log^2(N)\right) \right). $$

## B.8.2 Proof of Proposition B.5.2

*Proof.* Define for all function $h \in L^2(\mathbb{R})$ and from the diffusion process $X = (X_t)_{t \in [0,1]}$ the following pseudo-norm:

$$ \|h\|_X^2 := \int_0^1 h^2(X_s)ds. $$

Then, from Assumption B.5.1, we obtain that

$$ \Delta_b \le \|b_1^* - b_0^*\|_X \le \bar{\Delta}_b $$

where $\bar{\Delta}_b$ and $\Delta_b$ are positive real number such that $\bar{\Delta}_b > \Delta_b$. Let $m \ge 1$ be an integer. Consider the subdivision

$$ \Gamma_m := \left\{ t_k = \Delta_b + k \frac{\bar{\Delta}_b - \Delta_b}{m}, \ k = 0, \dots, m \right\} $$

of the compact interval $[\Delta_b, \bar{\Delta}_b]$. Set $f^* = b_1^* - b_0^*$, then $\Delta_b^* = \mathbb{E}[\int_0^1 f^{*2}(X_s)ds]$. Then, from the total probability formula, we have

$$ \mathbb{P}\left(0 < \left|\eta(X) - \frac{1}{2}\right| \le \varepsilon\right) = \sum_{k=0}^{m-1} \mathbb{P}\left(0 < \left|\eta(X) - \frac{1}{2}\right| \le \varepsilon \mid \|f^*\|_X \in [t_k, t_{k+1}]\right) \mathbb{P}(\|f^*\|_X \in [t_k, t_{k+1}]) $$

For all $k \in [0, m-1]$,

$$ \begin{aligned} \mathbb{P}\left(0 < \left|\eta(X) - \frac{1}{2}\right| \le \varepsilon \mid \|f^*\|_X \in [t_k, t_{k+1}]\right) &= \mathbb{P}\left(0 < \frac{|q_1(X) - q_0(X)|}{2(q_0(X) + q_1(X))} \le \varepsilon \mid \|f^*\|_X \in [t_k, t_{k+1}]\right) \\ &= T_1^k + T_2^k \end{aligned} $$

where,

$$ T_1^k = \mathbb{P}\left(\{q_1(X) < q_0(X)\} \cap \left\{\frac{|q_1(X) - q_0(X)|}{2(q_0(X) + q_1(X))} \le \varepsilon\right\} \mid \|f^*\|_X \in [t_k, t_{k+1}]\right) $$

$$ T_2^k = \mathbb{P}\left(\{q_1(X) > q_0(X)\} \cap \left\{\frac{|q_1(X) - q_0(X)|}{2(q_0(X) + q_1(X))} \le \varepsilon\right\} \mid \|f^*\|_X \in [t_k, t_{k+1}]\right). $$
**Upper-bound of $T_1^k$.** The first term $T_1$ satisfies

$$ T_1^k \le \mathbb{P} \left( \{q_1(X) < q_0(X)\} \cap \left\{ \left| \frac{q_1(X)}{q_0(X)} - 1 \right| \le 4\epsilon \right\} \mid \|f^*\|_X \in [t_k, t_{k+1}] \right) \\ \le \frac{1}{2} T_{1,1}^k + \frac{1}{2} T_{1,2}^k $$

where,

$$ T_{1,1}^k = \mathbb{P} \left( \{q_1(X) < q_0(X)\} \cap \left| \frac{q_1(X)}{q_0(X)} - 1 \right| \le 4\epsilon \mid \|b_1^* - b_0^*\|_X \in [t_k, t_{k+1}], Y = 0 \right), $$

$$ T_{1,2}^k = \mathbb{P} \left( \{q_1(X) < q_0(X)\} \cap \left| \frac{q_1(X)}{q_0(X)} - 1 \right| \le 4\epsilon \mid \|b_1^* - b_0^*\|_X \in [t_k, t_{k+1}], Y = 1 \right). $$

On the event $\{Y = 0\}$, we have

$$ \begin{aligned} \frac{q_1}{q_0}(X) &= \exp \left( \int_0^1 (b_1^* - b_0^*)(X_s)b_0^*(X_s)ds - \frac{1}{2} \int_0^1 (b_1^* - b_0^*)(b_0^* + b_1^*)(X_s)ds + \int_0^1 (b_1^* - b_0^*)(X_s)dW_s \right) \\ &= \exp \left( -\frac{1}{2} \int_0^1 f^{*2}(X_s)ds + \int_0^1 f^*(X_s)dW_s \right) \\ &= \exp \left( -\frac{1}{2} \|f^*\|_X^2 + \int_0^1 f^*(X_s)dW_s \right). \end{aligned} $$

On the event $\{Y = 1\}$, we have

$$ \begin{aligned} \frac{q_1}{q_0}(X) &= \exp \left( \int_0^1 (b_1^* - b_0^*)(X_s)b_1^*(X_s)ds - \frac{1}{2} \int_0^1 (b_1^* - b_0^*)(b_0^* + b_1^*)(X_s)ds + \int_0^1 (b_1^* - b_0^*)(X_s)dW_s \right) \\ &= \exp \left( \int_0^1 \frac{f^{*2}(X_s)}{2}ds + \int_0^1 f^*(X_s)dW_s \right) \\ &= \exp \left( \frac{1}{2} \|f^*\|_X^2 + \int_0^1 f^*(X_s)dW_s \right). \end{aligned} $$

We deduce for all $\epsilon < 1/8$ and for all $k \in [0, m-1]$ that

$$ \begin{aligned} T_{1,1}^k &= \mathbb{P} \left( \left| \exp \left( -\frac{1}{2} \|f^*\|_X^2 + \int_0^1 f^*(X_s) dW_s \right) - 1 \right| \le 4\epsilon \mid \|f^*\|_X \in [t_k, t_{k+1}], Y = 0 \right) \\ &\le \mathbb{P} \left( \log(1 - 4\epsilon) \le \int_0^1 f^*(X_s) dW_s - \frac{\|f^*\|_X^2}{2} \le \log(1 + 4\epsilon) \mid \|f^*\|_X \in [t_k, t_{k+1}], Y = 0 \right) \\ &\le \mathbb{P} \left( -\frac{8\epsilon}{\Delta_{b^*}} + \frac{\|f^*\|_X^2}{2\Delta_{b^*}} \le \int_0^1 \frac{f^*(X_s)}{\Delta_{b^*}} dW_s \le \frac{4\epsilon}{\Delta_{b^*}} + \frac{\|f^*\|_X^2}{2\Delta_{b^*}} \mid \|f^*\|_X \in [t_k, t_{k+1}], Y = 0 \right) \\ &\le \mathbb{P} \left( -\frac{8\epsilon}{\Delta_{b^*}} + \frac{t_k^2}{2\Delta_{b^*}} \le \int_0^1 \frac{f^*(X_s)}{\Delta_{b^*}} dW_s \le \frac{4\epsilon}{\Delta_{b^*}} + \frac{t_{k+1}^2}{2\Delta_{b^*}} \mid \|f^*\|_X \in [t_k, t_{k+1}], Y = 0 \right) \end{aligned} $$

and,

$$ \begin{aligned} T_{1,2}^k &= \mathbb{P} \left( \left| \exp \left( \frac{1}{2} \|f^*\|_X^2 + \int_0^1 f^*(X_s) dW_s \right) - 1 \right| \le 4\epsilon \mid \|f^*\|_X \in [t_k, t_{k+1}], Y = 1 \right) \\ &\le \mathbb{P} \left( \log(1 - 4\epsilon) \le \int_0^1 f^*(X_s) dW_s + \frac{\|f^*\|_X^2}{2} \le \log(1 + 4\epsilon) \mid \|f^*\|_X \in [t_k, t_{k+1}], Y = 0 \right) \\ &\le \mathbb{P} \left( -\frac{8\epsilon}{\Delta_{b^*}} - \frac{\|f^*\|_X^2}{2\Delta_{b^*}} \le \int_0^1 \frac{f^*(X_s)}{\Delta_{b^*}} dW_s \le \frac{4\epsilon}{\Delta_{b^*}} - \frac{\|f^*\|_X^2}{2\Delta_{b^*}} \mid \|f^*\|_X \in [t_k, t_{k+1}], Y = 0 \right) \\ &\le \mathbb{P} \left( -\frac{8\epsilon}{\Delta_{b^*}} - \frac{t_{k+1}^2}{2\Delta_{b^*}} \le \int_0^1 \frac{f^*(X_s)}{\Delta_{b^*}} dW_s \le \frac{4\epsilon}{\Delta_{b^*}} - \frac{t_k^2}{2\Delta_{b^*}} \mid \|f^*\|_X \in [t_k, t_{k+1}], Y = 0 \right). \end{aligned} $$
Under Assumption B.5.1, the random variable $Z = (1/\Delta_{b^*}) \int_0^1 f^*(X_s)dW_s$ admits a density function $p_Z$ that is bounded on the real line $\mathbb{R}$. Thus, there exists a constant $C_Z > 0$ such that

$$ \forall x \in \mathbb{R}, \quad p_Z(x) \le C_Z. $$

We finally obtain that for all $k \in [0, m-1]$,

$$ T_{1,1}^k \le \int_{-\frac{8\epsilon}{\Delta_{b^*}} + \frac{t_k^2}{2\Delta_{b^*}}}^{\frac{4\epsilon}{\Delta_{b^*}} + \frac{t_{k+1}^2}{2\Delta_{b^*}}} p_Z(x)dx \le C \left( \frac{12}{\Delta_{b^*}}\epsilon + \frac{1}{2}(t_{k+1}^2 - t_k^2) \right) \le C \left( \frac{12}{\Delta_{b^*}}\epsilon + \frac{\bar{\Delta}_b - \Delta_b}{2m} \right) $$

$$ T_{1,2}^k \le \int_{-\frac{8\epsilon}{\Delta_{b^*}} - \frac{t_{k+1}^2}{2\Delta_{b^*}}}^{\frac{4\epsilon}{\Delta_{b^*}} - \frac{t_k^2}{2\Delta_{b^*}}} p_Z(x)dx \le C \left( \frac{12}{\Delta_{b^*}}\epsilon + \frac{1}{2}(t_{k+1}^2 - t_k^2) \right) \le C \left( \frac{12}{\Delta_{b^*}}\epsilon + \frac{\bar{\Delta}_b - \Delta_b}{2m} \right) $$

where $C > 0$ is a new constant. Then, for all $k \in [0, m-1]$,

$$ T_1^k \le C \left( \frac{12}{\Delta_{b^*}}\epsilon + \frac{\bar{\Delta}_b - \Delta_b}{2m} \right). $$

Using similar reasoning, we obtain that for all $k \in [0, m-1]$,

$$ T_2^k \le C \left( \frac{12}{\Delta_{b^*}}\epsilon + \frac{\bar{\Delta}_b - \Delta_b}{2m} \right). $$

Thus, we conclude that for all $m \ge 1$,

$$ \begin{aligned} \mathbb{P} \left( 0 < \left| \eta(X) - \frac{1}{2} \right| \le \epsilon \right) &= \sum_{k=0}^{m-1} \mathbb{P} \left( 0 < \left| \eta(X) - \frac{1}{2} \right| \le \epsilon \mid \|f^*\|_X \in [t_k, t_{k+1}] \right) \mathbb{P} \left( \|f^*\|_X \in [t_k, t_{k+1}] \right) \\ &\le \sum_{k=0}^{m-1} (T_1^k + T_1^k) \mathbb{P} \left( \|f^*\|_X \in [t_k, t_{k+1}] \right) \\ &\le C \frac{12}{\Delta_{b^*}}\epsilon + \frac{\bar{\Delta}_b - \Delta_b}{2m}. \end{aligned} $$

Finally, we tend $m$ toward infinity and obtain

$$ \mathbb{P} \left( 0 < \left| \eta(X) - \frac{1}{2} \right| \le \epsilon \right) \le C \frac{12}{\Delta_{b^*}}\epsilon. $$

□
Bibliography

[1] J. Albert-Smet, A. Torrente, and J. Romo. Band depth based initialization of *k*-means for functional data clustering. *Adv. Data Anal. Classif.*, 17(2):463–484, 2023. 2

[2] B. Andrew, L. Birgé, and P. Massart. Risk bounds for model selection via penalization. *Probability theory and related fields*, 113(3):301–413, 1999. 41

[3] S. Arora, L. Babai, J. Stern, and Z. Sweedyk. The hardness of approximate optima in lattices, codes, and systems of linear equations. *Journal of Computer and System Sciences*, 54(2):317–331, 1997. 136

[4] J.-Y. Audibert, A.-B. Tsybakov, et al. Fast learning rates for plug-in classifiers. *The Annals of statistics*, 35(2):608–633, 2007. 27, 89, 99, 104

[5] A. Baíllo, A. Cuevas, and R. Fraiman. Classification methods for functional data. *The Oxford handbook of functional data analysis*, 2011. 88

[6] F.-M. Bandi and P.-C.-B. Phillips. Fully nonparametric estimation of scalar diffusion models. *Econometrica*, 71(1):241–283, 2003. 37

[7] Y. Baraud, F. Comte, and G. Viennet. Model selection for (auto-) regression with dependent data. *ESAIM: Probability and Statistics*, 5:33–49, 2001. 51, 77

[8] P.-L Bartlett, M.-I Jordan, and J.-D McAuliffe. Convexity, classification, and risk bounds. *Journal of the American Statistical Association*, 101(473):138–156, 2006. 134, 136, 140

[9] B. Biais, T. Björk, J. Cvitanić, N. El Karoui, E. Jouini, J.-C. Rochet, and M.-C. Quenez. Non-linear pricing theory and backward stochastic differential equations. *Financial Mathematics: Lectures given at the 3rd Session of the Centro Internazionale Matematico Estivo (CIME) held in Bressanone, Italy, July 8–13, 1996*, pages 191–246, 1997. 2

[10] G. Biau, F. Bunea, and M.-H. Wegkamp. Functional classification in hilbert spaces. *IEEE Transactions on Information Theory*, 51(6):2163–2172, 2005. 17

[11] B. Cadre. Supervised classification of diffusion paths. *Mathematical Methods of Statistics*, 22(3):213–225, 2013. 19, 20, 89, 134, 136, 142

[12] M.-P. Chandra et al. On the generalised distance in statistics. In *Proceedings of the National Institute of Sciences of India*, volume 2, pages 49–55, 1936. 17

[13] J. Chang. Functional data classification using the rotation forest method combined with the patch selection. *Comm. Statist. Simulation Comput.*, 52(7):3365–3378, 2023. 2

[14] G. Ciolek, D. Marushkevych, and M. Podolskij. On lasso estimator for the drift function in diffusion models. *arXiv preprint arXiv:2209.05974*, 2022. 30
[15] E. Clement. Estimation of diffusion processes by simulated moment methods. *Scandinavian Journal of Statistics*, 24(3):353–369, 1997. 1, 4, 37

[16] A. Cohen, M. Davenport, and D. Leviatan. On the stability and accuracy of least squares approximations. *Foundations of computational mathematics*, 13(5):819–834, 2013. 129

[17] F. Comte. *Nonparametric Estimation*. Spartacus IDH, 2017. 37

[18] F. Comte and V. Genon-Catalot. Nonparametric drift estimation for iid paths of stochastic differential equations. *The Annals of Statistics*, 48(6):3336–3365, 2020. 37, 50, 71, 89, 90, 126

[19] F. Comte and V. Genon-Catalot. Regression function estimation as a partly inverse problem. *Annals of the Institute of Statistical Mathematics*, 72(4):1023–1054, 2020. 41, 90, 129

[20] F. Comte and V. Genon-Catalot. Drift estimation on non compact support for diffusion models. *Stochastic Processes and their Applications*, 134:174–207, 2021. 2, 4, 5, 8, 25, 42, 49, 89, 90

[21] F. Comte, V. Genon-Catalot, et al. Regression function estimation as a partly inverse problem. *Preprint Hal*, 2018. 86

[22] F. Comte, V. Genon-Catalot, Y. Rozenholc, et al. Penalized nonparametric mean square estimation of the coefficients of diffusion processes. *Bernoulli*, 2007. 4, 9, 10, 37, 89

[23] A. Cuevas, M. Febrero, and R. Fraiman. Robust estimation and classification for functional data via projection-based depth notions. *Computational Statistics*, 22(3):481–496, 2007. 17

[24] S. Dabo-Niang, F. Ferraty, F. Rossi, and N. Villa. Recent advances in the use of svm for functional data classification. *Functional and operatorial statistics*, pages 273–280, 2008. 88

[25] C. De Boor. *A practical guide to splines*, volume 27. springer-verlag New York, 1978. 89

[26] P.-L. De Micheaux, Pavlo Mozharovskyi, and Myriam Vimond. Depth for curve data and applications. *Journal of the American Statistical Association*, 116(536):1881–1897, 2021. 88

[27] L. Della-Maestra and M. Hoffmann. Nonparametric estimation for interacting particle systems: Mckean-vlasov models. *Probability Theory and Related Fields*, 182(1):551–613, 2022. 89

[28] C. Denis, C. Dion-Blanc, E. Ella-Mintsa, and V.-C. Tran. Nonparametric plug-in classifier for multiclass classification of sde paths. *arXiv preprint arXiv:2212.10259*, 2022. 25, 37, 38, 49, 50, 58, 59, 86, 143, 144, 145

[29] C. Denis, C. Dion-Blanc, and M. Martinez. Consistent procedures for multiclass classification of discrete diffusion paths. *Scandinavian Journal of Statistics*, 47(2):516–554, 2020. 4, 12, 15, 20, 36, 89, 91, 92, 104, 134, 136, 139, 142

[30] C. Denis, C. Dion-Blanc, and M. Martinez. A ridge estimator of the drift from discrete repeated observations of the solutions of a stochastic differential equation. *Bernoulli*, 2021. 2, 4, 7, 8, 23, 25, 29, 37, 40, 42, 44, 48, 50, 61, 66, 67, 72, 83, 84, 89, 93, 95, 96, 98, 101, 110, 111, 113, 119, 121, 127, 129

[31] R.-A. DeVore and G.-G. Lorentz. *Constructive approximation*, volume 303. Springer Science & Business Media, 1993. 41

[32] L. Devroye, L. Györfi, and G. Lugosi. *A probabilistic theory of pattern recognition*, volume 31. Springer Science & Business Media, 2013. 12, 93

[33] D. Domingo, A. d'Onofrio, and F. Flandoli. Properties of bounded stochastic processes employed in biophysics. *Stochastic Analysis and Applications*, 38(2):277–306, 2020. 36, 88

[34] Darrell Duffie and Rui Kan. A yield-factor model of interest rates. *Mathematical finance*, 6(4):379–406, 1996. 36
[35] N. El Karoui, S. Peng, and M. C. Quenez. Backward stochastic differential equations in finance. *Mathematical finance*, 7(1):1–71, 1997. 36, 88

[36] E. Ella-Mintsa. Nonparametric estimation of the diffusion coefficient from sde paths. *arXiv preprint arXiv:2307.03960*, 2023. 29

[37] R. Erban and S. J. Chapman. Stochastic modelling of reaction–diffusion processes: algorithms for bimolecular reactions. *Physical biology*, 6(4):046001, 2009. 36, 88

[38] Alison Etheridge. *Some Mathematical Models from Population Genetics: École D'Été de Probabilités de Saint-Flour XXXIX-2009*, volume 2012. Springer Science & Business Media, 2011. 36

[39] S. N. Ethier and M. F. Norman. Error estimate for the diffusion approximation of the wright–fisher model. *Proceedings of the National Academy of Sciences*, 74(11):5096–5098, 1977. 36

[40] A. Fermanian. *Learning time-dependent data with the signature transform*. PhD thesis, Sorbonne université, 2021. 17

[41] F. Ferraty and P. Vieu. Curves discrimination: a nonparametric functional approach. *Computational Statistics & Data Analysis*, 44(1-2):161–173, 2003. 17

[42] F. Ferraty and P. Vieu. Reference manual for implementing nonparametric functional data analysis (npfda). *Companion Manual of the Book: Nonparametric Functional Data Analysis, Theory and Practice*, Springer-Verlag, New York, 2006. 1

[43] L. Ferré and N. Villa. Multilayer perceptron with functional inputs: an inverse regression approach. *Scandinavian Journal of Statistics*, 33(4):807–823, 2006. 17

[44] D. Florens. Estimation of the diffusion coefficient from crossings. *Statistical Inference for Stochastic Processes*, 1:175–195, 1998. 4, 37

[45] D. Florens-Zmirou. On estimating the diffusion coefficient from discrete observations. *Journal of applied probability*, 30(4):790–804, 1993. 2, 4, 37, 38, 52, 56

[46] A. Friedman. *Stochastic Differential Equations and Applications*, volume 1. Academic Press, 1975. 135

[47] S. Gadat, S. Gerchinovitz, and C. Marteau. Optimal functional supervised classification with separation condition. *Bernoulli*, 26(3):1797–1831, 2020. 18, 21, 36, 89, 99, 104, 139

[48] S. Gadat, T. Klein, and C. Marteau. Classification in general finite dimensional spaces with the k-nearest neighbor rule. *The Annals of Statistics*, 44(3):982–1009, 2016. 89, 90, 99, 104

[49] V. Genon-Catalot and J. Jacod. On the estimation of the diffusion coefficient for multi-dimensional diffusion processes. In *Annales de l'IHP Probabilités et statistiques*, volume 29, pages 119–151, 1993. 1, 4, 37

[50] V. Genon-Catalot and J. Jacod. Estimation of the diffusion coefficient for diffusion processes: random sampling. *Scandinavian Journal of Statistics*, pages 193–221, 1994. 37

[51] V. Genon-Catalot, T. Jeantheau, and C. Laredo. Parameter estimation for discretely observed stochastic volatility models. *Bernoulli*, pages 855–872, 1999. 37

[52] V. Genon-Catalot, C. Laredo, and D. Picard. Non-parametric estimation of the diffusion coefficient by wavelets methods. *Scandinavian Journal of Statistics*, pages 317–335, 1992. 37

[53] A. K Ghosh and P. Chaudhuri. On maximum depth and related classifiers. *Scandinavian Journal of Statistics*, 32(2):327–350, 2005. 17

[54] A. Gloter. Discrete sampling of an integrated diffusion process and parameter estimation of the diffusion coefficient. *ESAIM: Probability and Statistics*, 4:205–227, 2000. 1, 4, 37
[55] E. Gobet. Lan property for ergodic diffusions with discrete observations. *Annales de l'Institut Henri Poincare (B) Probability and Statistics*, 38:711-737, 2002. **4, 37, 38, 58, 90, 95, 99, 105, 106**, 107, 125, 135

[56] C. Gourieroux, H.-T. Nguyen, and S. Sriboonchitta. Nonparametric estimation of a scalar diffusion model from discrete time data: a survey. *Annals of Operations Research*, 256:203-219, 2017. 37

[57] L. Györfi, M. Kohler, A. Krzyzak, and H. Walk. *A distribution-free theory of nonparametric regression*. Springer Science & Business Media, 2006. **21, 24, 40, 93, 112, 121, 124**

[58] T. Hastie, R. Tibshirani, and J. Friedman. Springer series in statistics the elements of statistical learning data mining, inference. Technical report, and Prediction. Technical report, 2001. 43

[59] M. Hoffmann. Minimax estimation of the diffusion coefficient through irregular samplings. *Statistics & probability letters*, 32:11–24, 1997. **4, 37**

[60] M. Hoffmann. Adaptive estimation in diffusion processes. *Stochastic processes and their Applications*, 79(1):135–163, 1999. **37, 89**

[61] M. Hoffmann. Lp estimation of the diffusion coefficient. *Bernoulli*, pages 447–481, 1999. **4, 9**, 10, 22, 37, 44, 45, 46, 48, 58, 89

[62] S.-M. Iacus. *Simulation and inference for stochastic differential equations: with R examples*. Springer Science & Business Media, 2009. **53, 100**

[63] J. Jacod. Random sampling in estimation problems for continuous gaussian processes with independent increments. *Stochastic processes and their applications*, 44(2):181–204, 1993. **1, 4, 37**

[64] J. Jacod and A. Shiryaev. *Limit theorems for stochastic processes*, volume 288. Springer Science & Business Media, 2013. 117

[65] Patrick Jaillet, Damien Lamberton, and Bernard Lapeyre. Variational inequalities and the pricing of american options. *Acta Applicandae Mathematica*, 21:263–289, 1990. **36**

[66] G.-M. James and T.-J. Hastie. Functional linear discriminant analysis for irregularly sampled curves. *Journal of the Royal Statistical Society Series B: Statistical Methodology*, 63:533–550, 2001. 17

[67] I. Karatzas and S. Shreve. *Brownian motion and stochastic calculus*, volume 113. springer, 2014. 4, 15, 38, 90

[68] P. Kidger, J. Foster, X. Li, and T.-J. Lyons. Neural sdes as infinite-dimensional gans. In *International conference on machine learning*, pages 5453–5463. PMLR, 2021. 88

[69] Masaaki Kijima, Keiichi Tanaka, and Tony Wong. A multi-quality model of interest rates. *Quantitative Finance*, 9(2):133–145, 2009. 36

[70] D. Lamberton and B. Lapeyre. *Introduction to stochastic calculus applied to finance*. CRC press, 2011. 1, 2, 36

[71] J.-F. Le Gall. *Mouvement brownien, martingales et calcul stochastique*. Springer, 2013. 2, 118

[72] S.-J. Leon, A. Björck, and W. Gander. Gram-schmidt orthogonalization: 100 years and more. *Numerical Linear Algebra with Applications*, 20(3):492–532, 2013. 128

[73] S. López-Pintado and J. Romo. Depth-based classification for functional data. *DIMACS Series in Discrete Mathematics and Theoretical Computer Science*, 72:103, 2006. 17

[74] G.-G. Lorentz, M.-V. Golitschek, and Y. Makovoz. *Constructive approximation: advanced problems*, volume 304. Springer, 1996. 76, 150
[75] N. Marie and A. Rosier. Nadaraya-watson estimator for iid paths of diffusion processes. *arXiv preprint arXiv:2105.06884*, 2021. 4, 89

[76] P. Massart, E. Nédélec, et al. Risk bounds for statistical learning. *The Annals of Statistics*, 34(5):2326–2366, 2006. 134, 136

[77] F. Maturo and R. Verde. Supervised classification of curves via a combined use of functional data analysis and tree-based methods. *Comput. Statist.*, 38(1):419–459, 2023. 2

[78] K. Mosler and P. Mozharovskyi. Fast dd-classification of functional data. *Statistical Papers*, 58(4):1055–1089, 2017. 17

[79] D. Nualart. *The Malliavin calculus and related topics*, volume 1995. Springer, 2006. 140

[80] D.-W. Odell-Scott. Multivariate density estimation: theory, practice, and visualization, 1992. 56

[81] A. Oga and Y. Koike. Drift estimation for a multi-dimensional diffusion process using deep neural networks. *arXiv preprint arXiv:2112.13332*, 2021. 30

[82] J.-Y. Park and B. Wang. Nonparametric estimation of jump diffusion models. *Journal of Econometrics*, 222(1):688–715, 2021. 37

[83] J.-O. Ramsay and B.-W. Silverman. *Applied functional data analysis: methods and case studies*. Springer, 2002. 1

[84] J.-O. Ramsay and B.-W. Silverman. *Fitting differential equations to functional data: Principal differential analysis*. Springer, 2005. 1, 88

[85] R. Renò. Nonparametric estimation of stochastic volatility models. *Economics Letters*, 90(3):390–395, 2006. 37

[86] D. Revuz and M. Yor. *Continuous martingales and Brownian motion*, volume 293. Springer Science & Business Media, 2013. 2, 16, 32, 44, 82, 91, 139

[87] P. Romanczuk, M. Bär, W. Ebeling, B. Lindner, and L. Schimansky-Geier. Active brownian particles: From individual to collective stochastic dynamics. *The European Physical Journal Special Topics*, 202:1–162, 2012. 1

[88] Ivo F Sbalzarini. *Analysis, modeling, and simulation of diffusion processes in cell biology*. PhD thesis, ETH Zurich, 2006. 36

[89] E. Schmisser. Non-parametric estimation of the diffusion coefficient from noisy data. *Statistical inference for stochastic processes*, 15(3):193–223, 2012. 37

[90] E. Schmisser. Penalized nonparametric drift estimation for a multidimensional diffusion process. *Statistics*, 47(1):61–84, 2013. 30

[91] E. Schmisser. Non parametric estimation of the diffusion coefficients of a diffusion with jumps. *Stochastic processes and their applications*, 129(12):5364–5405, 2019. 37

[92] J. Sohn, S. Jeong, Y.-M. Cho, and T. Park. Functional clustering methods for binary longitudinal data with temporal heterogeneity. *Comput. Statist. Data Anal.*, 185:Paper No. 107766, 19, 2023. 2

[93] H. Sørensen. Estimation of diffusion parameters for discretely observed diffusion processes. *Bernoulli*, pages 491–508, 2002. 1, 4, 37

[94] P. Soulier. *Estimation fonctionnelle dans divers cadres de dépendance*. PhD thesis, Paris 11, 1993. 2, 37

[95] A.-B. Tsybakov. *Introduction to nonparametric estimation*. Springer Science & Business Media, 2008. 47, 96
[96] J.-W. Tukey. Mathematics and the picturing of data. In *Proceedings of the International Congress of Mathematicians, Vancouver, 1975*, volume 2, pages 523–531, 1975. 17

[97] S. Van-de Geer. Exponential inequalities for martingales, with application to maximum likelihood estimation for counting processes. *The Annals of Statistics*, pages 1779–1801, 1995. 117

[98] V.-N. Vapnik. An overview of statistical learning theory. *IEEE transactions on neural networks*, 10(5):988–999, 1999. 134, 136

[99] J.-L. Wang, J.-M. Chiou, and H.-G. Müller. Functional data analysis. *Annual Review of Statistics and its application*, 3:257–295, 2016. 88

[100] S. Wang, J. Cao, and S.-Y. Philip. Deep learning for spatio-temporal data mining: A survey. *IEEE transactions on knowledge and data engineering*, 34(8):3681–3700, 2020. 88

[101] Y. Yang. Minimax nonparametric classification: Rates of convergence. *IEEE Transactions on Information Theory*, 45(7):2271–2284, 1999. 99

[102] T. Zhang. Statistical behavior and consistency of classification methods based on convex risk minimization. *Annals of Statistics*, pages 56–85, 2004. 134, 137