<figure><img src="image_1.png" /></figure>

HAL
open science

# Drift Estimation for Diffusion Processes Using Neural Networks Based on Discretely Observed Independent Paths

Yuzhen Zhao, Yating Liu, Marc Hoffmann

► To cite this version:

Yuzhen Zhao, Yating Liu, Marc Hoffmann. Drift Estimation for Diffusion Processes Using Neural Networks Based on Discretely Observed Independent Paths. 2025. hal-05367231

HAL Id: hal-05367231

https://hal.science/hal-05367231v1

Preprint submitted on 15 Nov 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Drift Estimation for Diffusion Processes Using Neural Networks Based on Discretely Observed Independent Paths

Yuzhen Zhao¹, Yating Liu², Marc Hoffmann³

¹Sorbonne Université, and LEDa, Université Paris-Dauphine, PSL, Paris, France

²CEREMADE, CNRS, Université Paris-Dauphine, PSL, 75016 Paris, France

³CEREMADE, CNRS, Université Paris-Dauphine, PSL and Institut Universitaire de France, 75016 Paris, France
yuzhen.zhao@dauphine.eu, liu@ceremade.dauphine.fr, hoffmann@ceremade.dauphine.fr

## Abstract

This paper addresses the nonparametric estimation of the drift function over a compact domain for a time-homogeneous diffusion process, based on high-frequency discrete observations from $N$ independent trajectories. We propose a neural network-based estimator and derive a non-asymptotic convergence rate, decomposed into a training error, an approximation error, and a diffusion-related term scaling as $\log N/N$. For compositional drift functions, we establish an explicit rate. In the numerical experiments, we consider a drift function with local fluctuations generated by a double-layer compositional structure featuring local oscillations, and show that the empirical convergence rate becomes independent of the input dimension $d$. Compared to the $B$-spline method, the neural network estimator achieves better convergence rates and more effectively captures local features, particularly in higher-dimensional settings.

**Code** — https://github.com/yuzhen3001/nn-drift-estimation-diffusion-process

## 1 Introduction

In this paper, we study the estimation of the drift function of a time-homogeneous diffusion process using neural networks. Specifically, we consider an $\mathbb{R}^d$-valued diffusion process $X = (X_t)_{t \in [0,T]}$ that solves the stochastic differential equation (SDE)

$$dX_t = b(X_t) dt + \sigma(X_t) dB_t, \quad (1.1)$$

where $X_0$ is a random vector, $B = (B_t)_{t \in [0,T]}$ is a $d$-dimensional standard Brownian motion, and $b: \mathbb{R}^d \to \mathbb{R}^d$ and $\sigma: \mathbb{R}^d \to \mathbb{R}^d \times \mathbb{R}^d$ are measurable functions. To estimate the drift, we assume that $N$ independent and identically distributed (i.i.d.) sample paths $\bar{X}^{(n)} = (\bar{X}_t^{(n)})_{t \in [0,T]}$, $1 \le n \le N$, are available, each having the same distribution as $X$ and independent of $X$. Moreover, each sample path is observed at high frequency, that is, at discrete time points on a fine grid:

$$\bar{X}_{t_0:t_M}^{(n)} := (\bar{X}_{t_0=0}^{(n)}, \bar{X}_{t_1}^{(n)}, \dots, \bar{X}_{t_M=T}^{(n)}), \quad 1 \le n \le N, \quad (1.2)$$

where $t_m = m\Delta$ with the time step $\Delta = \frac{T}{M} \to 0$ as $M \to \infty$.

## Literature review

Diffusion processes, as defined by (1.1), are fundamental stochastic models with various applications in physics, biology, and mathematical finance (see, e.g., Gardiner (2004); Bressloff (2014); Karatzas and Shreve (1998)). A central task is the estimation of the drift function, which has been extensively studied under two standard frameworks: the long-time horizon regime, often assuming ergodicity (see, e.g., Hoffmann (1999); Kutoyants (2004); Dalalyan (2005); Frishman and Ronceray (2020)), and the high-frequency regime over a fixed time interval (see, e.g., Comte and Genon-Catalot (2020); Denis, Dion-Blanc, and Martinez (2021); Yildiz et al. (2018)).

In recent years, neural networks have become widely used as effective tools for nonparametric function estimation in statistical learning (see, e.g., Yarotsky (2017); Schmidt-Hieber (2020)). They have also been successfully applied to drift estimation in diffusion processes under the long-time horizon setting with ergodicity assumptions (see, e.g., Oga and Koike (2024); Ren et al. (2024)). On the other hand, the neural network-based drift estimation in the high-frequency regime over a fixed time interval is beginning to be explored (see, e.g., (Gao, Barzel, and Yan 2024; Bae, Ha, and Jeong 2025)), and theoretical analyses remain limited. Our work contributes to this direction by providing convergence guarantees.

## Contribution and organization of this paper

In this work, we address the problem of estimating the drift function $b$ over a compact set $K \subset \mathbb{R}^d$, using neural networks defined further in (2.2), based on discrete observations of the process over a fixed and finite time horizon $T$. Importantly, our approach does not rely on the ergodicity of the diffusion process, in contrast to Oga and Koike (2024). Moreover, since the drift function can be estimated component-wise, we can focus on estimating each component separately and combine these estimators to obtain an estimator for $b$. Specifically, for a given $i \in \{1, \dots, d\}$, we consider the $i$-th component of $b$ restricted to the compact domain $[0, 1]^d$. Our estimation target is therefore the function

$$f_0 := b^i \mathbf{1}_{[0,1]^d}, \quad (1.3)$$
where $\mathbf{1}_{[0,1]^d}$ denotes the indicator function of the domain $[0, 1]^d$. It is worth noting that the assumption of a compact domain is standard in the literature (see, e.g., Hoffmann (1999); Comte, Genon-Catalot, and Rozenholc (2007); Oga and Koike (2024)). In this paper, the choice of $[0, 1]^d$ is made for convenience in theoretical analysis and can be replaced by any compact set $K \subset \mathbb{R}^d$. In practice, the compact set $K$ can be determined from the training data distribution (e.g., based on sample coverage or through a sample-splitting procedure).

The main contribution of this paper is Theorem 2.1, which establishes a non-asymptotic upper bound on the $L^2$ estimation risk in terms of the training error from neural network optimization, the approximation error over the neural network class, the number of training samples $N$, and the time step $\Delta$ of the observation grid. Moreover, under a composition structure assumption on $f_0$, as considered in Schmidt-Hieber (2020) and Oga and Koike (2024), we derive an explicit upper bound on the test error as a function of $N$.

The paper is organized as follows. In Section 2, we present the mathematical framework, along with the assumptions and main result (Theorem 2.1). Section 3 provides a simulation study illustrating the proposed method, where we compare the performance of our estimator with that of Denis, Dion-Blanc, and Martinez (2021), which uses a B-spline-based estimator and ridge estimation. The numerical results show that our estimator outperforms the reference method in Denis, Dion-Blanc, and Martinez (2021) in terms of convergence rate and the ability to capture the local fluctuations, especially in high-dimensional settings. Finally, a sketch of the proof of the main result is provided in Section 4.

## Notations

For a vector or matrix $W$, let $|W|$ denote the Euclidean norm (if $W$ is a vector) or the Frobenius norm (if $W$ is a matrix). We write $|W|_\infty$ for the maximum-entry norm and $|W|_0$ for the number of nonzero entries. For $\beta \in \mathbb{R}$, $|\beta|$ denotes the largest integer strictly smaller than $\beta$. For two sequences $(a_N)$ and $(b_N)$, we write $a_N \lesssim b_N$ if there exists a constant $C$ such that $a_N \le Cb_N$ for all $N$, and we write $a_N \asymp b_N$ if $a_N \lesssim b_N$ and $b_N \lesssim a_N$. Moreover, for a function $f: \mathbb{R}^d \to \mathbb{R}$, let $\|f\|_\infty := \sup_{x \in [0,1]^d} |f(x)|$. For a random variable $X$, we write $\mathcal{L}(X)$ for its law. The index $m = 0, \dots, M-1$ corresponds to time steps on the discrete observation grid, while $n = 1, \dots, N$ indexes the samples in the training set.

# 2 Mathematical setting and main result

This section presents the mathematical setting and main result of the paper. We begin by introducing the neural network class used for estimation, followed by a description of the proposed estimator and the different loss functions considered. Finally, we present the assumptions and main result of the paper.

## Neural network class for estimation

We use feedforward neural networks to estimate the drift function component-wise. Specifically, let $\mathcal{F}_{L,p}$ denote the class of functions defined by neural networks with $L$ hidden layers, layer widths given by $\mathbf{p} = (p_0, p_1, \dots, p_{L+1}) \in \mathbb{N}^{L+2}$, where $p_0 = d$ is the input dimension and $p_{L+1} = 1$ is the output dimension. Each $f \in \mathcal{F}_{L,p}$ is a function from $\mathbb{R}^d$ to $\mathbb{R}$ of the form

$$f(x) = W_L \sigma_{v_L} W_{L-1} \sigma_{v_{L-1}} \cdots W_1 \sigma_{v_1} W_0 x, \quad (2.1)$$

where $W_i$ is a $p_{i+1} \times p_i$ weight matrix, $\sigma(x) = \max(x, 0)$ is the ReLU function applied component-wise, and for $\mathbf{v} = (v_1, \dots, v_r) \in \mathbb{R}^r$, $\sigma_{\mathbf{v}}: \mathbb{R}^r \to \mathbb{R}^r$ denotes the shifted ReLU function with shift vector $\mathbf{v}$, defined by

$$\sigma_{\mathbf{v}}((y_1, \dots, y_r)^\top) = (\sigma(y_1 - v_1), \dots, \sigma(y_r - v_r))^\top.$$

In this paper, we consider neural networks with sparsity parameter $s$, and we restrict the neural network functions to be uniformly bounded by a constant $F > 0$. Since the target function $f_0$ is supported on $[0, 1]^d$ (see (1.3)), we define the class of neural network estimators $\mathcal{F}(L, \mathbf{p}, s, F)$ as follows:

$$\begin{aligned} \mathcal{F}(L, \mathbf{p}, s, F) &:= \left\{ f \mathbf{1}_{[0,1]^d} : f \in \mathcal{F}_{L,\mathbf{p}} \text{ such that} \right. \\ & \qquad \max_{j=0,\dots,L} (\|W_j\|_\infty \vee |\mathbf{v}_j|_\infty) \le 1, \quad \|f\|_\infty \le F, \\ & \qquad \text{and} \quad \sum_{j=0}^L (\|W_j\|_0 + |\mathbf{v}_j|_0) \le s \right\}. \end{aligned} \quad (2.2)$$

When there is no ambiguity, we abbreviate $\mathcal{F}(L, \mathbf{p}, s, F)$ as $\mathcal{F}$ for simplicity, and we also use the notation $\mathcal{F}(L, \mathbf{p}, s) := \mathcal{F}(L, \mathbf{p}, s, \infty)$.

## Estimator and loss functions

Recall that our training data $\mathcal{D}_N$ for this estimation problem consists of discrete observations from independent sample paths of solutions to (1.1), that is,

$$\mathcal{D}_N = \left\{ \bar{X}_{t_0:t_M}^{(n)} = (\bar{X}_{t_0}^{(n)}, \dots, \bar{X}_{t_M}^{(n)}), 1 \le n \le N \right\}.$$

For a fixed $n$, let $\bar{X}^{(n),i}$ denote the $i$-th component of $\bar{X}^{(n)}$. The classical approach for estimating $b^i$ considers the increment

$$Y_{t_m}^{(n)} := \frac{1}{\Delta} \left( \bar{X}_{t_{m+1}}^{(n),i} - \bar{X}_{t_m}^{(n),i} \right), \quad (2.3)$$

which is approximately equal to $b^i(\bar{X}_{t_m}^{(n)})$ plus a noise term (see Denis, Dion-Blanc, and Martinez (2021, Appendix C), Oga and Koike (2024, Equation (4.1)); see also (4.2)). Unlike in classical regression with neural networks (see Schmidt-Hieber (2020)), the errors in our setting are neither normally distributed nor independent of $b^i(\bar{X}_{t_m}^{(n)})$, which presents a key challenge in analyzing the estimator's performance.

Based on the above definition of $Y_{t_m}^{(n)}$, our neural network training procedure consists in minimizing

$$Q_{\mathcal{D}_N}(f) := \frac{1}{NM} \sum_{n=1}^{N} \sum_{m=0}^{M-1} \left( Y_{t_m}^{(n)} - f(\bar{X}_{t_m}^{(n)}) \right)^2 \quad (2.4)$$
over $f \in \mathcal{F}(L, \mathbf{p}, s, F)$. In practice, finding an exact minimizer of $\mathcal{Q}_{\mathcal{D}_N}(f)$ may be challenging. To quantify the discrepancy between our estimator $\hat{f} = \hat{f}_N$, which is the output of the neural network training procedure applied to the empirical loss $\mathcal{Q}_{\mathcal{D}_N}(f)$, and an exact minimizer of $\mathcal{Q}_{\mathcal{D}_N}(f)$ over $\mathcal{F}(L, \mathbf{p}, s, F)$, we define

$$ \Psi^{\mathcal{F}}(\hat{f}) := \mathbb{E} \left[ \mathcal{Q}_{\mathcal{D}_N}(\hat{f}) - \inf_{f \in \mathcal{F}(L, \mathbf{p}, s, F)} \mathcal{Q}_{\mathcal{D}_N}(f) \right]. \quad (2.5) $$

The performance of $\hat{f}$ is measured by the estimation risk

$$ \mathcal{R}(\hat{f}, f_0) := \mathbb{E} \left[ \frac{1}{M} \sum_{m=0}^{M-1} \left( \hat{f}(X_{t_m}) - f_0(X_{t_m}) \right)^2 \right]. \quad (2.6) $$

where $X = (X_t)_{t \in [0,T]}$ is a solution to (1.1) that is independent of the training set $\mathcal{D}_N$. In practice (see also the numerical experiments in Section 3), the estimation risk $\mathcal{R}(\hat{f}, f_0)$ is often estimated by the following empirical test error

$$ \tilde{\mathcal{R}}(\hat{f}, f_0) := \frac{1}{MN'} \sum_{n=1}^{N'} \sum_{m=0}^{M-1} \left( \hat{f}(\tilde{X}_{t_m}^{(n)}) - f_0(\tilde{X}_{t_m}^{(n)}) \right)^2, \quad (2.7) $$

where $\tilde{X}^{(n)} = (\tilde{X}_t^{(n)})_{t \in [0,T]}$, $1 \le n \le N'$, are i.i.d. sample paths, independent of the training set $\mathcal{D}_N$.

## Assumptions and main result

Throughout this paper, we impose the following two assumptions. Assumption 1 ensures the existence and uniqueness of the solution to the SDE (1.1), while Assumption 2 is a technical condition that guarantees the convergence result stated in Theorem 2.1.

**Assumption 1.** The initial random variable $X_0$ satisfies $\mathbb{E}[|X_0|^2] < +\infty$. The coefficient functions $b$ and $\sigma$ are globally Lipschitz continuous; that is, there exist constants $L_b, L_\sigma > 0$ such that for all $x, y \in \mathbb{R}^d$,

$$ |b(x) - b(y)| \le L_b |x - y|, \quad |\sigma(x) - \sigma(y)| \le L_\sigma |x - y|. $$

Since $b$ and $\sigma$ are continuous, we define

$$ C_b := \sup_{x \in [0,1]^d} |b(x)| < \infty \quad \text{and} \quad C_\sigma := \sup_{x \in [0,1]^d} |\sigma(x)| < \infty. $$

It is obvious that $\|f_0\|_\infty \le C_b$.

**Assumption 2.** $F \ge \max(C_b, 1)$, $C_\sigma > 0$, $L \ge 1$, $s \ge 2$, $N \ge 2$ and $\Delta \le 1$.

Our main result is presented in the following theorem.

**Theorem 2.1.** Suppose that Assumptions 1 and 2 hold. There exists a constant $\mathcal{C}$, depending only on $C_b, C_\sigma, L_b, L_\sigma, T$, and a universal constant $C$ (introduced later in Lemma 4.3), such that

$$ \mathcal{R}(\hat{f}, f_0) \le 4\Psi^{\mathcal{F}}(\hat{f}) + 6 \inf_{f \in \mathcal{F}(L, \mathbf{p}, s, F)} \mathcal{R}(f, f_0) + \mathcal{C}F^2 \left( \Delta + \frac{s(L \log s + \log d) + s \log 4F}{N} + s \frac{\log N}{N} \right). $$

**Remark 2.2.** Theorem 2.1 provides a decomposition of the upper bound on the estimation risk into three components. The first term reflects the error arising from the training procedure for finding a global minimizer with respect to the empirical loss $\mathcal{Q}_{\mathcal{D}_N}$. The second term can be bounded by the classical regression error for i.i.d. samples, as analyzed in Schmidt-Hieber (2020), and does not depend on the diffusion setting. Finally, the third term captures the error due to the temporal dependence inherent in the diffusion process.

Now let

$$ C_r^\beta(D, K) = \left\{ \begin{array}{ll} f: D \subset \mathbb{R}^r \rightarrow \mathbb{R}: & \sum_{\alpha:|\alpha|<\beta} \|\partial^\alpha f\|_\infty \\ & + \sum_{\alpha:|\alpha|=\lfloor\beta\rfloor} \sup_{x,y \in D, x \neq y} \frac{|\partial^\alpha f(x) - \partial^\alpha f(y)|}{|x-y|_\infty^{\beta-\lfloor\beta\rfloor}} \le K \end{array} \right\}. $$

Let $\mathcal{G}(q, \mathbf{d}, \mathbf{t}, \beta, K)$ be the function space defined in Schmidt-Hieber (2020):

$$ \mathcal{G}(q, \mathbf{d}, \mathbf{t}, \beta, K) := \{f = g_q \circ \dots g_0: $$

$$ g_i = (g_{ij})_j: [a_i, b_i]^{d_i} \rightarrow [a_{i+1}, b_{i+1}]^{d_{i+1}}, $$

$$ g_{ij} \in C_{t_i}^{\beta_i}([a_i, b_i]^{t_i}, K), \text{ for some } |a_i|, |b_i| \le K \}. $$

with $\mathbf{d} := (d_0, \dots, d_{q+1})$, $\mathbf{t} := (t_0, \dots, t_q)$, $\beta := (\beta_0, \dots, \beta_q)$. Define $\beta_i^* := \beta_i \prod_{l=i+1}^q (\beta_l \wedge 1)$ and

$$ \phi_N := \max_{0 \le i \le q} N^{-\frac{2\beta_i^*}{2\beta_i^*+t_i}}. \quad (2.8) $$

**Corollary 2.3.** Suppose that Assumptions 1 and 2 hold. Assume moreover that $f_0 \in \mathcal{G}(q, \mathbf{d}, \mathbf{t}, \beta, K)$ and the neural network estimators set $\mathcal{F}(L, \mathbf{p}, s, F)$ satisfies

(i) $F \ge \max(K, 1)$, $L \asymp \log_2 N$

(ii) $N\phi_N \lesssim \min_{i=1,\dots,L} p_i$, $s \asymp N\phi_N \log N$.

Then there exists a constant $C$ depending on $q, \mathbf{d}, \mathbf{t}, \beta, F, C_b, C_\sigma, L_b, L_\sigma, T$ such that if

$$ \Delta \lesssim \phi_N \log^3 N \text{ and } \Psi^{\mathcal{F}}(\hat{f}) \le C\phi_N \log^3 N, $$

then $\mathcal{R}(\hat{f}, f_0) \le C\phi_N \log^3 N$.

# 3 Numerical experiments

In this section, we evaluate the performance of our estimator using the following drift functions:

$$ x \in \mathbb{R}^d \mapsto b(x) = -x + \phi \left( \frac{s(x)}{\theta} \right) \mathbf{1}_d, \quad (3.1) $$

where $\mathbf{1}_d = (1, \dots, 1)^\top$, $s(x) := \sum_{i=1}^d x_i$, $\theta = 0.2$ and $\phi(z) := 2z \exp(-z^2) - 8z \exp(-2z^2)$. In the following, Figure 3a (True function) illustrates $b(x)$ in one dimension. The initial condition $X_0$ is set as a random variable having standard normal distribution $N(0, I_d)$, and the diffusion coefficient $\sigma(x)$ is set to the identity matrix.

The numerical study consists of three parts. First, we examine the convergence rate of our estimator with respect to $N$. Then, we compare its performance to the B-spline-based estimator proposed in Denis, Dion-Blanc, and Martinez (2021), focusing on two aspects: the convergence rate
and the ability to capture the local fluctuations induced by the function $\phi$ in the drift. Finally, we conclude this section with a summary of the comparative results, followed by additional remarks on the memory requirements and computational feasibility of our estimator compared to the B-spline-based approach.

## Experimental setup

Throughout this section, we fix the time horizon to $T = 1$ and the time step to $\Delta = 0.01$, and we estimate the first component of the drift coefficient $b_1(x) : \mathbb{R}^d \to \mathbb{R}$. We set the test set size $N' = 1000$, and the estimation risk is approximated using its empirical version defined in (2.7). The experiments are carried out for a range of dimensions $d \in \{1, 2, 10, 50\}$ and training size $N \in \{100, 200, 500, 1000, 2000, 5000\}$.

For the neural network estimator, we perform hyperparameter tuning for each combination of $(N, d)$. We consider different hidden layer width

$$p \in \{(d, 16, 16, 1), (d, 16, 32, 16, 1), (d, 16, 32, 32, 16, 1)\}.$$

The number of non-zero parameters $s$ is set as a proportion $s_{\text{ratio}}$ of the total number of parameters, with $s_{\text{ratio}} \in \{0.25, 0.5, 0.75\}$. For the training procedure, we use the Adam optimizer with a learning rate of $10^{-3}$. The final number of training epochs is determined via early stopping triggered after 20 consecutive epochs.

Moreover, for each configuration, we repeat the experiment 50 times independently and compute the average error along with the corresponding 95% confidence interval. All implementations are performed using PyTorch.

## Numerical study of the convergence rate

We first examine how the estimation error evolves with the number of training trajectories $N$. In the setting of Corollary 2.3, we have $\phi_N = N^{-1}$ for the first component of the drift function $b$ defined in (3.1). Hence, if $\Psi^F(\hat{f}) \le C\phi_N \log^3 N$, then the theoretical upper bound of $\mathcal{R}(\hat{f}, f_0)$ will be $CN^{-1} \log^3 N$ for some constant $C$.

Figure 1 shows the log-log plot of the estimation risk $\mathcal{R}(\hat{f}, f_0)$ as a function of $N$ for $d \in \{1, 2, 10, 50\}$. The empirical convergence rate closely matches the theoretical upper bound for $d \in \{1, 2, 50\}$. For $d = 10$, however, the empirical error slightly exceeds the theoretical envelope for certain values of $N$. This deviation likely results from the restricted hyperparameter search space, which covered only a limited range of learning rates, training epochs, network widths and depths, and regularization parameters. However, the search grid was intentionally constrained to ensure comparability and to maintain a reasonable computational cost across dimensions.

## Comparison with B-spline-base estimator

We now compare our estimator with the B-spline-based estimator from Denis, Dion-Blanc, and Martinez (2021), using the same training and test datasets. The implementation of the B-spline estimator follows the procedure described in Denis, Dion-Blanc, and Martinez (2021), with an adaption to

the multivariate setting using tensor-product B-splines basis (see e.g. Györfi et al. (2002)), and the number of knots selected based on validation performance.

Figure 2 demonstrates that our method achieves a better convergence rate of the estimation risk in $N$. Furthermore, as illustrated in Figure 3, our estimator ($p = (d, 16, 32, 16, 1)$, $s_{\text{ratio}} = 0.75$) more accurately captures the local variations of the drift function, while the B-spline-based method fails to recover these fine-scale features.

<figure><img src="image_3.png" /><figcaption>Figure 1: Convergence rates of our estimator for $d = 1$ (top left), $d = 2$ (top right), $d = 10$ (bottom left), and $d = 50$ (bottom right).</figcaption></figure>

<figure><img src="image_4.png" /><figcaption>Figure 2: Comparison of convergence rates for $d = 1$ (left) and $d = 2$ (right).</figcaption></figure>

## Conclusion

In summary, Figures 2 and 3 clearly demonstrate that our neural network-based estimator outperforms the B-spline-based estimator both in terms of convergence rate and its ability to capture local fluctuations.

Furthermore, for the B-spline-based estimator, the number of basis functions grows exponentially with the dimension $d$, leading to a substantial increase in memory requirements. For example, when $d = 5$, assuming 8 basis func-
tions per dimension, the resulting coefficient matrix has size $MN \times 32768$. If each entry is stored as a 64-bit float (8 bytes), this amounts to over 24GB of memory for $M = 100$ and $N = 1000$, not including the additional memory required for matrix inversion. As a result, the B-spline estimator becomes computationally infeasible on a standard laptop even for moderate values of $d$, for example $d \ge 3$. In contrast, the neural network estimator only requires storing the network parameters and minibatch data during training, making it far more scalable in high-dimensional settings.

<figure><img src="image_5.png" /><figcaption>(a) NN estimator, $d = 1$</figcaption></figure>

<figure><img src="image_6.png" /><figcaption>(b) B-spline estimator, $d = 1$</figcaption></figure>

<figure><img src="image_7.png" /><figcaption>(c) NN estimator, $d = 2$</figcaption></figure>

<figure><img src="image_8.png" /><figcaption>(d) B-spline estimator, $d = 2$</figcaption></figure>

<caption>Figure 3: Comparison of the ability to capture local fluctuations (N=5000). For $d = 2$, we display $\hat{f}$ (blue and orange) together with $f_0 - 5$ (green) to improve visual clarity.</caption>

# 4 Proof sketch

The proof of Theorem 2.1 consists of two steps. In the first step (see Proposition 4.1), we establish an upper bound of the risk $\mathcal{R}(\hat{f}, f_0)$ defined in (2.6) by using the following expected empirical train risk $\hat{\mathcal{R}}_{\mathcal{D}_N}(\hat{f}, f_0)$, defined by

$$ \hat{\mathcal{R}}_{\mathcal{D}_N}(\hat{f}, f_0) := \mathbb{E} \left[ \frac{1}{NM} \sum_{n=1}^{N} \sum_{m=0}^{M-1} (\hat{f}(\bar{X}_{t_m}^{(n)}) - f_0(\bar{X}_{t_m}^{(n)}))^2 \right], \quad (4.1) $$

and an estimation of the covering number of the neural network estimator set $\mathcal{F}(L, \mathbf{p}, s, F)$. Recall that for any $\delta > 0$, a subset $\mathcal{G} \subset \mathcal{F}$ is called a $\delta$-net of $\mathcal{F}$ with respect to some norm $\|\cdot\|$ on $\mathcal{F}$, if for every $f \in \mathcal{F}$, there exists $g \in \mathcal{G}$ such that $\|f - g\| \le \delta$. The minimal cardinality of such $\delta$-net is called the covering number, denoted by $\mathfrak{N}(\delta, \mathcal{F}, \|\cdot\|)$ (see e.g. van der Vaart and Wellner (2023)). When there is no ambiguity about the training set, we simplify the notation by writing $\hat{\mathcal{R}}(\hat{f}, f_0)$ instead of $\hat{\mathcal{R}}_{\mathcal{D}_N}(\hat{f}, f_0)$. In the second step (see Proposition 4.4), we derive an upper bound on the expected empirical training risk $\hat{\mathcal{R}}(\hat{f}, f_0)$ in terms of $\Psi^{\mathcal{F}}(\hat{f})$,

the approximation error $\inf_{f \in \mathcal{F}} \|f - f_0\|_{\infty}^2$, the neural network parameters $(s, L, F)$, and the data parameters $\Delta, N$, and $d$.

## Upper bound of $\mathcal{R}(\hat{f}, f_0)$

This subsection is devoted to establishing the following proposition.

**Proposition 4.1.** Suppose that Assumptions 1 and 2 hold. Fix an arbitrary $\bar{\varepsilon} \in (0, 1)$, we have

$$ \mathcal{R}(\hat{f}, f_0) \le 3\bar{\varepsilon} + 2\hat{\mathcal{R}}(\hat{f}, f_0), \\ + \frac{44F^2 \log (Cs(L \log s + \log d + \log 4F - \log \bar{\varepsilon}))}{N}, $$

where $C > 0$ is a universal constant.

The proof of Proposition 4.1 relies on the following two lemmas.

**Lemma 4.2** (Lemma A.2 in Denis, Dion-Blanc, and Martínez (2021)). Let $X_1, \dots, X_N$ be independent copies of a random variable $X \in \mathcal{X}$. Let $\mathcal{G}$ be a class of real-valued functions on $\mathcal{X}$. For each $g \in \mathcal{G}$, and $x \in \mathcal{X}$, we assume that $0 \le g(x) \le L$, with $L > 0$. We consider $\mathcal{G}_{\delta}$ an $\delta$-net of $\mathcal{G}$ w.r.t. $\|\cdot\|_{\text{sup}}$ and we denote by $\mathfrak{N}(\delta, \mathcal{G}, \|\cdot\|_{\text{sup}})$ its cardinality. Then, the following holds

$$ \mathbb{E} \left[ \sup_{g \in \mathcal{G}} \left( \mathbb{E}[g(X)] - \frac{2}{N} \sum_{i=1}^{N} g(X_i) \right) \right] \le 3\varepsilon + \frac{11L \log \left( \mathfrak{N}(\delta, \mathcal{G}, \|\cdot\|_{\text{sup}}) \right)}{N}. $$

**Lemma 4.3** (Lemma 4.13 in Oga and Koike (2024)). Let $\mathcal{F} \subset \mathcal{F}(L, \mathbf{p}, s)$. If $s \ge 2$, we have for all $\delta \in (0, 1)$

$$ \log \mathfrak{N}(\delta, \mathcal{F}, \|\cdot\|_{\infty}) \le C_s(L \log s + \log d - \log \delta) $$

where $C > 0$ is a universal constant.

**Proof of Proposition 4.1.** In what follows, we adopt the notation $X_{t_0:t_{M-1}}$ for $(X_{t_0}, \dots, X_{t_{M-1}})$ and $\bar{X}_{t_0:t_{M-1}}^{(n)}$ for $(\bar{X}_{t_0}^{(n)}, \dots, \bar{X}_{t_{M-1}}^{(n)})$. For a fixed $\bar{f} \in \mathcal{F}(L, \mathbf{p}, s, F)$, we define the function $g_{\bar{f}}$ from $(\mathbb{R}^d)^M$ to $\mathbb{R}$ by

$$ x = (x_0, \dots, x_{M-1}) \mapsto g_{\bar{f}}(x) = \frac{1}{M} \sum_{m=0}^{M-1} (\bar{f}(x_m) - f_0(x_m))^2, $$

which is bounded above by $4F^2$ by Assumption 2 and supported on $([0, 1]^d)^M$ since both $f_0$ and $\bar{f}$ are supported on $[0, 1]^d$. Moreover, we define

$$ \mathcal{G} := \{g_{\bar{f}} : \bar{f} \in \mathcal{F}(L, \mathbf{p}, s, F)\}, $$

equipped with the norm $\|g\|_{\text{sup}} := \sup_{x \in ([0,1]^d)^M} |g(x)|$.

The estimation risk $\mathcal{R}(\hat{f}, f_0)$ can be decomposed by

$$ \mathcal{R}(\hat{f}, f_0) \le \mathbb{E} \left[ g_{\bar{f}}(X_{t_0:t_{M-1}}) - \frac{2}{N} \sum_{n=1}^{N} g_{\bar{f}}(\bar{X}_{t_0:t_{M-1}}^{(n)}) \right] + 2\hat{\mathcal{R}}(\hat{f}, f_0), $$
where the first term can be upper bounded as follows

$$
\begin{align*}
& \mathbb{E} \left[ g_{\hat{f}}(X_{t_0:t_{M-1}}) - \frac{2}{N} \sum_{n=1}^{N} g_{\hat{f}}(\bar{X}_{t_0:t_{M-1}}^{(n)}) \right] \\
& \le \mathbb{E} \left[ \sup_{g_{\hat{f}} \in \mathcal{G}} \left( g_{\bar{f}}(X_{t_0:t_{M-1}}) - \frac{2}{N} \sum_{n=1}^{N} g_{\bar{f}}(\bar{X}_{t_0:t_{M-1}}^{(n)}) \right) \right] \\
& \le 3\varepsilon + \frac{44F^2 \log \left( \mathfrak{N}(\varepsilon, \mathcal{G}, \| \cdot \|_{\text{sup}}) \right)}{N},
\end{align*}
$$

for any $\varepsilon \in (0, 1)$ by applying Lemma 4.2.

We now provide an upper bound on the covering number $\mathfrak{N}(\varepsilon, \mathcal{G}, \| \cdot \|_{\text{sup}})$ using the covering number over $\mathcal{F}(L, \mathbf{p}, s, F)$. Let $\mathfrak{N}_{\frac{\varepsilon}{4F}} := \mathfrak{N}(\frac{\varepsilon}{4F}, \mathcal{F}(L, \mathbf{p}, s, F), \| \cdot \|_{\infty})$ and let $\mathcal{F}_{\text{net}} := \{f_1, \dots, f_{\mathfrak{N}_{\frac{\varepsilon}{4F}}}\}$ be an $\frac{\varepsilon}{4F}$-net of $\mathcal{F}(L, \mathbf{p}, s, F)$. Then for every $\bar{f} \in \mathcal{F}(L, \mathbf{p}, s, F)$, there exists $\bar{f}_{\frac{\varepsilon}{4F}} \in \mathcal{F}_{\text{net}}$ such that $\|\bar{f} - \bar{f}_{\frac{\varepsilon}{4F}}\|_{\infty} \le \frac{\varepsilon}{4F}$. It follows that

$$
\|g_{\bar{f}} - g_{\bar{f}_{\frac{\varepsilon}{4F}}}\|_{\infty} \le \|\bar{f} - \bar{f}_{\frac{\varepsilon}{4F}}\|_{\infty} \|\bar{f} + \bar{f}_{\frac{\varepsilon}{4F}} - 2f_0\|_{\infty} \le \varepsilon.
$$

Therefore, Lemma 4.3 implies that

$$
\begin{align*}
\log \left( \mathfrak{N}(\varepsilon, \mathcal{G}, \| \cdot \|_{\text{sup}}) \right) & \\
& \le \log \left( \mathfrak{N}\left(\frac{\varepsilon}{4F}, \mathcal{F}(L, \mathbf{p}, s, F), \| \cdot \|_{\infty}\right) \right) \\
& \le \text{Cs}(L \log s + \log d + \log 4F - \log \varepsilon).
\end{align*}
$$

Combining the results above, we get

$$
\mathcal{R}(\hat{f}, f_0) \le 3\varepsilon + 2\hat{\mathcal{R}}(\hat{f}, f_0) + \frac{44F^2 \text{Cs}(L \log s + \log d + \log 4F - \log \varepsilon)}{N}. \quad \square
$$

**Upper bound of** $\hat{\mathcal{R}}(\hat{f}, f_0)$

This section aims to establish an upper bound on the ex-
pected empirical training risk $\hat{\mathcal{R}}(\hat{f}, f_0)$. We abbreviate $\mathcal{F}$ for
the neural network function class $\mathcal{F}(L, \mathbf{p}, s, F)$, and $\mathfrak{N}_\delta$ for
the covering number $\mathfrak{N}(\delta, \mathcal{F}(L, \mathbf{p}, s, F), \| \cdot \|_{\infty})$.

**Proposition 4.4.** Suppose that Assumptions 1 and 2 hold. There exists a constant $\mathcal{C}'$ depending on $C_b, C_\sigma, L_b, L_\sigma, T$ and the universal constant $C$ in Lemma 4.3 such that

$$
\hat{\mathcal{R}}(\hat{f}, f_0) \le 2\Psi^{\mathcal{F}}(\hat{f}) + 3 \inf_{\hat{f} \in \mathcal{F}} \mathcal{R}(f, f_0) \\
+ \mathcal{C}' \left( F^2 \Delta + \frac{s(L \log s + \log d) + F}{N} + s \frac{\log N}{N} \right).
$$

To prove Proposition 4.4, we first note that $Y_{t_m}^{(n)}$, defined
in (2.3), admits the following decomposition :

$$
\begin{align}
Y_{t_m}^{(n)} &= b^i(\bar{X}_{t_m}^{(n)}) \nonumber \\
&\quad + \frac{1}{\Delta} \int_{t_m}^{t_{m+1}} (b^i(\bar{X}_s^{(n)}) - b^i(\bar{X}_{t_m}^{(n)}))ds \nonumber \\
&\quad + \frac{1}{\Delta} \int_{t_m}^{t_{m+1}} \sigma^i(\bar{X}_s^{(n)})dB_s^{(n)} \nonumber \\
&:=: b^i(\bar{X}_{t_m}^{(n)}) + I_{t_m}^{(n)} + \Sigma_{t_m}^{(n)}, \tag{4.2}
\end{align}
$$

where $B^{(n)}$, $1 \le n \le N$ are i.i.d. $d$-dimensional stan-
dard Brownian motion. Given any two functions $f_1, f_2$ in
$\mathcal{F}(L, \mathbf{p}, s, F)$, we define

$$
\Psi_{\mathcal{D}_N}(f_1, f_2) := \mathbb{E} [\mathcal{Q}_{\mathcal{D}_N}(f_1) - \mathcal{Q}_{\mathcal{D}_N}(f_2)]
$$

with $\mathcal{Q}_{\mathcal{D}_N}$ defined in (2.4). Let $\bar{f}$ be an arbitrary function in $\mathcal{F}$. A direct computation yields the following decomposition of $\hat{\mathcal{R}}(\hat{f}, f_0)$ :

$$
\begin{align}
\hat{\mathcal{R}}(\hat{f}, f_0) ={}& \Psi_{\mathcal{D}_N}(\hat{f}, \bar{f}) \tag{4.3} \\
& + \mathbb{E} \left[ \frac{1}{NM} \sum_{n=1}^{N} \sum_{m=0}^{M-1} (\bar{f}(\bar{X}_{t_m}^{(n)}) - f_0(\bar{X}_{t_m}^{(n)}))^2 \right] \nonumber \\
& + 2\mathbb{E} \left[ \frac{1}{NM} \sum_{n=1}^{N} \sum_{m=0}^{M-1} I_{t_m}^{(n)} (\hat{f}(\bar{X}_{t_m}^{(n)}) - \bar{f}(\bar{X}_{t_m}^{(n)})) \right]. \nonumber \\
& + 2\mathbb{E} \left[ \frac{1}{NM} \sum_{n=1}^{N} \sum_{m=0}^{M-1} \Sigma_{t_m}^{(n)} (\hat{f}(\bar{X}_{t_m}^{(n)}) - f_0(\bar{X}_{t_m}^{(n)})) \right], \nonumber
\end{align}
$$

where the last term follows from the fact that $\bar{f}$ and $f_0$
are deterministic (i.e., independent of the training set), and
$\mathbb{E}[\Sigma_{t_m}^{(n)}] = 0$ and independent of $\bar{X}_{t_m}^{(n)}$ for each $m$ and $n$.

Let $\mathcal{F}_{\text{net}} := \{f_1, \dots, f_{\mathfrak{N}_{\delta}}\}$ denote a $\delta$-net of $\mathcal{F}$ with respect to $\| \cdot \|_{\infty}$. Then, there exists a random index $n^*$ valued in $\{1, \dots, \mathfrak{N}_{\delta}\}$ such that $|\hat{f} - f_{n^*}| \le \delta$. A reasoning similar to Oga and Koike (2024, Lemma 4.8, 4.11, 4.12) leads to the following lemma.

**Lemma 4.5.** Suppose that Assumptions 1 and 2 hold. Let $C_* = 4(C_b^2 + C_\sigma^2) \exp(4(L_b^2 + L_\sigma^2))$.

1. For every $\bar{f} \in \mathcal{F}$ and $\varepsilon \in (0, 1)$,

$$
\begin{align}
& \mathbb{E} \left[ \frac{1}{NM} \sum_{n=1}^{N} \sum_{m=0}^{M-1} I_{t_m}^{(n)} \left( \hat{f}(\bar{X}_{t_m}^{(n)}) - \bar{f}(\bar{X}_{t_m}^{(n)}) \right) \right] \nonumber \\
& \leq \frac{\varepsilon}{4} \hat{\mathcal{R}}(\hat{f}, f_0) + \frac{\varepsilon}{2} \hat{\mathcal{R}}(\bar{f}, f_0) + \frac{3L_b^2 C_*}{4\varepsilon} \Delta. \tag{4.4}
\end{align}
$$

2. There exists a constant $\bar{C}_\sigma$ depending only on $C_\sigma$ such that for every $n \in \{1, \dots, N\}$,

$$
\begin{align}
& \mathbb{E} \left[ \frac{1}{MN} \sum_{n=1}^{N} \sum_{m=0}^{M-1} \Sigma_{t_m}^{(n)} (\hat{f} - f_{n^*}) (\bar{X}_{t_m}^{(n)}) \right] \nonumber \\
& \leq \delta \sqrt{\frac{1}{2} L_{\sigma}^2 C_*} + \frac{\bar{C}_{\sigma}}{\sqrt{T}} \int_{0}^{\delta} \sqrt{\log \mathfrak{N}_u} \, du. \tag{4.5}
\end{align}
$$

Furthermore, the proof of Proposition 4.4 relies on the fol-
lowing lemma.

**Lemma 4.6.** Suppose that Assumptions 1 and 2 hold. For every $\varepsilon \in (0, 1)$, we have

$$
\begin{align*}
& \mathbb{E} \left[ \frac{1}{NM} \sum_{n=1}^{N} \sum_{m=0}^{M-1} \Sigma_{t_m}^{(n)} (f_{n^*}(\bar{X}_{t_m}^{(n)}) - f_0(\bar{X}_{t_m}^{(n)})) \right] \\
& \leq \frac{\varepsilon}{4} \hat{\mathcal{R}}(\hat{f}, f_0) + \gamma_{\varepsilon},
\end{align*}
$$

where $\gamma_{\varepsilon} = \varepsilon F \delta + \varepsilon \frac{F^2 L_{\sigma}^2 C_*}{C_{\sigma}^2} \Delta + \frac{4C_{\sigma}^2}{\varepsilon T N} (\log 2 + 2 \log \mathfrak{N}_{\delta})$.
**Proof sketch of Lemma 4.6.** For a fixed function $f \in \mathcal{F}$ and for a fixed $n$, we define the processes $\widehat{M}^{(n)}(f)$, $\widehat{A}^{(n)}(f)$, $\overline{M}(f)$ and $\overline{A}(f)$ by

$$ \widehat{M}^{(n)}(f)_s := \sum_{m=0}^{M-1} (f - f_0)(\bar{X}_{t_m}^{(n)}) \int_{s \wedge t_m}^{s \wedge t_{m+1}} \sigma^i(\bar{X}_u^{(n)}) dB_u^{(n)}, $$

$$ \overline{M}(f)_s := \frac{1}{N} \sum_{n=1}^{N} \widehat{M}^{(n)}(f) \\ = \frac{1}{N} \sum_{n=1}^{N} \sum_{m=0}^{M-1} (f - f_0)(\bar{X}_{t_m}^{(n)}) \int_{s \wedge t_m}^{s \wedge t_{m+1}} \sigma^i(\bar{X}_u^{(n)}) dB_u^{(n)}, \\ \widehat{A}^{(n)}(f) := \langle \widehat{M}^{(n)}(f) \rangle, $$

$$ \overline{A}(f) := \langle \overline{M}(f) \rangle = \frac{1}{N^2} \sum_{n=1}^{N} \widehat{A}^{(n)}(f). $$

It follows that

$$ \mathbb{E} \left[ \frac{1}{NM} \sum_{n=1}^{N} \sum_{m=0}^{M-1} \Sigma_{t_m}^{(n)} (f_{n^*}(\bar{X}_{t_m}^{(n)}) - f_0(\bar{X}_{t_m}^{(n)})) \right] \\ \le \mathbb{E} \left[ \left| \frac{1}{T} \overline{M}(f_{n^*})_T \right| \right] = \frac{2}{T} \mathbb{E} \left[ \left| \bar{\xi}_{f_{n^*}} \right| (\bar{A}(f_{n^*})_T + \overline{D}_{f_{n^*}})^{\frac{1}{2}} \right] \\ \le \frac{2}{T} \sqrt{\mathbb{E} \left[ \bar{\xi}_{f_{n^*}}^2 \right]} (2\mathbb{E} [\bar{A}(f_{n^*})_T] + \varepsilon'), \quad (4.6) $$

where for every $f \in \mathcal{F}$, we denote by $\bar{\xi}_f := \frac{\overline{M}(f)_T}{2\sqrt{\overline{A}(f)_T + \overline{D}_f}}$ and $\overline{D}_f = \mathbb{E}[\bar{A}(f)_T] + \varepsilon'$ for some $\varepsilon' > 0$. As $\overline{M}(f)$ is a continuous local martingale for every $f \in \mathcal{F}$, it follows from Oga and Koike (2024, Lemma 4.9) that

$$ \mathbb{E} \left[ \frac{\sqrt{\overline{D}_f}}{\sqrt{\overline{A}(f_{n^*})_T + \overline{D}_f}} \exp \left\{ 2 \bar{\xi}_f^2 \right\} \right] \le 1. \quad (4.7) $$

Hence, an argument analogous to Oga and Koike (2024, Lemma 4.10) yields $\mathbb{E}[\bar{\xi}_{f_{n^*}}^2] \le \frac{1}{4} \log 2 + \frac{1}{2} \log \mathfrak{N}_\delta$.

For $\mathbb{E}[\bar{A}(f_{n^*})_T]$, we have

$$ \mathbb{E}[\bar{A}(f_{n^*})_T] = \frac{1}{N} \cdot \mathbb{E} \left[ \frac{1}{N} \sum_{n=1}^{N} \widehat{A}^{(n)}(f_{n^*})_T \right], $$

and an argument analogous to Oga and Koike (2024, Lemma 4.10) yields

$$ \mathbb{E} \left[ \frac{1}{N} \sum_{n=1}^{N} \widehat{A}^{(n)}(f_{n^*})_T \right] \\ \le 2 C_{\sigma}^2 T (\hat{\mathcal{R}}(\hat{f}, f_0) + 4F\delta) + 8F^2 L_{\sigma}^2 C_{\star} T \Delta. $$

Finally, inserting the above bounds into (A.3) yields

$$ \mathbb{E} \left[ \frac{1}{NM} \sum_{n=1}^{N} \sum_{m=0}^{M-1} \Sigma_{t_m}^{(n)} (f_{n^*}(\bar{X}_{t_m}^{(n)}) - f_0(\bar{X}_{t_m}^{(n)})) \right] \\ \le \frac{1}{T} \sqrt{\log 2 + 2 \log \mathfrak{N}_\delta} \\ \cdot \sqrt{\frac{4}{N} \left[ C_\sigma^2 T (\hat{\mathcal{R}}(\hat{f}, f_0) + 4F\delta) + 4F^2 L_\sigma^2 C_\star T \Delta \right] + \varepsilon'}. $$

By letting $\varepsilon' \to 0$ and by applying the AM-GM inequality $\sqrt{x}y \le \frac{\varepsilon}{4}x + \frac{1}{\varepsilon}y$, we get the desired inequality. $\square$

**Proof sketch of Proposition 4.4.** By inserting the results of Lemmas 4.5 and 4.6 into (4.3), we obtain, for an arbitrary function $\hat{f}$ in $\mathcal{F}$, for $\varepsilon \in (0, 1)$,

$$ \hat{\mathcal{R}}(\hat{f}, f_0) \le \frac{\Psi^{\mathcal{F}}(\hat{f})}{1-\varepsilon} + \frac{1+\varepsilon}{1-\varepsilon} \mathcal{R}(\bar{f}, f_0) + \frac{\gamma'_{\varepsilon}}{1-\varepsilon}, $$

where we apply $\hat{\mathcal{R}}(\bar{f}, f_0) = \mathcal{R}(\bar{f}, f_0)$ and

$$ \gamma'_{\varepsilon} = \frac{3L_b^2 C_{\star}}{2\varepsilon} \Delta + \delta \sqrt{2L_{\sigma}^2 C_{\star}} + \frac{2\bar{C}_{\sigma}}{\sqrt{T}} \int_0^{\delta} \sqrt{\log \mathfrak{N}_u} \, du \\ + 2\varepsilon F \delta + 2\varepsilon \frac{F^2 L_{\sigma}^2 C_{\star}}{C_{\sigma}^2} \Delta + \frac{8C_{\sigma}^2}{\varepsilon T N} (\log 2 + 2 \log \mathfrak{N}_{\delta}). $$

By applying Lemma 4.3, we have

$$ \int_0^{\delta} \sqrt{\log \mathfrak{N}_u} \, du \le \int_0^{\delta} \sqrt{C_s(L \log s + \log d)} \, du \\ + \sqrt{C_s} \int_0^{\delta} \sqrt{-\log u} \, du. $$

Using the change of variable $t = -\log u$ and the inequality $\Gamma(\frac{3}{2}, -\log(\delta)) \le \frac{3}{4}\delta\sqrt{-\log\delta}$ valid for $\delta < \exp(-\frac{1}{2})$, we set $\varepsilon = \frac{1}{2}$ and $\delta = \frac{1}{N}$ to obtain

$$ \hat{\mathcal{R}}(\hat{f}, f_0) \le 2\Psi^{\mathcal{F}}(\hat{f}) + 3\mathcal{R}(\bar{f}, f_0) \\ + 2L_{\sigma}^2 C_{\star} (3 + \frac{F^2}{C_{\sigma}^2}) \Delta + C_1 \frac{1}{N} + C_2 \frac{\log N}{N}, $$

with

$$ C_1 = 2\sqrt{2L_{\sigma}^2 C_{\star}} + \frac{4\bar{C}_{\sigma}}{\sqrt{T}} \sqrt{C_s(L \log s + \log d)} + 2F \\ + \frac{32C_{\sigma}^2}{T} (\log 2 + 2C_s(L \log s + \log d)) $$

and $C_2 = \frac{6\bar{C}_{\sigma}\sqrt{C_s}}{\sqrt{T}} + \frac{64C_{\sigma}^2 C_s}{T}$. We conclude by noting that $\mathcal{R}(\bar{f}, f_0) \le \|\hat{f} - f_0\|_{\infty}$, and that the above inequality holds for any arbitrary function $\hat{f} \in \mathcal{F}$. $\square$

**Proof of Theorem 2.1.** Theorem 2.1 follows directly from Propositions 4.1 and 4.4 by setting $\bar{\varepsilon} = \frac{\log N}{N}$. $\square$

**Proof of Corollary 2.3.** Theorem 1 in Schmidt-Hieber (2020) provides an upper bound of $\inf_{f \in \mathcal{F}(L,p,s,F)} \mathcal{R}(f, f_0)$. Corollary 2.3 then follows directly from Theorem 2.1 and Theorem 1 in Schmidt-Hieber (2020). $\square$

Acknowledgments

This work was supported by the French National Research Agency (ANR) under the France 2030 program, reference "ANR-23-EXMA-0011" (MIRTE Project), and originates from the Master's thesis of Yuzhen Zhao, completed during her internship at the LEDa laboratory, Université Paris Dauphine-PSL, under the supervision of Yating Liu.
References

Bae, Y.; Ha, S.; and Jeong, H. 2025. Inferring the Langevin equation with uncertainty via Bayesian neural networks. *Chaos, Solitons & Fractals*, 197: 116440.

Bressloff, P. C. 2014. *Stochastic processes in cell biology*, volume 41 of *Interdisciplinary Applied Mathematics*. Springer, Cham. ISBN 978-3-319-08487-9; 978-3-319-08488-6.

Comte, F.; and Genon-Catalot, V. 2020. Nonparametric drift estimation for i.i.d. paths of stochastic differential equations. *Ann. Statist.*, 48(6): 3336-3365.

Comte, F.; Genon-Catalot, V.; and Rozenholc, Y. 2007. Personalized nonparametric mean square estimation of the coefficients of diffusion processes. *Bernoulli*, 13(2): 514–543.

Dalalyan, A. 2005. Sharp adaptive estimation of the drift function for ergodic diffusions. *Ann. Statist.*, 33(6): 2507–2528.

Denis, C.; Dion-Blanc, C.; and Martinez, M. 2021. A ridge estimator of the drift from discrete repeated observations of the solution of a stochastic differential equation. *Bernoulli*, 27(4): 2675–2713.

Frishman, A.; and Ronceray, P. 2020. Learning Force Fields from Stochastic Trajectories. *Phys. Rev. X*, 10: 021009.

Gao, T.-T.; Barzel, B.; and Yan, G. 2024. Learning interpretable dynamics of stochastic complex systems from experimental data. *Nature communications*, 15(1): 6029.

Gardiner, C. W. 2004. *Handbook of stochastic methods for physics, chemistry and the natural sciences*, volume 13 of Springer Series in Synergetics. Springer-Verlag, Berlin, third edition. ISBN 3-540-20882-8.

Györfi, L.; Kohler, M.; Krzyżak, A.; and Walk, H. 2002. *A distribution-free theory of nonparametric regression*. Springer Series in Statistics. Springer-Verlag, New York. ISBN 0-387-95441-4.

Hoffmann, M. 1999. Adaptive estimation in diffusion processes. *Stochastic Process. Appl.*, 79(1): 135-163.

Karatzas, I.; and Shreve, S. E. 1998. *Methods of mathematical finance*, volume 39 of Applications of Mathematics (New York). Springer-Verlag, New York. ISBN 0-387-94839-2.

Kutoyants, Y. A. 2004. *Statistical inference for ergodic diffusion processes*. Springer Series in Statistics. Springer-Verlag London, Ltd., London. ISBN 1-85233-759-1.

Oga, A.; and Koike, Y. 2024. Drift estimation for a multi-dimensional diffusion process using deep neural networks. *Stochastic Process. Appl.*, 170: Paper No. 104240, 25.

Pinelis, I. 2020. Exact lower and upper bounds on the incomplete gamma function. *Math. Inequal. Appl.*, 23(4): 1261–1278.

Ren, Y.; Lu, Y.; Ying, L.; and Rotskoff, G. M. 2024. Statistical spatially inhomogeneous diffusion inference. In *Proceedings of the AAAI Conference on Artificial Intelligence*, volume 38, 14820–14828.

Schmidt-Hieber, J. 2020. Nonparametric regression using deep neural networks with ReLU activation function. *Ann. Statist.*, 48(4): 1875–1897.

van der Vaart, A. W.; and Wellner, J. A. 2023. *Weak convergence and empirical processes—with applications to statistics*. Springer Series in Statistics. Springer, Cham. ISBN 978-3-031-29038-1; 978-3-031-29040-4. Second edition [of 1385671].

Yarotsky, D. 2017. Error bounds for approximations with deep ReLU networks. *Neural Networks*, 94: 103–114.

Yildiz, C.; Heinonen, M.; Intosalmi, J.; Mannerstrom, H.; and Lahdesmaki, H. 2018. Learning Stochastic Differential Equations with Gaussian Processes without Gradient Matching. In *2018 IEEE 28th International Workshop on Machine Learning for Signal Processing (MLSP)*, 1–6.
A Detailed Proofs

For every $n \in \{1, \dots, N\}$ and every $t \in [0, T]$, define

$$ \mathcal{A}_t^{(n)} := \{ \bar{X}_t^{(n)} \in [0, 1]^d \}. $$

Moreover, for every $n \in \{1, \dots, N\}$ and $m \in \{0, \dots, M-1\}$, define

$$ \bar{\Sigma}_{t_m}^{(n)} = \frac{1}{\Delta} \int_{t_m}^{t_{m+1}} \sigma^i(\bar{X}_{t_m}^{(n)}) dB_s^{(n)} \quad \text{and} \quad \tilde{\Sigma}_{t_m}^{(n)} = \frac{1}{\Delta} \int_{t_m}^{t_{m+1}} (\sigma^i(\bar{X}_s^{(n)}) - \sigma^i(\bar{X}_{t_m}^{(n)})) dB_s^{(n)}. $$

It is obvious that $\Sigma_{t_m}^{(n)} = \bar{\Sigma}_{t_m}^{(n)} + \tilde{\Sigma}_{t_m}^{(n)}$. Recall that $I_{t_m}^{(n)} = \frac{1}{\Delta} \int_{t_m}^{t_{m+1}} (b^i(\bar{X}_s^{(n)}) - b^i(\bar{X}_{t_m}^{(n)})) ds$ as given in equation (4.2), and the constant $C_* = 4(C_b^2 + C_\sigma^2) \exp(4(L_b^2 + L_\sigma^2))$ defined in Lemma 4.5. We also abbreviate $\mathcal{F}$ for the neural network function class $\mathcal{F}(L, \mathbf{p}, s, F)$, and $\mathfrak{N}_\delta$ for the covering number $\mathfrak{N}(\delta, \mathcal{F}(L, \mathbf{p}, s, F), \|\cdot\|_\infty)$. The following lemma is a direct consequence of Oga and Koike (2024, Lemma 4.6, Corollary 4.1, and Corollary 4.2).

**Lemma A.1.** Suppose that Assumptions 1 and 2 hold. For every $n \in \{1, \dots, N\}$, $m \in \{0, \dots, M-1\}$, and $s \in [t_m, t_{m+1}]$, we have

$$ 1. \quad \mathbb{E} [ |\bar{X}_s^{(n)} - \bar{X}_{t_m}^{(n)}|^2 \mathbf{1}_{A_{t_m}^{(n)}} ] \le C_* (s - t_m), $$

$$ 2. \quad \mathbb{E} [ (I_{t_m}^{(n)})^2 \mathbf{1}_{A_{t_m}^{(n)}} ] \le \frac{1}{2} L_b^2 C_* \Delta, $$

$$ 3. \quad \mathbb{E} [ (\tilde{\Sigma}_{t_m}^{(n)})^2 \mathbf{1}_{A_{t_m}^{(n)}} ] \le \frac{1}{2} L_\sigma^2 C_* . $$

We now present the proof of Lemma 4.5.

**Proof of Lemma 4.5.** 1. Consider two arbitrary functions $f_1, f_2 \in \mathcal{F}(L, \mathbf{p}, s, F)$. Then, since $f_1, f_2$ have support in $[0, 1]^d$, we have, for every $c > 0$,

$$ \begin{aligned} \mathbb{E} \left[ \frac{1}{NM} \sum_{n=1}^{N} \sum_{m=0}^{M-1} I_{t_m}^{(n)} (f_1(\bar{X}_{t_m}^{(n)}) - f_2(\bar{X}_{t_m}^{(n)})) \right] &= \mathbb{E} \left[ \frac{1}{NM} \sum_{n=1}^{N} \sum_{m=0}^{M-1} I_{t_m}^{(n)} \mathbf{1}_{A_{t_m}^{(n)}} (f_1(\bar{X}_{t_m}^{(n)}) - f_2(\bar{X}_{t_m}^{(n)})) \right] \\ &\le \frac{1}{NM} \sum_{n=1}^{N} \sum_{m=0}^{M-1} \left\{ \mathbb{E} \left[ \frac{1}{2c} (I_{t_m}^{(n)})^2 \mathbf{1}_{A_{t_m}^{(n)}} \right] + \mathbb{E} \left[ \frac{c}{2} (f_1(\bar{X}_{t_m}^{(n)}) - f_2(\bar{X}_{t_m}^{(n)}))^2 \right] \right\}, \end{aligned} \quad (A.1) $$

where the last inequality comes from the AM-GM inequality $ab \le \frac{a^2}{2c} + \frac{cb^2}{2}$ for every $a, b \in \mathbb{R}_+$. For the first term of (A.1), it follows that

$$ \begin{aligned} \frac{1}{NM} \sum_{n=1}^{N} \sum_{m=0}^{M-1} \mathbb{E} \left[ \frac{1}{2c} (I_{t_m}^{(n)})^2 \mathbf{1}_{A_{t_m}^{(n)}} \right] &= \frac{1}{NM} \frac{1}{2c} \sum_{n=1}^{N} \sum_{m=0}^{M-1} \mathbb{E} \left[ \frac{1}{\Delta^2} \left( \int_{t_m}^{t_{m+1}} [b^i(\bar{X}_s^{(n)}) - b^i(\bar{X}_{t_m}^{(n)})] ds \right)^2 \mathbf{1}_{A_{t_m}^{(n)}} \right] \\ &\le \frac{1}{NM} \frac{1}{2c} \sum_{n=1}^{N} \sum_{m=0}^{M-1} \mathbb{E} \left[ \frac{L_b^2}{\Delta^2} \left( \int_{t_m}^{t_{m+1}} |\bar{X}_s^{(n)} - \bar{X}_{t_m}^{(n)}| \mathbf{1}_{A_{t_m}^{(n)}} ds \right)^2 \right] \\ &\le \frac{1}{NM} \frac{L_b^2}{\Delta} \sum_{n=1}^{N} \sum_{m=0}^{M-1} \mathbb{E} \left[ \int_{t_m}^{t_{m+1}} |\bar{X}_s^{(n)} - \bar{X}_{t_m}^{(n)}|^2 \mathbf{1}_{A_{t_m}^{(n)}} ds \right] \\ &\le \frac{1}{NM} \frac{L_b^2}{\Delta} \sum_{n=1}^{N} \sum_{m=0}^{M-1} \int_{t_m}^{t_{m+1}} C_* (s - t_m) ds = \frac{1}{2c} \frac{L_b^2 C_*}{2} \Delta, \end{aligned} $$

where the first inequality follows from Assumption 1, the second uses the Cauchy-Schwarz inequality, the third is a direct application of Fubini's theorem, and the fourth follows from Lemma A.1.

For the second term of (A.1), we have

$$ \frac{1}{NM} \sum_{n=1}^{N} \sum_{m=0}^{M-1} \mathbb{E} \left[ \frac{c}{2} (f_1(\bar{X}_{t_m}^{(n)}) - f_2(\bar{X}_{t_m}^{(n)}))^2 \right] = \frac{c}{2} \mathbb{E} \left[ \frac{1}{NM} \sum_{n=1}^{N} \sum_{m=0}^{M-1} (f_1(\bar{X}_{t_m}^{(n)}) - f_2(\bar{X}_{t_m}^{(n)}))^2 \right] = \frac{c}{2} \hat{\mathcal{R}}(f_1, f_2). $$
Consequently,

$$ \mathbb{E} \left[ \frac{1}{NM} \sum_{n=1}^{N} \sum_{m=0}^{M-1} I_{t_m}^{(n)} (f_1(\bar{X}_{t_m}^{(n)}) - f_2(\bar{X}_{t_m}^{(n)})) \right] \leq \frac{c}{2} \hat{R}(f_1, f_2) + \frac{L_b^2 C_*}{2} \Delta. \quad (A.2) $$

Finally, by setting $(f_1, f_2, c)$ respectively to $(\hat{f}, f_0, \frac{\varepsilon}{2})$ $(\bar{f}, f_0, \varepsilon)$, and applying the triangle inequality, we obtain (4.4).

2. It follows from Lemma 4.11 and 4.12 of Oga and Koike (2024) that for every fixed $n \in \{1, ..., N\}$,

$$ \mathbb{E} \left[ \left| \frac{1}{M} \sum_{m=0}^{M-1} \tilde{\Sigma}_{t_m}^{(n)} (\hat{f} - f_{n^*}) (\bar{X}_{t_m}^{(n)}) \right| \right] \leq \delta \sqrt{\frac{L_b^2 C_*}{2}} \quad \text{and} \quad \mathbb{E} \left[ \left| \frac{1}{M} \sum_{m=0}^{M-1} \tilde{\Sigma}_{t_m}^{(n)} (\hat{f} - f_{n^*}) (\bar{X}_{t_m}^{(n)}) \right| \right] \leq \frac{\bar{C}_\sigma}{\sqrt{T}} \int_0^\delta \sqrt{\log \mathfrak{N}_u} \, du, $$

which implies directly (4.5) since $\Sigma_{t_m}^{(n)} = \bar{\Sigma}_{t_m}^{(n)} + \tilde{\Sigma}_{t_m}^{(n)}$.

Next, we provide the detailed proofs of Lemma 4.6 and Proposition 4.4.

*_Detailed proof of Lemma 4.6._* For a fixed function $f \in \mathcal{F}$ and for a fixed $n \in \{1, ..., N\}$, we define the processes $\hat{M}^{(n)}(f) = (\hat{M}^{(n)}(f)_s)_{s \in [0,T]}$, $\hat{A}^{(n)}(f) = (\hat{A}^{(n)}(f)_s)_{s \in [0,T]}$, $\bar{M}(f) = (\bar{M}(f)_s)_{s \in [0,T]}$ and $\bar{A}(f) = (\bar{A}(f)_s)_{s \in [0,T]}$ by

$$ \hat{M}^{(n)}(f)_s := \sum_{m=0}^{M-1} (f - f_0)(\bar{X}_{t_m}^{(n)}) \int_{s \wedge t_m}^{s \wedge t_{m+1}} \sigma^i(\bar{X}_u^{(n)}) dB_u^{(n)}, $$

$$ \bar{M}(f)_s := \frac{1}{N} \sum_{n=1}^{N} \hat{M}^{(n)}(f) = \frac{1}{N} \sum_{n=1}^{N} \sum_{m=0}^{M-1} (f - f_0)(\bar{X}_{t_m}^{(n)}) \int_{s \wedge t_m}^{s \wedge t_{m+1}} \sigma^i(\bar{X}_u^{(n)}) dB_u^{(n)}, $$

$$ \hat{A}^{(n)}(f)_s := \langle \hat{M}^{(n)}(f) \rangle_s = \sum_{m=0}^{M-1} \left( (f - f_0)(\bar{X}_{t_m}^{(n)}) \right)^2 \int_{s \wedge t_m}^{s \wedge t_{m+1}} \left| \sigma^i(\bar{X}_u^{(n)}) \right|^2 du, $$

$$ \bar{A}(f)_s := \langle \bar{M}(f) \rangle_s = \frac{1}{N^2} \sum_{n=1}^{N} \hat{A}^{(n)}(f)_s. $$

It follows that

$$ \begin{aligned} \mathbb{E} \left[ \frac{1}{NM} \sum_{n=1}^{N} \sum_{m=0}^{M-1} \Sigma_{t_m}^{(n)} (f_{n^*}(\bar{X}_{t_m}^{(n)}) - f_0(\bar{X}_{t_m}^{(n)})) \right] &\leq \mathbb{E} \left[ \left| \frac{1}{T} \overline{M}(f_{n^*})_T \right| \right] = \frac{2}{T} \mathbb{E} \left[ \left| \bar{\xi}_{f_{n^*}} \right| (\bar{A}(f_{n^*})_T + \overline{D}_{f_{n^*}})^{\frac{1}{2}} \right] \\ &\leq \frac{2}{T} \sqrt{\mathbb{E} \left[ \bar{\xi}_{f_{n^*}}^2 \right]} (2\mathbb{E} \left[ \bar{A}(f_{n^*})_T \right] + \varepsilon'), \end{aligned} \quad (A.3) $$

where for every $f \in \mathcal{F}$, we denote by $\bar{\xi}_f := \frac{\bar{M}(f)_T}{2\sqrt{\bar{A}(f)_T + \overline{D}_f}}$ and $\overline{D}_f = \mathbb{E}[\bar{A}(f)_T] + \varepsilon'$ for some $\varepsilon' > 0$.

For $\mathbb{E}[\bar{\xi}_f^2]$, as $\bar{M}(f)$ is a continuous local martingale for every $f \in \mathcal{F}$, it follows from (Oga and Koike 2024, Lemma 4.9) that

$$ \mathbb{E} \left[ \frac{\sqrt{\overline{D}_f}}{\sqrt{\bar{A}(f_n)_T + \overline{D}_f}} \exp \left\{ 2 \bar{\xi}_f^2 \right\} \right] \leq 1. \quad (A.4) $$

Moreover,

$$ \mathbb{E} \left[ \exp \left( \bar{\xi}_{f_{n^*}}^2 \right) \right] \leq \sqrt{\mathbb{E} \left[ \frac{\sqrt{\overline{D}_{f_{n^*}}}}{\sqrt{\bar{A}(f_{n^*})_T + \overline{D}_{f_{n^*}}}} \exp \left\{ 2 \bar{\xi}_{f_{n^*}}^2 \right\} \right]} \mathbb{E} \left[ \frac{\sqrt{\overline{A}(f_{n^*})_T + \overline{D}_{f_{n^*}}}}{\sqrt{\overline{D}_{f_{n^*}}}} \right]. $$

Since

$$ \mathbb{E} \left[ \frac{\sqrt{\overline{D}_{f_{n^*}}}}{\sqrt{\bar{A}(f_{n^*})_T + \overline{D}_{f_{n^*}}}} \exp \left\{ 2 \bar{\xi}_{f_{n^*}}^2 \right\} \right] \leq \mathbb{E} \left[ \max_{1 \leq n \leq \mathfrak{N}(\delta, \mathcal{F}, \|\cdot\|_\infty)} \frac{\sqrt{\overline{D}_{f_n}}}{\sqrt{\bar{A}(f_n)_T + \overline{D}_{f_n}}} \exp \left\{ 2 \bar{\xi}_{f_n}^2 \right\} \right] \leq \mathfrak{N}(\delta, \mathcal{F}, \|\cdot\|_\infty), $$
and

$$ \mathbb{E} \left[ \frac{\sqrt{\bar{A}(f_{n^*})_T + \bar{D}_{f_{n^*}}}}{\sqrt{\bar{D}_{f_{n^*}}}} \right] \le \sqrt{\mathbb{E} \left[ \frac{\bar{A}(f_{n^*})_T + \bar{D}_{f_{n^*}}}{\bar{D}_{f_{n^*}}} \right]} \le \sqrt{2}, \quad (A.5) $$

we obtain

$$ \mathbb{E} \left[ \exp \left( \bar{\xi}_{f_{n^*}}^2 \right) \right] \le 2^{\frac{1}{4}} \sqrt{\mathfrak{N}(\delta, \mathcal{F}, \| \cdot \|_{\infty})}. \quad (A.6) $$

Hence, the Jensen inequality yields

$$ \mathbb{E} \left[ \bar{\xi}_{f_{n^*}}^2 \right] \le \log \mathbb{E} \left[ \exp \left( \bar{\xi}_{f_{n^*}}^2 \right) \right] \le \frac{1}{2} \log 2 + \frac{1}{2} \mathfrak{N}(\delta, \mathcal{F}, \| \cdot \|_{\infty}). \quad (A.7) $$

For $\mathbb{E}[\bar{A}(f_{n^*})_T]$, we have

$$ \mathbb{E} [\bar{A}(f_{n^*})_T] = \mathbb{E} \left[ \frac{1}{N^2} \sum_{n=1}^{N} \hat{A}^{(n)}(f_{n^*})_T \right] = \frac{1}{N} \cdot \mathbb{E} \left[ \frac{1}{N} \sum_{n=1}^{N} \hat{A}^{(n)}(f_{n^*})_T \right] \quad (A.8) $$

and

$$ \begin{aligned} & \mathbb{E} \left[ \frac{1}{N} \sum_{n=1}^{N} \hat{A}^{(n)}(f_{n^*}) \right] \\ &= \mathbb{E} \left[ \frac{1}{N} \sum_{n=1}^{N} \sum_{m=0}^{M-1} \left( (f_{n^*} - f_0)(\bar{X}_{t_m}^{(n)}) \right)^2 \int_{t_m}^{t_{m+1}} \left| \sigma^i(\bar{X}_u^{(n)}) \right|^2 du \right] \\ &\le \mathbb{E} \left[ \frac{1}{N} \sum_{n=1}^{N} \sum_{m=0}^{M-1} \left( (f_{n^*} - f_0)(\bar{X}_{t_m}^{(n)}) \right)^2 \int_{t_m}^{t_{m+1}} \left[ 2 \left| \sigma^i(\bar{X}_{t_m}^{(n)}) \right|^2 + 2 \left| \sigma^i(\bar{X}_u^{(n)}) - \sigma^i(\bar{X}_{t_m}^{(n)}) \right|^2 \right] du \right] \\ &\le \mathbb{E} \left[ \frac{1}{N} \sum_{n=1}^{N} \sum_{m=0}^{M-1} \left( (f_{n^*} - f_0)(\bar{X}_{t_m}^{(n)}) \right)^2 \int_{t_m}^{t_{m+1}} \left[ 2 \left| \sigma^i(\bar{X}_{t_m}^{(n)}) \right|^2 \right] du \right] \\ &\quad + \mathbb{E} \left[ \frac{1}{N} \sum_{n=1}^{N} \sum_{m=0}^{M-1} \left( (f_{n^*} - f_0)(\bar{X}_{t_m}^{(n)}) \right)^2 \int_{t_m}^{t_{m+1}} \left[ 2 \left| \sigma^i(\bar{X}_u^{(n)}) - \sigma^i(\bar{X}_{t_m}^{(n)}) \right|^2 \right] du \right] \\ &=: (I) + (II). \end{aligned} $$

Since $f_{n^*}$ and $f_0$ have support in $[0, 1]^d$, we have for (I)

$$ (I) \le 2C_{\sigma}^2 T \mathbb{E} \left[ \frac{1}{NM} \sum_{n=1}^{N} \sum_{m=0}^{M-1} \left( (f_{n^*} - f_0)(\bar{X}_{t_m}^{(n)}) \right)^2 \right] = 2C_{\sigma}^2 T \hat{\mathcal{R}}(f_{n^*}, f_0), $$

and

$$ \begin{aligned} \hat{\mathcal{R}}(f_{n^*}, f_0) - \hat{\mathcal{R}}(\hat{f}, f_0) &= \mathbb{E} \left[ \frac{1}{NM} \sum_{n=1}^{N} \sum_{m=0}^{M-1} \left[ (\hat{f}_{n^*}(\bar{X}_{t_m}^{(n)}) - f_0(\bar{X}_{t_m}^{(n)}))^2 - \hat{f}(\bar{X}_{t_m}^{(n)}) - f_0(\bar{X}_{t_m}^{(n)}) \right]^2 \right] \\ &\le \mathbb{E} \left[ \frac{1}{NM} \sum_{n=1}^{N} \sum_{m=0}^{M-1} \left[ (\hat{f}_{n^*}(\bar{X}_{t_m}^{(n)}) - \hat{f}(\bar{X}_{t_m}^{(n)})) (\hat{f}_{n^*}(\bar{X}_{t_m}^{(n)}) + \hat{f}(\bar{X}_{t_m}^{(n)})) - 2f_0(\bar{X}_{t_m}^{(n)}) \right]^2 \right] \le \delta(2F + 2C_b) \le 4F\delta. \end{aligned} $$

For (II), we have

$$ \begin{aligned} (II) &\le 8F^2 \mathbb{E} \left[ \sum_{m=0}^{M-1} \int_{t_m}^{t_{m+1}} \left| \sigma^i(\bar{X}_u^{(1)}) - \sigma^i(\bar{X}_{t_m}^{(1)}) \right|^2 \mathbf{1}_{A_{t_m}^{(1)}} du \right] \le 8F^2 \mathbb{E} \left[ \sum_{m=0}^{M-1} \int_{t_m}^{t_{m+1}} L_{\sigma}^2 \left| \bar{X}_u^{(1)} - \bar{X}_{t_m}^{(1)} \right|^2 \mathbf{1}_{A_{t_m}^{(1)}} du \right] \\ &\le 8F^2 L_{\sigma}^2 \sum_{m=0}^{M-1} \int_{t_m}^{t_{m+1}} \mathbb{E} \left[ \left| \bar{X}_u^{(1)} - \bar{X}_{t_m}^{(1)} \right|^2 \mathbf{1}_{A_{t_m}^{(1)}} \right] du \le 8F^2 L_{\sigma}^2 C_* T \Delta, \end{aligned} $$
where the last inequality follows from Lemma A.1. Consequently,

$$ \mathbb{E} \left[ \frac{1}{N} \sum_{n=1}^{N} \hat{A}^{(n)}(f_{n^*})_T \right] \leq 2C_{\sigma}^2 T (\hat{\mathcal{R}}(\hat{f}, f_0) + 4F\delta) + 8F^2 L_{\sigma}^2 C_{\star} T \Delta. $$

Finally, inserting the above bounds into (A.3) yields

$$ \begin{aligned} & \mathbb{E} \left[ \frac{1}{NM} \sum_{n=1}^{N} \sum_{m=0}^{M-1} \sum_{t_m}^{n} (f_{n^*}(\bar{X}_{t_m}^{(n)}) - f_0(\bar{X}_{t_m}^{(n)})) \right] \\ & \leq \frac{1}{T} \sqrt{\log 2 + 2 \log \mathfrak{N}_{\delta}} \cdot \sqrt{\frac{4}{N} \left[ C_{\sigma}^2 T (\hat{\mathcal{R}}(\hat{f}, f_0) + 4F\delta) + 4F^2 L_{\sigma}^2 C_{\star} T \Delta \right]} + \varepsilon'. \end{aligned} $$

By letting $\varepsilon' \to 0$ and by applying the AM-GM inequality $\sqrt{xy} \leq \frac{\varepsilon}{4}x + \frac{1}{\varepsilon}y$ for some $\varepsilon \in (0, 1)$, we get the desired inequality. □

*Detailed proof of Proposition 4.4.* By inserting the results of Lemmas 4.5 and 4.6 into (4.3), we obtain, for an arbitrary function $\bar{f}$ in $\mathcal{F}$, for $\varepsilon \in (0, 1)$,

$$ \hat{\mathcal{R}}(\hat{f}, f_0) \leq \frac{\Psi^{\mathcal{F}}(\hat{f})}{1 - \varepsilon} + \frac{1 + \varepsilon}{1 - \varepsilon} \mathcal{R}(\bar{f}, f_0) + \frac{\gamma'_{\varepsilon}}{1 - \varepsilon}, $$

where we apply $\hat{\mathcal{R}}(\bar{f}, f_0) = \mathcal{R}(\bar{f}, f_0)$ and

$$ \gamma'_{\varepsilon} = \frac{3L_b^2 C_{\star}}{2\varepsilon} \Delta + \delta \sqrt{2L_{\sigma}^2 C_{\star}} + \frac{2\bar{C}_{\sigma}}{\sqrt{T}} \int_0^{\delta} \sqrt{\log \mathfrak{N}_u} du + 2\varepsilon F \delta + 2\varepsilon \frac{F^2 L_{\sigma}^2 C_{\star}}{C_{\sigma}^2} \Delta + \frac{8C_{\sigma}^2}{\varepsilon T N} (\log 2 + 2 \log \mathfrak{N}_{\delta}). $$

By applying Lemma 4.3, we have

$$ \int_{0}^{\delta} \sqrt{\log \mathfrak{N}_u} du \leq \int_{0}^{\delta} \sqrt{Cs(L \log s + \log d)} du + \sqrt{Cs} \int_{0}^{\delta} \sqrt{-\log u} du. $$

After a change of variable $t = -\log u$, we obtain

$$ \int_{0}^{\delta} \sqrt{-\log u} du = \Gamma(\frac{3}{2}, -\log(\delta)), $$

where $\Gamma(s, x)$ denotes the upper incomplete gamma function. Therefore, for $\delta < \exp(-\frac{1}{2})$, it follows from Pinelis (2020, Proposition 2.7) that $\Gamma(\frac{3}{2}, -\log(\delta)) \leq \frac{3}{4}\delta\sqrt{-\log\delta}$.

Finally, by considering $\varepsilon = \frac{1}{2}$ and $\delta = \frac{1}{N}$, we obtain

$$ \hat{\mathcal{R}}(\hat{f}, f_0) \leq 2\Psi^{\mathcal{F}}(\hat{f}) + 3\mathcal{R}(\bar{f}, f_0) + 2L_{\sigma}^2 C_{\star} \left(3 + \frac{F^2}{C_{\sigma}^2}\right)\Delta + C_1 \frac{1}{N} + C_2 \frac{\log N}{N}, $$

where

$$ C_1 = 2\sqrt{2L_{\sigma}^2 C_{\star}} + \frac{4\bar{C}_{\sigma}}{\sqrt{T}} \sqrt{Cs(L \log s + \log d)} + 2F + \frac{32C_{\sigma}^2}{T} (\log 2 + 2Cs(L \log s + \log d)) $$

and $C_2 = \frac{6\bar{C}_{\sigma}\sqrt{Cs}}{\sqrt{T}} + \frac{64C_{\sigma}^2Cs}{T}$. We conclude by noting that $\mathcal{R}(\bar{f}, f_0) \leq \|\bar{f} - f_0\|_{\infty}$, and observing that the above inequality holds for any arbitrary function $\bar{f} \in \mathcal{F}$. □

# B Experimental and Computational Details

## Data Generation

We simulate sample paths of (1.1) using the Euler-Maruyama discretization scheme. The time horizon is set to $T = 1$ and is discretized into $M = 100$ time steps, with the time step $\Delta = T/M = 0.01$.

For each experiment, we generate $N$ independent paths $\mathcal{D}_N := \{\bar{X}^{(n)}\}_{n=1}^N$ as training set and an $N' = 1000$ independent paths $\mathcal{D}_{\text{test}} := \{\bar{X}^{(n)}\}_{n=1}^{N'}$ for evaluation. Hyperparameter tuning is conducted using a separate independent validation set $\mathcal{D}_{\text{valid}} := \{\tilde{X}^{(n)}\}_{n=1}^{1000}$.
## Hyperparameter Tuning

As the empirical performance may vary across neural network architectures and training settings, we perform hyperparameter tuning for each combination of $(N, d)$ on $D_{valid}$ to ensure comparability of estimation errors across different combinations of $(N, d)$. Specifically, the tuning is performed over the following candidate sets:

* network architectures: $\mathbf{p} \in \{(d, 16, 16, 1), (d, 16, 32, 16, 1), (d, 16, 32, 32, 16, 1)\}$,

* percentage of non-zero parameters in total $s_{ratio}: \{0.25, 0.5, 0.75\}$.

The sparsity constraint $s$ used in the estimator class $\mathcal{F}(L, \mathbf{p}, s, F)$ is computed as

$$s = \left\lfloor s_{\text{per}} \cdot \sum_{i=0}^{L} (p_i + 1) \cdot p_{i+1} \right\rfloor,$$

where $L$ is the number of hidden layers and $(p_0, \dots, p_{L+1})$ is the layer width vector.

## Training Procedure

The network is trained using the Adam optimizer with learning rate of 0.001 and batch size of 256. At each epoch, the training samples are shuffled and fed to the model in mini-batches to update the network parameters. Following each optimizer step, we enforce the desired sparsity level by retaining only the top-$s$ parameters (in magnitude) and setting all remaining weights to zero. We then clip all parameters to lie within the range $[-1, 1]$.

After every epoch, the model is evaluated on the validation set to compute the validation error. Training is performed for up to 200 epochs, and the final number of epochs is selected via early stopping, which is triggered when the validation error fails to improve for 20 consecutive epochs.

## Evaluation

To account for the randomness in data generation and optimization process, for each pair $(N, d)$, we repeat the experiment 50 times with different random seeds, resulting in a series of different estimators and test set $((\hat{f}_{N,d}^{(j)}), D_{\text{test}}^{(j)})_{j=1}^{50}$. Let $E_{N,d}^{(j)}$ denote the generalization error computed in the $j$-th run with training size $N$ and dimension $d$. It is defined as

$$E_{N,d}^{(j)} := \tilde{\mathcal{R}}_{N}(\hat{f}_{N,d}^{(j)}, b_1),$$

where the error is evaluated on the test dataset $D_{\text{test}}^{(j)}$. For each $(N, d)$, we compute:

$$\bar{E}_{N,d} := \frac{1}{50} \sum_{j=1}^{50} E_{N,d}^{(j)}, \quad E_{N,d}^{\text{lower}} := \bar{E}_{N,d} - t_{0.975}^{(49)} \cdot \tau_{N,d}, \quad E_{N,d}^{\text{upper}} := \bar{E}_{N,d} + t_{0.975}^{(49)} \cdot \tau_{N,d}. \qquad (A.1)$$

where $\tau_{N,d} = \frac{1}{\sqrt{50}} (\frac{1}{49} \sum_{j=1}^{50} (E_{N,d}^{(j)} - \bar{E}_{N,d})^2)^{1/2}$ and $t_{0.975}^{(49)}$ is the 97.5 percentile of the Student's $t$ distribution with 49 degrees of freedom.

Log-log plots are used to present the average empirical error $\bar{E}_{N,d}$ and interval $[E_{N,d}^{\text{lower}}, E_{N,d}^{\text{upper}}]$ with respect to training size $N$.

## Memory Cost Discussion

The B-spline estimator involves constructing the matrix $\mathbf{B} \in \mathbb{R}^{NM \times p}$, where $p = (K_N + 3)^d$ denotes the total number of basis functions, and $K_N$ is the number of interior knots per coordinate. The memory cost is thus dominated by the storage of the matrix $\mathbf{B}$:

$$\text{Memory}_{\text{B-spline}} \sim O(NMp) = O(NM(K_N + 3)^d).$$

As $d$ increases, the number of basis functions $p$ grows exponentially, resulting in an exponential blow-up of memory cost. For instance, even with moderate values of $K_N$ (e.g., $K_N = 8$), when $d = 5$, we have

$$p = (8 + 3)^5 = 161051,$$

leading to a matrix $\mathbf{B}$ of size $(100N) \times 161051$ entries. Assuming each entry is stored as a 64-bit float (8 bytes), the total memory required is

$$\text{Memory}_{\text{B-spline}} \approx 100N \times 161051 \times 8 \text{ bytes.}$$

In particular, for $d = 5$, the required memory far exceeds the computational capacity of a standard laptop (Apple M2 chip, 16 GB RAM).
In contrast, the neural network estimator requires storing only the network parameters and minibatch data during training. The memory cost scales as

$$ \text{Memory}_{\text{NN}} \sim O \left( \sum_{i=1}^{L} (p_{i-1} + 1)p_i + \text{batch\_size} \times d \right), $$

where $p_i$ is the layer width and $d$ is the input dimension.

In particular, for $d = 5$, under the architecture $(d, 16, 32, 16, 1)$, with batch size 256, the total number of parameters is 1185, yielding under 64-bit float (8 bytes):

$$ \text{Memory}_{\text{NN}} \approx (1185 + 256 \times 5) \times 8 \text{ bytes.} $$

Thus, the overall memory required is approximately 19 KB. Crucially, the memory requirement of the neural network estimator grows linearly with $d$ and remains independent of the sample size $N$, making it more scalable for high-dimensional settings compared to the B-spline estimator.