Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: The paper proposes an extension of the Neural SDE model proposed by [Li et al. (AISTATS 2020)](https://proceedings.mlr.press/v108/li20i.html). This more general model is obtained by replacing the Brownian motion driving the NSDE with a Riemann–Liouville fractional Brownian motion (R–L fBM) that has a learnt time-varying Hurst function. To improve the scalability of this approach, the authors employ techniques from sparse Gaussian processes to reduce the computational cost of sampling the R–L fBM. Moreover, their approach of using inducing points allows the resulting model to be expressed as a “standard” Neural SDE, but with random drift and diffusion vector fields. A convergence analysis using rough path theory is presented to support this approximation. The paper concludes with a variety of numerical experiments, showing that the proposed model performs comparably to the existing approach of [Li et al. (2020)](https://proceedings.mlr.press/v108/li20i.html) and can recover Hurst functions reasonably well from noisy data. In addition, the authors show their approach can improve score-based generative modelling with SDEs.
strengths_and_weaknesses: Firstly, I would like to acknowledge the timing of this review (I was asked to be review this paper on 08/08/2022). Whilst I will include my questions/comments, I do not expect the authors to be able to respond to them.

Overall, I am positive about this paper and believe that learning more general noises is a sensible way of trying to improve NSDE models. The authors focus on learning fractional noises, which I believe constitutes a helpful first step in this direction.

Strengths

* I found the methodology presented in the paper understandable and rather interesting. Although I believe that increasing hidden state can allow "standard" NSDEs to handle real-world data, it does make sense to also consider driving processes other than Brownian motion.

* Using techniques from sparse Gaussian processes to sample R–L fBM is interesting in its own right. I have not seen this approach used in the fBM literature. In addition, the authors present an error analysis based on rough path theory – which is believable (though I have not thoroughly checked the details).

* The numerical experiments appear to be of a good quality. At the very least, they show that the proposed approach can give comparable performance to that of  [Li et al. (2020)](https://proceedings.mlr.press/v108/li20i.html). I found the application to score-based generative modelling on MNIST and FashionMNIST particularly interesting.

* The paper opens up a variety of future research directions, such as driving NSDEs by GPs or R–L fBMs with more general kernels. I also imagine that using different noises could have nice applications to score-based models.

* The paper is well structured and is good at conveying the key ideas.

Weaknesses

* I find the title a little misleading. Since the methodology imposes a Hurst function on the noise, I feel that "Learning Fractional Noises in Neural Stochastic Differential Equations" is more appropriate.

* The paper does not mention the term "Stochastic Volterra Equations" (SVE). Whilst the formulation with f(s,t) is more general, the methodology tested in the experiments can be viewed as a Neural SVE. One recent paper from the SVE literature that may be worth citing is [Bayer and Breneis (2021)](https://arxiv.org/abs/2108.05048).

* I believe aspects of Section 4 can be better presented (see questions section).

* It is not clear whether the proposed methodology performs better than the Latent SDE model of [Li et al. (2020)](https://proceedings.mlr.press/v108/li20i.html). However, I find this more than compensated by the interesting experiments on MNIST and FashionMNIST.

* The results on the real-world dataset against  the Latent SDE and ODE models are important and should be displayed in the main body. As an aside, I do not believe root mean square error is a sensible metric for generative models such as Neural SDEs.

* The results on FashionMNIST should be included in the main body of the paper.

* I find details on the architectures in the appendix lacking, particularly for the experiments on financial data. I think it is reasonable to expect such information to be in the appendix (such as in [Li et al (2020)](https://arxiv.org/abs/2001.01328) and [Kidger et al. (2021)](https://arxiv.org/abs/2102.03657))

* There are a number of grammatical errors (see minor suggestions in the questions section)
questions: Questions

* I believe aspects of Section 4 can be better presented.

For example, what is the "true" value of $k(t,s)$ in equation (8)? By Itô's Isometry, I believe it should be
$k_{\text{true}}(t,s) = E[B_t^\prime B_s^\prime] = \frac{1}{(\Delta t)^2}\int_0^{\min(s,t)} f(s, u) f(t, u) du\hspace{2.5mm}(\ast)$.

One can then apply any form of numerical integration to estimate $k_{\text{true}}(s,t)$, such as trapezoidal rule. Am I correct in thinking that applying the rectangle rule to $(\ast)$ is equivalent to what is done in the paper?

As a consequence, I do not believe the discussion on integral approximation in section 4.2 is that important. From what I understand, only $k(s,t)$ is needed to sample noise in the proposed model.

* Why is $B_t^\prime = \frac{\Delta B_t}{\Delta t}$ considered more numerically stable than the natural normalization  $B_t^\prime = \frac{\Delta B_t}{\sqrt{\Delta t}}$?

* I find details on the architectures in the appendix lacking, particularly for the experiments on financial data. How are the vector fields set up? (number of layers, choice of optimizer, etc) Are gradients obtained by optimize-then-discretize or discretize-then-optimize?

* In Theorem 1, does $L$ depend on $\varrho$? If so, the statement should fix $\varrho$ first.
* In the proof of Theorem 1, is $B_t$ a standard fractional Brownian or a Riemann-Liouville fractional Brownian motion?

For example, regarding the statements "As $B_t$ is a fractional Brownian motion" (line 726) and $\mathbb{E}[(B_t-B_{t+\tau})^2] = |\tau|^{2H}$ in the discussion. Generally speaking, I think it would be helpful to always distinguish between fBM and R–L fBM.


Minor suggestions

* "methods and sparse Gaussian process" → "methods and sparse Gaussian processes" (line 10)
* "Recent work on NODEs open a new direction" → "Recent work on NODEs opens a new direction" (line 21)
* "Extensions to SDEs is proposed to handling" → "Extensions of SDEs are proposed for handling" (line 22)
* "has been emphasized in a number of study" → "has been emphasized in a number of studies" (line 26)
* "expressiveness from representation powers" → "expressiveness from the representation powers" (line 30)
* "during solving SDEs" → "during SDE solves" (line 43)
* "using numerical method in solving DEs to perform learning parameters in neural network" → "using numerical methods in solving DEs to learn parameters in neural networks" (lines 57-58)
* "There are a number of follow-up work aiming to improvements in different aspects" → "There are a number of follow-up works aiming to improve different aspects" (line 59)
* "rough path" → "rough paths" (line 68)
* "between Gaussian process and" → "between Gaussian processes and" (line 70)
* "a function $t$" → "a function of $t$" (line 89)
* I am not sure the statement "If $h(t)$ changes over time, this means all such characteristics happen locally" can be made rigorous (lines 102-103)
* "We analyze the convergence of SDE solutions for this approximation" → "We analyze the convergence to SDE solutions for this approximation" (line 147)
* "$W_{s_{i+1}}-W_{s_i}\sim\mathcal{N}(0,\sqrt{\Delta s_i})$" → "$W_{s_{i+1}}-W_{s_i}\sim\mathcal{N}(0, \Delta s_i)$" (lines 162 and 163)
* I am not sure if the statement "The discretization is also inevitable whenever it is required to sample $B_t$" is true. (lines 166 and 167)
* "that of approximated solution" → "that of the approximated solution" (line 265)
* "distance on rough path space" → "distance in rough path space" (line 275)
* "Recent work on generative model" → "Recent work on generative models" (line 364)
* "We consider the case of perturbation in data under a new SDE" → "We consider the case where the perturbation in data is under a new SDE" (line 366)
* "this work can improve the current model with" → "our current model can be improved with" (line 393)
* Algorithms 1 and 2 in the appendix should be referenced in the main body of the paper.
* "(1)long-range" → "(1) long-range" (line 668)
* "Ito" → "Itô" (lines 676 and 678)
* "has the same" → "have the same" (between lines 685 and 686)
* "Here, Hurst parameter is" → "Here, the Hurst parameter is" (line 693)
* "we provide some backgrounds of" → "we provide some background on" (line 695)
* "is Itô-Lyons maps" → "is the Itô-Lyons map" (line between 714 and 715)
* "That is, followed" → "That is, following" (line 733)
* "i.e., its bijection and" → "i.e. its bijective and" (line 734)
* "via canonical lift $\boldsymbol{B}$ and initial $X_0$" → "via a canonical lift $\boldsymbol{B}$ and initial state $X_0$" (line 735)
* "one can have a local Lipschitz" → "we can obtain a local Lipschitz estimate" (end of line 737)
* "Combining with 20" → "Combining with (20)" (line 739)
* "Figure 10 and 11" → "Figures 10 and 11" (line 778)
* "Figure 10 and 10" → "Figures 10 and 11" (line 781)
* "Figure 14 and 13" → "Figures 13 and 14" (line 784)
* "Recent work on generative model" → "Recent work on generative models" (line 787)
* "while slightly underestimates those with small values" → "while slightly overestimates those with small values"  (figure 10)
* "Therefore, we resort a similar form to" → "Therefore, we resort to a similar form of" (line 791, bottom of p22)
* I would have thought it is "$\varsigma(t; Z)$" and not "$\varsigma(t)$"? (bottom of p22)
* "underestimate the Hurst having small value" → "overestimates when the true Hurst exponent is small" (figure 11)
* "depepend" → "depended" (line 792)
* "phrases" → "phases" (twice on line 795)
* "((long-range" → "(long-range" (line 803)
* "close to Gaussian distribution" → "close to a Gaussian distribution" (line 807)
limitations: The authors have addressed some technical limitations, particularly related to when the noise is rough (i.e. when $H$ is small) – which I think is reasonable. The authors could possibly include a statement along the lines of "We do not anticipate any significant negative societal impacts as a result of this work".
review_title: null

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: This paper presents an alternative class of conventional step-size neural stochastic differential equations (SDE). It provides an approximate sampling method based on sparse Gaussian Processes for the noise integral in order to put the neural SDEs into a computationally accessible form. Experiments have been conducted on the sequential finnancial time-series data and image recognition.
strengths_and_weaknesses: Strengths
1. The explanation of earlier work on neural SDEs is quite solid mathematically, which makes the propsoed method more convenient to understand for readers who have little experience.
2. The experiments are quite diverged, covering function re-production, financial data analysis and conventional MNIST image recognition.
3. The GP method itself is easy to understand, as well as the general presentation of the paper.

Weaknesses
1. Even though in the result section the title mentioned it, the evaluation lacks quantitative comparison, especially for image recognition task, but more in a qualitative way. Some numerical analysis would help.
2. There are some spots where the author explains the weaknesses of earlier work, but does not connect them to the proposed solution very tightly. e.g. In section 4.1, the approaches from [64] has the problem of un-scalable random variables. But then its problem on current models and relationship with the author's algorithm remain un-reached.
3. The baseline model compared with is latent SDEs, while stronger models such as GAN based SDE solvers may want to get in. Also, apart from Hurst function, some other approximation experiments might be helpful.
questions: The questions are addressed in the weaknesses. Two of them:
1. Is there any experiments on MNIST based on accuracy can be reported?
2. In section 4.1 - you said for method covered in [64]. "the number of these random vartiables prevents optimize-then-discretize methods to scale". But what is the problem and potential negative impact of this practically? And how your method may solve it?
limitations: There are no problem on the societal impact to me for this work. The technical limitations are addressed in the weaknesses.
review_title: null

final_score: rating: 3: Reject: For instance, a paper with technical flaws, weak evaluation, inadequate reproducibility and incompletely addressed ethical considerations.
review_content: summary: The paper studies a different noise model in SDEs. Namely, the model integrates the Wiener process in time scaling it with the function (that depends on time) called Hurst Exponent. It seems that this formulation is equivalent to the Ito SDE with a different scaling term. However, I don't have a background on the Hurst exponent, and the authors do not discuss such a relation. 

Throughout the paper, the authors claim that their main goal is to learn the Hurst exponent from the given data. They propose to sample the noise via the Gaussian process and learn the model via the variational inference as in neural SDEs [1]. Unfortunately, it's hard to describe the proposed method more precisely due to its poor description in the paper.

The authors provide two types of experiments: the estimation of the Hurst function from the given time series; the generation of the MNIST dataset with different noise models (different Hurst exponents).

1. Xuechen Li, Ting-Kam Leonard Wong, Ricky T. Q. Chen, and David Duvenaud. Scalable gradients for stochastic differential equations. In Proceedings of the Twenty Third International Conference on Artificial Intelligence and Statistics, pages 3870–3882, 2020.

strengths_and_weaknesses: Unfortunately, the paper does a poor job on all main aspects of a scientific publication: a motivation of the problem, a clear description of the proposed approach, and a rigorous comparison of the proposed approach to the competitors.

For the main motivation, the authors propose the learning of the Hurst exponent. However, it's not clear why learning the exponent is important in practice. I can assume that this can be used for further simulation of the stochastic process, but it's not clear how methods that do not take the Hurst exponent into account (just learning the scaling term in Ito SDE) compare to the proposed method.

For the method description, the paper does not describe the optimization problem and parameterization of the model (the authors claim to learn all the terms in Ito SDE). The main development of the paper is the sampling stage (section 4.3), but the authors do not describe how they propagate gradients through these samples that depend on the model. Furthermore, the whole algorithm seems to be independent of the exact form of the scaling term in SDE (whether it's introduced with the exponent or not). Thus, the whole discussion on the Hurst exponent doesn't seem to bring any practical value or modify the algorithm anyhow. In the considered context, the Hurst exponent can be considered simply as another parameterization of the scaling term in Ito SDE.

In what follows, there is a list of minor clarity issues throughout the paper.
- line 41. It's not clear w.r.t. what parameter the models cannot be scaled.
- Related work section rather enumerates a broad field of literature than relates the current paper to the existing literature.
- There is a broken reference on line 178.
- The sentence in lines 225-227 is not clear. Maybe there is a typo there.

The empirical evaluation of the proposed method does not give any evidence of its efficiency. For the Hurst exponent estimation, the authors provide experiments with two types of data. For the synthetic data, the authors report the quality of approximation in Fig 4(a). It's hard to measure the quality of the approximation since the further application is not clear, but I would say that the quality is unsatisfactory. The authors claim that the problem is challenging since they recover it by a single sample, but then I would expect demonstrations of the approach on less challenging problems. For the financial data, the performance is not measured quantitatively at all.

The image generation with different Hurst exponents is not evaluated properly. Once again, it's not clear why it is not an introduction of a different scaling term. Furthermore, some works study different noise scales [2,3], and no comparison with them is provided.

2. Song, Yang, Jascha Sohl-Dickstein, Diederik P. Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. "Score-based generative modeling through stochastic differential equations." arXiv preprint arXiv:2011.13456 (2020).
3. Kingma, Diederik, Tim Salimans, Ben Poole, and Jonathan Ho. "Variational diffusion models." Advances in neural information processing systems 34 (2021): 21696-21707.
questions: My questions and suggestions follow naturally from the section "Strengths And Weaknesses."
limitations: It's hard to assess the limitations of the approach due to its poor description and motivation.
review_title: null

final_score: rating: 8: Strong Accept: Technically strong paper, with novel ideas, excellent impact on at least one area, or high-to-excellent impact on multiple areas, with excellent evaluation, resources, and reproducibility, and no unaddressed ethical considerations.
review_content: summary: Neural SDEs are generalised so as to admit a certain class of non-Brownian drivers. This is done by aligning a Riemann--Stieltjes discretisation of the non-Brownian driver with an Euler--Maruyama discretisation to a separate SDE, so as to reduce to the case of a neural SDE with random coefficients; this randomness is specifically a dependence on a Gaussian process. Scalable sampling of this Gaussian process is performed via sparse inducing points.
strengths_and_weaknesses: **Strength: technical approach**

This paper is technically very strong. "Neural fractional SDEs" have been mooted for some time, with no real work in large part due to the non-scalability of sampling fractional Brownian motion (fBM). This paper overcomes this issue through a combination of (a) switching from fBM to another related process (due to Levy); (b) drawing on the theory of sparse inducing points; (c) demonstrating convergence via rough path theory; (d) carefully linking up all the pieces in specifically discrete time.

This is an excellent tour-de-force requiring expertise across multiple disciplines (GPs; SDEs; numerical methods; rough path theory; deep learning.). To the best of my knowledge this approach is novel.

**Strength: attention to computational cost etc.**

This paper does a good job discussing the varying computational costs of each procedure (e.g. $O(N)$ for sparse inducing points; $O(N^3)$ for Cholesky decompositions, etc.) This is of course important in practice. 

This is true even at the level of subtle technical details, such as the computational benefit of Stratonovich integration over Ito integration when backpropagating via optimise-then-discretise.

**Strength: discussion of limitations**

The authors are careful to state the limitations of their approach explicitly.

**Weakness: not self contained**

This paper obviously assumes a sophisticated background in stochastic processes. As such, at present, I do not believe the paper is self-contained: the results used are frequently not stated, and instead only a reference is given.

For example making clearer what is the relationship between equation (1) and fBM; the definition of mBm (line 88); the relationship between Hurst exponent and Holder continuity (line 100); etc.

(I admit I am hard-pressed to identify what should be cut in order to make space for such explanations, as the paper is already very dense. I suspect it may require some reorganisation -- for example the second paragraph of "Computational analysis" (line 271) could be replaced with a few extra words at the end of line 193.)

**Weakness: "adjoint method"**

The term "adjoint" has become heavily overloaded. There are multiple ways to backpropagate through a differential equation solve. In the current literature is is often used to mean specifically optimise-then-discretise, but in older literature it was often used to mean specifically discretise-then-optimise. Following [Ref. 37 of this paper, Remark 5.5], I would recommend avoiding the term "adjoint" altogether.

(In passing, the authors may be interested in [Ref. 37 of this paper, Section 5.2.3], who prove optimise-then-discretise backpropagation through SDEs via rough path theory.)

**Weakness: presentation**

The text could doing with being reviewed for typos (some are listed below) and mathematical clarity (as  above). In addition:

Equation (12) is a bit of a lie -- this is constructed for specifically the Euler--Maruyama discretisation, not the continuous limit.

Line 222 could be clarified: state precisely what is the nature of the association.

That the kernel on line 199 is the kernel of fBM should be stated explicitly. (Unless this is known to the reader it comes out of nowhere.)

I believe the presentation of Theorem 1 could be clarified: what is shown is a general result, that sort-of corresponds to the construction being presented. I believe this distinction should be emphasised a little more carefully, e.g. by turning "Convergence analysis" into a new section; or similar.

I am not that familiar with the details of inducing points for Gaussian processes -- at least for me I would have appreciated greater technical precision in Section 3.3.

**Minor issues**

Line 82: typo "of".

Line 141: "on sampling procedure" should be "on the sampling procedure" or similar.

Line 152: missing close bracket.

Line 155: "require computing Cholesky decomposition" should be "which require computing the Cholesky decomposition"

Line 156: equation (5) is referenced before it is introduced.

Line 137 (and elsewhere): the precise definition of $Z$ is not given.

Line 139: $m$ and $v$ should be notated as functions of $u$. This becomes important in equation (10), in which the need for sampling $B'_Z$ becomes unclear.

Line 227: should presumably state the only preliminary results are available for fBM.

Line 229: only $\xi_t$ -> only one $\xi_t$

Line 243: time->times

Line 248: I think this could be simplified to $\rho \in (1/(2H), 3/2]$?

Line 286: "SDEs model" does not parse.
questions: 1. In personal discussions around the possibility of "neural fractional SDEs", the general conclusion has been that it probably wasn't worth the effort -- because similar behaviour could probably be mimicked simply by taking sufficient complicated drifts and diffusions, and by working in a latent space.

   (As this hidden Markov structure implies the possibility of a non-Markov structure to its output -- which might be able to approximate the autocorrelation of a fractional SDE. And indeed latent spaces are standard when working with neural SDEs anyway.)

   As such I would really have expected to see experiments comparing the proposed approach against a "complicated" neural SDE with Brownian driver. This paper is technically impressive but the corollary to that is that it is also technically impenetrable -- I believe for this paper to see impact it will need to demonstrate a substantial improvement against this much-simpler-to-implement alternative.

   (Which I note is already not that simple to implement itself!)

   To my mind this is the single greatest limitation of this work, and it is a substantial limitation.

2. I would emphasise the need to improve several aspects of the presentation, as detailed in the main review.

3. The Remark of line 198: it is mentioned that working with the fBM kernel directly does not work, as it is not differentiable. Is this an issue because of the desire to backpropagate through it, so as to train the Hurst parameter? (Or is it something else?) Either way I think it worth including the rationale here.
4. The Hurst exponents obtained from synthetic data are moderately encouraging, but not that convincing. Can this experiment be tried with much larger amounts of data, and small step sizes, so that the true Hurst exponent really is recovered -- to verify the efficacy of the proposed methodology? The failure to work convincingly on relatively simple univariate data is unfortunate.

Overall I would emphasise that I nonetheless enjoyed this paper.

Unrelated to my review score -- what follows are some questions/suggestions just for my own curiosity.

1. This paper uses inducing points for sampling from Gaussian processes. I have not previously seen this applied in the context of non-Brownian drivers for SDEs -- is this particular combination new here?

2. The problem of simulating fractional Brownian motion is classical, and e.g. fractional volatility models are moderately popular. It seems to me that the approach proposed in this paper, of using sparse inducing points, should apply equally well to fBM (which is also a Gaussian process). What then happens? I assume this has been tried or otherwise fails for some reason.

3. Do the authors believe it would be possible/reasonable to e.g. modify a Heston model to use the alternate driver of (1), and then apply their efficient sampling scheme? This may well have different properties to the rough Heston model currently considered in the literature, so to what extent this is a reasonable model is unclear to me.

4. It is part of the folklore (and not really written anywhere in the literature, so far as I know) that latent SDEs consistently underestimate the variance. I haven't dug into the reason for this, but if the authors continue using latent SDES in future work then they wish to attempt to address this issue alongside. (e.g. by additionally introducing a discriminator and simultaneously training the model as an SDE-GAN -- I speculate).

5. The authors note that their analysis is restricted to the case of Euler--Maruyama with fixed step size, and that in future work they would like to consider (a) better convergence results; (b) GP drivers for NCDEs; (c) higher-order SDE solvers using Levy area. For future work, I would also like to suggest that handling the case of adaptive timestepping would also be particularly useful/interesting.

   (I am aware that adaptive SDEs are not a well-understood topic, with a reasonable number of errors in the published literature. For what it is worth I recommend the references listed in [0] as the best starting point for adaptive SDE timestepping. My own understanding is that general-case adaptivity -- i.e. arbitrary stepsize changes, rather than just halving/doubling -- currently requires the SDE to satisfy the commutativity condition.)

   [0] https://docs.kidger.site/diffrax/api/stepsize_controller/#diffrax.PIDController
limitations: This work has no ethical concerns.
review_title: null

final_score: rating: 4: Borderline reject: Technically solid paper where reasons to reject, e.g., limited evaluation, outweigh reasons to accept, e.g., good evaluation. Please use sparingly.
review_content: summary: The paper proposes an approach to neural SDEs using an alternative to standard Brownian noise. The authors propose a particular definition of alternative noise. They develop some theoretical results which allow standard neural SDE methods to be used to fit an approximation to their stochastic process. Some convergence results are also proved. Finally the method is applied to various experiments.
strengths_and_weaknesses: **Note**: My original score based on this review was 3 (Reject), but I've increased this to 4 (Borderline reject) following the rebuttal.

## Overview

* Originality: good. This seems like an interesting approach to investigate, and is novel to me, although I am not an expert in this area.
* Quality: poor. The experiments are not convincing (see below for details).
* Clarity: poor. I found the theoretical material unclear (see below for details).
* Significance: poor. In my opinion the paper did not make a strong case for the benefits of this approach.

## Strengths

The paper develops some non-trivial theoretical results.

## Weaknesses - overview

I found two main weaknesses in the paper.

1) The theoretical material is hard to follow, and seems to involves a lot of non-rigorous derivations.
2) The experiments do not demonstrate good performance of the proposed method.

I have less confidence about weakness 1) as I am not an expert in the subject area. It may be that the material would be more convincing to an expert. However in my opinion weakness 2) alone is a major limitation of the paper.

## Weaknesses - theory

* Equation (5) is an approximation, but is then treated as exact to derive results throughout the rest of Section 4 e.g. equation (7). Rigorous proofs should also be given.
* Line 178 says "Note that directly modeling $\Delta B_t$ in (6) is numerically unstable when computing (3.3)..." Is this a typo? There is no equation (3.3). This makes the material hard to follow.
* Section 4.4 says the original SDE (4) is "equivalent" to (12). But it seems that this result relies on many approximations e.g. the inducing points scheme. The authors should say that (12) is an approximation or prove that it is equivalent.
* Why does Theorem 1 treat the Hurst exponent as a constant, when earlier it was defined as a function?
* The authors do not give a summary of their proposed method. This makes it hard for others to implement.

## Weaknesses - experiments

* In Section 5.1 the baseline seems as good as the proposed method. It's also unclear to me where the exact details of the two methods can be found.
* In Section 5.2 the proposed method does not seem to give a good fit of the true function. Also, no baseline comparison is included.
* In Section 5.3, the proposed method only does well on training evaluation, and does poorly in test evaluation. This suggests it is overfitting.
* In Section 5.4, the authors write "Although we do not present a way to find optimal h(t), we show that there is a setting in which the NLL appears to be better". This is an interesting exploratory finding. However, given the extra cost of trying several scenarios to get this improvement, it's not clear that the proposed approach is superior to simply running more training iterations of the standard approach.
questions: Can you address the points listed under "weaknesses"?
limitations: The authors include a section on technical limitations. As this is a paper on foundational machine learning, extending existing methods, there do no not seem to be any direct negative societal impacts to discuss.
review_title: null

Meta-review: metareview: Ratings: 4/8/3/7/7.
Confidence: 3/4/4/2/4.
There was no discussion among reviewers, but there was a good amount of discussion between authors and reviewers.

Note that there are 5 reviewers since the original reviews had low confidence.

Summary: This paper studies a variant of neural SDEs that replace the standard Brownian motion driving the NSDE with a Riemann–Liouville fractional Brownian motion that has a learnt time-varying Hurst function. The paper and subsequent discussion unfortunately failed to persuade 2/5 of the reviewers, especially in terms of accessibility of the theory and the empirical validation. However, the reviewers agree that the topic and results are interesting.

Recommendation: I recommend to accept this submission.
confidence: Less certain
Decision: Accept