Paper: Parameter Estimation of Long Memory Stochastic Processes with Deep Neural Networks (ICLR-2024)

Reviewer comment (Experiments):
The paper misses significant breakthroughs in the domain of learning long-term dependencies using neural networks. In particular, [1,2,3,4] pushed the boundary on the sequence length of data that can be learned by an ML model. It is important to see how the proposed methods work compared to and in conjunction with these approaches.

Author response:
In our opinion, the problem of parameter estimation is not the same as the (general) task of modeling long sequences. While in the latter case, the proposed articles are clearly relevant, for the problem studied in our article, neural networks processing the raw sequences can produce similar or better results. Nevertheless, in the introduction the paper Kidger et al. (2019) is also cited. In practice, the usable sequence length ranges from a few hundred to a few thousand, e.g. due to the frequently occurring time dependence of the parameters.

Reviewer comment (Writing):
The writing of the paper can be improved. In particular, it is unclear what is new and what is just standard training methods. I think the paper would be much stronger if the contributions and relations to existing methods were stated more clearly.

Author response:
The new insight provided by the article is the realization that sequence processing neural architectures trained on a sufficient amount of data, can in itself replace the role of even sophisticated statistical descriptors in determining the memory parameters characterizing processes, somewhat similarly to the role of convolutional networks in image processing. This is formulated in the Introduction and in full concreteness in the Conclusion. The main part of the measurements is to argue on the power of these latent calculations leading to the parameters.

Reviewer comment (Novelty):
The major novel contribution of this work is not clearly specified at all.

Author response:
The new insight provided by the article is the realization that sequence processing neural architectures, trained on a sufficient amount of data, can in itself replace even sophisticated statistical descriptors in determining the memory parameters characterizing processes, somewhat similarly to the role of convolutional networks in image processing. This is formulated in the Introduction and, in full concreteness, in the Conclusion. The main part of the measurements is to argue on the power of these latent calculations leading to the parameters.

Reviewer comment (Novelty):
This work seems to be an application of available process generators to generate high quality synthetic data for the fBm and other long range stochastic processes to train standard neural network models and evaluate their performance. This contribution is not sufficient for an ICLR paper.

Author response:
Our result does indeed operate with existing tools, but it shows many previously unknown or unachieved insights. Obviously, the most important thing is the accuracy and speed of parameter extraction; the superior nature of our method is discussed in detail in Section 4.
The question also arises whether it is possible to show better performance if we do not work directly from the raw data of realization but rather extract some complex features, e.g. statistical descriptors, and use them as input for a neural network. In this regard, we have carried out several measurements, and neither the use of spectrum domain nor the signatures give better performance while they require significantly more computing resources.
We note that the codes used for measurements will be available, which also indicates that the paper is more than a compilation of off-the-shelf methods.