Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 10: strong accept, should be highlighted at the conference
review_content: summary_of_the_paper: The authors present a time-series augmentation method names Recursive Interpolation Method (RIM), designed to boost performance in learning tasks with limited data. The method is solidly based on a theoretical based, and the analytic results are constructed with numerical experiments that prove the goodness of the technique.
strength_and_weaknesses: Strengths:
1. The pipeline is based on a principled approach.
2. Stochastic type loss functions are used (cross-entropy) instead of deterministic ones.

clarity,_quality,_novelty_and_reproducibility: Very clear and solid work.
summary_of_the_review: The authors present a time-series data augmentation approach based on theoretical results. The work is solid and checked against experiments. This kind of work is in stark contrast to most of what can be seen in the field: ad-hoc pipelines with some experimental testing where it is almost impossible to assess the generality of the results.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This work propose a new time series augmentation technique, called Recursive Interpolation Method (RIM), to learn accurate models with limited data. They provide theoretical analysis with the guaranteeing faster convergence with reduced variance. Empirically, they show that the proposed method improves the performance on classification tasks over the SOTA on both real-world and synthetic datasets.
strength_and_weaknesses: Pros:
1. Data augmentation is critical for time series analysis. We have seen a lot of research for developing new data augmentation methods to improve the performance of downstream tasks, but few of them provided theoretical analysis. This work opens the door for analyzing the generalization capability of time series data augmentation. Besides, this work also presents a new data augmentation method based on recursive interpolation.

2. Empirical studies verify the efficacy of the proposed data augmentation method on both synthetic and real-world datasets.


Cons:

Major:
1. The proposed RIM in sec 3.1 lacks motivation. It would be better to provide more motivational examples showing how it can benefit the time series related tasks. Most of the existing data augmentation methods aim to enhance the temporal dependencies, but the proposed RIM in Eq 1. is temporal independent. I am unclear on how  RIM can help avoid overfitting and capture better temporal patterns. Since the proposed data augmentation method is temporal independent, it can be generalized to any tabular data. How does this method connect to time series data? Additionally, RIM in eq. 1 is a kind of mixup of two consecutive dimensions. How to justify its correctness, especially when there exists dependency over two non-consecutive dimensions?

2. The theoretical analysis is interesting but needs more clarification. It is clear to me the effect of sample size and the distance between the original time series and the augmented one. Still, it is unclear how to control and decrease the Rademacher complexity for the augmented data. Eq.6 does not guide how the proposed data augmentation method can reduce it.

3. The empirical studies are not promising. First, I am not sure why the authors do not compare with other data augmentation methods, but the time series generative model, TimeGAN. Besides, there are a lot of novel generative time series models, like the generative Fourier model,  the diffusion model beat TimeGAN with a margin. Finally, since the major contribution of this work is about the generalization capability, it would be more interesting to present the empirical analysis for each complexity term to show its vacuousness.

4. The proposed method is limited to point-wise time series classification, and it can not generalize to the more common settings, like window-wise classification or conventional time series forecasting problems. This might reduce the impact of this work.

Minor:
1. what's the definition of 'dem'?
2. It would be better to present assumptions 3 and 4 in the main content to make the whole manuscript self-contained.
clarity,_quality,_novelty_and_reproducibility: see above
summary_of_the_review: This paper studies an interesting and important problems, but the proposed method lacks motivation
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: Training set augmentation has been found to be an effective way of improving performance on downstream tasks such as classification accuracy, particularly for computer vision problems. Augmentation has also been considered for the time series domain, although much of the contributions in this space implicitly make assumptions on the structure of the time series, and can thus be less widely applied. It is also difficult to provide guarantees on the quality of generated samples using conventional methods. In this work, the authors propose a recursive approach for generating time series that follow the same dynamic process as a (possibly) limited number of observed time series, for which it is also possible to obtain guarantees on quality under certain conditions. The authors demonstrate how the approach compares favorably to augmentation methods leveraging GANs, while also being more computationally efficient.
strength_and_weaknesses: - The recursive approach detailed in this work is intuitive, and I appreciated how it presents a fairly straightforward solution to a problem that manifests itself in any domain where there is risk of overfitting to limited representative time series.
- The paper is mostly well-written, although there were several instances where I wished for more clarity or detail. I especially think that the authors should consider including more illustrative examples within the main paper itself such that the concepts become easier to follow. It is otherwise difficult to sometimes follow which time series structure the authors are referring to throughout the text, and it could also better illustrate the circumstances under which the guarantees put forward in the paper would not hold.
- One of the main motivations for undertaking this work was that existing augmentation techniques are problem dependent, so I would have liked to see more concrete examples of this. I consider such insight and analysis to be a crucial contribution of the paper, and yet it currently feels very underdeveloped.
- Another appealing aspect of this work is that the augmented time series can be leveraged in multiple different downstream tasks, which are highlighted in the introduction to the paper. However, besides a small concluding interlude in Section 4.3, experiments on next step prediction and reinforcement learning are left entirely to the Supplementary Material, with the focus in the main paper being exclusively on classification tasks. While there is a nice variety of synthetic and real-world classification problems here, featuring a different task entirely would have given a more complete picture.
- The actual discussion of the results is also quite surface-level, and I would have liked to see more insights provided throughout as well as ablation studies that tweak different aspects of the augmentation procedure.
clarity,_quality,_novelty_and_reproducibility: - The paper tackles an interesting problem, and I appreciated that the authors went beyond simply empirically verifying the benefit of the approach by also presenting theoretical guarantees on the expected quality of the augmented time series.
- A highlighted earlier, it would benefit from more illustrative examples and figures to properly convey which properties of time series influence the quality of the recursive procedure.
- While extensive, the experiment section could also be improved by adding more ablation studies and more insightful discussions on the results.
- The paper contains a few minor typos, but these should be easy to identify and fix.

summary_of_the_review: The contributions featured in this paper are interesting, and provide an intuitive way of overcoming sparsity in time series datasets. However, I believe there is still considerable room for improvement in the paper, and I encourage the authors to consider rewriting certain sections in order to include further illustrative example and ablation studies that will make the paper more complete and insightful to both readers and practitioners.  


** Post-rebuttal comments **

Thank you for your replies, and the updates to the paper. I am raising my score to a '5', although I still think this paper would benefit from another round of reviewing. The ablation study included as part of the rebuttal is interesting, but I believe there could still be more clarity on aspects such as settings where the RIC approach doesn’t perform as well.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: I have reviewed this paper in another conference.
This paper introduces a technique called Recursive Interpolation Method (RIM) for time series augmentation. The augmented samples preserve the original inherent time-series dynamics. Theoretical analysis is given to guarantee the test performance. Experiments are conducted on multiple time series tasks such as classification, regression and RL.
strength_and_weaknesses: Pros:

(1) The motivation to perform data augmentation that preserves the original inherent dynamics makes sense.

(2) The methodology is clearly described and theoretical analysis is given.

(3) Experiments are extensively conducted and it is shown that the model with data augmentation clearly outperforms the model without augmentation.

Cons:

(1) Same to my concerns in the previous review, the method in this paper assume that the data is stationary, which is not the case in real world. Though it is possible to use a large lambda close to 1 for non-stationary data, I still feel the assumption made in the method is a drawback.
clarity,_quality,_novelty_and_reproducibility: Good
summary_of_the_review: A good paper with theoretical insights and experimental support. But a flaw in the assumption of the model.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: Although there are mixed views and divided opinions among us about this paper, it stands out as a rare contribution to the topic of time series data augmentation by presenting a principled approach with solid theoretical justification. What’s more, the theoretical results are validated experimentally through extensive empirical studies. While it is worth giving this paper a chance to get published, its impact could be higher if the major weaknesses as pointed out by the reviewers can be addressed as much as possible before publication. The authors are highly recommended to revise their paper thoroughly by considering our comments and suggestions.

justification_for_why_not_higher_score: It is far below the threshold for spotlight acceptance unless the major weaknesses are thoroughly addressed.

justification_for_why_not_lower_score: It is worth giving the paper a chance due to its solid theoretical justification.
Decision: Accept: poster