Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: In this paper, the authors propose LEAVES, which is a lightweight module for learning views on time-series
data in contrastive learning. The LEAVES is optimized adversarially against the contrastive loss
to generate challenging views for the encoder in learning representations.


strength_and_weaknesses: Strength:
This paper proposed AutoAugment in their methods, which is quite useful for contrastive learning.

Weakness:
Their experiment just show a sole example on some medical case. It does not show a good contribution to the whole fields. Also their novelty is very empirical-based, I am not sure if this is suitable for this conference. 
clarity,_quality,_novelty_and_reproducibility: The paper has good clarity and good quality.

The novelty of this paper is acceptable but with low reproducibility since the datasets are closed.
summary_of_the_review: In this paper, the author introduced a simple but effective LEAVES module to learn augmentations for timeseries data in contrastive learning. With an adversarial training manner, the proposed method optimizes the hyper-parameters for data augmentation methods in contrastive learning. The idea is OK but novelty is not very obvious.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: The papers approaches the problem of contrastive learning of time series representations by learning parameters for time series augmentations. This allows good "views" of the time series to be directly incorporated into SimCLR, leading to reasonable performance gains against the baselines.
strength_and_weaknesses: Strenghts:
- The paper is easy to follow and the idea is clearly presented and explained.
- A diverse set of experiments are done to defend the claims of the paper. The model performs reasonably well against the proposed baselines.

Weaknesses:
- Differentiable augmentations are not a novel concept, and the adversarial framework to learn them is quite reminiscent of AdCo [1].
- One setup missing is the forecasting experiments in e.g. TS2Vec. This method could be directly applied to it, and results on a set of orthogonal tasks would be beneficial. More importantly, TS2Vec, CoST, etc. missing as (strong) baselines: comparing to them would strenghten the experiments section.


References:
[1] AdCo: Adversarial Contrast for Efficient Learning of Unsupervised Representations from Self-Trained Negative Adversaries

clarity,_quality,_novelty_and_reproducibility: Clarity: Clear.
Quality: Somewhat good. Well written, and extensive empirical evaluation on one hand. On the other hand, missing key baselines.
Originality: Derivative, to some extent. The proposed approaches have been shown to work well in other domains, such as computer vision.
summary_of_the_review: While I find this work well-written, and appreciate the extensive experiments, I also note the two main flaws I have listed above. Both those positive and negative points justify, in my opinion, the score I have given.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This paper tackles the problem of automatically generating augmentations for time-series data in the contrastive learning framework. It proposes LEAVES, a module that learns the hyper-parameters of the time-series data augmentations. The module is trained adversarially, i.e. maximizing the contrastive loss while the encoder network is minimizing the contrastive loss. Experiments on four public time-series datasets demonstrate the efficacy of the proposed method.
strength_and_weaknesses: Pros:
- Interesting topic. Contrastive learning has been shown to be very powerful, but it requires well-designed data augmentations, which has not been well studied for time-series data (compared to that for image). A good automatic time-series data augmentor should be beneficial for a great many real-world applications. 
- Good results. This paper conducts experiments on four real-world time-series datasets, and the proposed method outperforms the supervised baseline and ViewMaker (an automatic domain-agnostic data augmentor) consistently. It is also very helpful to have SOTA methods listed in the tables. 
- Nice ablation studies. The ablation studies show that the downstream performance for the four real-world tasks will be influenced by the choices of the hyper-parameters of data augmentations by a non-negligible margin, and the previous method, i.e. ViewMaker, is not able to generate satisfiable augmentations for time series data. Those underscore the need of designing a better automatic time-series data augmentor.

Cons:
- This paper only uses SimCLR as the contrastive learning framework to demonstrate everything. However, several contrastive learning frameworks (e.g. BYOL,  MoCo v2/v3) have been proposed after SimCLR and have been shown to outperform SimCLR empirically. How to adapt the automatic augmentation module into those frameworks (that should be easy for the proposed method) should be described, and whether the module can improve the performance of those frameworks empirically should be verified. 
- The novelty of the method is a little bit low. The adversarial framework is the same as ViewMaker, with the only difference being the types of hyperparameters to be learned. Moreover, the proposed method can only learn the hyperparameters of augmentations that are already well-designed, and it can not generate new augmentations that have not been discovered yet. Therefore, using such a module still requires researchers to manually design augmentation types first. 
- The organization of the paper can be improved. The definition and at least of a short explanation of the data augmentation used in the proposed module, i.e. Permutation, TimeDis, MagW, Scale and Jitter, should be included in the method section. Only from Figure 3, it is very unclear what they are. Moreover, the variables to be learned should be highlighted clearly, and why they are learnable (differentiable) should also be explained. 

Question:
- What is the difference between MagW and Jittering? Isn’t MagW included in Jittering?
- In Equation (1), what is \sigma_P? Isn’t the variable to be learned for Permutation the number of segments K?
- Why are Permutation and TimeDis differentiable?
- How do you know when to stop during the training process? As shown in Figure 5, some of the parameters, like Perm Max Segments on Sleep-EDFE and on Apnea-ECG, are not converging eventually. 
- Are the hyper-parameter values learned by LEAVES meaningful? Have you tried to re-train the encoder network using the (fixed) hyper-parameters learned by the adversarial training process? Will that deliver better or worse results compared to training the encoder and the hyper-parameters together?

clarity,_quality,_novelty_and_reproducibility: The novelty in this paper is a little bit low. Compared to ViewMaker, the novelty lies in the different types of augmentation to be learned. However, the types of augmentation used in the proposed method (Permutation, TimeDis, MagW, Scale and Jitter) are not designed delicately. It is possible that some are overlapped, e.g. Jitter and MagW, and other important types are missing, e.g. transformation in the frequency domain.

As mentioned above in “Cons” and “Questions”, the paper is not clear enough. The organization should be improved and the important details should be added. Without those details, the proposed method is not reproducible. Additionally, some words should be rephrased to avoid confusion. For example, in the title of 5.2, “fine-tune” has other meanings, i.e. updating the encoder in the downstream tasks, in that context. 
summary_of_the_review: This paper addresses an interesting and important question - how to automatically generate good time-series data augmentations for contrastive learning. The empirical results of the proposed method are good, but the proposed method is not designed delicately and important details and experiments are missing. Therefore, the paper needs more work before it is ready to be published in my opinion, but I am happy to raise the score if all my concerns are addressed.
review_title: null

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: The papers proposes to learn the view generation component for contrastive learning for time series data.
The alogithm is simple and includes usage of 5 common augmentations techniques and one new one (TimeDis) applied sequentialy to generate a view.
The loss function is similar to SimCLR and is maximized wrt parameters of LEAVES in an adversarial game.
The authors demonstrate improvement for the considered four medical datasets.
strength_and_weaknesses: Strengths:
- The name LEAVES is really good
- Results for three of four considered datasets are better than baseline results
- The number of hyperparameters to learn for LEAVES is small
- The idea of using the reparametrization trick is nice (while natural)

Weakness, methods:
- According to the presented approach, we should select maximum distortion all the time. So, it is not clear, why this effect is not observed in practice, making the solution trivial
- The comparison and overall presentation of the method can benefit from looking on additional relevant papers that follow simiar ideas [3, 4] 

Weakness, experiments:
- It seems, that to maximize the loss, we should distort the views with maximum values of sigma. It is true for most of the hyperparameters. Can you provide performance of LEAVES for hyperparameters that correspond to maximum distortion?
- What are the architectures for the considered datasets? Why they are selected in such way?
- Recent works on learning time series representations list particular important augmentations as well as ways to learn reporesentations to solve donwstream tasks [1, 2]. Why this work doesn't compare to those approaches and uses datasets different from that mentioned in these papers?
- It seems, that non-contrastive approaches are superior to SimCLR even for time series data [6]. The paper would benefit from adding this trainable view generator to e.g. BYOL similar to [3]
- Broader selection of the datasets for study would help to understand, what augmentations are useful in what settings. For example, it seems, that some of them can harm performance for periodic time series
- The ablation study doesn't demonstrate that all used augmentations are essential to the quality of the considered approach. In particular, does TimeDis contributes to the overall quality of the approach? Now it is not clear

Technical and presentation remarks:
- The exact training procedure is missing. Do we train the view generator and the encoder simultaneously? What other strategies are possible? Can other approaches improve the result?
- Figure 5: what is at the axes? Iterations or Epochs for x? Hyperparameter value for y?
- As the focus of the new method is performance, it makes sense to unite Table 1 - Table 4 in single Table in the main text, providing full version in appendix. It will help to save the space in the main text for desired technical details, that are missing now.

Literature:
1. Zhihan Yue, Yujing Wang, Juanyong Duan, Tianmeng Yang, Congrui Huang, Yunhai Tong, Bixiong Xu. TS2Vec: Towards Universal Representation of Time Series. AAAI. 2022
2. Eldele E., Ragab M., Chen Z., Wu M., Kwoh C. K., Li X., Guan C. Time-Series Representation Learning via Temporal and Contextual Contrasting. IJCAI. 2021. 
3. Shi, Yuge, et al. "Adversarial masking for self-supervised learning." ICML. 2022.
4. Reed, Colorado J., et al. "Selfaugment: Automatic augmentation policies for self-supervised learning." CVPR. 2021.
5. Koyama, Masanori, et al. "Contrastive Representation Learning with Trainable Augmentation Channel." arXiv preprint arXiv:2111.07679 (2021).
6. Marusov, Alexander, Valerii Baianov, and Alexey Zaytsev. "Non-contrastive approaches to similarity learning: positive examples are all you need." arXiv preprint arXiv:2209.14750 (2022).
clarity,_quality,_novelty_and_reproducibility: The writing is clear. The algorithm is reproducible or close to it.
summary_of_the_review: The approach seems interesting and simple, inheriting these qualities from a similar approach from CV ViewMaker.
While the obtained approach and results deserve attention, they provide a limited contribution to the community.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: The paper presents a simple approach to generate multiple views of time-series data augmentations towards contrastive learning. Though it has been demonstrated on SimCLR, it has potential for other contrastive learning methods too. The approach has been empirically evaluated on four public data sets.

The approach is simple and does not make a substantial contribution.
justification_for_why_not_higher_score: - Lacks novelty
- Lacks clarity of presentation and organization
justification_for_why_not_lower_score: The paper presents an approach to generate augmentations for Contrastive learning in time-series
Decision: Reject