Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: The paper proposes a generalisation of commonly used image augmentation to video, paying specific attention to temporal consistency and smoothness of the said augmentations. Efficacy of the proposed generalisation is supported by strong empirical results on a wide range of video understanding tasks.
strength_and_weaknesses: **Strengths**
* Well-written paper and well-motivated method.
* Convincing extensive experiments on a wide range of video recognition tasks.
* Simple widely applicable method.

**Weaknesses**
* Open source implementations of the proposed augmentations *in a range of existing frameworks* (e.g. numpy, tensorflow, etc) would greatly help adoption of the proposed methods.
clarity,_quality,_novelty_and_reproducibility: Simple, clear, novel idea. Open source code would help with reproduction.
summary_of_the_review: Clear simple video augmentation technique that is demonstrated to consistently improve model performance in upstream and downstream tasks.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: This paper presents a simple yet effective video data augmentation framework, called DynaAugment, to improve the robustness and accuracy of several video recognition tasks, e.g., action classification, action segmentation, action localization, video object detection. The proposed data augmentation framework respects and simulates the real-world video's temporal variations, instead of naively extend the image augmentation methods, and thus it enjoys good diversity and temporal smoothness. Specifically, a novel sampling function called Fourier Sampling is presented to simultaneously achieve the two goals, i.e., diversity and smoothness. With extensive ablation studies and experiments on multiple downstream tasks such as action recognition and corrupted video classification, impressive superiority of the proposed method is demonstrated.
strength_and_weaknesses: Strengths:

1. The work is fundamental and meaningful to the video research community, since most of video downstream tasks involve data augmentation strategies.
2. Good insight and meaningful motivation. And clear illustration of the main idea.
3. Extensive and consistent experimental support, including most of the commonly-used benchmark datasets. Remarkably, some of the experiments are interesting and novel. E.g., classification on corrupted videos. 
4. Clear writing style and good presentation.

clarity,_quality,_novelty_and_reproducibility: [Clarity] The clarity of the manuscript is good.
[Quality] The overall quality of this paper is impressive.
[Novelty] The novelty of this submission has reached the acceptance standard of ICLR, in my opinion.
[Reproducibility] It is hard to judge the reproducibility of the submission since no code is provided. Based on the implementation details, the re-implementation of the method should not be that hard.

summary_of_the_review: The overall quality of this submission is good. Please see "Strengths and Weaknesses" above.
I tend to accept this paper.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: The paper tackles the problem of data augmentation for training video models. It argues that existing augmentation techniques just mostly extend image augmentation strategies and don't fully utilize the temporal dynamics of videos. The paper proposes a new data augmentation framework specifically designed for videos (DynaAugment) that changes the magnitude of augmentation operations for each frame with the help of Fourier Sampling. The experiments are conducted on a diverse set of video datasets including large-scale, small-scale, and fine-grained. DynaAugment improves results in every scenario in combination with existing image-based data augmentation techniques.
strength_and_weaknesses: **Strengths**:

1) The paper is very well-written with good structure and nice visualizations. The motivation behind DynaAugment is clear and fitted to the nature of the video domain.
2) The proposed method is based on a strong theoretical framework (Fourier Sampling) with the application for general dynamic data augmentation with smooth and diverse properties.
3) The ablation study shows the benefits of the chosen methodology for dynamic variations over other possible alternatives.
On the large and diverse set of video datasets DynaAugment outperforms image-based data augmentation techniques using even different backbones including CNNs and transformers. The most significant improvement is shown on small-scale and fine-grained datasets of the most practical interest.
4) DynaAugment is also useful for transfer learning other than video action recognition tasks such as action localization, action segmentation, video object detection, and self-supervised learning
5) DynaAugment helps in the case of corrupted videos showing better results than other data augmentations.

**Weaknesses**:
1) A little bit of limited technical novelty which consists mainly of temporally-smoothed parameterized data augmentation.

clarity,_quality,_novelty_and_reproducibility: The clarity and the quality of the paper presentation are on a high level. It has nice visualizations with an easy-to-follow structure. The paper and its supplementary materials contain a detailed description of the models and experiments. So I would say that the work and the results are reproducible. However, the paper has a little bit of limited technical novelty. 
summary_of_the_review: I have a favorable opinion of the paper and I think the paper deserves the acceptance rating.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: This paper explores automatic data augmentation by introducing a simple yet effective Fourier sampling. It provides dynamic, smooth, and diverse augmentation, especially for videos with a time dimension. Many experiments on large-scale, small-scale, and fine-grained video action tasks are conducted and it shows an almost consistent improvement over multiple benchmarks.
strength_and_weaknesses: Strength:
-The motivation is straightforward and well-presented. The solution is simple yet effective and it can be seamlessly applied to the existing auto-augmentation methods such as RA, UA, and TA. 
-The experiments are enough to prove the effectiveness of the proposed Fourier sampling for dynamic augmentation. Extensive ablation studies are provided in the appendix.

Weaknesses:
-The baselines on experiments are somewhat weak. For example, Swin-Tiny and Uniformer-S in Table1 and SlowOnly in Table 3/4 have relatively low performance at the beginning. Stronger baselines are expected.


clarity,_quality,_novelty_and_reproducibility: This paper is well organized and the technical novelty is satisfactory.
Since it is implemented upon RA, UA, and TA, it should be easily reproduced.
summary_of_the_review: Considering the extensive experiments the authors provide, the effectiveness of the proposed sampling though simple is well proven. I feel positive for this paper at this moment.

After reading the rebuttal, I feel that my concerns have been well addressed. More strong baselines have been included to make this paper more solid.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: The authors propose a data augmentation strategy for training video understanding models which focus on temporal consistency, diversity, and smoothness. The magnitude of augmentation operations on each frame is modulated by Fourier Sampling. The proposed method shows consistent improvement across many classic benchmarks. The reviewers were appreciated the significance, clarity, and novelty of the proposed method, as well as the consistent improvements in practice. I will recommend the acceptance of this work.
justification_for_why_not_higher_score: - Limited scope: Impact limited to video understanding community dealing with primarily action recognition tasks
- No open-source implementation in PyTorch/JaX/TF
justification_for_why_not_lower_score: - Solid publication, practical applications in video understanding, very likely to be adapted by the community.
Decision: Accept: notable-top-25%