Source Code Data Augmentation for Deep Learning: A Survey
Authors: Terry Yue Zhuo, Zhou Yang, Zhensu Sun, Yufei Wang, Li Li, Xiaoning Du, Zhenchang Xing, David Lo


Reviewer paper summaries: This paper provides a comprehensive survey of data augmentation (DA) specifically focused on source code tasks. The author addresses the gap between source code and natural language, which has led to the traditional DA methods in natural language not being directly applicable to the code domain.
The survey compiles existing literature to offer a holistic overview, addressing the need for enhanced training data to improve model robustness and generalizability. It categorizes DA methods into rule-based, model-based, and example interpolation-based techniques, exploring their effectiveness in various source code tasks such as code completion, clone detection, and program repair.

The paper reviews various data augmentation (DA) techniques tailored for source code models, categorizing them into rule-based, model-based, and example-interpolation methods. It highlights the challenges and opportunities in applying DA to enhance the robustness and generalizability of deep learning models for source code tasks, emphasizing the need for further research in this field.
Summary of strengths: The strengths of this research work lie in its comprehensive and systematic compilation of data augmentation for source code, offering a categorization and in-depth analysis of rule-based, model-based, and example interpolation-based methods. It provides valuable insights into the effectiveness of these techniques in various source code tasks, highlights key challenges, and presents opportunities for future research, thereby serving as a valuable resource for further research work on programming language understanding and generation.

1. The paper conducts a thorough and systematic review of the literature, mapping out the evolution and trends in DA techniques for source code over the past five years. 

2. By highlighting existing gaps and challenges, such as the need for project-level source code augmentation and techniques for low-resource programming languages, the paper not only critiques the current state of research but also directs future research efforts.
Summary of weaknesses: 1. Since it's a survey paper and not very novel, it also does not cover declarative programming languages, and there is a lack of standardized benchmark tasks for evaluating different augmentation methods.
2. It focuses primarily on function-level data augmentation, lacking exploration of project-level augmentation. Otherwise, it would be very helpful for long-context LM training data, which is a very hot topic today.

1. The article is purely qualitative and does not contain any experimental or empirical results.

2. Although an overview of source code data augmentation (DA) techniques is provided, the descriptions are relatively brief. For example, Section 3 discusses method stacking and optimisation, but may not go into detail on how to select and combine different DA techniques and how to adapt the optimisation strategy for a particular task.

3. There is no clear indication in the article of how to assess the effectiveness of different DA methods, and there is a lack of harmonised metrics or assessment protocols.

4. The article focuses on function-level code snippets and does not fully consider project-level source code. Project-level data enhancements may involve more complex information, such as dependencies between different code modules and high-level architectural considerations.
Comments, suggestions and typos: Since there are many different programming languages with varying levels and grammar, is it possible to differentiate DA from the programming language perspective? For example, the coding style varies among different languages. Java source code may be easier to understand because the function names are more meaningful.

It will be better to mention declarative programming languages or other types of programming languages.
Overall assessment scores: 2.5

3.5