# Investigating Novice Developers' Code Commenting Trends Using Machine Learning Techniques
## 3 Methodology
### Data Extraction  
As already discussed, we split the methodology for **RQ-1** and **RQ-2** as \(M1\) and \(M2\), respectively. The data extraction process of the research questions is described in \(M1\) and \(M2\). \begin{table}
\begin{tabular}{p{42.7 pt} p{42.7 pt} p{42.7 pt} p{42.7 pt}} \hline \hline
**SNO** & **Category** & **Frequency** & **Description** \\ \hline
4 & **License** & **259 (5.18\%)** & **The code comments that contain information on the terms of use and the licensing of the source code. ** & **Thes code comments provide references to the authors and their ownership of the work, as well as source credentials in the form of an “@author” tag. ** & **The type of code comments for which it is not easy to comprehend their meaning and they do not clearly describe the associated code and are not related. ** & **Thes category includes all comments that contain source code commented out by developers. ** & **Organizational comments are used to communicate the structure of code. They typically take the form of a short comment that explains a module or block of code, separating one functional unit from another. This demonstrates that the programmer is attempting to present code in a way that helps other coders easily understand it. & **Thes category includes Auto-generated code. These are typically the metadata left behind by an IDE and contain only the skeleton with a placeholder provided by the IDE. ** & **Thes category includes comments that are not properly implemented, e.g., a comment should have an associated code directly below the comment body without any empty lines in between. ** & **Thes category includes the comments that do not contain anything,** & **Thes category includes Auto-generated code. These are typically the metadata left behind by an IDE and contain only the skeleton with a placeholder provided by the IDE. ** & **Thes category includes comments that are not properly implemented, e.g., a comment should have an associated code directly below the comment body without any empty lines in between. ** & **Thes category includes  
3.1 M1: Preparation of Dataset of Source Code Comments and Building Taxonomy of Source Code Comments  
As shown in Figure 3 and Listing 1, the steps from 1 to 4 of the figure and algorithm cover the data extraction of _RQ-1_. It defines the overall procedure of dataset preparation which is sub-divided into four parts, (i) pre-processing of java source code, (ii) building the parser, (iii) the JSON object is converted into CSV, and (iv) building taxonomy and dataset annotation, which are described as follows:  
* **Pre-processing of java source code:** The dataset required to carry out this research was prepared by pre-processing the raw source code. This raw source code is obtained from the lab assignments of sophomore-year students in the computer science discipline. We only considered the java source code files for our study. Initially, we prepared a dedicated script to obtain all the projects that are: (i) complete projects and (ii) programs built in java language. This results in a total of 70 projects. * **Build the parser and Parse the raw source code to create JSON object:** A parser was developed in JavaScript language to parse the java source code files and extract the code-comment pairs. The parser goes through a directory, traverses all sub-directories within that directory, and searches all files with a.java file extension. It reads the files one by one, extracts the code-comments pairs from those files, and creates a JSON object. That extended JSON object is later converted to CSV file format so that this can be used for machine learning experiments. * **The JSON object converted into CSV:** Data pre-processing is the first and most crucial phase in the research analysis. Data pre-processing is applied to the CSV file that is obtained by the parser. Intensive pre-processing would be required to convert the raw code into a usable dataset.