Empowering Automated Code Documentation with Code Metrics in Computational Notebooks
Creators: name: anynomouse
Publisher: name: Zenodo
Year: 2024

description: All source code and dataset for different artifacts that have been used throughout our experiments can be found here.

NB. We have frequently used Python 3.10 for all the experiments, please refer to requirements.txt for env specs.

 utilityThis component mainly consists of two submodules:- The abstractions (alongside their implementation) that handle the creation of checkpoints for machine learning models that are built upon torch package. You can read more about the functionalities provided by torch [here](https://pytorch.org/tutorials/recipes/recipes/saving_and_loading_a_general_checkpoint.html) in order to save and load model weights. Our abstractions for saving and loading model checkpoints can be found in `./utility/checkpoint`- The abstractions (alongside their implementation) that calculates NLP metrics like BLEU, ROUGE, BERT. All the logics can be found in `utility/metrics`.

 dataThis component encompasses the classes required for reading a dataset adhered to torch dataset contracts (read more about torch datasets [here](https://pytorch.org/tutorials/beginner/basics/data_tutorial.html)). It also provides the functionalities to apply operations like tokenizing, summarizing, and the information retrieval (required for few-shot learning) on the datasets. All the implementations can be found categorized under the `./data` path.

 modelsThis component implements the logics for the CNN-RNN model alongside the model which is augmented with code metrics (so-called CNN-RNN MA in our paper). Although never utilized in our final experiments, a model called AConvGNN is also implemented in this package which is the non-hierarchical version of HAConvGNN model proposed by X. Liu et al. (2021) (You can find their works on [here](https://aclanthology.org/2021.findings-emnlp.381/)).

As stated, all models are implemented purely using PyTorch and further trained on Kaggle T4x2 GPUs using CUDA. The implementations can be found in `./models`.

 experimentsThis component provides the classes that receive different traditional machine learning models and conduct an experiment of training and testing on them. All input models will be under the same experimental conditions (same train, evaluation and test sets, same number of epochs, etc.). The result of experiments can be used for comparing the input models. All implementations for these abstractions reside in `./experiments`.

 promptsThis component provides the functionality needed for using the information retrieval provided in `./data`, generating a prompt for some specific transformer-based model, and prompting the model. Different transformer-based models can be found in `./prompts/llm_api`; they are either imported models from HuggingFace or classes that do some api call. All the implementations for this part of our experiments can be found in `./prompts`.

 notebooksThis notebook implements our LLM-Based Model Architecture. In the notebooks, we first download and import required libraries, setup proper device for computation, configure and load our dataset. Then we do feature extraction and dataset preparation and after that, we do the k-fold cross-validation. The we generate tasks and prompts for our llm model, and use openai api to interact with gpt3.5 model. Finally, we save the results in  a json file.

 datasetour dataset exist as dataset.csv contains 22099 pairs of (code, markdown) after a rigorous filtering and processing approach.

dataset fields:



project_ID: identifier of project on Kaggle

cell_index: index of cell in notebook

source: code cell source

markdown: markdown cell content

markdown_text: markdown processed