DataRecipe
Creators: name: Kim, Kisub
name: Kim, Jounghoon
Publisher: name: figshare
Year: 2024
Subjects: classificationCode: 461201
subject: Automated software engineering

description: *This is a public release for a study related to Data Quality on training CodeLLMs for Code Generation.<b>Replication Package</b><br>Basic Setup<br>&gt; Install torch (https://pytorch.org/get-started/locally/) based on your device type, and install the other packages using:<br>&gt;&gt; pip install -e .<br>Dataset Download<br>&gt; Each dataset has to be downloaded to local directories to load.<br><br>The datasets can be found at:<br>MBPP: https://huggingface.co/datasets/google-research-datasets/mbpp<br>CoNaLa: https://huggingface.co/datasets/neulab/conala<br>CodeAlpaca: https://huggingface.co/datasets/antolin/codealpaca-filtered<br>DS-1000: https://huggingface.co/datasets/xlangai/DS-1000<br>ODEX: https://huggingface.co/datasets/loubnabnl/odex-data<br>Information to replicateTo replicate:<b>fine-tuning</b> each dataset, we used:<b>Commands</b>:<br>CUDA_VISIBLE_DEVICES= GPU_NUM \<br>python finetuning/train.py \<br>--model_ckpt ...../MODEL/PATH \<br>--dataset DATASET_PATH \<br>--max_length 512 \<br>--num_epochs 50 \<br>--optim OPTIMIZER \<br>--learning_rate LEARNING_RATE \<br>--weight_decay WEIGHT_DECAY \<br>--eval_freq EVAL_FREQ \<br>--batch_size BATCH_SIZE \<br>--gradient_accumulation_steps<br>performing <b>inference</b> using the checkpoints (i.e., fine-tuned models), we used:<b>./runs/dataset_name.sh</b> script files.we provide the example of hyperparameters inside script files but it should be considered when the experimental environment is different.the source code we primarily used:train.py: the main code for training/validating.main.py: the main code for testing.utils.py: the utils and processing-related code.<br><br><b>Characteristic Analysis</b> <b>Setup</b> Download the package and uncompress the .zip file. Install the libraries. <br>[command] <b>pip install -r requirements.txt </b>Instruction for each analysis Analysis based on conventional NLP techniques. <br>[command] <b>python nlpAnalysis.py </b><br>[results] The results will be in nlpAnalysis_result.csv Code-side issue extraction by using SonarQube. <br>[Instruction] <br>- Install SonarQube <br>- Create projects for each dataset <br>- Use <b>convertFromOriginalToPy.py</b> to convert the original dataset to a complete Python file <br>- Upload the input files to SonarQube <br>- Retrieve the SonarQube analysis report <br>[command] <b>python convertFromOriginalToPy.py </b><br>[results] The output files will be saved in the <i>dataset_before_refactoring</i> directory. Automated code refactoring <br>[command] <br>1) cp -r dataset_before_refactoring dataset_after_refactoring <br>2) autopep8 --in-place.py <br>3) black.py <br>4) yapf -i.py <br>5) autoflake --in-place --remove-unused-variable.py <br>6) docformatter --in-place.py <br>7) unify --in-place.py <br>[results] The output files will be saved in <i>dataset_after_refactoring</i> directory. Understandability measurement <br>[command] <b>python pylintUnderstandabilityScore.py</b> <br>[results] The results will be retrieved in <i>UnderstandabilityBeforeRefactoring.csv and UnderstandabilityAfterRefactoring.csv</i><br>