Package: gherald
gheraldA python package to assess the risk of a change during code review.Installationpipinstall-Ugheraldor install withPoetrypoetryaddgheraldUsageRun data pipelineBelow, we show an example of code to run the complete data pipeline, which includes the extraction of bug-fixing and bug-inducing commits from the Apache Commons-lang repo:fromgit.repo.baseimportRepo# extract bug-fixing and bug-inducing commits from Apache commons-langdefrun_data_pipeline(bugs_output_file,commits_output_file,buggy_commits_output_file,bug_inducing_commits_output_file,modified_files_commit_output_file,commit_code_changes_information_output_file,filtered_commits_output_file,filtered_fixing_commits_output_file):apache_commons_lang_suffix="LANG"cloned_repo_path="path/to/cloned/repo"cutoff_date="2022-01-01"branch_name="master"# clone Apache commons-langRepo.clone_from("https://github.com/apache/commons-lang",cloned_repo_path)extract_bugs_from_jira(bugs_output_file,apache_commons_lang_suffix,cutoff_date)extract_commits_from_vcs(commits_output_file,apache_commons_lang_suffix,cloned_repo_path,branch_name)merge_its_vcs_data(buggy_commits_output_file,bugs_output_file,commits_output_file)extract_defect_inducing_commits(bug_inducing_commits_output_file,cloned_repo_path,buggy_commits_output_file)extract_modified_files_commit(modified_files_commit_output_file,commits_output_file)extract_commit_code_changes(commit_code_changes_information_output_file,cloned_repo_path)data_filtering(filtered_commits_output_file,filtered_fixing_commits_output_file,commits_output_file,bug_inducing_commits_output_file,commit_code_changes_information_output_file,)returnrun_data_pipeline("apache_commons_lang_bugs.json","apache_commons_lang_commits.csv","apache_commons_lang_buggy_commits.csv","apache_commons_lang_bug_inducing_commit_ids.csv","apache_commons_lang_commits_modified_files.csv","apache_commons_lang_code_changes.csv","apache_commons_lang_commits_filtered.csv","apache_commons_lang_bug_fixing_commit_ids_filtered.csv")Therun_data_pipelineis a self-written function that calls theextract_bugs_from_jira,extract_commits_from_vcs,merge_its_vcs_data,extract_defect_inducing_commits,extract_modified_files_commit, andextract_commit_code_changesfunctions in order. Below, we explain each of the parameters ofrun_data_pipeline:bugs_output_file: The output file to which bug information will be written.commits_output_file: The output file to which commit information will be written.buggy_commits_output_file: The output file to which bug-fixing commits information will be written.bug_inducing_commits_output_file: The output file to which bug-inducing commits information will be written.modified_files_commit_output_file:  The output file to which filtered commit information will be written.commit_code_changes_information_output_file: The output file to which code change information will be written.filtered_commits_output_file: The output file to which filtered bug-inducing commits information will be written.filtered_fixing_commits_output_file: The output file to which filtered bug-fixing commits information will be written.Insiderun_data_pipeline, we define four constants,apache_commons_lang_suffix,cloned_repo_path,cutoff_date,branch_name, meaning, respectively, the suffix used by the repository in the Jira’s URL from which commit information is extracted (Apache Common-Lang in our example), the local path to the cloned Jira’s repository, the most recent date for which commit information is crawled from the Jira’s repository, and the branch from which commit information is extracted. Finally,Repo.clone_fromclones the repository in the specified folder.Run risk assessmentTherisk_assessmentis a self_written function that assesses the risk of each change after the filtering step.
We use the output files generated byrun_data_pipelineas parameters:commits_file: The file generated by the commits extraction step.filtered_commits_file: The file generated by the commit filtering step.commit_code_changes_file: The file generated by the step that extracts per-commit code changes information.bug_inducing_commit_file: The file generated by the step that extracts bug inducing commits.commit_modified_files: The file generated by the step that extracts modified files information.
Additionally, we store the risk assessment information for each file and method into file_risk_data_output_file and method_risk_data_output_file, respectively.Theprepare_experiment_commit_datais a self-written functions that computes the risk assessment information for the selected experimental changes.
Below, we explain each of the parameters ofprepare_experiment_commit_data:file_risk_data_output_file: The file generated by the risk assessment step, storing the risk assessment information for each file.method_risk_data_output_file: The file generated by the risk assessment step, storing the risk assessment information for each method.experiment_changes_out_file: The output file to which the risk assessment information for the experimental changes will be written.experiment_files_out_file: The output file to which the risk assessment information for the files in each experimental changes will be written.experiment_methods_out_file: The output file to which the risk assessment information for the methods in each experimental changes will be written.experiment_data: The experimental changes data stored in a dictionary, containing the experimental change IDs, the number of defects in each experiment change, etc.Below, we show an example of code to run the risk_assessment and prepare_experiment_commit_data, which includes the risk assessment for changes in the Apache Commons-lang repo and experimental changes selection process:risk_assessment("apache_commons_lang_commits.csv","apache_commons_lang_commits_filtered.csv","apache_commons_lang_code_changes.csv","apache_commons_lang_bug_inducing_commit_ids.csv","apache_commons_lang_commits_modified_files.csv","apache_commons_lang_file_risk_data.csv","apache_commons_lang_method_risk_data.csv")experiment_data={"id":["06aea7e74cfe4a1578cb76672f1562132090c205","9397608dd35a335d5e14813c0923f9419782980a","d844d1eb5e5b530a82b77302f1f284fd2f924be3","c1f9320476ab9e5f262fdf8a5b3e1ff70199aed8","5acf310d08b2bc5182cf936616ef70938cb2c499","16774d1c0d6d8aa4579f7c96b3fdb78bd118e5aa","188933345a7ebad94f74ba0fb6e8bc6eb99552a6","65392be352be6ccc8acf24405d819f60cd0d1a22","4f85c164a1a4eeb8813b61cf46132fb91971b323","eaa9269ac80c2a957cabed0c46173149a4137c24",],"practice":[0,0,0,0,0,1,1,1,0,0],"bug_count":[1.0,1.0,2.0,2.0,1.0,0.0,0.0,0.0,0.0,0.0],}prepare_experiment_commit_data("apache_commons_lang_file_risk_data.csv","apache_commons_lang_method_risk_data.csv","apache_commons_lang_experiment_changes.csv","apache_commons_lang_experiment_files.csv","apache_commons_lang_experiment_methods.csv",experiment_data)DevelopmentInitialize codeInitializegitinside the repo:cdgherald&&gitinitIf you don't havePoetryinstalled run:makepoetry-downloadInitialize poetry and installpre-commithooks:makeinstall
makepre-commit-installRun the codestyle:makecodestyleUpload initial code to GitHub:gitadd.
gitcommit-m":tada: Initial commit"gitbranch-Mmain
gitremoteaddoriginhttps://github.com/filipe-cogo/gherald.git
gitpush-uoriginmainPoetryCheckits documentation.Details about PoetryPoetry'scommandsare very intuitive and easy to learn, like:poetry add numpy@latestpoetry run pytestpoetry publish --buildetcBuilding and releasingBuilding a new version of the package:Bump the version of the packagepoetry version <version>. You can pass the new version explicitly, or a rule such asmajor,minor, orpatch. For more details, refer to theSemantic Versionsstandard.Make a commit toGitHub.Create aGitHub release.And... publish 🙂poetry publish --buildMakefile usageMakefilecontains a lot of functions for faster development.1. Download and remove PoetryTo download and install Poetry run:makepoetry-downloadTo uninstallmakepoetry-remove2. Install all dependencies and pre-commit hooksInstall requirements:makeinstallPre-commit hooks can be installed aftergit initviamakepre-commit-install3. CodestyleAutomatic formatting usespyupgrade,isortandblack.makecodestyle# or use synonymmakeformattingCodestyle checks only, without rewriting files:makecheck-codestyleNote:check-codestyleusesisort,blackanddarglintlibraryUpdate all dev libraries to the latest version using one comandmakeupdate-dev-deps4. Code securitymakecheck-safetyThis command launchesPoetryintegrity checks as well as identifies security issues withSafetyandBandit.makecheck-safety5. Type checksRunmypystatic type checkermakemypy6. Tests with coverage badgesRunpytestmaketest7. All lintersThere is a command to run all linters in one:makelintthe same as:maketest&&makecheck-codestyle&&makemypy&&makecheck-safety8. Dockermakedocker-buildwhich is equivalent to:makedocker-buildVERSION=latestRemove docker image withmakedocker-removeMore informationabout docker.9. CleanupDelete pycache filesmakepycache-removeRemove package buildmakebuild-removeDelete .DS_STORE filesmakedsstore-removeRemove .mypycachemakemypycache-removeOr to remove all above run:makecleanup📈 ReleasesYou can see the list of available releases on theGitHub Releasespage.We followSemantic Versionsspecification.We useRelease Drafter. As pull requests are merged, a draft release is kept up-to-date listing the changes, ready to publish when you’re ready. With the categories option, you can categorize pull requests in release notes using labels.List of labels and corresponding titlesLabelTitle in Releasesenhancement,feature🚀 Featuresbug,refactoring,bugfix,fix🔧 Fixes & Refactoringbuild,ci,testing📦 Build System & CI/CDbreaking💥 Breaking Changesdocumentation📝 Documentationdependencies⬆️ Dependencies updatesYou can update it inrelease-drafter.yml.GitHub creates thebug,enhancement, anddocumentationlabels for you. Dependabot creates thedependencieslabel. Create the remaining labels on the Issues tab of your GitHub repository, when you need them.🛡 LicenseThis project is licensed under the terms of theMITlicense. SeeLICENSEfor more details.📃 Citation@misc{gherald,author={University of Waterloo},title={A python package to assess the risk of a change during code review.},year={2023},publisher={GitHub},journal={GitHub repository},howpublished={\url{https://github.com/filipe-cogo/gherald}}}CreditsThis project was generated withpython-package-template