Package: pyr-benchmark-wrangling
IntroductionPyrepair Benchmark Wrangling is a Python package designed to facilitate and streamline the process of the data-wrangling benchmarks for Automated Program Repair (APR) tools. This package provides an easy-to-use command-line interface to interact with two main components:BugsInPy- a package to run BugsInPy benchmark (https://dl.acm.org/doi/abs/10.1145/3368089.3417943)diff_utils- a set of utilities for handling diffsFeatureslmeasures: A command-line tool to compute and report various metrics and measures for the benchmarks.bgp: A command-line interface to interact with the BugsInPy benchmark suite.sample_bip: A utility to sample bugs from the BugsInPy benchmark suite.run_custom_patch: A tool to apply custom patches to the bugs in the BugsInPy dataset.diff_utils: A command-line utility to handle diff files and changes.InstallationYou can either directly install Pyr Benchmark Wrangling via pip, or Docker.
After cloning the repository, switch to pyr_benchmark_wrangling:cdpyr_benchmark_wranglingDirect InstallationSystem Requirements:Before using the Pyr Benchmark Wrangling, make sure your system meets the following system requirements:Python 3.7 and Python 3.8Development packages for Python 3.7 and Python 3.8libffi7 libraryOn a Debian-based system, you can install these requirements usingapt-get:sudoapt-getinstallpython3.7python3.7-devpython3.8python3.8-devlibffi7Pip Command:You can install Pyr Benchmark Wrangling by running the following command:pipinstall.Using DockerPyr Benchmark Wrangling's Docker Space Requirements:Lite image: 2.8 GBFull image: 20 GBThe difference betweenliteandfullimage is that the virtual environments are lazily constructed inlite, and downloaded infull.To build the lite image (2.8 GB), use the following command:dockerbuild--targetlite-tpyr:lite.This will execute all instructions in the Dockerfile up until the lite stage is complete.
The lite image automatically runsupdate_bug_recordsandclone's all repositoriesTo build the full image (20 GB), use the following command:dockerbuild--targetfull-tpyr:full.This will execute all instructions in the Dockerfile.
The full image automatically runsupdate_bug_records,clone's all repositories, installs all required environments.BugsInPy CLIThe BugsInPy CLI is a command-line tool designed for interacting with and running Python bugs from the BugsInPy dataset. This script streamlines the process of setting up bug repositories, cloning specific bugs or repositories, preparing the environment, running tests, and more. Below, you'll find an overview of the available commands and their functionalities:
This tool requires Python3.10 and aboveUsagesetupCommandThesetupcommand is used to set up the BugsInPy repository. This step is essential before working with any bugs. It clones the BugsInPy repository to your local system.bgpsetupcloneCommandTheclonecommand allows you to clone specific bugs or repositories based on your requirements. You can specify the bugs to clone using the--bug_listflag or repositories using the--repo_listflag.Example:bgpclone--bugidsrepo1:id1,repo2:id2,...,repo3:id3checkout_buggyandcheckout_fixedCommandsThese commands are used to checkout the buggy or fixed version of a specific bug repository. You provide the bug ID to identify the repository.To checkout the buggy version:bgpcheckout_buggy--bugidsrepo:<bug_id>To checkout the fixed version:bgpcheckout_fixed--bugidsrepo:<bug_id>extract_featuresCommandTheextract_featurescommand extracts features of a specific bug.Example:bgpextract_features--bugidsrepo:<bug_id>prepCommandTheprepcommand prepares the environment for a specific bug. It installs the required dependencies and performs sanity checks to ensure the bug can be tested successfully.
The commandssetupandcloneshould be run beforeprepExample:bgpprep--bugidsrepo:<bug_id>run_testCommandTherun_testcommand runs the tests for a specific list of bugs. It executes the test commands associated with the bug.
The commandssetup,cloneandprepshould be run beforerun_test.bgprun_test--bugidsrepo:<bug_id>delete_bug_repoCommandThedelete_bug_repocommand deletes a specific bug repository from your local system.Example:bgpdelete_bug_repo<bug_id>Additional NotesThe CLI provides options to control the verbosity of the prep step and set the log level.Mutually exclusive flags such as--bug_list,--repo_listare available for listing bugs/repos on which the commands should be run.You can adjust the timeout for various system calls using the--timeoutflag.Unsupported reposThe following repositories are un-supported:Spacy: Due to the requirement of python version < "3.4"diff_utilsdiff_utilsis a Python module for analyzing and extracting data from unified diff outputs generated by tools such as Git. The module provides functionalities to compute localization measures on single file diffs, across multiple file diffs, and extract the modified line numbers and file names from diffs.FeaturesCompute hunk statistics such as count, gaps, and spans from a single file diff.Aggregate hunk information across multiple file diffs to calculate comprehensive statistics.Extract modified files and their respective line changes from a unified diff.Write the extracted data to CSV files for further analysis.UsageThediff_utilsmodule provides a set of functions that can be used independently or through a command-line interface.Command-line InterfaceThe module can be run as a script to perform actions based on the arguments provided:--measure: Accepts comma-delimited list of diff files to measure localization metrics.--locations: Accepts comma-delimited list of diff files from which to extract location sets.--quiet: Quiet mode, which suppresses the standard output.--output: Specifies the output CSV file name.Example Usage:>diff_utils--measure"diff_file1.txt,diff_file2.txt"--output"measures_output.csv">diff_utils--locations"diff_file1.txt,diff_file2.txt"--output"locations_output.csv"Module FunctionsYou can also use the functions provided bydiff_utilsin a Python script:fromdiffutilsimportmeasure_localisation_diff_file,locations_from_diff_file# Measure localization metrics for a given diff filemetrics=measure_localisation_diff_file("diff_file.txt")# Extract locations from a diff filelocations=locations_from_diff_file("diff_file.txt")