Package: picire
Parallel Delta Debugging FrameworkPicire(pronounced as /pitsirE/) is a Python implementation of theDelta Debuggingalgorithm supporting parallelization and further
configuration options. It can be used either as a command line tool
or as a library.Just like the original algorithm,Picireautomatically reduces “interesting”
tests while keeping their “interesting” behaviour. A common use case is
minimizing failing tests so that they still reproduce the original failure.The tool (and the algorithm) works iteratively. As a first step, it splits up
the input intonchunks either by lines or characters. Then, iteratively,
it inspects smaller test cases composed of these chunks whether they are still
interesting. The selection of chunks can happen two ways: either a small subset
of the chunks is kept (subset-based reduce), or that small subset is removed
and everything else is kept (complement-based reduce). If a new interesting
test case is found, it becomes the input of the next iteration. The iterations
stop if removing any further chunks would make the test uninteresting (e.g. the
test is1-minimal).RequirementsPython>= 3.5InstallTo usePicirein another project, it can be added tosetup.cfgas an
install requirement (if usingsetuptoolswith declarative config):[options]install_requires=picireTo installPiciremanually, e.g., into a virtual environment, usepip:pip install picireThe above approaches install the latest release ofPicirefromPyPI.
Alternatively, for the development version, clone the project and perform a
local install:pip install .UsagePicirehas two mandatory command line arguments: one that defines the input
test case to be reduced (--input) and another describing an executable
tester script or program (--test) that can decide about the interestingness
of an arbitrary input. This will be run in every iteration to check a test case.Common settings--parallel: EnablesPicireto run in multiprocess mode. (Otherwise, the
original single-process variant will run.)-j<num>: Defines the maximum number of parallel jobs.--combine-loops: The base algorithm had a dependency between subset and
complement-based reduce loops, but because of the sequential nature of its
implementation, it had no effect on efficiency. However, in parallel mode,
this separation becomes a potential sub-optimality. With this option, the
two reduce loops run combined for additional performance. Further details
about the algorithm variants are available in the cited papers.--complement-first: For some input types, subset-based reduce is not as
effective as the complement-based one (sometimes, aggressively removing too
big parts of the input eliminates the interestingness as well). By default,Picireperforms subset-based reduce before complement-based reduce, which
can result in many superfluous checks for such inputs. This flag forces to
start with complement checks.--subset-iterator/--complement-iterator: Guide the iteration
strategies of the subset and complement-based reduce loops.forward: Start investigating subsets (or complements) from the beginning
of the input.backward: Start investigating subsets (or complements) from the end of
the input. The goal is to reduce the number of semantic violations
(assuming that definitions - like variable declarations - appear before
uses).skip: Completely avoids the subset or complement checks (mostly used
with--subset-iterator).For the detailed options, seepicire--help.Tester scriptThe tester script is expected to take one command line argument, the path of a
test case, and it has to exit with 0 if the test is interesting and with
non-zero otherwise. An example tester script that runs an arbitrary target
application and checks if it fails on an assertion might look like the one
below:#! /bin/bash
timeout --foreground 10 <path/to/the/target/application> $1 2>&1 | grep -q "Assertion failed";Remarks:The<path/to/the/target/application>should either be an absolute path to
the target application or the application should be on the search path (i.e.,$PATH).$1is the single and mandatory command line argument containing the path
of a test case.If the target application is not guaranteed to exit, then it’s worth running
it withtimeoutto limit the amount of time waiting for producing the
expected behaviour.If the target is run with timeout then the--foregroundflag can also be
useful as it allows forwarding theKILLsignals (used by the parallel
implementation) through the timeout’s process group. This enables us to
stop all alive parallel processes when a new interesting configuration
is found already.If the interestingness decision is based on the content of the output then
usinggrep(perhaps with-qor--quiet) might be a right choice,
since it returns 0 if the pattern was found and 1 if not. Exactly the
return valuePicireexpects.A common form ofPicire’s usage:picire --input=<path/to/the/input> --test=<path/to/the/tester> \
       --parallel --subset-iterator=skip --complement-iterator=backwardCompatibilityPicirewas tested on:Linux (Ubuntu 14.04 / 16.04 / 18.04 / 20.04)OS X / macOS (10.11 / 10.12 / 10.13 / 10.14 / 10.15 / 11)Windows (Server 2012 R2 / Server version 1809 / Windows 10)Acknowledgement and CitationsThis software uses the delta debugging algorithm as described in:Andreas Zeller. Yesterday, My Program Worked. Today, It Does Not. Why?
In Proceedings of the 7th European Software Engineering Conference Held
Jointly with the 7th ACM SIGSOFT Symposium on the Foundations of Software
Engineering (ESEC/FSE ‘99), volume 1687 of Lecture Notes in Computer Science
(LNCS), pages 253-267, Toulouse, France, September 1999. Springer.https://doi.org/10.1007/3-540-48166-4_16Ralf Hildebrandt and Andreas Zeller. Simplifying Failure-Inducing Input.
In Proceedings of the 2000 ACM SIGSOFT International Symposium on Software
Testing and Analysis (ISSTA ‘00), pages 135-145, Portland, Oregon, USA, August
2000. ACM.https://doi.org/10.1145/347324.348938Further improvements are described in:Renata Hodovan and Akos Kiss. Practical Improvements to the Minimizing Delta
Debugging Algorithm.
In Proceedings of the 11th International Joint Conference on Software
Technologies (ICSOFT 2016) - Volume 1: ICSOFT-EA, pages 241-248, Lisbon,
Portugal, July 2016. SciTePress.https://doi.org/10.5220/0005988602410248Renata Hodovan, Akos Kiss, and Tibor Gyimothy. Tree Preprocessing and Test
Outcome Caching for Efficient Hierarchical Delta Debugging.
In Proceedings of the 12th IEEE/ACM International Workshop on Automation of
Software Testing (AST 2017), pages 23-29, Buenos Aires, Argentina, May 2017.
IEEE.https://doi.org/10.1109/AST.2017.4Akos Kiss. Generalizing the Split Factor of the Minimizing Delta Debugging
Algorithm.
IEEE Access, 8:219837-219846, December 2020. IEEE.https://doi.org/10.1109/ACCESS.2020.3043027Copyright and LicensingLicensed under the BSD 3-ClauseLicense.