Package: runperf
============
IntroductionThis project should help with executing the same tasks on pre-defined
scenarios/profiles. In case the tasks provide pbench-like json results
it also allows tools to analyze and compare the results with main
focus on monitoring performance in time.The main usecase of this tool is a performance regression CI.SetupRun-perf is available frompipso one can install it by executing::python3 -m pip install runperfor to install directly the latest version from git::python3 -m pip install git+https://github.com/distributed-system-analysis/run-perf.gitFor development purposes please check-out the :any:clone-and-deploysection.Componentsrun-perf      => run perf test(s) and report resultscompare-perf  => compare 2 or more runperf results together reporting
human as well as machine readable output optionally supporting model
to smooth the comparisonsanalyze-perf  => calculate a model based on one or multiple resultsBasic usageExecuteuperfandfio(with custom params) on machinefoo.example.orgthat will be provisioned viabeakerto Fedora-32. Execute the tests
underLocalhost(directly on thefoo.example.orgmachine) andTunedLibvirt(configures host, fetches guest image, configures it and
spawns guest VM) profiles and report results in./result_$datedirectory::run-perf -vvv --hosts foo:foo.example.org --provisioner Beaker --distro Fedora-32 --default-password password --profiles Localhost TunedLibvirt -- uperf fio:'{"type":"read", "ramptime":"1", "runtime":"10", "samples":"1", "file-size": "100", "targets": "/fio"}'Processresult*directories, compare the ranges and create alinear modelthat normalizes the ranges to<-3, +3>range::analyze-perf -vvv -l model1.json -t 3 -- result*Comparesrcanddstresults usingmodel1.jsonlinear model and report
the comparison in human readable form to the console, in XUNIT format inresult.xmlfile and as a standalone html page inresult.html. For
some tasks theresult*results are also added as reference for better
visualization of the changes::compare-perf -vvv --tolerance 5 --stddev-tolerance 10 -l model1.json --xunit result.xml --html result.html -- src result* dst.. image::https://github.com/distributed-system-analysis/run-perf/actions/workflows/ci.yml/badge.svg:target:https://github.com/distributed-system-analysis/run-perf/actions/workflows/ci.yml:alt: Github Actions CI.. image::https://readthedocs.org/projects/run-perf/badge/?version=latest:target:https://run-perf.readthedocs.io/en/latest/?badge=latest:alt: Documentation Status.. image::https://img.shields.io/lgtm/alerts/g/distributed-system-analysis/run-perf.svg?logo=lgtm&logoWidth=18:target:https://lgtm.com/projects/g/distributed-system-analysis/run-perf/alerts/:alt: LGTM alerts.. image::https://img.shields.io/lgtm/grade/python/g/distributed-system-analysis/run-perf.svg?logo=lgtm&logoWidth=18:target:https://lgtm.com/projects/g/distributed-system-analysis/run-perf/context:python:alt: LGTM Python code quality.. image::https://api.codeclimate.com/v1/badges/5a2ca7137e0094c24c18/maintainability:target:https://codeclimate.com/github/distributed-system-analysis/run-perf/maintainability:alt: Maintainability.. image::https://api.codeclimate.com/v1/badges/5a2ca7137e0094c24c18/test_coverage:target:https://codeclimate.com/github/distributed-system-analysis/run-perf/test_coverage:alt: Test Coverage