Package: temci
An advanced benchmarking tool written in Python 3 that supportssetting up an environment for benchmarkingand the generation ofvisually appealing reports.It runs on Linux systems and (rudimentarily) on macOS.Why should you use temci?temci allows you to easily measure the execution time (and other things)
of programs and compare them against each other resulting in a pretty
HTML5 based report. Furthermore it can set up the environment to ensure
benchmarking results with a low variance. The latter feature can be used
without using temci for benchmarking
by usingtemci short shell.UsageThe main commands of temci aretemci execandtemci report.Suppose you want to see whether grepping for the strings that consist ofaandbin the current
folder is slower than for strings that consist only ofa.First we have to install temci (usingNix, see below for more instructions):nix-env-fhttps://github.com/parttimenerd/temci/archive/master.tar.gz-iAfter this, we can benchmark both commands with temci:# benchmark both commands 20 timestemcishortexec"grep '[ab]*' -R .""grep 'a*' -R ."--runs10# append --watch to get report (in which you can move with the arrow keys and scroll)
# after every benchmark completed (use --watch_every to decrease interval)temcishortexec"grep '[ab]*' -R .""grep 'a*' -R ."--runs10--watch# if you want to improve the stability your benchmarks, run them with root privileges
# the benchmarked programs are run with your current privilegestemcishortexec"grep '[ab]*' -R .""grep 'a*' -R ."--runs10--sudo--presetusableThis results in arun_output.yamlfile that should look like:-attributes:{description:'grep''[ab]*''-R.'}data:etime:[0.03,0.02,0.02,0.03,0.03,0.03,0.02,0.03,0.03,0.02]…# other properties-attributes:{description:grep 'a*' -R .}data:etime:[0.02,0.03,0.02,0.03,0.03,0.02,0.03,0.03,0.02,0.02]…# other properties-property_descriptions:{etime:elapsed real (wall clock) time,…}For more information on the support measurement tools (likeperf statandrusage),
the supportedplugins for setting up the environmentand more, seetemci exec.We can now create a report from these benchmarking results usingtemci report.
We use the option--propertiesto include only the elapsed time in the
report to keep the report simple:>temcireportrun_output.yaml--propertiesetimeReportforsinglerunsgrep'[ab]*'-R.(10singlebenchmarks)etimemean=2(6).(000)m,deviation=18.84223%grep'a*'-R.(10singlebenchmarks)etimemean=2(5).(000)m,deviation=20.00000%Equalprogramblocksgrep'[ab]*'-R.⟷grep'a*'-R.etimeconfidence=67%,speedup=3.85%We see that there is no significant difference between the two commands.There are multiple reporters besides the defaultconsole reporter.
Another reporter is thehtml2 reporterthat produces an HTML report, use it by adding the--reporterhtml2option:InstallationThe simplest way is to use theNix package manager, after installing Nix, run:nix-env-fhttps://github.com/parttimenerd/temci/archive/master.tar.gz-iUsing pip requiring at least Python 3.6:sudopip3installtemciFor more information see theInstallationpage.Auto completionTemci can generate auto completion files for bash and zsh. Add the following line to your.bashrcor.zshrc:.`temci_completion$0`Using temci to set up a benchmarking environmentUse thetemci short shell COMMANDto run a command (shby default) in a shell that is inside
the benchmarking environment. Most options oftemci short execare supported.
For more information, seetemci shell.Why is temci called temci?The problem in naming programs is that most good program names are
already taken. A good program or project name has (in my opinion) the
following properties:it shouldn’t be used on the relevant platforms (in this case: github and pypi)it should be short (no one wants to type long program names)it should be pronounceableit should have at least something to do with the programtemci is such a name. It’s lojban for time (i.e. the time duration between two moments or events).ContributingBug reportsandcode contributionsare highly appreciated.For more information, see theContributingpage.