Package: pybe
Visualization//DocumentationPyBe - benchmark your Python functionsBenchmark any (Python) function, store (as csv or Excel), read andvisualizethe results with only a few lines of code!Table of Contents:Structure of a benchmarkInstallationGetting startedStructure of csvStructure of a benchmarkThe general structure of a benchmark script is as follows:you have some algorithmyou want to test the algorithm by varying over a set of inputsyou assess quantities of interest (i.e. some performance metric) to the output of the algorithmThis can be implemented as a Python function:def benchmark_function(input):
    result = algorithm(input)
    return {"name_performance_metric_1": performance_metric_1(result),"name_performance_metric_2": performance_metric_1(result),...}In order to benchmark your algorithm you simply need to call the above function to all sets of inputs.
This andstoring your resultsis taken care of by the Benchmark class in pybe.benchmark.
Lets look at a concrete example.Example: Optimization algorithmLets say you have an optimization algorithm implemented in Python
which takes as inputsa function to be optimized andthe number of runs.You want to evaluate the optimizer on a certain test function and benchmark how well the optimizer
performs for specific number of runs.
For this you have a performance metric which can be called to the output of the optimization and returns
a real number (float).Then, your benchmark function looks as follows:def benchmark_function(number_of_runs):
    result_optimizer = optimizer(test_function,number_of_runs)
    return {"name_performance_metric": performance_metric(result_optimizer)}Lets say you want to benchmark your optimization algorithm for number of runs 10,100 and 1000.
Now, you can simply benchmark your optimization algorithm by using the pybe Benchmark class.from pybe.benchmark import Benchmark
benchmark = Benchmark()
benchmark(function=benchmark_function,inputs=[10,100,1000],name="name_of_my_optimization_algorithm")Drag the resultingname_of_my_optimization_algorithm.csvinto theDashboardand thats it!InstallationThe official release is available at PyPi:pip install pybeYou can clone this repository by running the following command:git clone https://github.com/nicolaipalm/pybe
cd pybe
pip installGetting startedIn order to benchmark a Python function you only need to implement the function and
specify some data of the benchmark.frompybe.benchmarkimportBenchmarkfrompybe.wrappersimporttimerimporttimebenchmark=Benchmark()# initialize pybe's benchmark class@timer# additionally track the time needed in each iterationdeftest_function(i:int):time.sleep(0.1)return{"name_of_output":i}# specify the output in a dictionary# benchmark test_function on inputs [1,2,3] and evaluate each input 10 timesbenchmark(test_function,name="test_benchmark",# set the name of the benchmarkinputs=[1,2,3],store=True,# store the benchmark resultsnumber_runs=10)Look at the benchmark.csv file in your directory!You can view the results also directly in Python or write them to an Excel or csv fileprint(benchmark.inputs,benchmark.name_outputs)# print inputs and names of outputsprint(benchmark.result)# print results as stored in benchmark.csvbenchmark.to_excel(name="my_results")# write results as excelbenchmark.to_csv(name="my_results")# write results as csvYou can read any of the benchmark results by simply initializing the
benchmark class with parameter the .yaml benchmark file pathbenchmark=Benchmark(benchmark_file_path)Structure of benchmark csvThe structure of the resulting csv is supposed to be very intuitive:each row represents one call of the benchmarked function withone column for the inputone column with the name of the benchmarkone column for each outputFor example:the function has two outputs: time and valueis benchmarked at inputs 10 and 100has name Hellois evaluated once for each inputThen, the resulting csv/Excel has the following structure:valuetimeInputName00.1110hello10.0520100helloDashboardYou can easily visualize your benchmark results!