Package: cmdbench
CMDBenchA quick and easy benchmarking tool for any command's CPU, memory and disk usage.CLI and the library functionalities are both provided.Note: This library is written in, heavily tested and maintained for theLinuxoperating system primarily. ButWindowsandmacOSare also supported. Create an issue in case you run into a problem.InstallTo install the library from this github repository execute the following command in your terminal:pipinstallcmdbenchPython compatibility: 2.7 or >=3.4Table of contentsQuick Start: Command Line InterfaceQuick Start: LibraryMethod 1: EasierMethod 2: More customizableUsage IPython NotebookDocumentationbenchmark_command: methodbenchmark_command_generator: methodBenchmarkResults: ClassBenchmarkDict: ClassNotesmacOSWindowsQuick Start: Command Line InterfaceYou can use the CLI provided by the python package to benchmark any command.In the following demo, the commandnode test.js(a slightly modified version oftest.js) is being benchmarked 10 times, average of resources are being printed and a plot for the command's cpu and memory usage is being saved to the fileplot.png.The output plot fileplot.pngfor the demo will look like:Quick Start: LibraryMethod 1: EasierYou can simply use thebenchmark_commandfunction to benchmark a command.
Benchmarks the commandstress --cpu 10 --timeout 5over 20 iterations. But prints only the first one from the benchmark results.>>>importcmdbench>>>benchmark_results=cmdbench.benchmark_command("stress --cpu 10 --timeout 5",iterations_num=20)>>>first_iteration_result=benchmark_results.get_first_iteration()>>>first_iteration_result{'cpu':{'system_time':0.04,'total_time':49.75,'user_time':49.71,},'disk':{'read_bytes':0,'read_chars':5124,'total_bytes':0,'total_chars':5243,'write_bytes':0,'write_chars':119,},'memory':{'max':2166784,'max_perprocess':1060864,},'process':{'execution_time':5.0,'stderr_data':'','stdout_data':'stress: info: [20773] dispatching hogs: 10 cpu, 0 io, 0 vm, 0 hdd\n\nstress: info: [20773] successful runcompletedin5s\n',},'time_series':{'cpu_percentages':array([0.,0.,824.1,...,889.,998.3,0.])'memory_bytes':array([2166784,2166784,2166784,...,2166784,2166784,1060864])'sample_milliseconds':array([39,54,65,...,4979,4988,4997])},}>>>first_iteration_result.process.execution_time5.0Method 2: More customizableYou can also create one or more BenchmarkResults objects, and add benchmark results to them over time.So you are not forced to perform the benchmarking for the command consecutively when you simply can't.Could be helpful when you are trying to benchmark multiple commands that need to be executed in a certain order consecutively or depend on each other.>>>fromcmdbenchimportbenchmark_command,BenchmarkResults>>>benchmark_results=BenchmarkResults()>>>for_inrange(20):...new_benchmark_result=cmdbench.benchmark_command("stress --cpu 10 --timeout 5")...benchmark_results.add_benchmark_result(new_benchmark_result)...# The for loop above is equivalent to: benchmark_results = cmdbench.benchmark_command("stress --cpu 10 --timeout 5", iterations_num = 20)>>>benchmark_results.get_averages(){'cpu':{'system_time':0.012500000000000002,'total_time':48.468,'user_time':48.45550000000001,},'disk':{'read_bytes':0.0,'read_chars':5124.0,'total_bytes':0.0,'total_chars':5232.4,'write_bytes':0.0,'write_chars':108.4,},'memory':{'max':2094080.0,'max_perprocess':1020928.0,},'process':{'execution_time':5.0,'stderr_data':None,'stdout_data':None,},'time_series':{'cpu_percentages':array([0.,476.03157895,794.66363636,...,976.05555556,188.97777778,0.])'memory_bytes':array([2093924.84848485,2096074.10526316,2099013.81818182,...,2090552.88888889,1256561.77777778,810188.8])'sample_milliseconds':array([11.42424242,21.73684211,30.90909091,...,4986.44444444,4995.05555556,5000.2])},}Usage IPython notebookFor a more comprehensive demonstration on how to use the library and the resources plot, check the providedipython notebook.Documentationbenchmark_command(command: str, iterations_num = 1, raw_data = False)Argumentscommand: Target command to process.iterations_num: Number of times to measure the program's resources.raw_data: Whether or not to show all different info from different sources like psutil and GNU Time (if available).Returns a BenchmarkResults object containing the related results.benchmark_command_generator(command: str, interations_num = 1, raw_data = False)Arguments: Same as benchmark_commandReturns ageneratorobject allowing you to obtain a BenchmarkResults after each iteration of benchmarking until done (useful for monitoring the progress and recieving benchmarking data on the go).BenchmarkResults: ClassMethods:get_first_iteration()Returns the first iteration result in the benchmark results object.get_iterations()Returns the result for all of the iterations in the benchmark results object.get_values_per_attribute()Returns object containing lists for each type of value over different iterations.get_averages()Returns the average for all types of value over different iterations. Also calculates the average of the time series data.get_statistics()Returns different statistics (mean, stdev, min, max) for all types of values over different iterations.get_resources_plot(width: int, height: int)Returns matplotlib figure object of CPU and Memory usage of target process over time which can be viewed in an ipython notebook or be saved to an image file.add_benchmark_result(adding_result: BenchmarkResults)Adds another BenchmarkResults object's benchmark results iterations' data to the current object.BenchmarkDict: Class(defaultdict)A custom internal dictionary class used to represent the data for an iteration.Data inside objects from this class are accessible through both dot notationobj.keyand key accessobj["key"]NotesWindowsWhen benchmarking on windows, you will need to wrap your main code around theif __name__ == '__main__':statement.MacOSMacOS does not allow process-specific disk usage information collection, therefore disk usage will not be reported on macos when you perform benchmarking.