Package: timerit
TimeritA powerful multiline alternative to Python’s builtintimeitmodule.Docs are published athttps://timerit.readthedocs.io/en/latest/but this README
and code comments contain a walkthrough.Githubhttps://github.com/Erotemic/timeritPypihttps://pypi.org/project/timeritReadTheDocshttps://timerit.readthedocs.io/en/latest/DescriptionEasily do robust timings on existing blocks of code by simply indenting
them. There is no need to refactor into a string representation or
convert to a single line.InstallationpipinstalltimeritInteractive UseThetimeritlibrary provides a succinct API for interactive use:>>>importtimerit>>>for_intimerit:...sum(range(100000))Timedfor:288loops,bestof5timeperloop:best=616.740µs,mean=668.933±124.2µsCompare totimeit:$python-mtimeit'sum(range(100000))'500loops,bestof5:721usecperloopBy default, any code within the loop will be repeatedly executed until at least
200 ms have elapsed.  The timing results are then printed out.Here’s what each of the numbers means:“288 loops”: The code in the loop was run 288 times before the time limit was
reached.“best of 5”: Consider only the fastest of every 5 measured times, when
calculating the mean and standard deviation.  The reason for doing this is
that you can get slow times if the something in the background is consuming
resources, so you’re generally only interested in the fastest times.  This
idea is also described in thetimeitdocs.“best=616.740 µs”: How long the fastest iteration took to run.  For the reasons
described above, this is usually the most consistent number, and the primary
number you should focus on.“mean=668.933 ± 124.2 µs”: The mean and the standard deviation of the “best of 5”
iterations.  This statistic is usually not as robust or useful as the fastest
time, but sometimes its helpful to know if there’s high variance.The loop variable can be used as a context manager to only time a part of each
loop (e.g. to make the timings more accurate, or to incorporate a setup phase
that is not timed):>>>fortimerintimerit:...n=100*1000...withtimer:...sum(range(n))Timedfor:318loops,bestof5timeperloop:best=616.673µs,mean=617.545±0.9µsIt is also possible to provide arguments controlling how the timing
measurements are made.  See the online documentation for more information on
these arguments, but the snippet below runs for exactly 100 iterations, instead
of however many fit in 200 ms.>>>for_intimerit(num=100):...sum(range(100000))Timedfor:100loops,bestof5timeperloop:best=616.866µs,mean=619.120±5.3µsAutomatic ImportIf you want to maketimeriteven easier to use interactively, you can move
the import to thePYTHONSTARTUPfile.  If defined, this environment variable
gives the path to a python script that will be executed just before every
interactive session.  For example:$exportPYTHONSTARTUP=~/.pythonrc$cat$PYTHONSTARTUPimporttimerit$python>>>for_intimerit:...sum(range(100000))...Timedfor:59loops,bestof3timeperloop:best=2.532ms,mean=3.309±1.0msProgrammatic UseThe timerit library also provides aTimeritclass that can be used
programmatically.>>>importmath,timerit>>>fortimerintimerit:>>>setup_vars=10000>>>withtimer:>>>math.factorial(setup_vars)>>>print('t1.total_time =%r'%(t1.total_time,))Timingfor200loopsTimedfor:200loops,bestof3timeperloop:best=2.064ms,mean=2.115±0.05mst1.total_time=0.4427177629695507A common pattern is to create a singleTimeritinstance, then to repeatedly
“reset” it with different labels to test a number of different algorithms.  The
labels assigned in this way will be incorporated into the report strings that
theTimeritinstance produces.  The “Benchmark Recipe” below shows an example
of this pattern.  So do all of the scripts in theexamples/directory.There is also a simple one-liner that is comparable totimeit’s IPython magic:Compare the timeit version:>>>%timeitmath.factorial(100)564ns±5.46nsperloop(mean±std.dev.of7runs,1000000loopseach)With the Timerit version:>>>Timerit(100000).call(math.factorial,100).print()Timedfor:1loops,bestof1timeperloop:best=4.828µs,mean=4.828±0.0µsHow it worksThe timerit module definestimerit.Timerit, which is an iterable object
that yieldstimerit.Timercontext managers.>>>importmath>>>fromtimeritimportTimerit>>>fortimerinTimerit(num=200,verbose=2):>>>withtimer:>>>math.factorial(10000)The timer context manager measures how much time the body of it takes by
“tic”-ing on__enter__and “toc”-ing on__exit__. The parentTimeritobject has access to the context manager, so it is able to read its
measurement. These measurements are stored and then we compute some statistics
on them. Notably the minimum, mean, and standard-deviation of grouped (batched)
running times.Using the with statement inside the loop is nice because you can run untimed
setup code before you enter the context manager.In the case where no setup code is required, a more concise version of the
syntax is available.>>>importmath>>>fromtimeritimportTimerit>>>for_inTimerit(num=200,verbose=2):>>>math.factorial(10000)If the context manager is never called, theTimeritobject detects this and
the measurement is made in the__iter__method in theTimeritobject
itself. I believe that this concise method contains slightly more overhead than
the with-statement version. (I have seen evidence that this might actually be
more accurate, but it needs further testing).Benchmark Recipeimportubeltasubimportpandasaspdimporttimeritdefmethod1(x):ret=[]foriinrange(x):ret.append(i)returnretdefmethod2(x):ret=[iforiinrange(x)]returnretmethod_lut=locals()# can populate this some other wayti=timerit.Timerit(100,bestof=10,verbose=2)basis={'method':['method1','method2'],'x':list(range(7)),# 'param_name': [param values],}grid_iter=ub.named_product(basis)# For each variation of your experiment, create a row.rows=[]forparamsingrid_iter:key=ub.repr2(params,compact=1,si=1)kwargs=params.copy()method_key=kwargs.pop('method')method=method_lut[method_key]# Timerit will run some user-specified number of loops.# and compute time stats with similar methodology to timeitfortimerinti.reset(key):# Put any setup logic you dont want to time here.# ...withtimer:# Put the logic you want to time heremethod(**kwargs)row={'mean':ti.mean(),'min':ti.min(),'key':key,**params,}rows.append(row)# The rows define a long-form pandas data array.# Data in long-form makes it very easy to use seaborn.data=pd.DataFrame(rows)print(data)plot=Trueifplot:# import seaborn as sns# kwplot autosns works well for IPython and script execution.# not sure about notebooks.importkwplotsns=kwplot.autosns()# Your variables may changeax=kwplot.figure(fnum=1,doclf=True).gca()sns.lineplot(data=data,x='x',y='min',hue='method',marker='o',ax=ax)ax.set_title('Benchmark Name')ax.set_xlabel('x-variable description')ax.set_ylabel('y-variable description')