Package: memory-time-tracker
Memory time tracker is a simple python tool to track the memory and time requirements of software across both very brief (milliseconds) and large (days) time and memory requirements through adaptative log resolution.The reports are CSV files with two columns, the first one with the time delta and the second one with the required RAM.Tracking upwards to crashThis package handles gracefully also use cases where the tracked software
dies because of OOM or generally crashes by adding a0,0as the last line of the CSV document
it produces when the execution finishes nominally while adding a-1,-1when the execution
finishes with a detectable exception. When there are crashes not detectable through exceptions,
such as machine freezes because of OOM, kernel panics or other things, neither0,0``nor``1,1are (inevitably) written at the end of the CSV.To help distinguish the different possible completion statuses, we have prepared three methods:has_completed_successfullyto detect whether the execution has been completed without hiccups.has_crashed_gracefullyto detect crashes that raised “normal” exceptions.has_crashed_ungracefullyto detect crashes that did not raise “normal” exceptions, such as OOM and core dumps.See more below in the Examples section.RequirementsPlease do note that this package makes use ofproc/meminfo,
so it is strictly compatible only with Linux systems.Like most tracker systems, this one works best if there is a limited amount of noise in the system.
Do not run other software while running the benchmark, or your results may be skewed.Installing packageAs usual, to install this package from Pypi, just run:pipinstallmemory_time_trackerUsage exampleYou can use this package to track the execution of a given method as follows:frommemory_time_trackerimportTracker,has_completed_successfully,has_crashed_gracefully,has_crashed_ungracefully,plot_reportsfromtimeimportsleepimportnumpyasnpdefexample_function():"""Small example of function that takes 2 seconds."""arrays=[]for_inrange(10):arrays.append(np.zeros((10000,10000)))sleep(0.2)defexample_function_which_crashes():"""Small example of function that takes 2 seconds and crashes."""arrays=[]for_inrange(20):arrays.append(np.zeros((10000,5000)))sleep(0.1)raiseValueError("Argh! I'm crashig!")# The path where we will store the logpath1="/tmp/tracker_example.csv"# The path where we will store the log with the crashpath2="/tmp/tracker_example_with_crash.csv"# Create the tracker contextwithTracker(path1):example_function()# Wait between tracking to allow for memory to freesleep(20)# Create the tracker context to handle crashable librariestry:withTracker(path2,verbose=True):example_function_which_crashes()exceptException:passprint("Successful: ",has_completed_successfully(path1),"Crashed gracefully: ",has_crashed_gracefully(path1),"Crashed ungracefully: ",has_crashed_ungracefully(path1))# Successful:  True Crashed gracefully:  False Crashed ungracefully:  Falseprint("Successful: ",has_completed_successfully(path2),"Crashed gracefully: ",has_crashed_gracefully(path2),"Crashed ungracefully: ",has_crashed_ungracefully(path2))# Successful:  False Crashed gracefully:  True Crashed ungracefully:  Falseplot_reports([path1,path2])You canrun it on Colab here.The above example should generate a picture such as this one:Note that there is some noise in the RAM and time measurements as it was executed on COLAB.Authors and LicenseThis package was developed byLuca CappellettiandTommaso Fontanaand is released underMIT License.