Package: memo
Installationpip install memoDocumentationThe documentation can be foundhere.The quickstart guide is foundhere.UsageHere's an example of utility functions provided by our library.importnumpyasnpfrommemoimportmemlist,memfile,grid,time_takendata=[]@memfile(filepath="results.jsonl")@memlist(data=data)@time_taken()defbirthday_experiment(class_size,n_sim):"""Simulates the birthday paradox. Vectorized = Fast!"""sims=np.random.randint(1,365+1,(n_sim,class_size))sort_sims=np.sort(sims,axis=1)n_uniq=(sort_sims[:,1:]!=sort_sims[:,:-1]).sum(axis=1)+1proba=np.mean(n_uniq!=class_size)return{"est_proba":proba}forsettingsingrid(class_size=[5,10,20,30],n_sim=[1000,1_000_000]):birthday_experiment(**settings)The decoratorsmemlistandmemfileare making sure that the keyword arugments and
dictionary output of thebirthday_experimentare logged. The contents of theresults.jsonl-file
and thedatavariable looks like this;{"class_size": 5, "n_sim": 1000, "est_proba": 0.024, "time_taken": 0.0004899501800537109}
{"class_size": 5, "n_sim": 1000000, "est_proba": 0.027178, "time_taken": 0.19407916069030762}
{"class_size": 10, "n_sim": 1000, "est_proba": 0.104, "time_taken": 0.000598907470703125}
{"class_size": 10, "n_sim": 1000000, "est_proba": 0.117062, "time_taken": 0.3751380443572998}
{"class_size": 20, "n_sim": 1000, "est_proba": 0.415, "time_taken": 0.0009679794311523438}
{"class_size": 20, "n_sim": 1000000, "est_proba": 0.411571, "time_taken": 0.7928380966186523}
{"class_size": 30, "n_sim": 1000, "est_proba": 0.703, "time_taken": 0.0018239021301269531}
{"class_size": 30, "n_sim": 1000000, "est_proba": 0.706033, "time_taken": 1.1375510692596436}The nice thing about being able to log results to a file or to the web is that
you can also more easily parallize your jobs! For example now you can use theRunnerclass to parrallelize the function call withjoblib.importnumpyasnpfrommemoimportmemlist,memfile,grid,time_taken,Runnerdata=[]@memfile(filepath="results.jsonl")@memlist(data=data)@time_taken()defbirthday_experiment(class_size,n_sim):"""Simulates the birthday paradox. Vectorized = Fast!"""sims=np.random.randint(1,365+1,(n_sim,class_size))sort_sims=np.sort(sims,axis=1)n_uniq=(sort_sims[:,1:]!=sort_sims[:,:-1]).sum(axis=1)+1proba=np.mean(n_uniq!=class_size)return{"est_proba":proba}# declare all the settings to loop oversettings=grid(class_size=range(20,30),n_sim=[100,10_000,1_000_000])# use a runner to run over all the settingsrunner=Runner(backend="threading",n_jobs=-1)runner.run(func=birthday_experiment,settings=settings,progbar=True)FeaturesThis library also offers decorators to pipe to other sources.memlistssends the json blobs to a listmemfilesends the json blobs to a filememwebsends the json blobs to a server via http-post requestsmemfuncsends the data to a callable that you supply, likeprintgridgenerates a convenient grid for your experimentsrandom_gridgenerates a randomized grid for your experimentstime_takenalso logs the time the function takes to runWe also offer an option to parallelize function calls using joblib. This
is facilitated with aRunnerclass which supports multiple backends.Runner(backend="loky")Runner(backend="threading")Runner(backend="multiprocessing")Check the API docsherefor more information on
how these work.