Package: simple-metrics-manager
Simple Metrics Manager facilitates managing metrics.A "metric" as defined here consists of:A string name (and an associated python constant)A function that takes no arguments and returns some dataThe data returned from the metric functionStorageInterfacesAStorageInterfaceis a class that supports storing metric data using some
persistent backend. The current default is the fairly generic '.npz' format
used to store numpy arrays and simple python objectsNpzStorageInterface.To prevent data corruption, it always saves data to a temp file and then
moves it into the real file (potentially replacing any older copies).It has a simple save/load/exists API.Pre-defined StorageInterface classes:JsonStorageInterfaceusesjson.loadandjson.dumpand
automatically coerces numpy arrays to lists.NpyStorageInterfacejust usesnp.saveandnp.loaddirectly.NpzStorageInterfaceuses numpy'snp.loadand (hidden)_savezmethod.
It and does some simple checks to allow saving arrays as well as lists
of arrays and dictionaries of arrays.This is very fast, efficient, reliable, and gives quite a bit of flexibility.The option toallow_pickle=Falsemeans that saving object arrays will fail,
so change it toTrueif you need it, but pickle is considered volatile,
so changes to python or relevant package versions might mean your data
is unreadable. This is super convenient though, so it hasn't stopped big
packages like scikit-learn from relying on it.You could also easily add a pure pickle-based storage interface yourself
withpickle.loadandpickle.dump, but I don't include it here for
the reasons listed above :)Cache ManagersADatedCacheManageruses aStorageInterfaceand supports the following:Automatic caching to both memory and persistent storage,Automatic cache utilization, falling back on memory cache and then
persistent cache (override withforce=False)Dating of all metrics using "side-car" metrics (*_date)Printing of all major actions (disable withverbose=False)It has the following API:set_functions_dictSet the core data for the manager, a dictionary that actually defines the metrics.
Keys are metric names and values are metric functions (no arguments).
By calling this after\__init__, the manager itself can be used within metric functions.
See "Usage" below.existsBoolean, whether the metric is in the memory cache.clear_cacheRemove all metrics from the cache.computeCall a metric function.
Caches and returns the data.saveCompute a metric and save the result with the StorageInterface.
Caches to memory and disk and returns the data.loadLoad a metric (assumed to exist)
Tries the cache, then the StorageInterface, fails otherwise.
Returns the datagetCall save and then load.
Caches to disk and theStorageInterface.
Returns the cached data.
Overloaded with[](aka__getitem__).In addition,ParameterizedDatedCacheManageradds a powerful decorator
calledcollectthat collects metrics automatically.Basic usage:CM = DataCacheManager(NpyStorageInterface(SOME_DIRECTORY))

@CM.collect
def metric_1():
    stuff = do_something()
    return stuff

@CM.collect
def metric_2():
    stuff = do_something_else(CM.metric_1())  # <- use of the cached version here
    other_stuff = convert(stuff)
    return other_stuffNow calls toCM.metric_1()andCM.metric_2()seamlessly use the
cache (both in-memory and on disk).This allows for complex dependencies to be handled automatically
and efficiently.If you prefer,metric_1(cache=True)is equivalent toCM.metric_1().
Both will invokeCM['metric_1']which in turn will invoke the
"undecorated"metric_1()as it appears above when it does not exist.Passinguse_stored=Falseon any of these will invalidate the
cache and overwrite it with the new value.The original function (called withoutcache=True) will be unchanged.Example:metric_1()will intentionallyskipthe cache and just run
like normal.Parameterized usage:CM = ParameterizedDataCacheManager(NpyStorageInterface(SOME_DIRECTORY))

@CM.collect(params_list=[[1], [4]])
def poly_X(a):
    return do_something_hard(a)

@CM.collect(params_list=[1, 2], [1, 3], [4, 5])
def poly_Y(a, b):
    return CM.poly_X(a) ** 2 + do_something_else(b)CM.poly_Y(1, 2)will runpoly_Y(1, 2)which will runCM.poly_X(1)which will runpoly_X(1)Then another call toCM.poly_Y(1, 2)will loadCM['poly_Y_1_2']and return itA call topoly_Y(1, 3)will runpoly_Y(1, 3)which will runCM.poly_X(1)which will loadCM['poly_X_1]The point is that ifpoly_Xorpoly_Ytake a long time, subsequent calls
will be fast.Pre-running everything can be done like so:CM.poly_X.get_all()
CM.poly_Y.get_all()which is handy, for instance, if you need to run a large set of
computations overnight.The above will fail if you pass undefinied parameters (which is good if
you want to ensure you have pre-cached everything you might invoke).If youreallywant to call the functions in the cache manager with any
parameters, just add this option to the code above:CM = ParameterizedDataCacheManager(NpyStorageInterface(SOME_DIRECTORY),
                                   dynamic_metric_creation=True)With this change, things likeCM.poly_Y(12, 13)will also work.Finally, to reiterate, the original functionspoly_Xandpoly_Ywill
still work with any parameters regardless of this setting because they
areunchangedunless you specifycache=True.Manual usage of DatedCacheManager:METRIC_1 = 'metric_1'
METRIC_2 = 'metric_2'

CM = DataCacheManager(NpyStorageInterface(SOME_DIRECTORY))

def metric_1_function():
    stuff = do_something()
    return stuff

def metric_2_function():
    stuff = do_something_else(CM[METRIC_1])
    other_stuff = convert(stuff)
    return other_stuff

FUNCTIONS_DICT = {
  METRIC_1: metric_1_function,
  METRIC_2: metric_2_function,
  ...
}

CM.set_functions_dict(FUNCTIONS_DICT)Then in some other place invoke:CM[METRIC_1]orCM[METRIC_2]This has the same effect as the "Basic usage" example above but with a
lot more boilerplate. You can still use this if you despise decorator
magic though ;)