Package: cache.me
A library for caching the outputs of functions based on the inputted parameters to reduce recomputing expensive
computations and requesting frequently accessed content, and to improve performance.@cache(StaticCache(),on_hit=lambdah:print('Hit%stime(s)'%h),on_miss=lambdam:print('Missed%stime(s)'%m))deffibonacci(n):ifn<2:returnnreturnfib(n-1)+fib(n-2)InstallationThe latest version of cache.me is available viapip:pipinstallcache.meAlternatively, you can download and install from source:pythonsetup.pyinstallGetting StartedThecachedecorator contains the following signature:@cache(algorithm,include_types=False,on_hit=None,on_miss=None,key_func=None)deffunc(...)...It serves as both a function decorator, and a runnable wrapper and is configurable through it’s dynamic parameters.In its simplest form, the decorator accepts different algorithms that can be used to cache the outputs of the function
based on its inputs, otherwise known asmemoization. Different algorithms are provided to perform this memoization
for different access patterns, as described below.The creation of the key is based on the arguments and keyword arguments passed in. The type information can also be used
in the formation of the key by setting theinclude_typesparameter to True. In addition, the arguments used for the
function can be altered before use for the key creation via thekey_funcparameter. A common use case for
this is to remove a parameter since as a timer or logging object from the key creation. The function should accept the
same arguments as the calling function and return a tuple of the arguments and keyword arguments to use to create the
key.defkey_changer(x,logger):return(x,),{}@cache(RandomCache(size=30),key_func=key_changer)deffunc(x,logger)...Two callback functions,on_hitandon_miss, can be passed into the function to be called when either a cache hit
or cache miss occurs. For the most typical use cases, these are passed either the number of hits or number of misses
occurred.Bound Function MethodsIn addition to caching the function, other methods are bound to the function for interacting with the cache. The
following methods are added to the decorated function:cache_info(): Returns a tuple of information about the cache containing the number of hits, the number of misses, the current size, and the maximum size.cache_clear(): Clears the cache along with its statistics.@cache(NMRUCache(30))deffunc(...)...# Clear the cachefunc.cache_clear()# Grab cache informationhits,misses,current_size,max_size=func.cache_info()Caching AlgorithmsThe following caching algorithms are provided by the library (although others could be extended from theBaseCache):FIFO (First-in First-out)LIFO (Last-in First-outLFU (Least Frequently Used)LRU (Least Recently Used)MQ (Multi-Queue)MRU (Most Recently Used)NMRU (Not Most Recently Used)RR (Random Replacement)SLRU (Segmented Least Recently Used)StaticTLRU (Time-aware Least Recently Used)TwoQ (simple 2Q or Two-Queue)TwoQFull (full 2Q or Two-Queue)While each of these can be fed into thecachedecorator, they can also be used on their own by simply creating an
instance and calling the appropriate methods. Each implemented algorithm has an O(1) time complexity for accesses and
insertions, and have the following methods and properties:current_size: The current size of the cache.hits: The number of cache hits.max_size: The maximum size of the cache.misses: The number of cache misses.clear(): Clears the items in the cache.get(key, sentinel): Gets an item in the cache.put(key, value): Retrieves an item in the cache.dynamic_methods(): Provides dynamic binding of methods forcachedecorator.create_key(...): Creates a cache key.FIFO (First-in First-out)TheFIFOCacheis a First-in First-out cache where keys are evicted in order of arrival when the cache is full.
Accessing a key does not change the order of eviction.@cache(FIFOCache(size=50))deffunc(...)...LIFO (First-in First-out)TheLIFOCacheis a Last-in First-out cache where keys are evicted in reverse order of arrival when the cache is
full. Accessing a key does not change the order of eviction.@cache(LIFOCache(size=50))deffunc(...)...LFU (Least Frequently Used)TheLFUCacheis a Least Frequently Used cache where keys which have been accessed the least number of times are
evicted when the cache is full. The frequency list structure as described in“An O(1) algorithm for implementing the LFU cache eviction scheme”is implemented.@cache(LFUCache(size=50))deffunc(...)...LRU (Least Recently Used)TheLRUCacheis a Least Recently Used cache where keys which have been accessed the least recently are evicted when
the cache is full.@cache(LRUCache(size=50))deffunc(...)...MFU (Most Frequently Used)TheMFUCacheis a Most Frequently Used cache where keys which have been accessed the most number of times are
evicted when the cache is full. This uses the same frequency list structure as described in LFU.@cache(MFUCache(size=50))deffunc(...)...MQ (Multi-Queue)TheMQCacheis a Multi-Queue cache in which multiple queues are used to hold levels of varying temperature (i.e.
highly accessed and less accessed) along with a history buffer (similar to 2Q). This is implemented based on the
paper “The Multi-Queue Replacement Algorithm for Second Level Buffer Caches” which uses a LRU queues for each level. The
access count of each item is also recorded and used in determining which queue to promote the item to based on thequeue_funcparameter.Items are also susceptible to being evicted over time. If an item isn’t accessed within a certain time it is bumped
down to a lower queue and its expiration time is reset. This continues if the item isn’t accessed until it is
eventually evicted to the lowest queue.The history buffer is a FIFO queue that keeps track of items recently evicted from the queue. If an item is accessed
while in the history buffer, it is placed in the appropriate queue based on it’s previous access frequency + 1.@cache(MQCache(size=50,buffer_size=10,expire_time=100,num_queues=8,queue_func=lambdaf:math.log(f,2),access_based=True))deffunc(...)...MRU (Most Recently Used)TheMRUCacheis a Most Recently Used cache where keys which have been accessed the most recently are evicted when
the cache is full.@cache(MRUCache(size=50))deffunc(...)...NMRU (Not Most Recently Used)TheNMRUCacheis a Not Most Recently Used cache where keys which have not been access the most recently are evicted
when the cache is full. When the cache is full, a random key other than the most recently inserted is removed.@cache(NMRUCache(size=50))deffunc(...)...RR (Random Replacement)TheRRCacheis a Random Replacement cache where keys are evicted randomly, regardless of access of insertion order.@cache(RRCache(size=50))deffunc(...)...SLRU (Segmented Least Recently Used)TheSLRUCacheis a Segmented Least Recently Used cache which is implemented with two queues, a LRU (the protected),
and a FIFO (the probationary). Items are initially placed into the probationary queue when first placed into the cache.
If this cache is full, items are evicted in the order of their arrival. If items are accessed while they are in the
probationary queue, they are moved to the protected queue. They stay in this queue until it is full and the key
which has been least recently used is moved back to the probationary queue.Note that this cache implementation is very similar to the simple 2Q algorithm with the exception that items evicted
from the protected cache are moved to the probationary (opposed to being immediately evicted).@cache(SLRUCache(protected_size=50,probationary_size=20))deffunc(...)...StaticTheStaticCacheis a simple cache with no key eviction. Keys are stored permanently, or at least until the cache is
cleared.@cache(StaticCache())deffunc(...)...TLRU (Time-aware Least Recently Used)TheTLRUCacheis a Time-aware Least Recently-Used cache where keys are prematurely evicted if their last access time
is below a minimum limit, theexpire_time. Time in this case is either a simple clock that is incremented each time
the cache is accessed., or the actual time in seconds that has passed. This is determined by theaccess_basedparameter. Ifreset_on_accessis True, theexpire_timeis reset each time the item is accessed; otherwise it is
expired from the time of initial insertion in the cache.This is implemented with two LRU lists, one for LRU-based expiration and another for time-based expiration. This is
required due to allowingreset_on_accessto be False, thereby allowing items to be expired independent of how they
are accessed.@cache(TLRUCache(expire_time=100,size=50,access_based=False,reset_on_access=True))deffunc(...)...TwoQ (simple 2Q or Two-Queue)TheTwoQCacheis the simple 2Q cache described by the algorithm in “2Q: A Low Overhead High Performance Buffer
Management Replacement Algorithm”. Two queues are used, an LRU (the primary), and a FIFO (the secondary). Items are
initially placed into the secondary queue when placed into the cache. If the secondary queue is full, items are evicted
in the order of their arrival. If items are accessed while they are in the secondary queue, they are moved to the
primary queue. They stay in this queue until it is full and the key which has been least recently used is evicted.@cache(TwoQ(primary_size=50,secondary_size=20))deffunc(...)...TwoQFull (full 2Q or Two-Queue)TheTwoQFullCacheis the full 2Q cache as described by the algorithm in “2Q: A Low Overhead High Performance
Buffer Management Replacement Algorithm”. Three queues are used, an LRU (the primary), and two FIFO (the secondary
in and out). Items are initially placed into the secondary “in” queue when first placed into the cache. If this queue is
full, items are moved in the order of their arrival into the secondary “out” queue. If items are accessed while they are
in the secondary “in” queue they stay in this queue in their current position.If items are accessed while they are in the secondary “out” queue they are moved to the primary queue. If this cache
is full, items are evicted in the order of their arrival. Items in the primary queue stay in this queue until it is
full and the key which has been least recently used is evicted. This implementation yields O(1) access and insertion
time.@cache(TwoQFull(primary_size=50,secondary_in_size=20,secondary_out_size=20))deffunc(...)...Advanced UsageInstead of using as a decorator,cachecan be used as an instead for wrapping an arbitrary number of function calls.
This can be achieved via therunmethod.deffunc_a():...deffunc_b():...cacher=cache(algorithm=...)# Using same configured cache instancecache.run(func_a,args,kwargs)cache.run(func_b,args,kwargs)Besides using the providedrunmethod, like any decorator functions can be locally wrapped, passed around, and
executed.deffunc():...cacher=cache(algorithm=...)cache_func=cacher(func)cache_func(args,kwargs)# Or as a one-off like socache(...)(func)(args,kwargs)Both theon_hitandon_misscallback functions that can be passed intocachecan actually be configured to
accepts different number of parameters depending on the function. They can each either accept 0 parameters, the
parameters that would be typically passed in, or the wrapped function’s args and kwargs in addition to the parameters
typically given.Optionally passing in the args and kwargs allows for building more complex callback functions. Each of the possible
function variations are shown below.defon_hit():...defon_hit(error):...defon_hit(error,*args,**kwargs):...defon_miss():...defon_miss(value):...defon_miss(value,*args,**kwargs):...In addition to thecache_info()andcache_clear()methods bound to the function, others can be dynamically bound
to the function depending on the algorithm. None of the current basic implementations use this functionality, but this
has a case for when creating one’s own or extending the existing algorithms. The dynamic methods are prefixed with
“cache_<method-name>”.importcopyclassFancyCache(LRUCache):def__init__(self,size):super(size).__init__()defshow_all(self):# Retrieve a copy of the current items in the cachereturncopy(self._map)defdynamic_methods(self):return['show_all']@cache(FancyCache())deffunc(x,y)...func(1,2)func(3,4)# Dynamic methods are prefixed with 'cache_'items=func.cache_show_all()ContributionContributions or suggestions are welcome! Feel free toopen an issueif a bug is found or an enhancement is desired,
or even apull request.ChangelogAll changes and versioning information can be found in theCHANGELOG.LicenseCopyright (c) 2018 Jared Gillespie. SeeLICENSEfor details.