Package: py-memoize
Extended docs (including API docs) available atmemoize.readthedocs.io.What & WhyWhat:Caching library for asynchronous Python applications.Why:Python deserves library that works in async world
(for instance handlesdog-piling)
and has a proper, extensible API.EtymologyIn computing, memoization or memoisation is an optimization technique
used primarily to speed up computer programs by storing the results of
expensive function calls and returning the cached result when the same
inputs occur again. (…) The term “memoization” was coined by Donald
Michie in 1968 and is derived from the Latin word “memorandum” (“to be
remembered”), usually truncated as “memo” in the English language, and
thus carries the meaning of “turning [the results of] a function into
something to be remembered.”~WikipediaGetting StartedInstallationBasic InstallationTo get you up & running all you need is to install:pipinstallpy-memoizeInstallation of ExtrasIf you are going to usememoizewith tornado add a dependency on extra:pipinstallpy-memoize[tornado]To harness the power ofujson(if JSON SerDe is used) install extra:pipinstallpy-memoize[ujson]UsageProvided examples use default configuration to cache results in memory.
For configuration options seeConfigurability.You can usememoizewith bothasyncioandTornado-  please see the appropriate example:asyncioTo apply default caching configuration use:importasyncioimportrandomfrommemoize.wrapperimportmemoize@memoize()asyncdefexpensive_computation():return'expensive-computation-'+str(random.randint(1,100))asyncdefmain():print(awaitexpensive_computation())print(awaitexpensive_computation())print(awaitexpensive_computation())if__name__=="__main__":asyncio.get_event_loop().run_until_complete(main())TornadoIf your project is based on Tornado use:importrandomfromtornadoimportgenfromtornado.ioloopimportIOLoopfrommemoize.wrapperimportmemoize@memoize()@gen.coroutinedefexpensive_computation():return'expensive-computation-'+str(random.randint(1,100))@gen.coroutinedefmain():result1=yieldexpensive_computation()print(result1)result2=yieldexpensive_computation()print(result2)result3=yieldexpensive_computation()print(result3)if__name__=="__main__":IOLoop.current().run_sync(main)FeaturesAsync-firstAsynchronous programming is often seen as a huge performance boost in python programming.
But with all the benefits it brings there are also new concurrency-related caveats
likedog-piling.This library is built async-oriented from the ground-up, what manifests in, for example,
inDog-piling proofnessorAsync cache storage.Tornado & asyncio supportNo matter what are you using, build-inasyncioor its predecessorTornadomemoizehas you covered as you can use it with both.This may come handy if you are planning a migration from Tornado to asyncio.Under the hoodmemoizedetects if you are usingTornadoorasyncio(by checking ifTornadois installed and available to import).If haveTornadoinstalled but your application usesasyncioIO-loop,
setMEMOIZE_FORCE_ASYNCIO=1environment variable to force usingasyncioand ignoreTornadoinstalation.ConfigurabilityWithmemoizeyou have under control:timeout applied to the cached method;key generation strategy (seememoize.key.KeyExtractor);
already provided strategies use arguments (both positional & keyword) and method name (or reference);storage for cached entries/items (seememoize.storage.CacheStorage);
in-memory storage is already provided;
for convenience of implementing new storage adapters some SerDe (memoize.serde.SerDe) are provided;eviction strategy (seememoize.eviction.EvictionStrategy);
least-recently-updated strategy is already provided;entry builder (seememoize.entrybuilder.CacheEntryBuilder)
which has control overupdate_after&expires_afterdescribed inTunable eviction & async refreshingAll of these elements are open for extension (you can implement and plug-in your own).
Please contribute!Example how to customize default config (everything gets overridden):fromdatetimeimporttimedeltafrommemoize.configurationimportMutableCacheConfiguration,DefaultInMemoryCacheConfigurationfrommemoize.entrybuilderimportProvidedLifeSpanCacheEntryBuilderfrommemoize.evictionimportLeastRecentlyUpdatedEvictionStrategyfrommemoize.keyimportEncodedMethodNameAndArgsKeyExtractorfrommemoize.storageimportLocalInMemoryCacheStoragefrommemoize.wrapperimportmemoize@memoize(configuration=MutableCacheConfiguration.initialized_with(DefaultInMemoryCacheConfiguration()).set_method_timeout(value=timedelta(minutes=2)).set_entry_builder(ProvidedLifeSpanCacheEntryBuilder(update_after=timedelta(minutes=2),expire_after=timedelta(minutes=5))).set_eviction_strategy(LeastRecentlyUpdatedEvictionStrategy(capacity=2048)).set_key_extractor(EncodedMethodNameAndArgsKeyExtractor(skip_first_arg_as_self=False)).set_storage(LocalInMemoryCacheStorage()))asyncdefcached():return'dummy'Still, you can use default configuration which:sets timeout for underlying method to 2 minutes;uses in-memory storage;uses method instance & arguments to infer cache key;stores up to 4096 elements in cache and evicts entries according to least recently updated policy;refreshes elements after 10 minutes & ignores unrefreshed elements after 30 minutes.If that satisfies you, just use default config:frommemoize.configurationimportDefaultInMemoryCacheConfigurationfrommemoize.wrapperimportmemoize@memoize(configuration=DefaultInMemoryCacheConfiguration())asyncdefcached():return'dummy'Also, if you want to stick to the building blocks of the default configuration, but need to adjust some basic params:fromdatetimeimporttimedeltafrommemoize.configurationimportDefaultInMemoryCacheConfigurationfrommemoize.wrapperimportmemoize@memoize(configuration=DefaultInMemoryCacheConfiguration(capacity=4096,method_timeout=timedelta(minutes=2),update_after=timedelta(minutes=10),expire_after=timedelta(minutes=30)))asyncdefcached():return'dummy'Tunable eviction & async refreshingSometimes caching libraries allow providing TTL only. This may result in a scenario where when the cache entry expires
latency is increased as the new value needs to be recomputed.
To mitigate this periodic extra latency multiple delays are often used. In the case ofmemoizethere are two
(seememoize.entrybuilder.ProvidedLifeSpanCacheEntryBuilder):update_afterdefines delay after which background/async update is executed;expire_afterdefines delay after which entry is considered outdated and invalid.This allows refreshing cached value in the background without any observable latency.
Moreover, if some of those background refreshes fail they will be retried still in the background.
Due to this beneficial feature, it is recommended toupdate_afterbe significantly shorter thanexpire_after.Dog-piling proofnessIf some resource is accessed asynchronouslydog-pilingmay occur.
Caches designed for synchronous python code
(like built-inLRU)
will allow multiple concurrent tasks to observe a miss for the same resource and will proceed to flood underlying/cached
backend with requests for the same resource.As it breaks the purpose of caching (as backend effectively sometimes is not protected with cache)memoizehas built-in dog-piling protection.Under the hood, concurrent requests for the same resource (cache key) get collapsed to a single request to the backend.
When the resource is fetched all requesters obtain the result.
On failure, all requesters get an exception (same happens on timeout).An example of what it all is about:importasynciofromdatetimeimporttimedeltafromaiocacheimportcached,SimpleMemoryCache# version 0.11.1 (latest) used as example of other cache implementationfrommemoize.configurationimportDefaultInMemoryCacheConfigurationfrommemoize.wrapperimportmemoize# scenario configurationconcurrent_requests=5request_batches_execution_count=50cached_value_ttl_ms=200delay_between_request_batches_ms=70# results/statisticsunique_calls_under_memoize=0unique_calls_under_different_cache=0@memoize(configuration=DefaultInMemoryCacheConfiguration(update_after=timedelta(milliseconds=cached_value_ttl_ms)))asyncdefcached_with_memoize():globalunique_calls_under_memoizeunique_calls_under_memoize+=1awaitasyncio.sleep(0.01)returnunique_calls_under_memoize@cached(ttl=cached_value_ttl_ms/1000,cache=SimpleMemoryCache)asyncdefcached_with_different_cache():globalunique_calls_under_different_cacheunique_calls_under_different_cache+=1awaitasyncio.sleep(0.01)returnunique_calls_under_different_cacheasyncdefmain():foriinrange(request_batches_execution_count):awaitasyncio.gather(*[x()forxin[cached_with_memoize]*concurrent_requests])awaitasyncio.gather(*[x()forxin[cached_with_different_cache]*concurrent_requests])awaitasyncio.sleep(delay_between_request_batches_ms/1000)print("Memoize generated{}unique backend calls".format(unique_calls_under_memoize))print("Other cache generated{}unique backend calls".format(unique_calls_under_different_cache))predicted=(delay_between_request_batches_ms*request_batches_execution_count)//cached_value_ttl_msprint("Predicted (according to TTL){}unique backend calls".format(predicted))# Printed:# Memoize generated 17 unique backend calls# Other cache generated 85 unique backend calls# Predicted (according to TTL) 17 unique backend callsif__name__=="__main__":asyncio.get_event_loop().run_until_complete(main())Async cache storageInterface for cache storage allows you to fully harness benefits of asynchronous programming
(see interface ofmemoize.storage.CacheStorage).Currentlymemoizeprovides only in-memory storage for cache values (internally atRASPwe have others).
If you want (for instance) Redis integration, you need to implement one (please contribute!)
butmemoizewill optimally use your async implementation from the start.Manual InvalidationYou could also invalidate entries manually.
To do so you need to create instance ofmemoize.invalidation.InvalidationSupport)
and pass it alongside cache configuration.
Then you could just pass args and kwargs for which you want to invalidate entry.frommemoize.configurationimportDefaultInMemoryCacheConfigurationfrommemoize.invalidationimportInvalidationSupportimportasyncioimportrandomfrommemoize.wrapperimportmemoizeinvalidation=InvalidationSupport()@memoize(configuration=DefaultInMemoryCacheConfiguration(),invalidation=invalidation)asyncdefexpensive_computation(*args,**kwargs):return'expensive-computation-'+str(random.randint(1,100))asyncdefmain():print(awaitexpensive_computation('arg1',kwarg='kwarg1'))print(awaitexpensive_computation('arg1',kwarg='kwarg1'))print("Invalidation #1")awaitinvalidation.invalidate_for_arguments(('arg1',),{'kwarg':'kwarg1'})print(awaitexpensive_computation('arg1',kwarg='kwarg1'))print(awaitexpensive_computation('arg1',kwarg='kwarg1'))print("Invalidation #2")awaitinvalidation.invalidate_for_arguments(('arg1',),{'kwarg':'kwarg1'})print(awaitexpensive_computation('arg1',kwarg='kwarg1'))# Sample output:## expensive - computation - 98# expensive - computation - 98# Invalidation  # 1# expensive - computation - 73# expensive - computation - 73# Invalidation  # 2# expensive - computation - 59if__name__=="__main__":asyncio.get_event_loop().run_until_complete(main())