Package: cacheme
CachemeAsyncio cache framework with multiple cache storages.Organize cache better:Cache configuration with node, you can apply different strategies on different nodes.Multiple cache storages:in-memory/redis/mongodb/postgres..., also support chain storages.Multiple serializers:Pickle/Json/Msgpack serializers.Thundering herd protection:Simultaneously requests to same key are blocked by asyncio Event and only load from source once.Cache stats API:Stats of each node and colected automatically.Performance:See Benchemarks section.Related projects:High performance in-memory cache:https://github.com/Yiling-J/theineTable of ContentsInstallationAdd NodeRegister StorageCacheme APICache NodeKeyMeta ClassSerializersDoorKeeperCache StorageLocal StorageRedis StorageMongoDB StorageSqlite StoragePostgreSQL StorageMySQL StorageHow Thundering Herd Protection WorksBenchmarkscontinuous benchmark200k concurrent requests20k concurrent batch requestsRequirementsPython 3.7+Installationpip install cachemeMultiple storages are supported by drivers. You can install the required drivers with:pip install cacheme[redis]
pip install cacheme[aiomysql]
pip install cacheme[motor]
pip install cacheme[asyncpg]Add NodeNode is the core part of cache. Each node has its own key function, load function and storage options. Stats of each node are collected independently. You can place all node definations into one package/module, so everyone knows exactly what is cached and how they are cached. All cacheme API are based on node.Each node contains:Key attritubes andkeymethod,  which are used to generate cache key. Here theUserInfoNodeis a dataclass, so__init__method is generated automatically.Asyncloadmethod, which will be called to load data from data source on cache missing. This method can be omitted if you useMemoizedecorator only.Metaclass, node cache configurations. SeeCache Nodeimportcachemefromdataclassesimportdataclassfromcacheme.serializerimportMsgPackSerializer@dataclassclassUserInfoNode(cacheme.Node):user_id:intdefkey(self)->str:returnf"user:{self.user_id}:info"asyncdefload(self)->Dict:user=get_user_from_db(self.user_id)returnserialize(user)classMeta(cacheme.Node.Meta):version="v1"caches=[cacheme.Cache(storage="my-redis",ttl=None)]serializer=MsgPackSerializer()This simple example use a cache storage called "my-redis", which will be registered next step. Also we useMsgPackSerializerhere to dump and load data from redis. See [Cache Node] for more details.Register StorageRegister a redis storage called "my-redis", which you can use in node meta data. Theregister_storageis asynchronous and will try to establish connection to cache store.
See [Cache Storage] for more details.importcachemeawaitcacheme.register_storage("my-redis",cacheme.Storage(url="redis://localhost:6379"))Cacheme APIget: get data from single node.user=awaitcacheme.get(UserInfoNode(user_id=1))get_all: get data from multiple nodes, same node type.users=awaitcacheme.get_all([UserInfoNode(user_id=1),UserInfoNode(user_id=2)])invalidate: invalidate a node, remove data from cache.awaitcacheme.invalidate(UserInfoNode(user_id=1))refresh: reload node data usingloadmethod.awaitcacheme.refresh(UserInfoNode(user_id=1))Memoize: memoize function with this decorator.Decorate your function withcacheme.Memoizedecorator and cache node. Cacheme will load data using the decorated function and ignoreloadmethod.
Because your function may contain variable number of args/kwargs, we need one more step to map between args/kwargs to node. The decorated map function should have same input signature as memoized function, and return a cache node.@cacheme.Memoize(UserInfoNode)asyncdefget_user_info(user_id:int)->Dict:return{}# function name is not important, so just use _ here@get_user_info.to_nodedef_(user_id:int)->UserInfoNode:returnUserInfoNode(user_id=user_id)nodes: list all nodes.nodes=cacheme.nodes()stats: get node stats.metrics = cacheme.stats(UserInfoNode)

metrics.request_count() # total request count
metrics.hit_count() # total hit count
metrics.hit_rate() # hit_count/request_count
metrics.miss_count() # (request_count - hit_count)/request_count
metrics.miss_rate() # miss_count/request_count
metric.load_success_count() # total load success count
metrics.load_failure_count() # total load fail count
metrics.load_failure_rate() # load_failure_count/load_count
metrics.load_count() # total load count
metrics.total_load_time() # total load time in nanoseconds
metrics.average_load_time() # total_load_time/load_countset_prefix: set prefix for all keys. Default prefix iscacheme. Change prefix will invalid all keys, because prefix is part of the key.cacheme.set_prefix("mycache")Cache NodeKeyGenerated cache key will be:{prefix}:{key()}:{Meta.version}. So changeversionwill invalid all keys automatically.Meta Classversion[str]: Version of node, will be used as suffix of cache key.caches[List[Cache]]: Caches for node. EachCachehas 2 attributes,storage[str]andttl[Optional[timedelta]].storageis the name you registered withregister_storageandttlis how long this cache will live. Cacheme will try to get data from each cache from left to right. In most cases, use single cache or [local, remote] combination.serializer[Optional[Serializer]]: Serializer used to dump/load data. If storage type islocal, serializer is ignored. SeeSerializers.doorkeeper[Optional[DoorKeeper]]: SeeDoorKeeper.Multiple caches example. Local cache is not synchronized, so set a much shorter ttl compared to redis one. Then we don't need to worry too much about stale data.importcachemefromdataclassesimportdataclassfromdatetimeimporttimedeltafromcacheme.serializerimportMsgPackSerializer@dataclassclassUserInfoNode(cacheme.Node):user_id:intdefkey(self)->str:returnf"user:{self.user_id}:info"asyncdefload(self)->Dict:user=get_user_from_db(self.user_id)returnserialize(user)classMeta(cacheme.Node.Meta):version="v1"caches=[cacheme.Cache(storage="local",ttl=timedelta(seconds=30)),cacheme.Cache(storage="my-redis",ttl=timedelta(days=10))]serializer=MsgPackSerializer()Cacheme also support creating Node dynamically, you can use this together withMemoizedecorator:@Memoize(cacheme.build_node("TestNodeDynamic","v1",[Cache(storage="local",ttl=None)]))asyncdeffn(a:int)->int:return1@fn.to_nodedef_(a:int)->cacheme.DynamicNode:returnDynamicNode(key=f"bar:{a}")Here we useDynamicNode, which only support one param:keySerializersCacheme provides serveral builtin serializers, you can also write your own serializer.PickleSerializer: All python objects.JSONSerializer: Usepydantic_encoderandjson, support python primitive types, dataclass, pydantic model. Seepydantic types.MsgPackSerializer: Usepydantic_encoderandmsgpack, support python primitive types, dataclass, pydantic model. Seepydantic types.serializer with compression, use zlib level-3CompressedPickleSerializerCompressedJSONSerializerCompressedMsgPackSerializerDoorKeeperIdea fromTinyLfu paper.The Doorkeeper is a regular Bloom filter placed in front of the cahce. Upon
item arrival, we first check if the item is contained in the Doorkeeper. If it is not contained in the
Doorkeeper (as is expected with first timers and tail items), the item is inserted to the Doorkeeper and
otherwise, it is inserted to the cache.fromcachemeimportBloomFilter@dataclassclassUserInfoNode(cacheme.Node):classMeta(cacheme.Node.Meta):# size 100000, false positive probability 0.01doorkeeper=BloomFilter(100000,0.01)BloomFilter is cleared automatically when requests count == size.Cache StorageLocal StorageLocal storage use the state-of-the-art libraryTheineto store data. If your use case in simple, also consider usingTheinedirectly, which will have the best performance.# lru policyStorage(url="local://lru",size=10000)# w-tinylfu policyStorage(url="local://tlfu",size=10000)Parameters:url:local://{policy}. 2 policies are currently supported:lrutlfu: W-TinyLfu policysize: size of the storage. Policy will be used to evict key when cache is full.Redis StorageStorage(url="redis://localhost:6379")# clusterStorage(url="redis://localhost:6379",cluster=True)Parameters:url: redis connection url.cluster: bool, cluster or not, default False.pool_size: connection pool size, default 100.MongoDB StorageTo use mongodb storage, create index first. Seemongo.jsStorage(url="mongodb://test:password@localhost:27017",database="test",collection="cache")Parameters:url: mongodb connection url.database: mongodb database name.collection: mongodb collection name.pool_size: connection pool size, default 50.Sqlite StorageTo use sqlite storage, create table and index first. Seesqlite.sqlStorage(url="sqlite:///test",table="cache")Parameters:url: sqlite connection url.table: cache table name.pool_size: connection pool size, default 50.PostgreSQL StorageTo use postgres storage, create table and index first. Seepostgresql.sqlStorage(url="postgresql://username:password@127.0.0.1:5432/test",table="cache")Parameters:url: postgres connection url.table: cache table name.pool_size: connection pool size, default 50.MySQL StorageTo use mysql storage, create table and index first. Seemysql.sqlStorage("mysql://username:password@localhost:3306/test",table="cache")Parameters:url: mysql connection url.table: cache table name.pool_size: connection pool size, default 50.How Thundering Herd Protection WorksIf you are familar with Gosingleflight, you may have an idea how Cacheme works. Cacheme group concurrent requests to same resource(node) into a singleflight with asyncio Event, which willload from remote cache OR data source only once. That's why in next Benchmarks section, you will find Cacheme even reduce total redis GET command count under high concurrency.Benchmarkscontinuous benchmarkhttps://github.com/Yiling-J/cacheme-benchmark200k concurrent requestsaiocache:https://github.com/aio-libs/aiocachecashews:https://github.com/Krukov/cashewssource code:https://github.com/Yiling-J/cacheme/blob/master/benchmarks/trace.pyHow this benchmark run:Initialize Cacheme/Aiocache/Cashews with Redis backend, use Redis blocking pool and set pool size to 100.Decorate Aiocache/Cashews/Cacheme with a function which accept a number and sleep 0.1s. This function also record how many times it is called.Register Redis response callback, so we can know how many times GET command are called.Create 200k coroutines use a zipf generator and put them in async queue(around 50k-60k unique numbers).Run coroutines in queue with N concurrent workers.Collect results.Identifier:Cacheme: Cacheme redis storageAiocahce: Aiocahce cached decoratorCashews: Cashews cache decorateCacheme-2: Cacheme use cache chain [local, redis]Aiocache-2: Aiocache cached_stampede decoratorCashews-2: Cashews decorator with lock=TrueResult:Time: How long it takes to finish bench.Redis GET: How many times Redis GET command are called, use this to evaluate pressure to remote cache server.Load Hits: How many times the load function(which sleep 0.1s) are called, use this to evaluate pressure to load source(database or something else).1k concurrencyTimeRedis GETLoad HitsCacheme30 s16645455579Cacheme-218 s9068155632Aiocache46 s20000056367Aiocache-263 s25649255417Cashews51 s20000056920Cashews-2134 s2000005545010k concurrencyTimeRedis GETLoad HitsCacheme32 s12370456736Cacheme-220 s8375056635Aiocache67 s20000062568Aiocache-2113 s26319555507Cashews68 s20000066036Cashews-2175 s20000055709100k concurrencyTimeRedis GETLoad HitsCacheme30 s6099056782Cacheme-227 s5576255588Aiocache80 s200000125085Aiocache-2178 s32641765598Cashews88 s20000087894Cashews-2236 s2000005564720k concurrent batch requestssource code:https://github.com/Yiling-J/cacheme/blob/master/benchmarks/trace.pyHow this benchmark run:Initialize Cacheme with Redis backend, use Redis blocking pool and set pool size to 100.Decorate Cacheme with a function which accept a number and sleep 0.1s. This function also record how many times it is called.Register Redis response callback, so we can know how many times MGET command are called.Create 20kget_allcoroutines use a zipf generator and put them in async queue(around 50k-60k unique numbers). Eachget_allrequest will get 20 unique numbers in batch. So totally 400k numbers.Run coroutines in queue with N concurrent workers.Collect results.Result:Time: How long it takes to finish bench.Redis MGET: How many times Redis MGET command are called, use this to evaluate pressure to remote cache server.Load Hits: How many times the load function(which sleep 0.1s) are called, use this to evaluate pressure to load source(database or something else).1k concurrencyTimeRedis MGETLoad HitsCacheme12 s99965590210k concurrencyTimeRedis MGETLoad HitsCacheme11 s990842894