Package: shared-memory-dict
Shared Memory DictA very simpleshared memorydict implementation.Requires: Python >= 3.8>>># In the first Python interactive shell>>fromshared_memory_dictimportSharedMemoryDict>>smd=SharedMemoryDict(name='tokens',size=1024)>>smd['some-key']='some-value-with-any-type'>>smd['some-key']'some-value-with-any-type'>>># In either the same shell or a new Python shell on the same machine>>existing_smd=SharedMemoryDict(name='tokens',size=1024)>>>existing_smd['some-key']'some-value-with-any-type'>>>existing_smd['new-key']='some-value-with-any-type'>>># Back in the first Python interactive shell, smd reflects this change>>smd['new-key']'some-value-with-any-type'>>># Clean up from within the second Python shell>>>existing_smd.shm.close()# or "del existing_smd">>># Clean up from within the first Python shell>>>smd.shm.close()>>>smd.shm.unlink()# Free and release the shared memory block at the very end>>>delsmd# use of smd after call unlink() is unsupportedThe argnamedefines the location of the memory block, so if you want to share the memory between process use the same name.
The size (in bytes) occupied by the contents of the dictionary depends on the serialization used in storage. By default pickle is used.InstallationUsingpip:pipinstallshared-memory-dictLocksTo usemultiprocessing.Lockon write operations of shared memory dict set environment variableSHARED_MEMORY_USE_LOCK=1.SerializationWe usepickleas default to read and write the data into the shared memory block.You can create a custom serializer by implementing thedumpsandloadsmethods.Custom serializers should raiseSerializationErrorif the serialization fails andDeserializationErrorif the deserialization fails. Both are defined in theshared_memory_dict.serializersmodule.An example of a JSON serializer extracted from serializers module:NULL_BYTE:Final=b"\x00"classJSONSerializer:defdumps(self,obj:dict)->bytes:try:returnjson.dumps(obj).encode()+NULL_BYTEexcept(ValueError,TypeError):raiseSerializationError(obj)defloads(self,data:bytes)->dict:data=data.split(NULL_BYTE,1)[0]try:returnjson.loads(data)exceptjson.JSONDecodeError:raiseDeserializationError(data)Note: A null byte is used to separate the dictionary contents from the bytes that are in memory.To use the custom serializer you must set it when creating a new shared memory dict instance:>>>smd=SharedMemoryDict(name='tokens',size=1024,serializer=JSONSerializer())CaveatThe pickle module is not secure. Only unpickle data you trust.See morehere.Django Cache ImplementationThere's aDjango Cache Implementationwith Shared Memory Dict:# settings/base.pyCACHES={'default':{'BACKEND':'shared_memory_dict.caches.django.SharedMemoryCache','LOCATION':'memory','OPTIONS':{'MEMORY_BLOCK_SIZE':1024}}}Install with:pip install "shared-memory-dict[django]"CaveatWith Django cache implementation the keys only expire when they're read. Be careful with memory usageAioCache BackendThere's also aAioCache Backend Implementationwith Shared Memory Dict:Fromaiocacheimportcachescaches.set_config({'default':{'cache':'shared_memory_dict.caches.aiocache.SharedMemoryCache','size':1024,},})This implementation is very based on aiocacheSimpleMemoryCacheInstall with:pip install "shared-memory-dict[aiocache]"