Package: circular-dict
CircularDictCircularDictis a Pythondictthat acts as aCircular Buffer. This dictionary maintains acontrolled size, limited either by a specifiednumber of items(maxlen) ortotal size in bytes(maxsize_bytes). Upon reaching the defined limit,CircularDictautomatically removes the oldest entries, maintainingmemory usageunder control.Built upon Python'sOrderedDict,CircularDictinherits allstandard dictionary usageand operations, augmented by the capability ofmemory management. It's particularly useful in scenarios such ascaching, where limitingmemory consumptionis crucial. The class combinesdictionaryandcircular-queuebehaviors, providing an efficient and scalable solution for various use cases.InstallationTo installCircularDictsimply run:pipinstallcircular-dictUsageWorking withCircularDictis as simple as using a standard Pythondict, with additional parametersmaxlenormaxsize_byteson the initialization to control the buffer size. You can use one of them or both.fromcircular_dictimportCircularDict# Initialize a CircularDict with a maximum length of 3 items and a storage limit of 4Mbmy_dict=CircularDict(maxlen=3,maxsize_bytes=4*1024*1024)Example withmaxlenYou can usemaxlento define the maximum amount of items that the dictionary can store. It is useful for defining fixed size buffers.fromcircular_dictimportCircularDict# Initialize a CircularDict with a maximum length of 3my_buffer=CircularDict(maxlen=3)# Fill it with 3 itemsmy_buffer['item1']='value1'my_buffer['item2']='value2'my_buffer['item3']='value3'print(f"When filling it:{circ_dict}")# Add another itemmy_buffer['item4']='value4'print(f"After adding an element beyond maxlen:{circ_dict}")Output:Whenfillingit:{'item1':'value1','item2':'value2','item3':'value3'}Afteraddinganelementbeyondmaxlen:{'item2':'value2','item3':'value3','item4':'value4'}Example withmaxsize_bytesYou can usemaxsize_bytesto define the maximum amount of memory that thedictcan store. It is particularly beneficial when definingcaches, to preventmemory overflows.fromcircular_dictimportCircularDictimportnumpyasnpimportsys# Initialize a CircularDict with a maximum length of 100KBmy_buffer=CircularDict(maxsize_bytes=100*1024)# Add two arrays of ~40Kb (10*1024*4 bytes (int32) + 5 bytes (chars) + 100 bytes (numpy structure) + 50 bytes (str structure))my_buffer['item1']=np.zeros((10,1024),dtype=np.int32)my_buffer['item2']=np.ones((10,1024),dtype=np.int32)print(f"{len(my_buffer)}Elements{tuple(my_buffer.keys())}. Dict size:{my_buffer.current_size/1024}Kb")# Add a new element of ~32Kb will delete oldest elements ('item1') until fitting in the `dict`.my_buffer['item3']=np.ones((8,1024),dtype=np.int32)print(f"{len(my_buffer)}Elements{tuple(my_buffer.keys())}. Dict size:{my_buffer.current_size/1024}Kb")# Create an element of ~160Kb (bigger than the defined maximum storage) to trigger a MemoryErrortoo_big_array=np.ones((40,1024),dtype=np.int32)try:# Try to add it to the dictmy_buffer['item4']=too_big_arrayexceptMemoryError:print(f"Cannot add an element with{sys.getsizeof(too_big_array)/1024}Kb in a dict with"\f"maxsize_bytes of{my_buffer.maxsize_bytes/1024}Kb. Current elements{tuple(my_buffer.keys())}")Output2Elements('item1','item2').Dictsize:80.35Kb2Elements('item2','item3').Dictsize:72.35Kb
Cannotaddanelementwith160.12Kbinadictwithmaxsize_bytesof100.0Kb.Currentelements('item2','item3')Please remember that themaxsize_bytesparameter considers thetotalmemory footprint, including the sizes ofkeysandvalues. If you try to add an item that exceeds themaxsize_bytes, aMemoryErrorwill be raised.