Package: async-tinydb
What's This?An asynchronous version ofTinyDBwith extended capabilities.Almost every method is asynchronous. And it's based onTinyDB 4.7.0+.UnlikeTinyDBwhich has a minimal core,Async-TinyDBis designed to have max flexibility and performance.Incompatible ChangesAsynchronous: Say goodbye to blocking IO.Don't forget toawaitasync methods!Drop support: Only supports Python 3.10+.ujson:Usingujsoninstead ofjson. Some arguments aren't compatible withjson[^1]Dev-Changes: Changes that only matter to developers (Who customiseStorage,Query, etc).Miscellaneous: Differences that only matter in edge cases.New FeaturesEvent Hooks: You can now use event hooks to hook into an operation. SeeEvent Hooksfor more details.Redesigned ID & Doc Class: You canreplaceandcustomise themmore pleasingly.DB-level Caching: This significantly improves the performance of all operations. However, it may cause dirty reads with some types of storage [^disable-db-level].Built-inModifier: UseModifierto easilycompress,encryptandextend typesof your database. Sure you can do much more than these.(SeeModifier)Isolation Level: Performance or thread-safe or even ACID? It's up to you[^isolevel].Atomic Write: Shipped withJSONStorageBatch Search By IDs:searchmethod now takes an extradoc_idsargument (works like an additional condition)How to use it?InstallationMinimum:pip install async-tinydbEncryption:pip install async-tinydb[encryption]Compression:pip install async-tinydb[compression]Full:pip install async-tinydb[all]ImportingimportasynctinydbUsageRead theoriginalTinyDBdocuments. Insert anawaitin front of async methods.Notice that some codes are still blocking, for example, when callinglen()onTinyDBorTableObjects.That's it.Documents For Advanced UsageModifierEvent HooksReplacing ID & Document ClassNOTICE: Mixing classes in one table may cause errors!When a table exists in a file,Async-TinyDBwon't determine its classes by itself, it is your duty to make sure classes are matching.ID ClassesIncreID: Default ID class, mimics the behaviours of the originalintID but requires much fewer IO operations.UUID: Usesuuid.UUID[^uuid-version].Document ClassDocument: Default document class, usesdictunder the bonet.fromasynctinydbimportTinyDB,UUID,IncreID,Documentdb=TinyDB("database.db")# Setting ID class to `UUID`, document class to `Document`tab=db.table("table1",document_id_class=UUID,document_class=Document)SeeCustomisationfor more detailsEncryptionCurrently only supports AES-GCM encryption.There are two ways to use encryption:1. UseEncryptedJSONStoragedirectlyfromasynctinydbimportEncryptedJSONStorage,TinyDBasyncdefmain():db=TinyDB("db.json",key="your key goes here",storage=EncryptedJSONStorage)2. UseModifierclassSeeEncryptionIsolation LevelWhen operating the TinyDB concurrently, there might be racing conditions.Set a higher isolation level to mitigate this problem.db.isolevel=1isolevel:No isolation, best performance.Serialised(Atomic) CRUD operations. (Also ensures thread safety) (default)Deepcopy documents on insertion and retrieving. (CRUD) (EnsuresIndex&Query Cacheconsistency)DB-level cachingDB-level caching improves performance dramatically.However, this may cause data inconsistency betweenStorageandTinyDBif the file thatStoragereferred to is been shared.To disable it:db=TinyDB("./path",no_dbcache=True)Example Codes:Simple OneimportasynciofromasynctinydbimportTinyDB,Queryasyncdefmain():db=TinyDB('test.json')awaitdb.insert({"answer":42})print(awaitdb.search(Query().answer==42))# >>> [{'answer': 42}]asyncio.run(main())Event Hooks Exampleasyncdefmain():db=TinyDB('test.json')@db.storage.on.write.preasyncdefmul(ev:str,s:Storage,data:dict):data["_default"]["1"]['answer']*=2# directly manipulate on data@db.storage.on.write.postasyncdef_print(ev,s,anystr):print(anystr)# print json dumped stringawaitdb.insert({"answer":21})# insert() will trigger both write eventsawaitdb.close()# Reloaddb=TinyDB('test.json')print(awaitdb.search(Query().answer==42))# >>> [{'answer': 42}]Customise ID ClassInherit fromBaseIDand implement the following methods, and then you are good to go.fromasynctinydbimportBaseIDclassMyID(BaseID):def__init__(self,value:Any):"""You should be able to convert str into MyID instance if you want to use JSONStorage."""def__str__(self)->str:"""Optional.It should be implemented if you want to use JSONStorage."""def__hash__(self)->int:...def__eq__(self,other:object)->bool:...@classmethoddefnext_id(cls,table:Table,keys)->MyID:"""It should return a unique ID."""@classmethoddefmark_existed(cls,table:Table,new_id):"""Marks an ID as existing; the same ID shouldn't be generated by next_id."""@classmethoddefclear_cache(cls,table:Table):"""Clear cache of existing IDs, if such cache exists."""Customise Document ClassfromasynctinydbimportBaseDocumentclassMyDoc(BaseDocument):"""I am too lazy to write those necessary methods."""Anyways, a BaseDocument class looks like this:classBaseDocument(Mapping[IDVar,Any]):@property@abstractmethoddefdoc_id(self)->IDVar:raiseNotImplementedError()@doc_id.setterdefdoc_id(self,value:IDVar):raiseNotImplementedError()Make sure you have implemented all the methods required byBaseDocumentclass.Dev-ChangesStorageclosedproperty: OriginalTinyDBwon't raise exceptions when operating on a closed file. Now the propertyclosedofStorageclasses is required to be implemented[^why-closed][^operating-on-closed].Storage data converting: The responsibility of converting the data to the correct type is transferred to the Storage[^2]is_cacheablemethod inQueryInstanceis changed tocacheableproperty and will be deprecated.MiscLazy-load: File loading & dirs creating are delayed to the first IO operation.CachingMiddleWare:WRITE_CACHE_SIZEis now instance-specific.Example:TinyDB("test.db", storage=CachingMiddleWare(JSONStorage, 1024))searchaccepts optionalcond, returns all docs if no arguments are providedgetandcontainsraisesValueErrorinstead ofRuntimeErrorwhencondanddoc_idare bothNoneLRUCachestorestuples of ids instead oflists of docssearchandgettreatdoc_idanddoc_idsas extra conditions instead of ignoring conditions when IDs are provided. That is to say, whencondanddoc_id(s)are passed, they return docs satisfiescondand is indoc_id(s).[^1]: Why notorjson? Becauseujsonis fast enough and has more features.
[^2]: e.g.JSONStorageneeds to convert the keys tostrby itself.
[^UUID-version]:Currently using UUID4
[^disable-db-level]: SeeDB-level cachingto learn how to disable this feature if it causes dirty reads.
[^isolevel]: Seeisolevel[^why-closed]: This is forMiddlewareclasses to reliably determine whether theStorageis closed, so they can raiseIOError[^operating-on-closed]: AnIOErrorshould be raised when operating on closed storage.