Package: pydastic
PydasticPydastic is an elasticsearch python ORM based on Pydantic.💾 InstallationPip:pipinstallpydasticPoetry:poetryaddpydastic🚀 Core FeaturesSimple CRUD operations supportedSessions for simplifying bulk operations (a la SQLAlchemy)Dynamic index support when committing operations📋 UsageDefining ModelsfrompydasticimportESModelclassUser(ESModel):name:strphone:Optional[str]last_login:datetime=Field(default_factory=datetime.now)classMeta:index="user"Establishing ConnectionAn elasticsearch connection can be setup by using theconnectfunction. This function adopts the same signature as theelasticsearch.Elasticsearchclient and supports editor autocomplete.
Make sure to call this only once. No protection is put in place against multiple calls, might affect performance negatively.frompydasticimportconnectconnect(hosts="localhost:9200")CRUD: Create, Update# Create and save docuser=User(name="John",age=20)user.save(wait_for=True)# wait_for explained belowassertuser.id!=None# Update docuser.name="Sam"user.save(wait_for=True)CRUD: Read Documentgot=User.get(id=user.id)assertgot==userCRUD: Deleteuser=User(name="Marie")user.save(wait_for=True)user.delete(wait_for=True)SessionsSessions are inspired bySQL Alchemy's sessions, and are used for simplifying bulk operations using the Elasticsearch client. From what I've seen, the ES client makes it pretty hard to use the bulk API, so they created bulk helpers (which in turn have incomplete/wrong docs).john=User(name="John")sarah=User(name="Sarah")withSession()assession:session.save(john)session.save(sarah)session.commit()With an ORM, bulk operations can be exposed neatly through a simple API. Pydastic also offers more informative errors on issues encountered during bulk operations. This is possible by suppressing the built-in elastic client errors and extracting more verbose ones instead.Example error:pydastic.error.BulkError:[{"update":{"_index":"user","_type":"_doc","_id":"test","status":404,"error":{"type":"document_missing_exception","reason":"[_doc][test]: document missing","index_uuid":"cKD0254aQRWF-E2TMxHa4Q","shard":"0","index":"user"}}},{"update":{"_index":"user","_type":"_doc","_id":"test2","status":404,"error":{"type":"document_missing_exception","reason":"[_doc][test2]: document missing","index_uuid":"cKD0254aQRWF-E2TMxHa4Q","shard":"0","index":"user"}}}]The sessions API will also be available through a context manager before the v1.0 release.Dynamic Index SupportPydastic also supports dynamic index specification. The model Metaclass index definition is still mandatory, but if an index is specified when performing operations, that will be used instead.
The model Metaclass index is technically a fallback, although most users will probably be using a single index per model. For some users, multiple indices per model are needed (for example one user index per company).user=User(name="Marie")user.save(index="my-user",wait_for=True)user.delete(index="my-user",wait_for=True)Notes on testingWhen writing tests with Pydastic (even applies when writing tests with the elasticsearch client), remember to use thewait_for=Trueargument when executing operations. If this is not used, then the test will continue executing even if Elasticsearch hasn't propagated the change to all nodes, giving you weird results.For example if you save a document, then try getting it directly after, you'll get a document not found error. This is solved by using the wait_for argument in Pydastic (equivalent torefresh="wait_for"in Elasticsearch)Here isa referenceto where this argument is listed in the docs.It's also supported in the bulk helpers even though its not mentioned in their docs, but you wouldn't figure that out unless you dug into their source and traced back several function calls where*args**kwargsare just being forwarded across calls.. :)Support Elasticsearch VersionsPart of the build flow is running the tests using elasticsearch 7.12.0 DB as well as python client, and using 8.1.2 as well (DB as well as client, as part of a build matrix).
This ensures support for multiple versions.📈 ReleasesNone yet.You can see the list of available releases on theGitHub Releasespage.We followSemantic Versionsspecification.We useRelease Drafter. As pull requests are merged, a draft release is kept up-to-date listing the changes, ready to publish when you’re ready. With the categories option, you can categorize pull requests in release notes using labels.🛡 LicenseThis project is licensed under the terms of theMITlicense. SeeLICENSEfor more details.📃 Citation@misc{pydastic,author={Rami Awar},title={Pydastic is an elasticsearch python ORM based on Pydantic.},year={2022},publisher={GitHub},journal={GitHub repository},howpublished={\url{https://github.com/ramiawar/pydastic}}}