Package: pynocular
PynocularPynocular is a lightweight ORM that lets you query your database using Pydantic models and asyncio.With Pynocular you can decorate your existing Pydantic models to sync them with the corresponding table in your
database, allowing you to persist changes without ever having to think about the database. Transaction management is
automatically handled for you so you can focus on the important parts of your code. This integrates seamlessly with frameworks that use Pydantic models such as FastAPI.Features:Fully supports asyncio to write to SQL databasesProvides simple methods for basic SQLAlchemy support (create, delete, update, read)Contains access to more advanced functionality such as custom SQLAlchemy selectsContains helper functions for creating new database tablesAdvanced transaction management system allows you to conditionally put requests in transactionsTable of Contents:InstallationGuideBasic UsageAdvanced UsageCreating database tablesDevelopmentInstallationPynocular requires Python 3.6 or above.pipinstallpynocular# orpoetryaddpynocularGuideBasic UsagePynocular works by decorating your base Pydantic model with the functiondatabase_model. Once decorated
with the proper information, you can proceed to use that model to interface with your specified database table.The first step is to define aDBInfoobject. This will contain the connection information to your database.frompynocular.enginesimportDatabaseType,DBInfo# Example below shows how to connect to a locally-running Postgres databaseconnection_string=f"postgresql://{db_user_name}:{db_user_password}@localhost:5432/{db_name}?sslmode=disable")db_info=DBInfo(connection_string)Object ManagementOnce you define adb_infoobject, you are ready to decorate your Pydantic models and interact with your database!frompydanticimportBaseModel,Fieldfrompynocular.database_modelimportdatabase_model,UUID_STRfrommy_packageimportdb_info@database_model("organizations",db_info)classOrg(BaseModel):id:Optional[UUID_STR]=Field(primary_key=True,fetch_on_create=True)name:str=Field(max_length=45)slug:str=Field(max_length=45)tag:Optional[str]=Field(max_length=100)created_at:Optional[datetime]=Field(fetch_on_create=True)updated_at:Optional[datetime]=Field(fetch_on_update=True)#### Object management# Create a new Org via `create`org=awaitOrg.create(name="new org",slug="new-org")# Create a new Org via `save`org2=Org(name="new org2",slug="new-org2")awaitorg2.save()# Update an orgorg.name="renamed org"awaitorg.save()# Delete orgawaitorg.delete()# Get orgorg3=awaitOrg.get(org2.id)assertorg3==org2# Get a list of orgsorgs=awaitOrg.get_list()# Get a filtered list of orgsorgs=awaitOrg.get_list(tag="green")# Get orgs that have several different tagsorgs=awaitOrg.get_list(tag=["green","blue","red"])# Fetch the latest state of a table in the dborg3.name="fake name"awaitorg3.fetch()assertorg3.name=="new org2"SerializationDatabaseModels have their own serialization functions to convert to and from
dictionaries.# Serializing org with `to_dict()`org=Org.create(name="org serialize",slug="org-serialize")org_dict=org.to_dict()expected_org_dict={"id":"e64f6c7a-1bd1-4169-b482-189bd3598079","name":"org serialize","slug":"org-serialize","created_at":"2018-01-01 7:03:45","updated_at":"2018-01-01 9:24:12"}assertorg_dict==expected_org_dict# De-serializing org with `from_dict()`new_org=Org.from_dict(expected_org_dict)assertorg==new_orgUsing Nested DatabaseModelsPynocular also supports basic object relationships. If your database tables have a
foreign key reference you can leverage that in your pydantic models to increase the
accessibility of those related objects.frompydanticimportBaseModel,Fieldfrompynocular.database_modelimportdatabase_model,nested_model,UUID_STRfrommy_packageimportdb_info@database_model("users",db_info)classUser(BaseModel):id:Optional[UUID_STR]=Field(primary_key=True,fetch_on_create=True)username:str=Field(max_length=100)created_at:Optional[datetime]=Field(fetch_on_create=True)updated_at:Optional[datetime]=Field(fetch_on_update=True)@database_model("organizations",db_info)classOrg(BaseModel):id:Optional[UUID_STR]=Field(primary_key=True,fetch_on_create=True)name:str=Field(max_length=45)slug:str=Field(max_length=45)# `organizations`.`tech_owner_id` is a foreign key to `users`.`id`tech_owner:Optional[nested_model(User,reference_field="tech_owner_id")]# `organizations`.`business_owner_id` is a foreign key to `users`.`id`business_owner:nested_model(User,reference_field="business_owner_id")tag:Optional[str]=Field(max_length=100)created_at:Optional[datetime]=Field(fetch_on_create=True)updated_at:Optional[datetime]=Field(fetch_on_update=True)tech_owner=awaitUser.create("tech owner")business_owner=awaitUser.create("business owner")# Creating org with only business owner setorg=awaitOrg.create(name="org name",slug="org-slug",business_owner=business_owner)assertorg.business_owner==business_owner# Add tech ownerorg.tech_owner=tech_ownerawaitorg.save()# Fetch from the db and check idsorg2=Org.get(org.id)assertorg2.tech_owner.id==tech_owner.idassertorg2.business_owner.id==business_owner.id# Swap user rolesorg2.tech_owner=business_ownerorg2.business_owner=tech_ownerawaitorg2.save()org3=awaitOrg.get(org2.id)assertorg3.tech_owner.id==business_owner.idassertorg3.business_owner.id==tech_owner.id# Serialize orgorg_dict=org3.to_dict()expected_org_dict={"id":org3.id,"name":"org name","slug":"org-slug","business_owner_id":tech_owner.id,"tech_owner_id":business_owner.id,"tag":None,"created_at":org3.created_at,"updated_at":org3.updated_at}assertorg_dict==expected_org_dictWhen usingDatabaseModel.get(..), any foreign references will need to be resolved before any properties besides the primary ID can be accessed. If you try to access a property before callingfetch()on the nested model, aNestedDatabaseModelNotResolvederror will be thrown.org_get=awaitOrg.get(org3.id)org_get.tech_owner.id# Does not raise `NestedDatabaseModelNotResolved`org_get.tech_owner.username# Raises `NestedDatabaseModelNotResolved`org_get=awaitOrg.get(org3.id)awaitorg_get.tech_owner.fetch()org_get.tech_owner.username# Does not raise `NestedDatabaseModelNotResolved`Alternatively, callingDatabaseModel.get_with_refs()instead ofDatabaseModel.get()will
automatically fetch the referenced records and fully resolve those objects for you.org_get_with_refs=awaitOrg.get_with_refs(org3.id)org_get_with_refs.tech_owner.username# Does not raise `NestedDatabaseModelNotResolved`There are some situations where none of the objects have been persisted to the
database yet. In this situation, you can callDatabase.save(include_nested_models=True)on the object with the references and it will persist all of them in a transaction.# We create the objects but dont persist themtech_owner=User("tech owner")business_owner=User("business owner")org=Org(name="org name",slug="org-slug",business_owner=business_owner)awaitorg.save(include_nested_models=True)Special Type argumentsWith Pynocular you can set fields to be optional and set by the database. This is useful
if you want to let the database autogenerate your primary key orcreated_atandupdated_atfields
on your table. To do this you must:Wrap the typehint inOptionalProvide keyword arguments offetch_on_create=Trueorfetch_on_update=Trueto theFieldclassAdvanced UsageFor most use cases, the basic usage defined above should suffice. However, there are certain situations
where you don't necessarily want to fetch each object or you need to do more complex queries that
are not exposed by theDatabaseModelinterface. Below are some examples of how those situations can
be addressed using Pynocular.Tables with compound keysPynocular supports tables that use multiple fields as its primary key such as join tables.frompydanticimportBaseModel,Fieldfrompynocular.database_modelimportdatabase_model,nested_model,UUID_STRfrommy_packageimportdb_info@database_model("user_subscriptions",db_info)classUserSubscriptions(BaseModel):user_id:UUID_STR=Field(primary_key=True,fetch_on_create=True)subscription_id:UUID_STR=Field(primary_key=True,fetch_on_create=True)name:struser_sub=awaitUserSub.create(user_id="4d4254c4-8e99-45f9-8261-82f87991c659",subscription_id="3cc5d476-dbe6-4cc1-9390-49ebd7593a3d",name="User 1's subscriptions")# Get the users subscription and confirm its the sameuser_sub_get=awaitUserSub.get(user_id="4d4254c4-8e99-45f9-8261-82f87991c659",subscription_id="3cc5d476-dbe6-4cc1-9390-49ebd7593a3d",)assertuser_sub_get==user_sub# Change a property value like any other objectuser_sub_get.name="change name"awaituser_sub_get.save()Batch operations on tablesSometimes you want to insert a bunch of records into a database and you don't want to do an insert for each one.
This can be handled by thecreate_listfunction.org_list=[Org(name="org1",slug="org-slug1"),Org(name="org2",slug="org-slug2"),Org(name="org3",slug="org-slug3"),]awaitOrg.create_list(org_list)This function will insert all records into your database table in one batch.If you have a use case that requires deleting a bunch of records based on some field value, you can usedelete_records:# Delete all records with the tag "green"awaitOrg.delete_records(tag="green")# Delete all records with if their tag has any of the following: "green", "blue", "red"awaitOrg.delete_records(tag=["green","blue","red"])Sometimes you may want to update the value of a record in a database without having to fetch it first. This can be accomplished by using
theupdate_recordfunction:awaitOrg.update_record(id="05c0060c-ceb8-40f0-8faa-dfb91266a6cf",tag="blue")org=awaitOrg.get("05c0060c-ceb8-40f0-8faa-dfb91266a6cf")assertorg.tag=="blue"Complex queriesSometimes your application will require performing complex queries, such as getting the count of each unique field value for all records in the table.
Because Pynocular is backed by SQLAlchemy, we can access table columns directly to write pure SQLAlchemy queries as well!fromsqlalchemyimportfunc,selectfrompynocular.enginesimportDBEngineasyncdefgenerate_org_stats():query=(select([func.count(Org.column.id),Org.column.tag]).group_by(Org.column.tag).order_by(func.count().desc()))asyncwithawaitDBEngine.transaction(Org._database_info,is_conditional=False)asconn:result=awaitconn.execute(query)return[dict(row)asyncforrowinresult]NOTE:DBengine.transactionis used to create a connection to the database using the credentials passed in.
Ifis_conditionalisFalse, then it will add the query to any transaction that is opened in the call chain. This allows us to make database calls
in different functions but still have them all be under the same database transaction. If there is no transaction opened in the call chain it will open
a new one and any subsequent calls underneath that context manager will be added to the new transaction.Ifis_conditionalisTrueand there is no transaction in the call chain, then the connection will not create a new transaction. Instead, the query will be performed without a transaction.Creating database and tablesWith Pynocular you can use simple python code to create new databases and database tables. All you need is a working connection string to the database host, aDatabaseInfoobject that contains the information of the database you want to create, and a properly decorated pydantic model. When you decorate a pydantic model with Pynocular, it creates a SQLAlchemy table as a private variable. This can be accessed via the_tableproperty
(although accessing private variables is not recommended).frompynocular.db_utilimportcreate_new_database,create_tablefrommy_packageimportOrg,db_infoconnection_string="postgresql://postgres:XXXX@localhost:5432/postgres?sslmode=disable"# Creates a new database and "organizations" table in that databaseawaitcreate_new_database(connection_string,db_info)awaitcreate_table(db_info,Org._table)Unit Testing with DatabaseModelsPynocular comes with tooling to write unit tests against your DatabaseModels, giving you
the ability to test your business logic without the extra work and latency involved in
managing a database. All you have to do is use thepatch_database_modelcontext
manager provided in Pynocular.frompynocular.patch_modelsimportpatch_database_modelfrommy_packageimportOrg,Userwithpatch_database_model(Org):orgs=[Org(id=str(uuid4()),name="orgus borgus",slug="orgus_borgus"),Org(id=str(uuid4()),name="orgus borgus2",slug="orgus_borgus"),]awaitOrg.create_list(orgs)fetched_orgs=awaitOrg.get_list(name=orgs[0].name)assertorgs[0]==fetched_orgs[0]# patch_database_model also works with nested modelsusers=[User(id=str(uuid4()),username="Bob"),User(id=str(uuid4()),username="Sally"),]orgs=[Org(id=str(uuid4()),name="orgus borgus",slug="orgus_borgus",tech_owner=users[0],business_owner=users[1],),]withpatch_database_model(Org,models=orgs),patch_database_model(User,models=users):org=awaitOrg.get(orgs[0].id)org.name="new test name"users[0].username="bberkley"# Save the username update when saving the org model updateawaitorg.save(include_nested_models=True)# Get the org with the resolved nested modelorg_get=awaitOrg.get_with_refs(org.id)assertorg_get.name=="new test name"assertorg_get.tech_owner.username=="bberkley"DevelopmentTo develop Pynocular, install dependencies and enable the pre-commit hook.The example below is using Python 3.9 but you can replace this with any supported version of Python.Install Python 3.9 and activate it in your shell.sudoyuminstalllibffi-devel# Needed for ctypes to install poetrypyenvinstall3.9.7
pyenvshell3.9.7Install dependencies and enable the pre-commit hook.pipinstallpre-commitpoetry
poetryinstall
pre-commitinstallRun tests to confirm everything is installed correctly.poetryrunpytest