Package: bytehouse-sqlalchemy
ByteHouse SQLAlchemy ConnectorIntroductionByteHouse SQLAlchemy connector is ByteHouse dialect forSQLAlchemy, which is the Python SQL toolkit and object
relational mapper enabling application developers the full power and flexibility of SQL. The connector is built on
top of ByteHouse python driver which follows Python DB API 2.0 specification. The connector supports both SQLAlchemy
Core and SQLAlchemy ORM APIs.RequirementsPython v3.6 or higherInstallation from PyPIThe latest release version can be installed from here:pip install bytehouse-sqlalchemyInstallation from githubThe current development version can be installed from here:pip install git+https://github.com/bytehouse-cloud/bytehouse-sqlalchemy@master#egg=bytehouse-driverCreating ByteHouse AccountYou need to create a ByteHouse account in order to use Python Driver. You can simply create a free account with
the process mentioned in our official website documentation:https://docs.bytehouse.cloud/en/docs/quick-startYou can also create ByteHouse account through Volcano Engine by ByteDance:https://www.volcengine.com/product/bytehouse-cloudSQLAlchemy APIsSQLAlchemy has two distinct APIs, one building on top of the other. These APIs areCoreandORM.SQLAlchemy CoreSQLAlchemy Core APIs manage connectivity to a database, interact with database queries and results & manage programmatic
construction of SQL statements.Engine ConfigurationEngineis the starting point of any SQLAlchemy application.Enginerefers to aDialectandConnection Pool, whereDialectis aPythonobject that represents information and methods that allow database operations to proceed on a
particular kind of database backend and a particular kind of Python driver for that database.The engine and its underlying connection pool do not establish the first actual DBAPI connection until theEngine.connect()method is called, or an operation which is dependent on this method such asEngine.execute()is
invoked. In this way,EngineandPoolcan be said to have lazy initialization behaviour.ByteHouse RegionsCurrently, the driver supports the following region names across different cloud providers. Alternatively, if you know
the host address of ByteHouse server, you can directly use host address & omit region name.Region NameTarget ServerAP-SOUTHEAST-1gateway.aws-ap-southeast-1.bytehouse.cloud:19000VOLCANO-CN-NORTH-1bytehouse-cn-beijing.volces.com:19000Construction from ByteHouse URIRegion & Password FormatRequired parameters:regionaccountuserpasswordfromsqlalchemyimportcreate_engineengine=create_engine("bytehouse:///?region={}&account={}&user={}&password={}&database={}".format($REGION,$ACCOUNT,$USER,$PASSWORD,$DATABASE))Host Address & Password FormatRequired parameters:hostportaccountuserpasswordfromsqlalchemyimportcreate_engineengine=create_engine("bytehouse://{}:{}/?account={}&user={}&password={}&database={}".format($HOST,$PORT,$ACCOUNT,$USER,$PASSWORD,$DATABASE))For API Key authentication, user is always 'bytehouse'Region & API Key FormatRequired parameters:regionpasswordfromsqlalchemyimportcreate_engineengine=create_engine("bytehouse:///?region={}&user=bytehouse&password={}&database={}".format($REGION,$API_KEY,$DATABASE))Host Address & API Key FormatRequired parameters:hostportpasswordfromsqlalchemyimportcreate_engineengine=create_engine("bytehouse://{}:{}/?user=bytehouse&password={}&database={}".format($HOST,$PORT,$API_KEY,$DATABASE))Programmatic Constructionfromsqlalchemy.engineimportURLuri=URL.create("bytehouse",username="bytehouse",password="{}".format($API_KEY),host="{}".format($HOST),port="{}".format($PORT),database="{}".format($DATABASE),)engine=create_engine(uri)Working with ConnectionsThe most basic function of theEngineis to provide access to aConnection, which can execute SQL statements.
To execute a textual statement to the database looks like:fromsqlalchemyimporttextwithengine.connect()asconnection:result=connection.execute(text("SELECT 1"))forrowinresult:print(row[0])The object returned here is known as theCursorResult, which refers to a DBAPI cursor. The DBAPI cursor will be
closed by theCursorResultwhen all of its result rows are exhausted. When theConnectionis closed at the end of
the with block, the referenced DBAPI connection is released to the connection pool.Working with TransactionsTheConnectionobject provides aConnection.begin()method which returns aTransactionobject. The transaction is
committed when the block completes. If an exception is raised, the transaction would be rolled back, and the exception
would be propagated outwards.withengine.connect()asconnection:withconnection.begin():connection.execute(user_table.insert(),{"user_id":7,"user_name":"Jane"})connection.execute(user_table.insert(),{"user_id":8,"user_name":"Adam"})Database MetaDataMetaDatais a container object that keeps together different entities or features of a database.fromsqlalchemyimportMetaDatametadata_obj=MetaData()Table Definition : Constructor StyleTableclass would represent a table where two primary arguments are the table name and theMetaDataobject which
it will be associated with. The remaining positional arguments are Column objects describing eachColumnand engine
(CnchMergeTree) definition.fromsqlalchemyimportTable,Column,Integer,String,funcfrombytehouse_sqlalchemyimportenginesuser_table=Table("user",metadata,Column("user_id",Integer,primary_key=True),Column("user_name",String(16),nullable=False),engines.CnchMergeTree(order_by=func.tuple()))Creating and Dropping TablesThe general way of creating all tables is to execute thecreate_all()method on theMetaDataobject. This method will
first check the existence of each individual table, and if not found, then executeCREATEstatements for all tables.
Similarly, for dropping all tables, we can execute thedrop_all()method on the MetaData object. Creating and dropping
individual tables can be done via thecreate()anddrop()methods.fromsqlalchemyimportTable,Column,Integer,String,func,MetaDatafrombytehouse_sqlalchemyimportenginesmetadata_obj=MetaData()user_table=Table("user",metadata,Column("user_id",Integer,primary_key=True),Column("user_name",String(16),nullable=False),engines.CnchMergeTree(order_by=func.tuple()))metadata_obj.create_all(engine)metadata_obj.drop_all(engine)Insertion and SelectionTable.insert()can be used to insert rows into the table, whereasTable.select()would fetch the result rows from the server.withengine.connect()asconnection:connection.execute(user_table.insert(),{"user_id":7,"user_name":"Jane"})result_set=connection.execute(user_table.select())SQLAlchemy ORMSQLAlchemy ORM is built on top of SQLAlchemy Core which provides object relational mapping capabilities that allows
users to define Python classes mapped to database tables. It extends the Core SQL expression language to allow
SQL queries to be composed and invoked in user defined objects.Create EngineTheEngineis a factory class that will create and maintain database connections for us, where connections are held
inside of a Connection Pool for fast reuse. The details regarding theEngineare described at the beginning of this doc.fromsqlalchemyimportcreate_engineengine=create_engine("bytehouse:///?region={}&account={}&user={}&password={}&database={}".format($REGION,$ACCOUNT,$USER,$PASSWORD,$DATABASE))Declarative MappingTheDeclarative Mappingdefines a base class using thedeclarative_base()function, which returns a new base class
from which new classes to be mapped may inherit from. A mapped class typically refers to a single particular database
table, the name of which is indicated by using the__tablename__class level attribute.fromsqlalchemyimportColumn,Integer,String,func,MetaDatafromsqlalchemy.ormimportdeclarative_basefrombytehouse_sqlalchemyimportenginesmetadata=MetaData(bind=engine)Base=declarative_base(metadata=metadata)classUser(Base):__tablename__="user_account"id=Column(Integer,primary_key=True)name=Column(String(30))fullname=Column(String)__table_args__=(engines.CnchMergeTree(order_by=func.tuple()),)Table CreationUsing table metadata and engine, we can generate DDL schema & execute in ByteHouse at once usingTable.__table__.create()method.User.__table__.create()Session and Object PersistWe can create objects of previously defined classes and pass them to the database using an object calledSession,
which uses theEngineto interact with the database. TheSession.add_all()is used to add multiple objects at once,
and theSession.commit()method would flush any pending changes to the database.fromsqlalchemy.ormimportSessionwithSession(engine)assession:spongebob=User(id=1,name="spongebob",fullname="Spongebob Squarepants")sandy=User(id=2,name="sandy",fullname="Sandy Cheeks")session.add_all([spongebob,sandy])session.commit()SELECT statementWe can use theselect()method to create a newSelectobject, which can then be invoked using aSessionobject.
Optionally, we can also use theSelect.where()method to filter out the results.fromsqlalchemyimportselectsession=Session(engine)stmt=select(User).where(User.name.in_(["spongebob","sandy"]))foruserinsession.scalars(stmt):print(user.name)Local DevelopmentChangesetup.cfgfile to include your connection credentials. For running tests locally, follow these steps:python testsrequire.py && pip install .
python -m pytest tests/LicenseThis project is distributed under the terms of the MIT license:http://www.opensource.org/licenses/mit-license.php