Package: scyllapy
Async Scylla driver for pythonPython driver for ScyllaDB written in Rust. Though description says it's for scylla,
however it can be used with Cassandra and AWS keyspaces as well.This driver uses officialScyllaDB driverforRustand exposes python API to interact with it.InstallationTo install it, use your favorite package manager for python packages:pipinstallscyllapyAlso, you can build from sources. To do it, install stable rust,maturinand openssl libs.maturinbuild--release--outdist# Then install whl file from dist folder.pipinstalldist/*UsageThe usage is pretty straitforward. Create a Scylla instance, run startup and start executing queries.importasynciofromscyllapyimportScyllaasyncdefmain():scylla=Scylla(["localhost:9042"],keyspace="keyspace")awaitscylla.startup()awaitscylla.execute("SELECT * FROM table")awaitscylla.shutdown()if__name__=="__main__":asyncio.run(main())Parametrizing queriesWhile executing queries sometimes you may want to fine-tune some parameters, or dynamically pass values to the query.Passing parameters is simple. You need to add a paramters list to the query.awaitscylla.execute("INSERT INTO otps(id, otp) VALUES (?, ?)",[uuid.uuid4(),uuid.uuid4().hex],)Queries can be modified further by usingQueryclass. It allows you to define
consistency for query or enable tracing.fromscyllapyimportScylla,Query,Consistency,SerialConsistencyasyncdefmake_query(scylla:Scylla)->None:query=Query("SELECT * FROM table",consistency=Consistency.ALL,serial_consistency=SerialConsistency.LOCAL_SERIAL,request_timeout=1,timestamp=int(time.time()),is_idempotent=False,tracing=True,)result=awaitscylla.execute(query)print(result.all())Also, with queries you can tweak random parameters for a specific execution.query=Query("SELECT * FROM table")new_query=query.with_consistency(Consistency.ALL)Allwith_methods create new query, copying all other parameters.Named parametersAlso, you can provide named parameters to querties, by using name
placeholders instead of?.For example:asyncdefinsert(scylla:Scylla):awaitscylla.execute("INSERT INTO table(id, name) VALUES (:id, :name)",params={"id":uuid.uuid4(),"name":uuid.uuid4().hex})Important note: All variables should be in snake_case.
Otherwise the error may be raised or parameter may not be placed in query correctly.
This happens, because scylla makes all parameters in query lowercase.The scyllapy makes all parameters lowercase, but you may run into problems,
if you use multiple parameters that differ only in cases of some letters.Preparing queriesAlso, queries can be prepared. You can either prepare raw strings, orQueryobjects.fromscyllapyimportScylla,Query,PreparedQueryasyncdefprepare(scylla:Scylla,query:str|Query)->PreparedQuery:returnawaitscylla.prepare(query)You can execute prepared queries by passing them toexecutemethod.asyncdefrun_prepared(scylla:Scylla)->None:prepared=awaitscylla.prepare("INSERT INTO memse(title) VALUES (?)")awaitscylla.execute(prepared,("American joke",))BatchingWe support batches. Batching can help a lot when you have lots of queries that you want to execute at the same time.fromscyllapyimportScylla,Batchasyncdefrun_batch(scylla:Scylla,num_queries:int)->None:batch=Batch()for_inrange(num_queries):batch.add_query("SELECT * FROM table WHERE id = ?")awaitscylla.batch(batch,[(i,)foriinrange(num_queries)])Here we pass query as strings. But you can also add Prepared statements or Query objects.Also, note that we pass list of lists as parametes for execute. Each element of
the list is going to be used in the query with the same index. But named parameters
are not supported for batches.asyncdefrun_batch(scylla:Scylla,num_queries:int)->None:batch=Batch()batch.add_query("SELECT * FROM table WHERE id = :id")awaitscylla.batch(batch,[{"id":1}])# Will rase an error!PaginationSometimes you want to query lots of data. For such cases it's better not to
fetch all results at once, but fetch them using pagination. It reduces load
not only on your application, but also on a cluster.To execute query with pagination, simply addpaged=Truein execute method.
After doing so,executemethod will returnIterableQueryResult, instead ofQueryResult.
Instances ofIterableQueryResultcan be iterated withasync forstatements.
You, as a client, won't see any information about pages, it's all handeled internally within a driver.Please note, that paginated queries are slower to fetch all rows, but much more memory efficent for large datasets.result=awaitscylla.execute("SELECT * FROM table",paged=True)asyncforrowinresult:print(row)Of course, you can change how results returned to you, by either usingscalarsoras_cls. For example:asyncdeffunc(scylla:Scylla)->None:rows=awaitscylla.execute("SELECT id FROM table",paged=True)# Will print ids of each returned row.asyncfortest_idinrows.scalars():print(test_id)fromdataclassesimportdataclass@dataclassclassMyDTO:id:intval:intasyncdeffunc(scylla:Scylla)->None:rows=awaitscylla.execute("SELECT * FROM table",paged=True)# Will print ids of each returned row.asyncformy_dtoinrows.as_cls(MyDTO):print(my_dto.id,my_dto.val)Execution profilesYou can define profiles usingExecutionProfileclass. After that the
profile can be used while creating a cluster or when defining queries.fromscyllapyimportConsistency,ExecutionProfile,Query,Scylla,SerialConsistencyfromscyllapy.load_balancingimportLoadBalancingPolicy,LatencyAwarenessdefault_profile=ExecutionProfile(consistency=Consistency.LOCAL_QUORUM,serial_consistency=SerialConsistency.LOCAL_SERIAL,request_timeout=2,)asyncdefmain():query_profile=ExecutionProfile(consistency=Consistency.ALL,serial_consistency=SerialConsistency.SERIAL,# Load balancing cannot be constructed without running event loop.# If you won't do it inside async funcion, it will result in error.load_balancing_policy=awaitLoadBalancingPolicy.build(token_aware=True,prefer_rack="rack1",prefer_datacenter="dc1",permit_dc_failover=True,shuffling_replicas=True,latency_awareness=LatencyAwareness(minimum_measurements=10,retry_period=1000,exclusion_threshold=1.4,update_rate=1000,scale=2,),),)scylla=Scylla(["192.168.32.4"],default_execution_profile=default_profile,)awaitscylla.startup()awaitscylla.execute(Query("SELECT * FROM system_schema.keyspaces;",profile=query_profile,))ResultsEvery query returns a class that represents returned rows. It allows you to not fetch
and parse actual data if you don't need it.Please be awarethat if your query was
not expecting any rows in return. Like forUpdateorInsertqueries. TheRuntimeErroris raised when you callallorfirst.result=awaitscylla.execute("SELECT * FROM table")print(result.all())If you were executing query with tracing, you can get tracing id from results.result=awaitscylla.execute(Query("SELECT * FROM table",tracing=True))print(result.trace_id)Also it's possible to parse your data using custom classes. You
can use dataclasses or Pydantic.fromdataclassesimportdataclass@dataclassclassMyDTO:id:uuid.UUIDname:strresult=awaitscylla.execute("SELECT * FROM inbox")print(result.all(as_class=MyDTO))Or with pydantic.frompydanticimportBaseModelclassMyDTO(BaseModel):user_id:uuid.UUIDchat_id:uuid.UUIDresult=awaitscylla.execute("SELECT * FROM inbox")print(result.all(as_class=MyDTO))Extra typesSince Rust enforces typing, it's hard to identify which value
user tries to pass as a parameter. For example,1that comes from python
can be eithertinyint,smallintor evenbigint. But we cannot say for sure
how many bytes should we send to server. That's why we created some extra_types to
eliminate any possible ambigousnity.You can find these types inextra_typesmodule from scyllapy.fromscyllapyimportScylla,extra_typesasyncdefexecute(scylla:Scylla)->None:awaitscylla.execute("INSERT INTO table(id, name) VALUES (?, ?)",[extra_types.BigInt(1),"memelord"],)User defined typesWe also support user defined types. You can pass them as a parameter to query.
Or parse it as a model in response.Here's binding example. Imagine we have defined a type in scylla like this:CREATE TYPE IF NOT EXISTS test (
    id int,
    name text
);Now we need to define a model for it in python.fromdataclassesimportdataclassfromscyllapy.extra_typesimportScyllaPyUDT@dataclassclassTestUDT(ScyllaPyUDT):# Always define fields in the same order as in scylla.# Otherwise you will get an error, or wrong data.id:intname:strasyncdefexecute(scylla:Scylla)->None:awaitscylla.execute("INSERT INTO table(id, udt_col) VALUES (?, ?)",[1,TestUDT(id=1,name="test")],)We also support pydantic based models. Decalre them like this:frompydanticimportBaseModelfromscyllapy.extra_typesimportScyllaPyUDTclassTestUDT(BaseModel,ScyllaPyUDT):# Always define fields in the same order as in scylla.# Otherwise you will get an error, or wrong data.id:intname:strQuery buildingScyllaPy gives you ability to build queries,
instead of working with raw cql. The main advantage that it's harder to make syntax error,
while creating queries.Base classes for Query building can be found inscyllapy.query_builder.Usage example:fromscyllapyimportScyllafromscyllapy.query_builderimportInsert,Select,Update,Deleteasyncdefmain(scylla:Scylla):awaitscylla.execute("CREATE TABLE users(id INT PRIMARY KEY, name TEXT)")user_id=1# We create a user with id and name.awaitInsert("users").set("id",user_id).set("name","user").if_not_exists().execute(scylla)# We update it's name to be user2awaitUpdate("users").set("name","user2").where("id = ?",[user_id]).execute(scylla)# We select all users with id = user_id;res=awaitSelect("users").where("id = ?",[user_id]).execute(scylla)# Verify that it's correct.assertres.first()=={"id":1,"name":"user2"}# We delete our user.awaitDelete("users").where("id = ?",[user_id]).if_exists().execute(scylla)res=awaitSelect("users").where("id = ?",[user_id]).execute(scylla)# Verify that user is deleted.assertnotres.all()awaitscylla.execute("DROP TABLE users")Also, you can pass built queries into InlineBatches. You cannot use queries built with query_builder module with default batches. This constraint is exists, because we
need to use values from within your queries and should ignore all parameters passed inbatchmethod of scylla.Here's batch usage example.fromscyllapyimportScylla,InlineBatchfromscyllapy.query_builderimportInsertasyncdefexecute_batch(scylla:Scylla)->None:batch=InlineBatch()foriinrange(10):Insert("users").set("id",i).set("name","test").add_to_batch(batch)awaitscylla.batch(batch)PagingQueries that were built with QueryBuilder also support paged returns.
But it supported only for select, because update, delete and insert should
not return anything and it makes no sense implementing it.
To make builtSelectquery return paginated iterator, add paged parameter in execute method.rows=awaitSelect("test").execute(scylla,paged=True)asyncforrowinrows:print(row['id'])