Package: ksql
A python wrapper for the KSQL REST API. Easily interact with the KSQL REST API using this library.Supported KSQLDB version: 0.10.1+
Supported Python version: 3.5+InstallationpipinstallksqlOrgitclonehttps://github.com/bryanyang0528/ksql-pythoncdksql-pythonpythonsetup.pyinstallGetting StartedSetup for KSQLThis is the GITHUB page of KSQL.https://github.com/confluentinc/ksqlIf you have installed open source Confluent CLI (e.g. by installing Confluent Open Source or Enterprise Platform), you can start KSQL and its dependencies with one single command:confluentstartksql-serverSetup for ksql-python APISetup for the KSQL API:fromksqlimportKSQLAPIclient=KSQLAPI('http://ksql-server:8088')Setup for KSQl API with logging enabled:importloggingfromksqlimportKSQLAPIlogging.basicConfig(level=logging.DEBUG)client=KSQLAPI('http://ksql-server:8088')Setup for KSQL API with Basic AuthenticationfromksqlimportKSQLAPIclient=KSQLAPI('http://ksql-server:8088',api_key="your_key",secret="your_secret")OptionsOptionTypeRequiredDescriptionurlstringyesYour ksql-server url. Example:http://ksql-server:8080timeoutintegernoTimout for Requests. Default:5api_keystringnoAPI Key to use on the requestssecretstringnoSecret to use on the requestsMain MethodsksqlThis method can be used for some KSQL features which are not supported via other specific methods likequery,create_streamorcreate_stream_as.
The following example shows how to execute theshow tablesstatement:client.ksql('show tables')Example Response[{'tables':{'statementText': 'showtables;','tables':[]}}]queryIt will execute sql query and keep listening streaming data.client.query('select * from table1')This command returns a generator. It can be printed e.g. by reading its values vianext(query)or a for loop. Here is a complete example:fromksqlimportKSQLAPIclient=KSQLAPI('http://localhost:8088')query=client.query('select * from table1')foriteminquery:print(item)Example Response{"row":{"columns":[1512787743388,"key1",1,2,3]},"errorMessage":null}
{"row":{"columns":[1512787753200,"key1",1,2,3]},"errorMessage":null}
{"row":{"columns":[1512787753488,"key1",1,2,3]},"errorMessage":null}
{"row":{"columns":[1512787753888,"key1",1,2,3]},"errorMessage":null}Query with HTTP/2Execute queries with the new/query-streamendpoint. DocumentedhereTo execute a sql query use the same syntax as the regular query, with the additionaluse_http2=Trueparameter.client.query('select * from table1',use_http2=True)A generator is returned with the following example response{"queryId":"44d8413c-0018-423d-b58f-3f2064b9a312","columnNames":["ORDER_ID","TOTAL_AMOUNT","CUSTOMER_NAME"],"columnTypes":["INTEGER","DOUBLE","STRING"]}
[3,43.0,"Palo Alto"]
[3,43.0,"Palo Alto"]
[3,43.0,"Palo Alto"]To terminate the query above use theclose_querycall.
Provide thequeryIdreturned from thequerycall.client.close_query("44d8413c-0018-423d-b58f-3f2064b9a312")Insert rows into a Stream with HTTP/2Uses the new/inserts-streamendpoint. Seedocumentationrows=[{"ORDER_ID":1,"TOTAL_AMOUNT":23.5,"CUSTOMER_NAME":"abc"},{"ORDER_ID":2,"TOTAL_AMOUNT":3.7,"CUSTOMER_NAME":"xyz"}]results=self.api_client.inserts_stream("my_stream_name",rows)An array of object will be returned on success, with the status of each row inserted.Simplified APIcreate_stream/ create_tableclient.create_stream(table_name=table_name,columns_type=columns_type,topic=topic,value_format=value_format)OptionsOptionTypeRequiredDescriptiontable_namestringyesname of stream/tablecolumns_typelistyesex:['viewtimebigint','useridvarchar','pageidvarchar']topicstringyesKafka topicvalue_formatstringnoJSON(Default) orDELIMITEDorAVROkeystringfor TableKey (used for JOINs)ResponsesIf create table/stream succeed:return TrueIf failed:raise a CreateError(respose_from_ksql_server)create_stream_asa simplified api for creating stream as selectclient.create_stream_as(table_name=table_name,select_columns=select_columns,src_table=src_table,kafka_topic=kafka_topic,value_format=value_format,conditions=conditions,partition_by=partition_by,**kwargs)CREATESTREAM<table_name>[WITH(kafka_topic=<kafka_topic>,value_format=<value_format>,property_name=expression...)]ASSELECT<select_columns>FROM<src_table>[WHERE<conditions>]PARTITIONBY<partition_by>];OptionsOptionTypeRequiredDescriptiontable_namestringyesname of stream/tableselect_columnslistyesyou can select[*]or['columnA', 'columnB']src_tablestringyesname of source tablekafka_topicstringnoThe name of the Kafka topic of this new stream(table).value_formatstringnoDELIMITED,JSON``(Default)or ``AVROconditionsstringnoThe conditions in the where clause.partition_bystringnoData will be distributed across partitions by this column.kwargspairnoplease providekey=valuepairs. Please see more options.KSQL JOINsKSQL JOINs between Streams and Tables are not supported yet via explicit methods, but you can use theksqlmethod for this like the following:client.ksql("CREATE STREAM join_per_user WITH (VALUE_FORMAT='AVRO', KAFKA_TOPIC='join_per_user') AS SELECT Time, Amount FROM source c INNER JOIN users u on c.user = u.userid WHERE u.USERID = 1")FileUploaduploadRun commands from a .ksql file. Can only support ksql commands and not streaming queries.fromksqlimportFileUploadpointer=FileUpload('http://ksql-server:8080')pointer.upload('rules.ksql')OptionsOptionTypeRequiredDescriptionksqlfilestringyesname of file containing the rulesResponsesIf ksql-commands succesfully executed:return (List of server response for all commands)If failed:raise the appropriate errorMore OptionsThere are more properties (partitions, replicas, etc…) in the official document.KSQL Syntax ReferenceResponsesIf create table/stream succeed:return TrueIf failed:raise a CreatError(respose_from_ksql_server)