Project: SeaTunnel (App Definition and Development / Streaming & Messaging)
File: Kudu.md

Kudu source connector 1.11.1/1.12.0/1.13.0/1.14.0/1.15.0 Spark<br/> Flink<br/> SeaTunnel Zeta<br/> Used to read data from Kudu. The tested kudu version is 1.11.1. | kudu Data Type | SeaTunnel Data Type | |--|| | BOOL | BOOLEAN | | INT8<br/>INT16<br/>INT32 | INT | | INT64 | BIGINT | | DECIMAL | DECIMAL | | FLOAT | FLOAT | | DOUBLE | DOUBLE | | STRING | STRING | | UNIXTIME_MICROS | TIMESTAMP | | BINARY | BYTES | | Name | Type | Required | Default | Description | |-|--|-||--| | kudu_masters | String | Yes | - | Kudu master address. Separated by ',',such as '192.168.88.110:7051'. | | table_name | String | Yes | - | The name of kudu table. | | clientworkercount | Int | No | 2 * Runtime.getRuntime().availableProcessors() | Kudu worker count. Default value is twice the current number of cpu cores. | | clientdefaultoperationtimeoutms | Long | No | 30000 | Kudu normal operation time out. | | clientdefaultadminoperationtimeout_ms | Long | No | 30000 | Kudu admin operation time out. | | enable_kerberos | Bool | No | false | Kerberos principal enable. | | kerberos_principal | String | No | - | Kerberos principal. Note that all zeta nodes require have this file. | | kerberos_keytab | String | No | - | Kerberos keytab. Note that all zeta nodes require have this file. | | kerberos_krb5conf | String | No | - | Kerberos krb5 conf. Note that all zeta nodes require have this file. | | scantokenquery_timeout | Long | No | 30000 | The timeout for connecting scan token. If not set, it will be the same as operationTimeout. | | scantokenbatchsizebytes | Int | No | 1024 * 1024 | Kudu scan bytes. The maximum number of bytes read at a time, the default is 1MB. | | filter | Int | No | 1024 * 1024 | Kudu scan filter expressions,Not supported yet. | | schema | Map | No | 1024 * 1024 | SeaTunnel Schema. | | tablelist | Array | No | - | The list of tables to be read. you can use this configuration instead of `tablepath` example: ```tablelist = [{ tablename = "kudusourcetable1"},{ tablename = "kudusourcetable_2"}] ``` | | common-options | | No | - | Source plugin common parameters, please refer to for details. | The following example is for a Kudu table named "kudusourcetable", The goal is to print the data from this table on the console and write kudu table "kudusinktable" ```hocon env { parallelism = 2 job.mode = "BATCH" } source { kudu { kudu_masters = "kudu-master:7051" tablename = "kudusource_table" resulttablename = "kudu" enable_kerberos = true kerberos_principal = "xx@xx.COM" kerberos_keytab = "xx.keytab" } } transform { } sink { console { sourcetablename = "kudu" } kudu { sourcetablename = "kudu" kudu_masters = "kudu-master:7051" tablename = "kudusink_table" enable_kerberos = true kerberos_principal = "xx@xx.COM" kerberos_keytab = "xx.keytab" } } ``` ```hocon env { parallelism = 1 job.mode = "STREAMING" checkpoint.interval = 5000 } source { kudu{ kudu_masters = "kudu-master:7051" table_list = [ { tablename = "kudusourcetable1" },{ tablename = "kudusourcetable2" } ] resulttablename = "kudu" } } transform { } sink { Assert { rules { table-names = ["kudusourcetable1", "kudusourcetable2"] } } } ``` Add Kudu Source Connector Change plugin name from `KuduSource` to `Kudu`