Project: SeaTunnel (App Definition and Development / Streaming & Messaging)
File: OceanBase.md

JDBC OceanBase Source Connector Spark<br/> Flink<br/> SeaTunnel Zeta<br/> Read external data source data through JDBC. | Datasource | Supported versions | Driver | Url | Maven | ||--||--|-| | OceanBase | All OceanBase server versions. | com.oceanbase.jdbc.Driver | jdbc:oceanbase://localhost:2883/test | | Please download the support list corresponding to 'Maven' and copy it to the '$SEATNUNNEL_HOME/plugins/jdbc/lib/' working directory<br/> For example: cp oceanbase-client-xxx.jar $SEATNUNNEL_HOME/plugins/jdbc/lib/ | Mysql Data type | SeaTunnel Data type | |--|--| | BIT(1)<br/>TINYINT(1) | BOOLEAN | | TINYINT | BYTE | | TINYINT<br/>TINYINT UNSIGNED | SMALLINT | | SMALLINT UNSIGNED<br/>MEDIUMINT<br/>MEDIUMINT UNSIGNED<br/>INT<br/>INTEGER<br/>YEAR | INT | | INT UNSIGNED<br/>INTEGER UNSIGNED<br/>BIGINT | BIGINT | | BIGINT UNSIGNED | DECIMAL(20,0) | | DECIMAL(x,y)(Get the designated column's specified column size.<38) | DECIMAL(x,y) | | DECIMAL(x,y)(Get the designated column's specified column size.>38) | DECIMAL(38,18) | | DECIMAL UNSIGNED | DECIMAL((Get the designated column's specified column size)+1,<br/>(Gets the designated column's number of digits to right of the decimal point.))) | | FLOAT<br/>FLOAT UNSIGNED | FLOAT | | DOUBLE<br/>DOUBLE UNSIGNED | DOUBLE | | CHAR<br/>VARCHAR<br/>TINYTEXT<br/>MEDIUMTEXT<br/>TEXT<br/>LONGTEXT<br/>JSON<br/>ENUM | STRING | | DATE | DATE | | TIME | TIME | | DATETIME<br/>TIMESTAMP | TIMESTAMP | | TINYBLOB<br/>MEDIUMBLOB<br/>BLOB<br/>LONGBLOB<br/>BINARY<br/>VARBINAR<br/>BIT(n)<br/>GEOMETRY | BYTES | | Oracle Data type | SeaTunnel Data type | |--|| | Integer | DECIMAL(38,0) | | Number(p), p <= 9 | INT | | Number(p), p <= 18 | BIGINT | | Number(p), p > 18 | DECIMAL(38,18) | | Number(p,s) | DECIMAL(p,s) | | Float | DECIMAL(38,18) | | REAL<br/> BINARY_FLOAT | FLOAT | | BINARY_DOUBLE | DOUBLE | | CHAR<br/>NCHAR<br/>VARCHAR<br/>VARCHAR2<br/>NVARCHAR2<br/>NCLOB<br/>CLOB<br/>LONG<br/>XML<br/>ROWID | STRING | | DATE | TIMESTAMP | | TIMESTAMP<br/>TIMESTAMP WITH LOCAL TIME ZONE | TIMESTAMP | | BLOB<br/>RAW<br/>LONG RAW<br/>BFILE | BYTES | | UNKNOWN | Not supported yet | | Name | Type | Required | Default | Description | |||-|--|--| | url | String | Yes | - | The URL of the JDBC connection. Refer to a case: jdbc:oceanbase://localhost:2883/test | | driver | String | Yes | - | The jdbc class name used to connect to the remote data source, should be `com.oceanbase.jdbc.Driver`. | | user | String | No | - | Connection instance user name | | password | String | No | - | Connection instance password | | compatible_mode | String | Yes | - | The compatible mode of OceanBase, can be 'mysql' or

| | query | String | Yes | - | Query statement | | connectionchecktimeout_sec | Int | No | 30 | The time in seconds to wait for the database operation used to validate the connection to complete | | partition_column | String | No | - | The column name for parallelism's partition, only support numeric type column and string type column. | | partitionlowerbound | BigDecimal | No | - | The partition_column min value for scan, if not set SeaTunnel will query database get min value. | | partitionupperbound | BigDecimal | No | - | The partition_column max value for scan, if not set SeaTunnel will query database get max value. | | partition_num | Int | No | job parallelism | The number of partition count, only support positive integer. Default value is job parallelism. | | fetch_size | Int | No | 0 | For queries that return a large number of objects, you can configure <br/> the row fetch size used in the query to improve performance by <br/> reducing the number database hits required to satisfy the selection criteria.<br/> Zero means use jdbc default value. | | properties | Map | No | - | Additional connection configuration parameters,when properties and URL have the same parameters, the priority is determined by the <br/>specific implementation of the driver. For example, in MySQL, properties take precedence over the URL. | | common-options | | No | - | Source plugin common parameters, please refer to for details | If partitioncolumn is not set, it will run in single concurrency, and if partitioncolumn is set, it will be executed in parallel according to the concurrency of tasks. ``` env { parallelism = 2 job.mode = "BATCH" } source { Jdbc { driver = "com.oceanbase.jdbc.Driver" url = "jdbc:oceanbase://localhost:2883/test?useUnicode=true&characterEncoding=UTF-8&rewriteBatchedStatements=true" user = "root" password = "" compatible_mode = "mysql" query = "select * from source" } } transform { } sink { Console {} } ``` Read your query table in parallel with the shard field you configured and the shard data. You can do this if you want to read the whole table ``` env { parallelism = 10 job.mode = "BATCH" } source { Jdbc { driver = "com.oceanbase.jdbc.Driver" url = "jdbc:oceanbase://localhost:2883/test?useUnicode=true&characterEncoding=UTF-8&rewriteBatchedStatements=true" user = "root" password = "" compatible_mode = "mysql" query = "select * from source" partition_column = "id" partition_num = 10 } } sink { Console {} } ``` It is more efficient to read your data source according to the upper and lower boundaries you configured ``` source { Jdbc { driver = "com.oceanbase.jdbc.Driver" url = "jdbc:oceanbase://localhost:2883/test?useUnicode=true&characterEncoding=UTF-8&rewriteBatchedStatements=true" user = "root" password = "" compatible_mode = "mysql" query = "select * from source" partition_column = "id" partition_num = 10 partitionlowerbound = 1 partitionupperbound = 500 } } ```