Project: SeaTunnel (App Definition and Development / Streaming & Messaging)
File: Redis.md

Redis source connector Used to read data from Redis. | name | type | required | default value | ||--|--|| | host | string | yes | - | | port | int | yes | - | | keys | string | yes | - | | data_type | string | yes | - | | user | string | no | - | | auth | string | no | - | | db_num | int | no | 0 | | mode | string | no | single | | hashkeyparse_mode | string | no | all | | nodes | list | yes when mode=cluster | - | | schema | config | yes when format=json | - | | format | string | no | json | | common-options | | no | - | redis host redis port hash key parse mode, support `all` `kv`, used to tell connector how to parse hash key. when setting it to `all`, connector will treat the value of hash key as a row and use the schema config to parse it, when setting it to `kv`, connector will treat each kv in hash key as a row and use the schema config to parse it: for example, if the value of hash key is the following shown: ```text { "001": { "name": "tyrantlucifer", "age": 26 }, "002": { "name": "Zongwen", "age": 26 } } ``` if hashkeyparse_mode is `all` and schema config as the following shown, it will generate the following data: ```hocon schema { fields { 001 { name = string age = int } 002 { name = string age = int } } } ``` | 001 | 002 | ||| | Row(name=tyrantlucifer, age=26) | Row(name=Zongwen, age=26) | if hashkeyparse_mode is `kv` and schema config as the following shown, it will generate the following data: ```hocon schema { fields { hash_key = string name = string age = int } } ``` | hash_key | name | age | |-||--| | 001 | tyrantlucifer | 26 | | 002 | Zongwen | 26 | each kv that in hash key it will be treated as a row and send it to upstream. Tips: connector will use the first field information of schema config as the field name of each k that in each kv keys pattern Tips:Redis source connector support fuzzy key matching, user needs to ensure that the matched keys are the same type redis data types, support `key` `hash` `list` `set` `zset` key The value of each key will be sent downstream as a single row of

For example, the value of key is `SeaTunnel test message`, the data received downstream is `SeaTunnel test message` and only one message will be received. hash The hash key-value pairs will be formatted as json to be sent downstream as a single row of data. For example, the value of hash is `name:tyrantlucifer age:26`, the data received downstream is `{"name":"tyrantlucifer", "age":"26"}` and only one message will be received. list Each element in the list will be sent downstream as a single row of data. For example, the value of list is `[tyrantlucier, CalvinKirs]`, the data received downstream are `tyrantlucifer` and `CalvinKirs` and only two message will be received. set Each element in the set will be sent downstream as a single row of data For example, the value of set is `[tyrantlucier, CalvinKirs]`, the data received downstream are `tyrantlucifer` and `CalvinKirs` and only two message will be received. zset Each element in the sorted set will be sent downstream as a single row of data For example, the value of sorted set is `[tyrantlucier, CalvinKirs]`, the data received downstream are `tyrantlucifer` and `CalvinKirs` and only two message will be received. redis authentication user, you need it when you connect to an encrypted cluster redis authentication password, you need it when you connect to an encrypted cluster Redis database index ID. It is connected to db 0 by default redis mode, `single` or `cluster`, default is `single` redis nodes information, used in cluster mode, must like as the following format: ["host1:port1", "host2:port2"] the format of upstream data, now only support `json` `text`, default `json`. when you assign format is `json`, you should also assign schema option, for example: upstream data is the following: ```json {"code": 200, "data": "get success", "success": true} ``` you should assign schema as the following: ```hocon schema { fields { code = int data = string success = boolean } } ``` connector will generate data as the following: | code | data | success | ||-|| | 200 | get success | true | when you assign format is `text`, connector will do nothing for upstream data, for example: upstream data is the following: ```json {"code": 200, "data": "get success", "success": true} ``` connector will generate data as the following: | content | |-| | {"code": 200, "data": "get success", "success": true} | the schema fields of redis data Source plugin common parameters, please refer to for details simple: ```hocon Redis { host = localhost port = 6379 keys = "key_test*" data_type = key format = text } ``` ```hocon Redis { host = localhost port = 6379 keys = "key_test*" data_type = key format = json schema { fields { name = string age = int } } } ``` Add Redis Source Connector