Project: SeaTunnel (App Definition and Development / Streaming & Messaging)
File: Elasticsearch.md

Elasticsearch source connector Used to read data from Elasticsearch. support version >= 2.x and <= 8.x. | name | type | required | default value | |-||-|-| | hosts | array | yes | - | | username | string | no | - | | password | string | no | - | | index | string | yes | - | | source | array | no | - | | query | json | no | {"match_all": {}} | | scroll_time | string | no | 1m | | scroll_size | int | no | 100 | | tlsverifycertificate | boolean | no | true | | tlsverifyhostnames | boolean | no | true | | array_column | map | no | | | tlskeystorepath | string | no | - | | tlskeystorepassword | string | no | - | | tlstruststorepath | string | no | - | | tlstruststorepassword | string | no | - | | common-options | | no | - | Elasticsearch cluster http address, the format is `host:port`, allowing multiple hosts to be specified. Such as `["host1:9200", "host2:9200"]`. x-pack username. x-pack password. Elasticsearch index name, support * fuzzy matching. The fields of index. You can get the document id by specifying the field `id`.If sink id to other index,you need specify an alias for _id due to the Elasticsearch limit. If you don't config source, it is automatically retrieved from the mapping of the index. The fields of array type. Since there is no array index in es,so need assign array type,just like `{c_array = "array<tinyint>"}`. Elasticsearch DSL. You can control the range of data read. Amount of time Elasticsearch will keep the search context alive for scroll requests. Maximum number of hits to be returned with each Elasticsearch scroll request. Enable certificates validation for HTTPS endpoints Enable hostname validation for HTTPS endpoints The path to the PEM or JKS key store. This file must be readable by the operating system user running SeaTunnel. The key password for the key store specified The path to PEM or JKS trust store. This file must be readable by the operating system user running SeaTunnel. The key password for the trust store specified Source plugin common parameters, please refer to for details simple ```hocon Elasticsearch { hosts = ["localhost:9200"] index = "seatunnel-*" source = ["_id","name","age"] query = {"range":{"firstPacket":{"gte":1669225429990,"lte":1669225429990}}} } ``` complex ```hocon Elasticsearch { hosts = ["elasticsearch:9200"] index = "st_index" schema = { fields { c_map = "map<string, tinyint>" c_array = "array<tinyint>" c_string = string c_boolean = boolean c_tinyint = tinyint c_smallint = smallint c_int = int c_bigint = bigint c_float = float c_double = double c_decimal = "decimal(2, 1)" c_bytes = bytes c_date = date c_timestamp = timestamp } } query = {"range":{"firstPacket":{"gte":1669225429990,"lte":1669225429990}}} } ``` SSL (Disable certificates validation) ```hocon source { Elasticsearch { hosts = ["https://localhost:9200"] username = "elastic" password = "elasticsearch" tlsverifycertificate = false } } ``` SSL (Disable hostname validation) ```hocon source { Elasticsearch { hosts = ["https://localhost:9200"] username = "elastic" password = "elasticsearch" tlsverifyhostname = false } } ``` SSL (Enable certificates validation) ```hocon source { Elasticsearch { hosts = ["https://localhost:9200"] username = "elastic" password = "elasticsearch" tlskeystorepath = "${your elasticsearch home}/config/certs/http.p12" tlskeystorepassword = "${your password}" } } ``` Add Elasticsearch Source Connector ) [Feature] Support DSL