Project: SeaTunnel (App Definition and Development / Streaming & Messaging)
File: Gitlab.md

Gitlab source connector Used to read data from Gitlab. | name | type | required | default value | |--||-|| | url | String | Yes | - | | access_token | String | Yes | - | | method | String | No | get | | schema.fields | Config | No | - | | format | String | No | json | | params | Map | No | - | | body | String | No | - | | json_field | Config | No | - | | content_json | String | No | - | | pollintervalmillis | int | No | - | | retry | int | No | - | | retrybackoffmultiplier_ms | int | No | 100 | | retrybackoffmax_ms | int | No | 10000 | | enablemultilines | boolean | No | false | | common-options | config | No | - | http request url personal access token http request method, only supports GET, POST method http params http body request http api interval(millis) in stream mode The max retry times if request http return to `IOException` The retry-backoff times(millis) multiplier if request http failed The maximum retry-backoff times(millis) if request http failed the format of upstream data, now only support `json` `text`, default `json`. when you assign format is `json`, you should also assign schema option, for example: upstream data is the following: ```json { "code": 200, "data": "get success", "success": true } ``` you should assign schema as the following: ```hocon schema { fields { code = int data = string success = boolean } } ``` connector will generate data as the following: | code | data | success | ||-|| | 200 | get success | true | when you assign format is `text`, connector will do nothing for upstream data, for example: upstream data is the following: ```json { "code": 200, "data": "get success", "success": true } ``` connector will generate data as the following: | content | |-| | {"code": 200, "data": "get success", "success": true} | the schema fields of upstream data This parameter can get some json

you only need the data in the 'book' section, configure `content_field = "$.store.book.*"`. If your return data looks something like this. ```json { "store": { "book": [ { "category": "reference", "author": "Nigel Rees", "title": "Sayings of the Century", "price": 8.95 }, { "category": "fiction", "author": "Evelyn Waugh", "title": "Sword of Honour", "price": 12.99 } ], "bicycle": { "color": "red", "price": 19.95 } }, "expensive": 10 } ``` You can configure `content_field = "$.store.book.*"` and the result returned looks like this: ```json [ { "category": "reference", "author": "Nigel Rees", "title": "Sayings of the Century", "price": 8.95 }, { "category": "fiction", "author": "Evelyn Waugh", "title": "Sword of Honour", "price": 12.99 } ] ``` Then you can get the desired result with a simpler schema,like ```hocon Http { url = "http://mockserver:1080/contentjson/mock" method = "GET" format = "json" content_field = "$.store.book.*" schema = { fields { category = string author = string title = string price = string } } } ``` Here is an example: Test data can be found at this link See this link for task configuration . This parameter helps you configure the schema,so this parameter must be used with schema. If your data looks something like this: ```json { "store": { "book": [ { "category": "reference", "author": "Nigel Rees", "title": "Sayings of the Century", "price": 8.95 }, { "category": "fiction", "author": "Evelyn Waugh", "title": "Sword of Honour", "price": 12.99 } ], "bicycle": { "color": "red", "price": 19.95 } }, "expensive": 10 } ``` You can get the contents of 'book' by configuring the task as follows: ```hocon source { Http { url = "http://mockserver:1080/jsonpath/mock" method = "GET" format = "json" json_field = { category = "$.store.book[*].category" author = "$.store.book[*].author" title = "$.store.book[*].title" price = "$.store.book[*].price" } schema = { fields { category = string author = string title = string price = string } } } } ``` Test data can be found at this link See this link for task configuration . Source plugin common parameters, please refer to for details ```hocon Gitlab{ url = "https://gitlab.com/api/v4/projects" access_token = "xxxxx" schema { fields { id = int description = string name = string namewithnamespace = string path = string httpurlto_repo = string } } } ``` Add Gitlab Source Connector )