Project: SeaTunnel (App Definition and Development / Streaming & Messaging)
File: Github.md

Github source connector Used to read data from Github. | name | type | required | default value | |--||-|| | url | String | Yes | - | | access_token | String | No | - | | method | String | No | get | | schema.fields | Config | No | - | | format | String | No | json | | params | Map | No | - | | body | String | No | - | | json_field | Config | No | - | | content_json | String | No | - | | pollintervalmillis | int | No | - | | retry | int | No | - | | retrybackoffmultiplier_ms | int | No | 100 | | retrybackoffmax_ms | int | No | 10000 | | enablemultilines | boolean | No | false | | common-options | config | No | - | http request url Github personal access token, see: http request method, only supports GET, POST method http params http body request http api interval(millis) in stream mode The max retry times if request http return to `IOException` The retry-backoff times(millis) multiplier if request http failed The maximum retry-backoff times(millis) if request http failed the format of upstream data, now only support `json` `text`, default `json`. when you assign format is `json`, you should also assign schema option, for example: upstream data is the following: ```json { "code": 200, "data": "get success", "success": true } ``` you should assign schema as the following: ```hocon schema { fields { code = int data = string success = boolean } } ``` connector will generate data as the following: | code | data | success | ||-|| | 200 | get success | true | when you assign format is `text`, connector will do nothing for upstream data, for example: upstream data is the following: ```json { "code": 200, "data": "get success", "success": true } ``` connector will generate data as the following: | content | |-| | {"code": 200, "data": "get success", "success": true} | the schema fields of upstream data This parameter can get some json

you only need the data in the 'book' section, configure `content_field = "$.store.book.*"`. If your return data looks something like this. ```json { "store": { "book": [ { "category": "reference", "author": "Nigel Rees", "title": "Sayings of the Century", "price": 8.95 }, { "category": "fiction", "author": "Evelyn Waugh", "title": "Sword of Honour", "price": 12.99 } ], "bicycle": { "color": "red", "price": 19.95 } }, "expensive": 10 } ``` You can configure `content_field = "$.store.book.*"` and the result returned looks like this: ```json [ { "category": "reference", "author": "Nigel Rees", "title": "Sayings of the Century", "price": 8.95 }, { "category": "fiction", "author": "Evelyn Waugh", "title": "Sword of Honour", "price": 12.99 } ] ``` Then you can get the desired result with a simpler schema,like ```hocon Http { url = "http://mockserver:1080/contentjson/mock" method = "GET" format = "json" content_field = "$.store.book.*" schema = { fields { category = string author = string title = string price = string } } } ``` Here is an example: Test data can be found at this link See this link for task configuration . This parameter helps you configure the schema,so this parameter must be used with schema. If your data looks something like this: ```json { "store": { "book": [ { "category": "reference", "author": "Nigel Rees", "title": "Sayings of the Century", "price": 8.95 }, { "category": "fiction", "author": "Evelyn Waugh", "title": "Sword of Honour", "price": 12.99 } ], "bicycle": { "color": "red", "price": 19.95 } }, "expensive": 10 } ``` You can get the contents of 'book' by configuring the task as follows: ```hocon source { Http { url = "http://mockserver:1080/jsonpath/mock" method = "GET" format = "json" json_field = { category = "$.store.book[*].category" author = "$.store.book[*].author" title = "$.store.book[*].title" price = "$.store.book[*].price" } schema = { fields { category = string author = string title = string price = string } } } } ``` Test data can be found at this link See this link for task configuration . Source plugin common parameters, please refer to for details ```hocon Github { url = "https://api.github.com/orgs/apache/repos" access_token = "xxxx" method = "GET" format = "json" schema = { fields { id = int name = string description = string html_url = string stargazers_count = int forks = int } } } ```