Package: kafka-connect-healthcheck
kafka-connect-healthcheckA simple healthcheck wrapper to monitor Kafka Connect.Kafka Connect Healthcheck is a server that wraps the Kafka Connect API and provides a singular API endpoint to determine the health of a Kafka Connect instance. This can be used to alert or take action on unhealthy connectors and tasks.This can be used in numerous ways. It can sit as a standalone service for monitoring purposes, it can be used as a sidecar container to mark Kafka Connect workers as unhealthy in Kubernetes, or it can be used to provide logs of when connectors/tasks failed and reasons for their failures.By default, the root endpoint/will return200 OKhealthy if all connectors and tasks are in a state other thanFAILED. It will return503 Service Unavailableif any connector or tasks are in aFAILEDstate.UsageKafka Connect Healthcheck can be installed as a command-line tool throughpipor it can be used as a standalone Docker container. It could also be installed as a part of a custom Kafka Connect docker image.Command-LineTo usekafka-connect-healthcheckfrom the command-line, you must havepythonandpipinstalled. Currently, only Python 3 is supported.You can installkafka-connect-healthcheckvia pip:pipinstallkafka-connect-healthcheckTo start the healthcheck server, run:kafka-connect-healthcheckThe server will now be running onlocalhost:18083.DockerThekafka-connect-healthcheckimage can be found on Docker Hub.You can pull down the latest image by running:dockerpulldevshawn/kafka-connect-healthcheckTo start the healthcheck server, run:dockerrun--rm-it-p18083:18083devshawn/kafka-connect-healthcheckThe server will now be running onlocalhost:18083.ConfigurationKafka Connect Healthcheck can be configured via command-line arguments or by environment variables.PortThe port for thekafka-connect-healthcheckAPI.UsageValueEnvironment VariableHEALTHCHECK_PORTCommand-Line Argument--portDefault Value18083Connect URLThe full URL of the Kafka Connect REST API. This is used to determine the health of the connect instance.UsageValueEnvironment VariableHEALTHCHECK_CONNECT_URLCommand-Line Argument--connect-urlDefault Valuehttp://localhost:8083Connect Worker IDThe worker ID to monitor (usually the IP address of the connect worker). If none is set, all workers will be monitored and any failure will result in an unhealthy response.UsageValueEnvironment VariableHEALTHCHECK_CONNECT_WORKER_IDCommand-Line Argument--connect-worker-idDefault ValueNone (all workers monitored)Note: It is highly recommended to run an instance of the healthcheck for each worker if you're planning to restart containers based on the health.Considered ContainersA comma-separated list of which type of kafka connect container to be considered in the healthcheck calculation.UsageValueEnvironment VariableHEALTHCHECK_CONSIDERED_CONTAINERSCommand-Line Argument--considered-containersDefault ValueCONNECTOR,TASKValid ValuesCONNECTOR,TASKUnhealthy StatesA comma-separated list of connector and tasks states to be marked as unhealthy.UsageValueEnvironment VariableHEALTHCHECK_UNHEALTHY_STATESCommand-Line Argument--unhealthy-statesDefault ValueFAILEDValid ValuesFAILED,PAUSED,UNASSIGNED,RUNNINGNote: It's recommended to keep this defaulted toFAILED, but paused connectors or tasks can be marked as unhealthy by passingFAILED,PAUSED.Failure Threshold PercentageA number between 1 and 100. If set, this is the percentage of connectors that must fail for the healthcheck to fail.UsageValueEnvironment VariableHEALTHCHECK_FAILURE_THRESHOLD_PERCENTAGECommand-Line Argument--failure-threshold-percentageDefault Value0Valid Values1 to 100By default,anyfailures will cause the healthcheck to fail.Log LevelThe level of logs to be shown by the application.UsageValueEnvironment VariableHEALTHCHECK_LOG_LEVELCommand-Line Argument--log-levelDefault ValueINFOValid ValuesDEBUG,INFO,WARNING,ERRORAll healthy connector and task statuses are logged atINFO. Unhealthy ones are logged atWARNING. Any communication or HTTP errors are logged atERROR.APIThe server provides a very simple HTTP API which can be used for liveness probes and monitoring alerts. We expose two endpoints:GET /Get the current health status of the Kafka Connect system. This could be used as a sidecar to determine the health of each Kafka Connect worker and their associated connectors and tasks.Example Requestcurlhttp://localhost:18083Example Healthy Response200 OK{"failures":[],"failure_states":["FAILED"],"healthy":true}Example Unhealthy Response503 Service Unavailable{"failures":[{"type":"connector","connector":"jdbc-source","state":"FAILED","worker_id":"127.0.0.1:8083"},{"type":"task","connector":"jdbc-source","id":0,"state":"FAILED","worker_id":"127.0.0.1:8083","trace":"..."}],"failure_states":["FAILED"],"healthy":false}GET /pingGet the current health status of the healthcheck server. This will always be successful as long as the server is still able to serve requests. This can be used as a ready or liveness probe in Kubernetes.Example Requestcurlhttp://localhost:18083/pingExample Response200 OK{"status":"UP"}LicenseCopyright (c) 2019 Shawn Seymour.Licensed under theApache 2.0 license.