Package: healthchecker
HealthCheckerA very simple python script to check thehealthof a service: make a GET request to it and if it answers it on time, then it's alive.Additionally, it can write the result of the operation in a git repo hosted on Github or in a local file. Or notify dead services through a POST to some JSON API.Actions are logged using Python's standard logger and displayed according to the log level set.This app was thought mainly for ourstatus page. It's currently being maintained in ourrepository.RequirementsPython 3.7+PyGithub 1.43+requests 2.21+Usageusage: healthchecker [-h] [--gh-repo GH_REPO] [--gh-filename GH_FILENAME]
                     [--gh-branch GH_BRANCH] [--gh-token GH_TOKEN]
                     [--gh-email GH_EMAIL] [--notify-url NOTIFY_URL]
                     [--notify-payload NOTIFY_PAYLOAD]
                     [--notify-header NOTIFY_HEADER] [--notify-json]
                     [--version] [--validation VALIDATION] [--output OUTPUT]
                     [url [url ...]]

HealthChecker v1.1.0 by HacKan (https://hackan.net) FOSS under GNU GPL v3.0 or
newer. Checks URLs through HTTP GET requests to verify their availability.
Optionally writes the status result to a file in Github. Using Github requires
the repository name, the filename and an API token. Besides the ones listed
below, the following env vars exist: HEALTHCHECKER_LOG_LEVEL sets the minimal
logging level and defaults to info (can be: debug, info, warning, error,
critical); HEALTHCHECKER_REQUESTS_TIMEOUT sets the amount of time in seconds
to wait for services to respond and defaults to 10 seconds (setting a very low
value might cause several false positives). Note: command-line parameters will
always supersede env vars.

positional arguments:
  url                   (HEALTHCHECKER_URLS (comma-separated)) URL to check

optional arguments:
  -h, --help            show this help message and exit
  --version             show version and exit
  --validation VALIDATION
                        (HEALTHCHECKER_URLS_VALIDATION (comma-separated))
                        string to find in the body of a request to an URL as a
                        validation, one per URL (or the last one is used for
                        the remaining URLs) (this parameter can be repeated as
                        needed)
  --output OUTPUT, -o OUTPUT
                        (HEALTHCHECKER_OUTPUT) store result in a file,
                        overwriting if exists (use `-` for standard output)

github options:
  --gh-repo GH_REPO     (HEALTHCHECKER_GITHUB_REPO) repository in the form of
                        <user|org>/<repo> (case insensitive), i.e.:
                        HacKanCuBa/b2rsum
  --gh-filename GH_FILENAME
                        (HEALTHCHECKER_GITHUB_FILENAME) filename to modify
                        (include path if it is in a subdir such as
                        path/to/file.ext)
  --gh-branch GH_BRANCH
                        (HEALTHCHECKER_GITHUB_BRANCH) branch where commits are
                        done (defaults to main)
  --gh-token GH_TOKEN   (HEALTHCHECKER_GITHUB_API_TOKEN) API token or
                        client_id,client_secret (bypasses the one supplied
                        through the environment)
  --gh-email GH_EMAIL   (HEALTHCHECKER_GITHUB_COMMITTER_EMAIL) git committer
                        email (the committer name is hardcoded to
                        HealthChecker)

notify options:
  --notify-url NOTIFY_URL
                        (HEALTHCHECKER_NOTIFY_URL) URL to POST the status
                        notification
  --notify-payload NOTIFY_PAYLOAD
                        (HEALTHCHECKER_NOTIFY_PAYLOAD) payload to send to the
                        notify URL: it is prepended to the comma-separated
                        list of URLs that failed validation, unless that it
                        contains the string HEALTHCHECKER_FAILED_URLS (case
                        sensitive), where it will replace that string by the
                        comma-separated list of URLs, and send the entire
                        payload
  --notify-header NOTIFY_HEADER
                        (HEALTHCHECKER_NOTIFY_HEADERS (comma-separated))
                        header to send to the notify URL, which must be
                        specified as name and value separated by a semicolon:
                        <header name>:<header value> (this parameter can be
                        repeated as needed)
  --notify-json         (HEALTHCHECKER_NOTIFY_JSON (true/false)) send the
                        payload JSON encoded (it also adds the proper Content-
                        Type header)Environment variablesHEALTHCHECKER_GITHUB_REPO: repository in the form of <user|org>/ (case insensitive), i.e.: HacKanCuBa/b2rsum.HEALTHCHECKER_GITHUB_FILENAME: filename to modify (include path if it is in a subdir such as path/to/file.ext).HEALTHCHECKER_GITHUB_BRANCH: branch where commits are done (defaults to main).HEALTHCHECKER_GITHUB_API_TOKEN: API token or client_id,client_secret (bypasses the one supplied through the environment).HEALTHCHECKER_GITHUB_COMMITTER_EMAIL: git committer email (the committer name is hardcoded to HealthChecker).HEALTHCHECKER_URLS: URLs to check, comma-separated.HEALTHCHECKER_LOG_LEVEL: minimal logging level, defaults to info (can be: debug, info, warning, error, critical).HEALTHCHECKER_REQUESTS_TIMEOUT: amount of time in decimal seconds to wait for services to respond and defaults to 10 seconds (setting a very low value might cause several false positives).HEALTHCHECKER_NOTIFY_URL: URL to send failed checks via POST as notification, comma-separated.HEALTHCHECKER_URLS_VALIDATION: comma-separated list of validations to run on given URLs.HEALTHCHECKER_NOTIFY_PAYLOAD: payload to send to the notify URL: it is prepended to the comma-separated list of URLs that failed validation, unless that it contains the stringHEALTHCHECKER_FAILED_URLS(case sensitive), where it will replace that string by the comma-separated list of URLs, and send the entire payload.Example 1:HEALTHCHECKER_NOTIFY_PAYLOAD=here comes the failed urls...Example 2:HEALTHCHECKER_NOTIFY_PAYLOAD={"data": "HEALTHCHECKER_FAILED_URLS"}HEALTHCHECKER_NOTIFY_HEADERS: headers to send to the notify URL, which must be specified as name and value separated by a semicolon:header name:header value, and successive headers separated by comma.Example 1:HEALTHCHECKER_NOTIFY_HEADERS=X-Auth:4c18a291d7d8e7946cb9db9cbb3e1f49Example 2:HEALTHCHECKER_NOTIFY_HEADERS=Content-Type:application/json,X-MyVal:1HEALTHCHECKER_NOTIFY_JSON: (true/false) send the payload JSON encoded (it also adds the proper Content-Type header).HEALTHCHECKER_OUTPUT: store result in a file (use-for standard output).ResponsesHealthChecker reports two values for each service checked:alive: means that the service has responded (no timeout and status code lower than 500).ok: means that the service has responded and has a status code of 2xx; additionally, if checks are given for the service this means that checks passed.The app will follow any 3xx response accordingly.ExamplesSimply print checks result::~$ healthchecker https://rlab.be adm.rlab.be
INFO 2019-04-09 01:06:37 Begin checking URL https://rlab.be...
INFO 2019-04-09 01:06:37 Begin checking URL http://adm.rlab.be...
INFO 2019-04-09 01:06:37 Finish checking URL https://rlab.be: alive and OK
INFO 2019-04-09 01:06:37 Finish checking URL http://adm.rlab.be: alive and OKOr show output throughstdout, hiding logs::~$ HEALTHCHECKER_LOG_LEVEL=critical healthchecker -o - https://rlab.be adm.rlab.be 2> /dev/null
[
  {
    "uid": "416ff55a",
    "url": "https://rlab.be",
    "alive": true,
    "ok": true
  },
  {
    "uid": "5d961608",
    "url": "http://adm.rlab.be",
    "alive": true,
    "ok": true
  }
]That output can be parsed byjqor any other JSON tool! Additionally, it can be saved to a file for later reference::~$ healthchecker -o checks.json https://rlab.be adm.rlab.be
INFO 2019-04-09 01:06:37 Begin checking URL https://rlab.be...
INFO 2019-04-09 01:06:37 Begin checking URL http://adm.rlab.be...
INFO 2019-04-09 01:06:37 Finish checking URL https://rlab.be: alive and OK
INFO 2019-04-09 01:06:37 Finish checking URL http://adm.rlab.be: alive and OK
INFO 2019-04-09 01:06:37 Result stored as "/home/hackan/Workspace/healthchecker/checks.json"

:~$ cat checks.json
[
  {
    "uid": "416ff55a",
    "url": "https://rlab.be",
    "alive": true,
    "ok": true
  },
  {
    "uid": "5d961608",
    "url": "http://adm.rlab.be",
    "alive": true,
    "ok": true
  }
]Notify failed services to an endpoint:healthchecker --notify-url https://eoc.rlab.be/api/v1/status/ https://rlab.be http://wiki.rlab.beWrite to a file in Github and be very verbose::~$ HEALTHCHECKER_LOG_LEVEL=debug healthchecker --gh-repo rlyehlab/sysadmins --gh-filename data/healthcheck.json --gh-token ab410...2cc https://git.rlab.be
INFO 2019-04-09 01:07:32 Begin checking URL https://git.rlab.be...
ERROR 2019-04-09 01:07:32 Error GETing data from/to https://git.rlab.be: ConnectionError(MaxRetryError("HTTPConnectionPool(host='git.rlab.be', port=80): Max retries exceeded with url: / (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x6ac59ffbd001>: Failed to establish a new connection: [Errno -2] Name or service not known'))"))
DEBUG 2019-04-09 01:07:32 Request to https://git.rlab.be took 0.18 seconds
INFO 2019-04-09 01:07:32 Finish checking URL http://git.rlab.be: dead
INFO 2019-04-09 01:07:33 Getting repository information...
INFO 2019-04-09 01:07:38 File data/healthcheck.json updated: 0335a8088f5aff42f078a9396916c8adbcc1a6c3Parameters can be passed through env vars and/or through command-line indistinctly (command-line parameters will always supersede env vars)::~$ HEALTHCHECKER_URLS_VALIDATION="Services | Administration,Adventurous writings by R'lyeh Sysadmins" healthchecker --notify-url 127.0.0.1:8000 https://adm.rlab.be https://blog.adm.rlab.be
INFO 2019-04-09 00:59:44 Begin checking URL https://adm.rlab.be...
INFO 2019-04-09 00:59:44 Begin checking URL https://blog.adm.rlab.be...
INFO 2019-04-09 00:59:44 Finish checking URL https://blog.adm.rlab.be: alive and OK
INFO 2019-04-09 00:59:44 Finish checking URL https://adm.rlab.be: alive and OKAll checks were OK, but should one fail...:HEALTHCHECKER_LOG_LEVEL=debug HEALTHCHECKER_URLS_VALIDATION="Services | Administration,Adventurous writings by R'lyeh Sysadmins" python -m healthchecker --validation "non-existent string" --notify-url 127.0.0.1:8000 https://adm.rlab.be https://blog.adm.rlab.be
INFO 2019-04-09 01:03:39 Begin checking URL https://adm.rlab.be...
INFO 2019-04-09 01:03:39 Begin checking URL https://blog.adm.rlab.be...
DEBUG 2019-04-09 01:03:40 Request to https://blog.adm.rlab.be took 0.18 seconds
INFO 2019-04-09 01:03:40 Finish checking URL https://blog.adm.rlab.be: alive but not OK
DEBUG 2019-04-09 01:03:40 Request to https://adm.rlab.be took 0.30 seconds
INFO 2019-04-09 01:03:40 Finish checking URL https://adm.rlab.be: alive but not OK
DEBUG 2019-04-09 01:03:40 Notifying http://127.0.0.1:8000 with headers: {} and payload: https://adm.rlab.be,https://blog.adm.rlab.be
ERROR 2019-04-09 01:03:40 Error POSTing data from/to http://127.0.0.1:8000: ConnectionError(MaxRetryError("HTTPConnectionPool(host='127.0.0.1', port=8000): Max retries exceeded with url: / (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f49d9767f28>: Failed to establish a new connection: [Errno 111] Connection refused'))"))
DEBUG 2019-04-09 01:03:40 Request to http://127.0.0.1:8000 took 0.00 seconds
ERROR 2019-04-09 01:03:40 Could not notify http://127.0.0.1:8000Note that nobody was listening at127.0.0.1:8000so the notification failed as well.RunCan be run as a python modulepython -m healthchecker ...or directly as a CLI toolhealthchecker .... Read below on how to run thedockerimage.PyPiHealthChecker is in PyPi and can be installed with any standard tool such aspip(pip install healthchecker) orpoetry(poetry add healthchecker). To add it as a dependency in your project it's recommended to use thehash parameter for pip. Hashes (and more) are listed in everyrelease.RepoClonethe repoand install requirements:installing requirements withpoetry(recommended):poetry install. Then start virtualenv withpoetry shell.DockerBuildBuild locally withdocker build --compress --pull --rm --tag registry.rlab.be/sysadmins/healthchecker:latest .orinvoke buildPullYou can pull from our registry withdocker pull registry.rlab.be/sysadmins/healthchecker:latestRunRun withdocker run --rm registry.rlab.be/sysadmins/healthchecker:latest ...Alternatively, use env vars by creating an env file and passing it to docker:cp sample.env .env
vim .env  # edit and populate vars
docker run --rm --env-file .env registry.rlab.be/sysadmins/healthchecker:latestDeployThis can be deployed in a server creating a SystemD service and timer (the image will be pulled by Docker on first run):ServiceCreate an env file where ever you want, as in/srv/healthchecker/.env(protect access to it with linux permissions). It can be anywhere with any name, just point it in the parameter--env-filein the service file. Then create the service file/etc/systemd/system/healthchecker.service:[Unit]
Description=HealthChecker Service
Requires=docker.service
After=network.target docker.service

[Service]
Type=simple
ExecStart=/usr/bin/docker run --rm --env-file /srv/healthchecker/.env registry.rlab.be/sysadmins/healthchecker:latest
User=root
Group=dockerAlternatively, you can skip env file usage and write every parameter in theExecStartline, but writing the API token there means it will be visible in the process list which is usually not a good idea.Timer:Create the timer file/etc/systemd/system/healthchecker.timer(use the same name as the service but with the.timerextension):[Unit]
Description=HealthChecker Service Timer

[Timer]
OnBootSec=600
OnUnitActiveSec=5m

[Install]
WantedBy=multi-user.targetThis sample is set to run the service 5 minutes after boot and then every 5 minutes. Read thedocumentationif you need to set different parameters.Reload SystemD services:systemctl daemon-reloadEnable the timer:systemctl enable healthchecker.timerStart it:systemctl start healthchecker.timerTip: you can see the execution log withjournalctl -u healthchecker(or the service name used). Check journalctl help for additional filtering options.Thanks @snkisuke for your help with this section.Developing and PRingFor more information and use cases, refer toDEVELOPERS.md.CollaboratorsMany thanks to those that collaborate with this project (in alphabetical order): @erus, @seykron, @snkisuke.LicenseHealthCheckeris made byHacKanunder GNU GPL v3.0+. You are free to use, share, modify and share modifications under the terms of thatlicense.Copyright (C) 2019 HacKan (https://hackan.net)

This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

You should have received a copy of the GNU General Public License
along with this program.  If not, see <http://www.gnu.org/licenses/>.