Package: job-runner-worker
This package contains the Job-Runner Worker, which is responsible for executing
the scheduled jobs managed by the Job-Runner.LinksdocumentationsourceInstallationRequirements (depending on your distro, the naming might be a bit different):python-devbuild-essentiallibevent-devThen you should be able to install this package withpip installjob-runner-worker.If you want to install this package in development mode, clone this repository
and then executepython setup.py develop. In the latter, you might want
to install the testing requirements by executingpip install-rtest-requirements.txt.See the getting started section in the Job-Runner documentation (
in thejob-runnerrepo) for setting up the whole project.ConfigurationExample with required settings:[job_runner_worker]
api_base_url=http://domain.of.job.runner/
api_key=worker1
secret=verysecret
script_temp_path=/tmp
ws_server_hostname=domain.of.websocket.server
broadcaster_server_hostname=domain.of.broadcast.serverAll available settingsapi_base_urlThe base URL which will be used to access the API. This should start withhttp://orhttps://.api_keyPublic-key to access the API.secretPrivate-key to access the API.concurrent_jobsThe number of jobs to run concurrently. Default:4.log_levelThe log level. Default:'info'. Valid options are:debuginfowarningerrormax_log_bytesThe maximum number of bytes of the log that is sent back to the API. This
is to avoid413 Request Entity Too Largeerrors. If the log will be
larger than this value, 20% of the allowed size will be taken from the top
of the log, the remaining 80% will be taken from the bottom. Everything
in between will be truncated. Default:819200(800kb).ws_server_hostnameThe hostname of the WebSocket Server.ws_server_portThe port of the WebSocket Server. Default:5555.script_temp_pathThe path where the scripts that are being executed through the Job-Runner
are temporarily stored. Default:'/tmp'.broadcaster_server_hostnameThe hostname of the queue broadcaster server.broadcaster_server_portThe port of the queue broadcaster server. Default:5556.reconnect_after_inactivitySeconds after which the subscriber is re-connecting to the publisher
when no data has been received. Default:300. This is useful when you
are loadbalancing the publisher and it keeps the TCP connection open on the
front-end, when the connection on the back-end has been closed. Because of
this ZMQ doesn’t detect that it is not connected anymore and jobs get
stuck.Command-line usageFor starting the worker, you can use thejob_runner_workercommand:usage: job_runner_worker [-h] [--config-path CONFIG_PATH]

Job Runner worker (v2.1.0)

optional arguments:
  -h, --help            show this help message and exit
  --config-path CONFIG_PATH
                        absolute path to config file (default: CONFIG_PATH env
                        variable)Changesv2.1.2Rollback retry on 4xx errors. Instead, recover when an unexpected error
occurs in theexecute_run,enqueue_actions, orkill_run. This
will recover from when a run was claimed by two workers (e.g. in the case
when it was sent to worker a, which doesn’t respond directly, then it was
sent to worker b which claims it after which a claims it too).v2.1.1Make sure a shebang does exist on scripts to be run. Use shlex to make
Popen safer.Retry request 5x when the response is in the 4xx range before raising an
exception.v2.1.0On ping response, send back the version of the worker and the number of
concurrent jobs. This version requires that you havejob-runner>=3.4.0running.v2.0.3Update error message when job does not start to be more verbose and specific.v2.0.2Fix the case where in case of an exception, the run was marked as completed
but not started.v2.0.1Make sure to only cleanup runs that are assigned to the worker. This version
is dependent onjob-runner>=3.0.1.v2.0.0Make the worker compatible with the new worker-pool structure.IMPORTANT:This version is dependent onjob-runner>=2.0.0!ChangeSETTINGS_PATHenvironment variable toCONFIG_PATHfor better
naming consistency.Make sure that when a run already has log, it is updated (before it would
hang on the database integrity error).v1.2.1Make the worker crash early instead of hanging on errors happening before the
actual job starts, to give the user a visible cue that something went wrong.v1.2.0The worker will now terminate gracefully when receiving theTERMsignal.
This means that all pending jobs will be completed, but that it will not
accept any new jobs. After finishing the last pending job, the worker will
terminate.v1.1.4Setreconnect_after_inactivitydefault to 10 minutes. This is 2 x theJOB_RUNNER_WORKER_PING_INTERVALdefault setting in Job-Runner.v1.1.3Implement handler forpingaction.v1.1.2Add and implementreconnect_after_inactivitysetting.v1.1.1Run script by finding their shebang without the x bit being needed.v1.1.0Handle separate run log-output resource. This requires Job-Runner >= v1.3.0.v1.0.7Fix killing job-runs. Wherev1.0.5was killing children processes, it did
not kill children of children, … This should kill the full tree of
child-processes.v1.0.6Freeze requests library version, since 1.0.0 contains backwards compatible
changes.v1.0.5Fix killing job-runs. When the process had sub-processes, only the parent
process was killed and the worker was waiting for the child-processes to
complete.v1.0.4Add config variablemax_log_bytesto limit the amount of logdata that
will be send back to the API (to avoid413 Request Entity Too Largeerrors).v1.0.3Sendpidback to the REST API when a job has been started.Kill a job-run when akillaction is received.v1.0.2Make sure that the API exactly matches.v1.0.1Make the timezones send to the REST API timezone aware.v1.0.0Deployar related changes.v0.7.1Fix encoding issue when writing the file.v0.7.0Refactor to make the worker compatible with the 0.7 version of thejob-runnerpackage.Make it consume runs from the queue broadcaster instead of hitting the REST
interface every x seconds.Add retry on error to recover from temporary REST interface errors.v0.6.1Merge fixes v0.5.1 and v0.5.2 into v0.6.x version.v0.6.0Refactor to make use of separate WebSocket Server.v0.5.2Make temporary path for scripts configurable.v0.5.1Disable SSL certificate validation.v0.5.0Initial release.