Package: yacron
Yet Another CronA modern Cron replacement that is Docker-friendlyFree software: MIT licenseFeatures“Crontab” is in YAML format;Builtin sending of Sentry and Mail outputs when cron jobs fail;Flexible configuration: you decide how to determine if a cron job fails or not;Designed for running in Docker, Kubernetes, or 12 factor environments:Runs in the foreground;Logs everything to stdout/stderr[1];Option to automatically retry failing cron jobs, with exponential backoff;Optional HTTP REST API, to fetch status and start jobs on demand;Arbitrary timezone support;[1]Whereas vixie cron only logs to syslog, requiring a syslog daemon to be running in the background or else you don’t get logs!StatusThe project is in beta stage: essential features are complete, and the focus is
finding and fixing bugs before the first stable release.InstallationInstall using pipyacron requires Python >= 3.6 (for systems with older Python, use the binary instead).  It is advisable to install it in a Python
virtual environment, for example:python3-mvenvyacronenv.yacronenv/bin/activatepipinstallyacronInstall using pipxpipxautomates creating a virtualenv and installing a python program in the
newly created virtualenv.  It is as simple as:pipxinstallyacronInstall using binaryAlternatively, a self-contained binary can be downloaded
from github:https://github.com/gjcarneiro/yacron/releases. This binary should
work on any Linux 64-bit system post glibc 2.23 (e.g. Ubuntu:16.04).  Python is not required on the target system (it is embedded in the executable).UsageConfiguration is in YAML format.  To start yacron, give it a configuration file
or directory path as the-cargument.  For example:yacron -c /tmp/my-crontab.yamlThis starts yacron (always in the foreground!), reading/tmp/my-crontab.yamlas configuration file.  If the path is a directory,
any*.yamlor*.ymlfiles inside this directory are taken as
configuration files.Configuration basicsThis configuration runs a command every 5 minutes:jobs:-name:test-01command:echo "foobar"shell:/bin/bashschedule:"*/5****"The command can be a string or a list of strings.  If command is a string,
yacron runs it through a shell, which is/bin/bashin the above example, but
is/bin/shby default.If the command is a list of strings, the command is executed directly, without a
shell.  The ARGV of the command to execute is extracted directly from the
configuration:jobs:-name:test-01command:-echo-foobarschedule:"*/5****"Thescheduleoption can be a string in a crontab format specified byhttps://github.com/josiahcarlson/parse-crontab(this module is used by yacron).
Additionally @reboot can be included , which will only run the job when yacron is initially
executed. Furtherschedulecan be an object with properties.  The following configuration
runs a command every 5 minutes, but only on the specific date 2017-07-19, and
doesn’t run it in any other date:jobs:-name:test-01command:echo "foobar"schedule:minute:"*/5"dayOfMonth:19month:7year:2017dayOfWeek:"*"Important: by default all time is interpreted to be in UTC, but you can
request to use local time instead.  For instance, the cron job below runs
every day at 19h27local timebecause of theutc: falseoption:jobs:-name:test-01command:echo "hello"schedule:"2719***"utc:falsecaptureStdout:trueSince Yacron version 0.11, you can also request that the schedule be
interpreted in an arbitrary timezone, using thetimezoneattribute:jobs:-name:test-01command:echo "hello"schedule:"2719***"timezone:America/Los_AngelescaptureStdout:trueYou can ask for environment variables to be defined for command execution:jobs:-name:test-01command:echo "foobar"shell:/bin/bashschedule:"*/5****"environment:-key:PATHvalue:/bin:/usr/binYou can also provide an environment file to define environments for command execution:jobs:-name:test-01command:echo "foobar"shell:/bin/bashschedule:"*/5****"env_file:.envThe env file must be a list ofKEY=VALUEpairs. Empty lines and lines starting with#will be ignored.Variables declared in theenvironmentoption will override those found in theenv_file.Specifying defaultsThere can be a specialdefaultssection in the config.  Any attributes
defined in this section provide default values for cron jobs to inherit.
Although cron jobs can still override the defaults, as needed:defaults:environment:-key:PATHvalue:/bin:/usr/binshell:/bin/bashutc:falsejobs:-name:test-01command:echo "foobar"# runs with /bin/bash as shellschedule:"*/5****"-name:test-02# runs with /bin/sh as shellcommand:echo "zbr"shell:/bin/shschedule:"*/5****"Note: if the configuration option is a directory and there are multiple configuration files in that directory, then thedefaultssection in each configuration file provides default options only for cron jobs inside that same file; the defaults have no effect beyond any individual YAML file.ReportingYacron has builtin support for reporting jobs failure (more on that below) by
email, Sentry and shell command (additional reporting methods might be added in the future):-name:test-01command:|echo "hello" 1>&2sleep 1exit 10schedule:minute:"*/2"captureStderr:trueonFailure:report:sentry:dsn:value:example# Alternatively:# fromFile: /etc/secrets/my-secret-dsn# fromEnvVar: SENTRY_DSNfingerprint:# optional, since yacron 0.6-yacron-"{{environment.HOSTNAME}}"-"{{name}}"extra:foo:barzbr:123level:warningenvironment:productionmail:from:example@foo.comto:example@bar.comsmtpHost:127.0.0.1# optional fields:username:"username1"# set username and password to enable loginpassword:value:example# Alternatively:# fromFile: /etc/secrets/my-secret-password# fromEnvVar: MAIL_PASSWORDtls:false# set to true to enable TLSstarttls:false# set to true to enable StartTLSshell:shell:/bin/bashcommand:...Here, theonFailureobject indicates that what to do when a job failure
is detected.  In this case we ask for it to be reported both to sentry and by
sending an email.ThecaptureStderr: truepart instructs yacron to capture output from the the
program’sstandard error, so that it can be included in the report.  We could
also turn onstandard outputcapturing via thecaptureStdout: trueoption.
By default, yacron captures only standard error.  If a cron job’s standard error
or standard output capturing is not enabled, these streams will simply write to
the same standard output and standard error as yacron itself.Bothstdoutandstderrstream lines are by default prefixed with[{job_name} {stream_name}], i.e.[test-01stdout], if for any reason you
need to change this, provide the optionstreamPrefix(new in version 0.16)
with your own custom string.-name:test-01command:echo "hello world"schedule:minute:"*/2"captureStdout:truestreamPrefix:"[{job_name}job]"In some cases, for instance when you’re logging JSON objects you might want to
completely get rid of the prefix altogether:-name:test-01command:echo "hello world"schedule:minute:"*/2"captureStdout:truestreamPrefix:""It is possible also to report job success, as well as failure, via theonSuccessoption.-name:test-01command:echo "hello world"schedule:minute:"*/2"captureStdout:trueonSuccess:report:mail:from:example@foo.comto:example@bar.comsmtpHost:127.0.0.1Since yacron 0.5, it is possible to customise the format of the report. Formailreporting, the optionsubjectindicates what is the subject of the
email, whilebodyformats the email body.  For Sentry reporting, there is
onlybody.  In all cases, the values of those options are strings that are
processed by thejinja2templating engine.  The following variables are
available in templating:name(str): name of the cron jobsuccess(bool): whether or not the cron job succeededstdout(str): standard output of the processstderr(str): standard error of the processexit_code(int): process exit codecommand(str): cron job commandshell(str): cron job shellenvironment(dict): subprocess environment variablesExample:-name:test-01command:|echo "hello" 1>&2sleep 1exit 10schedule:minute:"*/2"captureStderr:trueonFailure:report:mail:from:example@foo.comto:example@bar.comsmtpHost:127.0.0.1subject:Cron job '{{name}}' {% if success %}completed{% else %}failed{% endif %}body:|{{stderr}}(exit code: {{exit_code}})The shell reporter (since yacron 0.13) executes a user given shell command in
the specified shell. It passes all environment variables from the python
executable and specifies some additional ones to inform about the state of the
job:YACRON_FAIL_REASON (str)YACRON_FAILED (“1” or “0”)YACRON_JOB_NAME (str)YACRON_JOB_COMMAND (str)YACRON_JOB_SCHEDULE (str)YACRON_RETCODE (str)YACRON_STDERR (str)YACRON_STDOUT (str)A simple example configuration:-name:test-01command:echo "foobar" && exit 123shell:/bin/bashschedule:"*****"onFailure:report:shell:shell:/bin/bashcommand:echo "Error code $YACRON_RETCODE"Since yacron 0.15, it is possible to send emails formatted as html, by  adding
thehtml: trueproperty.  For example, here the standard output of a shell
command is captured and interpreted as html and placed in the email message:-name:test-01command:echo "hello <b>world</b>"schedule:"@reboot"captureStdout:trueonSuccess:report:mail:from:example@foo.comto:example@bar.com, zzz@sleep.comhtml:truesmtpHost:127.0.0.1smtpPort:1025subject:This is a cron job with html bodyMetricsYacron has builtin support for writing job metrics toStatsd:jobs:-name:test01command:echo "hello"schedule:"*****"statsd:host:my-statsd.example.comport:8125prefix:my.cron.jobs.prefix.test01With this config Yacron will write the following metrics over UDP
to the Statsd listening onmy-statsd.example.com:8125:my.cron.jobs.prefix.test01.start:1|g  # this one is sent when the job starts
my.cron.jobs.prefix.test01.stop:1|g   # the rest are sent when the job stops
my.cron.jobs.prefix.test01.success:1|g
my.cron.jobs.prefix.test01.duration:3|ms|@0.1Handling failureBy default, yacron considers that a job hasfailedif either the process
returns a non-zero code or if it generates output tostandard error(and
standard error capturing is enabled, of course).You can instruct yacron how to determine if a job has failed or not via thefailsWhenoption:failsWhen:producesStdout:falseproducesStderr:truenonzeroReturn:truealways:falseproducesStdoutIf true, any captured standard output causes yacron to consider the job
as failed.  This is false by default.producesStderrIf true, any captured standard error causes yacron to consider the job
as failed.  This is true by default.nonzeroReturnIf true, if the job process returns a code other than zero causes yacron
to consider the job as failed.  This is true by default.alwaysIf true, if the job process exits that causes yacron to consider the job as
failed.  This is false by default.It is possible to instruct yacron to retry failing cron jobs by adding aretryoption insideonFailure:-name:test-01command:|echo "hello" 1>&2sleep 1exit 10schedule:minute:"*/10"captureStderr:trueonFailure:report:mail:from:example@foo.comto:example@bar.comsmtpHost:127.0.0.1retry:maximumRetries:10initialDelay:1maximumDelay:30backoffMultiplier:2The above settings tell yacron to retry the job up to 10 times, with the delay
between retries defined by an exponential backoff process: initially 1 second,
doubling for every retry up to a maximum of 30 seconds. A value of -1 for
maximumRetries will mean yacron will keep retrying forever, this is mostly
useful with a schedule of “@reboot” to restart a long running process when it
has failed.If the cron job is expected to fail sometimes, you may wish to report only in
the case the cron job ultimately fails after all retries and we give up on it.
For that situation, you can use theonPermanentFailureoption:-name:test-01command:|echo "hello" 1>&2sleep 1exit 10schedule:minute:"*/10"captureStderr:trueonFailure:retry:maximumRetries:10initialDelay:1maximumDelay:30backoffMultiplier:2onPermanentFailure:report:mail:from:example@foo.comto:example@bar.comsmtpHost:127.0.0.1ConcurrencySometimes it may happen that a cron job takes so long to execute that when the moment its next scheduled execution is reached a previous instance may still be running.  How yacron handles this situation is controlled by the optionconcurrencyPolicy, which takes one of the following values:Allowallows concurrently running jobs (default)Forbidforbids concurrent runs, skipping next run if previous hasn’t finished yetReplacecancels currently running job and replaces it with a new oneExecution timeout(new in version 0.4)If you have a cron job that may possibly hang sometimes, you can instruct yacron
to terminate the process after N seconds if it’s still running by then, via theexecutionTimeoutoption.  For example, the following cron job takes 2
seconds to complete, yacron will terminate it after 1 second:-name:test-03command:|echo "starting..."sleep 2echo "all done."schedule:minute:"*"captureStderr:trueexecutionTimeout:1# in secondsWhen terminating a job, it is always a good idea to give that job process some
time to terminate properly.  For example, it may have opened a file, and even if
you tell it to shutdown, the process may need a few seconds to flush buffers and
avoid losing data.On the other hand, there are times when programs are buggy and simply get stuck,
refusing to terminate nicely no matter what.  For this reason, yacron always
checks if a process exited some time after being asked to do so. If it hasn’t,
it tries to forcefully kill the process.  The optionkillTimeoutoption
indicates how many seconds to wait for the process to gracefully terminate
before killing it more forcefully.  In Unix systems, we first send a SIGTERM,
but if the process doesn’t exit afterkillTimeoutseconds (30 by default)
then we send SIGKILL.  For example, this cron job ignores SIGTERM, and so yacron
will send it a SIGKILL after half a second:-name:test-03command:|trap "echo '(ignoring SIGTERM)'" TERMecho "starting..."sleep 10echo "all done."schedule:minute:"*"captureStderr:trueexecutionTimeout:1killTimeout:0.5Change to another user/group(new in version 0.11)You can request that Yacron change to another user and/or group for a specific
cron job.  The fielduserindicates the user (uid or userame) under which
the subprocess must be executed.  The fieldgroup(gid or group name)
indicates the group id.  If onlyuseris given, the group defaults to the
main group of that user.  Example:-name:test-03command:idschedule:minute:"*"captureStderr:trueuser:www-dataNaturally, yacron must be running as root in order to have permissions to
change to another user.Remote web/HTTP interface(new in version 0.10)If you wish to remotely control yacron, you can optionally enable an HTTP REST
interface, with the following configuration (example):web:listen:-http://127.0.0.1:8080-unix:///tmp/yacron.sockNow you have the following options to control it (using HTTPie as example):Get the version of yacron:$httpgethttp://127.0.0.1:8080/versionHTTP/1.1200OKContent-Length:22Content-Type:text/plain;charset=utf-8Date:Sun,03Nov201919:48:15GMTServer:Python/3.7aiohttp/3.6.20.10.0b3.dev7+g45bc4ceGet the status of cron jobs:$httpgethttp://127.0.0.1:8080/statusHTTP/1.1200OKContent-Length:104Content-Type:text/plain;charset=utf-8Date:Sun,03Nov201919:44:45GMTServer:Python/3.7aiohttp/3.6.2test-01:scheduled(in14seconds)test-02:scheduled(in74seconds)test-03:scheduled(in14seconds)You may also get status info in json format:$httpgethttp://127.0.0.1:8080/statusAccept:application/jsonHTTP/1.1200OKContent-Length:206Content-Type:application/json;charset=utf-8Date:Sun,03Nov201919:45:53GMTServer:Python/3.7aiohttp/3.6.2[{"job":"test-01","scheduled_in":6.16588,"status":"scheduled"},{"job":"test-02","scheduled_in":6.165787,"status":"scheduled"},{"job":"test-03","scheduled_in":6.165757,"status":"scheduled"}]Start a job right now:Sometimes it’s useful to start a cron job right now, even if it’s not
scheduled to run yet, for example for testing:$httpposthttp://127.0.0.1:8080/jobs/test-02/startHTTP/1.1200OKContent-Length:0Content-Type:application/octet-streamDate:Sun,03Nov201919:50:20GMTServer:Python/3.7aiohttp/3.6.2Includes(new in version 0.13)You may have a use case where it’s convenient to have multiple config files,
and choose at runtime which one to use.  In that case, it might be useful if
you can put common definitions (such as defaults for reporting, shell, etc.)
in a separate file, that is included by the other files.To support this use case, it is possible to ask one config file to include
another one, via theincludedirective.  It takes a list of file names:
those files will be parsed as configuration and merged in with this file.Example, your main config file could be:include:-_inc.yamljobs:-name:my job...And your included_inc.yamlfile could contain some useful defaults:defaults:shell:/bin/bashonPermanentFailure:report:sentry:...Custom loggingIt’s possible to provide a custom logging configuration, via theloggingconfiguration section.  For example, the following configuration displays log lines with
an embedded timestamp for each message.logging:# In the format of:# https://docs.python.org/3/library/logging.config.html#dictionary-schema-detailsversion:1disable_existing_loggers:falseformatters:simple:format:'%(asctime)s[%(processName)s/%(threadName)s]%(levelname)s(%(name)s):%(message)s'handlers:console:class:logging.StreamHandlerlevel:DEBUGformatter:simplestream:ext://sys.stdoutroot:level:INFOhandlers:-consoleObscure configuration optionsenabled: true|false (default true)(new in yacron 0.18)It is possible to disable a specific cron job by adding aenabled: falseoption.  Jobs
withenabled: falsewill simply be skipped, as if they aren’t there, apart from
validating the configuration.jobs:-name:test-01enabled:false# this cron job will not run until you change this to `true`command:echo "foobar"shell:/bin/bashschedule:"*****"History0.19.0 (2023-03-11)Add ability to configure yacron’s own logging (#81 #82 #83, gjcarneiro, bdamian)Add config value for SMTP(validate_certs=False) (David Batley)0.18.0 (2023-01-01)fixes “Job is always executed immediately on yacron start” (#67)add anenabledoption in jobs (#73)give a better error message when no configuration file is provided or exists (#72)0.17.0 (2022-06-26)Support Additional Shell Report Vars (RJ Garcia)Shell reporter: handle long lines truncatation (Hannes Hergeth)exe: undo pyinstaller LD_LIBRARY_PATH changes in subprocesses (#68, Gustavo Carneiro)0.16.0 (2021-12-05)make the capture max line length configurable and change the default
from 64K to 16M (#56)Add config option to change prefix of subprocess stream lines (#58, eelkeh)0.15.1 (2021-11-19)Fix a bug in the –validate option (#57, Leonid Repin)0.15.0 (2021-11-10)Allow emails to be html formattedFix an error when reading cmd output with huge lines (#56)0.14.0 (2021-10-04)Sentry: increase the size of messages before getting truncated #54Sentry: allow specifying the environment option #53Minor fixes0.13.1 (2021-08-10)unicode fixes for the exe binary version0.13.0 (2021-06-28)Add ability for one config file to include another one #38Add shell command reporting ability (Hannes Hergeth, #50)0.12.2 (2021-05-31)constrain ruamel.yaml to version 0.17.4 or below, later versions are buggy0.12.1 (2021-05-30)blacklist ruamel.yaml version 0.17.5 in requirements #470.12.0 (2021-04-22)web: don’t crash when receiving a web request without Accept header (#45)add env_file configuration option (Alessandro Romani, #43)email: add missing Date header (#39)0.11.2 (2020-11-29)Add back a self contained binary, this time based on PyInstaller0.11.1 (2020-07-29)Fix email reporting when multiple recipients given0.11.0 (2020-07-20)reporting: add a failure reason line at the top of sentry/email (#36)mail: new tls, startls, username, and password options (#21)allow jobs to run as a different user (#18)Support timezone schedule (#26)0.10.1 (2020-06-02)Minor bugfixes0.10.0 (2019-11-03)HTTP remote interface, allowing to get job status and start jobs on demandSimple Linux binary including all dependencies (built using PyOxidizer)0.10.0b2 (2019-10-26)Build Linux binary inside Docker Ubuntu 16.04, so that it is compatible with
older glibc systems0.10.0b1 (2019-10-13)Build a standalone Linux binary, using PyOxidizerSwitch from raven to sentry-sdk0.9.0 (2019-04-03)Added an option to just check if the yaml file is valid without running the scheduler.Fix missingbodyin the schema for sentry config0.8.1 (2018-10-16)Fix a bug handling@rebootin schedule (#22)0.8.0 (2018-05-14)Sentry: add newextraandleveloptions.0.7.0 (2018-03-21)Added theutcoption and document that times are utc by default (#17);If an email body is empty, skip sending it;Added docker and k8s example.0.6.0 (2017-11-24)Add custom Sentry fingerprint supportAbility to send job metrics to statsd (thanks bofm)alwaysflag to consider any cron job that exits to be failed
(thanks evanjardineskinner)maximumRetriescan now be-1to never stop retrying (evanjardineskinner)schedulecan be the string@rebootto always run that cron job on startup
(evanjardineskinner)saveLimitcan be set to zero (evanjardineskinner)0.5.0Templating support for reportsRemove deprecated smtp_host/smtp_port0.4.3 (2017-09-13)Bug fixes0.4.2 (2017-09-07)Bug fixes0.4.1 (2017-08-03)More polished handling of configuration errors;Unit tests;Bug fixes.0.4.0 (2017-07-24)New optionexecutionTimeout, to terminate jobs that get stuck;If a job doesn’t terminate gracefully kill it.  New optionkillTimeoutcontrols how much time to wait for graceful termination before killing it;Switch parsing to strictyaml, for more user friendly parsing validation error
messages.