Package: crontabber
A cron job runner with self-healing and job dependencies.License:MPL 2How to run testsFirst you need to create a dedicated test database. We recommend you
call ittest_crontabber. Then you need the necessary credentials for
it.Before running the tests you need to install some extras to be able to
run tests at all:pip install -r test-requirements.txtNext, in the root directory of the project create a file calledtest-crontabber.iniand it should look something like this:[crontabber]
user=myusername
password=mypassword
dbname=test_crontabberTo start all the tests run:PYTHONPATH=. nosetestsIf you want to run a specific test in a specific file in a specific
class you can define it per thenosetestsstandard like this for
example:PYTHONPATH=. nosetests tests crontabber/tests/test_crontabber.py:TestCrontabber.test_basic_run_jobIf you want the tests to stop as soon as the first test fails add-xto that same command above.Also, if you wantnoseteststonotcapturestdoutadd-sto that same command as above.How to do code coverage analysisFirst you need to install thecoveragemodule. Then,
withnosetests, you can run this:PYTHONPATH=. nosetests --with-coverage --cover-erase --cover-html --cover-package=crontabberAfter it has run, you can open the filecover/index.htmlin browser.How to run the exampleappThe example app helps you set up a playground to play around with and
test crontabber to gain a better understanding of how it works.The best place to start with is to read theexampleapp/README.mdfile and go through its steps. Once you get the basics to work you can
start experimenting with adding your job classes.How locking workscrontabber supports locking. It basically means if you start a second
instance of crontabber whilst it’s already ongoing in another
terminal/server the second one will exist early. This is only applicable
if there is an actual job ongoing.There are two kinds of locking.General locking.The first thing crontabber does before it starts
an app is to ask the state (stored in PostgreSQL) if it’s ongoing and
if it is, it exists with an error code of3.Sub-second locking.If the general locking (see point above) says
“No, the job is not ongoing”, it’s going to proceed to update the
state with arow-level locking transaction in
PostgreSQL.
That basically means PostgreSQL only allows one singleUPDATEfrom the process that gets there first. The second crontabber process
will will exit early with an error code of2if the first
crontabber process managed to run theUPDATEfirst.Imagine two separate terminals starting crontabber at the almost same
time:# Terminal 1
$ python crontabber.py --admin.conf=crontabber.ini
$ echo $?
0# Terminal 2 (started almost simultaneously)
$ python crontabber.py --admin.conf=crontabber.ini
$ echo $?
3Note!If a job has been ongoing to a maximum period of time, the
locking is ignored. This is controlled by the config optioncrontabber.max_ongoing_age_hourswhich defaults to12 hours.
This is applicable if crontabber, updates the state that it’s starting a
job, then when it tries to update the state that it finished
(successfully or not) and that write fails, if for example it’s unable
to make a connection to PostgreSQL. If this happens crontabber will just
ignore the lock and run it anyway.