Package: taskrunner
# TaskRunnerExecute a certain sequence of tasks and later their cleanups. It is useful forrunning tasks with many varying configurations. It doesn't have anydependencies, just the standard library.```python# file examples/simple.pyimport taskrunnerclass ExampleTask(taskrunner.Task):def __init__(self, msg, clean_msg, **kwargs):super(ExampleTask, self).__init__(**kwargs)self.msg = msgself.clean_msg = clean_msgdef run(self, context):print self.msgdef cleanup(self, context):print self.clean_msgtask1 = {'task': ExampleTask,'name': 'task1','msg': 'hello world','clean_msg': 'goodbye'}task2 = {'task': ExampleTask,'name': 'task2','msg': 'hello again','clean_msg': ':)'}pipeline = [task1, task2]``````$ bin/taskrunner -f examples/simple.py pipeline2013-12-02 19:46:37,952 - taskrunner - INFO - =========== run task1 ===========hello world2013-12-02 19:46:37,953 - taskrunner - INFO - =========== run task2 ===========hello again2013-12-02 19:46:37,953 - taskrunner - INFO - --------- cleanup task2 ---------:)2013-12-02 19:46:37,953 - taskrunner - INFO - --------- cleanup task1 ---------goodbye```### How it worksThe pipeline is a list of task configurations, which are normal Pythondictionaries with the special item `'task':ExampleTask`. It goes trough thelist and for each task, it instantiates `ExampleTask` with the rest of thedictionary content as parameters. Then it executes `ExampleTask.run()` for allof the tasks. After it passes trough the whole list, it goes trough it inreverse order and executes `ExampleTask.cleanup()` for each item. The tasks canwrite into `context` and the content of it will be passed to the next task.### UsageYou can specify the pipeline directly as arguments:$ bin/taskrunner -f examples/simple.py task1 task2Or you can combine multiple pipelines, which will run all the tasks from eachpipeline:$ bin/taskrunner -f examples/simple.py pipeline another_task_pipelineOr even combine pipelines and tasks (this will run *task2* twice):$ bin/taskrunner -f examples/simple.py pipeline task2To use the tool as a library, you can directly use `execute`:```pythontaskrunner.execute([task1, task2])```#### Taking control of the cleanup executionSometimes you want to only execute the `run()` part of the tasks, debugsomething and only run the cleanups after you are done. To skip the cleanups,you can do:$ bin/taskrunner -f examples/simple.py pipeline --cleanup=neverTo run the cleanups only:$ bin/taskrunner -f examples/simple.py pipeline --cleanup=prontoYou can also use the options `--cleanup=on_success` or `--cleanup=on_failure`,which will get executed based on how the `run` turned out.Don't forget to make the cleanups independent of the runs, otherwise this won'twork.#### Exception and signal handlingWhen an exception occurs in the task run, its traceback is logged and it jumpsright into the cleanups. However, it doesn't clean up the tasks that didn't run(but it does clean up the task which failed and got only partially executed).If you get an exception during some cleanup, the traceback is logged butexecution continues with the next task's cleanup.The list of errors gets logged again after everything else finishes, in theorder they happened.If you terminate the run using `ctrl-c` (also known as *SIGINT*), it will gostraight to the cleanups. Sending the termination signal again will stop itcompletely. This works for the *SIGTERM* signal too.#### The name of a taskBy default, the name of a task is the class name. To have more readable logs,you can specify the keyword `name` in the task configuration. The task namescan be important for configuration redefinition from the command line.#### Redefining the task configuration trough CLI argumentsSometimes you want to run a sequence of tasks with some changes in theirconfiguration, but don't want to change the files. You can redefine it usingthe parameter `-D`.$ bin/taskrunner -f examples/simple.py pipeline -D task1.msg=pingIt can't contain any spaces, has to be in the exact format of`taskname.key1.key2.key3...=newvalue`, where `taskname` is either the name ofthe task specified in the configuration dictionary or the class name. If moretasks have the same name, it will get rewritten for all of them. For example,$ bin/taskrunner -f examples/simple.py pipeline -D ExampleTask.msg=pingwill change the message for both `task1` and `task2`, because they have thesame class name.#### Using multiple files for the task configurationsYou probably don't want to have everything in a single file. You can loadmultiple modules and reference the tasks normally.$ bin/taskrunner -f examples/advanced.py -f examples/simple.py \mytask task1 task2In case you have any name conflicts, you can specify the name of the module.$ bin/taskrunner -f examples/advanced.py -f examples/simple.py \advanced.pipeline### Best practices for writing tasks and their configurations* don't make the `cleanup` method dependent on `run`, because with theoption `--cleanup=pronto`, the `run` method won't get executed* don't assume that the `run` got executed completely* put the tasks into a separate file, which will be imported in the file withthe task configurations* use the minimum of Python features in the task configuration files (which arejust `.py` files), variable definitions and if conditions are usuallysufficient. You will be later able to switch to some other configurationformat.