Package: torque
OverviewTorque is a web hook task queue based onTornadoandredis.  It’s designed to
solve two problems in the context of a web application:you want to do something lateryou want to do a number of things in parallelThere are many ways of approaching these problems.  For example, in python, you
might look atTwisted,CeleryandStackless.Torque is inspired by Google App Engine’staskqueue, whichmodelstasks aswebhooks.  This approach allows you to handle tasks within your normal web
application environment by writing request handlers, just as you would to handle
a user initiated request.To use it, you need to run:aredisdatabase./bin/torque-serve, which exposes aTornadoapplication (by default onhttp://localhost:8889)one./bin/torque-processper queueOr, if you’re just using the default queue, you can replace./bin/torque-serveand./bin/torque-processwith:./bin/torque-runWhich runs the web application and the process loop in seperate threads.You can process queues ad infinitum, or until they are empty.  Seetorque.process.QueueProcessor.__doc__for the details.You can add tasks to the queue in two ways:by posting an HTTP request to theTornadoapplication run by./bin/torque-serveor by using the python client api intorque.clientThis first method allows you to use Torque from any programming language.  The second
makes it much simpler if you’re using python.To add a task using an HTTP request, post to/add_taskwith two params:urlwhich is the url to the webhook you want the task to requestparamswhich is a json encoded dictionary of the params you want
to post to the webhook you’re requestingAn example in python (with the Tornado application available onlocalhost,
running on port8889) would be:import json
import urllib

mytask = {
    'url': 'http://mywebservice.com/hooks/do/foo',
    'params': json.dumps({'foo', 'somevalue', 'baz': 99})
}
target_url = 'http://localhost:8889/add_task'
urllib.urlopen(target_url, urllib.urlencode(mytask))This queued a POST request tohttp://mywebservice.com/hooks/do/foowith
the paramsfoo=somevalueandbaz=99to be made as soon as possible.You can do something similar using any programming language that can make url
requests.  However, if you are using python, it’s much simpler to use the client
api that Torque provides:from torque.client import add_task
t = add_task(url='http://mywebservice.com/hooks/do/foo', params={'a': 1})Note that this doesn’t require json encoding the params.  For all the client api
options, seetorque.client.Task.__doc__.Individual tasks backoff exponentially if they error, upto a maximum backoff delay
that’s configurable as--max_task_delay, until they error--max_task_errorstimes (at which point they get deleted).InstallInstall theredisandTornadodependencies.  Then install Torque:$ easy_install torqueOr manually from source:$ git clone git://github.com/thruflo/torque.git
$ cd torque
$ python setup.py installRunRunredis:$ ./redis-serverStart theTornadoapplication:$ ./bin/torque-serveIf you want to run the tests, use:$ ./bin/nosetests -w ./src/torque --with-doctest
.......
----------------------------------------------------------------------
Ran 7 tests in 22.627s

OKStart the default task queue running ad infinitum:$ ./bin/torque-processSee--helpagainst either of the torque console scripts for a list of configuration
options.  For example, to run a second queue calledfoobar, you might use:./bin/torque-process --queue_name=foobarOr to process the default queue once until empty you might use:./bin/torque-process --finish_on_empty=true --max_task_errors=3Or to do exactly the same from python code:from torque.processor import QueueProcessor
QueueProcessor(max_task_errors=3).process(finish_on_empty=true)Read the source code for more information.