gresrun/jesque — pull request #14 by tjhruska (2012-03-25 18:44:06+00:00)
Title: Some updates to give greater flexibility to a server utilizing jesque

We deploy a set of servers that pull jobs from hundreds of queues, but we limit server load by only starting up a fixed number of workers per server instance. Each queue is given a single threaded worker as we can't allow more than a single job to work each queue as they utilize limited resources. Our queues are randomly empty so in order to balance work to all of the queues we added the ability to have a worker shutdown if there are no jobs in queue after a fixed number of polls.

We also have a great need to not interrupt currently running jobs when we shut down the server as they would otherwise leave data in partially completed states on a system where we are unable to commit changes atomically. So we added the ability to signal workers to shutdown, and then have the server join to the worker threads until they have all finished running before shutting down the server.

Since jesque doesn't have the ability to prioritize jobs within a queue we decided a simple expedient is to add our high priority jobs to the head of the job queue so we added support to jesque to 'headQueue' jobs.

The last change we added was the ability to have jobs acquire system locks to coordinate work needed in some limited situations.

Each one of these items is in a single commit, and the can be accepted or rejected independently.