seomoz/qsome-core — pull request #1 by dlecocq (2013-05-08 15:12:02+00:00)
Title: HC-9

This is finally passing unit tests in qsome-py, and implements queues comprising of subqueues. Unfortunately there is code in the initial commit as well :-/

Each queue has a configurable concurrency limit, which determines how many concurrent jobs may be run from any given subqueue. By default, this is set to 1, meaning that no two jobs from the same subqueue can be run at the same time.

Jobs are put into a subqueue based on a hash provided when submitting a job. Queues can be resized, changing the number of subqueues it consists of. A resize doesn't impact the current state of the jobs contained within, but obviously it does affect job availability (whether jobs can be popped off or not).