Yelp/python-gearman — Issue #17: Better handling of heterogenous server/task pairings
Author: josharian
Created: 2012-01-20 20:11:19+00:00
State: open

The following (simplified) setup goes awry:

* Start two gearmand instances A and B (say, on different ports).
* Start one worker for task T that connects to A.
* Have a client submit a job T to servers A and B.

The outcome is based on a race condition: Whichever server (A or B) replies first gets the task. If it is A, the task succeeds; if it is B, the task fails.

In general, unless all servers have the exact same set of worker tasks available, clients can't rely on requests succeeding.

This makes it difficult to rely on multiple gearmand servers for load balancing and redundancy, even if the intent is to have all workers connect to all servers (config errors, connection flukes, etc.).