celery/celery — pull request #2410 by tweedmorris (2014-12-10 02:41:47+00:00)
Title: filter out latest messages for all tasks, not just the one being queried

Hello!
I was playing around with the RPC backend and noticed that groups of tasks were generally failing in interactive mode. Easy test case using classical add() task with RPC backend and AMQP storage:

`>>> t1 = tasks.add.delay(1,2)`
`>>> t2 = tasks.add.delay(3,4)`
`# ... wait a few milliseconds for both tasks to complete ...`
`>>> print(t1.ready(), t2.ready())`
`True False`
`# ohno!`

t2 will never report ready() despite having executed, and worse, a request to t2.get() will now hang indefinitely. In general, only the first task from any batch that is queried for its status will have a sane status.

The culprit is get_task_meta() in the amqp backend, inherited by the rpc backend. Since the amqp backend uses one channel per task, it safely assumes that it can discard all queued messages except the last one targeted to task_id.
The rpc backend cannot afford such profligacy: any time a task metadata is queried, all pending messages for other tasks are discarded, and subsequent metadata requests by those tasks will wait forever for a message that will never come, until perhaps its 128-bit GUID is duplicated some trillions of years hence.

This patch provides a suitable get_task_meta() implementation for amqp. I would be thrilled if you found it suitable.

cheers
Morris