ask/celery — Issue #537: ACKS_LATE doesn't ensure eventual completion when results can't be stored 
Author: jcspray
Created: 2011-11-12 11:44:45+00:00
State: open

See http://groups.google.com/group/celery-users/browse_thread/thread/651f1b0ff5ed9fc2/65c661f56cde166c?show_docid=65c661f56cde166c

If I lose database connectivity such that I get Mysql OperationalError exceptions, celery fails tasks that I want to retry.
* Task experiences a database-related exception, I handle that, I call .retry()
* celery's mark_as_retry throws an exception, because it can't get to the DB
* TaskRequest.on_failure gets called, and calls self.acknowledge,
consuming the message from RabbitMQ
* The task remains in state STARTED, and never gets executed, even
when I bring the database back online and restart the worker.

It seems to me that when using ACKS_LATE, the late ack should not be
sent unless the result was stored successfully -- if this was the case, I could handle this scenario. As it is
there seems to be no way to guarantee my tasks get restarted.