gem/openquake — pull request #590 by al-maisan (2011-11-29 22:44:17+00:00)
Title: Hazard curve calculation and serialization is now interleaved

Hello there!

This branch is the last piece of
https://bugs.launchpad.net/openquake/+bug/894398
and also addresses
https://bugs.launchpad.net/openquake/+bug/894433

All hazard calculation tasks run fully asynchronously and their
respective serializers are now used as asynchronous task handlers.
The main enhancements are:
- hazard curve calculation and serialization is now interleaved
- we can remove intermediate hazard results from redis after we are
done with a block of sites (scalability improvement!)
- completed hazard celery tasks will not have their return value
sent back to the control node in a message (reduced load on
RabbitMQ, scalability improvement!)

The splitting of the overall set of sites into blocks is now done in
ClassicalMixin.execute(). The main reason being that the redis content
cannot be purged until all the different kind of hazard curves/maps have
been computed and serialized for a particular block of sites.

Please let me know what you think!