gem/oq-engine — pull request #1366 by micheles (2014-02-11 10:07:03+00:00)
Title: Getting a more uniform distribution of the tasks

In https://github.com/gem/oq-engine/pull/1357 we introduced the concept of "source weight". Tasks should have homogeneous weight, i.e. a task may contain a lot of light sources or a few heavy sources. After some analysis I discovered that the simple rule weight = number of ruptures is appropriated for point sources but not for are other source typologies. One must give a bigger weight, it must be more than linear in the number of ruptures, otherwise too many big sources will be collected in a single task. While a correct assessment of the weight would require a more serious analysis, this PR adopt the following heuristic rules:

1. for point sources weight = num_ruptures
2. for other sources weight = num_ruptures ** 1.5

In the case of the USA calculation the adoption of these rules reduces the total runtime from 8.5 hours to 1.2 hours. Good enough. Other experiments (for instance giving a quadratic weight to non-point-sources) gave worse results, because in that case too many light sources get collected in a single task.

This PR also adds more logging, so that an analysis of the computation time vs number of ruptures can be performed later on.