praekelt/vumi-go — pull request #793 by jerith (2013-11-26 14:46:00+00:00)
Title: Better conversation metrics collection machinery

Our metrics collection worker is loading every active conversation every few minutes to collect metrics. The following process will mitigate this somewhat:

1. We look up conversations per account as normal. This will bite us eventually, but it should be okay for now.
2. For each account, we `HGETALL` a redis hash with a mapping of conversation keys to worker names for that account.
* We `HDEL` any conversations in the hash but not the list. (They're archived now.)
* We fetch any conversations in the list but not the hash and `HSET` them with their `worker_name`.
* We cache the worker name in-process along with the conversation key for collection when the right time comes around.