alphagov/govuk_crawler_worker — pull request #45 by mattbostock (2014-07-31 16:53:17+00:00)
Title: For discussion: When should we store a statsd metric?

__NOTE: This PR depends on #43; please ignore all commits except the last, 35d5904. Best viewed [without whitespace differences](https://github.com/alphagov/govuk_crawler_worker/commit/35d5904b16d5e37dc5c7d89c83149821ef0663a8?w=1).__

__For discussion:__

Do we want to only log statsd timings on successful operations? For example, if we crawl a URL but it returns a HTTP 5XX error, do we still want to instrument that operation?

The changes in this PR wrap the code within the `for` loops that range over the `inbound` channel inside inline functions.

The advantage to this is that we can use `defer` for `util.StatsDTiming()`, which means that statsd metrics will always be fired even if we encounter an error. I'm not sure that's what we want, however.

Possibly we should add a new 'namespace' to the statsd keys so that we can differentiate between different kinds of errors, though that might be overkill.