alphagov/govuk_crawler_worker — pull request #59 by mattbostock (2014-08-08 10:44:48+00:00)
Title: [#76585878] Mark redirect urls as seen

Mark URLs returning a HTTP 3XX redirect status as 'seen' in the
Redis-backed TTLHashSet so that we don't try crawling them again until
the TTLHashSet expiry time (currently 48 hours) has passed.

Also, change the if block to use `switch` for readability.

* * *

Also, change an `INFO` log line to an `ERROR`.

* * *

Finally, use `fallthrough`[1] to reduce the amount of duplication. I'm not sure
about this however because I think it makes the logic difficult to
understand.

[1]: http://code.google.com/p/go-wiki/wiki/Switch#Fall_through