PuerkitoBio/gocrawl — pull request #8 by PuerkitoBio (2012-12-12 02:15:37+00:00)
Title: fixes #3, closes #4

Fetch() doesn't follow redirects automatically, instead it re-enqueues the redirect-to URLs so that they always have to be whitelisted by the Filter(). Also, with the new EnqueueChan that may be present on the Extender, it is possible to re-enqueue a URL following an error, for example.