github.com/fnkr/gocrawl (go): Simple web crawler.
Keywords: crawler|http-client