github.com/amartorelli/millipedes (go): A webcrawler written in Go which builds a graph of all the links found in a domain
Keywords: crawler|go|golang|webcrawler