github.com/schollz/crawdad (go): Cross-platform persistent and distributed web crawler :crab:
Keywords: crawler|golang|redis|web