github.com/hu17889/go_spider (go): [爬虫框架 (golang)] An awesome Go concurrent Crawler(spider) framework. The crawler is flexible and modular. It can be expanded to an Individualized crawler easily or you can use the default crawl components only. 
Keywords: crawler|go|pipeline|schedule|spider