smoothcrawler (pypi): Build crawler humanly as different roles which be combined with different components.
Keywords: crawler|multi-runnable|python|web-spider