A flexible web crawler based on Scrapy for fetching most of Ajax(js rendered) or other various types of web pages. Easy to use: To customize a new web crawler-You just need to write a config file and run.