scrapy/scrapy — Issue #72: CrawlSpider shouldn't follow links from non-HTML responses
Author: pablohoffman
Created: 2011-12-27 23:21:56+00:00
State: open

When CrawlSpider tries to extract links from a non-HTML response, it fails.

CrawlSpider should only follow links from HTML responses.