CrawSpider: support process_links as generator

Adding tests for CrawlSpider's process_links