Try to effectivelly capture the errors raised while processing response in downloader middlewares. Whenever the error occurs, start processing the exceptions.
Added strip_utm_tags as a submethod of url canonicalization.
Output dropped items in WARNING mode. It makes more sense. Simply don't drop good items.
Output more details when error occurs.
Replaced yields in CrawlSpider with array return.
yields in spiders doesn't work well with exceptions.
Added "modelargs" property to DjangoItem.