Ignore exceptions produced by crawlers
Merge pull request #122 from mezuro/ignore_crawlers

Ignore exceptions produced by crawlers