Fixes #8 AggregateException when cancellation toke is used to shutdown the crawl.