BBC-News/wraith — pull request #126 by mattsenior (2014-04-10 07:58:15+00:00)
Title: Parallel execution for performance improvements

Implementation of [grosser/parallel](https://github.com/grosser/parallel) for parallel execution of `save_images`, `crop_images`, `compare_images`, and `generate_thumbnails` tasks.

Takes a 15-minute batch down to under 3 mins on my local tests.

Occasionally runs into the bug where screenshots fail to save from the headless browser, so includes a (crude) retry loop to mitigate this. This should be investigated at source to see what’s causing it, but I’ve never encountered it failing repeatedly enough times to cause a problem with the comparison…

TODO:
- If a thumbnail capture fails enough times, we could grab the fallback image, resize it, and copy it into the expected position - I’ve not yet implemented this.