New crawl methods

Run crawl as these default, backwards or random. Open multiple SSH
windows and run them all in parallel.