Updating README and config to bump cluster up to 20 nodes
Changed crawler.py to be more reslient to file failures (had an issue with trying to write to a directory thathad already been uploaded) and to progressively update the success and failure files so that it's easier to recover partial progress