Added support for flags and subdirectories and cleaned up error handling

Now you can run scrape.py with the flags indicated in the README.