More detailed output, including links to previous step
Started adding ability to cache results so a crawl can be resumed