Here are all the tweaks I made

Mostly about cleaning up output a little, getting it to crawl a sitemap, handling some ascii errors, and handling some types of ignorable 404/url errors that we didn't think of before