update the final crawl and fix time-format