canonical, cleanup, and fix crawl_urls return data