Canonicalise URLs in the generic crawler.