Follow http-equiv refresh url
Canonicalize the url
Ignore txt and html files