remove debugging info
check the effective url
don't check for hash here
cache runs
check for hash before getting links
don't check 404 pages either