removed url scheme before checking if url was previously seen, also fixed host url if redirected