Don't check URLs which we have previously scanned

This check needs to be further up to prevent unnecessary HEAD requests.