[#75579526] Remove #fragment when extracting URLs from the DOM

This change strips the #fragment part of the URL when extracting URLs.
It fixes a bug where we were crawling the same URL multiple times, but
with different fragment identifiers.

Rather than crawling both `gov.uk/` and `gov.uk/#foo`, we'd now only
crawl the former case.