[#75962980] Check URL is blacklisted when reading from queue 

Check if a URL is blacklisted as we read it from the RabbitMQ queue to
prevent us from crawling blacklisted URLs.

Previously we would only check URLs that were extracted from a crawled
HTML document.