[#75962980] Check URL is blacklisted when reading from queue 

Check if a URL is blacklisted as we read it from the RabbitMQ queue to
prevent us from crawling blacklisted URLs.

Previously we would only check URLs that were extracted from a crawled
HTML document.
Let's work on it together tomorrow. It's an item of tech debt we've acknowledged and should get fixed before it goes into master.