Each URL should be logged somehow so the user will know it was crawled.

We shouldn't just drop URLs without logging them (warn / debug, depends on the case)