throw out urls from uniqueness checking