Change scraper to remove duplicates and empty links