updates to webscraper. closed certain pages depending on date