some simplification to the scraper logic