Created pipeline for automated crawling of all data