GitHub repository: scrapinghub/frontera
Primary language: Python
Description: A scalable frontier for web crawlers