GitHub repository: ConnectionMaster/frontera
Primary language: Python
Description: A scalable frontier for web crawlers