GitHub repository: pombredanne/crawl-frontier
Primary language: Python
Description: A flexible frontier for web crawlers