GitHub repository: rudaporto/aiocomcrawl
Primary language: Python
Created: 2021-11-08T13:16:20Z
Description: This is the aiocrawler utility. It uses asyncio and multiprocessing to quickly search and download crawl   data from the Common Crawl public dataset [https://commoncrawl.org].