GitHub repository: milan1310/distributed-scrapy-scraping
Primary language: Python
Created: 2024-05-15T09:45:57Z
Description: This project demonstrates a distributed web scraping setup using Scrapy, Celery, Redis, and scrapy-redis, enabling efficient and scalable data extraction across multiple nodes. Ideal for high-performance scraping tasks.