GitHub repository: ml-lubich/website-scraper
Created: 2024-12-07T20:27:12Z
Description: 🕷️ Multiprocessing-enabled web scraper with rate limiting and bot detection avoidance. Use as a CLI tool or Python module. Features progress tracking, comprehensive logging, and JSON output. Perfect for efficient, respectful web scraping.