GitHub repository: natzar/pythonSimpleCrawlerBot
Primary language: Python
Created: 2023-10-22T08:14:01Z
Description: A Simple Crawler Bot written in Python. This script is a version of the one used at Domstry.com. You can use it to crawl the internet. It fetches domains, store the data, and adds the links found in current domain to the queue.