GitHub repository: p4u/domaincrawler
Primary language: Python
Description: It is an HTTP crawler which looks for domains in <a> tags and sotres them into a sqlite database next to their IP address. it works recursively among the stored domains.