GitHub repository: tbisiar/buildit_crawler
Primary language: Python
Created: 2020-04-07T12:27:54Z
Description: A simple web crawler.  Given a domain url it should visit all pages within the domain, but not follow the links to external sites such as Google or Twitter