GitHub repository: DunaevViktor/WebCrawler
Primary language: HTML
Created: 2020-07-12T10:27:57Z
Description: Simple web crawler that traverses websites following predefined link depth and max visited pages limit. Web crawler starts from predefined URL and follows links found to dive deeper. The main purpose of this crawler to detect the presence of some terms on the page and collect statistics.