GitHub repository: mjorgens/web-crawler
Primary language: PHP
Created: 2020-01-12T17:21:33Z
Description: This is a PHP library that takes a starting URL and then parses the page Html and extracts the URLs. It then follows the URL and parses those pages until the max number of URLs is reached.