GitHub repository: mondongos/web-crawl
Primary language: JavaScript
Description: A web crawler which, given a starting URL, visits every reachable page on the domain and returns the URL of every static asset (images, javascript, stylesheets) on each page in JSON format.