GitHub repository: zrashwani/arachnid
Primary language: PHP
Description: Crawl all unique internal links found on a given website, and extract SEO related information - supports javascript based sites