GitHub repository: HubSpot/heritrix3
Primary language: Java
Description: Heritrix is the Internet Archive's open-source, extensible, web-scale, archival-quality web crawler project.