GitHub repository: n644dc/pywebcrawl
Primary language: Python
Created: 2017-12-01T20:20:41Z
Description: Crawls the web looking for provided content. Assigns pages thresholds for recording or throwing away the page.