robotstxt (npm): a robotstxt parser for node.js
Keywords: robots.txt|parser|web|crawl|seo