robots-txt-component (npm): Lightweight robots.txt parsing component without any external dependencies for Node.js.
Keywords: robots.txt|robots|robots-parser|robots-exclusion-standard|javascript|js|node-js|robots-txt|crawler|crawlers|nodejs|robots-node|robots-txt-node