robotstxt (cargo): A native Rust port of Google's robots.txt parser and matcher C++ library.
Keywords: parser|robots|robotstxt|google-robots-parser|rust