robots (hex): A parser for robots.txt.
Keywords: crawling|erlang|erlang-library|parser|parsing|parsing-library|robots-parser|robots-txt