spiderbar (cran): Parse and Test Robots Exclusion Protocol Files and Rules
Keywords: r|r-cyber|robots-exclusion-protocol|robots-txt|rstats