nickmoline/robots-checker (packagist): Class to check a URL for robots exclusion using all possible methods of robots exclusion
Keywords: robots.txt|x-robots-tag|meta robots