Only check if the domain is not set

This was only working for the first request.  Subsequent requests to other domains were causing an undefined index error because `empty(self::$robotstxt)` was `false`