Added a global disallow in the robots.txt