Updated robots.txt for cpanel based deployments.