Changed robots.txt directives
Added proper meta tags