SEO improvement to robot.txt

SEO improvement to robot.txt allowing .css and .js file to be accessed by bots in their crawlings.
See https://github.com/joomla/joomla-cms/issues/3937