SEO improvement to robot.txt

SEO improvement to robot.txt allowing .css and .js file to be accessed by bots in their crawlings.
See https://github.com/joomla/joomla-cms/issues/3937
Update robots.txt.dist

I've revised a bit the code again, giving total access to both `templates` and `media`, since there could be images and other useful stuff apart `CSS` and `js` files.

Should be evaluated if totally delete the filters or leave them commented.
Update robots.txt.dist

as per https://github.com/joomla/joomla-cms/pull/3965 removed the commented folders (templates and media)
Merge pull request #3965 from UberGruber/robot.txt-seo-update

Update robots.txt for SEO