SEO improvement to robot.txt

SEO improvement to robot.txt allowing .css and .js file to be accessed by bots in their crawlings.
See https://github.com/joomla/joomla-cms/issues/3937
Update robots.txt.dist

I've revised a bit the code again, giving total access to both `templates` and `media`, since there could be images and other useful stuff apart `CSS` and `js` files.

Should be evaluated if totally delete the filters or leave them commented.
Update robots.txt.dist

as per https://github.com/joomla/joomla-cms/pull/3965 removed the commented folders (templates and media)
Forward port #924
Change to self
Use JLog rather than JError
Codestyle
[imp][32833] Implements a translatable messages in com_ajax. Closes #2611
Add purge argument to Smart Search CLI indexer.
Closes #3346
Add purge argument to Smart Search CLI indexer. Closes #3466
Merge pull request #3965 from UberGruber/robot.txt-seo-update

Update robots.txt for SEO
use "class" if exist
use "advancedSelect" as default class
do not use default class
Fixes #3070
[bug][33988] Option "float none" missing in frontend article edit view. Fixes #3969
[bug][33340] vCard not downloading with cache activated. Fixes #4001
[imp][25616] Delete URL-Parameter limitstart (also when its value is 0). Closes #3725
Codestyle
[bug][33994] Bug in plaintext mode of email plugin. Fixes #3981
[bug][33061] Protostar error.php has different layout and does not find module's css . Fixes #3983