Added different robots.txt files
Fine-tuned robots.txt a bit
Merge branch 'master' into production