declared rules in robots.txt
added legacy routes