Updated robots.txt file and added some new modules.