keyword search and robot.txt file added