add robots.txt and modify search