change robots.txt
refactor keyword search logic