disallow gambit searches in robots.txt