add robots.txt for all hosts;