regenerate with proper robots.txt