Disallow robot crawling of federated search content.